{"id":459,"date":"2016-05-04T18:10:04","date_gmt":"2016-05-04T18:10:04","guid":{"rendered":"http:\/\/www.proftesting.com\/blog\/?p=459"},"modified":"2016-05-06T14:55:12","modified_gmt":"2016-05-06T14:55:12","slug":"determinants-of-success","status":"publish","type":"post","link":"https:\/\/www.proftesting.com\/blog\/2016\/05\/04\/determinants-of-success\/","title":{"rendered":"How to Determine Successful Performance on an Exam: A Case Study"},"content":{"rendered":"

In 2011, the Council of Landscape Architectural Registration Boards (CLARB)<\/a> wanted to know what factors would lead to successful performance on the Landscape Architect Registration Exam (L.A.R.E.).\u00a0 At the time, the L.A.R.E. was a five-part exam consisting of three multiple-choice question exams and two hand-drawing practical exams.\u00a0 The practical exams were administered twice per year for one day in a handful of locations, creating barriers to obtaining a license in landscape architecture.\u00a0 Since the L.A.R.E. is a licensure exam regulated by each state, decisions on eligibility requirements were made at the state level and often varied by state.\u00a0 For example, the number of years of practice required to sit for the L.A.R.E. is different from state to state.\u00a0 If CLARB could identify which factors lead to successful performance on each exam, they could provide insights to both exam candidates and State Regulatory Boards.<\/p>\n

<\/p>\n

The \u201cDeterminants of Success\u201d study consisted of three phases.\u00a0 During the first phase, a research team interviewed members of three stakeholder groups via semi-structured phone interviews to determine what factors could lead to successful performance on the L.A.R.E.\u00a0 In the second phase, data collected from phone interviews was turned into an online survey, which was sent to candidates after each administration of the L.A.R.E. for one year.\u00a0 The third phase of the study included a statistical analysis whereby data collected from the online survey was analyzed to determine what factors best predict successful performance on the L.A.R.E.<\/p>\n

\"CLARB\"The phone interviews took place over the course of a few weeks.\u00a0 Three stakeholder groups were interviewed \u2013 employers of exam candidates, educators teaching content found on the L.A.R.E., and exam candidates.\u00a0 Each interviewee was asked to describe specific factors that they believe influenced successful performance on the L.A.R.E.\u00a0 The semi-structured interview protocol had series of prompts separated into six factors: testing environment, formal education, work experience, preparation, demographics, and candidate skills.\u00a0 Overall, six employers, five educators, and four exam candidates were interviewed for a total of 15 interviews.<\/p>\n

Utilizing the data from the phone interviews, an online survey was developed and administered to exam candidates.\u00a0 The survey was administered to L.A.R.E. candidates following the completion of four separate examinations (two administrations of the multiple choice sections of the L.A.R.E. and two administrations of the hand-drawing practical sections of the L.A.R.E.).\u00a0 It was critical that the surveys were administered after <\/em>the exam candidates had taken the L.A.R.E., but before <\/em>they received their scores on the exam to avoid pass\/fail status influencing responses.\u00a0 Like the phone interviews, survey items were separated into six factors: testing environment, formal education, work experience, preparation, demographics, and candidate skills.<\/p>\n

To determine which factors, as established in the first phase of the study, predicted success on each of the five sections of the L.A.R.E., a statistical analysis was conducted using both multiple linear regression and logistic regression.\u00a0 In both cases, the same predictor variables were entered into the statistical model, but with multiple linear regression the outcome variable was candidate exam score and with logistic regression the dependent variable was pass\/fail status on the exam.\u00a0 For the purposes of this study, each of the five sections of the L.A.R.E. were analyzed separately using five different factor models:<\/p>\n