From the Item Bank

The Professional Testing Blog

CLEAR Exam Review

In our study, we asked how large a change item statistics should make us suspect content exposure in a licensure or certification exam. To find out, we built hundreds of simulated exams where different portions of candidates had advance access to varying sets of questions (compromised items), then compared two outcomes — the percent of candidates who answered each question correctly and a statistic that shows how well the question separates higher and lower scorers (discrimination value). The simulations were clear: when roughly one-quarter of candidates have the answers, a difficult question can become about 20 percentage points easier, far beyond normal random noise. The discrimination statistic was also affected by item compromise, but in a different way. The discrimination values were influenced both by the number of candidates that had unfair access to the question, as well as how many other questions they had access too. An items ability to discrimination dropped when the pool of compromised items was small, but increased when the pool of items was large. In practice, if a tough item suddenly looks a significantly easier, we should investigate; however, shifts in the discrimination statistic alone are a less discernable signal of item compromise.

Categorized in: Industry News