From the Item Bank

The Professional Testing Blog

Eight Tips for Reporting Failing Test Scores on Licensing and Certification Tests

For the people who take your test, after all the studying and stressing out, nothing beats getting a certificate with a shiny gold seal in your mailbox. That makes a passing score report fairly easy to design. It’s going to be a variation on “Hooray! You made it!”

Chances are, though, that not every candidate is going to get that letter. What are you going to tell the candidate who fails?

Here are eight pro tips:

1.       Tell them their score
You should give failing candidates as much helpful information as you can. This includes their score. If I know I am one point short of passing, I will review my notes and retake the test. If I know I am a few points away, I may take a training course. Fifty points away? Maybe I should consider a new career.

2.       Consider reporting raw scores
If you use Classical Test Theory, the simplest and most understandable way to report scores is to tell candidates how many questions they got right (their raw score) or what proportion of the questions they got right (their percent-correct score).

When different forms have different passing scores, consider using a scale (see below). But if you think your constituency can digest the information, why not give it to them straight? Announce,

Your passing score will vary depending on the difficulty of the particular set of questions you get.

Then, when someone fails, tell them,

You took a form of the test that had a passing score of 72/100. You answered 68 questions correctly. Unfortunately, you did not pass the test.

3.       Avoid a scale that can be confused with another scale
You may choose to use a score scale. In choosing the scale, it is wise to avoid 0 to 100, which is easily confused with percent-correct scores.

By the same token, if your test has 180 questions, don’t use a 120-to-180 scale! Someone is sure to confuse scale scores with raw scores.

4.       If you’re using a scale, don’t overdo it
Avoid giving the illusion of greater accuracy than the test can provide. With a 50-question test, a scale that goes from 0 to 900 may be disingenuous. A score of 541 seems a whole lot lower than a score of 573, but it might reflect a difference of a single question! (For details on setting a scale, see “Scoring Precision Information in Tong & Kolen, Scaling: An ITEMS Module.)

5.       Plan ahead to report subscores
You’re trying to give candidates helpful information. You ought to give an indication of how they did in the various content domains included in the test. They can use that information to guide their studies.

If you intend to provide feedback about performance in a content area, you must make sure you have enough questions in the test to make that feedback meaningful. That takes advance planning.

6.       Raw subscores aren’t always helpful
If you’re reporting raw (or percent-correct) scores for the test overall, you can  do that for domain-level scores too. Even with good planning, though, subscores will be less reliable than overall scores.

Testing standards require us to let score users know about imprecision in scores. I asked Ron Hambleton, Distinguished University Professor at UMass Amherst, how to achieve this without leaving the impression that these subscores are a wild guess. He recommended communicating “the concept of imprecision” without necessarily “being quite explicit about the breadth of the [error] band.” One way of doing that is to use performance-level indicators instead of raw numbers.

7.       Use defensible performance-level indicators
Use helpful performance-level descriptors like “Needs Review” or “In this area, your score was above the average score of test-takers who passed the test.” But be careful  not to give test takers labels you cannot defend: “I’m glad you got all 12 questions in the “Anatomy” section right, but that doesn’t make you a doctor, does it?” Use labels like “Expert” with caution.

8.       Include other useful information
If you have other useful information, consider including it too. For example, if “two-thirds of people who retake this test pass the second time,” then let candidates know that. In any case, make sure they know how to proceed if they want to retest.

Categorized in: Industry News