If you ask psychometricians and test developers, “What is the first step in developing a certification program?” a majority of them will respond that the first step is to conduct a job/task analysis. From a measurement perspective, they would be right. Once an organization has determined that they want to go through the process of developing a certification program, a logical first step is to conduct a job/task analysis. However, there is a critical step that every organization should take before reaching that point, although few do take it. That first step is to conduct a needs assessment.

The process for developing a certification exam has evolved over a number of years as more and different types of organizations seek to assess the competence of workers and to certify people. The certification exam is often at the center of this initiative and subject to the most scrutiny, so it’s important to start with a solid foundation and good planning.  For most professional certification programs, the steps for developing certification examinations include:  conducting a job/task analysis, developing an examination blueprint, writing and reviewing items, assembling and reviewing an initial exam form, conducting a pilot or beta test of that exam form, and establishing a passing score.

Do you have multiple choice items that are not accurately assessing the content of an exam?  If so you may want to implement innovative items.  In our workshop at the 2015 ATP Conference we presented four innovative item types, along with when each one should be used.

The key factor that should be considered before adding an innovative item type is whether it can improve the measurement quality of the exam.  The first step in implementing innovative items is choosing the right type.  To help guide your decision on which types to implement, it is helpful to have a general understanding of the structure, strengths, and weaknesses of each item type.