A renowned American Professor of Measurement and Testing, Robert Ebel, published the following definition of a performance test:

“In a performance test the subject is required to demonstrate his or her skill by manipulating objects or instruments.”

This definition was first published in a measurement textbook in 1965. The definition seems solid, except that evolving technology may have changed the concept of manipulating objects or instruments since the time of publication.  For example, robotic surgery to perform a kidney transplant from a location 7,000 miles away from the patient.  This type of ultra-high speed data transfer and related virtual environment begins to change the concept of manipulating objects or instruments.

There has been a lot of buzz about ISO/IEC 17024: 2012 – Conformity Assessment – General requirements for bodies operating certification of persons.  With so many acronyms associated with this standard (ISO, IEC, ANSI, IAF, etc.) it is easy to get confused about who owns this standard and who accredits against it.   We will explore specifics of the standard itself in another post.  The purpose of this post is to help explain the various bodies associated with ISO 17024.

In a recent Internet search, I came across Professional Certifications: Can we Avoid Them? by Uganda-based consultant, Michael Niyitegeka. In this article, he suggests that in the wake of education commoditization, education qualifications have become “tradable goods,” forcing employers to find alternative avenues to gauge one’s competency above and beyond academic qualifications.  While we might share a different view about education, the point that certification plays a role in qualifying people for employment or takes over where education leaves off is worth examining and assigning a value to.  

The Standards for Educational and Psychological Testing (APA, AERA, & NCME, 2014) (Standards) was recently revised and updated.  As credentialing organizations begin to process what the revised version of the Standards means to them, it is important to point out most of the workplace testing and credentialing chapter remained the same.  However, statements about the need for reliable subscore reporting are of vast importance, as is the design of examination score reports. 

Computer Adaptive Testing (CAT) is a testing methodology that weds two processes—adaptive testing and computer administration—for efficient measurement and administration. When compared to fixed length exams delivered linearly on a computer, CAT exams measure a candidate’s ability with fewer delivered exam questions and higher precision. By precision, it is suggested that error is reduced and reliability is increased. 

Overview

The multiple response (MR) item type is, hands-down, one of the most useful alternative item types available today.  It isn’t a new item type; the MR been used in paper-based assessments for many years (under various names, including multiple select, multiple-answer, multiple-choice, etc.). Now that the MR item type is provided as a standard option within many CBT applications, it is becoming a more popular option.

The process for developing a certification exam has evolved over a number of years as more and different types of organizations seek to assess the competence of workers and to certify people. The certification exam is often at the center of this initiative and subject to the most scrutiny, so it’s important to start with a solid foundation and good planning.  For most professional certification programs, the steps for developing certification examinations include:  conducting a job/task analysis, developing an examination blueprint, writing and reviewing items, assembling and reviewing an initial exam form, conducting a pilot or beta test of that exam form, and establishing a passing score.

The Census Bureau recently revealed that the 2050 tipping point, when the majority would become the minority and non-Hispanic whites would no longer account for more than 50% of U.S. inhabitants, may now occur as early as 2044. Yes, the landscape is shifting quickly and NOW is the time to make sure you’ve done your work to integrate multicultural marketing into your outreach efforts.

Many evolving forms of credentialing are clamoring for our attention these days—digital badges, alternative credentials, verified certificates, nano-degrees and micro-credentials.  But what are these forms of “credentialing” and how can they can complement and even enhance certification programs, you know, the traditional ones that assess competency and verify the skills and knowledge required to do a job or define the body of knowledge for a profession/occupation.  And how does a certification body assess if “alt creds” are right for them?  These are two aspects of a complex topic we as certification and test developers are talking about.   

Do you have multiple choice items that are not accurately assessing the content of an exam?  If so you may want to implement innovative items.  In our workshop at the 2015 ATP Conference we presented four innovative item types, along with when each one should be used.

The key factor that should be considered before adding an innovative item type is whether it can improve the measurement quality of the exam.  The first step in implementing innovative items is choosing the right type.  To help guide your decision on which types to implement, it is helpful to have a general understanding of the structure, strengths, and weaknesses of each item type.