{"id":33,"date":"2015-06-05T15:30:08","date_gmt":"2015-06-05T15:30:08","guid":{"rendered":"http:\/\/proftesting.com\/blog\/?p=33"},"modified":"2016-01-29T14:19:25","modified_gmt":"2016-01-29T14:19:25","slug":"201565how-to-develop-a-certification-exam","status":"publish","type":"post","link":"https:\/\/www.proftesting.com\/blog\/2015\/06\/05\/201565how-to-develop-a-certification-exam\/","title":{"rendered":"How to Develop a Certification Exam"},"content":{"rendered":"

The process for developing a certification exam has evolved over a number of years as more and different types of organizations seek to assess the competence of workers and to certify people. The certification exam is often at the center of this initiative and subject to the most scrutiny, so it\u2019s important to start with a solid foundation and good planning.\u00a0 For most professional certification programs, the steps for developing certification examinations include:\u00a0 conducting a job\/task analysis, developing an examination blueprint, writing and reviewing items, assembling and reviewing an initial exam form, conducting a pilot or beta test of that exam form, and establishing a passing score.\u00a0 As organizationsplan their exam development activities, it is imperative that each step be detailed and agreed upon prior to development<\/em> to ensure a successful and timely launch. \u00a0There are key decisions related to each step in the exam development process that have the potential of making or breaking your exam\u2019s debut, for example, have you secured qualified and enough SMEs? Will you have enough items?\u00a0 While we understand that a certification program is more than an exam, the exam is often the candidate\u2019s first encounter with the certification body and the biggest hurdle to earning certification, so it\u2019s best to make a good first impression.<\/p>\n

Job\/Task Analysis<\/strong><\/p>\n

A job\/task analysis (JTA) is the foundation of any certification exam and often lays the foundation for other program requirements.\u00a0 The knowledge, skills, and abilities (KSAs) identified during the JTA will help form the examination blueprint or content outline for the certification exam.\u00a0 In other words, the content being assessed by the certification exam will largely be determined by the JTA.\u00a0 The two key decisions in conducting a JTA are choosing the right JTA method and selecting the right Subject Matter Experts or SMEs (for more on selecting the right SMEs, see here<\/a><\/strong>.\u00a0 There are many methods for conducting a JTA (DACUM, Critical Incident Technique, Functional Job Analysis, Task Inventory Analysis, etc.).\u00a0 Some methods include conducting focus groups, some include one-on-one interviews, job shadowing, literature reviews, small group interviews, or large-scale surveys.\u00a0 Each method and process has an impact on budgets, timelines, and the number of individuals involved.\u00a0 In the end, you want to select a JTA method that will give you the most accurate representation of the KSAs you would like to assess, while saving you time and money.<\/p>\n

Item Writing<\/strong><\/p>\n

The item types and content of each item on your certification exam has a huge impact on the perception of the exam.\u00a0 Item types that are confusing to respond to and items that are poorly written will give test takers a negative impression of the exam.\u00a0 To help determine what item types may work for your certification exam see here<\/a><\/strong>.\u00a0 Once item types are determined, the next goal is writing items that are clear, concise, and straight to the point.\u00a0 A certification exam should never include items that are intentionally tricky; you want a test taker to get an item right if he\/she knows the content being assessed and wrong if he\/she doesn\u2019t know the content.\u00a0 Training a group of SMEs on item writing best practices (or general do\u2019s and don\u2019ts associated with item writing) will greatly improve the content of each item, and the overall impression of the exam.<\/p>\n

Item Review, Form Assembly, and Form Reviews<\/strong><\/p>\n

It is imperative that all newly written items are reviewed by a group of SMEs (preferably a different <\/em>group of SMEs than the ones who wrote the items).\u00a0 The initial exam form should be assembled based upon the examination blueprint or content outline, and that form should also reviewed by a group of SMEs.\u00a0 While most would agree that conducting item reviews is a critical stage in the development process, not all organizations conduct form reviews.\u00a0 The exam form should be reviewed as a whole before it is administered to look for content imbalances, item enemies, and any missed grammatical or typographical errors.\u00a0 All of the items on the exam form should seem to \u201cgo\u201d together.\u00a0 In our experience, a group of SMEs can tell you really quickly how the exam as a whole feels (too easy, too hard, over-weighted in a content area).\u00a0 Having a final form review before launching your initial exam will add one more layer of confidence that test takers will see the quality of the exam and have confidence in it.<\/p>\n

Pilot Testing and Passing Score Studies<\/strong><\/p>\n

The final two steps in developing a certification exam is pilot or beta testing the initial exam form and establishing a passing score (or cut-score) on that initial form.\u00a0 These two steps can be done in either order, although it is the preference of many psychometricians to pilot test first, withhold scores, and then establish a passing score so that actual candidate data can be used to inform the decision.\u00a0 Key considerations with these last two steps are ensuring a large enough candidate volume and representative candidate population at the initial pilot test, and selecting a cut score methodology that will produce the most accurate passing point for your exam.\u00a0 The right number of test takers for the pilot administration of your exam will depend on many factors, but most agree that you will need a minimum of 30-50 test takers in your initial administration.\u00a0 If the process used to establish your passing point produces a cut score that both your SMEs and Board are happy with, it is likely you will have a passing rate that makes sense to your candidate population as well.<\/p>\n

Candidate Experience and Reputation<\/strong><\/p>\n

Candidates will talk about the exam and will form options, right or wrong, about its content, accuracy and relevance.\u00a0 In certification the old adage that \u201cperceptions are realities\u201d is still true.\u00a0 You will know what your candidates think.\u00a0 Following acceptable practices, making the right decisions throughout the examination development process, and focusing on producing a quality exam goes a long way in creating a good experience.\u00a0 While technology and more exam administration opportunities (think CBT, web-based, mobile apps) have facilitated the outreach of exams, the most important factor to the reputation of a certification program is still the quality of its products\u2014consider your exam one of your most important ones.<\/p>\n","protected":false},"excerpt":{"rendered":"

The process for developing a certification exam has evolved over a number of years as more and different types of organizations seek to assess the competence of workers and to certify people. The certification exam is often at the center of this initiative and subject to the most scrutiny, so it\u2019s important to start with a solid foundation and good planning.  For most professional certification programs, the steps for developing certification examinations include:  conducting a job\/task analysis, developing an examination blueprint, writing and reviewing items, assembling and reviewing an initial exam form, conducting a pilot or beta test of that exam form, and establishing a passing score.<\/p>","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-33","post","type-post","status-publish","format-standard","hentry","category-industry-news"],"_links":{"self":[{"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/posts\/33","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/comments?post=33"}],"version-history":[{"count":1,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/posts\/33\/revisions"}],"predecessor-version":[{"id":135,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/posts\/33\/revisions\/135"}],"wp:attachment":[{"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/media?parent=33"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/categories?post=33"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/tags?post=33"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}