{"id":1023,"date":"2018-10-10T18:26:59","date_gmt":"2018-10-10T18:26:59","guid":{"rendered":"http:\/\/www.proftesting.com\/blog\/?p=1023"},"modified":"2018-10-10T18:26:59","modified_gmt":"2018-10-10T18:26:59","slug":"testing-on-demand-vs-windows","status":"publish","type":"post","link":"https:\/\/www.proftesting.com\/blog\/2018\/10\/10\/testing-on-demand-vs-windows\/","title":{"rendered":"Continuous, On-Demand Testing vs. Testing Windows"},"content":{"rendered":"

One of the first questions an organization must answer when establishing a computer-based examination<\/a> program is whether they want to offer their exam continuously, on-demand or during testing windows.\u00a0 Many organizations automatically default to choosing continuous, on-demand testing because of its availability and the belief that it will provide the greatest benefit to their candidate population.\u00a0 However, there are many factors that must be considered when making this decision.<\/p>\n

Definitions<\/em><\/h1>\n

The difference between testing continuously, on-demand and testing in windows is that continuously, on-demand means the candidate can schedule their computer examination any day that the computer testing center is open and has availability.\u00a0 Testing windows is when the exam is only available during specific periods of times and the candidate may only schedule their examination during those windows of time.<\/p>\n

Retake Policies<\/em><\/h1>\n

When testing in windows, the retake policy might state that a candidate who fails the exam must wait until the next \u201cwindow\u201d to sit for the exam again.\u00a0 When testing continuously, on-demand it is more critical that the organization establishes a policy outlining the number of days\/months a candidate must wait between exam administrations and how many times the candidate may retake the exam.\u00a0 Otherwise there could be the possibility of a candidate testing over and over almost every day.<\/p>\n

Exam Form Development<\/em><\/h1>\n

When testing continuously, on-demand the organization should have at least two forms in rotation so that a candidate who fails one form is then exposed to a second form (although this can be mitigated with a longer \u201csit out\u201d period as part of the retake policy).\u00a0 When testing continuously, on-demand, more items will need to be generated to minimize high item exposure rates.<\/p>\n

Exam Authorizations<\/em><\/h1>\n

When a candidate has met all eligibility criteria, and has been approved to sit for the exam, he or she should be given the authorization to test.\u00a0 When testing in windows, it\u2019s easy for an organization to state a candidate is approved for the next \u201cwindow\u201d.\u00a0 However, when testing continuously, on-demand, there should be a time frame for which someone is authorized to test.\u00a0 For example, an organization might grant a candidate the authorization to test for the next 90 days.\u00a0 This is to encourage candidates to actually sit for their certification exam.\u00a0 Without time frames around authorizations to test, some candidates will delay taking their test indefinitely.<\/p>\n

Item Analyses<\/em><\/h1>\n

When testing in windows, all data from each window can be analyzed at the close of the window.\u00a0 Decisions about promoting items from pretest<\/a> to active, decisions related to key changes, and even post-equating can all be completed at the close of a window.\u00a0 When testing continuously, on-demand, a schedule must be established for analyzing item- and form-level data since exam administrations don\u2019t stop and start at specific points in time.<\/p>\n

Exam Applications<\/em><\/h1>\n

When testing in windows, exam applications tend to come in waves (a lot of applicants in the few weeks prior to an exam window).\u00a0 However, when testing continuously, on-demand, exam applications are more likely to be spread out across time.<\/p>\n

Availability<\/em><\/h1>\n

When testing in windows, there are less days of availability at computer test centers and during some peak times such as summer months, seats may be limited.\u00a0 Windows testing might require the windows to be selected during less popular times depending on the candidate volume.\u00a0 However, when testing continuously, on-demand this ceases to be a problem since there are more days available.<\/p>\n

Conclusions<\/em><\/h1>\n

The decision of whether to offer computer-based examinations continuously, on-demand or in windows is a decision that should be made in conjunction with your psychometric consultant<\/a> and your computer testing vendor.\u00a0 By working closely with both of these experts, the pros and cons of both modes of computer administration can be discussed (including many of the issues mentioned above) and a solution that best meets the candidates\u2019 and organization\u2019s needs can be selected.<\/p>\n

Additional Posts<\/h1>\n

For other more articles on options for your examination program check out\u00a0Key Considerations for Translating Examinations and Pitfalls to Avoid<\/a>\u00a0and\u00a0Is it Time to Drop Option D? Considerations for Choosing the Three-Option Multiple Choice Item<\/a>.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"One of the first questions an organization must answer when establishing a computer-based examination program is whether they want to offer their exam continuously, on-demand or during testing windows.\u00a0 Many organizations automatically default to choosing continuous, on-demand testing because of its availability and the belief that it will provide the greatest benefit to their candidate… View Article<\/a>","protected":false},"author":4,"featured_media":1025,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[44],"tags":[10,68],"class_list":["post-1023","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-test-development","tag-certification","tag-test-development"],"_links":{"self":[{"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/posts\/1023","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/comments?post=1023"}],"version-history":[{"count":2,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/posts\/1023\/revisions"}],"predecessor-version":[{"id":1026,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/posts\/1023\/revisions\/1026"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/media\/1025"}],"wp:attachment":[{"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/media?parent=1023"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/categories?post=1023"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.proftesting.com\/blog\/wp-json\/wp\/v2\/tags?post=1023"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}