In the certification arena, a traditional model of item development consists of representative panels of content experts from various locations gathering for a period of time in a face-to-face meeting to perform specific tasks associated with the development of the examination (e.g., item writing, item review).\u00a0 At times, the meetings are held in conjunction with an association conference or at a desirable location to serve as an incentive for content expert participation.\u00a0 The content experts attending the meeting also have the added incentive of getting to network with peers within their profession and discuss commonalities and issues within practice.<\/p>\n
While desirable in many ways, time and travel costs to host these meetings can be high.\u00a0 Meetings have to be planned well in advanced and panel members may have difficulty coordinating schedules to devote multiple days or a week to the meeting.\u00a0 Also, particularly when meetings are required to occur during winter months, weather and other travel related factors may delay or cancel a meeting which can prove to be even more costly.<\/p>\n
With technological advances including the internet, cloud computing, video conferencing, and other telecommunication systems, web conferencing and telecommunication are becoming the main stay with many organizations, a trend that has also penetrated into the certification testing world. Many organizations are integrating at least some components of remote item development into their processes, while others have gone to a completely remote model. Cost savings, flexibility and \u201cjust-in-time\u201d development are often cited as reasons for using a remote development model.<\/p>\n
Remote item development consists of any aspect of item development that is performed where content experts, test developers, and\/or psychometricians are working on the development of the items from physically different locations.\u00a0 This can involve activities that are performed synchronously (e.g., panel of content experts meeting via a web conference) or asynchronously (content experts independently write items and submit them to a web-based item banking system).\u00a0 Remote item development can happen at any of the various phases of item development, most often during item writing or item review.<\/p>\n
Remote item writing<\/strong><\/p>\n In remote item writing, item writers are first typically trained on general item writing practices and on the specifics related to the particular examination. They then write and submit items remotely.<\/p>\n Remote item writing training can be conducted via web conference or via pre-recorded presentation.\u00a0 A web-conference is ideal as item writers can benefit from the questions and discussions presented by others in the group. \u00a0However, sometimes schedules and time frames may prove difficult in scheduling a group webinar.\u00a0 Either way, it is recommended that certification organizations provide interactive components throughout the presentation and a follow up assignment to encourage the participants to remain engaged.<\/p>\n Some organizations take a hybrid approach where training and initial item writing occurs at an introductory face-to-face meeting and then continued via remote item writing.<\/p>\n Items are typically submitted on some sort of template, whether it is a word processing form submitted via a secure FTP, a cloud based site\u00a0 via an on-line submittal form, or directly into the item bank.\u00a0 The increase of web-based item banking systems is making the submittal of items directly into the item bank more feasible.\u00a0 However, the high prices of these systems can be a factor for some organizations; therefore other options are still viable choices.\u00a0 While secure emailing options (encryption) are available, submitting items by email is generally not recommended.<\/p>\n Not all content experts are suited for remote item writing. \u00a0Those who succeed the most tend to be self-starters who are able to work independently within specified guidelines and when given feedback.<\/p>\n Management of remote item writing can vary provided the needs and goals of the program.\u00a0 Some programs will engage a particular group of item writers for a brief period of time for a one time project; other programs will maintain a standing group of item writers who will provide on-going item writing over a longer period time.\u00a0 Regardless of the term of the item writers, it is recommended that programs schedule periodic check-in sessions with\u00a0 item writers (preferably as a group) to allow opportunities for discussion and feedback.\u00a0 This also helps encourage continued engagement with the project.<\/p>\n Remote Item Review<\/strong><\/p>\n Remote item review can be done synchronously through webinars, asynchronously where a group of item reviewers each independently review and comment on the same set of items, or by a hybrid approach where item reviewers do an initial filter review independently and then hold a web-conference to discuss items that are shown to be questionable or where the reviewers views differed.<\/p>\n Unlike a face-to-face meeting where item review activities happen over multiple days and where hundreds of items may be reviewed, remote item review will occur in more-frequent shorter-duration sessions with fewer items reviewed in an individual session.\u00a0 A typical review session may last 2-3 hours vs. 2-3 days.\u00a0 Facilitation can be the key to the success of these sessions.\u00a0 Unlike a face-to-face meeting, it is easy for item reviewers on web conference calls to get distracted by competing demands and can more likely tune out (or \u201cmute\u201d out) of the discussion on the call.\u00a0 The facilitator needs to be particularly skilled in the finesse of keeping each member of the call focused on the topic at hand and each view point adequately expressed.\u00a0 This can be challenging as facilitators\u00a0 may be at a disadvantage in not being able to interpret body language when gauging level of agreement from individuals in the group.<\/p>\n Security Considerations<\/strong><\/p>\n Remote item development brings with it an added layer of security considerations due to the added risk of exposure to items via the internet, or individuals working from varied and remote locations.\u00a0 Both the Handbook of Test Development<\/em> (2015) and the Handbook of Test Security<\/em> (2014) provide discussions related to added security measures to consider when engaging in remote item development or using web-based item banking systems.\u00a0 Some major topics to consider when planning for remote item writing include:<\/p>\n\n