Identifying the unauthorized use of examination material.
Item disclosure is one of the most serious threats to the validity of high stakes examinations, and identifying examinees that may have had unauthorized access to material is an important step in ensuring the integrity of an examination. A procedure was developed to identify examinees that potentially had unauthorized prior access to examination content. A standardized difference score is created by comparing examinee ability estimates for potentially exposed items to ability estimates for unexposed items. Outliers in this distribution are then flagged for further review. The steps associated with this procedure are described and followed by an example of applying the procedure. In addition, the use of this procedure is supported by the results of a simulation that models the use of unauthorized access to examination material.
Wood
Timothy J
TJ
Medical Council of Canada, 2283 St. Laurent Blvd., Ottawa, Ontario, Canada. twood@mcc.ca
St-Onge
Christina
C
Boulais
André-Philippe
AP
Blackmore
David E
DE
Maguire
Thomas O
TO
D000704:Analysis of Variance; D002170:Canada; D002983:Clinical Competence / Q000592:standards* / Q000706:statistics & numerical data; D008192:Deception; D004521:Educational Measurement / Q000592:standards* / Q000706:statistics & numerical data; D004522:Educational Status; D005240:Feasibility Studies; D006281:Health Occupations / Q000193:education / Q000941:ethics*; D006801:Humans; D009010:Monte Carlo Method; D011594:Psychometrics; D012044:Regression Analysis; D013044:Specialty Boards / Q000592:standards* / Q000706:statistics & numerical data