Differential weighting of errors on a test of clinical reasoning skills.
OBJECTIVE
Examinees can make three types of errors on the short-menu questions in the Clinical Reasoning Skills component of the Medical Council of Canada's Qualifying Examination Part I: (1) failing to select any correct responses, (2) selecting too many responses, or (3) selecting a response that is inappropriate or harmful to the patient. This study compared the information provided by equal and differential weighting of these errors.


METHODS
The item response theory nominal model was applied to fit examinees' response patterns on the 1998 test.


RESULTS
Differential error weighting resulted in improved model fit and increased test information for examinees in the lower half of the achievement continuum.


CONCLUSIONS
Differential error weighting appears promising. The pass score is near the lower end of the achievement continuum; therefore, this approach may improve the accuracy of pass-fail decisions.
Childs
Ruth A
RA
Ontario Institute for Studies in Education of the University of Toronto, Canada. rchilds@oise.utoronto.ca
Dunn
Jennifer L
JL
Van Barneveld
Christina
C
Jaciw
Andrew P
AP
McIlroy
Jodi Herold
JH
D002170:Canada; D002983:Clinical Competence / Q000706:statistics & numerical data*; D004521:Educational Measurement / Q000706:statistics & numerical data*; D006801:Humans; D013223:Statistics as Topic / Q000379:methods*; D013337:Students, Medical