The effect of incorporating normative data into a criterion-referenced standard setting in medical education.
OBJECTIVE
Determining standards for assessing clinical performance is a controversial issue. Purely item-based methods such as the Angoff method often produce unrealistic judgments, even when used by experienced judges. The rather unstudied compromise methods combine absolute and relative judgments and thereby incorporate normative data into criterion-based standard-setting judgments. The purpose of this study was to compare the quality and implications of standards set by three methods used for the Objective Structured Clinical Examination (OSCE).


METHODS
Ninety-six judges set standards for 36 surgical year-4 undergraduate OSCE stations. All judges had normative student performance data when judgments were made with the Angoff, Ebel, or Hofstee methods.


RESULTS
The Hofstee method gave more realistic cutoff scores and standard errors and better Meskauskas and Jaeger indices than the Angoff and Ebel methods.


CONCLUSIONS
Medical educators setting standards for an OSCE should consider adopting the Hofstee method.
Cusimano
Michael D
MD
mountain@smh.toronto.on.ca
Rothman
Arthur I
AI
D002982:Clinical Clerkship; D002983:Clinical Competence; D004504:Education, Medical, Undergraduate; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D011897:Random Allocation