A comparison of standard-setting procedures for an OSCE in undergraduate medical education.
OBJECTIVE
To compare four standard-setting procedures for an objective structure clinical examination (OSCE).


METHODS
A 12-station OSCE was administered to 84 students in each of the final (fourth-) year medical classes of 1996 and 1997 at Dalhousie University Faculty of Medicine. Four standard-setting procedures (Angoff, borderline, relative, and holistic) were applied to the data to establish a cutoff score for a pass/fail decision.


RESULTS
The procedures yielded highly inconsistent results. The Angoff and borderline procedures gave similar results; however, the relative and holistic methods gave widely divergent results. The Angoff procedure yielded results reliable enough to use in decision making for a high-stakes examination, but would have required more judges or more stations.


CONCLUSIONS
The Angoff and borderline procedures provide reasonable and defensible approaches to standard setting and are practical to apply by non-psychometricians in medical schools. Further investigation of the other procedures is needed.
Kaufman
D M
DM
Division of Medical Education, Dalhousie University, Halifax, Nova Scotia, Canada. david.kaufman@dal.ca
Mann
K V
KV
Muijtjens
A M
AM
van der Vleuten
C P
CP
D004504:Education, Medical, Undergraduate; D004521:Educational Measurement / Q000379:methods*; D006801:Humans