Setting school-level outcome standards.
BACKGROUND
To establish international standards for medical schools, an appropriate panel of experts must decide on performance standards. A pilot test of such standards was set in the context of a multidimensional (multiple-choice question examination, objective structured clinical examination, faculty observation) examination at 8 leading schools in China.


METHODS
A group of 16 medical education leaders from a broad array of countries met over a 3-day period. These individuals considered competency domains, examination items, and the percentage of students who could fall below a cut-off score if the school was still to be considered as meeting competencies. This 2-step process started with a discussion of the borderline school and the relative difficulty of a borderline school in achieving acceptable standards in a given competency domain. Committee members then estimated the percentage of students falling below the standard that is tolerable at a borderline school and were allowed to revise their ratings after viewing pilot data.


RESULTS
Tolerable failure rates ranged from 10% to 26% across competency domains and examination types. As with other standard-setting exercises, standard deviations from initial to final estimates of the tolerable failure rates fell, but the cut-off scores did not change significantly. Final, but not initial cut-off scores were correlated with student failure rates (r = 0.59, P = 0.03).


CONCLUSIONS
This paper describes a method to set school-level outcome standards at an international level based on prior established standard-setting methods. Further refinement of this process and validation using other examinations in other countries will be needed to achieve accurate international standards.
Stern
David T
DT
Department of Internal Medicine , University of Michigan Medical School, Ann Arbor, Michigan, USA. dstern@umich.edu
Friedman Ben-David
Miriam
M
Norcini
John
J
Wojtczak
Andrzej
A
Schwarz
M Roy
MR
D002681:China; D002983:Clinical Competence / Q000592:standards*; D004504:Education, Medical, Undergraduate / Q000592:standards*; D005240:Feasibility Studies; D007391:International Cooperation; D012015:Reference Standards; D012577:Schools, Medical / Q000592:standards*