title: Setting school-level outcome standards.
abstract: BACKGROUND
  To establish international standards for medical schools, an appropriate panel of experts must decide on performance standards. A pilot test of such standards was set in the context of a multidimensional (multiple-choice question examination, objective structured clinical examination, faculty observation) examination at 8 leading schools in China.
  
  
  METHODS
  A group of 16 medical education leaders from a broad array of countries met over a 3-day period. These individuals considered competency domains, examination items, and the percentage of students who could fall below a cut-off score if the school was still to be considered as meeting competencies. This 2-step process started with a discussion of the borderline school and the relative difficulty of a borderline school in achieving acceptable standards in a given competency domain. Committee members then estimated the percentage of students falling below the standard that is tolerable at a borderline school and were allowed to revise their ratings after viewing pilot data.
  
  
  RESULTS
  Tolerable failure rates ranged from 10% to 26% across competency domains and examination types. As with other standard-setting exercises, standard deviations from initial to final estimates of the tolerable failure rates fell, but the cut-off scores did not change significantly. Final, but not initial cut-off scores were correlated with student failure rates (r = 0.59, P = 0.03).
  
  
  CONCLUSIONS
  This paper describes a method to set school-level outcome standards at an international level based on prior established standard-setting methods. Further refinement of this process and validation using other examinations in other countries will be needed to achieve accurate international standards.
authors.affiliation: Department of Internal Medicine , University of Michigan Medical School, Ann Arbor, Michigan, USA. dstern@umich.edu
authors.forename: David T
authors.initials: DT
authors.lastname: Stern
authors.forename: Miriam
authors.initials: M
authors.lastname: Friedman Ben-David
authors.forename: John
authors.initials: J
authors.lastname: Norcini
authors.forename: Andrzej
authors.initials: A
authors.lastname: Wojtczak
authors.forename: M Roy
authors.initials: MR
authors.lastname: Schwarz
mesh_terms: D002681:China; D002983:Clinical Competence / Q000592:standards*; D004504:Education, Medical, Undergraduate / Q000592:standards*; D005240:Feasibility Studies; D007391:International Cooperation; D012015:Reference Standards; D012577:Schools, Medical / Q000592:standards*