title: How to set the bar in competency-based medical education: standard setting after an Objective Structured Clinical Examination (OSCE).
abstract: BACKGROUND
  The goal of the Objective Structured Clinical Examination (OSCE) in Competency-based Medical Education (CBME) is to establish a minimal level of competence. The purpose of this study was to 1) to determine the credibility and acceptability of the modified Angoff method of standard setting in the setting of CBME, using the Borderline Group (BG) method and the Borderline Regression (BLR) method as a reference standard; 2) to determine if it is feasible to set different standards for junior and senior residents, and 3) to determine the desired characteristics of the judges applying the modified Angoff method.
  
  
  METHODS
  The results of a previous OSCE study (21 junior residents, 18 senior residents, and six fellows) were used. Three groups of judges performed the modified Angoff method for both junior and senior residents: 1) sports medicine surgeons, 2) non-sports medicine orthopedic surgeons, and 3) sports fellows. Judges defined a borderline resident as a resident performing at a level between competent and a novice at each station. For each checklist item, the judges answered yes or no for "will the borderline/advanced beginner examinee respond correctly to this item?" The pass mark was calculated by averaging the scores. This pass mark was compared to that created using both the BG and the BLR methods.
  
  
  RESULTS
  A paired t-test showed that all examiner groups expected senior residents to get significantly higher percentage of checklist items correct compared to junior residents (all stations p < 0.001). There were no significant differences due to judge type. For senior residents, there were no significant differences between the cut scores determined by the modified Angoff method and the BG/BLR method. For junior residents, the cut scores determined by the modified Angoff method were lower than the cut scores determined by the BG/BLR Method (all p < 0.01).
  
  
  CONCLUSIONS
  The results of this study show that the modified Angoff method is an acceptable method of setting different pass marks for senior and junior residents. The use of this method enables both senior and junior residents to sit the same OSCE, preferable in the regular assessment environment of CBME.
authors.affiliation: Women's College Hospital, 76 Grenville St, Toronto, M5S 1B1, Canada. drtimdwyer@yahoo.com.au.|Mt Sinai Hospital, 600 University Avenue, Toronto, M5G 1X5, Canada. drtimdwyer@yahoo.com.au.
authors.forename: Tim
authors.initials: T
authors.lastname: Dwyer
authors.affiliation: Toronto East General, 825 Coxell Avenue, Toronto, M4C 3E7, Canada. sarah.wright@utoronto.ca.
authors.forename: Sarah
authors.initials: S
authors.lastname: Wright
authors.affiliation: The Wilson Centre, 200 Elizabeth St, Toronto, M5G 2C4, Canada. mahan777@gmail.com.
authors.forename: Kulamakan Mahan
authors.initials: KM
authors.lastname: Kulasegaram
authors.affiliation: Women's College Hospital, 76 Grenville St, Toronto, M5S 1B1, Canada. john.theodoropoulos@gmail.com.|Mt Sinai Hospital, 600 University Avenue, Toronto, M5G 1X5, Canada. john.theodoropoulos@gmail.com.
authors.forename: John
authors.initials: J
authors.lastname: Theodoropoulos
authors.affiliation: Women's College Hospital, 76 Grenville St, Toronto, M5S 1B1, Canada. jchahal@utosm.com.
authors.forename: Jaskarndip
authors.initials: J
authors.lastname: Chahal
authors.affiliation: Sunnybrook Health Sciences Centre, 2075 Bayview Ave, Toronto, M4N 3 M5, Canada. david.wasserstein@mail.utoronto.ca.
authors.forename: David
authors.initials: D
authors.lastname: Wasserstein
authors.affiliation: The Wilson Centre, 200 Elizabeth St, Toronto, M5G 2C4, Canada. charlotte.ringsted@au.dk.
authors.forename: Charlotte
authors.initials: C
authors.lastname: Ringsted
authors.affiliation: The Wilson Centre, 200 Elizabeth St, Toronto, M5G 2C4, Canada. brian.hodges@uhn.ca.
authors.forename: Brian
authors.initials: B
authors.lastname: Hodges
authors.affiliation: Women's College Hospital, 76 Grenville St, Toronto, M5S 1B1, Canada. leafdoc1@rogers.com.
authors.forename: Darrell
authors.initials: D
authors.lastname: Ogilvie-Harris
mesh_terms: D000328:Adult; D002983:Clinical Competence / Q000592:standards; D003162:Competency-Based Education / Q000592:standards*; D004503:Education, Medical, Graduate / Q000379:methods; D004521:Educational Measurement / Q000592:standards*; D005257:Fellowships and Scholarships; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000379:methods; D008297:Male; D009985:Orthopedics / Q000193:education*; D012044:Regression Analysis; D013178:Sports Medicine / Q000193:education*