title: Setting pass scores for assessment of technical performance by surgical trainees.
abstract: BACKGROUND
  One of the major challenges of competency-based training is defining a score representing a competent performance. The objective of this study was to set pass scores for the Objective Structured Assessment of Technical Skill.
  
  
  METHODS
  Pass scores for the examination were set using three standard setting methods applied to data collected prospectively from first-year surgical residents (trainees). General surgery residents were then assigned an overall pass-fail status for each method. Using a compensatory model, residents passed the eight station examinations if they met the overall pass score; using a conjunctive model, residents passed if they met the overall pass score and passed at least 50 per cent of the stations. The consistency of the pass-fail decision across the three methods, and between a compensatory and conjunctive model, were compared.
  
  
  RESULTS
  Pass scores were stable across all three methods using data from 513 residents, 133 of whom were general surgeons. Consistency of the pass-fail decision across the three methods was 95.5 and 93.2 per cent using compensatory and conjunctive models respectively. Consistency of the pass-fail status between compensatory and conjunctive models for all three methods was also very high (91.7, 95.5 and 96.2 per cent).
  
  
  CONCLUSIONS
  Consistency in pass-fail status between the various methods builds evidence of validity for the set scores. These methods can be applied and studied across a variety of assessment platforms, helping to increase the use of standard setting for competency-based training.
authors.affiliation: Department of Surgery, University of Toronto, Toronto, Ontario, Canada.|Division of General Surgery, St Michael's Hospital, Toronto, Ontario, Canada.
authors.forename: S
authors.initials: S
authors.lastname: de Montbrun
authors.affiliation: University of Toronto, Surgical Skills Centre, Mount Sinai Hospital, Toronto, Ontario, Canada.
authors.forename: L
authors.initials: L
authors.lastname: Satterthwaite
authors.affiliation: Department of Surgery, University of Toronto, Toronto, Ontario, Canada.|Division of General Surgery, St Michael's Hospital, Toronto, Ontario, Canada.
authors.forename: T P
authors.initials: TP
authors.lastname: Grantcharov
mesh_terms: D002983:Clinical Competence; D004502:Education, Medical, Continuing / Q000592:standards*; D004521:Educational Measurement; D005180:Faculty, Medical / Q000592:standards*; D013502:General Surgery / Q000193:education*; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D009864:Ontario; D013043:Specialties, Surgical / Q000193:education*