title: Using a comprehensive examination to assess multiple competencies in surgical residents: does the oral examination still have a role?
abstract: BACKGROUND
  While specialty-level evaluations evolve from traditional examinations to objective structured clinical examination-like assessments, a broader range of competencies are tested; consequently, examiners are forced to integrate results when making a determination of competency. The aim of this study was to describe how experts weigh relative performances on specific components of a comprehensive examination to make decisions of overall competency.
  
  
  METHODS
  The Patient Assessment and Management Examination is a standardized patient-based assessment of general surgery in which each 25-minute station encompasses four components: history and physical examination, investigation interpretation, diagnosis and treatment discussion with the patient, and a structured oral examination (SOE). A six-station Patient Assessment and Management Examination was administered to 21 senior surgery residents. Surgeons marked each station with global rating scales and, in addition, provided an end-of-station overall global assessment of performance. A "gold-standard" examination pass-or-fail decision was determined through videotape review of each candidate's performance across six stations by two blinded surgeons. Multiple linear regression analysis was used to determine which components were associated with the end-of-station overall global assessments. Multivariable logistic regression was used to determine which components were associated with the final "gold-standard" pass-or-fail assessment.
  
  
  RESULTS
  The only component notably (p < 0.005) associated with end-of-station global assessment for all six stations was the SOE. Mean SOE score was the only notable independent variable associated with the gold-standard pass-or-fail decision (R(2) = 0.63, p < 0.001).
  
  
  CONCLUSIONS
  Performance on the SOE section of a multicompetency examination is markedly associated with the final determination of competency. These results have implications for the design and implementation of comprehensive specialty-level assessments.
authors.affiliation: Department of Surgery, University of British Columbia, Vancouver, British Columbia, Canada.
authors.forename: Ravi S
authors.initials: RS
authors.lastname: Sidhu
authors.forename: Jodi Herold
authors.initials: JH
authors.lastname: McIlroy
authors.forename: Glenn
authors.initials: G
authors.lastname: Regehr
mesh_terms: D002983:Clinical Competence; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D013502:General Surgery / Q000193:education*; D006801:Humans; D007396:Internship and Residency