title: Evaluation of patient simulator performance as an adjunct to the oral examination for senior anesthesia residents.
abstract: BACKGROUND
  Patient simulators possess features for performance assessment. However, the concurrent validity and the "added value" of simulator-based examinations over traditional examinations have not been adequately addressed. The current study compared a simulator-based examination with an oral examination for assessing the management skills of senior anesthesia residents.
  
  
  METHODS
  Twenty senior anesthesia residents were assessed sequentially in resuscitation and trauma scenarios using two assessment modalities: an oral examination, followed by a simulator-based examination. Two independent examiners scored the performances with a previously validated global rating scale developed by the Anesthesia Oral Examination Board of the Royal College of Physicians and Surgeons of Canada. Different examiners were used to rate the oral and simulation performances.
  
  
  RESULTS
  Interrater reliability was good to excellent across scenarios and modalities: intraclass correlation coefficients ranged from 0.77 to 0.87. The within-scenario between-modality score correlations (concurrent validity) were moderate: r = 0.52 (resuscitation) and r = 0.53 (trauma) (P < 0.05). Forty percent of the average score variance was accounted for by the participants, and 30% was accounted for by the participant-by-modality interaction.
  
  
  CONCLUSIONS
  Variance in participant scores suggests that the examination is able to perform as expected in terms of discriminating among test takers. The rather large participant-by-modality interaction, along with the pattern of correlations, suggests that an examinee's performance varies based on the testing modality and a trainee who "knows how" in an oral examination may not necessarily be able to "show how" in a simulation laboratory. Simulation may therefore be considered a useful adjunct to the oral examination.
authors.affiliation: St. Michael's Anesthesia Research into Teaching Simulation Group, Department of Anesthesia, St. Michael's Hospital, University of Toronto, Ontario, Canada.
authors.forename: Georges L
authors.initials: GL
authors.lastname: Savoldelli
authors.forename: Viren N
authors.initials: VN
authors.lastname: Naik
authors.forename: Hwan S
authors.initials: HS
authors.lastname: Joo
authors.forename: Patricia L
authors.initials: PL
authors.lastname: Houston
authors.forename: Marianne
authors.initials: M
authors.lastname: Graham
authors.forename: Bevan
authors.initials: B
authors.lastname: Yee
authors.forename: Stanley J
authors.initials: SJ
authors.lastname: Hamstra
mesh_terms: D000328:Adult; D000776:Anesthesiology / Q000193:education*; D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006801:Humans; D007396:Internship and Residency; D008297:Male; D016544:Patient Simulation