title: Assessment of a clinical performance evaluation tool for use in a simulator-based testing environment: a pilot study.
abstract: OBJECTIVE
  This study assessed a clinical performance evaluation tool for use in a simulator-based testing environment.
  
  
  METHODS
  Twenty-three subjects were evaluated during five standardized encounters using a patient simulator (six emergency medicine students, seven house officers, ten chief resident-fellows). Performance in each 15-minute session was compared with performance on an identical number of oral objective-structured clinical examination (OSCE) sessions used as controls. Each was scored by a faculty rater using a scoring system previously validated for oral certification examinations in emergency medicine (eight skills rated 1-8; passing = 5.75).
  
  
  RESULTS
  On both simulator exams and oral controls, chief resident-fellows earned (mean) "passing" scores [sim = 6.4 (95% CI: 6.0-6.8), oral = 6.4 (95% CI: 6.1-6.7)]; house officers earned "borderline" scores [sim = 5.6 (95% CI: 5.2-5.9), oral = 5.5 (95% CI: 5.0-5.9)]; and students earned "failing" scores [sim = 4.3 (95% CI: 3.8-4.7), oral = 4.5 (95% CI: 3.8-5.1)]. There were significant differences among mean scores for the three cohorts, for both oral and simulator test arms (p <.01).
  
  
  CONCLUSIONS
  In this pilot, a standardized oral OSCE scoring system performed equally well in a simulator-based testing environment.
authors.affiliation: Department of Emergency Medicine, Harvard Medical School, Massachusetts General Hospital, Boston 02114, USA. gordon3@partners.org
authors.forename: James A
authors.initials: JA
authors.lastname: Gordon
authors.forename: David N
authors.initials: DN
authors.lastname: Tancredi
authors.forename: William D
authors.initials: WD
authors.lastname: Binder
authors.forename: William M
authors.initials: WM
authors.lastname: Wilkerson
authors.forename: David W
authors.initials: DW
authors.lastname: Shaffer
mesh_terms: D002983:Clinical Competence / Q000592:standards / Q000706:statistics & numerical data*; D015331:Cohort Studies; D003198:Computer Simulation; D004521:Educational Measurement / Q000379:methods / Q000706:statistics & numerical data; D005257:Fellowships and Scholarships; D006801:Humans; D007396:Internship and Residency; D008404:Massachusetts; D016544:Patient Simulation; D010865:Pilot Projects; D011446:Prospective Studies; D013337:Students, Medical; D011795:Surveys and Questionnaires