title: Objective structured clinical examinations provide valid clinical skills assessment in emergency medicine education.
abstract: BACKGROUND
  Evaluation of emergency medicine (EM) learners based on observed performance in the emergency department (ED) is limited by factors such as reproducibility and patient safety. EM educators depend on standardized and reproducible assessments such as the objective structured clinical examination (OSCE). The validity of the OSCE as an evaluation tool in EM education has not been previously studied. The objective was to assess the validity of a novel management-focused OSCE as an evaluation instrument in EM education through demonstration of performance correlation with established assessment methods and case item analysis.
  
  
  METHODS
  We conducted a prospective cohort study of fourth-year medical students enrolled in a required EM clerkship. Students enrolled in the clerkship completed a five-station EM OSCE. We used Pearson's coefficient to correlate OSCE performance with performance in the ED based on completed faculty evaluations. Indices of difficulty and discrimination were computed for each scoring item.
  
  
  RESULTS
  We found a moderate and statistically-significant correlation between OSCE score and ED performance score [r(239) =0.40, p<0.001]. Of the 34 OSCE testing items the mean index of difficulty was 63.0 (SD =23.0) and the mean index of discrimination was 0.52 (SD =0.21).
  
  
  CONCLUSIONS
  Student performance on the OSCE correlated with their observed performance in the ED, and indices of difficulty and differentiation demonstrated alignment with published best-practice testing standards. This evidence, along with other attributes of the OSCE, attest to its validity. Our OSCE can be further improved by modifying testing items that performed poorly and by examining and maximizing the inter-rater reliability of our evaluation instrument.
authors: Emory University, Department of Emergency Medicine, Atlanta, Georgia.
authors: Joshua
authors: J
authors: Wallenstein
authors: Emory University, Department of Emergency Medicine, Atlanta, Georgia.
authors: Douglas
authors: D
authors: Ander
mesh_terms: D002982:Clinical Clerkship; D002983:Clinical Competence / Q000592:standards; D004521:Educational Measurement / Q000379:methods*; D004635:Emergency Medicine / Q000193:education* / Q000592:standards; D006801:Humans; D010808:Physical Examination; D011446:Prospective Studies; D015203:Reproducibility of Results; D014481:United States