title: Sharpening the eye of the OSCE with critical action analysis.
abstract: OBJECTIVE
  When interpreting performance scores on an objective structured clinical examination (OSCE), are all checklist items created equal? Although assigning priority through checklist item weighting is often used to emphasize the clinical importance of selected checklist items, the authors propose the use of critical action analysis as an additional method for analyzing and discriminating clinical performance in clinical skill assessment exercises. A critical action is defined as an OSCE checklist item whose performance is critical to ensure an optimal patient outcome and avoid medical error. In this study, the authors analyzed a set of clerkship OSCE performance outcome data and compared the results of critical action analysis versus traditional checklist item performance scores.
  
  
  METHODS
  OSCE performance scores of 398 third-year clerkship students from 2003 to 2006 at the University of Virginia School of Medicine were analyzed using descriptive statistics and a logistic regression model. Through a consensus process, 10 of 25 OSCE cases were identified as containing critical actions.
  
  
  RESULTS
  Students who scored above the median correctly performed the critical actions more often than those scoring lower. However, for 9 of 10 cases, 6% to 46% of higher-scoring students failed to perform the critical action correctly.
  
  
  CONCLUSIONS
  Failure to address this skill assessment outcome is a missed opportunity to more fully understand and apply the results of such examinations to the clinical performance development of medical students. Including critical action analysis in OSCE data interpretation sharpens the eye of the OSCE and enhances its value in clinical skill assessment.
authors.affiliation: University of Virginia School of Medicine, Charlottesville, Virginia 22908, USA.
authors.forename: Nancy J
authors.initials: NJ
authors.lastname: Payne
authors.forename: Elizabeth B
authors.initials: EB
authors.lastname: Bradley
authors.forename: Evan B
authors.initials: EB
authors.lastname: Heald
authors.forename: Karen L
authors.initials: KL
authors.lastname: Maughan
authors.forename: Veronica E
authors.initials: VE
authors.lastname: Michaelsen
authors.forename: Xin-Qun
authors.initials: XQ
authors.lastname: Wang
authors.forename: Eugene C
authors.initials: EC
authors.lastname: Corbett
mesh_terms: D000328:Adult; D002982:Clinical Clerkship / Q000592:standards*; D002983:Clinical Competence; D003422:Critical Care / Q000592:standards; D004504:Education, Medical, Undergraduate / Q000379:methods / Q000592:standards; D004521:Educational Measurement; D005260:Female; D006801:Humans; D007388:Internal Medicine / Q000193:education* / Q000592:standards; D016015:Logistic Models; D008297:Male; D016017:Odds Ratio; D010808:Physical Examination / Q000592:standards; D011336:Probability; D018794:Problem-Based Learning / Q000379:methods / Q000592:standards*; D012577:Schools, Medical / Q000592:standards; D012680:Sensitivity and Specificity; D013337:Students, Medical / Q000706:statistics & numerical data; D013647:Task Performance and Analysis; D014768:Virginia