Sharpening the eye of the OSCE with critical action analysis.
OBJECTIVE
When interpreting performance scores on an objective structured clinical examination (OSCE), are all checklist items created equal? Although assigning priority through checklist item weighting is often used to emphasize the clinical importance of selected checklist items, the authors propose the use of critical action analysis as an additional method for analyzing and discriminating clinical performance in clinical skill assessment exercises. A critical action is defined as an OSCE checklist item whose performance is critical to ensure an optimal patient outcome and avoid medical error. In this study, the authors analyzed a set of clerkship OSCE performance outcome data and compared the results of critical action analysis versus traditional checklist item performance scores.


METHODS
OSCE performance scores of 398 third-year clerkship students from 2003 to 2006 at the University of Virginia School of Medicine were analyzed using descriptive statistics and a logistic regression model. Through a consensus process, 10 of 25 OSCE cases were identified as containing critical actions.


RESULTS
Students who scored above the median correctly performed the critical actions more often than those scoring lower. However, for 9 of 10 cases, 6% to 46% of higher-scoring students failed to perform the critical action correctly.


CONCLUSIONS
Failure to address this skill assessment outcome is a missed opportunity to more fully understand and apply the results of such examinations to the clinical performance development of medical students. Including critical action analysis in OSCE data interpretation sharpens the eye of the OSCE and enhances its value in clinical skill assessment.
Payne
Nancy J
NJ
University of Virginia School of Medicine, Charlottesville, Virginia 22908, USA.
Bradley
Elizabeth B
EB
Heald
Evan B
EB
Maughan
Karen L
KL
Michaelsen
Veronica E
VE
Wang
Xin-Qun
XQ
Corbett
Eugene C
EC
D000328:Adult; D002982:Clinical Clerkship / Q000592:standards*; D002983:Clinical Competence; D003422:Critical Care / Q000592:standards; D004504:Education, Medical, Undergraduate / Q000379:methods / Q000592:standards; D004521:Educational Measurement; D005260:Female; D006801:Humans; D007388:Internal Medicine / Q000193:education* / Q000592:standards; D016015:Logistic Models; D008297:Male; D016017:Odds Ratio; D010808:Physical Examination / Q000592:standards; D011336:Probability; D018794:Problem-Based Learning / Q000379:methods / Q000592:standards*; D012577:Schools, Medical / Q000592:standards; D012680:Sensitivity and Specificity; D013337:Students, Medical / Q000706:statistics & numerical data; D013647:Task Performance and Analysis; D014768:Virginia