Characteristics and Implications of Diagnostic Justification Scores Based on the New Patient Note Format of the USMLE Step 2 CS Exam.
BACKGROUND
To determine the psychometric characteristics of diagnostic justification scores based on the patient note format of the United States Medical Licensing Examination Step 2 Clinical Skills exam, which requires students to document history and physical findings, differential diagnoses, diagnostic justification, and plan for immediate workup.


METHODS
End-of-third-year medical students at one institution wrote notes for five standardized patient cases in May 2013 (n = 180) and 2014 (n = 177). Each case was scored using a four-point rubric to rate each of the four note components. Descriptive statistics and item analyses were computed and a generalizability study done.


RESULTS
Across cases, 10% to 48% provided no diagnostic justification or had several missing or incorrect links between history and physical findings and diagnoses. The average intercase correlation for justification scores ranged from 0.06 to 0.16; internal consistency reliability of justification scores (coefficient alpha across cases) was 0.38. Overall, justification scores had the highest mean item discrimination across cases. The generalizability study showed that person-case interaction (12%) and task-case interaction (13%) had the largest variance components, indicating substantial case specificity.


CONCLUSIONS
The diagnostic justification task provides unique information about student achievement and curricular gaps. Students struggled to correctly justify their diagnoses; performance was highly case specific. Diagnostic justification was the most discriminating element of the patient note and had the greatest variability in student performance across cases. The curriculum should provide a wide range of clinical cases and emphasize recognition and interpretation of clinically discriminating findings to promote the development of clinical reasoning skills.
Rachel
R
Yudkowsky
Yoon Soo
YS
Park
Abbas
A
Hyderi
Georges
G
Bordage
D002983:Clinical Competence; D000066491:Clinical Decision-Making; D003479:Curriculum; D004504:Education, Medical, Undergraduate; D005260:Female; D006801:Humans; D008007:Licensure, Medical; D008297:Male; D008487:Medical History Taking; D016544:Patient Simulation; D010808:Physical Examination; D011594:Psychometrics; D015203:Reproducibility of Results; D014481:United States