OSCE checklists do not capture increasing levels of expertise.
OBJECTIVE
To evaluate the effectiveness of binary content checklists in measuring increasing levels of clinical competence.


METHODS
Fourteen clinical clerks, 14 family practice residents, and 14 family physicians participated in two 15-minute standardized patient interviews. An examiner rated each participant's performance using a binary content checklist and a global process rating. The participants provided a diagnosis two minutes into and at the end of the interview.


RESULTS
On global scales, the experienced clinicians scored significantly better than did the residents and clerks, but on checklists, the experienced clinicians scored significantly worse than did the residents and clerks. Diagnostic accuracy increased for all groups between the two-minute and 15-minute marks without significant differences between the groups.


CONCLUSIONS
These findings are consistent with the hypothesis that binary checklists may not be valid measures of increasing clinical competence.
Hodges
B
B
Department of Psychiatry, Faculty of Medicine, University of Toronto, Ontario, Canada. brian.hodges@utoronto.ca
Regehr
G
G
McNaughton
N
N
Tiberius
R
R
Hanson
M
M
D000704:Analysis of Variance; D002982:Clinical Clerkship; D002983:Clinical Competence; D004501:Education, Medical / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D005194:Family Practice / Q000193:education; D006801:Humans; D007396:Internship and Residency; D001523:Mental Disorders / Q000175:diagnosis; D009864:Ontario; D011570:Psychiatry / Q000193:education; D015203:Reproducibility of Results