Surgery Clerkship Evaluations Are Insufficient for Clinical Skills Appraisal: The Value of a Medical Student Surgical Objective Structured Clinical Examination.
OBJECTIVE
Optimal methods for medical student assessment in surgery remain elusive. Faculty- and housestaff-written evaluations constitute the chief means of student assessment in medical education. However, numerous studies show that this approach has poor specificity and a high degree of subjectivity. We hypothesized that an objective structured clinical examination (OSCE) in the surgery clerkship would provide additional data on student performance that would confirm or augment other measures of assessment.


METHODS
We retrospectively reviewed data from OSCEs, National Board of Medical Examiners shelf examinations, oral presentations, and written evaluations for 51 third-year Harvard Medical School students rotating in surgery at Massachusetts General Hospital from 2014 to 2015. We expressed correlations between numeric variables in Pearson coefficients, stratified differences between rater groups by one-way analysis of variance, and compared percentages with 2-sample t-tests. We examined commentary from both OSCE and clinical written evaluations through textual analysis and summarized these results in percentages.


RESULTS
OSCE scores and clinical evaluation scores correlated poorly with each other, as well as with shelf examination scores and oral presentation grades. Textual analysis of clinical evaluation comments revealed a heavy emphasis on motivational factors and praise, whereas OSCE written comments focused on cognitive processes, patient management, and methods to improve performance.


CONCLUSIONS
In this single-center study, an OSCE provided clinical skills data that were not captured elsewhere in the surgery clerkship. Textual analysis of faculty evaluations reflected an emphasis on interpersonal skills, rather than appraisal of clinical acumen. These findings suggest complementary roles of faculty evaluations and OSCEs in medical student assessment.
Department of Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts. Electronic address: klbutler@partners.org.
Kathryn L
KL
Butler
Cambridge Health Alliance, Cambridge, Massachusetts; Department of Medicine, Harvard Medical School, Boston, Massachusetts.
David A
DA
Hirsh
Department of Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts.
Emil R
ER
Petrusa
Department of Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts.
D Dante
DD
Yeh
Department of Emergency Medicine, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts.
Dana
D
Stearns
Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts.
David E
DE
Sloane
Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts.
Jeffrey A
JA
Linder
Department of Medicine, Cambridge Health Alliance and Harvard Medical School, Cambridge, Massachusetts.
Gaurab
G
Basu
Harvard School of Dental Medicine, Cambridge, Massachusetts.
Lisa A
LA
Thompson
Department of Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts.
Marc A
MA
de Moya
D000068:Accreditation; D002982:Clinical Clerkship; D002983:Clinical Competence; D016208:Databases, Factual; D004504:Education, Medical, Undergraduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D013502:General Surgery / Q000193:education* / Q000706:statistics & numerical data; D006801:Humans; D008297:Male; D008404:Massachusetts; D008487:Medical History Taking; D010808:Physical Examination; D012189:Retrospective Studies; D012577:Schools, Medical / Q000458:organization & administration; D013337:Students, Medical / Q000706:statistics & numerical data; D055815:Young Adult
Interpersonal Skills and Communication; Medical Knowledge; Patient Care; Professionalism; clinical evaluations; medical student assessment; objective structured clinical skills examination; simulation; surgery clerkship; textual analysis