Factors affecting the reliability of ratings of students' clinical skills in a medicine clerkship.
OBJECTIVE
To determine the overall reliability and factors that might affect the reliability of ratings of students' clinical skills in a medicine clerkship.


METHODS
A nine-item instrument was used to evaluate students' clinical skills. Raters were also asked to provide a grade of each student's overall clinical performance. Generalizability studies were performed to estimate the reliability of the ratings. The effects of rater experience and clerkship setting were investigated by regression analysis.


METHODS
Teaching hospitals and community-based sites in three Northwestern states.


METHODS
All students (328) who had completed the 12-week clerkship in internal medicine at one medical school during the academic years 1987-1989. Raters included attending physicians, chief residents, and other residents.


RESULTS
Seven observations were needed to provide a reliable rating of the overall clinical grade. More observations were needed to obtain reliable ratings for individual items, ranging from seven observations needed for the rating of data gathering skills to 27 observations needed for the rating of interpersonal relationships with patients. Rater experience and clerkship setting (i.e., teaching hospitals vs. community-based clinics) were found, in general, not to affect significantly the ratings received by students.


CONCLUSIONS
Reliable ratings of students' overall clinical skills, including overall clinical grades, can be achieved by collecting a minimum of seven observations. More observations are needed to measure reliably the interpersonal aspects of clinical performance. These findings support the use of performance ratings to evaluate clinical skills and knowledge of students in clerkship settings.
Carline
J D
JD
Department of Medical Education, University of Washington School of Medicine, Seattle 98195.
Paauw
D S
DS
Thiede
K W
KW
Ramsey
P G
PG
D002982:Clinical Clerkship; D002983:Clinical Competence; D015986:Confounding Factors, Epidemiologic; D004521:Educational Measurement / Q000592:standards*; D006801:Humans; D007388:Internal Medicine; D015205:Northwestern United States; D015203:Reproducibility of Results; D013337:Students, Medical