A generalizability study of a new standardized rating form used to evaluate students' clinical clerkship performances.
OBJECTIVE
To investigate the measurement characteristics of standardized clinical evaluation forms (CEFs) used to assign grades for clerkship performance.


METHODS
In 1996-97, the authors reviewed 5,168 CEFs completed for 175 students in eight clerkships. Limiting their analysis to the three clerkships that produced the most CEFs, the authors conducted a generalizability study to determine the five variance components for each clerkship. A decision study then calculated the generalizability coefficients and standard errors of measurement in each clerkship for varied numbers of raters and CEF items.


RESULTS
The generalizability study found large variance components attributable to rater and rating context. The decision study found that, when three or more raters completed CEFs for a student, the generalizability coefficient and standard error of measurement reached levels acceptable for grading. Increasing the number of items on the CEF had no significant effect.


CONCLUSIONS
The reliability of assigning students clerkship grades based on single CEFs is unacceptably low. However, CEFs can accurately measure students' clerkship performances if completed by three or more raters.
Kreiter
C D
CD
Office of Consultation and Research in Medical Education, University of Iowa, Iowa City 52242-1008, USA.
Ferguson
K
K
Lee
W C
WC
Brennan
R L
RL
Densen
P
P
D002982:Clinical Clerkship / Q000458:organization & administration / Q000592:standards*; D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D007484:Iowa; D018794:Problem-Based Learning; D015397:Program Evaluation / Q000379:methods; D015203:Reproducibility of Results; D012189:Retrospective Studies; D013663:Teaching