Evaluating evaluation: assessment of the American Board of Internal Medicine Resident Evaluation Form.
The American Board of Internal Medicine suggests use of a standard form to rate residents on nine dimensions (such as clinical judgment and overall clinical competence) on a scale of 1 to 9. The authors examined the psychometric evidence for reliability and validity of 1,039 ratings of 85 residents by 135 attendings in a single internal medicine residency program. Of these ratings, 95.6% were from 6 to 9. Factor analysis revealed that high correlations among the nine dimensions (r ranged from 0.72 to 0.92) resulted from a single global factor accounting for 86% of the variance. The study also examined whether the form reliably distinguishes among residents scoring between 6 and 9. Agreement among attendings rating the same individual was weak (average reliability = 0.64, by the method of James). The rating method fails to discriminate dimensions of clinical care and has low reliability for distinguishing among competent residents.
Thompson
W G
WG
Department of Medicine, New York University School of Medicine.
Lipkin
M
M
Gilbert
D A
DA
Guzzo
R A
RA
Roberson
L
L
D002983:Clinical Competence / Q000706:statistics & numerical data*; D004521:Educational Measurement / Q000379:methods; D005069:Evaluation Studies as Topic; D006801:Humans; D007388:Internal Medicine / Q000193:education*; D007396:Internship and Residency / Q000592:standards*; D013044:Specialty Boards; D014481:United States