title: Evaluating evaluation: assessment of the American Board of Internal Medicine Resident Evaluation Form.
abstract: The American Board of Internal Medicine suggests use of a standard form to rate residents on nine dimensions (such as clinical judgment and overall clinical competence) on a scale of 1 to 9. The authors examined the psychometric evidence for reliability and validity of 1,039 ratings of 85 residents by 135 attendings in a single internal medicine residency program. Of these ratings, 95.6% were from 6 to 9. Factor analysis revealed that high correlations among the nine dimensions (r ranged from 0.72 to 0.92) resulted from a single global factor accounting for 86% of the variance. The study also examined whether the form reliably distinguishes among residents scoring between 6 and 9. Agreement among attendings rating the same individual was weak (average reliability = 0.64, by the method of James). The rating method fails to discriminate dimensions of clinical care and has low reliability for distinguishing among competent residents.
authors.affiliation: Department of Medicine, New York University School of Medicine.
authors.forename: W G
authors.initials: WG
authors.lastname: Thompson
authors.forename: M
authors.initials: M
authors.lastname: Lipkin
authors.forename: D A
authors.initials: DA
authors.lastname: Gilbert
authors.forename: R A
authors.initials: RA
authors.lastname: Guzzo
authors.forename: L
authors.initials: L
authors.lastname: Roberson
mesh_terms: D002983:Clinical Competence / Q000706:statistics & numerical data*; D004521:Educational Measurement / Q000379:methods; D005069:Evaluation Studies as Topic; D006801:Humans; D007388:Internal Medicine / Q000193:education*; D007396:Internship and Residency / Q000592:standards*; D013044:Specialty Boards; D014481:United States