title: The validity and reliability of attending evaluations of medicine residents.
abstract: OBJECTIVE
  To assess the reliability and validity of faculty evaluations of medicine residents.
  
  
  METHODS
  We conducted a retrospective study (2004-2012) involving 228 internal medicine residency graduates at the Medical College of Wisconsin who were evaluated by 334 attendings. Measures included evaluations of residents by attendings, based on six competencies and interns and residents' performance on the American Board of Internal Medicine certification exam and annual in-service training examination. All residents had at least one in-service training examination result and 80% allowed the American Board of Internal Medicine to release their scores.
  
  
  RESULTS
  Attending evaluations had good consistency (Cronbach's α = 0.96). There was poor construct validity with modest inter-rater reliability and evidence that attendings were rating residents on a single factor rather than the six competencies intended to be measured. There was poor predictive validity as attending ratings correlated weakly with performance on the in-service training examination or American Board of Internal Medicine certification exam.
  
  
  CONCLUSIONS
  We conclude that attending evaluations are poor measures for assessing progress toward competency. It may be time to move beyond evaluations that rely on global, end-of-rotation appraisals.
authors.affiliation: C, GIM Section, Zablocki VAMC, Department of Medicine, Medical College of Wisconsin, Milwaukee, WI, USA.
authors.forename: Jeffrey L
authors.initials: JL
authors.lastname: Jackson
authors.affiliation: Department of Medicine, Medical College of Wisconsin, Milwaukee, WI, USA.
authors.forename: Cynthia
authors.initials: C
authors.lastname: Kay
authors.affiliation: Program Director, Internal Medicine Residency, Department of Medicine, Medical College of Wisconsin, Milwaukee, WI, USA.
authors.forename: Michael
authors.initials: M
authors.lastname: Frank
keywords: Education; evaluation; residents