title: A Milestone-Based Evaluation System-The Cure for Grade Inflation?
abstract: OBJECTIVE
  Controversy exists over the optimal use of the Milestones in the process of resident evaluation and feedback. We sought to evaluate the performance of a Milestones-based feedback system in comparison to a traditional model.
  
  
  METHODS
  The traditional evaluation system (TES) consisted of a generic 16-item survey using a 5-point Likert scale ranging from 1 to 5, and a free-text comments section. The Milestones-based evaluation system (MBES) was launched in July 2014, ranging from 0 to 4. Individual milestones were mapped to rotations based on resident educational goals by postgraduate year (PGY). The MBES consisted of a survey with a maximum of 7 items, followed by a free-text comment section. Within each evaluation system, an overall composite score was calculated for each categorical general surgical resident. To scale the 2 systems for comparison, TES scores were adjusted downward by 1 point. Descriptive statistics were performed. Univariate analysis was performed with the Wilcoxon signed-rank test. A test for trend across PGY was used for the MBES only.
  
  
  RESULTS
  In the traditional system, the median score was 3.66 (range: 3.2-4.0). There was no meaningful difference in the median score by PGY. In the new system, the median score was 2.69 (range: 1.5-3.7, p < 0.01). The median score differed across PGY and increased by PGY of training (p < 0.01). There was an increase in differences between median scores by PGY.
  
  
  CONCLUSIONS
  On using the milestones to facilitate faculty evaluation of resident knowledge and skill, there was a trend in increasing score by PGY of training. In the MBES, scores could be used to better discriminate resident skill and knowledge levels and resulted in improved differentiation in scoring by PGY. The use of the milestones as a basis for evaluation enabled the program to provide more meaningful feedback to residents and represents an improvement in surgical education.
authors.affiliation: Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania. Electronic address: Lindsay.Kuo@uphs.upenn.edu.
authors.forename: Lindsay E
authors.initials: LE
authors.lastname: Kuo
authors.affiliation: Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania.
authors.forename: Rebecca L
authors.initials: RL
authors.lastname: Hoffman
authors.affiliation: Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania.
authors.forename: Jon B
authors.initials: JB
authors.lastname: Morris
authors.affiliation: Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania.
authors.forename: Noel N
authors.initials: NN
authors.lastname: Williams
authors.affiliation: Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania.
authors.forename: Mark
authors.initials: M
authors.lastname: Malachesky
authors.affiliation: Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania.
authors.forename: Laura E
authors.initials: LE
authors.lastname: Huth
authors.affiliation: Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania.
authors.forename: Rachel R
authors.initials: RR
authors.lastname: Kelz
mesh_terms: D002983:Clinical Competence / Q000706:statistics & numerical data*; D000068156:Formative Feedback; D013502:General Surgery / Q000193:education*; D007396:Internship and Residency; D011996:Records
keywords: Interpersonal and Communication Skills; Likert; Medical Knowledge; Milestones; Next Accreditation System; Patient Care; Practice-Based Learning and Improvement; Professionalism; Systems-Based Practice; competencies; feedback; resident evaluation