title: Impact on house staff evaluation scores when changing from a Dreyfus- to a Milestone-based evaluation model: one internal medicine residency program's findings.
abstract: OBJECTIVE
  As graduate medical education (GME) moves into the Next Accreditation System (NAS), programs must take a critical look at their current models of evaluation and assess how well they align with reporting outcomes. Our objective was to assess the impact on house staff evaluation scores when transitioning from a Dreyfus-based model of evaluation to a Milestone-based model of evaluation. Milestones are a key component of the NAS.
  
  
  METHODS
  We analyzed all end of rotation evaluations of house staff completed by faculty for academic years 2010-2011 (pre-Dreyfus model) and 2011-2012 (post-Milestone model) in one large university-based internal medicine residency training program. Main measures included change in PGY-level average score; slope, range, and separation of average scores across all six Accreditation Council for Graduate Medical Education (ACGME) competencies.
  
  
  RESULTS
  Transitioning from a Dreyfus-based model to a Milestone-based model resulted in a larger separation in the scores between our three post-graduate year classes, a steeper progression of scores in the PGY-1 class, a wider use of the 5-point scale on our global end of rotation evaluation form, and a downward shift in the PGY-1 scores and an upward shift in the PGY-3 scores.
  
  
  CONCLUSIONS
  For faculty trained in both models of assessment, the Milestone-based model had greater discriminatory ability as evidenced by the larger separation in the scores for all the classes, in particular the PGY-1 class.
authors.affiliation: Department of Internal Medicine, Hofstra North Shore LIJ School of Medicine, Hempstead, New York, NY, USA; kfriedma@nshs.edu.
authors.forename: Karen A
authors.initials: KA
authors.lastname: Friedman
authors.affiliation: Department of Internal Medicine, Hofstra North Shore LIJ School of Medicine, Hempstead, New York, NY, USA.
authors.forename: Sandy
authors.initials: S
authors.lastname: Balwan
authors.affiliation: Department of Internal Medicine, Hofstra North Shore LIJ School of Medicine, Hempstead, New York, NY, USA.
authors.forename: Frank
authors.initials: F
authors.lastname: Cacace
authors.affiliation: Department of Internal Medicine, Hofstra North Shore LIJ School of Medicine, Hempstead, New York, NY, USA.
authors.forename: Kyle
authors.initials: K
authors.lastname: Katona
authors.affiliation: Department of Psychiatry, Hofstra North Shore LIJ School of Medicine, Hempstead, New York, NY, USA.
authors.forename: Suzanne
authors.initials: S
authors.lastname: Sunday
authors.affiliation: Department of Internal Medicine, Hofstra North Shore LIJ School of Medicine, Hempstead, New York, NY, USA.
authors.forename: Saima
authors.initials: S
authors.lastname: Chaudhry
mesh_terms: D000068:Accreditation; D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate / Q000592:standards; D004521:Educational Measurement / Q000379:methods*; D004650:Employee Performance Appraisal / Q000379:methods; D007388:Internal Medicine / Q000193:education*; D007396:Internship and Residency; D008505:Medical Staff, Hospital; D014481:United States
keywords: ACGME core competencies; Medical Education-Graduate; Medical Education-assessment methods; Milestones; Next Accreditation System