title: Initial Validity Analysis of the Emergency Medicine Milestones.
abstract: OBJECTIVE
  The Accreditation Council for Graduate Medical Education (ACGME) Milestones describe behavioral markers for the progressive acquisition of competencies during residency. As a key component of the Next Accreditation System, all residents are evaluated for the acquisition of specialty-specific Milestones. The objective was to determine the validity and reliability of the emergency medicine (EM) Milestones.
  
  
  METHODS
  The ACGME and the American Board of Emergency Medicine performed this single-event observational study. The data included the initial EM Milestones performance ratings of all categorical EM residents submitted to the ACGME from October 31, 2013, to January 6, 2014. Mean performance ratings were determined for all 23 subcompetencies for every year of residency training. The internal consistency (reliability) of the Milestones was determined using a standardized Cronbach's alpha coefficient. Exploratory factor analysis was conducted to determine how the subcompetencies were interrelated.
  
  
  RESULTS
  EM Milestone performance ratings were obtained on 100% of EM residents (n = 5,805) from 162 residency programs. The mean performance ratings of the aggregate and individual subcompetency scores showed discrimination between residency years, and the factor structure further supported the validity of the EM Milestones. The reliability was α = 0.96 within each year of training.
  
  
  CONCLUSIONS
  The EM Milestones demonstrated validity and reliability as an assessment instrument for competency acquisition. EM residents can be assured that this evaluation process has demonstrated validity and reliability; faculty can be confident that the Milestones are psychometrically sound; and stakeholders can know that the Milestones are a nationally standardized, objective measure of specialty-specific competency acquisition.
authors.affiliation: The Department of Emergency Medicine, Akron General Medical Center, Akron, OH.
authors.forename: Michael S
authors.initials: MS
authors.lastname: Beeson
authors.affiliation: Milestones Development and Evaluation, Chicago, IL.|Accreditation Council for Graduate Medical Education, Chicago, IL.
authors.forename: Eric S
authors.initials: ES
authors.lastname: Holmboe
authors.affiliation: American Board of Emergency Medicine, East Lansing, MI.
authors.forename: Robert C
authors.initials: RC
authors.lastname: Korte
authors.affiliation: Accreditation Council for Graduate Medical Education, Chicago, IL.|Jefferson Medical Center, Philadelphia, PA.
authors.forename: Thomas J
authors.initials: TJ
authors.lastname: Nasca
authors.affiliation: Accreditation Council for Graduate Medical Education, Chicago, IL.|Jefferson Medical Center, Philadelphia, PA.
authors.forename: Timothy
authors.initials: T
authors.lastname: Brigham
authors.affiliation: American Board of Emergency Medicine, East Lansing, MI.
authors.forename: Chad M
authors.initials: CM
authors.lastname: Russ
authors.affiliation: The Department of Sociology, Michigan State University, East Lansing, MI.
authors.forename: Cameron T
authors.initials: CT
authors.lastname: Whitley
authors.affiliation: American Board of Emergency Medicine, East Lansing, MI.
authors.forename: Earl J
authors.initials: EJ
authors.lastname: Reisdorff
mesh_terms: D000068:Accreditation / Q000592:standards*; D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D004635:Emergency Medicine / Q000193:education*; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D015203:Reproducibility of Results