title: A multi-source feedback tool for measuring a subset of Pediatrics Milestones.
abstract: BACKGROUND
  The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context.
  
  
  OBJECTIVE
  To report validity evidence for the MSF tool for informing milestone classification decisions.
  
  
  METHODS
  We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework.
  
  
  RESULTS
  One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores.
  
  
  CONCLUSIONS
  We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.
authors.affiliation: a Department of Medical Education , University of Illinois at Chicago , Chicago , IL , USA.|b Association of Pediatric Program Directors , Longitudinal Educational Assessment Research Network , McLean , VA , USA.
authors.forename: Alan
authors.initials: A
authors.lastname: Schwartz
authors.affiliation: c Measurement Consulting Services, National Board of Medical Examiners , Philadelphia , PA , USA.
authors.forename: Melissa J
authors.initials: MJ
authors.lastname: Margolis
authors.affiliation: d Department of Pediatrics , University of Louisville , Louisville , KY , USA.
authors.forename: Sara
authors.initials: S
authors.lastname: Multerer
authors.affiliation: e Department of Pediatrics , University of Michigan , Ann Arbor , MI , USA.
authors.forename: Hilary M
authors.initials: HM
authors.lastname: Haftel
authors.affiliation: f Department of Emergency Medicine , Cincinnati Children's Hospital Medical Center , Cincinnati , OH , USA.
authors.forename: Daniel J
authors.initials: DJ
authors.lastname: Schumacher
authors.affiliation: b Association of Pediatric Program Directors , Longitudinal Educational Assessment Research Network , McLean , VA , USA.|c Measurement Consulting Services, National Board of Medical Examiners , Philadelphia , PA , USA.
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D003162:Competency-Based Education; D004521:Educational Measurement / Q000379:methods / Q000592:standards*; D005163:Factor Analysis, Statistical; D005178:Faculty; D000068156:Formative Feedback; D006801:Humans; D007396:Internship and Residency; D009726:Nurses; D010372:Pediatrics / Q000193:education / Q000592:standards*; D011594:Psychometrics; D012955:Societies, Medical