title: Assessing residents' written learning goals and goal writing skill: validity evidence for the learning goal scoring rubric.
abstract: OBJECTIVE
  To provide validity evidence for use of the Learning Goal Scoring Rubric to assess the quality of written learning goals and residents' goal writing skills.
  
  
  METHODS
  This two-part study used the rubric to assess University of Colorado third-year pediatric residents' written learning goals to obtain validity evidence. In study 1, five raters independently scored 48 goals written in 2010-2011 and 2011-2012 by 48 residents, who also responded to the Jefferson Scale of Physician Lifelong Learning (JeffSPLL). In study 2, two raters independently scored 48 goals written in 2011-2012 by 12 residents. Intraclass correlation coefficients (ICCs) assessed rater agreement to provide evidence for response process. Generalizability theory assessed internal structure. Independent-samples Mann-Whitney U tests and correlations assessed relationship to other variables. Content was matched to published literature and instructional methods.
  
  
  RESULTS
  The ICC was 0.71 for the overall rubric. In study 1, where the generalizability study's (G study's) object of measurement was learning goals, the phi coefficient was 0.867. In study 2, where the G study's object of measurement was the resident (goal writing skill), the phi coefficient was 0.751. The total mean score of residents with goal writing training was significantly higher than that of those without (7.54 versus 4.98, P < .001). Correlation between goal quality and JeffSPLL score was not significant. Investigators agreed that the content matched the published literature and instructional methods.
  
  
  CONCLUSIONS
  Preliminary validity evidence indicates that this scoring rubric can assess learning goal quality and goal writing skill.
authors: Dr. Lockspeiser is assistant professor, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Ms. Schmitter is program manager for medical education, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Lane is professor and vice chair for medical education, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Hanson is professor and director of medical education research and development, Departments of Pediatrics and Family Medicine, University of Colorado School of Medicine, Aurora, Colorado. Dr. Rosenberg is professor and director, Pediatric Residency Program, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Park is assistant professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois.
authors: Tai M
authors: TM
authors: Lockspeiser
authors: Patricia A
authors: PA
authors: Schmitter
authors: J Lindsey
authors: JL
authors: Lane
authors: Janice L
authors: JL
authors: Hanson
authors: Adam A
authors: AA
authors: Rosenberg
authors: Yoon Soo
authors: YS
authors: Park
mesh_terms: D003120:Colorado; D004502:Education, Medical, Continuing; D004503:Education, Medical, Graduate; D006040:Goals; D006801:Humans; D007396:Internship and Residency; D007858:Learning; D010372:Pediatrics / Q000193:education*; D014956:Writing