title: Can Residents Assess Other Providers' Infant Lumbar Puncture Skills?: Validity Evidence for a Global Rating Scale and Subcomponent Skills Checklist.
abstract: OBJECTIVE
  The aims of this study were to provide validity evidence for infant lumbar puncture (ILP) checklist and global rating scale (GRS) instruments when used by residents to assess simulated ILP performances and to compare these metrics to previously obtained attending rater data.
  
  
  METHODS
  In 2009, the International Network for Simulation-based Pediatric Innovation, Research, and Education (INSPIRE) developed checklist and GRS scoring instruments, which were previously validated among attending raters when used to assess simulated ILP performances. Video recordings of 60 subjects performing an LP on an infant simulator were collected; 20 performed by subjects in 3 categories (beginner, intermediate, and expert). Six blinded pediatric residents independently scored each performance (3 via the GRS, 3 via the checklist). Four of the 5 domains of validity evidence were collected: content, response process, internal structure (reliability and discriminant validity), and relations to other variables.
  
  
  RESULTS
  Evidence for content and response process validity is presented. When used by residents, the checklist performed similarly to what was found for attending raters demonstrating good internal consistency (Cronbach α = 0.77) and moderate interrater agreement (intraclass correlation coefficient = 0.47). Residents successfully discerned beginners (P < 0.01, effect size = 2.1) but failed to discriminate between expert and intermediate subjects (P = 0.68, effect size = 0.34). Residents, however, gave significantly higher GRS scores than attending raters across all subject groups (P < 0.001). Moderate correlation was found between GRS and total checklist scores (P = 0.49, P < 0.01).
  
  
  CONCLUSIONS
  This study provides validity evidence for the checklist instrument when used by pediatric residents to assess ILP performances. Compared with attending raters, residents appeared to over-score subjects on the GRS instrument.
authors.affiliation: From the *Department of Pediatrics, Saint Louis University School of Medicine, St Louis, MO; †Department of Pediatrics, Columbia University Medical Center, New York, NY; ‡Department of Pediatrics, Yale University School of Medicine, New Haven, CT; and §Department of Pediatrics, Medical College of Georgia at Georgia Regents University, Augusta, GA.
authors.forename: Colleen
authors.initials: C
authors.lastname: Braun
authors.forename: David O
authors.initials: DO
authors.lastname: Kessler
authors.forename: Marc
authors.initials: M
authors.lastname: Auerbach
authors.forename: Renuka
authors.initials: R
authors.lastname: Mehta
authors.forename: Anthony J
authors.initials: AJ
authors.lastname: Scalzo
authors.forename: James M
authors.initials: JM
authors.lastname: Gerard
mesh_terms: D057189:Checklist; D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D007223:Infant; D007396:Internship and Residency / Q000379:methods*; D016544:Patient Simulation; D010372:Pediatrics / Q000193:education*; D010820:Physicians; D015203:Reproducibility of Results; D013129:Spinal Puncture / Q000592:standards*; D014741:Video Recording