title: Design, implementation, and psychometric analysis of a scoring instrument for simulated pediatric resuscitation: a report from the EXPRESS pediatric investigators.
abstract: BACKGROUND
  Robustly tested instruments for quantifying clinical performance during pediatric resuscitation are lacking. Examining Pediatric Resuscitation Education through Simulation and Scripting Collaborative was established to conduct multicenter trials of simulation education in pediatric resuscitation, evaluating performance with multiple instruments, one of which is the Clinical Performance Tool (CPT). We hypothesize that the CPT will measure clinical performance during simulated pediatric resuscitation in a reliable and valid manner.
  
  
  METHODS
  Using a pediatric resuscitation scenario as a basis, a scoring system was designed based on Pediatric Advanced Life Support algorithms comprising 21 tasks. Each task was scored as follows: task not performed (0 points); task performed partially, incorrectly, or late (1 point); and task performed completely, correctly, and within the recommended time frame (2 points). Study teams at 14 children's hospitals went through the scenario twice (PRE and POST) with an interposed 20-minute debriefing. Both scenarios for each of eight study teams were scored by multiple raters. A generalizability study, based on the PRE scores, was conducted to investigate the sources of measurement error in the CPT total scores. Inter-rater reliability was estimated based on the variance components. Validity was assessed by repeated measures analysis of variance comparing PRE and POST scores.
  
  
  RESULTS
  Sixteen resuscitation scenarios were reviewed and scored by seven raters. Inter-rater reliability for the overall CPT score was 0.63. POST scores were found to be significantly improved compared with PRE scores when controlled for within-subject covariance (F1,15 = 4.64, P < 0.05). The variance component ascribable to rater was 2.4%.
  
  
  CONCLUSIONS
  Reliable and valid measures of performance in simulated pediatric resuscitation can be obtained from the CPT. Future studies should examine the applicability of trichotomous scoring instruments to other clinical scenarios, as well as performance during actual resuscitations.
authors.affiliation: Divisions of Emergency Medicine, The Children's Hospital of Philadelphia, Philadelphia, PA, USA. donoghue@email.chop.edu
authors.forename: Aaron
authors.initials: A
authors.lastname: Donoghue
authors.forename: Kathleen
authors.initials: K
authors.lastname: Ventre
authors.forename: John
authors.initials: J
authors.lastname: Boulet
authors.forename: Marisa
authors.initials: M
authors.lastname: Brett-Fleegler
authors.forename: Akira
authors.initials: A
authors.lastname: Nishisaki
authors.forename: Frank
authors.initials: F
authors.lastname: Overly
authors.forename: Adam
authors.initials: A
authors.lastname: Cheng
mesh_terms: D000465:Algorithms; D000704:Analysis of Variance; D016887:Cardiopulmonary Resuscitation / Q000379:methods*; D002648:Child; D039702:Current Procedural Terminology; D004521:Educational Measurement; D004522:Educational Status; D006801:Humans; D010372:Pediatrics; D011594:Psychometrics / Q000379:methods*; D015203:Reproducibility of Results; D012107:Research Design; D013223:Statistics as Topic; D013647:Task Performance and Analysis; D014741:Video Recording