title: Design, validity, and reliability of a pediatric resident JumpSTART disaster triage scoring instrument.
abstract: OBJECTIVE
  To design an instrument for scoring residents learning pediatric disaster triage (PDT), and to test the validity and reliability of the instrument.
  
  
  METHODS
  We designed a checklist-based scoring instrument including PDT knowledge and skills and performance, as well as a global assessment. Learners' performance in a 10-patient school bus crash simulation was video recorded and scored with the instrument. Learners triaged the patients with a color-coded algorithm (JumpSTART, Simple Triage and Rapid Treatment). Three evaluators observed the recordings and scored triage performance for each learner. Internal and construct validity of the instrument were established via comparison of resident performance by postgraduate year (PGY) and correlating instrument items with overall score. Validity was assessed with analysis of variance and the D statistic. We calculated evaluators' intraclass correlation coefficient (ICC) for each patient, skill, triage decision, and global assessment.
  
  
  RESULTS
  There were 37 learners and 111 observations. There was no difference in total scores by PGY (P = .77), establishing internal validity. Regarding construct validity, most instrument items had a D statistic of >0.5. The overall ICC among scores was 0.83 (95% confidence interval [CI] 0.74-0.89). Individual patient score reliability was high and was greatest among patients with head injury (ICC 0.86; 95% CI 0.79-0.91). Reliability was low for an ambulatory patient (ICC 0.29; 95% CI 0.07-0.48). Triage skills evaluation showed excellent reliability, including airway management (ICC 0.91; 95% CI 0.86-0.94) and triage speed (ICC 0.81; 95% CI 0.72-0.88). The global assessment had moderate reliability for skills (ICC 0.63; 95% CI 0.47-0.75) and knowledge (ICC 0.64; 95% CI 0.49-0.76).
  
  
  CONCLUSIONS
  We report the validity and reliability testing of a PDT-scoring instrument. Validity was confirmed with no performance differential by PGY. Reliability of the scoring instrument for most patient-level triage, knowledge, and specific skills was high.
authors.affiliation: Department of Pediatrics, Yale University School of Medicine, New Haven, CT 06511, USA. mark.cicero@yale.edu
authors.forename: Mark X
authors.initials: MX
authors.lastname: Cicero
authors.forename: Antonio
authors.initials: A
authors.lastname: Riera
authors.forename: Veronika
authors.initials: V
authors.lastname: Northrup
authors.forename: Marc
authors.initials: M
authors.lastname: Auerbach
authors.forename: Kevin
authors.initials: K
authors.lastname: Pearson
authors.forename: Carl R
authors.initials: CR
authors.lastname: Baum
mesh_terms: D002983:Clinical Competence; D054597:Disaster Medicine / Q000193:education*; D004503:Education, Medical, Graduate; D006801:Humans; D007396:Internship and Residency; D010372:Pediatrics / Q000193:education*; D015203:Reproducibility of Results; D014218:Triage / Q000379:methods*