title: Structured performance assessment in three pediatric emergency scenarios: a validation study.
abstract: OBJECTIVE
  To develop and validate 3 performance evaluation checklists (PECs) for systematic performance assessment in 3 clinical scenarios: cardiopulmonary arrest, dyspnea with oxygen desaturation after intubation, and respiratory syncytial virus (RSV).
  
  
  METHODS
  The 3 PECs were developed using an integrative approach and used to rate 50 training sessions in a simulator environment by different raters. Construct validity was tested by correlating the checklist scores with external constructs (ie, global rating, team experience level, and time to action). Further interrater reliability was tested for all 3 PECs.
  
  
  RESULTS
  The PECs for the desaturation and cardiopulmonary arrest scenarios were valid and reliable, whereas the PEC for RSV had limited validity and reliability.
  
  
  CONCLUSIONS
  For 2 pediatric emergencies, the PEC is a valid and reliable tool for systematic performance assessment. The unsatisfactory results for the PEC for RSV may be related to limitations of the simulation setting and require further investigation. Structured assessment of clinical performance can augment feedback on technical performance aspects and is essential for training purposes as well as for research. Only reliable and valid performance measures will allow medical educators to accurately evaluate the behavioral effects of training interventions and further enhance the quality of patient care.
authors.affiliation: Department of Management, Technology, and Economics, ETH Zurich, Zurich, Switzerland. Electronic address: jschmutz@ethz.ch.
authors.forename: Jan
authors.initials: J
authors.lastname: Schmutz
authors.affiliation: Institute for Patient Safety, University Hospital Bonn, Bonn, Germany.
authors.forename: Tanja
authors.initials: T
authors.lastname: Manser
authors.affiliation: Dr. von Hauner University Children's Hospital, Munich, Germany.
authors.forename: Julia
authors.initials: J
authors.lastname: Keil
authors.affiliation: Department of Pediatric Cardiology, Pulmology and Pediatric Intensive Care Medicine, University Children's Hospital, Tübingen, Germany.
authors.forename: Ellen
authors.initials: E
authors.lastname: Heimberg
authors.affiliation: Dr. von Hauner University Children's Hospital, Munich, Germany.
authors.forename: Florian
authors.initials: F
authors.lastname: Hoffmann
mesh_terms: D057189:Checklist / Q000592:standards*; D002648:Child; D004417:Dyspnea / Q000175:diagnosis* / Q000628:therapy*; D004630:Emergencies; D004638:Emergency Treatment; D005260:Female; D006323:Heart Arrest / Q000175:diagnosis* / Q000628:therapy*; D006801:Humans; D007223:Infant; D008297:Male; D015203:Reproducibility of Results; D018357:Respiratory Syncytial Virus Infections / Q000175:diagnosis* / Q000628:therapy*