title: Development and validation of clinical performance assessment in simulated medical emergencies: an observational study.
abstract: BACKGROUND
  Critical illness is a time-sensitive process which requires practitioners to process vast quantities of data and make decisions rapidly. We have developed a tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN), aimed at enhancing care delivery in such situations. To determine the efficacy of CERTAIN and similar cognitive aids, we developed rubric for evaluating provider performance in a simulated medical resuscitation environments.
  
  
  METHODS
  We recruited 18 clinicians with current valid ACLS certification for evaluation in three simulated medical scenarios designed to mimic typical medical decompensation events routinely experienced in clinical care. Subjects were stratified as experienced or novice based on prior critical care training. A checklist of critical actions was designed using face validity for each scenario to evaluate task completion and performance. Simulation sessions were video recorded and scored by two independent raters. Construct validity was assessed under the assumption that experienced clinicians should perform better than novice clinicians on each task. Reliability was assessed as percentage agreement, kappa statistics and Bland-Altman plots as appropriate.
  
  
  RESULTS
  Eleven experts and seven novices completed evaluation. The overall agreement on common checklist item completion was 84.8 %. The overall model achieved face validity and was consistent with our construct, with experienced clinicians trending towards better performance compared to novices for accuracy and speed of task completion.
  
  
  CONCLUSIONS
  A standardized video assessment tool has potential to provide a valid and reliable method to assess 12 performances of clinicians facing simulated medical emergencies.
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. aydoganer@gmail.com.|Department of Anesthesiology, Mayo Clinic, Rochester, MN, USA. aydoganer@gmail.com.|Department of Anesthesiology and Reanimation, Suleyman Demirel University, Isparta, Turkey. aydoganer@gmail.com.|METRIC group, Mayo Clinic, Rochester, MN, USA. aydoganer@gmail.com.
authors.forename: Aysen
authors.initials: A
authors.lastname: Erdogan
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. dong.yue@mayo.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. dong.yue@mayo.edu.
authors.forename: Yue
authors.initials: Y
authors.lastname: Dong
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. chen.xiaomei@mayo.edu.|Department of Critical Care Medicine, Qilu Hospital of Shandong University, Shandong, China. chen.xiaomei@mayo.edu.
authors.forename: Xiaomei
authors.initials: X
authors.lastname: Chen
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. cns866@mail.harvard.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. cns866@mail.harvard.edu.|Department of Internal Medicine, Boston University Medical Center, Boston, MA, USA. cns866@mail.harvard.edu.
authors.forename: Christopher
authors.initials: C
authors.lastname: Schmickl
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. sevillaberrios.ronaldo@mayo.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. sevillaberrios.ronaldo@mayo.edu.
authors.forename: Ronaldo A
authors.initials: RA
authors.lastname: Sevilla Berrios
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. Garciaarguello.lisbeth@mayo.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. Garciaarguello.lisbeth@mayo.edu.
authors.forename: Lisbeth Y
authors.initials: LY
authors.lastname: Garcia Arguello
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. kashyap.rahul@mayo.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. kashyap.rahul@mayo.edu.
authors.forename: Rahul
authors.initials: R
authors.lastname: Kashyap
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. kilickaya.oguz@mayo.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. kilickaya.oguz@mayo.edu.|Department of Anesthesiology and Reanimation, Gulhane Medical Faculty, Ankara, Turkey. kilickaya.oguz@mayo.edu.
authors.forename: Oguz
authors.initials: O
authors.lastname: Kilickaya
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. pickering.brian@mayo.edu.|Department of Anesthesiology, Mayo Clinic, Rochester, MN, USA. pickering.brian@mayo.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. pickering.brian@mayo.edu.
authors.forename: Brian
authors.initials: B
authors.lastname: Pickering
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. gajic.ognjen@mayo.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. gajic.ognjen@mayo.edu.
authors.forename: Ognjen
authors.initials: O
authors.lastname: Gajic
authors.affiliation: Department of Medicine, Mayo Clinic, Rochester, MN, USA. ohoro.john@mayo.edu.|METRIC group, Mayo Clinic, Rochester, MN, USA. ohoro.john@mayo.edu.
authors.forename: John C
authors.initials: JC
authors.lastname: O'Horo
mesh_terms: D057189:Checklist; D002983:Clinical Competence / Q000592:standards*; D003422:Critical Care; D006801:Humans; D016730:Program Development; D011446:Prospective Studies