A Taxonomy of Delivery and Documentation Deviations During Delivery of High-Fidelity Simulations.
BACKGROUND
We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations.


METHODS
Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development.


RESULTS
The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy.


CONCLUSIONS
This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.
From the Department of Anesthesiology, University of Pittsburgh Medical Center and the Winter Institute for Simulation Education and Research (WISER), Pittsburgh, PA (W.R.M.); The Division of Anesthesiology Critical Care Medicine, Department of Anesthesiology, Vanderbilt University School of Medicine, Nashville, TN (A.B.); Research and Data Resources, Foundation for Advancement of International Medical Education and Research, Philadelphia, PA (J.R.B.); Department of Biomedical Informatics, University of Pittsburgh School of Medicine; and Department of Dental Public Health, University of Pittsburgh School of Dental Medicine, Pittsburgh, PA (T.B.); Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Pittsburgh, PA (E.T.); Department of Anesthesiology, The Mayo Clinic, Rochester, MN (L.T.); Anesthesiology, Icahn School of Medicine at the Mt Sinai Medical Center, New York City, NY (S.DM.); Department of Anesthesiology and Critical Care Medicine, and UNM BATCAVE Simulation Center, University of New Mexico School of Medicine, Albuquerque, NM (J.P.R.); Department of Biostatistics, Vanderbilt University, Nashville, TN (M.S.S.); Clinical Skills and Simulation, Cooper Medical School of Rowan University, Cooper University Hospital, Camden, NJ (A.B.); Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, and Center for Medical Simulation, Boston, MA (J.C) Immersive Learning, Stanford University, Stanford, CA; and VA Palo Alto, Palo Alto, CA (D.M.G.); Department of Anesthesiology, Icahn School of Medicine at the Mt Sinai Medical Center, New York City, NY (A.L.); Department of Anesthesiology, Northwestern University Feinberg School of Medicine, Chicago, IL (C.P.); Department of Anesthesiology and Neurosurgery, Pennsylvania State University College of Medicine, Hershey, PA (E.S.); Anesthesiology Department, University of California, Los Angeles, CA (R.S.); and Center for Research and Innovation in Systems Safety, Institute for Medicine and Public Health, Vanderbilt University; Geriatric Research, Education, and Clinical Center (GRECC); VA Tennessee Valley Healthcare System; and Anesthesiology, Biomedical Informatics, and Medical Education, Vanderbilt University School of Medicine, Nashville, TN (M.B.W.).
William R
WR
McIvor
Arna
A
Banerjee
John R
JR
Boulet
Tanja
T
Bekhuis
Eugene
E
Tseytlin
Laurence
L
Torsher
Samuel
S
DeMaria
John P
JP
Rask
Matthew S
MS
Shotwell
Amanda
A
Burden
Jeffrey B
JB
Cooper
David M
DM
Gaba
Adam
A
Levine
Christine
C
Park
Elizabeth
E
Sinz
Randolph H
RH
Steadman
Matthew B
MB
Weinger
D002983:Clinical Competence / Q000592:standards; D003697:Delphi Technique; D004282:Documentation / Q000145:classification* / Q000592:standards*; D004521:Educational Measurement; D006801:Humans; D008348:Manikins; D016544:Patient Simulation; D018875:Vocabulary, Controlled