Simulation With PARTS (Phase-Augmented Research and Training Scenarios): A Structure Facilitating Research and Assessment in Simulation.
BACKGROUND
Assessment in simulation is gaining importance, as are scenario design methods increasing opportunity for assessment. We present our approach to improving measurement in complex scenarios using PARTS [Phase-Augmented Research and Training Scenarios], essentially separating cases into clearly delineated phases.


METHODS
We created 7 PARTS with real-time rating instruments and tested these in 63 cases during 4 weeks of simulation. Reliability was tested by comparing real-time rating with postsimulation video-based rating using the same instrument. Validity was tested by comparing preintervention and postintervention total results, by examining the difference in improvement when focusing on the phase-specific results addressed by the intervention, and further explored by trying to demonstrate the discrete improvement expected from proficiency in the rare occurrence of leader inclusive behavior.


RESULTS
Intraclass correlations [3,1] between real-time and postsimulation ratings were 0.951 (95% confidence interval [CI], 0.794-0.990), 1.00 (95% CI, --to--), 0.948 (95% CI, 0.783-0.989), and 0.995 (95% CI, 0.977-0.999) for 3 phase-specific scores and total scenario score, respectively. Paired t tests of prelecture-postlecture performance showed an improvement of 14.26% (bias-corrected and accelerated bootstrap [BCa] 95% CI, 4.71-23.82; P = 0.009) for total performance but of 28.57% (BCa 95% CI, 13.84-43.30; P = 0.002) for performance in the respective phase. The correlation of total scenario performance with leader inclusiveness was not significant (rs = 0.228; BCa 95% CI. -0.082 to 0.520; P = 0.119) but significant for specific phase performance (rs = 0.392; BCa 95% CI, 0.118-0.632; P = 0.006).


CONCLUSIONS
The PARTS allowed for improved reliability and validity of measurements in complex scenarios.
From the Institute of Anesthesiology (C.J.S., A.M., M.D., A.K., D.R.S., B.G.), University Hospital Zürich; and Department of Management, Technology, and Economics Organization (M.W., M.K., G.G.), Work and Technology Group, ETH Zürich, Zürich, Switzerland.
Carl J
CJ
Schick
Mona
M
Weiss
Michaela
M
Kolbe
Adrian
A
Marty
Micha
M
Dambach
Axel
A
Knauth
Donat R
DR
Spahn
Gudela
G
Grote
Bastian
B
Grande
D002983:Clinical Competence; D006801:Humans; D007396:Internship and Residency / Q000458:organization & administration* / Q000592:standards; D015203:Reproducibility of Results; D000066908:Simulation Training / Q000458:organization & administration* / Q000592:standards