Analysis of simulated angiographic procedures. Part 2: extracting efficiency data from audio and video recordings.
OBJECTIVE
To create and test methods of extracting efficiency data from recordings of simulated renal stent procedures.


METHODS
Task analysis was performed and used to design a standardized testing protocol. Five experienced angiographers then performed 16 renal stent simulations using the Simbionix AngioMentor angiographic simulator. Audio and video recordings of these simulations were captured from multiple vantage points. The recordings were synchronized and compiled. A series of efficiency metrics (procedure time, contrast volume, and tool use) were then extracted from the recordings. The intraobserver and interobserver variability of these individual metrics was also assessed. The metrics were converted to costs and aggregated to determine the fixed and variable costs of a procedure segment or the entire procedure.


RESULTS
Task analysis and pilot testing led to a standardized testing protocol suitable for performance assessment. Task analysis also identified seven checkpoints that divided the renal stent simulations into six segments. Efficiency metrics for these different segments were extracted from the recordings and showed excellent intra- and interobserver correlations. Analysis of the individual and aggregated efficiency metrics demonstrated large differences between segments as well as between different angiographers. These differences persisted when efficiency was expressed as either total or variable costs.


CONCLUSIONS
Task analysis facilitated both protocol development and data analysis. Efficiency metrics were readily extracted from recordings of simulated procedures. Aggregating the metrics and dividing the procedure into segments revealed potential insights that could be easily overlooked because the simulator currently does not attempt to aggregate the metrics and only provides data derived from the entire procedure. The data indicate that analysis of simulated angiographic procedures will be a powerful method of assessing performance in interventional radiology.
Duncan
James R
JR
Mallinckrodt Institute of Radiology, Washington University School of Medicine, 510 S. Kingshighway Blvd., St. Louis, MO 63110, USA. Duncanj@mir.wustl.edu
Kline
Benjamin
B
Glaiberman
Craig B
CB
D000792:Angiography; D002983:Clinical Competence; D003198:Computer Simulation; D003194:Computer-Assisted Instruction; D004503:Education, Medical, Graduate / Q000379:methods*; D006801:Humans; D015588:Observer Variation; D010865:Pilot Projects; D015641:Radiography, Interventional; D015642:Radiology, Interventional / Q000193:education*; D012077:Renal Artery / Q000000981:diagnostic imaging / Q000601:surgery; D015203:Reproducibility of Results; D015607:Stents; D013647:Task Performance and Analysis; D014584:User-Computer Interface; D014656:Vascular Surgical Procedures; D014742:Videodisc Recording