Validation of two instruments to assess technical bronchoscopic skill using virtual reality simulation.
BACKGROUND
In an era of increased emphasis on patient safety and competency-based education, demonstration of significant variability in the teaching of flexible bronchoscopy has led to initiatives for new standardized curricula and assessment tools.


OBJECTIVE
To evaluate the efficacy of such curricula and to assess bronchoscopic skill, 2 measuring instruments have been developed: the Bronchoscopy Skills and Tasks Assessment Tool (BSTAT) and the Bronchoscopy Step-by-Step Evaluation Tool (BSET). We studied the validity and reliability of these 2 instruments.


METHODS
Two independent testers simultaneously scored 22 volunteer participants at 3 levels [novice (n = 7), fellow (n = 8) and attending (n = 7)] on a virtual reality bronchoscopy simulator using the 2 instruments; each participant was tested twice, in 2 separate sessions. Intertester and test-retest reliability were analyzed with intraclass correlations (ICC); ANOVA was used to assess concurrent validity based on the subjects' expected skill level.


RESULTS
The ICCs between the testers were 0.98 for both the BSTAT and BSET. Comparison of the scores between the sessions showed high test-retest reliability by ICC (0.86 and 0.85 for BSTAT and BSET respectively), with a small yet statistically significant learning effect. The novice group's scores were lower than the fellows' and attendings' (p < 0.001) for both the BSTAT and BSET; the fellows' scores were consistently lower than the attendings' on both tests, yet the differences were not statistically significant.


CONCLUSIONS
This validation study of 2 objective tests of bronchoscopic skill demonstrated high reliability and concurrent validity. These instruments can now be used to evaluate the effectiveness of new competency-based bronchoscopy curricula.
Davoudi
Mohsen
M
Medical Center, University of California, Irvine, CA 92868, USA.
Osann
Kathryn
K
Colt
Henri G
HG
D001999:Bronchoscopy; D002983:Clinical Competence; D003198:Computer Simulation; D004521:Educational Measurement; D006801:Humans; D015272:Pulmonary Medicine / Q000193:education*; D014584:User-Computer Interface