title: Evaluating the ORSIM® simulator for assessment of anaesthetists' skills in flexible bronchoscopy: aspects of validity and reliability.
abstract: BACKGROUND
  Developing expertise in flexible bronchoscopy is limited by inadequate opportunities to train on difficult airways. The new ORSIM bronchoscopy simulator aims to address this by creating virtual patients with difficult airways. This study aims to provide evidence on the validity and reliability of the ORSIM for assessment of subjects on both normal and abnormal airway simulations.
  
  
  METHODS
  Novice, trainee, and expert subjects performed seven simulations of varying difficulty and scored the perceived difficulty for each. Time to completion was measured. Three blinded raters independently scored videos of each subject's performance. We measured inter-rater agreement and the difference in raters' scores between subject groups.
  
  
  RESULTS
  We recruited 28 study subjects, generating 196 videos for analysis. Expert subjects consistently completed the scenarios faster than novices. Overall performance scores showed significant differences between subject groups (P<0.0001). Inter-rater reliability of scores was >0.8.
  
  
  CONCLUSIONS
  Our results provide initial evidence on the validity and reliability of the ORSIM bronchoscopy simulator, supporting its potential value in training and assessment.
authors.affiliation: University of Auckland, Auckland, New Zealand Auckland City Hospital, Auckland, New Zealand.
authors.forename: P A
authors.initials: PA
authors.lastname: Baker
authors.affiliation: University of Auckland, Auckland, New Zealand Auckland City Hospital, Auckland, New Zealand.
authors.forename: J M
authors.initials: JM
authors.lastname: Weller
authors.affiliation: University of Auckland, Auckland, New Zealand.
authors.forename: M J
authors.initials: MJ
authors.lastname: Baker
authors.affiliation: Middlemore Hospital, Auckland, New Zealand.
authors.forename: G L
authors.initials: GL
authors.lastname: Hounsell
authors.affiliation: Auckland City Hospital, Auckland, New Zealand Middlemore Hospital, Auckland, New Zealand.
authors.forename: J
authors.initials: J
authors.lastname: Scott
authors.affiliation: Auckland City Hospital, Auckland, New Zealand.
authors.forename: P J
authors.initials: PJ
authors.lastname: Gardiner
authors.affiliation: University of Auckland, Auckland, New Zealand.
authors.forename: J M D
authors.initials: JM
authors.lastname: Thompson
mesh_terms: D000776:Anesthesiology / Q000193:education*; D020680:Bronchoscopes; D001999:Bronchoscopy / Q000193:education* / Q000295:instrumentation / Q000592:standards; D002983:Clinical Competence; D003198:Computer Simulation; D004502:Education, Medical, Continuing / Q000379:methods*; D005336:Fiber Optic Technology / Q000193:education; D006801:Humans; D009520:New Zealand; D015588:Observer Variation; D015203:Reproducibility of Results
keywords: airway management; computer simulation; education, continuing; scoring methods