title: Using performance in multiple simulated scenarios to assess bronchoscopy skills.
abstract: BACKGROUND
  International guidelines suggest that trainees should perform at least 100 flexible bronchoscopies in a supervised setting, but this number is not evidence based. An objective assessment method could provide educational feedback to trainees and help supervisors decide when basic competency is established. No former assessment instrument has been able to distinguish between trainees and experts.
  
  
  OBJECTIVE
  The aim of this study was to explore the validity and reliability of a new assessment procedure relating to testing operators across multiple tasks with increasing difficulty using a standardized scoring form.
  
  
  METHODS
  The test was administered on a virtual reality bronchoscopy simulator to a total of 42 test subjects (14 senior consultants, 14 trainees and 14 medical students). The inter-rater reliability of the test procedure was explored according to examination of test results from 10 subjects using 3 raters (2 blinded).
  
  
  RESULTS
  We found a high inter-rater reliability (Cronbach's α = 0.90). The assessment procedure sufficiently differentiated the performance of the 3 groups (p < 0.001). Assessment of 6 procedures was necessary to secure a generalizability coefficient >0.80.
  
  
  CONCLUSIONS
  The new assessment procedure proved both valid and reliable.
authors.affiliation: Centre for Clinical Education, University of Copenhagen and Capital Region of Denmark, Copenhagen, Denmark. lkonge @ yahoo.dk
authors.forename: Lars
authors.initials: L
authors.lastname: Konge
authors.forename: Henrik
authors.initials: H
authors.lastname: Arendrup
authors.forename: Christian
authors.initials: C
authors.lastname: von Buchwald
authors.forename: Charlotte
authors.initials: C
authors.lastname: Ringsted
mesh_terms: D000328:Adult; D000368:Aged; D001999:Bronchoscopy / Q000592:standards*; D002983:Clinical Competence; D003198:Computer Simulation; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D017064:Process Assessment, Health Care; D015203:Reproducibility of Results; D013997:Time Factors; D055815:Young Adult