title: Testing basic competency in knee arthroscopy using a virtual reality simulator: exploring validity and reliability.
abstract: BACKGROUND
  Diagnostic knee arthroscopy is a common procedure that orthopaedic residents are expected to learn early in their training. Arthroscopy requires a different skill set from traditional open surgery, and many orthopaedic residents feel less prepared for arthroscopic procedures. Virtual reality simulation training and testing provide an opportunity to ensure basic competency before proceeding to supervised procedures in patients.
  
  
  METHODS
  Twenty-six physicians (thirteen novices and thirteen experienced arthroscopic surgeons) were voluntarily recruited to perform a test consisting of five arthroscopic procedures on a knee arthroscopy simulator. Performance was evaluated by obtaining predefined metrics from the simulator for each procedure, and z-scores, describing suboptimal performance, were calculated from the metrics. The intercase reliability of the simulator metrics was explored by calculating an intraclass correlation coefficient. Finally, a pass-or-fail standard was set with use of the contrasting groups method, and the consequences of the pass-or-fail standard were explored.
  
  
  RESULTS
  One procedure was excluded from the final test because of a lack of validity. The total Z-scores for the four procedures included in the final test showed an intercase reliability of 0.87 (95% confidence interval, 0.78 to 0.93). The total mean z-score (and standard deviation) was 38.6 ± 27.3 points for the novices and 0.0 ± 9.1 points for the experienced surgeons (p < 0.0005). The pass-or-fail standard was set at a total z-score of 15.5 points, resulting in two of the novices passing the test and a single experienced surgeon failing the test.
  
  
  CONCLUSIONS
  By combining four procedures on a virtual reality arthroscopy simulator, it was possible to create a valid, reliable, and feasible test of basic arthroscopic competency and to establish a credible pass-or-fail standard.
  
  
  CONCLUSIONS
  The simulation-based test and pass-or-fail standard could aid in assessing and ensuring basic competency of future orthopaedic residents before proceeding to supervised procedures in patients.
authors.affiliation: Faculty of Health and Medical Sciences, University of Copenhagen, Matthæusgade 48C, Nr. 418, 1666 Copenhagen V, Denmark. E-mail address: madsemil4@gmail.com.
authors.forename: Mads Emil
authors.initials: ME
authors.lastname: Jacobsen
authors.affiliation: Department of Orthopedic Surgery, Copenhagen University Hospital Hvidovre, Himmerlandsvej 20, No. 3, 2720 Vanløse, Denmark.
authors.forename: Morten Jon
authors.initials: MJ
authors.lastname: Andersen
authors.affiliation: Arthroscopic Center Amager, Amager Hospital, Italiensvej 1, 2300 Copenhagen S, Denmark.
authors.forename: Claus Ol
authors.initials: CO
authors.lastname: Hansen
authors.affiliation: Centre for Clinical Education (CEKU), University of Copenhagen and the Capital Region of Denmark Rigshospitalet afsnit 5404, Blegdamsvej 9, 2100 Copenhagen Ø, Denmark.
authors.forename: Lars
authors.initials: L
authors.lastname: Konge
mesh_terms: D000328:Adult; D001182:Arthroscopy / Q000193:education*; D005260:Female; D006801:Humans; D007396:Internship and Residency; D007719:Knee Joint; D008297:Male; D009985:Orthopedics / Q000193:education*; D015203:Reproducibility of Results; D014584:User-Computer Interface