title: Validation of the updated ArthroS simulator: face and construct validity of a passive haptic virtual reality simulator with novel performance metrics.
abstract: OBJECTIVE
  To assess the construct and face validity of ArthroS, a passive haptic VR simulator. A secondary aim was to evaluate the novel performance metrics produced by this simulator.
  
  
  METHODS
  Two groups of 30 participants, each divided into novice, intermediate or expert based on arthroscopic experience, completed three separate tasks on either the knee or shoulder module of the simulator. Performance was recorded using 12 automatically generated performance metrics and video footage of the arthroscopic procedures. The videos were blindly assessed using a validated global rating scale (GRS). Participants completed a survey about the simulator's realism and training utility.
  
  
  RESULTS
  This new simulator demonstrated construct validity of its tasks when evaluated against a GRS (p ≤ 0.003 in all cases). Regarding it's automatically generated performance metrics, established outputs such as time taken (p ≤ 0.001) and instrument path length (p ≤ 0.007) also demonstrated good construct validity. However, two-thirds of the proposed 'novel metrics' the simulator reports could not distinguish participants based on arthroscopic experience. Face validity assessment rated the simulator as a realistic and useful tool for trainees, but the passive haptic feedback (a key feature of this simulator) is rated as less realistic.
  
  
  CONCLUSIONS
  The ArthroS simulator has good task construct validity based on established objective outputs, but some of the novel performance metrics could not distinguish between surgical experience. The passive haptic feedback of the simulator also needs improvement. If simulators could offer automated and validated performance feedback, this would facilitate improvements in the delivery of training by allowing trainees to practise and self-assess.
authors.affiliation: Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK. patrick.garfjeldroberts@ndorms.ox.ac.uk.
authors.forename: Patrick
authors.identifier: http://orcid.org/0000-0003-2316-5089
authors.initials: P
authors.lastname: Garfjeld Roberts
authors.affiliation: MDHU Derriford, Plymouth Hospitals NHS Trust, Oxford, UK.
authors.forename: Paul
authors.initials: P
authors.lastname: Guyver
authors.affiliation: Health Education Thames Valley, Oxford, UK.
authors.forename: Mathew
authors.initials: M
authors.lastname: Baldwin
authors.affiliation: The Blizard Institute, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London, UK.
authors.forename: Kash
authors.initials: K
authors.lastname: Akhtar
authors.affiliation: Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.
authors.forename: Abtin
authors.initials: A
authors.lastname: Alvand
authors.affiliation: Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Price
authors.affiliation: Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.
authors.forename: Jonathan L
authors.initials: JL
authors.lastname: Rees
mesh_terms: D001182:Arthroscopy / Q000193:education*; D002983:Clinical Competence; D003198:Computer Simulation; D004521:Educational Measurement; D005246:Feedback; D006801:Humans; D007719:Knee Joint / Q000601:surgery; D015203:Reproducibility of Results; D012785:Shoulder Joint / Q000601:surgery; D000066908:Simulation Training
keywords: Arthroscopy; Simulation; Training; Virtual reality