title: Toward feasible, valid, and reliable video-based assessments of technical surgical skills in the operating room.
abstract: OBJECTIVE
  To determine the feasibility, validity, inter-rater, and intertest reliability of 4 previously published video-based rating scales, for technical skills assessment on a benchmark laparoscopic procedure.
  
  
  BACKGROUND
  Assessment of technical skills is crucial to the demonstration and maintenance of competent healthcare practitioners. Traditional assessment methods are prone to subjectivity through a lack of proven validity and reliability.
  
  
  METHODS
  Nineteen surgeons (6 novice and 13 experienced) performed a median of 2 laparoscopic cholecystectomies each (range 1-5) on 53 patients within 2 Academic Surgical Departments. All patients had a diagnosis of biliary colic. Surgical technical skills were rated posthoc in a blinded manner by 2 experienced observers on 4 video-based rating scales. The different scales used had been developed to assess generic or procedure-specific technical skills in a global manner, or on a procedure-specific checklist.
  
  
  RESULTS
  Six of 53 procedures were excluded on the basis of intraoperative difficulty. Of the remaining 47 procedures, 14 were performed by 6 novice surgeons and 33 by the 13 experienced surgeons. There were statistically significant differences between performance of the 2 groups on the generic global rating scale (median 24 vs. 27, P = 0.031), though not on procedural or checklist-based scales. All scales demonstrated inter-rater reliability (alpha = 0.58-0.76), though only the global rating scales exhibited intertest reliability (alpha = 0.72).
  
  
  CONCLUSIONS
  Video-based technical skills evaluation in the operating room is feasible, valid and reliable. Global rating scales hold promise for summative assessment, though further work is necessary to elucidate the value of procedural rating scales.
authors.affiliation: Department of Biosurgery and Surgical Technology, Imperial College London, London, UK. rajesh.aggarwal@impeiral.ac.uk
authors.forename: Rajesh
authors.initials: R
authors.lastname: Aggarwal
authors.forename: Teodor
authors.initials: T
authors.lastname: Grantcharov
authors.forename: Krishna
authors.initials: K
authors.lastname: Moorthy
authors.forename: Thor
authors.initials: T
authors.lastname: Milland
authors.forename: Ara
authors.initials: A
authors.lastname: Darzi
mesh_terms: D001649:Bile Duct Diseases / Q000601:surgery; D017081:Cholecystectomy, Laparoscopic / Q000379:methods* / Q000592:standards; D002983:Clinical Competence / Q000592:standards*; D005240:Feasibility Studies; D006801:Humans; D015588:Observer Variation; D009873:Operating Rooms / Q000600:supply & distribution*; D015203:Reproducibility of Results; D020535:Video-Assisted Surgery / Q000592:standards*