title: Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool.
abstract: Errors in judgment during laparoscopic cholecystectomy can lead to bile duct injuries and other complications. Despite correlations between outcomes, expertise and advanced cognitive skills, current methods to evaluate these skills remain subjective, rater- and situation-dependent and non-systematic. The purpose of this study was to develop objective metrics using a Web-based platform and to obtain validity evidence for their assessment of decision-making during laparoscopic cholecystectomy.
  
  
  
  An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from six institutions completed a 12-item assessment, developed based on a cognitive task analysis. Five items required subjects to draw their answer on the surgical field, and accuracy scores were calculated based on an algorithm derived from experts' responses ("visual concordance test", VCT). Test-retest reliability, internal consistency, and correlation with self-reported experience, Global Operative Assessment of Laparoscopic Skills (GOALS) score and Objective Performance Rating Scale (OPRS) score were calculated. Questionnaires were administered to evaluate the platform's usability, feasibility and educational value.
  
  
  
  Thirty-nine subjects (17 surgeons, 22 trainees) participated. There was high test-retest reliability (intraclass correlation coefficient = 0.95; n = 10) and internal consistency (Cronbach's α = 0.87). The assessment demonstrated significant differences between novices, intermediates and experts in total score (p < 0.01) and VCT score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.83, p < 0.01) and between total case number and VCT (ρ = 0.82, p < 0.01), and moderate to high correlations between total score and GOALS (ρ = 0.66, p = 0.05), VCT and GOALS (ρ = 0.83, p < 0.01), total score and OPRS (ρ = 0.67, p = 0.04), and VCT and OPRS (ρ = 0.78, p = 0.01). Most subjects agreed or strongly agreed that the platform and assessment was easy to use [n = 29 (78 %)], facilitates learning intra-operative decision-making [n = 28 (81 %)], and should be integrated into surgical training [n = 28 (76 %)].
  
  
  
  This study provides preliminary validity evidence for a novel interactive platform to objectively assess decision-making during laparoscopic cholecystectomy.
authors.affiliation: Department of Surgery, McGill University, Montreal, Canada. amin.madani@mail.mcgill.ca.|Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada. amin.madani@mail.mcgill.ca.
authors.forename: Amin
authors.initials: A
authors.lastname: Madani
authors.affiliation: Department of Gastroenterological Surgery II, Hokkaido University, Sapporo, Japan.
authors.forename: Yusuke
authors.initials: Y
authors.lastname: Watanabe
authors.affiliation: Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada.
authors.forename: Elif
authors.initials: E
authors.lastname: Bilgic
authors.affiliation: Department of Surgery and Cancer, Imperial College London, London, UK.
authors.forename: Philip H
authors.initials: PH
authors.lastname: Pucher
authors.affiliation: Department of Surgery, McGill University, Montreal, Canada.|Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada.
authors.forename: Melina C
authors.initials: MC
authors.lastname: Vassiliou
authors.affiliation: Department of Surgery, McGill University, Montreal, Canada.|Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada.|Faculty of Medicine, Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, Canada.
authors.forename: Rajesh
authors.initials: R
authors.lastname: Aggarwal
authors.affiliation: Department of Surgery, McGill University, Montreal, Canada.|Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada.|Faculty of Medicine, Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, Canada.
authors.forename: Gerald M
authors.initials: GM
authors.lastname: Fried
authors.affiliation: Department of Surgery, McGill University, Montreal, Canada.
authors.forename: Elliot J
authors.initials: EJ
authors.lastname: Mitmaker
authors.affiliation: Department of Surgery, McGill University, Montreal, Canada.|Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada.
authors.forename: Liane S
authors.initials: LS
authors.lastname: Feldman
mesh_terms: D000328:Adult; D017081:Cholecystectomy, Laparoscopic / Q000193:education*; D000066491:Clinical Decision-Making; D003194:Computer-Assisted Instruction; D005260:Female; D006801:Humans; D020407:Internet; D008297:Male; D011446:Prospective Studies; D015203:Reproducibility of Results; D066231:Surgeons; D014741:Video Recording
keywords: Assessment; Cholecystectomy; Competency; Education; Performance; Visual concordance test