title: What errors make a laparoscopic cancer surgery unsafe? An ad hoc analysis of competency assessment in the National Training Programme for laparoscopic colorectal surgery in England.
abstract: BACKGROUND
  The National Training Programme for laparoscopic colorectal surgery in England was implemented to ensure training was supervised, structured, safe and effective. Delegates were required to pass a competency assessment (sign-off) before undertaking independent practice. This study described the types of errors identified and associated these errors with competency to progress to independent laparoscopic colorectal practice.
  
  
  METHODS
  All sign-off submissions from the start of the process in January 2008 until July 2013 were included. Content analysis was used to categorise errors. Bayes factor (BF) was used to measure the impact of individual error on assessment outcome. A smaller BF indicates that an error has stronger associations with unsuccessful assessments. Bayesian network was employed to graphically represent the reasoning process whereby the chance of successful assessment diminished with the identification of each error. Quality of the error feedback was measured by the area under the ROC curve which linked the predictions from the Bayesian model to the expert verdict.
  
  
  RESULTS
  Among 370 assessments analysed, 240 passed and 130 failed. On average, 2.5 different types of error were identified in each assessment. Cases that were more likely to fail had three or more different types of error (χ(2) = 72, p < 0.0001) and demonstrated poorer technical skills (CAT score <2.7, χ(2) = 164, p < 0.0001). Case complexity or right- versus left-sided resection did not have a significant impact. Errors associated with dissection (BF = 0.18), anastomosis (BF = 0.23) and oncological quality (BF = 0.19) were critical determinants of surgical competence, each reducing the odds of pass by at least fourfold. The area under the ROC curve was 0.84.
  
  
  CONCLUSIONS
  Errors associated with dissection, anastomosis and oncological quality were critical determinants of surgical competency. The detailed error analysis reported in this study can guide the design of future surgical education and clinical training programmes.
authors.affiliation: Division of Surgery, Department of Surgery and Cancer, Imperial College London, 10th Floor, QEQM, St Mary's Hospital, Praed St, London, W2 1NY, UK.
authors.forename: Melody
authors.initials: M
authors.lastname: Ni
authors.affiliation: Division of Surgery, Department of Surgery and Cancer, Imperial College London, 10th Floor, QEQM, St Mary's Hospital, Praed St, London, W2 1NY, UK.
authors.forename: Hugh
authors.initials: H
authors.lastname: Mackenzie
authors.affiliation: Department of Surgery, Royal Cornwall Hospitals Trust, Cornwall, UK.
authors.forename: Adam
authors.initials: A
authors.lastname: Widdison
authors.affiliation: Department of Surgery, St Mark's Hospital, London, UK.
authors.forename: John T
authors.initials: JT
authors.lastname: Jenkins
authors.affiliation: Department of Surgery, Royal Devon and Exeter NHS Foundation Trust, Exeter, UK.
authors.forename: Steve
authors.initials: S
authors.lastname: Mansfield
authors.affiliation: Department of Colorectal Surgery, North Bristol NHS Trust, Southmead Hospital, Bristol, UK.
authors.forename: Tony
authors.initials: T
authors.lastname: Dixon
authors.affiliation: Department of Surgery, Salford Royal NHS Foundation Trust, Salford, UK.
authors.forename: Dominic
authors.initials: D
authors.lastname: Slade
authors.affiliation: Department of Surgery, Plymouth Hospitals NHS Trust, Plymouth, UK.
authors.forename: Mark G
authors.initials: MG
authors.lastname: Coleman
authors.affiliation: Division of Surgery, Department of Surgery and Cancer, Imperial College London, 10th Floor, QEQM, St Mary's Hospital, Praed St, London, W2 1NY, UK. g.hanna@imperial.ac.uk.
authors.forename: George B
authors.initials: GB
authors.lastname: Hanna
mesh_terms: D001499:Bayes Theorem; D002983:Clinical Competence / Q000592:standards / Q000706:statistics & numerical data*; D003082:Colectomy / Q000193:education* / Q000379:methods / Q000592:standards; D015179:Colorectal Neoplasms / Q000601:surgery*; D003107:Colorectal Surgery / Q000193:education* / Q000592:standards; D004739:England; D006801:Humans; D010535:Laparoscopy / Q000193:education* / Q000592:standards; D019300:Medical Errors / Q000009:adverse effects / Q000517:prevention & control / Q000706:statistics & numerical data*; D061214:Patient Safety / Q000592:standards; D012372:ROC Curve; D012007:Rectum / Q000601:surgery*; D012189:Retrospective Studies
keywords: Bayes factor; Bayesian network; Colorectal cancer; Surgical technique; Surgical training