title: Video assessment of laparoscopic skills by novices and experts: implications for surgical education.
abstract: Previous investigators have shown that novices are able to assess surgical skills as reliably as expert surgeons. The purpose of this study was to determine how novices and experts arrive at these graded scores when assessing laparoscopic skills and the potential implications this may have for surgical education.
  
  
  
  Four novices and four general laparoscopic surgeons evaluated 59 videos of a suturing task using a 5-point scale. Average novice and expert evaluator scores for each video and the average number of times that scores were changed were compared. Intraclass correlation coefficients were used to determine inter-rater and test-retest reliability. Evaluators were asked to define the number of videos they needed to watch before they could confidently grade and to describe how they were able to distinguish between different levels of expertise.
  
  
  
  There were no significant differences in mean scores assigned by the two evaluator groups. Novices changed their scores more frequently compared to experts, but this did not reach statistical significance. There was excellent inter-rater reliability between the two groups (ICC = 0.91, CI 0.85-0.95) and good test-retest reliability (ICC > 0.83). On average, novices and experts reported that they needed to watch 13.8 ± 2.4 and 8.5 ± 2.5 videos, respectively, before they could confidently grade. Both groups also identified similar qualitative indicators (e.g., instrument control).
  
  
  
  Evaluators with varying levels of expertise can reliably grade performance of an intracorporeal suturing task. While novices were less confident in their grading, both groups were able to assign comparable scores and identify similar elements of a suturing skill as being important in terms of assessment.
authors.affiliation: Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
authors.forename: Celine
authors.initials: C
authors.lastname: Yeung
authors.affiliation: Division of General and Thoracic Surgery, The Hospital for Sick Children, Toronto, ON, Canada.|Centre for Image-Guided Innovation and Therapeutic Intervention, Toronto, ON, Canada.
authors.forename: Brian
authors.initials: B
authors.lastname: Carrillo
authors.affiliation: Division of Otolaryngology, The Hospital for Sick Children, Toronto, ON, Canada.
authors.forename: Victor
authors.initials: V
authors.lastname: Pope
authors.affiliation: Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
authors.forename: Shahob
authors.initials: S
authors.lastname: Hosseinpour
authors.affiliation: Division of General and Thoracic Surgery, The Hospital for Sick Children, Toronto, ON, Canada.|Centre for Image-Guided Innovation and Therapeutic Intervention, Toronto, ON, Canada.
authors.forename: J Ted
authors.initials: JT
authors.lastname: Gerstle
authors.affiliation: Division of General and Thoracic Surgery, The Hospital for Sick Children, Toronto, ON, Canada. georges.azzie@sickkids.ca.|Centre for Image-Guided Innovation and Therapeutic Intervention, Toronto, ON, Canada. georges.azzie@sickkids.ca.
authors.forename: Georges
authors.initials: G
authors.lastname: Azzie
mesh_terms: D000293:Adolescent; D000328:Adult; D002983:Clinical Competence / Q000706:statistics & numerical data*; D006801:Humans; D010535:Laparoscopy / Q000193:education*; D015203:Reproducibility of Results; D066231:Surgeons; D013536:Suture Techniques / Q000193:education*; D014741:Video Recording; D055815:Young Adult
keywords: Laparoscopic; Novice evaluators; Suturing skill; Video assessment