title: Reliable assessment of operative performance.
abstract: BACKGROUND
  There is no consensus regarding the number of intraoperative assessments required to reliably measure trainee performance. This study used generalizability theory (GT) to describe factors contributing to score variance and to estimate the number of assessments needed to achieve high standards of reliability.
  
  
  METHODS
  While performing laparoscopic procedures, trainees were assessed by the attending surgeon using Global Operative Assessment of Laparoscopic Skills (GOALS). Data were collected prospectively (2-month intervals), assessing each trainee multiple times. Reliability coefficient was calculated using trainees, cases, and raters as factors.
  
  
  RESULTS
  Eighteen trainees were included for a total of 65 assessments. Total variance in scores was accounted for as follows: 66.1% by trainees, 31.6% by the interaction between trainees and cases, and 2.3% by raters. At least 3 cases are required for reliable scores using GOALS.
  
  
  CONCLUSIONS
  Trainees accounted for most of the variance in GOALS scores with a minimum of 3 cases required to improve the reliability of the scores obtained. These data may guide the implementation of performance assessments in surgical training programs.
authors.affiliation: Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, D6-257, Montreal, QC, H3G1A4, Canada.
authors.forename: Elif
authors.initials: E
authors.lastname: Bilgic
authors.affiliation: Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, D6-257, Montreal, QC, H3G1A4, Canada; Department of Gastroenterological Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Hokkaido, Japan.
authors.forename: Yusuke
authors.initials: Y
authors.lastname: Watanabe
authors.affiliation: Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, D6-257, Montreal, QC, H3G1A4, Canada.
authors.forename: Katherine
authors.initials: K
authors.lastname: McKendy
authors.affiliation: Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, D6-257, Montreal, QC, H3G1A4, Canada.
authors.forename: Amani
authors.initials: A
authors.lastname: Munshi
authors.affiliation: Department of Biostatistics, Hokkaido University Graduate School of Medicine, Sapporo, Hokkaido, Japan.
authors.forename: Yoichi M
authors.initials: YM
authors.lastname: Ito
authors.affiliation: Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, D6-257, Montreal, QC, H3G1A4, Canada.
authors.forename: Gerald M
authors.initials: GM
authors.lastname: Fried
authors.affiliation: Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, D6-257, Montreal, QC, H3G1A4, Canada.
authors.forename: Liane S
authors.initials: LS
authors.lastname: Feldman
authors.affiliation: Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, D6-257, Montreal, QC, H3G1A4, Canada. Electronic address: melina.vassiliou@mcgill.ca.
authors.forename: Melina C
authors.initials: MC
authors.lastname: Vassiliou
mesh_terms: D002983:Clinical Competence; D005793:Generalization, Psychological; D006801:Humans; D007396:Internship and Residency; D010535:Laparoscopy / Q000193:education*; D059032:Learning Curve; D011214:Practice, Psychological; D011446:Prospective Studies; D015203:Reproducibility of Results
keywords: Assessment; Generalizability theory; Reliability; Surgery