title: Direct Observation versus Endoscopic Video Recording-Based Rating with the Objective Structured Assessment of Technical Skills for Training of Laparoscopic Cholecystectomy.
abstract: OBJECTIVE
  The validated Objective Structured Assessment of Technical Skills (OSATS) score is used for evaluating laparoscopic surgical performance. It consists of two subscores, a Global Rating Scale (GRS) and a Specific Technical Skills (STS) scale. The OSATS has accepted construct validity for direct observation ratings by experts to discriminate between trainees' levels of experience. Expert time is scarce. Endoscopic video recordings would facilitate assessment with the OSATS. We aimed to compare video OSATS with direct OSATS.
  
  
  METHODS
  We included 79 participants with different levels of experience [58 medical students, 15 junior residents (novices), and 6 experts]. Performance of a cadaveric porcine laparoscopic cholecystectomy (LC) was evaluated with OSATS by blinded expert raters by direct observation and then as an endoscopic video recording. Operative time was recorded.
  
  
  RESULTS
  Direct OSATS rating and video OSATS rating correlated significantly (x03C1; = 0.33, p = 0.005). Significant construct validity was found for direct OSATS in distinguishing between students or novices and experts. Students and novices were not different in direct OSATS or video OSATS. Mean operative times varied for students (73.4 ± 9.0 min), novices (65.2 ± 22.3 min), and experts (46.8 ± 19.9 min). Internal consistency was high between the GRS and STS subscores for both direct and video OSATS with Cronbach's α of 0.76 and 0.86, respectively. Video OSATS and operative time in combination was a better predictor of direct OSATS than each single parameter.
  
  
  CONCLUSIONS
  Direct OSATS rating was better than endoscopic video rating for differentiating between students or novices and experts for LC and should remain the standard approach for the discrimination of experience levels. However, in the absence of experts for direct rating, video OSATS supplemented with operative time should be used instead of single parameters for predicting direct OSATS scores.
authors.affiliation: Department of General, Visceral and Transplantation Surgery, University of Heidelberg, Heidelberg, Germany.
authors.forename: Felix
authors.initials: F
authors.lastname: Nickel
authors.forename: Jonathan D
authors.initials: JD
authors.lastname: Hendrie
authors.forename: Christian
authors.initials: C
authors.lastname: Stock
authors.forename: Mohamed
authors.initials: M
authors.lastname: Salama
authors.forename: Anas A
authors.initials: AA
authors.lastname: Preukschas
authors.forename: Jonas D
authors.initials: JD
authors.lastname: Senft
authors.forename: Karl F
authors.initials: KF
authors.lastname: Kowalewski
authors.forename: Martin
authors.initials: M
authors.lastname: Wagner
authors.forename: Hannes G
authors.initials: HG
authors.lastname: Kenngott
authors.forename: Georg R
authors.initials: GR
authors.lastname: Linke
authors.forename: Lars
authors.initials: L
authors.lastname: Fischer
authors.forename: Beat P
authors.initials: BP
authors.lastname: Müller-Stich
mesh_terms: D017081:Cholecystectomy, Laparoscopic / Q000193:education*; D002983:Clinical Competence; D004521:Educational Measurement; D016145:Endoscopy, Digestive System; D006801:Humans; D061646:Operative Time; D014741:Video Recording