Structured assessment format for evaluating operative reports in general surgery.
BACKGROUND
Despite its multifaceted importance, no validated or reliable tools assess the quality of the dictated operative note. This study determined the construct validity, interrater reliability, and internal consistency of a Structured Assessment Format for Evaluating Operative Reports (SAFE-OR) in general surgery.


METHODS
SAFE-OR was developed by using consensus criteria set forth by the Canadian Association of General Surgeons. This instrument includes a structured assessment and a global quality rating scale. Residents divided into novice and experienced groups viewed and dictated a videotaped laparoscopic sigmoid colectomy. Blinded, independent faculty evaluators graded the transcribed reports using SAFE-OR.


RESULTS
Twenty-one residents participated in the study. Mean structured assessment scores (out of 44) were significantly lower for novice versus experienced residents (23.3 +/- 5.2 vs 34.1 +/- 6.0, t = .001). Mean global quality scores (out of 45) were similarly lower for novice residents (25.6 +/- 4.7 vs 35.9 +/- 7.6, t = .006). Interclass correlation coefficients were .98 (95% confidence interval, .96-.99) for structured assessment and .93 (95% confidence interval, .83-.97) for global quality scales. Cronbach alpha coefficients for internal consistency were .85 for structured assessment and .96 for global quality assessment scales.


CONCLUSIONS
SAFE-OR shows significant construct validity, excellent interrater reliability, and high internal consistency. This tool will allow educators to objectively evaluate the quality of trainee operative reports and provide a mechanism for implementing, monitoring, and refining curriculum for dictation skills.
Vergis
Ashley
A
Section of General Surgery, University of Manitoba, Winnipeg, Manitoba, Canada. ashleyvergis@hotmail.com <ashleyvergis@hotmail.com>
Gillman
Lawrence
L
Minor
Samuel
S
Taylor
Mark
M
Park
Jason
J
D013505:Digestive System Surgical Procedures; D004521:Educational Measurement; D013502:General Surgery / Q000193:education*; D006801:Humans; D007396:Internship and Residency; D008499:Medical Records / Q000592:standards*