Value of an objective assessment tool in the operating room.
BACKGROUND
Concerns about the achievement of surgical proficiency during residency are increasing. To objectify surgical skills, the Objective Structured Assessment of Technical Skills (OSATS) was developed and proven valid, feasible and reliable for use in laboratory settings. This study aimed to evaluate the value of this tool for intraoperative use.


METHODS
Residents were assessed with an OSATS after every procedure they performed as the primary surgeon during a 3-month clinical rotation in gynecological surgery. We mapped individual learning curves (OSATS scores plotted against experience) and established the average procedure-specific learning curve. We used linear mixed models to assess the relation between performance and experience.


RESULTS
Nine residents were recruited and 319 OSATS analyzed. Individual learning curves revealed progression beyond 24 of 30 OSATS points for 7 residents. Performance on the average procedure improved with experience, and the OSATS score increased by an average of 1.10 points per assessed procedure (p=0.008, 95% confidence interval 0.44-1.77). Median OSATS scores ranged from 18 to 30 among the 21 assessors.


CONCLUSIONS
Intraoperative implementation of OSATS seems to offer important advantages: structured feedback is facilitated, and learning curves enable insight into individual progression. However, doubts have been raised about the objectivity of the tool. Therefore, caution is warranted in using it for graduation and certification.
Department of Gynecology, Leiden University Medical Center, Leiden, the Netherlands. E.Hiemstra@lumc.nl
Ellen
E
Hiemstra
Wendela
W
Kolkman
Ron
R
Wolterbeek
Baptist
B
Trimbos
Frank Willem
FW
Jansen
D000328:Adult; D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006176:Gynecology / Q000193:education*; D006801:Humans; D007396:Internship and Residency / Q000592:standards; D007721:Knowledge of Results, Psychological; D016014:Linear Models; D008297:Male; D009426:Netherlands; D009873:Operating Rooms
J Pediatr Surg. 2003 May;38(5):720-4
12720179
Med Teach. 2008;30(2):212-6
18464149
Am J Obstet Gynecol. 2006 Aug;195(2):617-21; discussion 621-3
16890561
Surg Endosc. 2001 Oct;15(10):1066-70
11727071
Ann Surg. 2008 Feb;247(2):372-9
18216547
Best Pract Res Clin Obstet Gynaecol. 2006 Feb;20(1):173-87
16278096
Am J Surg. 2001 Aug;182(2):110-6
11574079
J Endourol. 2002 May;16(4):251-6
12042110
Obstet Gynecol. 2000 May;95(5):783-5
10775747
Gynecol Obstet Invest. 2007;64(1):1-7
17192714
Br J Surg. 2007 Nov;94(11):1315-6
17939136
BMJ. 2003 Nov 1;327(7422):1032-7
14593041
J Minim Invasive Gynecol. 2006 Jan-Feb;13(1):4-9
16431316
Surg Endosc. 2005 Nov;19(11):1498-502
16206008
Obstet Gynecol. 2000 Jul;96(1):146-50
10862857
BMJ. 1999 Apr 3;318(7188):887-8
10102830
Arch Surg. 2000 Jul;135(7):786-92
10896371
J Am Coll Surg. 2004 Jan;198(1):105-10
14698317
Obstet Gynecol. 2004 Aug;104(2):308-13
15292004
N Engl J Med. 2006 Dec 21;355(25):2664-9
17182991
Br J Surg. 1997 Feb;84(2):273-8
9052454