Ensuring Competency of Novice Laparoscopic Surgeons-Exploring Standard Setting Methods and their Consequences.
OBJECTIVE
Simulation-based assessment tools have been developed to allow for proficiency-based simulator training in laparoscopy. However, few studies have examined the consequences of different standard setting methods or examined what level of proficiency is considered adequate for trainees. The objectives of the present study were to explore the consequences of different standard setting methods and to examine the proficiency level that surgical trainees are expected to reach, before performing supervised surgery on patients.


METHODS
Study participants undertook the Training and Assessment of Basic Laparoscopic Techniques test. The tests were video-recorded and rated using a simple scoring system based on number of errors and time. Participants were then asked to assess how high a score a novice should reach before performing supervised surgery on a patient. We then compared 3 methods of standard setting: expert performance level, contrasting groups method, and a modified Angoff method.


METHODS
The study was conducted at the Copenhagen Academy for Medical Education and Simulation. The academy provides surgical simulation training in laparoscopy for trainees at the hospitals in the Capital Region and the Zealand Region of Denmark.


METHODS
Participants were recruited among surgical trainees in their first year of specialty training from surgery, gynecology, and urology departments. A total of 40 participants were included and completed the trial.


RESULTS
The different standard setting methods resulted in different pass/fail levels. At the expert performance level, the pass/fail level was 474 points-the contrasting groups method resulted in 358 points and the modified Angoff method resulted in 311 points among experienced surgeons, and 386 points among trainees. The different proficiency levels resulted in a failure rate of 0% to 50% of experienced surgeons and a pass rate of 0% to 25% of novices. Novice laparoscopic surgeons set a higher pass/fail level than experienced surgeons did (p = 0.008).


CONCLUSIONS
Required proficiency levels varies depending on the standard setting method used, which highlights the importance of using an established standard setting method to set the pass/fail level.
Department of Surgery, Roskilde and Koege Hospital, Koege, Denmark; Copenhagen Academy for Medical Education and Simulation, Rigshospitalet, Copenhagen, Denmark. Electronic address: ebbe.thinggaard@gmail.com.
Ebbe
E
Thinggaard
Department of Surgery, Roskilde and Koege Hospital, Koege, Denmark.
Flemming
F
Bjerrum
Department of Gynecology, Juliane Marie Centre, Rigshospitalet, Copenhagen, Denmark.
Jeanett
J
Strandbygaard
Department of Surgery, Roskilde and Koege Hospital, Koege, Denmark.
Ismail
I
Gögenur
Copenhagen Academy for Medical Education and Simulation, Rigshospitalet, Copenhagen, Denmark.
Lars
L
Konge
D000328:Adult; D002983:Clinical Competence; D003198:Computer Simulation; D003718:Denmark; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods; D005260:Female; D006801:Humans; D010535:Laparoscopy / Q000193:education*; D008297:Male; D000066908:Simulation Training / Q000379:methods*; D013043:Specialties, Surgical / Q000193:education*; D014584:User-Computer Interface; D014741:Video Recording
Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; laparoscopy; medical education; minimally invasive surgery; simulation; standard setting; training