title: Construct and face validity of the educational computer-based environment (ECE) assessment scenarios for basic endoneurosurgery skills.
abstract: In neurosurgery education, there is a paradigm shift from time-based training to criterion-based model for which competency and assessment becomes very critical. Even virtual reality simulators provide alternatives to improve education and assessment in neurosurgery programs and allow for several objective assessment measures, there are not many tools for assessing the overall performance of trainees. This study aims to develop and validate a tool for assessing the overall performance of participants in a simulation-based endoneurosurgery training environment.
  
  
  
  A training program was developed in two levels: endoscopy practice and beginning surgical practice based on four scenarios. Then, three experiments were conducted with three corresponding groups of participants (Experiment 1, 45 (32 beginners, 13 experienced), Experiment 2, 53 (40 beginners, 13 experienced), and Experiment 3, 26 (14 novices, 12 intermediate) participants). The results analyzed to understand the common factors among the performance measurements of these experiments. Then, a factor capable of assessing the overall skill levels of surgical residents was extracted. Afterwards, the proposed measure was tested to estimate the experience levels of the participants. Finally, the level of realism of these educational scenarios was assessed.
  
  
  
  The factor formed by time, distance, and accuracy on simulated tasks provided an overall performance indicator. The prediction correctness was very high for the beginners than the one for experienced surgeons in Experiments 1 and 2. When non-dominant hand is used in a surgical procedure-based scenario, skill levels of surgeons can be better predicted. The results indicate that the scenarios in Experiments 1 and 2 can be used as an assessment tool for the beginners, and scenario-2 in Experiment 3 can be used as an assessment tool for intermediate and novice levels. It can be concluded that forming the balance between perceived action capacities and skills is critical for better designing and developing skill assessment surgical simulation tools.
authors.affiliation: Department of Software Engineering, Faculty of Engineering, Atilim University, Ankara, Turkey. necagiltay@gmail.com.
authors.forename: Nergiz Ercil
authors.initials: NE
authors.lastname: Cagiltay
authors.affiliation: Department of Psychology, Faculty of Arts and Sciences, Çankaya University, Ankara, Turkey.
authors.forename: Erol
authors.initials: E
authors.lastname: Ozcelik
authors.affiliation: Department of Computer Engineering, Faculty of Engineering, Atilim University, Ankara, Turkey.
authors.forename: Gokhan
authors.initials: G
authors.lastname: Sengul
authors.affiliation: Department of Neurosurgery, Faculty of Medicine, Hacettepe University, Ankara, Turkey.
authors.forename: Mustafa
authors.initials: M
authors.lastname: Berker
mesh_terms: D000328:Adult; D002983:Clinical Competence / Q000706:statistics & numerical data*; D003198:Computer Simulation; D004521:Educational Measurement / Q000379:methods; D004724:Endoscopy / Q000193:education*; D005260:Female; D006801:Humans; D008297:Male; D019635:Neurosurgical Procedures / Q000193:education*; D015203:Reproducibility of Results; D000066908:Simulation Training / Q000379:methods*; D014584:User-Computer Interface
keywords: Assessment; Basic endoneurosurgery skills; Navigation skills; Simulation-based surgical education; Surgical experience levels