title: Development and validation of a theoretical test of proficiency for video-assisted thoracoscopic surgery (VATS) lobectomy.
abstract: BACKGROUND
  Testing stimulates learning, improves long-term retention, and promotes technical performance. No purpose-orientated test of competence in the theoretical aspects of VATS lobectomy has previously been presented. The purpose of this study was, therefore, to develop and gather validity evidence for a theoretical test on VATS lobectomy consisting of multiple-choice questions.
  
  
  METHODS
  Four European VATS lobectomy experts were interviewed to explore their views on important theoretical VATS lobectomy knowledge (step 1). This information was used to construct the test items in compliance with existing guidelines for multiple-choice questions (step 2). The experts rated the relevance of the items to confirm content validity in a modified Delphi approach (step 3). Finally, the test was administered to physicians, who were categorised into different experience levels based on their experience in VATS procedures overall and in VATS lobectomies specifically. Their answers were used to achieve construct validity (step 4).
  
  
  RESULTS
  Initially, 81 items were constructed and two Delphi iterations reduced the test to 50 items. Item analysis led to the exclusion of 19 items and the mean discrimination index of the 31 final items was 0.26. Cronbach's alpha for internal consistency was 0.75. The mean item difficulty was calculated to 0.63. According to performed VATS procedures, significantly different test performances were detected when comparing the group performances (p = 0.002) and the experts performed significantly better than the novices (p < 0.001) and intermediates (p = 0.01). In the category of performed VATS lobectomies, significant group performances were also found. In this category, the experts were also significantly better than the novices (p < 0.001), the trainees (p = 0.002), and the intermediates (p = 0.01).
  
  
  CONCLUSIONS
  This study led to the development of a theoretical test on VATS lobectomy consisting of multiple-choice questions. Both content and construct validity evidence were established.
authors.affiliation: Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Blegdamsvej 9, 2100, Copenhagen, Denmark, pjl251@alumni.ku.dk.
authors.forename: Mona Meral
authors.initials: MM
authors.lastname: Savran
authors.forename: Henrik Jessen
authors.initials: HJ
authors.lastname: Hansen
authors.forename: René Horsleben
authors.initials: RH
authors.lastname: Petersen
authors.forename: William
authors.initials: W
authors.lastname: Walker
authors.forename: Thomas
authors.initials: T
authors.lastname: Schmid
authors.forename: Signe Rolskov
authors.initials: SR
authors.lastname: Bojsen
authors.forename: Lars
authors.initials: L
authors.lastname: Konge
mesh_terms: D002983:Clinical Competence; D006801:Humans; D008175:Lung Neoplasms / Q000601:surgery*; D011013:Pneumonectomy / Q000379:methods / Q000592:standards*; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires; D020775:Thoracic Surgery, Video-Assisted