title: MCQ tests in Advanced Trauma Life Support (ATLS©): Development and revision.
abstract: BACKGROUND
  In Advanced Trauma Life Support (ATLS©) courses, multiple choice question (MCQ) tests are used to assess student's post course knowledge. As part of the ninth Edition Revision Process, existing MCQ tests were reviewed and revised by an International MCQ Revision group. The aim of this study was to evaluate the revision procedure and its effects.
  
  
  METHODS
  Based on psychometric data and evidence based guidelines for adequate MCQ item and test development, a detailed stepwise approach was determined and followed to evaluate the existing MCQs, and to guide test item revision or replacement.
  
  
  RESULTS
  The MCQ Revision group composed three new draft test versions comprising of 40 MCQs each. These were beta-tested among ATLS Instructors in various countries involved in ATLS. Psychometric analysis demonstrated that a minority of MCQ items required revision to create three equally balanced tests. After these final adjustments, a new set of three validated MCQ tests was available for use in 9th edition ATLS provider courses. Beta testing was performed using instructors but not students. The failure rate amongst students of ATLS provider courses increased significantly after introduction of the new MCQ tests.
  
  
  CONCLUSIONS
  ATLS tests were revised and updated using current evidence based guidelines and psychometric analysis. Difficulty of the tests was not initially beta-tested on students. Increasing test item discrimination and quality resulted in lower test scores by students.
authors.affiliation: PLATO, Leiden University, P.O. Box 9555, 2300 RB Leiden, The Netherlands. Electronic address: j.bustraan@lumc.nl.
authors.forename: Jacqueline
authors.initials: J
authors.lastname: Bustraan
authors.affiliation: ATLS Netherlands, Tilburg, The Netherlands. Electronic address: whenny@online.nl.
authors.forename: Walter
authors.initials: W
authors.lastname: Henny
authors.affiliation: Department of Surgery, University of Calgary and Alberta Health Services - Calgary Foothills Medical Center, Department of Surgery, 10th Floor N. Tower, 1403-29th Street, NW Calgary, AB, Canada T2N 2T9.
authors.forename: John B
authors.initials: JB
authors.lastname: Kortbeek
authors.affiliation: Department of Surgery, Medical College of Wisconsin, Milwaukee, USA. Electronic address: kbrasel@mcw.edu.
authors.forename: Karen J
authors.initials: KJ
authors.lastname: Brasel
authors.affiliation: Faculty of Health Witten-Herdecke University, Alfred-Herrhausen-Str 50, 584 Witten, NRW, Germany. Electronic address: Marzellus.Hofmann@uni-wh.de.
authors.forename: Marzellus
authors.initials: M
authors.lastname: Hofmann
authors.affiliation: Department of Trauma Surgery, Leiden University Medical Center, Leiden, The Netherlands. Electronic address: i.b.schipper@lumc.nl.
authors.forename: Inger B
authors.initials: IB
authors.lastname: Schipper
mesh_terms: D061906:Advanced Trauma Life Support Care / Q000592:standards; D002983:Clinical Competence / Q000592:standards*; D004502:Education, Medical, Continuing / Q000592:standards*; D004521:Educational Measurement; D006801:Humans; D015397:Program Evaluation; D014194:Traumatology / Q000193:education*
keywords: ATLS; Assessment; MCQ; Trauma education