title: [To what extent do reviewers of multiple-choice questions need to be trained? A comparison between handing out information sheets and brief workshop sessions].
abstract: BACKGROUND
  A competent review process is crucial to ensure the quality of multiple-choice (MC) questions. However, the acquisition of reviewing skills should not cause any unnecessary additional burden for a medical staff that is already facing heavy workloads.
  
  
  METHODS
  100 MC questions, for which an expert review existed, were presented to 12 novices. In advance, six participants received a specific information sheet covering critical information for high-calibre review; the other six participants attended a 2.5-hour workshop covering the same information. The review results of both groups were analysed with a licensed version of the IBM software SPSS 19.0 (SPSS Inc., Chicago, IL).
  
  
  RESULTS
  The results of the workshop group were distinctly closer to the experts' results (gold standard) than those of the information sheet group. For the quantitatively important category of medium quality MC questions, the results of the workshop group did not significantly differ from the experts' results. In the information sheet group the results were significantly poorer than the experts', regardless of the quality of the questions.
  
  
  CONCLUSIONS
  Distributing specific information sheets to MC question reviewers is not sufficient for ensuring the quality of the review so that - regardless of the increased effort involved - a recommendation to conduct specific workshops must be made.
authors.affiliation: Abt. Kardioanästhesiologie, Universitätsklinikum Ulm, Deutschland. Electronic address: wolfgang.oechsner@uniklinik-ulm.de.
authors.forename: Wolfgang
authors.initials: W
authors.lastname: Öchsner
authors.affiliation: Abt. für Anatomie und Zellbiologie, Universität Ulm, Deutschland.
authors.forename: Anja
authors.initials: A
authors.lastname: Böckers
mesh_terms: D005858:Germany; D006801:Humans; D011795:Surveys and Questionnaires
keywords: Effekte von Merkblättern; Effekte von Workshops; MC-Aufgaben; Multiple-choice questions; Qualitätssicherung; Review-Prozess; effects of information sheets; effects of workshops; efficient use of resources; effizienter Ressourceneinsatz; quality assurance; review process