title: A comparison of methods for dealing with troublesome examination questions.
abstract: This study compared two methods for identifying troublesome examination questions: a postexam student "challenge committee" process and a "question citation" procedure allowing students to identify troublesome questions during an exam. Challenged and cited questions on eight examinations given to second-year medical students were compared with respect to the percentage of correct responses (%CR) and the discrimination index (DI). Challenged questions had a significantly lower mean value for %CR (53.2 +/- 22.5, n = 74) than unchallenged questions (84.7 +/- 15.4, n = 428) but the two groups did not differ with respect to DI. For cited questions, the number of citations was correlated with %CR (r = -0.5480, p < 0.001) but not with DI. The group of questions that received three or more citations (n = 145) included 67.6% of all challenged questions and 78.3% of challenges that eventually led to a change in an answer key. A simple citation procedure completed during an exam could replace a complicated, post-exam challenge committee process and achieve the same goal of identifying troublesome questions.
authors.affiliation: University of New England, College of Osteopathic Medicine, Biddeford, Maine 04005, USA. jnorton@mailbox.une.edu
authors.forename: J M
authors.initials: JM
authors.lastname: Norton
mesh_terms: D004501:Education, Medical / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D005069:Evaluation Studies as Topic; D006801:Humans