A comparison of methods for dealing with troublesome examination questions.
This study compared two methods for identifying troublesome examination questions: a postexam student "challenge committee" process and a "question citation" procedure allowing students to identify troublesome questions during an exam. Challenged and cited questions on eight examinations given to second-year medical students were compared with respect to the percentage of correct responses (%CR) and the discrimination index (DI). Challenged questions had a significantly lower mean value for %CR (53.2 +/- 22.5, n = 74) than unchallenged questions (84.7 +/- 15.4, n = 428) but the two groups did not differ with respect to DI. For cited questions, the number of citations was correlated with %CR (r = -0.5480, p < 0.001) but not with DI. The group of questions that received three or more citations (n = 145) included 67.6% of all challenged questions and 78.3% of challenges that eventually led to a change in an answer key. A simple citation procedure completed during an exam could replace a complicated, post-exam challenge committee process and achieve the same goal of identifying troublesome questions.
Norton
J M
JM
University of New England, College of Osteopathic Medicine, Biddeford, Maine 04005, USA. jnorton@mailbox.une.edu
D004501:Education, Medical / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D005069:Evaluation Studies as Topic; D006801:Humans