title: The effect of a 'don't know' option on test scores: number-right and formula scoring compared.
abstract: OBJECTIVE
  In multiple-choice tests using a 'don't-know' option the number of correct minus incorrect answers was used as the test score (formula scoring) in order to reduce the measurement error resulting from random guessing. In the literature diverging results are reported when comparing formula scoring and number-right scoring, the scoring method without the don't-know option.
  
  
  METHODS
  To investigate which method was most appropriate, both scoring methods were used in true-false tests (block tests) taken at the end of a second- and third-year educational module (block). The students were asked to answer each item initially by choosing from the response options true, false or don't know, and secondly to replace all don't-know answers by a true-false answer.
  
  
  METHODS
  Maastricht University, The Netherlands.
  
  
  METHODS
  Medical students.
  
  
  RESULTS
  The correct scores for the don't-know answered items were found to be 4.5% and 5.9%, respectively, higher than expected with pure random guesswork. This represents a source of bias with formula scoring, because students who were less willing to guess obtained lower scores. The average difference in the correct minus incorrect score for the two scoring methods (2.5%, P < 0.001, and 3.4%, P < 0. 001, respectively) indicates the size of the bias (compare: the standard deviation of the score equals 11%). Test reliability was higher with formula scoring (0.72 vs. 0.66 and 0.74 vs. 0.66), but the difference decreased when the test was restricted to items which were close to the core content of the block (0.81 vs. 0.77, resp. 0. 75 vs. 0.70).
  
  
  CONCLUSIONS
  In deciding what scoring method to use, less bias (number-right scoring) has to be weighed against higher reliability (formula scoring). Apart from these psychometric reasons educational factors must be considered.
authors.affiliation: Department of Medical Informatics, Anatomy and Embryology, Maastricht University, The Netherlands.
authors.forename: A M
authors.initials: AM
authors.lastname: Muijtjens
authors.forename: H V
authors.initials: HV
authors.lastname: Mameren
authors.forename: R J
authors.initials: RJ
authors.lastname: Hoogenboom
authors.forename: J L
authors.initials: JL
authors.lastname: Evers
authors.forename: C P
authors.initials: CP
authors.lastname: van der Vleuten
mesh_terms: D004504:Education, Medical, Undergraduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D012680:Sensitivity and Specificity