title: Reducing the number of options on multiple-choice questions: response time, psychometrics and standard setting.
abstract: OBJECTIVE
  Despite significant evidence supporting the use of three-option multiple-choice questions (MCQs), these are rarely used in written examinations for health professions students. The purpose of this study was to examine the effects of reducing four- and five-option MCQs to three-option MCQs on response times, psychometric characteristics, and absolute standard setting judgements in a pharmacology examination administered to health professions students.
  
  
  METHODS
  We administered two versions of a computerised examination containing 98 MCQs to 38 Year 2 medical students and 39 Year 3 pharmacy students. Four- and five-option MCQs were converted into three-option MCQs to create two versions of the examination. Differences in response time, item difficulty and discrimination, and reliability were evaluated. Medical and pharmacy faculty judges provided three-level Angoff (TLA) ratings for all MCQs for both versions of the examination to allow the assessment of differences in cut scores.
  
  
  RESULTS
  Students answered three-option MCQs an average of 5 seconds faster than they answered four- and five-option MCQs (36 seconds versus 41 seconds; p = 0.008). There were no significant differences in item difficulty and discrimination, or test reliability. Overall, the cut scores generated for three-option MCQs using the TLA ratings were 8 percentage points higher (p = 0.04).
  
  
  CONCLUSIONS
  The use of three-option MCQs in a health professions examination resulted in a time saving equivalent to the completion of 16% more MCQs per 1-hour testing period, which may increase content validity and test score reliability, and minimise construct under-representation. The higher cut scores may result in higher failure rates if an absolute standard setting method, such as the TLA method, is used. The results from this study provide a cautious indication to health professions educators that using three-option MCQs does not threaten validity and may strengthen it by allowing additional MCQs to be tested in a fixed amount of testing time with no deleterious effect on the reliability of the test scores.
authors.affiliation: Pharmacy Team-Based Learning Programme, Skaggs School of Pharmacy and Pharmaceutical Sciences, School of Medicine, University of California San Diego (UCSD), La Jolla, California, USA.
authors.forename: Stephen D
authors.initials: SD
authors.lastname: Schneid
authors.forename: Chris
authors.initials: C
authors.lastname: Armour
authors.forename: Yoon Soo
authors.initials: YS
authors.lastname: Park
authors.forename: Rachel
authors.initials: R
authors.lastname: Yudkowsky
authors.forename: Georges
authors.initials: G
authors.lastname: Bordage
mesh_terms: D000328:Adult; D002140:California; D004504:Education, Medical, Undergraduate / Q000379:methods*; D004512:Education, Pharmacy / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006801:Humans; D008297:Male; D011594:Psychometrics; D011930:Reaction Time; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires / Q000592:standards*; D055815:Young Adult