The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value.
BACKGROUND
Failure to adhere to standard item-writing guidelines may render examination questions easier or more difficult than intended. Item complexity describes the cognitive skill level required to obtain a correct answer. Higher cognitive examination items promote critical thinking and are recommended to prepare students for clinical training. This study evaluated faculty-authored examinations to determine the impact of item-writing flaws and item complexity on the difficulty and discrimination value of examination items used to assess third year veterinary students.


METHODS
The impact of item-writing flaws and item complexity (cognitive level I-V) on examination item difficulty and discrimination value was evaluated on 1925 examination items prepared by clinical faculty for third year veterinary students.


RESULTS
The mean (± SE) percent correct (83.3 % ± 17.5) was consistent with target values in professional education, and the mean discrimination index (0.18 ± 0.17) was slightly lower than recommended (0.20). More than one item-writing flaw was identified in 37.3 % of questions. The most common item-writing flaws were awkward stem structure, implausible distractors, longest response is correct, and responses are series of true-false statements. Higher cognitive skills (complexity level III-IV) were required to correctly answer 38.4 % of examination items. As item complexity increased, item difficulty and discrimination values increased. The probability of writing discriminating, difficult examination items decreased when implausible distractors and all of the above were used, and increased if the distractors were comprised of a series of true/false statements. Items with four distractors were not more difficult or discriminating than items with three distractors.


CONCLUSIONS
Preparation of examination questions targeting higher cognitive levels will increase the likelihood of constructing discriminating items. Use of implausible distractors to complete a five-option multiple choice question does not strengthen the discrimination value.
Department of Clinical Sciences, College of Veterinary Medicine, Kansas State University, Manhattan, KS, USA. brush@vet.ksu.edu.
Bonnie R
BR
Rush
Department of Clinical Sciences, College of Veterinary Medicine, Kansas State University, Manhattan, KS, USA.
David C
DC
Rankin
Department of Clinical Sciences, College of Veterinary Medicine, Kansas State University, Manhattan, KS, USA.
Brad J
BJ
White
Item-writing flaws; Multiple choice examinations; Pre-clinical education; True-false examinations; Veterinary education
Am J Pharm Educ. 2012 Aug 10;76(6):114
22919090
Indian J Community Med. 2014 Jan;39(1):17-20
24696535
Med Teach. 2006 Sep;28(6):566-8
17074708
Acad Emerg Med. 2010 Oct;17 Suppl 2:S31-7
21199081
Pak J Med Sci. 2014 Jan;30(1):3-6
24639820
Med Educ. 2004 Sep;38(9):974-9
15327679
BMC Med Educ. 2007 Nov 28;7:49
18045500
Int J Health Sci (Qassim). 2011 Jan;5(1):39-43
22489228
Med Klin (Munich). 2007 Dec 15;102(12):993-1001
18075719
Perspect Med Educ. 2013 Nov;2(5-6):252-63
24203858
Adv Health Sci Educ Theory Pract. 2012 Aug;17(3):369-76
21837548
Med Educ Online. 2013 Mar 06;18:1-5
23469935
Natl Med J India. 2008 May-Jun;21(3):130-3
19004145
Adv Health Sci Educ Theory Pract. 2015 May;20(2):305-20
24973998
Med Educ. 2010 Jan;44(1):109-17
20078762
Med Teach. 2010;32(7):e300-7
20653373
Adv Health Sci Educ Theory Pract. 2002;7(3):235-41
12510145
Acad Med. 2002 Feb;77(2):156-61
11841981
Adv Health Sci Educ Theory Pract. 2005;10(2):133-43
16078098
Acad Med. 2002 Oct;77(10 Suppl):S103-4
12377719
Nurse Educ Pract. 2006 Dec;6(6):354-63
19040902
Nurse Educ Today. 2010 Aug;30(6):539-43
20053488
BMC Med Educ. 2010 Nov 24;10:85
21106066
BMC Med Educ. 2009 Jul 07;9:40
19580681
Med Educ. 2008 Oct;42(10):959-66
18823514
J Cancer Educ. 1996 Spring;11(1):17-24
8777151