Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments.
BACKGROUND
Multiple-choice questions (MCQs) are frequently used to assess students in health science disciplines. However, few educators have formal instruction in writing MCQs and MCQ items often have item-writing flaws. The purpose of this study was to examine the impact of item-writing flaws on student achievement in high-stakes assessments in a nursing programme in an English-language university in Hong Kong.


METHODS
From a larger sample, we selected 10 summative test papers that were administered to undergraduate nursing students in 1 nursing department. All test items were reviewed for item-writing flaws by a 4-person consensus panel. Items were classified as 'flawed' if they contained > or = 1 flaw. Items not containing item-writing violations were classified as 'standard'. For each paper, 2 separate scales were computed: a total scale which reflected the characteristics of the assessment as administered and a standard scale which reflected the characteristics of a hypothetical assessment including only unflawed items.


RESULTS
The proportion of flawed items on the 10 test papers ranged from 28-75%; 47.3% of all items were flawed. Fewer examinees passed the standard scale than the total scale (748 [90.6%] versus 779 [94.3%]). Conversely, the proportion of examinees obtaining a score > or = 80% was higher on the standard scale than the total scale (173 [20.9%] versus 120 [14.5%]).


CONCLUSIONS
Flawed MCQ items were common in high-stakes nursing assessments but did not disadvantage borderline students, as has been previously demonstrated. Conversely, high-achieving students were more likely than borderline students to be penalised by flawed items.
Tarrant
Marie
M
Department of Nursing Studies, Faculty of Medicine, University of Hong Kong, Hong Kong, China. tarrantm@hku.hk
Ware
James
J
D002755:Choice Behavior; D004506:Education, Nursing / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D004522:Educational Status; D006723:Hong Kong; D013663:Teaching / Q000379:methods*