The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.
Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.
Tarrant
Marie
M
Department of Nursing Studies, Faculty of Medicine, University of Hong Kong, 4/F, William M.W. Mong Block, 21 Sassoon Road, Hong Kong. tarrantm@hku.hk
Knierim
Aimee
A
Hayes
Sasha K
SK
Ware
James
J
D015982:Bias; D003071:Cognition; D032882:Comprehension; D003463:Cues; D016002:Discriminant Analysis; D004507:Education, Nursing, Associate / Q000592:standards; D004508:Education, Nursing, Baccalaureate / Q000592:standards; D004518:Education, Professional, Retraining / Q000592:standards; D004521:Educational Measurement / Q000379:methods* / Q000592:standards; D005181:Faculty, Nursing / Q000592:standards; D019983:Guideline Adherence / Q000592:standards; D017408:Guidelines as Topic; D006301:Health Services Needs and Demand; D006723:Hong Kong; D006801:Humans; D008037:Linguistics / Q000592:standards; D011939:Mental Recall; D015402:Nursing Education Research; D011594:Psychometrics; D011786:Quality Control; D013338:Students, Nursing / Q000523:psychology; D011795:Surveys and Questionnaires / Q000592:standards*; D014956:Writing / Q000592:standards*