Using automatic item generation to create multiple-choice test items.
BACKGROUND
Many tests of medical knowledge, from the undergraduate level to the level of certification and licensure, contain multiple-choice items. Although these are efficient in measuring examinees' knowledge and skills across diverse content areas, multiple-choice items are time-consuming and expensive to create. Changes in student assessment brought about by new forms of computer-based testing have created the demand for large numbers of multiple-choice items. Our current approaches to item development cannot meet this demand.


METHODS
We present a methodology for developing multiple-choice items based on automatic item generation (AIG) concepts and procedures. We describe a three-stage approach to AIG and we illustrate this approach by generating multiple-choice items for a medical licensure test in the content area of surgery.


RESULTS
To generate multiple-choice items, our method requires a three-stage process. Firstly, a cognitive model is created by content specialists. Secondly, item models are developed using the content from the cognitive model. Thirdly, items are generated from the item models using computer software. Using this methodology, we generated 1248 multiple-choice items from one item model.


CONCLUSIONS
Automatic item generation is a process that involves using models to generate items using computer technology. With our method, content specialists identify and structure the content for the test items, and computer technology systematically combines the content to generate new test items. By combining these outcomes, items can be generated automatically.
Department of Surgery, University of Alberta, Edmonton, Alberta, Canada. mark.gierl@ualberta.ca
Mark J
MJ
Gierl
Hollis
H
Lai
Simon R
SR
Turner
D002983:Clinical Competence; D003194:Computer-Assisted Instruction / Q000379:methods / Q000592:standards; D003201:Computers; D004504:Education, Medical, Undergraduate / Q000379:methods* / Q000592:standards; D004521:Educational Measurement / Q000379:methods* / Q000592:standards; D013502:General Surgery / Q000193:education*; D006801:Humans; D017145:Models, Educational