title: Using cognitive models to develop quality multiple-choice questions.
abstract: With the recent interest in competency-based education, educators are being challenged to develop more assessment opportunities. As such, there is increased demand for exam content development, which can be a very labor-intense process. An innovative solution to this challenge has been the use of automatic item generation (AIG) to develop multiple-choice questions (MCQs). In AIG, computer technology is used to generate test items from cognitive models (i.e. representations of the knowledge and skills that are required to solve a problem). The main advantage yielded by AIG is the efficiency in generating items. Although technology for AIG relies on a linear programming approach, the same principles can also be used to improve traditional committee-based processes used in the development of MCQs. Using this approach, content experts deconstruct their clinical reasoning process to develop a cognitive model which, in turn, is used to create MCQs. This approach is appealing because it: (1) is efficient; (2) has been shown to produce items with psychometric properties comparable to those generated using a traditional approach; and (3) can be used to assess higher order skills (i.e. application of knowledge). The purpose of this article is to provide a novel framework for the development of high-quality MCQs using cognitive models.
authors.affiliation: a Department of Medicine , The Ottawa Hospital, University of Ottawa , Ottawa , Ontario , Canada ;
authors.forename: Debra
authors.initials: D
authors.lastname: Pugh
authors.affiliation: b Medical Council of Canada , Ottawa , Ontario , Canada ;
authors.forename: Andre
authors.initials: A
authors.lastname: De Champlain
authors.affiliation: c Faculty of Education , University of Alberta , Edmonton , Alberta , Canada ;
authors.forename: Mark
authors.initials: M
authors.lastname: Gierl
authors.affiliation: d Faculty of Medicine and Dentistry , University of Alberta , Edmonton , Alberta , Canada.
authors.forename: Hollis
authors.initials: H
authors.lastname: Lai
authors.affiliation: a Department of Medicine , The Ottawa Hospital, University of Ottawa , Ottawa , Ontario , Canada ;|b Medical Council of Canada , Ottawa , Ontario , Canada ;
authors.forename: Claire
authors.initials: C
authors.lastname: Touchie
mesh_terms: D003162:Competency-Based Education; D004504:Education, Medical, Undergraduate; D004521:Educational Measurement / Q000379:methods* / Q000592:standards*; D006801:Humans; D008960:Models, Psychological