title: A General Method of Empirical Q-matrix Validation.
abstract: In contrast to unidimensional item response models that postulate a single underlying proficiency, cognitive diagnosis models (CDMs) posit multiple, discrete skills or attributes, thus allowing CDMs to provide a finer-grained assessment of examinees' test performance. A common component of CDMs for specifying the attributes required for each item is the Q-matrix. Although construction of Q-matrix is typically performed by domain experts, it nonetheless, to a large extent, remains a subjective process, and misspecifications in the Q-matrix, if left unchecked, can have important practical implications. To address this concern, this paper proposes a discrimination index that can be used with a wide class of CDM subsumed by the generalized deterministic input, noisy "and" gate model to empirically validate the Q-matrix specifications by identifying and replacing misspecified entries in the Q-matrix. The rationale for using the index as the basis for a proposed validation method is provided in the form of mathematical proofs to several relevant lemmas and a theorem. The feasibility of the proposed method was examined using simulated data generated under various conditions. The proposed method is illustrated using fraction subtraction data.
authors.affiliation: Department of Educational Psychology, Rutgers, The State University of New Jersey, 10 Seminary Place, New Brunswick, NJ, 08901, USA. j.delatorre@rutgers.edu.
authors.forename: Jimmy
authors.initials: J
authors.lastname: de la Torre
authors.affiliation: Department of Educational Psychology, Rutgers, The State University of New Jersey, 10 Seminary Place, New Brunswick, NJ, 08901, USA.
authors.forename: Chia-Yi
authors.initials: CY
authors.lastname: Chiu
mesh_terms: D000293:Adolescent; D000465:Algorithms; D002648:Child; D003071:Cognition; D004521:Educational Measurement; D005240:Feasibility Studies; D006801:Humans; D008960:Models, Psychological; D015233:Models, Statistical; D011594:Psychometrics; D015203:Reproducibility of Results; D013223:Statistics as Topic
keywords: G-DINA; MMLE; Q-matrix; cognitive diagnosis; validation