title: Testlet-Based Multidimensional Adaptive Testing.
abstract: Multidimensional adaptive testing (MAT) is a highly efficient method for the simultaneous measurement of several latent traits. Currently, no psychometrically sound approach is available for the use of MAT in testlet-based tests. Testlets are sets of items sharing a common stimulus such as a graph or a text. They are frequently used in large operational testing programs like TOEFL, PISA, PIRLS, or NAEP. To make MAT accessible for such testing programs, we present a novel combination of MAT with a multidimensional generalization of the random effects testlet model (MAT-MTIRT). MAT-MTIRT compared to non-adaptive testing is examined for several combinations of testlet effect variances (0.0, 0.5, 1.0, and 1.5) and testlet sizes (3, 6, and 9 items) with a simulation study considering three ability dimensions with simple loading structure. MAT-MTIRT outperformed non-adaptive testing regarding the measurement precision of the ability estimates. Further, the measurement precision decreased when testlet effect variances and testlet sizes increased. The suggested combination of the MTIRT model therefore provides a solution to the substantial problems of testlet-based tests while keeping the length of the test within an acceptable range.
authors.affiliation: Department of Research Methods in Education, Institute of Educational Science, Friedrich Schiller University JenaJena, Germany; Faculty of Education, Centre for Educational Measurement, University of OsloOslo, Norway.
authors.forename: Andreas
authors.initials: A
authors.lastname: Frey
authors.affiliation: Department of Research Methods in Education, Institute of Educational Science, Friedrich Schiller University Jena Jena, Germany.
authors.forename: Nicki-Nils
authors.initials: NN
authors.lastname: Seitz
authors.affiliation: Art of Reduction Altenholz, Germany.
authors.forename: Steffen
authors.initials: S
authors.lastname: Brandt
keywords: computerized adaptive testing; item response theory; large-scale assessment; multidimensional IRT models; testlets