title: Developing new online calibration methods for multidimensional computerized adaptive testing.
abstract: Multidimensional computerized adaptive testing (MCAT) has received increasing attention over the past few years in educational measurement. Like all other formats of CAT, item replenishment is an essential part of MCAT for its item bank maintenance and management, which governs retiring overexposed or obsolete items over time and replacing them with new ones. Moreover, calibration precision of the new items will directly affect the estimation accuracy of examinees' ability vectors. In unidimensional CAT (UCAT) and cognitive diagnostic CAT, online calibration techniques have been developed to effectively calibrate new items. However, there has been very little discussion of online calibration in MCAT in the literature. Thus, this paper proposes new online calibration methods for MCAT based upon some popular methods used in UCAT. Three representative methods, Method A, the 'one EM cycle' method and the 'multiple EM cycles' method, are generalized to MCAT. Three simulation studies were conducted to compare the three new methods by manipulating three factors (test length, item bank design, and level of correlation between coordinate dimensions). The results showed that all the new methods were able to recover the item parameters accurately, and the adaptive online calibration designs showed some improvements compared to the random design under most conditions.
authors.affiliation: Beijing Normal University, Beijing, China.
authors.forename: Ping
authors.identifier: http://orcid.org/0000-0002-2920-4205
authors.initials: P
authors.lastname: Chen
authors.affiliation: University of Minnesota, Minneapolis, Minnesota, USA.
authors.forename: Chun
authors.initials: C
authors.lastname: Wang
authors.affiliation: Beijing Normal University, Beijing, China.
authors.forename: Tao
authors.initials: T
authors.lastname: Xin
authors.affiliation: University of Illinois at Urbana-Champaign, Illinois, USA.
authors.forename: Hua-Hua
authors.initials: HH
authors.lastname: Chang
mesh_terms: D000465:Algorithms; D002138:Calibration / Q000592:standards*; D003198:Computer Simulation; D004521:Educational Measurement / Q000379:methods* / Q000592:standards*; D015233:Models, Statistical; D009862:Online Systems; D011594:Psychometrics / Q000379:methods / Q000592:standards*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity
keywords: item bank construction; item replenishment; multidimensional computerized adaptive testing; multidimensional two-parameter logistic model; online calibration