Evaluating cognitive theory: a joint modeling approach using responses and response times.
In current psychological research, the analysis of data from computer-based assessments or experiments is often confined to accuracy scores. Response times, although being an important source of additional information, are either neglected or analyzed separately. In this article, a new model is developed that allows the simultaneous analysis of accuracy scores and response times of cognitive tests with a rule-based design. The model is capable of simultaneously estimating ability and speed on the person side as well as difficulty and time intensity on the task side, thus dissociating information that is often confounded in current analysis procedures. Further, by integrating design matrices on the task side, it becomes possible to assess the effects of design parameters (e.g., cognitive processes) on both task difficulty and time intensity, offering deeper insights into the task structure. A Bayesian approach, using Markov Chain Monte Carlo methods, has been developed to estimate the model. An application of the model in the context of educational assessment is illustrated using a large-scale investigation of figural reasoning ability.
Klein Entink
Rinke H
RH
Department of Research Methodology, Measurement, and Data Analysis, University of Twente, Enschede, The Netherlands. r.h.kleinentink@gw.utwente.nl
Kuhn
Jörg-Tobias
JT
Hornke
Lutz F
LF
Fox
Jean-Paul
JP
D001077:Aptitude Tests / Q000706:statistics & numerical data; D001288:Attention; D001499:Bayes Theorem; D003071:Cognition; D003210:Concept Formation; D004521:Educational Measurement / Q000706:statistics & numerical data; D006801:Humans; D015233:Models, Statistical; D010364:Pattern Recognition, Visual; D011340:Problem Solving; D011581:Psychological Tests / Q000706:statistics & numerical data*; D011594:Psychometrics / Q000706:statistics & numerical data*; D011930:Reaction Time; D015203:Reproducibility of Results