title: Rasch modeling of accuracy and confidence measures from cognitive tests.
abstract: The use of IRT models has not been rigorously applied in studies of the relationship between test-takers' confidence and accuracy. This study applied the Rasch measurement models to investigate the relationship between test-takers' confidence and accuracy on English proficiency tests, proposing potentially useful measures of under or overconfidence. The Rasch approach provided the scaffolding to formulate indices that can assess the discrepancy between confidence and accuracy at the item or total test level, as well as at particular ability levels locally. In addition, a "disattenuated" measure of association between accuracy and confidence, which takes measurement error into account, was obtained through a multidimensional Rasch modeling of the two constructs where the latent variance-covariance structure is directly estimated from the data. The results indicate that the participants tend to show overconfidence bias in their own cognitive abilities.
authors.affiliation: Educational Psychology and Learning Systems, Florida State University, Tallahassee, FL 32306-4453, USA. ipaek@fsu.edu
authors.forename: Insu
authors.initials: I
authors.lastname: Paek
authors.forename: Jihyun
authors.initials: J
authors.lastname: Lee
authors.forename: Lazar
authors.initials: L
authors.lastname: Stankov
authors.forename: Mark
authors.initials: M
authors.lastname: Wilson
mesh_terms: D003072:Cognition Disorders / Q000175:diagnosis*; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D004521:Educational Measurement / Q000379:methods* / Q000706:statistics & numerical data; D006801:Humans; D015233:Models, Statistical; D011594:Psychometrics / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D011795:Surveys and Questionnaires