title: A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings.
abstract: How should we measure metacognitive ("type 2") sensitivity, i.e. the efficacy with which observers' confidence ratings discriminate between their own correct and incorrect stimulus classifications? We argue that currently available methods are inadequate because they are influenced by factors such as response bias and type 1 sensitivity (i.e. ability to distinguish stimuli). Extending the signal detection theory (SDT) approach of Galvin, Podd, Drga, and Whitmore (2003), we propose a method of measuring type 2 sensitivity that is free from these confounds. We call our measure meta-d', which reflects how much information, in signal-to-noise units, is available for metacognition. Applying this novel method in a 2-interval forced choice visual task, we found that subjects' metacognitive sensitivity was close to, but significantly below, optimality. We discuss the theoretical implications of these findings, as well as related computational issues of the method. We also provide free Matlab code for implementing the analysis.
authors.affiliation: Department of Psychology, Columbia University, New York, NY 10027, USA. brian@psych.columbia.edu
authors.forename: Brian
authors.initials: B
authors.lastname: Maniscalco
authors.forename: Hakwan
authors.initials: H
authors.lastname: Lau
mesh_terms: D003071:Cognition; D006801:Humans; D007600:Judgment; D011582:Psychological Theory; D012680:Sensitivity and Specificity; D017603:Signal Detection, Psychological; D059629:Signal-To-Noise Ratio; D013028:Space Perception