Type I error rates and power analyses for single-point sensitivity measures.
Experiments often produce a hit rate and a false alarm rate in each of two conditions. These response rates are summarized into a single-point sensitivity measure such as d', and t tests are conducted to test for experimental effects. Using large-scale Monte Carlo simulations, we evaluate the Type I error rates and power that result from four commonly used single-point measures: d', A', percent correct, and gamma. We also test a newly proposed measure called gammaC. For all measures, we consider several ways of handling cases in which false alarm rate = 0 or hit rate = 1. The results of our simulations indicate that power is similar for these measures but that the Type I error rates are often unacceptably high. Type I errors are minimized when the selected sensitivity measure is theoretically appropriate for the data.
Rotello
Caren M
CM
Department of Psychology, University of Massachusetts, Amherst, Massachusetts 01003-7710, USA. caren@psych.umass.edu
Masson
Michael E J
ME
Verde
Michael F
MF
D001288:Attention; D015982:Bias; D003463:Cues; D003627:Data Interpretation, Statistical; D003657:Decision Making; D004193:Discrimination Learning; D006801:Humans; D015233:Models, Statistical; D009010:Monte Carlo Method; D016011:Normal Distribution; D009949:Orientation; D010364:Pattern Recognition, Visual; D011589:Psychology, Experimental / Q000706:statistics & numerical data*; D011601:Psychophysics; D012372:ROC Curve; D012107:Research Design / Q000706:statistics & numerical data*; D012680:Sensitivity and Specificity; D017603:Signal Detection, Psychological