Psychometric considerations in the measurement of event-related brain potentials: Guidelines for measurement and reporting.
Failing to consider psychometric issues related to reliability and validity, differential deficits, and statistical power potentially undermines the conclusions of a study. In research using event-related brain potentials (ERPs), numerous contextual factors (population sampled, task, data recording, analysis pipeline, etc.) can impact the reliability of ERP scores. The present review considers the contextual factors that influence ERP score reliability and the downstream effects that reliability has on statistical analyses. Given the context-dependent nature of ERPs, it is recommended that ERP score reliability be formally assessed on a study-by-study basis. Recommended guidelines for ERP studies include 1) reporting the threshold of acceptable reliability and reliability estimates for observed scores, 2) specifying the approach used to estimate reliability, and 3) justifying how trial-count minima were chosen. A reliability threshold for internal consistency of at least 0.70 is recommended, and a threshold of 0.80 is preferred. The review also advocates the use of generalizability theory for estimating score dependability (the generalizability theory analog to reliability) as an improvement on classical test theory reliability estimates, suggesting that the latter is less well suited to ERP research. To facilitate the calculation and reporting of dependability estimates, an open-source Matlab program, the ERP Reliability Analysis Toolbox, is presented.
Department of Psychology, University of California, Los Angeles, Los Angeles, CA, United States. Electronic address: peter.clayson@gmail.com.
Peter E
PE
Clayson
Department of Psychology, University of California, Los Angeles, Los Angeles, CA, United States; Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, United States.
Gregory A
GA
Miller
D004569:Electroencephalography / Q000592:standards*; D005071:Evoked Potentials / Q000502:physiology*; D017408:Guidelines as Topic / Q000592:standards*; D006801:Humans; D011594:Psychometrics / Q000592:standards*; D011603:Psychophysiology / Q000592:standards*
Dependability; ERP reliability analysis toolbox; Event-related potentials; Guidelines; Psychometrics