title: Use of internal consistency coefficients for estimating reliability of experimental task scores.
abstract: Reliabilities of scores for experimental tasks are likely to differ from one study to another to the extent that the task stimuli change, the number of trials varies, the type of individuals taking the task changes, the administration conditions are altered, or the focal task variable differs. Given that reliabilities vary as a function of the design of these tasks and the characteristics of the individuals taking them, making inferences about the reliability of scores in an ongoing study based on reliability estimates from prior studies is precarious. Thus, it would be advantageous to estimate reliability based on data from the ongoing study. We argue that internal consistency estimates of reliability are underutilized for experimental task data and in many applications could provide this information using a single administration of a task. We discuss different methods for computing internal consistency estimates with a generalized coefficient alpha and the conditions under which these estimates are accurate. We illustrate use of these coefficients using data for three different tasks.
authors.affiliation: Arizona State University, Social Sciences Building, 951 S Cady Mall, P.O. Box 873701, Tempe, AZ, 85287-3701, USA. samgreen@asu.edu.
authors.forename: Samuel B
authors.initials: SB
authors.lastname: Green
authors.affiliation: Florida State University, Tallahassee, FL, USA.
authors.forename: Yanyun
authors.initials: Y
authors.lastname: Yang
authors.affiliation: University of Arizona, Tucson, AZ, USA.
authors.forename: Mary
authors.initials: M
authors.lastname: Alt
authors.affiliation: Arizona State University, Social Sciences Building, 951 S Cady Mall, P.O. Box 873701, Tempe, AZ, 85287-3701, USA.
authors.forename: Shara
authors.initials: S
authors.lastname: Brinkley
authors.affiliation: Arizona State University, Social Sciences Building, 951 S Cady Mall, P.O. Box 873701, Tempe, AZ, 85287-3701, USA.
authors.forename: Shelley
authors.initials: S
authors.lastname: Gray
authors.affiliation: MGH Institute of Health Professions, Charlestown, MA, USA.
authors.forename: Tiffany
authors.initials: T
authors.lastname: Hogan
authors.affiliation: University of Missouri, Columbia, MO, USA.
authors.forename: Nelson
authors.initials: N
authors.lastname: Cowan
mesh_terms: D006801:Humans; D015203:Reproducibility of Results; D013223:Statistics as Topic; D013647:Task Performance and Analysis
keywords: Coefficient alpha; Generalized coefficient alpha; Reliability; Split-half reliability