title: Reliability concerns in the repeated computerized assessment of attention in children.
abstract: Assessment of attentional processes via computerized assessment is frequently used to quantify intra-individual cognitive improvement or decline in response to treatment. However, assessment of intra-individual change is highly dependent on sufficient test reliability. We examined the test-retest reliability of selected variables from one popular computerized continuous performance test (CPT)-i.e., the Conners' CPT - Second Edition (CPT-II). Participants were 39 healthy children (20 girls) ages 6-18 without intellectual impairment (mean PPVT-III SS = 102.6), LD, or psychiatric disorders (DICA-IV). Test-retest reliability over the 3-8 month interval (mean = 6 months) was acceptable (Intraclass Correlations [ICC] =.82 to.92) on comparison measures (Beery Test of Visual Perception, WISC-IV Block Design, PPVT-III). In contrast, test-retest reliability was only modest for CPT-II raw scores (ICCs ranging from.62 to.82) and T-scores (ICCs ranging from.33 to.65) for variables of interest (Omissions, Commissions, Variability, Hit Reaction Time, and Attentiveness). Using test-retest reliability information published in the CPT-II manual, 90% confidence intervals based on reliable change index (RCI) methodology were constructed to examine the significance of test-retest difference/change scores. Of the participants in this sample of typically developing youth, 30% generated intra-individual changes in T-scores on the Omissions and Attentiveness variables that exceeded the 90% confidence intervals and qualified as "statistically rare" changes in score. These results suggest a considerable degree of normal variability in CPT-II test scores over extended test-retest intervals, and suggest a need for caution when interpreting test score changes in neurologically unstable clinical populations.
authors.affiliation: Kennedy Krieger Institute, Baltimore, MD 21231, USA. zabela@kennedykrieger.org
authors.forename: T Andrew
authors.initials: TA
authors.lastname: Zabel
authors.forename: Christian
authors.initials: C
authors.lastname: von Thomsen
authors.forename: Carolyn
authors.initials: C
authors.lastname: Cole
authors.forename: Rebecca
authors.initials: R
authors.lastname: Martin
authors.forename: E Mark
authors.initials: EM
authors.lastname: Mahone
mesh_terms: D000293:Adolescent; D041923:Adolescent Development / Q000502:physiology; D001288:Attention; D002648:Child; D002657:Child Development / Q000502:physiology; D039721:Diagnostic and Statistical Manual of Mental Disorders; D005260:Female; D006801:Humans; D008297:Male; D009483:Neuropsychological Tests / Q000706:statistics & numerical data*; D018579:Patient Selection; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires