title: The impact of probe variability on brief experimental analysis of reading skills.
abstract: The purpose of this study was to examine the impact of probe variability on the ability to replicate results in brief experimental analysis (BEA) of reading. In the first phase of the study, 41 first- and second- grade students completed 16 oral reading fluency probes. Calculations of probe difficulty were used to identify Low and High Variability probe sets. In the second phase of the study, the performance of 40 second- through fifth-grade students during two reading interventions was compared. The best-performing intervention for each student in the initial trial was replicated during a second trial for only 43% of students regardless of probe variability. The best-performing intervention was replicated for 60% of students when average performance across two trials was compared. Rules for determining the best-performing intervention in academic BEA should consider the standard error of measurement (SEM) for the probe set to be used, the reliability for absolute decisions using the probe set, and the number of replications relative to SEM needed to adequately demonstrate experimental control.
authors.affiliation: Department of Educational and Counselling Psychology and Special Education, University of British Columbia.
authors.forename: Sterett H
authors.identifier: 0000-0002-7940-4221
authors.initials: SH
authors.lastname: Mercer
authors.affiliation: Department of Psychology, The University of Southern Mississippi.
authors.forename: Lauren Lestremau
authors.initials: LL
authors.lastname: Harpole
authors.affiliation: Department of Psychology, The University of Southern Mississippi.
authors.forename: Rachel R
authors.initials: RR
authors.lastname: Mitchell
authors.affiliation: Department of Psychology, The University of Southern Mississippi.
authors.forename: Chandler
authors.initials: C
authors.lastname: McLemore
authors.affiliation: Department of Psychology, The University of Southern Mississippi.
authors.forename: Christina
authors.initials: C
authors.lastname: Hardy
mesh_terms: D002648:Child; D004521:Educational Measurement / Q000379:methods* / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D007858:Learning / Q000502:physiology*; D008297:Male; D011932:Reading; D015203:Reproducibility of Results