title: Generalizability and decision studies to inform observational and experimental research in classroom settings.
abstract: Attaining reliable estimates of observational measures can be challenging in school and classroom settings, as behavior can be influenced by multiple contextual factors. Generalizability (G) studies can enable researchers to estimate the reliability of observational data, and decision (D) studies can inform how many observation sessions are necessary to achieve a criterion level of reliability. We conducted G and D studies using observational data from a randomized control trial focusing on social and academic participation of students with severe disabilities in inclusive secondary classrooms. Results highlight the importance of anchoring observational decisions to reliability estimates from existing or pilot data sets. We outline steps for conducting G and D studies and address options when reliability estimates are lower than desired.
authors.forename: Kristen
authors.initials: K
authors.lastname: Bottema-Beutel
authors.forename: Blair
authors.initials: B
authors.lastname: Lloyd
authors.forename: Erik W
authors.initials: EW
authors.lastname: Carter
authors.forename: Jennifer M
authors.initials: JM
authors.lastname: Asmus
mesh_terms: D000293:Adolescent; D000328:Adult; D002659:Child Development Disorders, Pervasive / Q000523:psychology; D003627:Data Interpretation, Statistical; D004519:Education, Special / Q000592:standards*; D036262:Empirical Research; D005260:Female; D006801:Humans; D008607:Intellectual Disability / Q000523:psychology; D007398:Interpersonal Relations; D008297:Male; D064887:Observational Studies as Topic / Q000592:standards*; D010379:Peer Group; D016032:Randomized Controlled Trials as Topic; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards; D012574:Schools / Q000592:standards; D055815:Young Adult
keywords: inclusion; observational measurement; secondary classrooms; severe disabilities; social interaction