title: Sensitivity analyses for clustered data: an illustration from a large-scale clustered randomized controlled trial in education.
abstract: In this paper, we demonstrate the importance of conducting well-thought-out sensitivity analyses for handling clustered data (data in which individuals are grouped into higher order units, such as students in schools) that arise from cluster randomized controlled trials (RCTs). This is particularly relevant given the rise in rigorous impact evaluations that use cluster randomized designs across various fields including education, public health and social welfare. Using data from a recently completed cluster RCT of a school-based teacher professional development program, we demonstrate our use of four commonly applied methods for analyzing clustered data. These methods include: (1) hierarchical linear modeling (HLM); (2) feasible generalized least squares (FGLS); (3) generalized estimating equations (GEE); and (4) ordinary least squares (OLS) regression with cluster-robust (Huber-White) standard errors. We compare our findings across each method, showing how inconsistent results - in terms of both effect sizes and statistical significance - emerged across each method and our analytic approach to resolving such inconsistencies.
authors.affiliation: IMPAQ International, 1333 Broadway, Suite 300 , Oakland, CA 94612, United States. Electronic address: yabe@impaqint.com.
authors.forename: Yasuyo
authors.initials: Y
authors.lastname: Abe
authors.affiliation: University of California, Davis, One Shields Ave., Davis, CA 95616, United States. Electronic address: kagee@ucdavis.edu.
authors.forename: Kevin A
authors.initials: KA
authors.lastname: Gee
mesh_terms: D006801:Humans; D016018:Least-Squares Analysis; D016014:Linear Models; D015397:Program Evaluation / Q000379:methods*; D016032:Randomized Controlled Trials as Topic / Q000379:methods*; D012107:Research Design; D012574:Schools / Q000458:organization & administration*; D013192:Staff Development / Q000458:organization & administration*
keywords: Data analysis for clustered randomized trials; Generalized estimating equations; Hierarchical linear modeling; Impact evaluation; Sensitivity analyses