title: Recommendations for choosing an analysis method that controls Type I error for unbalanced cluster sample designs with Gaussian outcomes.
abstract: We used theoretical and simulation-based approaches to study Type I error rates for one-stage and two-stage analytic methods for cluster-randomized designs. The one-stage approach uses the observed data as outcomes and accounts for within-cluster correlation using a general linear mixed model. The two-stage model uses the cluster specific means as the outcomes in a general linear univariate model. We demonstrate analytically that both one-stage and two-stage models achieve exact Type I error rates when cluster sizes are equal. With unbalanced data, an exact size α test does not exist, and Type I error inflation may occur. Via simulation, we compare the Type I error rates for four one-stage and six two-stage hypothesis testing approaches for unbalanced data. With unbalanced data, the two-stage model, weighted by the inverse of the estimated theoretical variance of the cluster means, and with variance constrained to be positive, provided the best Type I error control for studies having at least six clusters per arm. The one-stage model with Kenward-Roger degrees of freedom and unconstrained variance performed well for studies having at least 14 clusters per arm. The popular analytic method of using a one-stage model with denominator degrees of freedom appropriate for balanced data performed poorly for small sample sizes and low intracluster correlation. Because small sample sizes and low intracluster correlation are common features of cluster-randomized trials, the Kenward-Roger method is the preferred one-stage approach.
authors.affiliation: Department of Psychiatry, University of North Carolina, Chapel Hill, NC, U.S.A.
authors.forename: Jacqueline L
authors.initials: JL
authors.lastname: Johnson
authors.affiliation: Neptune and Company, Lakewood, CO, U.S.A.
authors.forename: Sarah M
authors.identifier: http://orcid.org/0000-0002-2104-5166
authors.initials: SM
authors.lastname: Kreidler
authors.affiliation: RTI International, Research Triangle Park, NC, U.S.A.
authors.forename: Diane J
authors.initials: DJ
authors.lastname: Catellier
authors.affiliation: Biostatistics and Bioinformatics BranchDivision of Epidemiology Statistics, and Prevention Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Rockville, MD, U.S.A.
authors.forename: David M
authors.identifier: http://orcid.org/0000-0003-0797-9269
authors.initials: DM
authors.lastname: Murray
authors.affiliation: Department of Health Outcomes and Policy, University of Florida, Gainesville, FL, U.S.A.
authors.forename: Keith E
authors.initials: KE
authors.lastname: Muller
authors.affiliation: Department of Biostatistics and Informatics, University of Colorado Denver, Aurora, CO, U.S.A.
authors.forename: Deborah H
authors.initials: DH
authors.lastname: Glueck
mesh_terms: D015982:Bias; D016000:Cluster Analysis; D016011:Normal Distribution; D058996:Quality Improvement / Q000706:statistics & numerical data; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data; D012107:Research Design / Q000592:standards / Q000706:statistics & numerical data
keywords: Gaussian; Type I error; cluster randomized; group randomized; unbalanced