title: An Evaluation of Weighting Methods Based on Propensity Scores to Reduce Selection Bias in Multilevel Observational Studies.
abstract: Observational studies of multilevel data to estimate treatment effects must consider both the nonrandom treatment assignment mechanism and the clustered structure of the data. We present an approach for implementation of four propensity score (PS) methods with multilevel data involving creation of weights and three types of weight scaling (normalized, cluster-normalized and effective), followed by estimation of multilevel models with the multilevel pseudo-maximum likelihood estimation method. Using a Monte Carlo simulation study, we found that the multilevel model provided unbiased estimates of the Average Treatment Effect on the Treated (ATT) and its standard error across manipulated conditions and combinations of PS model, PS method, and type of weight scaling. Estimates of between-cluster variances of the ATT were biased, but improved as cluster sizes increased. We provide a step-by-step demonstration of how to combine PS methods and multilevel modeling to estimate treatment effects using multilevel data from the Early Childhood Longitudinal Study-Kindergarten Cohort (ECLS-K).
authors.affiliation: a College of Education , University of Florida.
authors.forename: Walter L
authors.initials: WL
authors.lastname: Leite
authors.affiliation: a College of Education , University of Florida.
authors.forename: Francisco
authors.initials: F
authors.lastname: Jimenez
authors.affiliation: a College of Education , University of Florida.
authors.forename: Yasemin
authors.initials: Y
authors.lastname: Kaya
authors.affiliation: b College of Education , University of Maryland.
authors.forename: Laura M
authors.initials: LM
authors.lastname: Stapleton
authors.affiliation: a College of Education , University of Florida.
authors.forename: Jann W
authors.initials: JW
authors.lastname: MacInnes
authors.affiliation: c Santa Fe College.
authors.forename: Robert
authors.initials: R
authors.lastname: Sandbach
mesh_terms: D003198:Computer Simulation; D006801:Humans; D016013:Likelihood Functions; D015233:Models, Statistical; D009010:Monte Carlo Method; D055361:Multilevel Analysis / Q000379:methods*; D064887:Observational Studies as Topic; D057216:Propensity Score; D015983:Selection Bias