Comparison of data analysis strategies for intent-to-treat analysis in pre-test-post-test designs with substantial dropout rates.
The pre-test-post-test design (PPD) is predominant in trials of psychotherapeutic treatments. Missing data due to withdrawals present an even bigger challenge in assessing treatment effectiveness under the PPD than under designs with more observations since dropout implies an absence of information about response to treatment. When confronted with missing data, often it is reasonable to assume that the mechanism underlying missingness is related to observed but not to unobserved outcomes (missing at random, MAR). Previous simulation and theoretical studies have shown that, under MAR, modern techniques such as maximum-likelihood (ML) based methods and multiple imputation (MI) can be used to produce unbiased estimates of treatment effects. In practice, however, ad hoc methods such as last observation carried forward (LOCF) imputation and complete-case (CC) analysis continue to be used. In order to better understand the behaviour of these methods in the PPD, we compare the performance of traditional approaches (LOCF, CC) and theoretically sound techniques (MI, ML), under various MAR mechanisms. We show that the LOCF method is seriously biased and conclude that its use should be abandoned. Complete-case analysis produces unbiased estimates only when the dropout mechanism does not depend on pre-test values even when dropout is related to fixed covariates including treatment group (covariate-dependent: CD). However, CC analysis is generally biased under MAR. The magnitude of the bias is largest when the correlation of post- and pre-test is relatively low.
Salim
Agus
A
Centre for Mental Health Research, Australian National University, Acton, Australia. agus.salim@anu.edu.au
Mackinnon
Andrew
A
Christensen
Helen
H
Griffiths
Kathleen
K
D015982:Bias; D002986:Clinical Trials as Topic / Q000379:methods / Q000706:statistics & numerical data*; D005069:Evaluation Studies as Topic; D006801:Humans; D016013:Likelihood Functions; D016015:Logistic Models; D008137:Longitudinal Studies; D015233:Models, Statistical; D010352:Patient Dropouts / Q000706:statistics & numerical data*; D011336:Probability; D011613:Psychotherapy / Q000379:methods / Q000706:statistics & numerical data*; D011634:Public Health / Q000379:methods / Q000706:statistics & numerical data; D016032:Randomized Controlled Trials as Topic / Q000379:methods / Q000706:statistics & numerical data; D012107:Research Design / Q000706:statistics & numerical data; D013223:Statistics as Topic / Q000379:methods*; D016896:Treatment Outcome