Using machine learning to identify structural breaks in single-group interrupted time series designs.
OBJECTIVE
Single-group interrupted time series analysis (ITSA) is a popular evaluation methodology in which a single unit of observation is being studied, the outcome variable is serially ordered as a time series and the intervention is expected to 'interrupt' the level and/or trend of the time series, subsequent to its introduction. Given that the internal validity of the design rests on the premise that the interruption in the time series is associated with the introduction of the treatment, treatment effects may seem less plausible if a parallel trend already exists in the time series prior to the actual intervention. Thus, sensitivity analyses should focus on detecting structural breaks in the time series before the intervention.


METHODS
In this paper, we introduce a machine-learning algorithm called optimal discriminant analysis (ODA) as an approach to determine if structural breaks can be identified in years prior to the initiation of the intervention, using data from California's 1988 voter-initiated Proposition 99 to reduce smoking rates.


RESULTS
The ODA analysis indicates that numerous structural breaks occurred prior to the actual initiation of Proposition 99 in 1989, including perfect structural breaks in 1983 and 1985, thereby casting doubt on the validity of treatment effects estimated for the actual intervention when using a single-group ITSA design.


CONCLUSIONS
Given the widespread use of ITSA for evaluating observational data and the increasing use of machine-learning techniques in traditional research, we recommend that structural break sensitivity analysis is routinely incorporated in all research using the single-group ITSA design.
Linden Consulting Group, LLC, Ann Arbor, MI, USA.|Division of General Medicine, Medical School, University of Michigan, Ann Arbor, MI, USA.
Ariel
A
Linden
Optimal Data Analysis, LLC, Chicago, IL, USA.
Paul R
PR
Yarnold
D000328:Adult; D016002:Discriminant Analysis; D005260:Female; D006801:Humans; D065186:Interrupted Time Series Analysis; D000069550:Machine Learning; D008297:Male; D008875:Middle Aged; D012107:Research Design; D015983:Selection Bias
causal inference; data mining; interrupted time series analysis; machine learning; maximum-accuracy model; optimal discriminant analysis; quasi-experimental; structural breaks