title: Combining machine learning and matching techniques to improve causal inference in program evaluation.
abstract: OBJECTIVE
  Program evaluations often utilize various matching approaches to emulate the randomization process for group assignment in experimental studies. Typically, the matching strategy is implemented, and then covariate balance is assessed before estimating treatment effects. This paper introduces a novel analytic framework utilizing a machine learning algorithm called optimal discriminant analysis (ODA) for assessing covariate balance and estimating treatment effects, once the matching strategy has been implemented. This framework holds several key advantages over the conventional approach: application to any variable metric and number of groups; insensitivity to skewed data or outliers; and use of accuracy measures applicable to all prognostic analyses. Moreover, ODA accepts analytic weights, thereby extending the methodology to any study design where weights are used for covariate adjustment or more precise (differential) outcome measurement.
  
  
  METHODS
  One-to-one matching on the propensity score was used as the matching strategy. Covariate balance was assessed using standardized difference in means (conventional approach) and measures of classification accuracy (ODA). Treatment effects were estimated using ordinary least squares regression and ODA.
  
  
  RESULTS
  Using empirical data, ODA produced results highly consistent with those obtained via the conventional methodology for assessing covariate balance and estimating treatment effects.
  
  
  CONCLUSIONS
  When ODA is combined with matching techniques within a treatment effects framework, the results are consistent with conventional approaches. However, given that it provides additional dimensions and robustness to the analysis versus what can currently be achieved using conventional approaches, ODA offers an appealing alternative.
authors.affiliation: Linden Consulting Group, LLC, Ann Arbor, MI, USA.|Division of General Medicine, Medical School, University of Michigan, Ann Arbor, MI, USA.
authors.forename: Ariel
authors.initials: A
authors.lastname: Linden
authors.affiliation: Optimal Data Analysis, LLC, Chicago, IL, USA.|Southern Network on Adverse Reactions (SONAR), College of Pharmacy, University of South Carolina, Columbia, SC, USA.
authors.forename: Paul R
authors.initials: PR
authors.lastname: Yarnold
mesh_terms: D000328:Adult; D000465:Algorithms; D015984:Causality; D002908:Chronic Disease / Q000191:economics; D016002:Discriminant Analysis; D005260:Female; D006801:Humans; D000069550:Machine Learning; D008297:Male; D008875:Middle Aged; D015397:Program Evaluation / Q000379:methods*; D057216:Propensity Score; D011897:Random Allocation
keywords: balance; causal inference; machine learning; matching; propensity score