title: Graphical displays for assessing covariate balance in matching studies.
abstract: OBJECTIVE
  An essential requirement for ensuring the validity of outcomes in matching studies is that study groups are comparable on observed pre-intervention characteristics. Investigators typically use numerical diagnostics, such as t-tests, to assess comparability (referred to as 'balance'). However, such diagnostics only test equality along one dimension (e.g. means in the case of t-tests), and therefore do not adequately capture imbalances that may exist elsewhere in the distribution. Furthermore, these tests are generally sensitive to sample size, raising the concern that a reduction in power may be mistaken for an improvement in covariate balance. In this paper, we demonstrate the shortcomings of numerical diagnostics and demonstrate how visual displays provide a complete representation of the data to more robustly assess balance.
  
  
  METHODS
  We generate artificial datasets specifically designed to demonstrate how widely used equality tests capture only a single-dimension of the data and are sensitive to sample size. We then plot the covariate distributions using several graphical displays.
  
  
  RESULTS
  As expected, tests showing perfect covariate balance in means failed to reflect imbalances at higher moments (variances). However, these discrepancies were easily detected upon inspection of the graphic displays. Additionally, smaller sample sizes led to the appearance of covariate balance, when in fact it was a result of lower statistical power.
  
  
  CONCLUSIONS
  Given the limitations of numerical diagnostics, we advocate using graphical displays for assessing covariate balance and encourage investigators to provide such graphs when reporting balance statistics in their matching studies.
authors.affiliation: Linden Consulting Group, LLC, Ann Arbor, MI, USA; Department of Health Management and Policy, School of Public Health, University of Michigan, Ann Arbor, MI, USA.
authors.forename: Ariel
authors.initials: A
authors.lastname: Linden
mesh_terms: D001296:Audiovisual Aids; D006801:Humans; D015233:Models, Statistical; D017063:Outcome Assessment, Health Care / Q000379:methods*; D015203:Reproducibility of Results; D012107:Research Design
keywords: covariate balance; matching; propensity score; standardized differences; visual displays