title: Frequency and impact of confounding by indication and healthy vaccinee bias in observational studies assessing influenza vaccine effectiveness: a systematic review.
abstract: BACKGROUND
  Evidence on influenza vaccine effectiveness (VE) is commonly derived from observational studies. However, these studies are prone to confounding by indication and healthy vaccinee bias. We aimed to systematically investigate these two forms of confounding/bias.
  
  
  METHODS
  Systematic review of observational studies reporting influenza VE and indicators for bias and confounding. We assessed risk of confounding by indication and healthy vaccinee bias for each study and calculated ratios of odds ratios (crude/adjusted) to quantify the effect of confounder adjustment. VE-estimates during and outside influenza seasons were compared to assess residual confounding by healthy vaccinee effects.
  
  
  RESULTS
  We identified 23 studies reporting on 11 outcomes. Of these, 19 (83 %) showed high risk of bias: Fourteen due to confounding by indication, two for healthy vaccinee bias, and three studies showed both forms of confounding/bias. Adjustment for confounders increased VE on average by 12 % (95 % CI: 7-17 %; all-cause mortality), 9 % (95 % CI: 4-14 %; all-cause hospitalization) and 7 % (95 % CI: 4-10 %; influenza-like illness). Despite adjustment, nine studies showed residual confounding as indicated by significant off-season VE-estimates. These were observed for five outcomes, but more frequently for all-cause mortality as compared to other outcomes (p = 0.03) and in studies which indicated healthy vaccinee bias at baseline (p = 0.01).
  
  
  CONCLUSIONS
  Both confounding by indication and healthy vaccinee bias are likely to operate simultaneously in observational studies on influenza VE. Although adjustment can correct for confounding by indication to some extent, the resulting estimates are still prone to healthy vaccinee bias, at least as long as unspecific outcomes like all-cause mortality are used. Therefore, cohort studies using administrative data bases with unspecific outcomes should no longer be used to measure the effects of influenza vaccination.
authors.affiliation: Immunization Unit, Robert Koch Institute, Seestrasse 10, 13353, Berlin, Germany. remschmidtc@rki.de.
authors.forename: Cornelius
authors.initials: C
authors.lastname: Remschmidt
authors.affiliation: Immunization Unit, Robert Koch Institute, Seestrasse 10, 13353, Berlin, Germany. wichmanno@rki.de.
authors.forename: Ole
authors.initials: O
authors.lastname: Wichmann
authors.affiliation: Immunization Unit, Robert Koch Institute, Seestrasse 10, 13353, Berlin, Germany. hardert@rki.de.
authors.forename: Thomas
authors.initials: T
authors.lastname: Harder
mesh_terms: D002423:Cause of Death; D016208:Databases, Factual; D006760:Hospitalization; D006801:Humans; D007252:Influenza Vaccines / Q000009:adverse effects / Q000627:therapeutic use*; D007251:Influenza, Human / Q000401:mortality / Q000473:pathology / Q000517:prevention & control*; D016017:Odds Ratio; D012306:Risk; D014611:Vaccination
chemical_list: D007252:Influenza Vaccines