title: Challenges to estimating vaccine impact using hospitalization data.
abstract: Because the real-world impact of new vaccines cannot be known before they are implemented in national programs, post-implementation studies at the population level are critical. Studies based on analysis of hospitalization rates of vaccine-preventable outcomes are typically used for this purpose. However, estimates of vaccine impact based on hospitalization data are particularly prone to confounding, as hospitalization rates are tightly linked to changes in the quality, access and use of the healthcare system, which often occur simultaneously with introduction of new vaccines. Here we illustrate how changes in healthcare delivery coincident with vaccine introduction can influence estimates of vaccine impact, using as an example reductions in infant pneumonia hospitalizations after introduction of the 10-valent pneumococcal conjugate vaccine (PCV10) in Brazil. To this end, we explore the effect of changes in several metrics of quality and access to public healthcare on trends in hospitalization rates before (2008-09) and after (2011-12) PCV10 introduction in 2010. Changes in infant pneumonia hospitalization rates following vaccine introduction were significantly associated with concomitant changes in hospital capacity and the fraction of the population using public hospitals. Importantly, reduction of pneumonia hospitalization rates after PCV10 were also associated with the expansion of outpatient services in several Brazilian states, falling more sharply where primary care coverage and the number of health units offering basic and emergency care increased more. We show that adjustments for unrelated (non-vaccine) trends commonly employed by impact studies, such as use of single control outcomes, are not always sufficient for accurate impact assessment. We discuss several ways to identify and overcome such biases, including sensitivity analyses using different denominators to calculate hospitalizations rates and methods that track changes in the outpatient setting. Employing these practices can improve the accuracy of vaccine impact estimates, particularly in evolving healthcare settings typical of low- and middle-income countries.
authors.affiliation: Sage Analytica, Portland, ME, United States. Electronic address: cschuck@sageanalytica.com.
authors.forename: Cynthia
authors.initials: C
authors.lastname: Schuck-Paim
authors.affiliation: Sage Analytica, Portland, ME, United States.
authors.forename: Robert J
authors.initials: RJ
authors.lastname: Taylor
authors.affiliation: Department of Global Health, Milken Institute School of Public Health, George Washington University, Washington, DC, United States.
authors.forename: Lone
authors.initials: L
authors.lastname: Simonsen
authors.affiliation: Sage Analytica, Portland, ME, United States.
authors.forename: Roger
authors.initials: R
authors.lastname: Lustig
authors.affiliation: Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, CT, United States; Department of Statistics, University of California, Riverside, CA, United States.
authors.forename: Esra
authors.initials: E
authors.lastname: Kürüm
authors.affiliation: Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, CT, United States.
authors.forename: Christian A W
authors.initials: CA
authors.lastname: Bruhn
authors.affiliation: Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, CT, United States.
authors.forename: Daniel M
authors.initials: DM
authors.lastname: Weinberger
mesh_terms: D056808:Biostatistics; D001938:Brazil / Q000453:epidemiology; D005260:Female; D006760:Hospitalization; D006801:Humans; D007223:Infant; D008297:Male; D022242:Pneumococcal Vaccines / Q000008:administration & dosage* / Q000276:immunology*; D011018:Pneumonia, Pneumococcal / Q000453:epidemiology* / Q000517:prevention & control*; D016896:Treatment Outcome
chemical_list: C586648:10-valent pneumococcal conjugate vaccine; D022242:Pneumococcal Vaccines
keywords: Bias; Brazil; Confounding factors; Delivery of health care; Health impact assessment; Hospitalization; Latin America; Observational studies; Pneumococcal conjugate vaccines; Pneumococcal vaccines; Pneumococcus; Pneumonia; Public health; Vaccines