title: Model distinguishability and inference robustness in mechanisms of cholera transmission and loss of immunity.
abstract: Mathematical models of cholera and waterborne disease vary widely in their structures, in terms of transmission pathways, loss of immunity, and a range of other features. These differences can affect model dynamics, with different models potentially yielding different predictions and parameter estimates from the same data. Given the increasing use of mathematical models to inform public health decision-making, it is important to assess model distinguishability (whether models can be distinguished based on fit to data) and inference robustness (whether inferences from the model are robust to realistic variations in model structure). In this paper, we examined the effects of uncertainty in model structure in the context of epidemic cholera, testing a range of models with differences in transmission and loss of immunity structure, based on known features of cholera epidemiology. We fit these models to simulated epidemic and long-term data, as well as data from the 2006 Angola epidemic. We evaluated model distinguishability based on fit to data, and whether the parameter values, model behavior, and forecasting ability can accurately be inferred from incidence data. In general, all models were able to successfully fit to all data sets, both real and simulated, regardless of whether the model generating the simulated data matched the fitted model. However, in the long-term data, the best model fits were achieved when the loss of immunity structures matched those of the model that simulated the data. Two parameters, one representing person-to-person transmission and the other representing the reporting rate, were accurately estimated across all models, while the remaining parameters showed broad variation across the different models and data sets. The basic reproduction number (R0) was often poorly estimated even using the correct model, due to practical unidentifiability issues in the waterborne transmission pathway which were consistent across all models. Forecasting efforts using noisy data were not successful early in the outbreaks, but once the epidemic peak had been achieved, most models were able to capture the downward incidence trajectory with similar accuracy. Forecasting from noise-free data was generally successful for all outbreak stages using any model. Our results suggest that we are unlikely to be able to infer mechanistic details from epidemic case data alone, underscoring the need for broader data collection, such as immunity/serology status, pathogen dose response curves, and environmental pathogen data. Nonetheless, with sufficient data, conclusions from forecasting and some parameter estimates were robust to variations in the model structure, and comparative modeling can help to determine how realistic variations in model structure may affect the conclusions drawn from models and data.
authors.affiliation: Department of Biology, Georgetown University, 37th and O Streets, NW, Washington, DC 20057, United States. Electronic address: ecl48@georgetown.edu.
authors.forename: Elizabeth C
authors.initials: EC
authors.lastname: Lee
authors.affiliation: Division of Natural Sciences and Mathematics, Transylvania University, 300 North Broadway, Lexington, KY 40508, United States; Department of Mathematics, The Ohio State University, 231 West 18th Ave, Columbus, OH 43210, United States. Electronic address: mikelly@transy.edu.
authors.forename: Michael R
authors.initials: MR
authors.lastname: Kelly
authors.affiliation: Department of BioSciences, Program in Ecology and Evolutionary Biology, Rice University, 6100 Main Street, MS-170, Houston, TX 77005-1892, United States. Electronic address: brad.ochocki@rice.edu.
authors.forename: Brad M
authors.initials: BM
authors.lastname: Ochocki
authors.affiliation: Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB, Canada T6G 2G1. Electronic address: segunmic@ualberta.ca.
authors.forename: Segun M
authors.initials: SM
authors.lastname: Akinwumi
authors.affiliation: Division of Global Pediatrics and Division of Epidemiology and Community Health, University of Minnesota, 717 Delaware Street SE, 3rd Floor, Minneapolis, MN 55414, United States. Electronic address: hamr0091@umn.edu.
authors.forename: Karen E S
authors.initials: KES
authors.lastname: Hamre
authors.affiliation: Department of Mathematics, The Ohio State University, 231 West 18th Ave, Columbus, OH 43210, United States. Electronic address: jtien@math.ohio-state.edu.
authors.forename: Joseph H
authors.initials: JH
authors.lastname: Tien
authors.affiliation: Departments of Epidemiology and Mathematics, University of Michigan, Ann Arbor, 1415 Washington Heights, Ann Arbor 48109, United States. Electronic address: marisae@umich.edu.
authors.forename: Marisa C
authors.initials: MC
authors.lastname: Eisenberg
mesh_terms: D000810:Angola; D050936:Basic Reproduction Number; D002771:Cholera / Q000453:epidemiology* / Q000276:immunology / Q000635:transmission; D003198:Computer Simulation; D058872:Epidemics; D006801:Humans; D008962:Models, Theoretical; D035501:Uncertainty
keywords: Comparative modeling; Model misspecification; Model structure; Parameter estimation