Evaluating the performance of infectious disease forecasts: A comparison of climate-driven and seasonal dengue forecasts for Mexico.
Dengue viruses, which infect millions of people per year worldwide, cause large epidemics that strain healthcare systems. Despite diverse efforts to develop forecasting tools including autoregressive time series, climate-driven statistical, and mechanistic biological models, little work has been done to understand the contribution of different components to improved prediction. We developed a framework to assess and compare dengue forecasts produced from different types of models and evaluated the performance of seasonal autoregressive models with and without climate variables for forecasting dengue incidence in Mexico. Climate data did not significantly improve the predictive power of seasonal autoregressive models. Short-term and seasonal autocorrelation were key to improving short-term and long-term forecasts, respectively. Seasonal autoregressive models captured a substantial amount of dengue variability, but better models are needed to improve dengue forecasting. This framework contributes to the sparse literature of infectious disease prediction model evaluation, using state-of-the-art validation techniques such as out-of-sample testing and comparison to an appropriate reference model.
Dengue Branch, Division of Vector-Borne Diseases, Centers for Disease Control and Prevention, San Juan, Puerto Rico.|Center for Communicable Disease Dynamics, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, USA.
Michael A
MA
Johansson
Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, Massachusetts, USA.
Nicholas G
NG
Reich
Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts, USA.
Aditi
A
Hota
Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts, USA.|Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA.
John S
JS
Brownstein
Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts, USA.|Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA.|J.A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA.
Mauricio
M
Santillana
PLoS One. 2012;7(11):e50972
23226436
PLoS Negl Trop Dis. 2012;6(11):e1908
23209852
PLoS Negl Trop Dis. 2014 Apr 10;8(4):e2771
24722434
Cad Saude Publica. 2012 Nov;28(11):2189-97
23147960
Am J Trop Med Hyg. 2014 Sep;91(3):598-610
24957546
Ambio. 2008 Jun;37(4):292-4
18686509
BMC Infect Dis. 2011 Jun 09;11:166
21658238
Indian J Public Health. 2012 Oct-Dec;56(4):281-5
23354138
Epidemiol Infect. 2010 Jul;138(7):962-7
20370955
Am J Trop Med Hyg. 2013 Dec;89(6):1066-80
23958906
Acta Trop. 2007 Jul;103(1):50-7
17612499
BMC Public Health. 2009 Nov 20;9:422
19930557
Trop Med Int Health. 2007 Nov;12(11):1327-37
17956543
Acta Trop. 2015 Jan;141(Pt A):88-96
25447266
Biometrics. 1988 Dec;44(4):1019-31
3148334
Stat Med. 2013 Feb 28;32(5):864-83
22927252
PLoS Comput Biol. 2015 Oct 29;11(10):e1004513
26513245
Nature. 2009 Feb 19;457(7232):1012-4
19020500
Proc Natl Acad Sci U S A. 2015 Nov 24;112(47):14473-8
26553980
Int J Environ Health Res. 2004 Dec;14(6):429-41
15545038
PLoS One. 2013 May 09;8(5):e63717
23671694
Am J Trop Med Hyg. 2013 May;88(5):971-81
23478584
Environ Res Lett. 2009 Mar 4;4:140111-140118
19763186
J R Soc Interface. 2013 Feb 13;10(81):20120921
23407571
J Med Entomol. 1993 Nov;30(6):1003-17
8271242
Epidemiol Infect. 2012 Jul;140(7):1244-51
21906411
Southeast Asian J Trop Med Public Health. 2013 Mar;44(2):206-17
23691630
Int J Environ Health Res. 2008 Aug;18(4):253-65
18668414
Parasit Vectors. 2013 Dec 12;6:351
24330720
Rev Soc Bras Med Trop. 2011 Jul-Aug;44(4):436-40
21860888
J R Soc Interface. 2013 Jul 03;10(86):20130414
23825116
Southeast Asian J Trop Med Public Health. 2008 Jan;39(1):90-8
18567447
BMC Med Inform Decis Mak. 2012 Nov 05;12:124
23126401
Vaccine. 2011 Aug 11;29(35):5860-8
21699949
PLoS Negl Trop Dis. 2012;6(8):e1760
22880140
Biomedica. 2013 Sep;33 Suppl 1:142-52
24652258
Epidemics. 2015 Jun;11:92-102
25979287
Am J Trop Med Hyg. 2008 Dec;79(6):933-9
19052308
Asian Pac J Trop Med. 2012 Mar;5(3):249-52
22305794
PLoS Negl Trop Dis. 2014 May 01;8(5):e2805
24786517
PLoS Negl Trop Dis. 2009;3(2):e382
19221592
Science. 2014 Mar 14;343(6176):1203-5
24626916
PLoS Med. 2009 Nov;6(11):e1000168
19918363
PLoS Negl Trop Dis. 2012;6(2):e1470
22348154
Artif Intell Med. 2014 Jan;60(1):41-51
24388398
Indian J Med Res. 2012 Jul;136(1):32-9
22885261
Int J Biometeorol. 2001 Jul;45(2):81-9
11513051
Trop Biomed. 2012 Sep;29(3):339-48
23018496
Asia Pac J Public Health. 2014 Jan;26(1):48-57
23761588
Occup Environ Med. 2010 May;67(5):307-11
19819860
PLoS Negl Trop Dis. 2012;6(5):e1648
22629476
Comput Math Methods Med. 2012;2012:758674
22481978
Microbes Infect. 2002 Jun;4(7):699-705
12067829
PLoS Negl Trop Dis. 2010 Nov 16;4(11):e890
21103381
J R Soc Interface. 2010 Feb 6;7(43):297-307
19570792