title: The reliability of tweets as a supplementary method of seasonal influenza surveillance.
abstract: BACKGROUND
  Existing influenza surveillance in the United States is focused on the collection of data from sentinel physicians and hospitals; however, the compilation and distribution of reports are usually delayed by up to 2 weeks. With the popularity of social media growing, the Internet is a source for syndromic surveillance due to the availability of large amounts of data. In this study, tweets, or posts of 140 characters or less, from the website Twitter were collected and analyzed for their potential as surveillance for seasonal influenza.
  
  
  OBJECTIVE
  There were three aims: (1) to improve the correlation of tweets to sentinel-provided influenza-like illness (ILI) rates by city through filtering and a machine-learning classifier, (2) to observe correlations of tweets for emergency department ILI rates by city, and (3) to explore correlations for tweets to laboratory-confirmed influenza cases in San Diego.
  
  
  METHODS
  Tweets containing the keyword "flu" were collected within a 17-mile radius from 11 US cities selected for population and availability of ILI data. At the end of the collection period, 159,802 tweets were used for correlation analyses with sentinel-provided ILI and emergency department ILI rates as reported by the corresponding city or county health department. Two separate methods were used to observe correlations between tweets and ILI rates: filtering the tweets by type (non-retweets, retweets, tweets with a URL, tweets without a URL), and the use of a machine-learning classifier that determined whether a tweet was "valid", or from a user who was likely ill with the flu.
  
  
  RESULTS
  Correlations varied by city but general trends were observed. Non-retweets and tweets without a URL had higher and more significant (P<.05) correlations than retweets and tweets with a URL. Correlations of tweets to emergency department ILI rates were higher than the correlations observed for sentinel-provided ILI for most of the cities. The machine-learning classifier yielded the highest correlations for many of the cities when using the sentinel-provided or emergency department ILI as well as the number of laboratory-confirmed influenza cases in San Diego. High correlation values (r=.93) with significance at P<.001 were observed for laboratory-confirmed influenza cases for most categories and tweets determined to be valid by the classifier.
  
  
  CONCLUSIONS
  Compared to tweet analyses in the previous influenza season, this study demonstrated increased accuracy in using Twitter as a supplementary surveillance tool for influenza as better filtering and classification methods yielded higher correlations for the 2013-2014 influenza season than those found for tweets in the previous influenza season, where emergency department ILI rates were better correlated to tweets than sentinel-provided ILI rates. Further investigations in the field would require expansion with regard to the location that the tweets are collected from, as well as the availability of more ILI data.
authors.affiliation: Graduate School of Public Health, San Diego State University, San Diego, CA, United States.
authors.forename: Anoshé A
authors.identifier: http://orcid.org/0000-0003-0248-3192
authors.initials: AA
authors.lastname: Aslam
authors.forename: Ming-Hsiang
authors.identifier: http://orcid.org/0000-0003-3421-486X
authors.initials: MH
authors.lastname: Tsou
authors.forename: Brian H
authors.identifier: http://orcid.org/0000-0003-3838-6052
authors.initials: BH
authors.lastname: Spitzberg
authors.forename: Li
authors.identifier: http://orcid.org/0000-0002-7933-5174
authors.initials: L
authors.lastname: An
authors.forename: J Mark
authors.identifier: http://orcid.org/0000-0003-2744-937X
authors.initials: JM
authors.lastname: Gawron
authors.forename: Dipak K
authors.identifier: http://orcid.org/0000-0002-2540-5069
authors.initials: DK
authors.lastname: Gupta
authors.forename: K Michael
authors.identifier: http://orcid.org/0000-0003-4107-8050
authors.initials: KM
authors.lastname: Peddecord
authors.forename: Anna C
authors.identifier: http://orcid.org/0000-0003-1929-8006
authors.initials: AC
authors.lastname: Nagel
authors.forename: Christopher
authors.identifier: http://orcid.org/0000-0002-3336-2601
authors.initials: C
authors.lastname: Allen
authors.forename: Jiue-An
authors.identifier: http://orcid.org/0000-0003-4246-0470
authors.initials: JA
authors.lastname: Yang
authors.forename: Suzanne
authors.identifier: http://orcid.org/0000-0002-4983-0748
authors.initials: S
authors.lastname: Lindsay
mesh_terms: D002140:California / Q000453:epidemiology; D004636:Emergency Service, Hospital / Q000706:statistics & numerical data; D006801:Humans; D007251:Influenza, Human / Q000453:epidemiology*; D011159:Population Surveillance / Q000379:methods*; D015203:Reproducibility of Results; D012621:Seasons; D061108:Social Media; D014481:United States / Q000453:epidemiology
keywords: Internet; Twitter; influenza; infodemiology; infoveillance; syndromic surveillance; tweets