title: Automatically quantifying the scientific quality and sensationalism of news records mentioning pandemics: validating a maximum entropy machine-learning model.
abstract: To develop and validate a method for automatically quantifying the scientific quality and sensationalism of individual news records.
  
  
  
  After retrieving 163,433 news records mentioning the Severe Acute Respiratory Syndrome (SARS) and H1N1 pandemics, a maximum entropy model for inductive machine learning was used to identify relationships among 500 randomly sampled news records that correlated with systematic human assessments of their scientific quality and sensationalism. These relationships were then computationally applied to automatically classify 10,000 additional randomly sampled news records. The model was validated by randomly sampling 200 records and comparing human assessments of them to the computer assessments.
  
  
  
  The computer model correctly assessed the relevance of 86% of news records, the quality of 65% of records, and the sensationalism of 73% of records, as compared to human assessments. Overall, the scientific quality of SARS and H1N1 news media coverage had potentially important shortcomings, but coverage was not too sensationalizing. Coverage slightly improved between the two pandemics.
  
  
  
  Automated methods can evaluate news records faster, cheaper, and possibly better than humans. The specific procedure implemented in this study can at the very least identify subsets of news records that are far more likely to have particular scientific and discursive qualities.
authors: Global Strategy Lab, Faculty of Law, University of Ottawa, 57 Louis Pasteur St, Ottawa, Ontario, Canada K1N 6N5. Electronic address: steven.hoffman@uottawa.ca.
authors: Steven J
authors: SJ
authors: Hoffman
authors: Harvard Global Health Institute, Harvard University, 42 Church St, Cambridge, Massachusetts 02138, USA.
authors: Victoria
authors: V
authors: Justicz
mesh_terms: D016208:Databases, Factual; D006801:Humans; D053118:Influenza A Virus, H1N1 Subtype; D007251:Influenza, Human / Q000453:epidemiology*; D033181:Information Dissemination / Q000379:methods*; D000069550:Machine Learning / Q000592:standards*; D008402:Mass Media / Q000592:standards*; D058873:Pandemics; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D045169:Severe Acute Respiratory Syndrome / Q000453:epidemiology*
keywords: Health communication; Machine learning; Mass media; News; Pandemics; Validation studies