title: Does ignoring model selection when assessing the effect of particulate matter air pollution on mortality make us too vigilant?
abstract: OBJECTIVE
  To investigate the extent to which standard errors can be underestimated in time-series studies of the association between particulate matter air pollution (PM) and mortality if model selection variation is not accounted for.
  
  
  METHODS
  Actual-time series data from Cook County, Illinois, and Salt Lake County, Utah, for the period 1987 to 2000 were used to generate mortality time series. These series were used to examine the overconfidence resulting from ignoring variability introduced by the model selection process.
  
  
  RESULTS
  When variation associated with a model selection process is not accounted for, we found that the estimated standard errors for the effect of PM on mortality were substantially smaller than the true standard errors that necessarily incorporate model selection variability. Because of this, the true standard errors are approximately 70% larger than the reported standard errors. We also found that not accounting for model selection effects can result in the observed size of tests of no association between PM and mortality being up to about five times the nominal significance level.
  
  
  CONCLUSIONS
  Failing to account properly for the effect of model selection can reduce the accepted burden of proof for concluding a statistically significant association between PM and mortality.
authors.affiliation: School of Finance, Actuarial Studies and Applied Statistics, College of Business and Economics, Australian National University, Canberra ACT 0200, Australia. roberts@anu.edu.au
authors.forename: Steven
authors.initials: S
authors.lastname: Roberts
authors.forename: Michael A
authors.initials: MA
authors.lastname: Martin
mesh_terms: D000368:Aged; D002641:Chicago / Q000453:epidemiology; D004812:Epidemiologic Methods; D006801:Humans; D009026:Mortality; D052638:Particulate Matter; D012189:Retrospective Studies; D013997:Time Factors; D014589:Utah / Q000453:epidemiology
chemical_list: D052638:Particulate Matter