Does ignoring model selection when assessing the effect of particulate matter air pollution on mortality make us too vigilant?
OBJECTIVE
To investigate the extent to which standard errors can be underestimated in time-series studies of the association between particulate matter air pollution (PM) and mortality if model selection variation is not accounted for.


METHODS
Actual-time series data from Cook County, Illinois, and Salt Lake County, Utah, for the period 1987 to 2000 were used to generate mortality time series. These series were used to examine the overconfidence resulting from ignoring variability introduced by the model selection process.


RESULTS
When variation associated with a model selection process is not accounted for, we found that the estimated standard errors for the effect of PM on mortality were substantially smaller than the true standard errors that necessarily incorporate model selection variability. Because of this, the true standard errors are approximately 70% larger than the reported standard errors. We also found that not accounting for model selection effects can result in the observed size of tests of no association between PM and mortality being up to about five times the nominal significance level.


CONCLUSIONS
Failing to account properly for the effect of model selection can reduce the accepted burden of proof for concluding a statistically significant association between PM and mortality.
Roberts
Steven
S
School of Finance, Actuarial Studies and Applied Statistics, College of Business and Economics, Australian National University, Canberra ACT 0200, Australia. roberts@anu.edu.au
Martin
Michael A
MA
D000368:Aged; D002641:Chicago / Q000453:epidemiology; D004812:Epidemiologic Methods; D006801:Humans; D009026:Mortality; D052638:Particulate Matter; D012189:Retrospective Studies; D013997:Time Factors; D014589:Utah / Q000453:epidemiology
D052638:Particulate Matter