Researcher judgment and study design: challenges of using administrative data.
BACKGROUND
Questions have been raised about methods of studies finding substantial undercounting of workplace injuries and illnesses by the Bureau of Labor Statistics (BLS) and workers' compensation agencies. A more recent study of Minnesota concluded that the BLS survey captures 84-90% of workers' compensation cases.


METHODS
We examined the sensitivity of findings in two studies to alternate sample definitions and study assumptions.


RESULTS
Applying alternate sample construction rules to the earlier study increased estimated BLS reporting rates from 68% to 77%, assuming source independence. Applying alternate assumptions to the more recent Minnesota study reduced its high estimate of BLS reporting from 90% to 53-64%.


CONCLUSIONS
Studies linking administrative data from different sources require substantial judgment in constructing research datasets and choosing analytic methods. Moreover, different sample construction rules lead to different results. This suggests that sensitivity analysis should be carried out when alternatives cannot be ruled out. In this case, sensitivity analysis supports the hypothesis of substantial underreporting.
Boden
Leslie I
LI
Department of Environmental Health, Boston University School of Public Health, 715 Albany Street, Boston, MA 02118, USA. lboden@bu.edu
Ozonoff
Al
A
D000062:Accidents, Occupational / Q000706:statistics & numerical data*; D015982:Bias; D003430:Cross-Sectional Studies; D003625:Data Collection / Q000706:statistics & numerical data*; D016208:Databases, Factual / Q000706:statistics & numerical data*; D004608:Eligibility Determination; D006801:Humans; D007600:Judgment; D008910:Minnesota; D009784:Occupational Diseases / Q000453:epidemiology*; D011159:Population Surveillance; D015203:Reproducibility of Results; D012107:Research Design / Q000706:statistics & numerical data*; D014942:Workers' Compensation / Q000331:legislation & jurisprudence*