title: How credible are the study results? Evaluating and applying internal validity tools to literature-based assessments of environmental health hazards.
abstract: Environmental health hazard assessments are routinely relied upon for public health decision-making. The evidence base used in these assessments is typically developed from a collection of diverse sources of information of varying quality. It is critical that literature-based evaluations consider the credibility of individual studies used to reach conclusions through consistent, transparent and accepted methods. Systematic review procedures address study credibility by assessing internal validity or "risk of bias" - the assessment of whether the design and conduct of a study compromised the credibility of the link between exposure/intervention and outcome. This paper describes the commonalities and differences in risk-of-bias methods developed or used by five groups that conduct or provide methodological input for performing environmental health hazard assessments: the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group, the Navigation Guide, the National Toxicology Program's (NTP) Office of Health Assessment and Translation (OHAT) and Office of the Report on Carcinogens (ORoC), and the Integrated Risk Information System of the U.S. Environmental Protection Agency (EPA-IRIS). Each of these groups have been developing and applying rigorous assessment methods for integrating across a heterogeneous collection of human and animal studies to inform conclusions on potential environmental health hazards. There is substantial consistency across the groups in the consideration of risk-of-bias issues or "domains" for assessing observational human studies. There is a similar overlap in terms of domains addressed for animal studies; however, the groups differ in the relative emphasis placed on different aspects of risk of bias. Future directions for the continued harmonization and improvement of these methods are also discussed.
authors: Office of Health Assessment and Translation, Division of the National Toxicology Program, National Institute of Environmental Health Sciences (NIEHS), National Institutes of Health (NIH), Research Triangle Park, NC, USA.
authors: Andrew A
authors: AA
authors: Rooney
authors: National Center for Environmental Assessment, Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC, USA.
authors: Glinda S
authors: GS
authors: Cooper
authors: Office of the Report on Carcinogens, Division of the National Toxicology Program, National Institute of Environmental Health Sciences (NIEHS), National Institutes of Health (NIH), Research Triangle Park, NC, USA.
authors: Gloria D
authors: GD
authors: Jahnke
authors: University of California San Francisco, Program on Reproductive Health and the Environment, San Francisco, CA, USA.
authors: Juleen
authors: J
authors: Lam
authors: McMaster University, Department of Clinical Epidemiology and Biostatistics, Hamilton, Ontario, Canada.
authors: Rebecca L
authors: RL
authors: Morgan
authors: Office of Health Assessment and Translation, Division of the National Toxicology Program, National Institute of Environmental Health Sciences (NIEHS), National Institutes of Health (NIH), Research Triangle Park, NC, USA.
authors: Abee L
authors: AL
authors: Boyles
authors: Integrated Laboratory Systems (ILS), Morrisville, NC, USA.
authors: Jennifer M
authors: JM
authors: Ratcliffe
authors: National Center for Environmental Assessment, Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC, USA.
authors: Andrew D
authors: AD
authors: Kraft
authors: McMaster University, Department of Clinical Epidemiology and Biostatistics, Hamilton, Ontario, Canada.
authors: Holger J
authors: HJ
authors: Schünemann
authors: Integrated Laboratory Systems (ILS), Morrisville, NC, USA.
authors: Pamela
authors: P
authors: Schwingl
authors: National Center for Environmental Assessment, Office of Research and Development, U.S. Environmental Protection Agency, Washington, DC, USA.
authors: Teneille D
authors: TD
authors: Walker
authors: Office of Health Assessment and Translation, Division of the National Toxicology Program, National Institute of Environmental Health Sciences (NIEHS), National Institutes of Health (NIH), Research Triangle Park, NC, USA.
authors: Kristina A
authors: KA
authors: Thayer
authors: Office of the Report on Carcinogens, Division of the National Toxicology Program, National Institute of Environmental Health Sciences (NIEHS), National Institutes of Health (NIH), Research Triangle Park, NC, USA. Electronic address: lunn@niehs.nih.gov.
authors: Ruth M
authors: RM
authors: Lunn
mesh_terms: D003657:Decision Making; D004782:Environmental Health / Q000379:methods*; D006801:Humans; D011634:Public Health / Q000379:methods*; D012196:Review Literature as Topic
keywords: Environmental health; Hazard assessment; Internal validity; Risk of bias; Systematic review