title: Systematic comparison of study quality criteria.
abstract: Approaches for the systematic review and evaluation of chemical toxicity are currently being reconsidered, with a specific focus on the evaluation of individual studies and their integration into the overall body of evidence. This renewed interest has arisen, in part, as a result of several prominent reviews of these approaches by special committees of the National Research Council (NRC), among others. We conducted a critical evaluation of several available frameworks for evaluating study quality. We assessed the criteria separately for human, animal, and in vitro studies as well as for systematic reviews. We then evaluated commonalities across disciplines. We also considered the potential implications of applying criteria frameworks and how they bear on fundamental risk assessment questions. We found that the available frameworks within each discipline differed in terms of their intended purpose and level of guidance for decision making. All the frameworks across disciplines shared common themes, however, including the adequate reporting of specific details of study conditions and design/protocol, selection and randomization of study groups (where applicable), outcome assessment methods and applicability (e.g., validity and reliability), avoidance of selective reporting, and the consideration of potential confounders or bias. We identified the most informative study quality considerations, which will enable researchers to implement more objective and standardized methods for evaluating studies and, ultimately, improve risk assessment methods.
authors.affiliation: Gradient, 20 University Road, Cambridge, MA 02138, USA.
authors.forename: Heather N
authors.initials: HN
authors.lastname: Lynch
authors.affiliation: Gradient, 20 University Road, Cambridge, MA 02138, USA.
authors.forename: Julie E
authors.initials: JE
authors.lastname: Goodman
authors.affiliation: Gradient, 600 Stewart Street, Suite 1900, Seattle, WA 98101, USA.
authors.forename: Jade A
authors.initials: JA
authors.lastname: Tabony
authors.affiliation: Gradient, 20 University Road, Cambridge, MA 02138, USA. Electronic address: lrhomberg@gradientcorp.com.
authors.forename: Lorenz R
authors.initials: LR
authors.lastname: Rhomberg
mesh_terms: D000818:Animals; D015982:Bias; D000068598:Data Accuracy; D003627:Data Interpretation, Statistical; D017408:Guidelines as Topic; D006801:Humans; D012107:Research Design / Q000592:standards* / Q000706:statistics & numerical data; D012196:Review Literature as Topic; D018570:Risk Assessment; D018675:Toxicity Tests / Q000592:standards* / Q000706:statistics & numerical data
keywords: Evidence integration; Study quality; Systematic review