title: Perspective: NutriGrade: A Scoring System to Assess and Judge the Meta-Evidence of Randomized Controlled Trials and Cohort Studies in Nutrition Research.
abstract: The objective of this study was to develop a scoring system (NutriGrade) to evaluate the quality of evidence of randomized controlled trial (RCT) and cohort study meta-analyses in nutrition research, building upon previous tools and expert recommendations. NutriGrade aims to assess the meta-evidence of an association or effect between different nutrition factors and outcomes, taking into account nutrition research-specific requirements not considered by other tools. In a pretest study, 6 randomly selected meta-analyses investigating diet-disease relations were evaluated with NutriGrade by 5 independent raters. After revision, NutriGrade was applied by the same raters to 30 randomly selected meta-analyses in the same thematic area. The reliability of ratings of NutriGrade items was calculated with the use of a multirater κ, and reliability of the total (summed scores) was calculated with the use of intraclass correlation coefficients (ICCs). The following categories for meta-evidence evaluation were established: high (8-10), moderate (6-7.99), low (4-5.99), and very low (0-3.99). The NutriGrade scoring system (maximum of 10 points) comprises the following items: 1) risk of bias, study quality, and study limitations, 2) precision, 3) heterogeneity, 4) directness, 5) publication bias, 6) funding bias, 7) study design, 8) effect size, and 9) dose-response. The NutriGrade score varied between 2.9 (very low meta-evidence) and 8.8 (high meta-evidence) for meta-analyses of RCTs, and it ranged between 3.1 and 8.8 for meta-analyses of cohort studies. The κ value of the ratings for each scoring item varied from 0.32 (95% CI: 0.22, 0.42) for risk of bias for cohort studies and 0.95 (95% CI: 0.91, 0.99) for study design, with a mean κ of 0.66 (95% CI: 0.53, 0.79). The ICC of the total score was 0.81 (95% CI: 0.69, 0.90). The NutriGrade scoring system showed good agreement and reliability. The initial findings regarding the performance of this newly established scoring system need further evaluation in independent analyses.
authors.affiliation: Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany; lukas.schwingshackl@dife.de.
authors.forename: Lukas
authors.identifier: 0000-0003-3407-7594
authors.initials: L
authors.lastname: Schwingshackl
authors.affiliation: Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany.
authors.forename: Sven
authors.initials: S
authors.lastname: Knüppel
authors.affiliation: Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany.
authors.forename: Carolina
authors.initials: C
authors.lastname: Schwedhelm
authors.affiliation: Department of Nutritional Sciences, University of Vienna, Vienna, Austria.
authors.forename: Georg
authors.initials: G
authors.lastname: Hoffmann
authors.affiliation: Department of Nutritional Sciences, University of Vienna, Vienna, Austria.
authors.forename: Benjamin
authors.initials: B
authors.lastname: Missbach
authors.affiliation: Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany.|Department of Pediatric Gastroenterology and Metabolic Diseases, Poznan University of Medical Sciences, Poznan, Poland.
authors.forename: Marta
authors.initials: M
authors.lastname: Stelmach-Mardas
authors.affiliation: Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany.
authors.forename: Stefan
authors.initials: S
authors.lastname: Dietrich
authors.affiliation: Nutrition, Immunity, and Metabolism Start-Up Lab.|Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany.
authors.forename: Fabian
authors.initials: F
authors.lastname: Eichelmann
authors.affiliation: Centre for Primary Care, Institute of Population Health, University of Manchester, Manchester, United Kingdom.
authors.forename: Evangelos
authors.initials: E
authors.lastname: Kontopantelis
authors.affiliation: Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany.
authors.forename: Khalid
authors.initials: K
authors.lastname: Iqbal
authors.affiliation: Nutrition, Immunity, and Metabolism Start-Up Lab.|Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany.
authors.forename: Krasimira
authors.initials: K
authors.lastname: Aleksandrova
authors.affiliation: Institute of Nutrition, Friedrich Schiller University Jena, Jena, Germany.|Competence Cluster of Nutrition and Cardiovascular Health, Halle-Jena-Leipzig, Germany.
authors.forename: Stefan
authors.initials: S
authors.lastname: Lorkowski
authors.affiliation: Department of Epidemiology and Preventive Medicine, University of Regensburg, Regensburg, Germany; and.
authors.forename: Michael F
authors.initials: MF
authors.lastname: Leitzmann
authors.affiliation: Department of Nutritional, Food, and Consumer Sciences, University of Applied Sciences, Fulda, Germany.
authors.forename: Anja
authors.initials: A
authors.lastname: Kroke
authors.affiliation: Department of Epidemiology, German Institute of Human Nutrition Potsdam Rehbruecke, Nuthetal, Germany.
authors.forename: Heiner
authors.initials: H
authors.lastname: Boeing
mesh_terms: D035843:Biomedical Research / Q000592:standards*; D015331:Cohort Studies; D005069:Evaluation Studies as Topic; D006801:Humans; D015201:Meta-Analysis as Topic; D052756:Nutritional Sciences / Q000592:standards*; D016032:Randomized Controlled Trials as Topic / Q000592:standards*; D015203:Reproducibility of Results; D012107:Research Design
keywords: meta-analysis; meta-evidence; nutrition; reliability; scoring