title: Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods.
abstract: BACKGROUND
  Thresholds for statistical significance when assessing meta-analysis results are being insufficiently demonstrated by traditional 95% confidence intervals and P-values. Assessment of intervention effects in systematic reviews with meta-analysis deserves greater rigour.
  
  
  METHODS
  Methodologies for assessing statistical and clinical significance of intervention effects in systematic reviews were considered. Balancing simplicity and comprehensiveness, an operational procedure was developed, based mainly on The Cochrane Collaboration methodology and the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) guidelines.
  
  
  RESULTS
  We propose an eight-step procedure for better validation of meta-analytic results in systematic reviews (1) Obtain the 95% confidence intervals and the P-values from both fixed-effect and random-effects meta-analyses and report the most conservative results as the main results. (2) Explore the reasons behind substantial statistical heterogeneity using subgroup and sensitivity analyses (see step 6). (3) To take account of problems with multiplicity adjust the thresholds for significance according to the number of primary outcomes. (4) Calculate required information sizes (≈ the a priori required number of participants for a meta-analysis to be conclusive) for all outcomes and analyse each outcome with trial sequential analysis. Report whether the trial sequential monitoring boundaries for benefit, harm, or futility are crossed. (5) Calculate Bayes factors for all primary outcomes. (6) Use subgroup analyses and sensitivity analyses to assess the potential impact of bias on the review results. (7) Assess the risk of publication bias. (8) Assess the clinical significance of the statistically significant review results.
  
  
  CONCLUSIONS
  If followed, the proposed eight-step procedure will increase the validity of assessments of intervention effects in systematic reviews of randomised clinical trials.
authors.affiliation: Rigshospitalet, Copenhagen Trial Unit, Centre for Clinical Intervention Research, Department 7812, Copenhagen University Hospital, Copenhagen, Denmark. jcj@ctu.dk.
authors.forename: Janus Christian
authors.initials: JC
authors.lastname: Jakobsen
authors.forename: Jørn
authors.initials: J
authors.lastname: Wetterslev
authors.forename: Per
authors.initials: P
authors.lastname: Winkel
authors.forename: Theis
authors.initials: T
authors.lastname: Lange
authors.forename: Christian
authors.initials: C
authors.lastname: Gluud
mesh_terms: D001499:Bayes Theorem; D006801:Humans; D015201:Meta-Analysis as Topic; D015233:Models, Statistical; D017063:Outcome Assessment, Health Care / Q000706:statistics & numerical data*; D017594:Publication Bias; D016032:Randomized Controlled Trials as Topic; D016008:Statistical Distributions; D054928:Validation Studies as Topic