title: The interpretation of systematic reviews with meta-analyses: an objective or subjective process?
abstract: BACKGROUND
  Discrepancies between the conclusions of different meta-analyses (quantitative syntheses of systematic reviews) are often ascribed to methodological differences. The objective of this study was to determine the discordance in interpretations when meta-analysts are presented with identical data.
  
  
  METHODS
  We searched the literature for all randomized clinical trials (RCT) and review articles on the efficacy of intravenous magnesium in the early post-myocardial infarction period. We organized the articles chronologically and grouped them in packages. The first package included the first RCT, and a summary of the review articles published prior to first RCT. The second package contained the second and third RCT, a meta-analysis based on the data, and a summary of all review articles published prior to the third RCT. Similar packages were created for the 5th RCT, 10th RCT, 20th RCT and 23rd RCT (all articles). We presented the packages one at a time to eight different reviewers and asked them to answer three clinical questions after each package based solely on the information provided. The clinical questions included whether 1) they believed magnesium is now proven beneficial, 2) they believed magnesium will eventually be proven to be beneficial, and 3) they would recommend its use at this time.
  
  
  RESULTS
  There was considerable disagreement among the reviewers for each package, and for each question. The discrepancies increased when the heterogeneity of the data increased. In addition, some reviewers became more sceptical of the effectiveness of magnesium over time, and some reviewers became less sceptical.
  
  
  CONCLUSIONS
  The interpretation of the results of systematic reviews with meta-analyses includes a subjective component that can lead to discordant conclusions that are independent of the methodology used to obtain or analyse the data.
authors.affiliation: Centre for Clinical Epidemiology and Community Studies, Lady Davis Institute for Medical Research, SMBD-Jewish General Hospital, McGill University, Montreal, Canada. ian.shrier@mcgill.ca
authors.forename: Ian
authors.initials: I
authors.lastname: Shrier
authors.forename: Jean-François
authors.initials: JF
authors.lastname: Boivin
authors.forename: Robert W
authors.initials: RW
authors.lastname: Platt
authors.forename: Russell J
authors.initials: RJ
authors.lastname: Steele
authors.forename: James M
authors.initials: JM
authors.lastname: Brophy
authors.forename: Franco
authors.initials: F
authors.lastname: Carnevale
authors.forename: Mark J
authors.initials: MJ
authors.lastname: Eisenberg
authors.forename: Andrea
authors.initials: A
authors.lastname: Furlan
authors.forename: Ritsuko
authors.initials: R
authors.lastname: Kakuma
authors.forename: Mary Ellen
authors.initials: ME
authors.lastname: Macdonald
authors.forename: Louise
authors.initials: L
authors.lastname: Pilote
authors.forename: Michel
authors.initials: M
authors.lastname: Rossignol
mesh_terms: D000889:Anti-Arrhythmia Agents / Q000627:therapeutic use; D003657:Decision Making; D003661:Decision Support Techniques; D006801:Humans; D008278:Magnesium Sulfate / Q000627:therapeutic use; D015201:Meta-Analysis as Topic; D009203:Myocardial Infarction / Q000188:drug therapy / Q000401:mortality; D015588:Observer Variation; D016017:Odds Ratio; D036301:Qualitative Research; D016032:Randomized Controlled Trials as Topic; D015203:Reproducibility of Results; D012196:Review Literature as Topic
chemical_list: D000889:Anti-Arrhythmia Agents; D008278:Magnesium Sulfate