Susceptibility to fraud in systematic reviews: lessons from the Reuben case.
BACKGROUND
Dr. Scott Reuben allegedly fabricated data. The authors of the current article examined the impact of Reuben reports on conclusions of systematic reviews.


METHODS
The authors searched in ISI Web of Knowledge systematic reviews citing Reuben reports. Systematic reviews were grouped into one of three categories: I, only cited but did not include Reuben reports; II, retrieved and considered, but eventually excluded Reuben reports; III, included Reuben reports. For quantitative systematic reviews (i.e., meta-analyses), a relevant difference was defined as a significant result becoming nonsignificant (or vice versa) by excluding Reuben reports. For qualitative systematic reviews, each author decided independently whether noninclusion of Reuben reports would have changed conclusions.


RESULTS
Twenty-five systematic reviews (5 category I, 6 category II, 14 category III) cited 27 Reuben reports (published 1994-2007). Most tested analgesics in surgical patients. One of 6 quantitative category III reviews would have reached different conclusions without Reuben reports. In all 6 (30 subgroup analyses involving Reuben reports), exclusion of Reuben reports never made any difference when the number of patients from Reuben reports was less than 30% of all patients included in the analysis. Of 8 qualitative category III reviews, all authors agreed that one would certainly have reached different conclusions without Reuben reports. For another 4, the authors' judgment was not unanimous.


CONCLUSIONS
Carefully performed systematic reviews proved robust against the impact of Reuben reports. Quantitative systematic reviews were vulnerable if the fraudulent data were more than 30% of the total. Qualitative systematic reviews seemed at greater risk than quantitative.
Marret
Emmanuel
E
Division of Anesthesiology, Department of Anesthesiology, Critical Care and Clinical Pharmacology, Geneva University Hospitals, Geneva, Switzerland.
Elia
Nadia
N
Dahl
Jørgen B
JB
McQuay
Henry J
HJ
Møiniche
Steen
S
Moore
R Andrew
RA
Straube
Sebastian
S
Tramèr
Martin R
MR
D000700:Analgesics / Q000009:adverse effects / Q000627:therapeutic use; D052246:Cyclooxygenase 2 Inhibitors / Q000009:adverse effects / Q000627:therapeutic use; D006801:Humans; D015201:Meta-Analysis as Topic; D010149:Postoperative Pain / Q000188:drug therapy; D016032:Randomized Controlled Trials as Topic; D012196:Review Literature as Topic; D015871:Scientific Misconduct; D016896:Treatment Outcome
D000700:Analgesics; D052246:Cyclooxygenase 2 Inhibitors