A randomized controlled study of reviewer bias against an unconventional therapy.
A study was designed to test the hypothesis that experts who review papers for publication are prejudiced against an unconventional form of therapy. Two versions were produced (A and B) of a 'short report' that related to treatments of obesity, identical except for the nature of the intervention. Version A related to an orthodox treatment, version B to an unconventional treatment. 398 reviewers were randomized to receive one or the other version for peer review. The primary outcomes were the reviewers' rating of 'importance' on a scale of 1-5 and their verdict regarding rejection or acceptance of the paper. Reviewers were unaware that they were taking part in a study. The overall response rate was 41.7%, and 141 assessment forms were suitable for statistical evaluation. After dichotomization of the rating scale, a significant difference in favour of the orthodox version with an odds ratio of 3.01 (95% confidence interval, 1.03 to 8.25), was found. This observation mirrored that of the visual analogue scale for which the respective medians and interquartile ranges were 67% (51% to 78.5%) for version A and 57% (29.7% to 72.6%) for version B. Reviewers showed a wide range of responses to both versions of the paper, with a significant bias in favour of the orthodox version. Authors of technically good unconventional papers may therefore be at a disadvantage in the peer review process. Yet the effect is probably too small to preclude publication of their work in peer-reviewed orthodox journals.
Resch
K I
KI
Forschungsinstitut für Balneologie und Kurortwissenschaft, Bad Elster, Germany.
Ernst
E
E
Garrow
J
J
D015982:Bias; D002951:Citrates / Q000627:therapeutic use; D000529:Complementary Therapies; D006801:Humans; D009765:Obesity / Q000188:drug therapy; D017712:Peer Review, Research / Q000592:standards*; D013455:Sulfur / Q000627:therapeutic use
D002951:Citrates; D013455:Sulfur