Attention should be given to multiplicity issues in systematic reviews.
OBJECTIVE
The objective of this paper is to describe the problem of multiple comparisons in systematic reviews and to provide some guidelines on how to deal with it in practice.


METHODS
We describe common reasons for multiplicity in systematic reviews, and present some examples. We provide guidance on how to deal with multiplicity when it is unavoidable.


RESULTS
We identified six common reasons for multiplicity in systematic reviews: multiple outcomes, multiple groups, multiple time points, multiple effect measures, subgroup analyses, and multiple looks at accumulating data. The existing methods to deal with multiplicity in single trials can not always be applied in systematic reviews.


CONCLUSIONS
There is no simple and completely satisfactory solution to the problem of multiple comparisons in systematic reviews. More research is required to develop multiple comparison procedures for use in systematic reviews. Authors and consumers of systematic reviews should give serious attention to multiplicity in systematic reviews when presenting, interpreting and using the results of these reports.
Bender
Ralf
R
Institute for Quality and Efficiency in Health Care, Dillenburger Street 27, Cologne, Germany. ralf.bender@iqwig.de
Bunce
Catey
C
Clarke
Mike
M
Gates
Simon
S
Lange
Stefan
S
Pace
Nathan L
NL
Thorlund
Kristian
K
D015982:Bias; D035843:Biomedical Research / Q000592:standards*; D003627:Data Interpretation, Statistical; D017408:Guidelines as Topic / Q000592:standards*; D006801:Humans; D015201:Meta-Analysis as Topic; D012196:Review Literature as Topic