Adjustment of meta-analyses on the basis of quality scores should be abandoned.
OBJECTIVE
To find if a particular quality score was better than others at validly scoring the quality of randomized controlled trials, both by examining the consistency of dividing studies into high and low quality and using a large study as a reference standard.


METHODS
Observational study of meta-analyses from the Cochrane Library. These had to have binary outcomes that included more than 10 studies, one or more of which randomized more than 500 people into each group.


RESULTS
Eighteen systematic reviews, with 65 meta-analyses using binary outcomes, were included and the included trials were scored for 43 different quality scores. None of these scores was better at dividing the studies in to low and high quality, and none of the scores was better over the 65 meta-analyses in making the result closer to the reference standard.


CONCLUSIONS
None of the quality scores found appeared to measure quality validly. It is a mistake to assign meaning to the result of a quality score.
Herbison
Peter
P
Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, PO Box 913, Dunedin, New Zealand. peter.herbison@otago.ac.nz
Hay-Smith
Jean
J
Gillespie
William J
WJ
D015982:Bias; D006801:Humans; D015201:Meta-Analysis as Topic; D016032:Randomized Controlled Trials as Topic / Q000592:standards*; D012015:Reference Standards; D015203:Reproducibility of Results; D012196:Review Literature as Topic