The quality of reports of critical care meta-analyses in the Cochrane Database of Systematic Reviews: an independent appraisal.
OBJECTIVE
To independently appraise the methodological quality of a sample of reports of meta-analyses that address critical care topics in the Cochrane Database of Systematic Reviews compared with the quality of reports published in regular journals, using a validated assessment instrument, the Overview Quality Assessment Questionnaire (OQAQ).


METHODS
Studies were selected from a search of MEDLINE, EMBASE, and the Cochrane Database of Systematic Reviews from 1994 to 2003, using multiple search terms for critical care and sensitive filters to identify meta-analyses.


METHODS
Two authors independently selected meta-analyses that addressed topics pertinent to critical care medicine.


METHODS
Two authors independently extracted the data. The proportion of reports that met each component of the OQAQ was determined, as was the overall quality score. Meta-analyses published in the Cochrane Database of Systematic Reviews were compared with those published in regular journals.


RESULTS
There were 36 reports of meta-analyses in the Cochrane Database of Systematic Reviews and 103 reports of meta-analyses published in regular journals; 11 of these were reports of Cochrane reviews. The meta-analyses published in the Cochrane Database of Systematic Reviews were more likely to fulfill most components of the OQAQ. The median overall OQAQ scores indicated significant methodological problems in the reports regardless of the source of publication, although the reports in the Cochrane database scored higher than those in regular journals (five compared with two, p<.001). Major methodological flaws, notably failure to appropriately refer to the validity of included studies, were found in meta-analyses in both the Cochrane Database of Systematic Reviews and regular journals (44.4% and 79.3%, respectively).


CONCLUSIONS
Although the quality of reports of meta-analyses published in the Cochrane Database of Systematic Reviews is superior to the quality of reports of meta-analyses published in regular journals, there is significant room for improvement. Clinicians should critically appraise all reports of meta-analyses before considering the results, regardless of the source of publication.
Delaney
Anthony
A
Department of Intensive Care Medicine, Royal North Shore Hospital, and Northern Clinical School, University of Sydney, St. Leonards, NSW, Australia.
Bagshaw
Sean M
SM
Ferland
Andre
A
Laupland
Kevin
K
Manns
Braden
B
Doig
Christopher
C
D003422:Critical Care; D015201:Meta-Analysis as Topic; D011786:Quality Control; D012107:Research Design; D012196:Review Literature as Topic; D011795:Surveys and Questionnaires