title: The quality of reports of critical care meta-analyses in the Cochrane Database of Systematic Reviews: an independent appraisal.
abstract: OBJECTIVE
  To independently appraise the methodological quality of a sample of reports of meta-analyses that address critical care topics in the Cochrane Database of Systematic Reviews compared with the quality of reports published in regular journals, using a validated assessment instrument, the Overview Quality Assessment Questionnaire (OQAQ).
  
  
  METHODS
  Studies were selected from a search of MEDLINE, EMBASE, and the Cochrane Database of Systematic Reviews from 1994 to 2003, using multiple search terms for critical care and sensitive filters to identify meta-analyses.
  
  
  METHODS
  Two authors independently selected meta-analyses that addressed topics pertinent to critical care medicine.
  
  
  METHODS
  Two authors independently extracted the data. The proportion of reports that met each component of the OQAQ was determined, as was the overall quality score. Meta-analyses published in the Cochrane Database of Systematic Reviews were compared with those published in regular journals.
  
  
  RESULTS
  There were 36 reports of meta-analyses in the Cochrane Database of Systematic Reviews and 103 reports of meta-analyses published in regular journals; 11 of these were reports of Cochrane reviews. The meta-analyses published in the Cochrane Database of Systematic Reviews were more likely to fulfill most components of the OQAQ. The median overall OQAQ scores indicated significant methodological problems in the reports regardless of the source of publication, although the reports in the Cochrane database scored higher than those in regular journals (five compared with two, p<.001). Major methodological flaws, notably failure to appropriately refer to the validity of included studies, were found in meta-analyses in both the Cochrane Database of Systematic Reviews and regular journals (44.4% and 79.3%, respectively).
  
  
  CONCLUSIONS
  Although the quality of reports of meta-analyses published in the Cochrane Database of Systematic Reviews is superior to the quality of reports of meta-analyses published in regular journals, there is significant room for improvement. Clinicians should critically appraise all reports of meta-analyses before considering the results, regardless of the source of publication.
authors.affiliation: Department of Intensive Care Medicine, Royal North Shore Hospital, and Northern Clinical School, University of Sydney, St. Leonards, NSW, Australia.
authors.forename: Anthony
authors.initials: A
authors.lastname: Delaney
authors.forename: Sean M
authors.initials: SM
authors.lastname: Bagshaw
authors.forename: Andre
authors.initials: A
authors.lastname: Ferland
authors.forename: Kevin
authors.initials: K
authors.lastname: Laupland
authors.forename: Braden
authors.initials: B
authors.lastname: Manns
authors.forename: Christopher
authors.initials: C
authors.lastname: Doig
mesh_terms: D003422:Critical Care; D015201:Meta-Analysis as Topic; D011786:Quality Control; D012107:Research Design; D012196:Review Literature as Topic; D011795:Surveys and Questionnaires