Evaluating the quality of systematic reviews in the emergency medicine literature.
OBJECTIVE
The objective of this study was to examine the scientific quality of systematic reviews published in 5 leading emergency medicine journals.


METHODS
MEDLINE and EMBASE databases were electronically searched to identify published systematic reviews. Searches were only conducted in emergency medicine journals during the past 10 years; 4 of the journals were also hand searched. Potential reviews were assessed independently by 2 reviewers for inclusion. Data regarding methods were extracted from each review independently by 2 reviewers. All systematic reviews were retrieved and rated for quality by using the 10 questions from the overview quality assessment questionnaire.


RESULTS
Twenty-nine reviews were identified from more than 100 citations. The overall scientific quality of the systematic reviews was low (mean score, 2.7; 95% confidence interval 2.1 to 3.2; maximum possible score, 7.0). Selection and publication biases were rarely addressed in this collection of reviews. For example, the search strategies were only identified in 9 (31%) reviews, whereas independent study selection (6 [21%]) and quality assessment of included studies (9 [31%]) were infrequently performed. Overall, the majority of reviews had extensive flaws, and only 3 (10%) had minimal flaws.


CONCLUSIONS
The results of the study indicate that many of the systematic reviews published in the emergency medicine literature contain major flaws; reviews with poor methodology may limit the validity of reported results. Further efforts should be made to improve the design, reporting, and publication of systematic reviews in emergency medicine.
Kelly
K D
KD
Division of Emergency Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada.
Travers
A
A
Dorgan
M
M
Slater
L
L
Rowe
B H
BH
D004635:Emergency Medicine; D006801:Humans; D007256:Information Systems; D016239:MEDLINE; D010506:Periodicals as Topic; D011643:Publishing / Q000592:standards*; D011785:Quality Assurance, Health Care; D011786:Quality Control; D012196:Review Literature as Topic; D014481:United States