title: Evaluating meta-analyses in the general surgical literature: a critical appraisal.
abstract: OBJECTIVE
  To assess the methodologic quality of meta-analyses of general surgery topics published in peer-reviewed journals.
  
  
  BACKGROUND
  Systematic reviews and meta-analysis are used to seek, summarize, and interpret primary studies on a given topic. Accordingly, systematic reviews and meta-analyses of high-quality primary studies may be the highest level of evidence for issues of prevention and treatment in evidence-based medicine. However, not all published meta-analyses are rigorously performed.
  
  
  METHODS
  We searched MEDLINE (from January 1, 1997, to September 1, 2002) and reference lists and solicited general surgery specialists to identify relevant meta-analyses. Inclusion criteria were use of meta-analytic methods to pool the results of primary studies in general surgery on issues of diagnosis, causation, prognosis, or treatment. Our search strategies identified 487 potentially relevant articles. After excluding articles based on a priori criteria, 51 meta-analyses fulfilled eligibility criteria. In duplicate and independently, 2 reviewers assessed the quality of these meta-analyses using a 10-item index called the Overview Quality Assessment Questionnaire.
  
  
  RESULTS
  Overall concordance between 2 independent reviewers was good (interobserver agreement 81%, and a kappa of 0.62 (95% CI 0.55-0.69). Of 51 relevant articles, 38 were published in surgical journals. Most studies had major methodologic flaws (median score of 3.3, scale of 1-7). Factors associated with low overall scientific quality included the absence of any prior meta-analyses publications by authors and meta-analyses produced by surgical department members without external collaboration.
  
  
  CONCLUSIONS
  This critical appraisal of meta-analyses published in the general surgery literature demonstrates frequent methodologic flaws. The quality of these reports limits the validity of the findings and the inferences that can be made about the primary studies reviewed. To improve the quality of future meta-analyses, we recommend following guidelines for the optimal conduct and reporting of meta-analyses in general surgery.
authors.affiliation: Department of Surgery, University of Calgary, Calgary, Alberta, Canada.
authors.forename: Elijah
authors.initials: E
authors.lastname: Dixon
authors.forename: Morad
authors.initials: M
authors.lastname: Hameed
authors.forename: Francis
authors.initials: F
authors.lastname: Sutherland
authors.forename: Deborah J
authors.initials: DJ
authors.lastname: Cook
authors.forename: Christopher
authors.initials: C
authors.lastname: Doig
mesh_terms: D013502:General Surgery; D015201:Meta-Analysis as Topic; D011643:Publishing / Q000592:standards; D012196:Review Literature as Topic
references.citation: J Gastrointest Surg. 1999 May-Jun;3(3):225-32
references.pmid: 10481115
references.citation: Ann Thorac Surg. 2001 Jul;72(1):306-13
references.pmid: 11465217
references.citation: Lancet. 1998 Aug 22;352(9128):609-13
references.pmid: 9746022
references.citation: Ann Emerg Med. 2001 Nov;38(5):518-26
references.pmid: 11679863
references.citation: Semin Liver Dis. 1999;19(4):457-73
references.pmid: 10643629
references.citation: Drug Saf. 2000 Nov;23(5):449-61
references.pmid: 11085349
references.citation: JAMA. 1990 Mar 9;263(10):1392-5
references.pmid: 2406473
references.citation: Lancet. 1995 Mar 25;345(8952):772-6
references.pmid: 7891492
references.citation: J Clin Epidemiol. 1991;44(1):91-8
references.pmid: 1824710
references.citation: Arch Surg. 2000 Aug;135(8):948-58
references.pmid: 10922258
references.citation: Br J Surg. 1998 Feb;85(2):171-8
references.pmid: 9501809
references.citation: Am Surg. 2001 May;67(5):421-6
references.pmid: 11379640
references.citation: Onkologie. 2000 Dec;23(6):597-602
references.pmid: 11441269
references.citation: Br J Cancer. 2000 Mar;82(5):1017-21
references.pmid: 10737383
references.citation: Aust N Z J Surg. 1997 Nov;67(11):750-4
references.pmid: 9396988
references.citation: Br J Surg. 2002 Jun;89(6):695-703
references.pmid: 12027978
references.citation: Br J Surg. 1997 Jun;84(6):855-7
references.pmid: 9189108
references.citation: Dis Colon Rectum. 1998 Feb;41(2):180-9
references.pmid: 9556242
references.citation: Dis Esophagus. 2001;14(3-4):212-7
references.pmid: 11869322
references.citation: J Clin Epidemiol. 1991;44(11):1271-8
references.pmid: 1834807
references.citation: Can J Surg. 2002 Jun;45(3):173-80
references.pmid: 12067168
references.citation: Dis Colon Rectum. 1998 Jul;41(7):875-82; discussion 882-3
references.pmid: 9678373
references.citation: JAMA. 1995 Feb 1;273(5):408-12
references.pmid: 7823387
references.citation: Surg Endosc. 1999 Jul;13(7):689-94
references.pmid: 10384076
references.citation: JAMA. 2002 Jun 12;287(22):2973-82
references.pmid: 12052127
references.citation: Br J Surg. 2001 Jul;88(7):913-30
references.pmid: 11442521
references.citation: Am J Surg. 2002 Jul;184(1):6-10
references.pmid: 12135710
references.citation: Control Clin Trials. 1997 Dec;18(6):568-79; discussion 661-6
references.pmid: 9408719
references.citation: Br J Surg. 1997 Jun;84(6):750-9
references.pmid: 9189079
references.citation: JAMA. 1998 Apr 8;279(14):1089-93
references.pmid: 9546568
references.citation: BMJ. 2002 Apr 6;324(7341):813
references.pmid: 11934773
references.citation: Int J Technol Assess Health Care. 1996 Spring;12(2):195-208
references.pmid: 8707495
references.citation: Ann Surg. 2002 Jul;236(1):17-27
references.pmid: 12131081
references.citation: Ann R Coll Surg Engl. 1998;80 Suppl 1:S1-80
references.pmid: 11432408
references.citation: Dis Colon Rectum. 1999 Nov;42(11):1424-8; discussion 1428-31
references.pmid: 10566530
references.citation: Can J Surg. 1999 Oct;42(5):377-83
references.pmid: 10526524
references.citation: J Reprod Med. 1999 Apr;44(4):325-31
references.pmid: 10319300
references.citation: Surg Endosc. 2002 Jul;16(7):1098-104
references.pmid: 12165830
references.citation: Lancet. 2000 Feb 26;355(9205):756-7
references.pmid: 10703836
references.citation: Am J Gastroenterol. 1997 Jan;92(1):109-13
references.pmid: 8995948
references.citation: Am J Surg. 1998 Dec;176(6):666-70
references.pmid: 9926810
references.citation: Cancer. 1999 Feb 1;85(3):616-28
references.pmid: 10091735
references.citation: BMJ. 2001 Oct 6;323(7316):773-6
references.pmid: 11588077
references.citation: Obes Surg. 2000 Oct;10(5):391-401
references.pmid: 11054242
references.citation: Stat Med. 2002 Jun 15;21(11):1513-24
references.pmid: 12111917
references.citation: Arch Surg. 2002 Apr;137(4):458-61
references.pmid: 11926952
references.citation: J Clin Epidemiol. 2001 Jul;54(7):651-4
references.pmid: 11438404
references.citation: J Hepatol. 2001 Feb;34(2):342-5
references.pmid: 11281568
references.citation: Surg Laparosc Endosc. 1999 Jan;9(1):17-26
references.pmid: 9950122
references.citation: Dis Colon Rectum. 1998 Sep;41(9):1116-26
references.pmid: 9749495
references.citation: JAMA. 1995 Dec 13;274(22):1800-4
references.pmid: 7500513
references.citation: Am Surg. 1998 Nov;64(11):1050-8
references.pmid: 9798767
references.citation: J Surg Res. 2000 May 15;90(2):161-5
references.pmid: 10792958
references.citation: J Clin Epidemiol. 2000 Dec;53(12):1193-9
references.pmid: 11146264
references.citation: Surgery. 2000 Feb;127(2):136-41
references.pmid: 10686977
references.citation: Haemostasis. 1997 Mar-Apr;27(2):65-74
references.pmid: 9212354
references.citation: Ann Intern Med. 2001 Dec 4;135(11):982-9
references.pmid: 11730399
references.citation: BMJ. 1997 Sep 13;315(7109):629-34
references.pmid: 9310563
references.citation: Surgery. 2000 Dec;128(6):1075-81
references.pmid: 11114645
references.citation: Am J Surg. 2001 Nov;182(5):470-5
references.pmid: 11754853
references.citation: Cancer J Sci Am. 1997 Jan-Feb;3(1):6-12
references.pmid: 9072310
references.citation: Ann Surg. 2002 Mar;235(3):322-32
references.pmid: 11882753
references.citation: Am J Surg. 2000 Jul;180(1):58-61
references.pmid: 11036143
references.citation: Ann Surg. 1999 Apr;229(4):467-77
references.pmid: 10203078
references.citation: Anesth Analg. 2001 Mar;92(3):700-9
references.pmid: 11226105
references.citation: JAMA. 1998 Jul 15;280(3):278-80
references.pmid: 9676681
references.citation: Liver Transpl. 2000 May;6(3):302-8
references.pmid: 10827230
references.citation: Ann Surg. 2001 Nov;234(5):590-606
references.pmid: 11685021
references.citation: N Engl J Med. 1987 Feb 19;316(8):450-5
references.pmid: 3807986
references.citation: Ann Surg. 2000 Mar;231(3):436-42
references.pmid: 10714638
references.citation: Ann Surg Oncol. 1999 Jan-Feb;6(1):109-16
references.pmid: 10030423
references.citation: Ann Surg. 1999 Feb;229(2):174-80
references.pmid: 10024097
references.citation: J Clin Gastroenterol. 1997 Dec;25(4):628-33
references.pmid: 9451677
references.citation: Dis Colon Rectum. 2000 Jul;43(7):911-9
references.pmid: 10910235
references.citation: N Engl J Med. 1997 Aug 21;337(8):536-42
references.pmid: 9262498