title: A tool to assess the quality of a meta-analysis.
abstract: BACKGROUND
  Because meta-analyses are increasingly prevalent and cited in the medical literature, it is important that tools are available to assess their methodological quality. When performing an empirical study of the quality of published meta-analyses, we found that existing tools did not place a strong emphasis on statistical and interpretational issues.
  
  
  METHODS
  We developed a quality-assessment tool using existing materials and expert judgment as a starting point, followed by multiple iterations of input from our working group, piloting, and discussion. After having used the tool for our empirical study, agreement for four key items in the tool was measured using weighted kappa coefficients.
  
  
  RESULTS
  Our tool contained 43 items divided into four key areas (data sources, analysis of individual studies, meta-analysis methods, and interpretation), and each area ended with a summary question. We also produced guidance for completing the tool. Agreement between raters was fair to moderate.
  
  
  CONCLUSIONS
  The tool should usefully inform subsequent initiatives to develop quality-assessment tools for meta-analysis. We advocate use of consensus between independent raters when assessing statistical appropriateness and adequacy of interpretation in meta-analyses.
authors.affiliation: School of Social and Community Medicine, University of Bristol, Bristol, UK.
authors.forename: Julian P T
authors.initials: JP
authors.lastname: Higgins
authors.affiliation: Statistical Consultancy Group; GlaxoSmithKline R&D, Stevenage, UK.
authors.forename: Peter W
authors.initials: PW
authors.lastname: Lane
authors.affiliation: Royal Free Hospital Medical Library, University College London, London, UK.
authors.forename: Betsy
authors.initials: B
authors.lastname: Anagnostelis
authors.affiliation: Biostatistics, Roche Products Ltd, Welwyn Garden City, UK.
authors.forename: Judith
authors.initials: J
authors.lastname: Anzures-Cabrera
authors.affiliation: Amgen Ltd, Cambridge, UK.
authors.forename: Nigel F
authors.initials: NF
authors.lastname: Baker
authors.affiliation: Biostatistics, Pfizer Inc, Groton, Connecticut, USA.
authors.forename: Joseph C
authors.initials: JC
authors.lastname: Cappelleri
authors.affiliation: Primary Care Business Unit, Pfizer Global R&D, Sandwich, UK.
authors.forename: Scott
authors.initials: S
authors.lastname: Haughie
authors.affiliation: Global Medicines Development, AstraZeneca, Macclesfield, UK.
authors.forename: Sally
authors.initials: S
authors.lastname: Hollis
authors.affiliation: Centre for Population Health Sciences, University of Edinburgh Medical School, Teviot Place, Edinburgh, EH8 9AG, UK.
authors.forename: Steff C
authors.initials: SC
authors.lastname: Lewis
authors.affiliation: Biometrics, Vifor Pharma, Glattbrugg, Switzerland.
authors.forename: Patrick
authors.initials: P
authors.lastname: Moneuse
authors.affiliation: Medical and Pharmaceutical Statistics Research Unit, Lancaster University, Lancaster, UK.
authors.forename: Anne
authors.initials: A
authors.lastname: Whitehead
mesh_terms: D015982:Bias; D056808:Biostatistics; D000068598:Data Accuracy; D006801:Humans; D015201:Meta-Analysis as Topic; D015233:Models, Statistical
keywords: bias; meta-analysis; quality; systematic reviews