title: External validation of a measurement tool to assess systematic reviews (AMSTAR).
abstract: BACKGROUND
  Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews.
  
  
  METHODS
  AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently.
  
  
  CONCLUSIONS
  The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were -0.19+/-1.38. This translates to a minimum detectable difference between reviews of 0.64 'AMSTAR points'. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.
authors.affiliation: Community Information and Epidemiological Technologies, Ottawa, Ontario, Canada. bshea@ciet.org
authors.forename: Beverley J
authors.initials: BJ
authors.lastname: Shea
authors.forename: Lex M
authors.initials: LM
authors.lastname: Bouter
authors.forename: Joan
authors.initials: J
authors.lastname: Peterson
authors.forename: Maarten
authors.initials: M
authors.lastname: Boers
authors.forename: Neil
authors.initials: N
authors.lastname: Andersson
authors.forename: Zulma
authors.initials: Z
authors.lastname: Ortiz
authors.forename: Tim
authors.initials: T
authors.lastname: Ramsay
authors.forename: Annie
authors.initials: A
authors.lastname: Bai
authors.forename: Vijay K
authors.initials: VK
authors.lastname: Shukla
authors.forename: Jeremy M
authors.initials: JM
authors.lastname: Grimshaw
mesh_terms: D005744:Gastric Acid; D005764:Gastroesophageal Reflux / Q000628:therapy; D006801:Humans; D015201:Meta-Analysis as Topic; D010437:Peptic Ulcer / Q000628:therapy; D015203:Reproducibility of Results