title: Can AMSTAR also be applied to systematic reviews of non-randomized studies?
abstract: BACKGROUND
  There is a lack of an instrument to evaluate systematic reviews of non-randomized studies in epidemiological research. The Assessment of Multiple Systematic Reviews (AMSTAR) is widely used to evaluate the scientific quality of systematic reviews, but it has not been validated for SRs of non-randomized studies. The objective of this paper is to report our experience in applying AMSTAR to systematic reviews of non-randomized studies in terms of applicability, reliability and feasibility. Thus, we applied AMSTAR to a recently published review of 32 systematic reviews of non-randomized studies investigating the hospital volume-outcome relationship in surgery.
  
  
  RESULTS
  The inter-rater reliability was high (0.76), albeit items 8 (scientific quality used in formulating conclusions), 9 (appropriate method to combine studies), and 11 (conflicts of interest) scored moderate (≤0.58). However, there was a high heterogeneity between the two pairs of reviewers. In terms of feasibility, AMSTAR proved easy to apply to systematic reviews of non-randomized studies, each review taking 5-10 minutes to complete. We faced problems in applying three items, mainly related to scientific quality of the included studies.
  
  
  CONCLUSIONS
  AMSTAR showed good psychometric properties, comparable to prior findings in systematic reviews of randomized controlled trials. AMSTAR can be applied to systematic reviews of non-randomized studies, although there are some item specific issues users should be aware of. Revisions and extensions of AMSTAR might be helpful.
authors.affiliation: Institute for Research in Operative Medicine, Witten/Herdecke University, Ostmerheimer Str, 200, Building 38, D-51109 Cologne, Germany. dawid.pieper@uni-wh.de.
authors.forename: Dawid
authors.initials: D
authors.lastname: Pieper
authors.forename: Tim
authors.initials: T
authors.lastname: Mathes
authors.forename: Michaela
authors.initials: M
authors.lastname: Eikermann
mesh_terms: D002986:Clinical Trials as Topic; D005240:Feasibility Studies; D015588:Observer Variation; D011594:Psychometrics; D015203:Reproducibility of Results; D012196:Review Literature as Topic