title: Methodological quality of systematic reviews in subfertility: a comparison of Cochrane and non-Cochrane systematic reviews in assisted reproductive technologies.
abstract: OBJECTIVE
  Are there differences in the methodological quality of Cochrane systematic reviews (CRs) and non-Cochrane systematic reviews (NCRs) of assisted reproductive technologies?
  
  
  CONCLUSIONS
  CRs on assisted reproduction are of higher methodological quality than similar reviews published in other journals.
  
  
  BACKGROUND
  The quality of systematic reviews varies.
  
  
  METHODS
  This was a cross-sectional study of 30 CR and 30 NCR systematic reviews that were randomly selected from the eligible reviews identified from a literature search for the years 2007-2011.
  
  
  METHODS
  We extracted data on the reporting and methodological characteristics of the included systematic reviews. We assessed the methodological quality of the reviews using the 11-domain Measurement Tool to Assess the Methodological Quality of Systematic Reviews (AMSTAR) tool and subsequently compared CR and NCR systematic reviews.
  
  
  RESULTS
  The AMSTAR quality assessment found that CRs were superior to NCRs. For 10 of 11 AMSTAR domains, the requirements were met in >50% of CRs, but only 4 of 11 domains showed requirements being met in >50% of NCRs. The strengths of CRs are the a priori study design, comprehensive literature search, explicit lists of included and excluded studies and assessments of internal validity. Significant failings in the CRs were found in duplicate study selection and data extraction (67% meeting requirements), assessment for publication bias (53% meeting requirements) and reporting of conflicts of interest (47% meeting requirements). NCRs were more likely to contain methodological weaknesses as the majority of the domains showed <40% of reviews meeting requirements, e.g. a priori study design (17%), duplicate study selection and data extraction (17%), assessment of study quality (27%), study quality in the formulation of conclusions (23%) and reporting of conflict of interests (10%).
  
  
  CONCLUSIONS
  The AMSTAR assessment can only judge what is reported by authors. Although two of the five authors are involved in the production of CRs, the risk of bias was reduced by not involving these authors in the assessment of the systematic review quality.
  
  
  CONCLUSIONS
  Not all systematic reviews are equal. The reader needs to consider the quality of the systematic review when they consider the results and the conclusions of a systematic review.
  
  
  BACKGROUND
  There are no conflicts with any commercial organization. Funding was provided for the students by the summer studentship programme of the Faculty of Medical and Health Sciences of the University of Auckland.
authors.affiliation: Faculty of Medical and Health Sciences,  The University of Auckland, Private Bag 92019, Auckland 1142,  New Zealand.
authors.forename: B
authors.initials: B
authors.lastname: Windsor
authors.forename: I
authors.initials: I
authors.lastname: Popovich
authors.forename: V
authors.initials: V
authors.lastname: Jordan
authors.forename: M
authors.initials: M
authors.lastname: Showell
authors.forename: B
authors.initials: B
authors.lastname: Shea
authors.forename: C
authors.initials: C
authors.lastname: Farquhar
mesh_terms: D003430:Cross-Sectional Studies; D005260:Female; D006801:Humans; D007246:Infertility; D008297:Male; D015201:Meta-Analysis as Topic; D011247:Pregnancy; D017594:Publication Bias; D011643:Publishing / Q000592:standards; D012107:Research Design / Q000592:standards; D012196:Review Literature as Topic