title: [A reproducibility and validity study of a systematic review on ischemic cardiopathy. The Study Group of the Quality of Care (GRECA)].
abstract: BACKGROUND
  The systematic literature reviews have been proposed as a method of scientific evidence identification since they protect the final product from the subjectivity of each primary source reviewer. However, it is not known whether the different ways of evidence synthesis accomplish suitable criteria of objectivity, reliability and biases protection so as to be considered scientifically valid. An experiment of reliability and validity of a systematic literature review about coronary heart disease was carried out.
  
  
  METHODS
  Study of blind concordance between two independent reviewers for the identification, selection, retrieval and quality evaluation of the articles by using the same protocol. The concordance was analysed by the kappa index for two observers in different categories. The validity was evaluated throughout the acceptability of the review users.
  
  
  RESULTS
  The concordance for their identification capacity was poor although they used the same key words (869 versus 476). But the concordance improved when considering selection (26.6% versus 29.2%), retrieval (agreement = 76%) classification by kind of article (kappa = 0.60) and scoring by strength of the evidence (kappa = 0.87). The acceptability was high among review users.
  
  
  CONCLUSIONS
  It would be assumed that, even under tight rules of performance, the systematic literature reviews are not completely protected against some biases which could damage their validity in a non easily controllable form. The implication of reviewers, experts in documentation and users of the literature, together with pilot studies performed previous to the review, seems to be the best way to yield better results.
authors.affiliation: Servicio de Urgencias, Fundación Hospital de Alcorcón, Comunidad de Madrid.
authors.forename: E
authors.initials: E
authors.lastname: Gómez Gras
authors.forename: E
authors.initials: E
authors.lastname: de Villar Conde
authors.forename: J R
authors.initials: JR
authors.lastname: Lacalle Remigio
authors.forename: E B
authors.initials: EB
authors.lastname: Pérez de la Blanca
authors.forename: A
authors.initials: A
authors.lastname: Reyes Domínguez
authors.forename: R
authors.initials: R
authors.lastname: Alvarez Gil
authors.forename: M J
authors.initials: MJ
authors.lastname: Pérez Lozano
authors.forename: I
authors.initials: I
authors.lastname: Marín León
mesh_terms: D006801:Humans; D000787:Angina Pectoris; D001634:Bibliographies as Topic; D019317:Evidence-Based Medicine / Q000379:methods / Q000592:standards; D016239:MEDLINE; D017202:Myocardial Ischemia; D015588:Observer Variation; D011786:Quality Control; D011787:Quality of Health Care; D015203:Reproducibility of Results; D013030:Spain; D000078202:Systematic Reviews as Topic