title: Comparison of two independent systematic reviews of trials of recombinant human bone morphogenetic protein-2 (rhBMP-2): the Yale Open Data Access Medtronic Project.
abstract: It is uncertain whether the replication of systematic reviews, particularly those with the same objectives and resources, would employ similar methods and/or arrive at identical findings. We compared the results and conclusions of two concurrent systematic reviews undertaken by two independent research teams provided with the same objectives, resources, and individual participant-level data.
  
  
  
  Two centers in the USA and UK were each provided with participant-level data on 17 multi-site clinical trials of recombinant human bone morphogenetic protein-2 (rhBMP-2). The teams were blinded to each other's methods and findings until after publication. We conducted a retrospective structured comparison of the results of the two systematic reviews. The main outcome measures included (1) trial inclusion criteria; (2) statistical methods; (3) summary efficacy and risk estimates; and (4) conclusions.
  
  
  
  The two research teams' meta-analyses inclusion criteria were broadly similar but differed slightly in trial inclusion and research methodology. They obtained similar results in summary estimates of most clinical outcomes and adverse events. Center A incorporated all trials into summary estimates of efficacy and harms, while Center B concentrated on analyses stratified by surgical approach. Center A found a statistically significant, but small, benefit whereas Center B reported no advantage. In the analysis of harms, neither showed an increased cancer risk at 48 months, although Center B reported a significant increase at 24 months. Conclusions reflected these differences in summary estimates of benefit balanced with small but potentially important risk of harm.
  
  
  
  Two independent groups given the same research objectives, data, resources, funding, and time produced broad general agreement but differed in several areas. These differences, the importance of which is debatable, indicate the value of the availability of data to allow for more than a single approach and a single interpretation of the data.
  
  
  
  PROSPERO CRD42012002040 and CRD42012001907 .
authors.affiliation: Department of Internal Medicine, Yale School of Medicine, 333 Cedar Street, New Haven, CT, 06510, USA.
authors.forename: Jeffrey
authors.initials: J
authors.lastname: Low
authors.affiliation: Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, 1 Church Street, Suite 200, New Haven, CT, 06510, USA.|Section of General Internal Medicine, Department of Internal Medicine, Yale School of Medicine, P.O. Box 208093, New Haven, CT, 06510, USA.|Robert Wood Johnson Foundation Clinical Scholars Program, P.O. Box 208088, New Haven, CT, 06510, USA.|Department of Health Policy and Management, Yale School of Public Health, P.O. Box 208034, New Haven, CT, 06520, USA.
authors.forename: Joseph S
authors.initials: JS
authors.lastname: Ross
authors.affiliation: Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, 1 Church Street, Suite 200, New Haven, CT, 06510, USA.
authors.forename: Jessica D
authors.initials: JD
authors.lastname: Ritchie
authors.affiliation: Section of General Internal Medicine, Department of Internal Medicine, Yale School of Medicine, P.O. Box 208093, New Haven, CT, 06510, USA.|Robert Wood Johnson Foundation Clinical Scholars Program, P.O. Box 208088, New Haven, CT, 06510, USA.
authors.forename: Cary P
authors.initials: CP
authors.lastname: Gross
authors.affiliation: Department of Primary Care Health Sciences, Oxford University, 33 St Ebbes Street, Oxford, OX1 1PU, UK.
authors.forename: Richard
authors.initials: R
authors.lastname: Lehman
authors.affiliation: Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, 1 Church Street, Suite 200, New Haven, CT, 06510, USA.|Department of Biostatistics, Yale School of Public Health, P.O. Box 208034, New Haven, CT, 06520, USA.
authors.forename: Haiqun
authors.initials: H
authors.lastname: Lin
authors.affiliation: Department of Public Health and Preventive Medicine, Pacific Northwest Evidence-based Practice Center, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Road, Portland, OR, 97239-3098, USA.
authors.forename: Rongwei
authors.initials: R
authors.lastname: Fu
authors.affiliation: Centre for Reviews and Dissemination, University of York, Heslington, York, YO10 5DD, UK.
authors.forename: Lesley A
authors.initials: LA
authors.lastname: Stewart
authors.affiliation: Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, 1 Church Street, Suite 200, New Haven, CT, 06510, USA. harlan.krumholz@yale.edu.|Robert Wood Johnson Foundation Clinical Scholars Program, P.O. Box 208088, New Haven, CT, 06510, USA. harlan.krumholz@yale.edu.|Department of Health Policy and Management, Yale School of Public Health, P.O. Box 208034, New Haven, CT, 06520, USA. harlan.krumholz@yale.edu.|Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, P.O. Box 208093, New Haven, CT, 06510, USA. harlan.krumholz@yale.edu.
authors.forename: Harlan M
authors.initials: HM
authors.lastname: Krumholz
mesh_terms: D055396:Bone Morphogenetic Protein 2 / Q000009:adverse effects / Q000627:therapeutic use*; D003627:Data Interpretation, Statistical; D006801:Humans; D015201:Meta-Analysis as Topic; D011994:Recombinant Proteins / Q000009:adverse effects / Q000627:therapeutic use; D012189:Retrospective Studies; D012196:Review Literature as Topic; D012307:Risk Factors; D016212:Transforming Growth Factor beta / Q000009:adverse effects / Q000627:therapeutic use*; D016896:Treatment Outcome
chemical_list: D055396:Bone Morphogenetic Protein 2; D011994:Recombinant Proteins; D016212:Transforming Growth Factor beta; C499469:recombinant human bone morphogenetic protein-2
keywords: Data interpretation; Data sharing; Meta-analysis; Systematic review