title: The impact of Cochrane Reviews: a mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research.
abstract: BACKGROUND
  The last few decades have seen a growing emphasis on evidence-informed decision-making in health care. Systematic reviews, such as those produced by Cochrane, have been a key component of this movement. The National Institute for Health Research (NIHR) Systematic Review Programme currently supports 20 Cochrane Review Groups (CRGs) in the UK and it is important that this funding represents value for money.
  
  
  OBJECTIVE
  The overall aim was to identify the impacts and likely impacts on health care, patient outcomes and value for money of Cochrane Reviews published by 20 NIHR-funded CRGs during the years 2007-11.
  
  
  METHODS
  We sent questionnaires to CRGs and review authors, undertook interviews with guideline developers (GDs) and used bibliometrics and documentary review to get an overview of CRG impact and to evaluate the impact of a sample of 60 Cochrane Reviews. The evaluation was guided by a framework with four categories (knowledge production, research targeting, informing policy development and impact on practice/services).
  
  
  RESULTS
  A total of 3187 new and updated reviews were published on the Cochrane Database of Systematic Reviews between 2007 and 2011, 1502 (47%) of which were produced by the 20 CRGs funded by the NIHR. We found 40 examples where reviews appeared to have influenced primary research and reviews had contributed to the creation of new knowledge and stimulated debate. Twenty-seven of the 60 reviews had 100 or more citations in Google Scholar™ (Google, CA, USA). Overall, 483 systematic reviews had been cited in 247 sets of guidance. This included 62 sets of international guidance, 175 sets of national guidance (87 from the UK) and 10 examples of local guidance. Evidence from the interviews suggested that Cochrane Reviews often play an instrumental role in informing guidance, although reviews being a poor fit with guideline scope or methods, reviews being out of date and a lack of communication between CRGs and GDs were barriers to their use. Cochrane Reviews appeared to have led to a number of benefits to the health service including safer or more appropriate use of medication or other health technologies or the identification of new effective drugs or treatments. However, whether or not these changes were directly as a result of the Cochrane Review and not the result of subsequent clinical guidance was difficult to judge. Potential benefits of Cochrane Reviews included economic benefits through budget savings or the release of funds, improvements in clinical quality, the reduction in the use of unproven or unnecessary procedures and improvements in patient and carer experiences.
  
  
  CONCLUSIONS
  This study identified a number of impacts and likely impacts of Cochrane Reviews. The clearest impacts of Cochrane Reviews are on research targeting and health-care policy, with less evidence of a direct impact on clinical practice and the organisation and delivery of NHS services. Although it is important for researchers to consider how they might increase the influence of their work, such impacts are difficult to measure. More work is required to develop suitable methods for defining and quantifying the impact of research.
  
  
  BACKGROUND
  The NIHR Health Technology Assessment programme.
authors: Centre for Research in Primary and Community Care, University of Hertfordshire, Hatfield, Hertfordshire, UK.
authors: Frances
authors: F
authors: Bunn
authors: Centre for Research in Primary and Community Care, University of Hertfordshire, Hatfield, Hertfordshire, UK.
authors: Daksha
authors: D
authors: Trivedi
authors: Centre for Clinical Practice, National Institute for Health and Care Excellence, Manchester, UK.
authors: Phil
authors: P
authors: Alderson
authors: Centre for Research in Primary and Community Care, University of Hertfordshire, Hatfield, Hertfordshire, UK.
authors: Laura
authors: L
authors: Hamilton
authors: Centre for Research in Primary and Community Care, University of Hertfordshire, Hatfield, Hertfordshire, UK.
authors: Alice
authors: A
authors: Martin
authors: Centre for Research in Primary and Community Care, University of Hertfordshire, Hatfield, Hertfordshire, UK.
authors: Emma
authors: E
authors: Pinkney
authors: Research Department of Primary Care and Population Health, UCL Medical School, London, UK.
authors: Steve
authors: S
authors: Iliffe
mesh_terms: D003245:Consensus Statements as Topic; D003362:Cost-Benefit Analysis; D003657:Decision Making; D019317:Evidence-Based Medicine / Q000191:economics / Q000379:methods / Q000592:standards*; D062489:Health Impact Assessment / Q000191:economics / Q000379:methods / Q000592:standards*; D006801:Humans; D007407:Interviews as Topic; D055811:Journal Impact Factor; D011050:Policy Making; D017410:Practice Guidelines as Topic / Q000592:standards*; D012196:Review Literature as Topic; D011795:Surveys and Questionnaires; D006113:United Kingdom