title: How hard can it be to include research evidence and evaluation in local health policy implementation? Results from a mixed methods study.
abstract: BACKGROUND
  Although an evidence-based approach is the ideal model for planning and delivering healthcare, barriers exist to using research evidence to implement and evaluate service change. This paper aims to inform policy implementation and evaluation by understanding the role of research evidence at the local level through implementation of a national chronic conditions management policy.
  
  
  METHODS
  We conducted a national email survey of health service commissioners at the most devolved level of decision-making in Wales (Local Health Boards - LHBs) followed by in-depth interviews with representatives of LHBs, purposively selecting five to reflect geographic and economic characteristics. Survey data were analysed descriptively; we used thematic analysis for interview data.
  
  
  RESULTS
  All LHBs (n = 22) completed questionnaires. All reported they routinely assessed the research literature before implementing interventions, but free-text answers revealed wide variation in approach. Most commonly reported information sources included personal contacts, needs assessments, information or research databases. No consistent approach to evaluation was reported. Frequently reported challenges were: insufficient staff capacity (17/22); limited skills, cost, limited time, competing priorities (16/22); availability and quality of routine data (15/22). Respondents reported they would value central guidance on evaluation.Five interviews were held with managers from the five LHBs contacted. Service delivery decisions were informed by Welsh Government initiatives and priorities, budgets, perceived good practice, personal knowledge, and local needs, but did not include formal research evidence, they reported. Decision making was a collaborative process including clinical staff, patient representatives, and partner organization managers with varying levels of research experience. Robust evaluation data were required, but they were constrained by a lack of skills, time, and resources. They viewed evaluation as a means of demonstrating that targets had been met.
  
  
  CONCLUSIONS
  There is a gap between evidence-based aims of national health policy and how health services are commissioned, implemented, and evaluated at local level. Commissioners and managers are unable to routinely incorporate research evidence. If health services research is to identify most effective ways to implement high quality care, it should be incorporated into commissioning and service delivery. Local commissioners and managers need to build the critical use of research evidence and evaluation into health policy implementation at local level in order to provide consistent and effective healthcare services.
authors: CHIRAL, ILS2, College of Medicine, Swansea University, Swansea SA2 8PP, UK. b.a.evans@swansea.ac.uk
authors: Bridie Angela
authors: BA
authors: Evans
authors: Helen
authors: H
authors: Snooks
authors: Helen
authors: H
authors: Howson
authors: Myfanwy
authors: M
authors: Davies
mesh_terms: D002908:Chronic Disease / Q000628:therapy*; D003657:Decision Making; D003695:Delivery of Health Care / Q000458:organization & administration; D004059:Diffusion of Innovation; D019317:Evidence-Based Medicine; D006291:Health Policy; D006302:Health Services Research / Q000706:statistics & numerical data*; D006801:Humans; D015397:Program Evaluation; D011795:Surveys and Questionnaires; D014852:Wales
references: BMJ. 2001 Aug 4;323(7307):275-9
references: 11485961
references: J R Soc Med. 2003 Sep;96(9):429-31
references: 12949196
references: BMC Public Health. 2001;1:1
references: 11208260
references: BMJ. 1994 May 28;308(6941):1426-9
references: 8019258
references: BMJ. 2009 Feb 17;338:b49
references: 19223339
references: J Health Serv Res Policy. 2010 Apr;15 Suppl 2:3-11
references: 20147422
references: J Health Serv Res Policy. 2008 Apr;13 Suppl 2:26-31
references: 18416926
references: Implement Sci. 2009 Apr 07;4:18
references: 19351400
references: Ann Intern Med. 2005 Jun 21;142(12 Pt 2):1120-5
references: 15968037
references: Implement Sci. 2012 Jan 16;7:3
references: 22248385
references: J Health Serv Res Policy. 2009 Jul;14(3):129-30
references: 19541869
references: J Health Serv Res Policy. 2005 Jul;10 Suppl 1:1-5
references: 16053579
references: Implement Sci. 2011 Nov 19;6:122
references: 22098945
references: J Health Serv Res Policy. 2002 Oct;7(4):239-44
references: 12425783
references: BMJ. 1995 Jan 14;310(6972):71-2
references: 7833721
references: Health Aff (Millwood). 2000 May-Jun;19(3):236-40
references: 10812803
references: Lancet. 2009 May 30;373(9678):1829-31
references: 19446328
references: J Health Serv Res Policy. 2010 Apr;15 Suppl 2:26-34
references: 20176663
references: Milbank Q. 2003;81(2):221-48, 171-2
references: 12841049
references: BMJ. 2001 Jan 27;322(7280):222-5
references: 11159625
references: BMJ. 2006 Jan 14;332(7533):109-12
references: 16410590
references: Implement Sci. 2012 Jan 11;7:2
references: 22236561
references: J Health Serv Res Policy. 2003 Oct;8 Suppl 2:26-34
references: 14596745
references: J Health Serv Res Policy. 2010 Apr;15 Suppl 2:1-2
references: 20354113
references: BMC Health Serv Res. 2008 Dec 04;8:248
references: 19055843
references: J Epidemiol Community Health. 2003 Jul;57(7):527-9
references: 12821702
references: BMJ. 2004 Oct 30;329(7473):988-9
references: 15514318
references: Health Serv Res. 2009 Aug;44(4):1111-36
references: 19490163