title: What to expect when you're evaluating healthcare improvement: a concordat approach to managing collaboration and uncomfortable realities.
abstract: Evaluation of improvement initiatives in healthcare is essential to establishing whether interventions are effective and to understanding how and why they work in order to enable replication. Although valuable, evaluation is often complicated by tensions and friction between evaluators, implementers and other stakeholders. Drawing on the literature, we suggest that these tensions can arise from a lack of shared understanding of the goals of the evaluation; confusion about roles, relationships and responsibilities; data burdens; issues of data flows and confidentiality; the discomforts of being studied and the impact of disappointing or otherwise unwelcome results. We present a possible approach to managing these tensions involving the co-production and use of a concordat. We describe how we developed a concordat in the context of an evaluation of a complex patient safety improvement programme known as Safer Clinical Systems Phase 2. The concordat development process involved partners (evaluators, designers, funders and others) working together at the outset of the project to agree a set of principles to guide the conduct of the evaluation. We suggest that while the concordat is a useful resource for resolving conflicts that arise during evaluation, the process of producing it is perhaps even more important, helping to make explicit unspoken assumptions, clarify roles and responsibilities, build trust and establish open dialogue and shared understanding. The concordat we developed established some core principles that may be of value for others involved in evaluation to consider. But rather than seeing our document as a ready-made solution, there is a need for recognition of the value of the process of co-producing a locally agreed concordat in enabling partners in the evaluation to work together effectively.
authors.affiliation: Department of Health Sciences, University of Leicester, Leicester, UK.
authors.forename: Liz
authors.identifier: http://orcid.org/0000-0003-3604-2897
authors.initials: L
authors.lastname: Brewster
authors.affiliation: Department of Health Sciences, University of Leicester, Leicester, UK.
authors.forename: Emma-Louise
authors.initials: EL
authors.lastname: Aveling
authors.affiliation: Department of Health Sciences, University of Leicester, Leicester, UK.
authors.forename: Graham
authors.initials: G
authors.lastname: Martin
authors.affiliation: Department of Health Sciences, University of Leicester, Leicester, UK.
authors.forename: Carolyn
authors.initials: C
authors.lastname: Tarrant
authors.affiliation: Department of Health Sciences, University of Leicester, Leicester, UK.
authors.forename: Mary
authors.initials: M
authors.lastname: Dixon-Woods
mesh_terms: D003299:Cooperative Behavior; D006801:Humans; D007400:Interprofessional Relations; D010043:Outcome and Process Assessment, Health Care / Q000458:organization & administration*; D061214:Patient Safety; D024382:Professional Role; D058996:Quality Improvement / Q000458:organization & administration*
keywords: Evaluation methodology; Patient safety; Quality improvement; Quality improvement methodologies
references.citation: Ergonomics. 2000 Jul;43(7):960-74
references.pmid: 10929830
references.citation: Milbank Q. 2011 Jun;89(2):167-205
references.pmid: 21676020
references.citation: Hastings Cent Rep. 2011 Jul-Aug;41(4):3
references.pmid: 21845906
references.citation: Implement Sci. 2013;8:70
references.pmid: 23786847
references.citation: J Health Psychol. 2014 Jan;19(1):34-45
references.pmid: 24195915
references.citation: BMJ. 2014;348:g1687
references.pmid: 24609605
references.citation: BMJ Qual Saf. 2015 Mar;24(3):228-38
references.pmid: 25616279