title: Design of an impact evaluation using a mixed methods model--an explanatory assessment of the effects of results-based financing mechanisms on maternal healthcare services in Malawi.
abstract: BACKGROUND
  In this article we present a study design to evaluate the causal impact of providing supply-side performance-based financing incentives in combination with a demand-side cash transfer component on equitable access to and quality of maternal and neonatal healthcare services. This intervention is introduced to selected emergency obstetric care facilities and catchment area populations in four districts in Malawi. We here describe and discuss our study protocol with regard to the research aims, the local implementation context, and our rationale for selecting a mixed methods explanatory design with a quasi-experimental quantitative component.
  
  
  METHODS
  The quantitative research component consists of a controlled pre- and post-test design with multiple post-test measurements. This allows us to quantitatively measure 'equitable access to healthcare services' at the community level and 'healthcare quality' at the health facility level. Guided by a theoretical framework of causal relationships, we determined a number of input, process, and output indicators to evaluate both intended and unintended effects of the intervention. Overall causal impact estimates will result from a difference-in-difference analysis comparing selected indicators across intervention and control facilities/catchment populations over time.To further explain heterogeneity of quantitatively observed effects and to understand the experiential dimensions of financial incentives on clients and providers, we designed a qualitative component in line with the overall explanatory mixed methods approach. This component consists of in-depth interviews and focus group discussions with providers, service user, non-users, and policy stakeholders. In this explanatory design comprehensive understanding of expected and unexpected effects of the intervention on both access and quality will emerge through careful triangulation at two levels: across multiple quantitative elements and across quantitative and qualitative elements.
  
  
  CONCLUSIONS
  Combining a traditional quasi-experimental controlled pre- and post-test design with an explanatory mixed methods model permits an additional assessment of organizational and behavioral changes affecting complex processes. Through this impact evaluation approach, our design will not only create robust evidence measures for the outcome of interest, but also generate insights on how and why the investigated interventions produce certain intended and unintended effects and allows for a more in-depth evaluation approach.
authors.forename: Stephan
authors.initials: S
authors.lastname: Brenner
authors.forename: Adamson S
authors.initials: AS
authors.lastname: Muula
authors.forename: Paul Jacob
authors.initials: PJ
authors.lastname: Robyn
authors.forename: Till
authors.initials: T
authors.lastname: Bärnighausen
authors.forename: Malabika
authors.initials: M
authors.lastname: Sarker
authors.forename: Don P
authors.initials: DP
authors.lastname: Mathanga
authors.forename: Thomas
authors.initials: T
authors.lastname: Bossert
authors.affiliation: Institute of Public Health, Ruprecht-Karls-University, Heidelberg, Germany. manuela.de.allegri@urz.uni-heidelberg.de.
authors.forename: Manuela
authors.initials: M
authors.lastname: De Allegri
mesh_terms: D005069:Evaluation Studies as Topic; D005260:Female; D006801:Humans; D008295:Malawi; D008427:Maternal Health Services / Q000191:economics*; D012052:Reimbursement, Incentive
references.citation: Nurse Res. 2004;12(1):47-61
references.pmid: 15493214
references.citation: Health Aff (Millwood). 2011 Aug;30(8):1518-27
references.pmid: 21821568
references.citation: Trop Med Int Health. 2009 Jul;14(7):830-7
references.pmid: 19497081
references.citation: J Health Soc Behav. 1995 Mar;36(1):1-10
references.pmid: 7738325
references.citation: Cochrane Database Syst Rev. 2012 Feb 15;(2):CD007899
references.pmid: 22336833
references.citation: Annu Rev Psychol. 2009;60:607-29
references.pmid: 18616391
references.citation: Womens Health Issues. 2009 Jan-Feb;19(1):14-20
references.pmid: 19111783
references.citation: BMC Health Serv Res. 2011 Aug 24;11:203
references.pmid: 21864335
references.citation: Lancet. 2003 Jul 19;362(9379):233-41
references.pmid: 12885488
references.citation: J Can Acad Child Adolesc Psychiatry. 2006 Aug;15(3):126-34
references.pmid: 18392182
references.citation: JAMA. 2000 Apr 5;283(13):1715-22
references.pmid: 10755498
references.citation: BMC Res Notes. 2010 Jul 27;3:209
references.pmid: 20663202
references.citation: JAMA. 1988 Sep 23-30;260(12):1743-8
references.pmid: 3045356
references.citation: BMC Health Serv Res. 2013 Apr 25;13:149
references.pmid: 23617375
references.citation: Lancet. 2006 Oct 28;368(9546):1535-41
references.pmid: 17071287
references.citation: Prev Med. 2012 Nov;55 Suppl:S95-S105
references.pmid: 22178043
references.citation: Soc Sci Med. 2012 Jul;75(2):377-83
references.pmid: 22560799
references.citation: J Econ Perspect. 2006 Winter;20(1):91-116
references.pmid: 17162836
references.citation: Am J Community Psychol. 1999 Oct;27(5):711-31
references.pmid: 10676545
references.citation: Med Care. 1981 Dec;19(12 Suppl):4-27
references.pmid: 7339313
references.citation: Lancet. 2006 Sep 30;368(9542):1189-200
references.pmid: 17011946
references.citation: Health Res Policy Syst. 2011 Jan 17;9:4
references.pmid: 21241477
references.citation: BMJ. 2003 Oct 4;327(7418):785-9
references.pmid: 14525877
references.citation: BMJ. 2006 Feb 18;332(7538):413-6
references.pmid: 16484270
references.citation: Health Econ. 2010 Dec;19(12):1461-77
references.pmid: 19960481
references.citation: Qual Health Res. 2012 May;22(5):619-29
references.pmid: 22232296
references.citation: Lancet. 2011 Jan 1;377(9759):85-95
references.pmid: 20619886
references.citation: Lancet. 2011 Apr 23;377(9775):1421-8
references.pmid: 21515164
references.citation: BMJ. 1999 Jul 10;319(7202):94-8
references.pmid: 10398635
references.citation: BMC Public Health. 2013 Feb 01;13:92
references.pmid: 23368959
references.citation: Bull World Health Organ. 2011 Sep 1;89(9):695-8
references.pmid: 21897491
references.citation: Global Health. 2012 Sep 10;8:33
references.pmid: 22963317
references.citation: Trop Med Int Health. 2010 Feb;15(2):182-90
references.pmid: 19930141
references.citation: Bull World Health Organ. 2010 Jun;88(6):402
references.pmid: 20539847
references.citation: Health Econ. 2003 Aug;12(8):685-96
references.pmid: 12898665
references.citation: Bull World Health Organ. 2012 Aug 1;90(8):559-559A
references.pmid: 22893735
references.citation: Lancet. 2006 Apr 8;367(9517):1193-208
references.pmid: 16616562
references.citation: Bull World Health Organ. 2011 Feb 1;89(2):153-6
references.pmid: 21346927