title: Evaluating complex health financing interventions: using mixed methods to inform further implementation of a novel PBI intervention in rural Malawi.
abstract: Gaps remain in understanding how performance-based incentive (PBI) programs affect quality of care and service quantity, whether programs are cost effective and how programs could be tailored to meet client and provider needs while remaining operationally viable. In 2014, Malawi's Ministry of Health launched the Service Delivery Integration-PBI (SSDI-PBI) program. The program is unique in that no portion of performance bonuses are paid to individual health workers, and it shifts responsibility for infrastructure and equipment procurement from facility staff to implementing partners. This protocol outlines an approach that analyzes processes and outcomes, considers expected and unexpected consequences of the program and frames the program's outputs relative to its costs. Findings from this evaluation will inform the intended future scale-up of PBI in Malawi.
  
  
  
  This study employs a prospective controlled before-and-after triangulation design to assess effects of the PBI program by analyzing quantitative and qualitative data from intervention and control facilities. Guided by a theoretical framework, the evaluation consists of four main components: service provision, health worker motivation, implementation processes and costing. Quality and access outcomes are assessed along four dimensions: (1) structural elements (related to equipment, drugs, staff); (2) process elements (providers' compliance with standards); (3) outputs (service utilization); (4) experiential elements (experiences of service delivery). The costing component includes costs related to start-up, ongoing management, and the cost of incentives themselves. The cost analysis considers costs incurred within the Ministry of Health, funders, and the implementing agency. The evaluation relies on primary data (including interviews and surveys) and secondary data (including costing and health management information system data).
  
  
  
  Through the lens of a PBI program, we illustrate how complex interventions can be evaluated via not only primary, mixed-methods data collection, but also through a wealth of secondary data from program implementers (including monitoring, evaluation and financial data), and the health system (including service utilization and service readiness data). We also highlight the importance of crafting a theory and using theory to inform the nature of data collected. Finally, we highlight the need to be responsive to stakeholders in order to enhance a study's relevance.
authors.affiliation: Institute of Public Health, Faculty of Medicine, Heidelberg University, INF 324, 69120, Heidelberg, Germany. mcmahon@uni-heidelberg.de.
authors.forename: Shannon A
authors.initials: SA
authors.lastname: McMahon
authors.affiliation: Institute of Public Health, Faculty of Medicine, Heidelberg University, INF 324, 69120, Heidelberg, Germany.
authors.forename: Stephan
authors.initials: S
authors.lastname: Brenner
authors.affiliation: Institute of Public Health, Faculty of Medicine, Heidelberg University, INF 324, 69120, Heidelberg, Germany.
authors.forename: Julia
authors.initials: J
authors.lastname: Lohmann
authors.affiliation: Department of Public Health, University of Malawi, College of Medicine, Blantyre, Malawi.
authors.forename: Christopher
authors.initials: C
authors.lastname: Makwero
authors.affiliation: Department of Policy Analysis and Public Management, Centre for Research on Health and Social Care Management (CERGAS), Bocconi University, Milano, Italy.
authors.forename: Aleksandra
authors.initials: A
authors.lastname: Torbica
authors.affiliation: Department of Public Health, University of Malawi, College of Medicine, Blantyre, Malawi.
authors.forename: Don P
authors.initials: DP
authors.lastname: Mathanga
authors.affiliation: Department of Public Health, University of Malawi, College of Medicine, Blantyre, Malawi.
authors.forename: Adamson S
authors.initials: AS
authors.lastname: Muula
authors.affiliation: Institute of Public Health, Faculty of Medicine, Heidelberg University, INF 324, 69120, Heidelberg, Germany.
authors.forename: Manuela
authors.initials: M
authors.lastname: De Allegri
mesh_terms: D003362:Cost-Benefit Analysis; D006282:Health Personnel / Q000191:economics* / Q000523:psychology; D061889:Healthcare Financing; D006801:Humans; D008295:Malawi; D009042:Motivation; D011446:Prospective Studies; D011787:Quality of Health Care; D012052:Reimbursement, Incentive; D012107:Research Design; D019035:Rural Health Services / Q000191:economics*; D012456:Salaries and Fringe Benefits
keywords: Health financing; Malawi; Performance-based financing; Performance-based incentives; Process evaluation; Research protocol; Results based financing