title: Measuring implementation strength: lessons from the evaluation of public health strategies in low- and middle-income settings.
abstract: Evaluation of strategies to ensure evidence-based, low-cost interventions reach those in need is critical. One approach is to measure the strength, or intensity, with which packages of interventions are delivered, in order to explore the association between implementation strength and public health gains. A recent systematic review suggested methodological guidance was needed. We described the approaches used in three examples of measures of implementation strength in evaluation. These addressed important public health topics with a substantial disease burden in low-and middle-income countries; they involved large-scale implementation; and featured evaluation designs without comparison areas. Strengths and weaknesses of the approaches were discussed. In the evaluation of Ethiopia's Health Extension Programme, implementation strength scoring for each kebele (ward) was based on aggregated data from interviews with mothers of children aged 12-23 months, reflecting their reports of contact with four elements of the programme. An evaluation of the Avahan HIV prevention programme in India used the cumulative amount of Avahan funding per HIV-infected person spent each year in each district. In these cases, a single measure was developed and the association with hypothesised programme outcomes presented. In the evaluation of the Affordable Medicines Facility-malaria, several implementation strength measures were developed based on the duration of activity of the programme and the level of implementation of supporting interventions. Measuring the strength of programme implementation and assessing its association with outcomes is a promising approach to strengthen pragmatic impact evaluation. Five key aspects of developing an implementation strength measure are to: (a) develop a logic model; (b) identify aspects of implementation to be assessed; (c) design and implement data collection from a range of data sources; (d) decide whether and how to combine data into a single measure; and, (e) plan whether and how to use the measure(s) in outcome analysis.
authors.affiliation: Faculty of Public Health and Policy, james.hargreaves@lshtm.ac.uk.
authors.forename: James R M
authors.initials: JR
authors.lastname: Hargreaves
authors.affiliation: Faculty of Public Health and Policy.
authors.forename: Catherine
authors.initials: C
authors.lastname: Goodman
authors.affiliation: Faculty of Public Health and Policy.
authors.forename: Calum
authors.initials: C
authors.lastname: Davey
authors.affiliation: Faculty of Epidemiology and Population Health and.
authors.forename: Barbara A
authors.initials: BA
authors.lastname: Willey
authors.affiliation: Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, Keppel Street, London, WC1E 7HT, UK.
authors.forename: Bilal Iqbal
authors.initials: BI
authors.lastname: Avan
authors.affiliation: Faculty of Infectious and Tropical Diseases, London School of Hygiene & Tropical Medicine, Keppel Street, London, WC1E 7HT, UK.
authors.forename: Joanna Rm Armstrong
authors.initials: JR
authors.lastname: Schellenberg
mesh_terms: D003695:Delivery of Health Care; D005002:Ethiopia; D019317:Evidence-Based Medicine / Q000379:methods; D015658:HIV Infections / Q000628:therapy; D006284:Health Plan Implementation; D006801:Humans; D007194:India; D007223:Infant; D008288:Malaria / Q000517:prevention & control; D015397:Program Evaluation; D011634:Public Health
keywords: Africa; Asia; Evaluation; HIV/AIDS; implementation strength; malaria; maternal; newborn and child health