title: Application of balanced scorecard in the evaluation of a complex health system intervention: 12 months post intervention findings from the BHOMA intervention: a cluster randomised trial in Zambia.
abstract: BACKGROUND
  In many low income countries, the delivery of quality health services is hampered by health system-wide barriers which are often interlinked, however empirical evidence on how to assess the level and scope of these barriers is scarce. A balanced scorecard is a tool that allows for wider analysis of domains that are deemed important in achieving the overall vision of the health system. We present the quantitative results of the 12 months follow-up study applying the balanced scorecard approach in the BHOMA intervention with the aim of demonstrating the utility of the balanced scorecard in evaluating multiple building blocks in a trial setting.
  
  
  METHODS
  The BHOMA is a cluster randomised trial that aims to strengthen the health system in three rural districts in Zambia. The intervention aims to improve clinical care quality by implementing practical tools that establish clear clinical care standards through intensive clinic implementations. This paper reports the findings of the follow-up health facility survey that was conducted after 12 months of intervention implementation. Comparisons were made between those facilities in the intervention and control sites. STATA version 12 was used for analysis.
  
  
  RESULTS
  The study found significant mean differences between intervention(I) and control (C) sites in the following domains: Training domain (Mean I:C; 87.5.vs 61.1, mean difference 23.3, p = 0.031), adult clinical observation domain (mean I:C; 73.3 vs.58.0, mean difference 10.9, p = 0.02 ) and health information domain (mean I:C; 63.6 vs.56.1, mean difference 6.8, p = 0.01. There was no gender differences in adult service satisfaction. Governance and motivation scores did not differ between control and intervention sites.
  
  
  CONCLUSIONS
  This study demonstrates the utility of the balanced scorecard in assessing multiple elements of the health system. Using system wide approaches and triangulating data collection methods seems to be key to successful evaluation of such complex health intervention.
  
  
  BACKGROUND
  ClinicalTrials.gov NCT01942278.
authors.affiliation: Department of Public Health, University of Zambia School of Medicine, Lusaka, Zambia; Clinical Research Department, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom.
authors.forename: Wilbroad
authors.initials: W
authors.lastname: Mutale
authors.affiliation: Clinical Research Department, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom.
authors.forename: Jeffrey
authors.initials: J
authors.lastname: Stringer
authors.affiliation: Department of Global Health and Development, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, United Kingdom.
authors.forename: Namwinga
authors.initials: N
authors.lastname: Chintu
authors.affiliation: Centre for Infectious Disease control in Zambia, Northmead Lusaka, Zambia.
authors.forename: Roma
authors.initials: R
authors.lastname: Chilengi
authors.affiliation: ZAMBART Project, Ridgeway Campus, University of Zambia, Lusaka, Zambia.
authors.forename: Margaret Tembo
authors.initials: MT
authors.lastname: Mwanamwenge
authors.affiliation: ZAMBART Project, Ridgeway Campus, University of Zambia, Lusaka, Zambia.
authors.forename: Nkatya
authors.initials: N
authors.lastname: Kasese
authors.affiliation: Department of Global Health and Development, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, United Kingdom.
authors.forename: Dina
authors.initials: D
authors.lastname: Balabanova
authors.affiliation: Department of Global Health and Development, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, United Kingdom.
authors.forename: Neil
authors.initials: N
authors.lastname: Spicer
authors.affiliation: Department of Infectious Disease Epidemiology, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, United Kingdom.
authors.forename: James
authors.initials: J
authors.lastname: Lewis
authors.affiliation: Clinical Research Department, Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, United Kingdom; ZAMBART Project, Ridgeway Campus, University of Zambia, Lusaka, Zambia.
authors.forename: Helen
authors.initials: H
authors.lastname: Ayles
mesh_terms: D000328:Adult; D016000:Cluster Analysis; D003710:Demography; D006268:Health Facilities / Q000592:standards; D006302:Health Services Research / Q000592:standards*; D006801:Humans; D016014:Linear Models; D011787:Quality of Health Care / Q000592:standards; D013997:Time Factors; D015024:Zambia
references.citation: Implement Sci. 2011 Dec 02;6:124
references.pmid: 22132875
references.citation: Reprod Health. 2012 Nov 12;9:27
references.pmid: 23140196
references.citation: BMC Health Serv Res. 2013;13 Suppl 2:S7
references.pmid: 23819614
references.citation: Health Policy. 2011 Dec;103(2-3):200-8
references.pmid: 21723641
references.citation: Harv Bus Rev. 1992 Jan-Feb;70(1):71-9
references.pmid: 10119714
references.citation: Cost Eff Resour Alloc. 2011 May 17;9(1):7
references.pmid: 21586111
references.citation: BMC Health Serv Res. 2013;13 Suppl 2:S1
references.pmid: 23819761
references.citation: PLoS One. 2013;8(3):e58650
references.pmid: 23555590
references.citation: Health Policy Plan. 2012 Oct;27 Suppl 4:iv9-19
references.pmid: 23014156
references.citation: J Med Pract Manage. 2011 May-Jun;26(6):331-5
references.pmid: 21815545
references.citation: J Healthc Manag. 2002 May-Jun;47(3):179-95; discussion 195-6
references.pmid: 12055900
references.citation: Rev Esp Cardiol (Engl Ed). 2012 Oct;65(10):919-27
references.pmid: 22917775
references.citation: Health Educ Behav. 2012 Dec;39(6):745-51
references.pmid: 22467637
references.citation: Health Policy. 2011 Dec;103(2-3):305-14
references.pmid: 21658787
references.citation: J Eval Clin Pract. 2016 Feb;22(1):112-121
references.pmid: 24814988
references.citation: Qual Manag Health Care. 2000 Summer;8(4):21-39
references.pmid: 11183582
references.citation: J Healthc Manag. 2005 Jul-Aug;50(4):251-62; discussion 262-3
references.pmid: 16130808
references.citation: Health Policy Plan. 2012 Oct;27 Suppl 4:iv20-31
references.pmid: 23014150
references.citation: Bull World Health Organ. 2011 Dec 1;89(12):907-12
references.pmid: 22271948
references.citation: Health Policy Plan. 2012 Oct;27 Suppl 4:iv1-3
references.pmid: 23014149
references.citation: Health Policy Plan. 2012 Aug;27(5):365-73
references.pmid: 21821667
references.citation: PLoS Med. 2010 Jan;7(1):e1000089
references.pmid: 20069038
references.citation: Malar J. 2010 Sep 18;9:261
references.pmid: 20849650
references.citation: PLoS Med. 2011 Jul;8(7):e1001066
references.pmid: 21814499
references.citation: Am J Epidemiol. 1975 May;101(5):391-9
references.pmid: 1130402
references.citation: Pain. 2013 Mar;154(3):468-475
references.pmid: 23375161
references.citation: Int J Health Plann Manage. 2013 Apr-Jun;28(2):202-15
references.pmid: 22887590
references.citation: Health Policy Plan. 2012 Oct;27 Suppl 4:iv4-8
references.pmid: 23014152
references.citation: Health Policy Plan. 2012 Oct;27 Suppl 4:iv32-43
references.pmid: 23014151
references.citation: Health Policy Plan. 2012 Oct;27 Suppl 4:iv54-61
references.pmid: 23014154
references.citation: Trials. 2013 Jan 12;14:15
references.pmid: 23311722
references.citation: Nurs Leadersh (Tor Ont). 2011 Apr;24(1):47-58
references.pmid: 21512337
references.citation: Implement Sci. 2013 Mar 28;8:39
references.pmid: 23537192