title: Indicators for tracking programmes to strengthen health research capacity in lower- and middle-income countries: a qualitative synthesis.
abstract: BACKGROUND
  The monitoring and evaluation of health research capacity strengthening (health RCS) commonly involves documenting activities and outputs using indicators or metrics. We sought to catalogue the types of indicators being used to evaluate health RCS and to assess potential gaps in quality and coverage.
  
  
  METHODS
  We purposively selected twelve evaluations to maximize diversity in health RCS, funders, countries, and approaches to evaluation. We explored the quality of the indicators and extracted them into a matrix across individual, institutional, and national/regional/network levels, based on a matrix in the ESSENCE Planning, Monitoring and Evaluation framework. We synthesized across potential impact pathways (activities to outputs to outcomes) and iteratively checked our findings with key health RCS evaluation stakeholders.
  
  
  RESULTS
  Evaluations varied remarkably in the strengths of their evaluation designs. The validity of indicators and potential biases were documented in a minority of reports. Indicators were primarily of activities, outputs, or outcomes, with little on their inter-relationships. Individual level indicators tended to be more quantitative, comparable, and attentive to equity considerations. Institutional and national-international level indicators were extremely diverse. Although linkage of activities through outputs to outcomes within evaluations was limited, across the evaluations we were able to construct potential pathways of change and assemble corresponding indicators.
  
  
  CONCLUSIONS
  Opportunities for improving health RCS evaluations include work on indicator measurement properties and development of indicators which better encompass relationships with knowledge users. Greater attention to evaluation design, prospective indicator measurement, and systematic linkage of indicators in keeping with theories of change could provide more robust evidence on outcomes of health RCS.
authors.affiliation: Dalla Lana School of Public Health, University of Toronto, Toronto, ON M5T 3M7, Canada. donald.cole@utoronto.ca.
authors.forename: Donald C
authors.initials: DC
authors.lastname: Cole
authors.forename: Alan
authors.initials: A
authors.lastname: Boyd
authors.forename: Garry
authors.initials: G
authors.lastname: Aslanyan
authors.forename: Imelda
authors.initials: I
authors.lastname: Bates
mesh_terms: D057191:Capacity Building / Q000458:organization & administration; D003625:Data Collection / Q000592:standards; D003906:Developing Countries / Q000706:statistics & numerical data*; D006302:Health Services Research / Q000592:standards / Q000706:statistics & numerical data*; D006801:Humans; D015397:Program Evaluation; D011787:Quality of Health Care; D012107:Research Design / Q000592:standards; D012108:Research Personnel / Q000193:education / Q000592:standards; D015983:Selection Bias
references.citation: PLoS Med. 2012;9(4):e1001209
references.pmid: 22545025
references.citation: PLoS Med. 2010 Jul 06;7(7):e1000299
references.pmid: 20625547
references.citation: Int J Health Plann Manage. 2011 Jan-Mar;26(1):89-106
references.pmid: 20422620
references.citation: Health Res Policy Syst. 2010 Oct 06;8:29
references.pmid: 20925953
references.citation: PLoS Negl Trop Dis. 2011 Oct;5(10):e1351
references.pmid: 22022630
references.citation: J Contin Educ Health Prof. 2008 Spring;28(2):67-72
references.pmid: 18521874
references.citation: BMJ. 2000 Jan 8;320(7227):114-6
references.pmid: 10625273
references.citation: Health Res Policy Syst. 2013 Dec 14;11:46
references.pmid: 24330628
references.citation: Health Res Policy Syst. 2011 Jun 24;9:26
references.pmid: 21702930
references.citation: Health Res Policy Syst. 2014 Mar 03;12:11
references.pmid: 24581148
references.citation: BMC Health Serv Res. 2012 Aug 27;12:276
references.pmid: 22920443
references.citation: PLoS Med. 2011 Nov;8(11):e1001115
references.pmid: 22069378
references.citation: Health Res Policy Syst. 2012 Feb 09;10:5
references.pmid: 22321566
references.citation: Glob Public Health. 2013;8 Suppl 1:S104-24
references.pmid: 23651463
references.citation: BMC Health Serv Res. 2006 Oct 18;6:134
references.pmid: 17049092
references.citation: BMC Public Health. 2006 Jun 12;6:151
references.pmid: 16768792
references.citation: Health Res Policy Syst. 2013 Dec 17;11:47
references.pmid: 24341347
references.citation: Health Res Policy Syst. 2012 Jul 02;10:21
references.pmid: 22748169