title: The effect of performance indicator category on estimates of intervention effectiveness.
abstract: BACKGROUND
  A challenge for systematic reviews on improving health worker performance is that included studies often use different performance indicators, and the validity of comparing interventions with different indicators is unclear. One potential solution is to adjust comparisons by indicator category, with categories based on steps of the case-management process that can be easily recognized (assessment of symptoms, treatment etc.) and that might require different levels of effort to bring about improvements. However, this approach would only be useful if intervention effect sizes varied by indicator category. To explore this approach, studies were analyzed that evaluated the Integrated Management of Childhood Illness (IMCI) strategy.
  
  
  METHODS
  Performance indicators were grouped into four categories: patient assessment, diagnosis, treatment and counseling. An effect size of IMCI was calculated for each indicator. Linear regression modeling was used to test for differences among the mean effect sizes of the indicator categories.
  
  
  RESULTS
  Six studies were included, with data from 3136 ill child consultations. Mean effect sizes for 63 assessment indicators, 12 diagnosis indicators, 31 treatment indicators and 34 counseling indicators were 50.9 percentage-points (%-points), 44.7, 36.5 and 46.6%-points, respectively. After adjusting for baseline indicator value, compared with the assessment mean effect size, the diagnosis mean was 7.3%-points lower (P = 0.23), the treatment mean was 15.2%-points lower (P = 0.0004) and the counseling mean was 12.9%-points lower (P = 0.0027).
  
  
  CONCLUSIONS
  Adjusting the results of systematic reviews for indicator category and baseline indicator value might be useful for improving the validity of intervention comparisons.
authors.affiliation: Malaria Branch, Division of Parasitic Diseases and Malaria, Center for Global Health, Centers for Disease Control and Prevention, Atlanta, GA 30333, USA. axr9@cdc.gov
authors.forename: Alexander K
authors.initials: AK
authors.lastname: Rowe
mesh_terms: D002648:Child; D002662:Child Health Services / Q000379:methods / Q000592:standards; D006801:Humans; D010043:Outcome and Process Assessment, Health Care / Q000379:methods / Q000592:standards; D011785:Quality Assurance, Health Care / Q000379:methods / Q000458:organization & administration / Q000592:standards*; D019984:Quality Indicators, Health Care / Q000458:organization & administration / Q000592:standards*
keywords: child health; developing country; health services research; methods; quality improvement; systematic reviews