title: The practice of 'doing' evaluation: lessons learned from nine complex intervention trials in action.
abstract: BACKGROUND
  There is increasing recognition among trialists of the challenges in understanding how particular 'real-life' contexts influence the delivery and receipt of complex health interventions. Evaluations of interventions to change health worker and/or patient behaviours in health service settings exemplify these challenges. When interpreting evaluation data, deviation from intended intervention implementation is accounted for through process evaluations of fidelity, reach, and intensity. However, no such systematic approach has been proposed to account for the way evaluation activities may deviate in practice from assumptions made when data are interpreted.
  
  
  METHODS
  A collective case study was conducted to explore experiences of undertaking evaluation activities in the real-life contexts of nine complex intervention trials seeking to improve appropriate diagnosis and treatment of malaria in varied health service settings. Multiple sources of data were used, including in-depth interviews with investigators, participant-observation of studies, and rounds of discussion and reflection.
  
  
  CONCLUSIONS
  From our experiences of the realities of conducting these evaluations, we identified six key 'lessons learned' about ways to become aware of and manage aspects of the fabric of trials involving the interface of researchers, fieldworkers, participants and data collection tools that may affect the intended production of data and interpretation of findings. These lessons included: foster a shared understanding across the study team of how individual practices contribute to the study goals; promote and facilitate within-team communications for ongoing reflection on the progress of the evaluation; establish processes for ongoing collaboration and dialogue between sub-study teams; the importance of a field research coordinator bridging everyday project management with scientific oversight; collect and review reflective field notes on the progress of the evaluation to aid interpretation of outcomes; and these approaches should help the identification of and reflection on possible overlaps between the evaluation and intervention.
  
  
  CONCLUSIONS
  The lessons we have drawn point to the principle of reflexivity that, we argue, needs to become part of standard practice in the conduct of evaluations of complex interventions to promote more meaningful interpretations of the effects of an intervention and to better inform future implementation and decision-making.
authors.affiliation: Department of Social and Environmental Health Research, London School of Hygiene & Tropical Medicine, 15-17 Tavistock Place, London WC1H 9SH, UK. joanna.reynolds@lshtm.ac.uk.
authors.forename: Joanna
authors.initials: J
authors.lastname: Reynolds
authors.forename: Deborah
authors.initials: D
authors.lastname: DiLiberto
authors.forename: Lindsay
authors.initials: L
authors.lastname: Mangham-Jefferies
authors.forename: Evelyn K
authors.initials: EK
authors.lastname: Ansah
authors.forename: Sham
authors.initials: S
authors.lastname: Lal
authors.forename: Hilda
authors.initials: H
authors.lastname: Mbakilwa
authors.forename: Katia
authors.initials: K
authors.lastname: Bruxvoort
authors.forename: Jayne
authors.initials: J
authors.lastname: Webster
authors.forename: Lasse S
authors.initials: LS
authors.lastname: Vestergaard
authors.forename: Shunmay
authors.initials: S
authors.lastname: Yeung
authors.forename: Toby
authors.initials: T
authors.lastname: Leslie
authors.forename: Eleanor
authors.initials: E
authors.lastname: Hutchinson
authors.forename: Hugh
authors.initials: H
authors.lastname: Reyburn
authors.forename: David G
authors.initials: DG
authors.lastname: Lalloo
authors.forename: David
authors.initials: D
authors.lastname: Schellenberg
authors.forename: Bonnie
authors.initials: B
authors.lastname: Cundill
authors.forename: Sarah G
authors.initials: SG
authors.lastname: Staedke
authors.forename: Virginia
authors.initials: V
authors.lastname: Wiseman
authors.forename: Catherine
authors.initials: C
authors.lastname: Goodman
authors.forename: Clare I R
authors.initials: CI
authors.lastname: Chandler
mesh_terms: D003142:Communication; D003299:Cooperative Behavior; D003625:Data Collection; D015438:Health Behavior; D006282:Health Personnel / Q000193:education / Q000458:organization & administration*; D006302:Health Services Research / Q000458:organization & administration* / Q000592:standards; D006801:Humans; D007318:Inservice Training; D007400:Interprofessional Relations; D008288:Malaria / Q000175:diagnosis / Q000188:drug therapy; D011203:Poverty; D015397:Program Evaluation / Q000379:methods* / Q000592:standards; D012107:Research Design; D012108:Research Personnel / Q000193:education / Q000458:organization & administration*