title: Evaluating a complex system-wide intervention using the difference in differences method: the Delivering Choice Programme.
abstract: OBJECTIVE
  We report the use of difference in differences (DiD) methodology to evaluate a complex, system-wide healthcare intervention. We use the worked example of evaluating the Marie Curie Delivering Choice Programme (DCP) for advanced illness in a large urban healthcare economy.
  
  
  METHODS
  DiD was selected because a randomised controlled trial was not feasible. The method allows for before and after comparison of changes that occur in an intervention site with a matched control site. This enables analysts to control for the effect of the intervention in the absence of a local control. Any policy, seasonal or other confounding effects over the test period are assumed to have occurred in a balanced way at both sites. Data were obtained from primary care trusts. Outcomes were place of death, inpatient admissions, length of stay and costs.
  
  
  RESULTS
  Small changes were identified between pre- and post-DCP outputs in the intervention site. The proportion of home deaths and median cost increased slightly, while the number of admissions per patient and the average length of stay per admission decreased slightly. None of these changes was statistically significant.
  
  
  CONCLUSIONS
  Effects estimates were limited by small numbers accessing new services and selection bias in sample population and comparator site. In evaluating the effect of a complex healthcare intervention, the choice of analysis method and output measures is crucial. Alternatives to randomised controlled trials may be required for evaluating large scale complex interventions and the DiD approach is suitable, subject to careful selection of measured outputs and control population.
authors.affiliation: Marie Curie Palliative Care Research Unit, University College London, London, UK.
authors.forename: Jeff
authors.initials: J
authors.lastname: Round
authors.affiliation: Marie Curie Palliative Care Research Unit, University College London, London, UK.
authors.forename: Robyn
authors.initials: R
authors.lastname: Drake
authors.affiliation: Health Care Evaluation Group, University College London, London, UK.
authors.forename: Edward
authors.initials: E
authors.lastname: Kendall
authors.affiliation: The King's Fund, London, UK.
authors.forename: Rachael
authors.initials: R
authors.lastname: Addicott
authors.affiliation: Marie Curie Cancer Care, London, UK.
authors.forename: Nicky
authors.initials: N
authors.lastname: Agelopoulos
authors.affiliation: Marie Curie Palliative Care Research Unit, University College London, London, UK.
authors.forename: Louise
authors.initials: L
authors.lastname: Jones
mesh_terms: D065187:Controlled Before-After Studies; D003695:Delivery of Health Care; D017048:Health Care Costs / Q000706:statistics & numerical data; D006760:Hospitalization / Q000706:statistics & numerical data; D006801:Humans; D007902:Length of Stay; D009026:Mortality; D065228:Non-Randomized Controlled Trials as Topic / Q000379:methods*; D057240:Patient Preference; D015397:Program Evaluation / Q000379:methods*; D012107:Research Design; D013727:Terminal Care; D014505:Urban Population
keywords: Health Economics; Methodological Research; Palliative Care; Research Design; Terminal Care