title: Quantifying uncertainty in intervention effectiveness with structured expert judgement: an application to obstetric fistula.
abstract: OBJECTIVE
  To demonstrate a new application of structured expert judgement to assess the effectiveness of surgery to correct obstetric fistula in a low-income setting. Intervention effectiveness is a major input of evidence-informed priority setting in healthcare, but information on intervention effectiveness is generally lacking. This is particularly problematic in the context of poorly resourced healthcare settings where even efficacious interventions fail to translate into improvements in health. The few intervention effectiveness studies related to obstetric fistula treatment focus on the experience of single facilities and do not consider the impact of multiple factors that may affect health outcomes.
  
  
  METHODS
  We use the classical model of structured expert judgement, a method that has been used to quantify uncertainty in the areas of engineering and environmental risk assessment when data are unavailable. Under this method, experts quantify their uncertainty about rates of long-term disability in patients with fistula following treatment in different contexts, but the information content drawn from their responses is statistically conditioned on the accuracy and informativeness of their responses to a set of calibration questions. Through this method, we develop best estimates and uncertainty bounds for the rate of disability associated with each treatment scenario and setting.
  
  
  METHODS
  Eight experts in obstetric fistula repair in low and middle income countries.
  
  
  RESULTS
  Estimates developed using performance weights were statistically superior to those involving a simple averaging of expert responses. The performance-weight decision maker's assessments are narrower for 9 of the 10 calibration questions and 21 of 23 variables of interest.
  
  
  CONCLUSIONS
  We find that structured expert judgement is a viable approach to investigating the effectiveness of medical interventions where randomised controlled trials are not possible. Understanding the effectiveness of surgery performed at different types of facilities can guide programme planning to increase access to fistula treatment.
authors.affiliation: Center for Disease Dynamics, Economics & Policy, Washington DC, USA Princeton Environmental Institute, Princeton University, Princeton, New Jersey, USA.
authors.forename: Abigail R
authors.initials: AR
authors.lastname: Colson
authors.affiliation: Center for Disease Dynamics, Economics & Policy, Washington DC, USA Columbia University Mailman School of Public Health, New York, New York, USA.
authors.forename: Sweta
authors.initials: S
authors.lastname: Adhikari
authors.affiliation: Columbia University Mailman School of Public Health, New York, New York, USA Maimonides Medical Center, Brooklyn, New York, USA.
authors.forename: Ambereen
authors.initials: A
authors.lastname: Sleemi
authors.affiliation: Center for Disease Dynamics, Economics & Policy, Washington DC, USA Princeton Environmental Institute, Princeton University, Princeton, New Jersey, USA Public Health Foundation of India, New Delhi, India.
authors.forename: Ramanan
authors.initials: R
authors.lastname: Laxminarayan
mesh_terms: D003661:Decision Support Techniques; D019317:Evidence-Based Medicine; D005260:Female; D006801:Humans; D007600:Judgment; D011204:Poverty Areas; D011247:Pregnancy; D018570:Risk Assessment; D016896:Treatment Outcome; D035501:Uncertainty; D014624:Vaginal Fistula / Q000601:surgery*
keywords: GENITOURINARY MEDICINE; GYNAECOLOGY