title: Methods to elicit probability distributions from experts: a systematic review of reported practice in health technology assessment.
abstract: BACKGROUND
  Elicitation is a technique that can be used to obtain probability distribution from experts about unknown quantities. We conducted a methodology review of reports where probability distributions had been elicited from experts to be used in model-based health technology assessments.
  
  
  METHODS
  Databases including MEDLINE, EMBASE and the CRD database were searched from inception to April 2013. Reference lists were checked and citation mapping was also used. Studies describing their approach to the elicitation of probability distributions were included. Data was abstracted on pre-defined aspects of the elicitation technique. Reports were critically appraised on their consideration of the validity, reliability and feasibility of the elicitation exercise.
  
  
  RESULTS
  Fourteen articles were included. Across these studies, the most marked features were heterogeneity in elicitation approach and failure to report key aspects of the elicitation method. The most frequently used approaches to elicitation were the histogram technique and the bisection method. Only three papers explicitly considered the validity, reliability and feasibility of the elicitation exercises.
  
  
  CONCLUSIONS
  Judged by the studies identified in the review, reports of expert elicitation are insufficient in detail and this impacts on the perceived usability of expert-elicited probability distributions. In this context, the wider credibility of elicitation will only be improved by better reporting and greater standardisation of approach. Until then, the advantage of eliciting probability distributions from experts may be lost.
authors.affiliation: Peninsula Technology Assessment Group (PenTAG), Institute of Health Research, University of Exeter Medical School, University of Exeter, Exeter, UK, bogdan.grigore@pcmd.ac.uk.
authors.forename: Bogdan
authors.initials: B
authors.lastname: Grigore
authors.forename: Jaime
authors.initials: J
authors.lastname: Peters
authors.forename: Christopher
authors.initials: C
authors.lastname: Hyde
authors.forename: Ken
authors.initials: K
authors.lastname: Stein
mesh_terms: D020811:Biomedical Technology / Q000592:standards*; D006801:Humans; D008962:Models, Theoretical; D011336:Probability; D015203:Reproducibility of Results; D013673:Technology Assessment, Biomedical / Q000379:methods*