title: THE GHOST IN THE MACHINE? THE VALUE OF EXPERT ADVICE IN THE PRODUCTION OF EVIDENCE-BASED GUIDANCE: A MIXED METHODS STUDY OF THE NICE INTERVENTIONAL PROCEDURES PROGRAMME.
abstract: OBJECTIVE
  The aim of this study was to determine the aspects of expert advice that decision makers find most useful in the development of evidence-based guidance and to identify the characteristics of experts providing the most useful advice.
  
  
  METHODS
  First, semi-structured interviews were conducted with seventeen members of the Interventional Procedures Advisory Committee of the UK's National Institute of Health and Care Excellence. Interviews examined the usefulness of expert advice during guidance development. Transcripts were analyzed inductively to identify themes. Second, data were extracted from 211 experts' questionnaires for forty-one consecutive procedures. Usefulness of advice was scored using an index developed through the qualitative work. Associations between usefulness score and characteristics of the expert advisor were investigated using univariate and multivariate analyses.
  
  
  RESULTS
  Expert opinion was seen as a valued complement to empirical evidence, providing context and tacit knowledge unavailable in published literature, but helpful for interpreting it. Interviewees also valued advice on the training and experience required to perform a procedure, on patient selection criteria and the place of a procedure within a clinical management pathway. Limitations of bias in expert opinion were widely acknowledged and skepticism expressed regarding the anecdotal nature of advice on safety or efficacy outcomes. Quantitative analysis demonstrated that the most useful advice was given by clinical experts with direct personal experience of the procedure, particularly research experience.
  
  
  CONCLUSIONS
  Evidence-based guidance production is often characterized as a rational, pipeline process. This ignores the valuable role that expert opinion plays in guidance development, complementing and supporting the interpretation of empirical data.
authors.affiliation: University of Warwick Medical School,Gibbet Hill Campuso.r.o.oyebode@warwick.ac.uk.
authors.forename: Oyinlola
authors.identifier: http://orcid.org/0000-0003-0925-9839
authors.initials: O
authors.lastname: Oyebode
authors.affiliation: National Institute for Health and Care Excellence.
authors.forename: Hannah
authors.initials: H
authors.lastname: Patrick
authors.affiliation: Royal Infirmary of Edinburgh.
authors.forename: Alexander
authors.initials: A
authors.lastname: Walker
authors.affiliation: National Institute for Health and Care Excellence.
authors.forename: Bruce
authors.initials: B
authors.lastname: Campbell
authors.affiliation: National Institute for Health and Care Excellence.
authors.forename: John
authors.initials: J
authors.lastname: Powell
mesh_terms: D026683:Advisory Committees / Q000458:organization & administration; D003657:Decision Making; D019317:Evidence-Based Medicine / Q000458:organization & administration*; D005260:Female; D006801:Humans; D008297:Male; D013222:State Medicine / Q000458:organization & administration*; D013673:Technology Assessment, Biomedical / Q000458:organization & administration*; D006113:United Kingdom
keywords: Biomedical; Evidence-based medicine; Expert testimony; Technology assessment