title: Developing a quality criteria framework for patient decision aids: online international Delphi consensus process.
abstract: OBJECTIVE
  To develop a set of quality criteria for patient decision support technologies (decision aids).
  
  
  METHODS
  Two stage web based Delphi process using online rating process to enable international collaboration.
  
  
  METHODS
  Individuals from four stakeholder groups (researchers, practitioners, patients, policy makers) representing 14 countries reviewed evidence summaries and rated the importance of 80 criteria in 12 quality domains on a 1 to 9 scale. Second round participants received feedback from the first round and repeated their assessment of the 80 criteria plus three new ones.
  
  
  METHODS
  Aggregate ratings for each criterion calculated using medians weighted to compensate for different numbers in stakeholder groups; criteria rated between 7 and 9 were retained.
  
  
  RESULTS
  212 nominated people were invited to participate. Of those invited, 122 participated in the first round (77 researchers, 21 patients, 10 practitioners, 14 policy makers); 104/122 (85%) participated in the second round. 74 of 83 criteria were retained in the following domains: systematic development process (9/9 criteria); providing information about options (13/13); presenting probabilities (11/13); clarifying and expressing values (3/3); using patient stories (2/5); guiding/coaching (3/5); disclosing conflicts of interest (5/5); providing internet access (6/6); balanced presentation of options (3/3); using plain language (4/6); basing information on up to date evidence (7/7); and establishing effectiveness (8/8).
  
  
  CONCLUSIONS
  Criteria were given the highest ratings where evidence existed, and these were retained. Gaps in research were highlighted. Developers, users, and purchasers of patient decision aids now have a checklist for appraising quality. An instrument for measuring quality of decision aids is being developed.
authors.affiliation: Department of General Practice, Centre for Health Sciences Research, Cardiff University, Cardiff CF14 4YS. elwyng@cardiff.ac.uk
authors.forename: Glyn
authors.initials: G
authors.lastname: Elwyn
authors.forename: Annette
authors.initials: A
authors.lastname: O'Connor
authors.forename: Dawn
authors.initials: D
authors.lastname: Stacey
authors.forename: Robert
authors.initials: R
authors.lastname: Volk
authors.forename: Adrian
authors.initials: A
authors.lastname: Edwards
authors.forename: Angela
authors.initials: A
authors.lastname: Coulter
authors.forename: Richard
authors.initials: R
authors.lastname: Thomson
authors.forename: Alexandra
authors.initials: A
authors.lastname: Barratt
authors.forename: Michael
authors.initials: M
authors.lastname: Barry
authors.forename: Steven
authors.initials: S
authors.lastname: Bernstein
authors.forename: Phyllis
authors.initials: P
authors.lastname: Butow
authors.forename: Aileen
authors.initials: A
authors.lastname: Clarke
authors.forename: Vikki
authors.initials: V
authors.lastname: Entwistle
authors.forename: Deb
authors.initials: D
authors.lastname: Feldman-Stewart
authors.forename: Margaret
authors.initials: M
authors.lastname: Holmes-Rovner
authors.forename: Hilary
authors.initials: H
authors.lastname: Llewellyn-Thomas
authors.forename: Nora
authors.initials: N
authors.lastname: Moumjid
authors.forename: Al
authors.initials: A
authors.lastname: Mulley
authors.forename: Cornelia
authors.initials: C
authors.lastname: Ruland
authors.forename: Karen
authors.initials: K
authors.lastname: Sepucha
authors.forename: Alan
authors.initials: A
authors.lastname: Sykes
authors.forename: Tim
authors.initials: T
authors.lastname: Whelan
mesh_terms: D003661:Decision Support Techniques; D003697:Delphi Technique; D006801:Humans; D010358:Patient Participation