title: Testing an Adapted Modified Delphi Method: Synthesizing Multiple Stakeholder Ratings of Health Care Service Effectiveness.
abstract: BACKGROUND
  The Affordable Care Act incentivizes health systems for better meeting patient needs, but often guidance about patient preferences for particular health services is limited. All too often vulnerable patient populations are excluded from these decision-making settings. A community-based participatory approach harnesses the in-depth knowledge of those experiencing barriers to health care.
  
  
  METHODS
  We made three modifications to the RAND-UCLA appropriateness method, a modified Delphi approach, involving patients, adding an advisory council group to characterize existing knowledge in this little studied area, and using effectiveness rather than "appropriateness" as the basis for rating. As a proof of concept, we tested this method by examining the broadly delivered but understudied nonmedical services that community health centers provide.
  
  
  RESULTS
  This method created discrete, new knowledge about these services by defining 6 categories and 112 unique services and by prioritizing among these services based on effectiveness using a 9-point scale. Consistent with the appropriateness method, we found statistical convergence of ratings among the panelists.
  
  
  CONCLUSIONS
  Challenges include time commitment and adherence to a clear definition of effectiveness of services. This diverse stakeholder engagement method efficiently addresses gaps in knowledge about the effectiveness of health care services to inform population health management.
authors.affiliation: AltaMed Health Services Corporation, Los Angeles, CA, USA.
authors.forename: Anne L
authors.initials: AL
authors.lastname: Escaron
authors.affiliation: Association of Asian Pacific Community Health Organizations, Oakland, CA, USA.
authors.forename: Rosy
authors.initials: R
authors.lastname: Chang Weir
authors.affiliation: St. Joseph's Medical Center, Stockton, CA, USA.
authors.forename: Petra
authors.initials: P
authors.lastname: Stanton
authors.affiliation: University of California Los Angeles, Los Angeles, CA, USA.
authors.forename: Sitaram
authors.initials: S
authors.lastname: Vangala
authors.affiliation: University of California Los Angeles, Los Angeles, CA, USA.
authors.forename: Tristan R
authors.initials: TR
authors.lastname: Grogan
authors.affiliation: University of California Los Angeles, Los Angeles, CA, USA rclarke@mednet.ucla.edu.
authors.forename: Robin M
authors.initials: RM
authors.lastname: Clarke
mesh_terms: D003695:Delivery of Health Care / Q000592:standards; D003697:Delphi Technique; D006801:Humans; D011785:Quality Assurance, Health Care / Q000379:methods*; D019984:Quality Indicators, Health Care; D011795:Surveys and Questionnaires; D014481:United States
keywords: access to health care; enabling services; health disparities; modified Delphi approach; patient-centered research method; social determinants of health; uninsured/safety net providers