title: Comparative analysis of stakeholder experiences with an online approach to prioritizing patient-centered research topics.
abstract: OBJECTIVE
  Little evidence exists about effective and scalable methods for meaningful stakeholder engagement in research. We explored patient/caregiver experiences with a high-tech online engagement approach for patient-centered research prioritization, compared their experiences with those of professional stakeholders, and identified factors associated with favorable participant experiences.
  
  
  METHODS
  We conducted 8 online modified-Delphi (OMD) panels. Panelists participated in 2 rating rounds with a statistical feedback/online discussion round in between. Panels focused on weight management/obesity, heart failure, and Kawasaki disease. We recruited a convenience sample of adults with any of the 3 conditions (or parents/guardians of Kawasaki disease patients), clinicians, and researchers. Measures included self-reported willingness to use OMD again, the panelists' study participation and online discussion experiences, the system's perceived ease of use, and active engagement metrics.
  
  
  RESULTS
  Out of 349 panelists, 292 (84%) completed the study. Of those, 46% were patients, 36% were clinicians, and 19% were researchers. In multivariate models, patients were not significantly more actively engaged (Odds ratio (OR) = 1.69, 95% confidence interval (CI), 0.94-3.05) but had more favorable study participation (β = 0.49; P  ≤ .05) and online discussion (β = 0.18; P  ≤ .05) experiences and were more willing to use OMD again (β = 0.36; P  ≤ .05), compared to professional stakeholders. Positive perceptions of the OMD system's ease of use (β = 0.16; P  ≤ .05) and favorable study participation (β = 0.26; P  ≤ .05) and online discussion (β = 0.57; P  ≤ .05) experiences were also associated with increased willingness to use OMD in the future. Active engagement was not associated with online experience indices or willingness to use OMD again.
  
  
  CONCLUSIONS
  Online approaches to engaging large numbers of stakeholders are a promising and efficient adjunct to in-person meetings.
authors.affiliation: RAND Corporation, Santa Monica, CA, USA.
authors.forename: Dmitry
authors.initials: D
authors.lastname: Khodyakov
authors.affiliation: RAND Corporation, Santa Monica, CA, USA.
authors.forename: Sean
authors.initials: S
authors.lastname: Grant
authors.affiliation: RAND Corporation, Santa Monica, CA, USA.|University of Southern California, Los Angeles, CA, USA.
authors.forename: Daniella
authors.initials: D
authors.lastname: Meeker
authors.affiliation: RAND Corporation, Santa Monica, CA, USA.
authors.forename: Marika
authors.initials: M
authors.lastname: Booth
authors.affiliation: RAND Corporation, Santa Monica, CA, USA.
authors.forename: Nathaly
authors.initials: N
authors.lastname: Pacheco-Santivanez
authors.affiliation: University of California, Davis, CA, USA.
authors.forename: Katherine K
authors.initials: KK
authors.lastname: Kim
mesh_terms: D000328:Adult; D035843:Biomedical Research; D003697:Delphi Technique; D005260:Female; D006801:Humans; D016015:Logistic Models; D008297:Male; D015999:Multivariate Analysis; D009862:Online Systems; D063868:Patient Outcome Assessment; D010361:Patients; D012108:Research Personnel; D011795:Surveys and Questionnaires
keywords: ExpertLens; online modified-Delphi; pSCANNER; patient engagement; patient-centered outcomes research; stakeholder engagement