title: Evaluating Digital Health Interventions: Key Questions and Approaches.
abstract: Digital health interventions have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety, and personalization. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of digital health interventions. However, evaluations of digital health interventions present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the research questions needed to appraise such interventions. As they are at the intersection of biomedical, behavioral, computing, and engineering research, methods drawn from all of these disciplines are required. Relevant research questions include defining the problem and the likely benefit of the digital health intervention, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components, and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost effectiveness, and harms. Although RCTs are important for evaluation of effectiveness and cost effectiveness, they are best undertaken only when: (1) the intervention and its delivery package are stable; (2) these can be implemented with high fidelity; and (3) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at lower cost). Broadening the portfolio of research questions and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice, and research.
authors.affiliation: Research Department of Primary Care and Population Health, University College London, London, United Kingdom. Electronic address: elizabeth.murray@ucl.ac.uk.
authors.forename: Elizabeth
authors.initials: E
authors.lastname: Murray
authors.affiliation: Designing Health Lab, School of Nutrition and Health Promotion, Arizona State University, Phoenix, Arizona.
authors.forename: Eric B
authors.initials: EB
authors.lastname: Hekler
authors.affiliation: Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden; Department of Clinical Neuroscience, Karolinska Institute, Stockholm, Sweden.
authors.forename: Gerhard
authors.initials: G
authors.lastname: Andersson
authors.affiliation: The Methodology Center and Department of Human Development and Family Studies, The Pennsylvania State University, State College, Pennsylvania.
authors.forename: Linda M
authors.initials: LM
authors.lastname: Collins
authors.affiliation: MRC Clinical Trial Service Unit Hub, Nuffield Department of Population Health, University of Oxford, Oxford, United Kingdom.
authors.forename: Aiden
authors.initials: A
authors.lastname: Doherty
authors.affiliation: NIHR MindTech HTC, University of Nottingham, Nottingham, United Kingdom.
authors.forename: Chris
authors.initials: C
authors.lastname: Hollis
authors.affiliation: School for the Engineering of Matter, Transport, and Energy, Ira A. Fulton Schools of Engineering, Arizona State University, Phoenix, Arizona.
authors.forename: Daniel E
authors.initials: DE
authors.lastname: Rivera
authors.affiliation: Research Department of Epidemiology and Public Health, University College London, London, United Kingdom.
authors.forename: Robert
authors.initials: R
authors.lastname: West
authors.affiliation: Wessex Institute, University of Southampton, Southampton, United Kingdom.
authors.forename: Jeremy C
authors.initials: JC
authors.lastname: Wyatt
mesh_terms: D015438:Health Behavior; D006293:Health Promotion; D006801:Humans; D015397:Program Evaluation; D012107:Research Design; D013685:Telecommunications