title: Engagement and retention: measuring breadth and depth of participant use of an online intervention.
abstract: BACKGROUND
  The Internet provides us with tools (user metrics or paradata) to evaluate how users interact with online interventions. Analysis of these paradata can lead to design improvements.
  
  
  OBJECTIVE
  The objective was to explore the qualities of online participant engagement in an online intervention. We analyzed the paradata in a randomized controlled trial of alternative versions of an online intervention designed to promote consumption of fruit and vegetables.
  
  
  METHODS
  Volunteers were randomized to 1 of 3 study arms involving several online sessions. We created 2 indirect measures of breadth and depth to measure different dimensions and dynamics of program engagement based on factor analysis of paradata measures of Web pages visited and time spent online with the intervention materials. Multiple regression was used to assess influence of engagement on retention and change in dietary intake.
  
  
  RESULTS
  Baseline surveys were completed by 2513 enrolled participants. Of these, 86.3% (n = 2168) completed the follow-up surveys at 3 months, 79.6% (n = 2027) at 6 months, and 79.4% (n = 1995) at 12 months. The 2 tailored intervention arms exhibited significantly more engagement than the untailored arm (P < .01). Breadth and depth measures of engagement were significantly associated with completion of follow-up surveys (odds ratios [OR] = 4.11 and 2.12, respectively, both P values < .001). The breadth measure of engagement was also significantly positively associated with a key study outcome, the mean increase in fruit and vegetable consumption (P < .001).
  
  
  CONCLUSIONS
  By exploring participants' exposures to online interventions, paradata are valuable in explaining the effects of tailoring in increasing participant engagement in the intervention. Controlling for intervention arm, greater engagement is also associated with retention of participants and positive change in a key outcome of the intervention, dietary change. This paper demonstrates the utility of paradata capture and analysis for evaluating online health interventions.
  
  
  BACKGROUND
  NCT00169312; http://clinicaltrials.gov/ct2/show/NCT00169312 (Archived by WebCite at http://www.webcitation.org/5u8sSr0Ty).
authors.affiliation: University of Michigan, Survey Research Center, Ann Arbor, 48106, USA. mcouper@umich.edu
authors.forename: Mick P
authors.initials: MP
authors.lastname: Couper
authors.forename: Gwen L
authors.initials: GL
authors.lastname: Alexander
authors.forename: Nanhua
authors.initials: N
authors.lastname: Zhang
authors.forename: Roderick J A
authors.initials: RJ
authors.lastname: Little
authors.forename: Noel
authors.initials: N
authors.lastname: Maddy
authors.forename: Michael A
authors.initials: MA
authors.lastname: Nowak
authors.forename: Jennifer B
authors.initials: JB
authors.lastname: McClure
authors.forename: Josephine J
authors.initials: JJ
authors.lastname: Calvi
authors.forename: Sharon J
authors.initials: SJ
authors.lastname: Rolnick
authors.forename: Melanie A
authors.initials: MA
authors.lastname: Stopponi
authors.forename: Christine
authors.initials: C
authors.lastname: Cole Johnson
mesh_terms: D000328:Adult; D003256:Community Participation / Q000706:statistics & numerical data*; D003376:Counseling / Q000379:methods*; D005247:Feeding Behavior; D005260:Female; D005638:Fruit; D019983:Guideline Adherence / Q000706:statistics & numerical data*; D006293:Health Promotion / Q000379:methods*; D006801:Humans; D020407:Internet / Q000706:statistics & numerical data*; D008297:Male; D008875:Middle Aged; D012153:Retention, Psychology; D012648:Self Care / Q000379:methods; D011795:Surveys and Questionnaires; D013813:Therapy, Computer-Assisted / Q000379:methods; D014675:Vegetables; D055815:Young Adult