title: Sociotechnical Human Factors Involved in Remote Online Usability Testing of Two eHealth Interventions.
abstract: BACKGROUND
  Research in the fields of human performance technology and human computer interaction are challenging the traditional macro focus of usability testing arguing for methods that help test moderators assess "use in context" (ie, cognitive skills, usability understood over time) and in authentic "real world" settings. Human factors in these complex test scenarios may impact on the quality of usability results being derived yet there is a lack of research detailing moderator experiences in these test environments. Most comparative research has focused on the impact of the physical environment on results, and rarely on how the sociotechnical elements of the test environment affect moderator and test user performance. Improving our understanding of moderator roles and experiences with conducting "real world" usability testing can lead to improved techniques and strategies
  
  
  OBJECTIVE
  To understand moderator experiences of using Web-conferencing software to conduct remote usability testing of 2 eHealth interventions.
  
  
  METHODS
  An exploratory case study approach was used to study 4 moderators' experiences using Blackboard Collaborate for remote testing sessions of 2 different eHealth interventions. Data collection involved audio-recording iterative cycles of test sessions, collecting summary notes taken by moderators, and conducting 2 90-minute focus groups via teleconference. A direct content analysis with an inductive coding approach was used to explore personal accounts, assess the credibility of data interpretation, and generate consensus on the thematic structure of the results.
  
  
  RESULTS
  Following the convergence of data from the various sources, 3 major themes were identified: (1) moderators experienced and adapted to unpredictable changes in cognitive load during testing; (2) moderators experienced challenges in creating and sustaining social presence and untangling dialogue; and (3) moderators experienced diverse technical demands, but were able to collaboratively troubleshoot with test users.
  
  
  CONCLUSIONS
  Results highlight important human-computer interactions and human factor qualities that impact usability testing processes. Moderators need an advanced skill and knowledge set to address the social interaction aspects of Web-based usability testing and technical aspects of conferencing software during test sessions. Findings from moderator-focused studies can inform the design of remote testing platforms and real-time usability evaluation processes that place less cognitive burden on moderators and test users.
authors.affiliation: Centre for Research in Family Health, IWK Health Centre, Halifax, NS, Canada. loriwozney@gmail.com.
authors.forename: Lori M
authors.identifier: http://orcid.org/0000-0001-6933-9596
authors.initials: LM
authors.lastname: Wozney
authors.forename: Pamela
authors.identifier: http://orcid.org/0000-0002-8396-8476
authors.initials: P
authors.lastname: Baxter
authors.forename: Hilary
authors.identifier: http://orcid.org/0000-0003-1976-5532
authors.initials: H
authors.lastname: Fast
authors.forename: Laura
authors.identifier: http://orcid.org/0000-0001-8712-3027
authors.initials: L
authors.lastname: Cleghorn
authors.forename: Amos S
authors.identifier: http://orcid.org/0000-0001-8257-2202
authors.initials: AS
authors.lastname: Hundert
authors.forename: Amanda S
authors.identifier: http://orcid.org/0000-0003-3020-674X
authors.initials: AS
authors.lastname: Newton
keywords: HCI design and evaluation methods; Web-conferencing; human centered-computing; real-time systems; telemedicine; usability inspection; walk-through evaluation