title: Usability study of clinical exome analysis software: top lessons learned and recommendations.
abstract: OBJECTIVE
  New DNA sequencing technologies have revolutionized the search for genetic disruptions. Targeted sequencing of all protein coding regions of the genome, called exome analysis, is actively used in research-oriented genetics clinics, with the transition to exomes as a standard procedure underway. This transition is challenging; identification of potentially causal mutation(s) amongst ∼10(6) variants requires specialized computation in combination with expert assessment. This study analyzes the usability of user interfaces for clinical exome analysis software. There are two study objectives: (1) To ascertain the key features of successful user interfaces for clinical exome analysis software based on the perspective of expert clinical geneticists, (2) To assess user-system interactions in order to reveal strengths and weaknesses of existing software, inform future design, and accelerate the clinical uptake of exome analysis.
  
  
  METHODS
  Surveys, interviews, and cognitive task analysis were performed for the assessment of two next-generation exome sequence analysis software packages. The subjects included ten clinical geneticists who interacted with the software packages using the "think aloud" method. Subjects' interactions with the software were recorded in their clinical office within an urban research and teaching hospital. All major user interface events (from the user interactions with the packages) were time-stamped and annotated with coding categories to identify usability issues in order to characterize desired features and deficiencies in the user experience.
  
  
  RESULTS
  We detected 193 usability issues, the majority of which concern interface layout and navigation, and the resolution of reports. Our study highlights gaps in specific software features typical within exome analysis. The clinicians perform best when the flow of the system is structured into well-defined yet customizable layers for incorporation within the clinical workflow. The results highlight opportunities to dramatically accelerate clinician analysis and interpretation of patient genomic data.
  
  
  CONCLUSIONS
  We present the first application of usability methods to evaluate software interfaces in the context of exome analysis. Our results highlight how the study of user responses can lead to identification of usability issues and challenges and reveal software reengineering opportunities for improving clinical next-generation sequencing analysis. While the evaluation focused on two distinctive software tools, the results are general and should inform active and future software development for genome analysis software. As large-scale genome analysis becomes increasingly common in healthcare, it is critical that efficient and effective software interfaces are provided to accelerate clinical adoption of the technology. Implications for improved design of such applications are discussed.
authors.affiliation: Centre for Molecular Medicine and Therapeutics, Child & Family Research Institute, 950 28th Ave W, Vancouver, BC V5Z 4H4, Canada; Bioinformatics Graduate Program, University of British Columbia, 2329 West Mall, Vancouver, BC V6T 1Z4, Canada.
authors.forename: Casper
authors.initials: C
authors.lastname: Shyr
authors.affiliation: School of Health Information Science, University of Victoria, 3800 Finnerty Rd., Victoria, BC V8P 5C2, Canada.
authors.forename: Andre
authors.initials: A
authors.lastname: Kushniruk
authors.affiliation: Centre for Molecular Medicine and Therapeutics, Child & Family Research Institute, 950 28th Ave W, Vancouver, BC V5Z 4H4, Canada; Department of Medical Genetics, University of British Columbia, 2329 West Mall, Vancouver, BC V6T 1Z4, Canada. Electronic address: wyeth@cmmt.ubc.ca.
authors.forename: Wyeth W
authors.initials: WW
authors.lastname: Wasserman
mesh_terms: D002874:Chromosome Mapping / Q000379:methods* / Q000706:statistics & numerical data*; D003258:Consumer Behavior / Q000706:statistics & numerical data; D059472:Exome / Q000235:genetics*; D062527:Meaningful Use / Q000706:statistics & numerical data*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D017422:Sequence Analysis, DNA / Q000706:statistics & numerical data*; D012984:Software; D012986:Software Validation; D014584:User-Computer Interface
keywords: Exome; Genome; Next-generation sequencing; Software interface; Usability; Whole-genome