Exploring the Usability of Mobile Apps Supporting Radiologists' Training in Diagnostic Decision Making.
OBJECTIVE
The objective of this study was to conduct a usability evaluation of mobile apps for supporting education and training in radiologic diagnostic decision-making processes.


METHODS
Of 381 mobile apps available at two major stores (Google Play and iTunes), eight iOS apps were selected for laboratory-based usability tests. Six staff radiologists completed eight app-specific task sets, using a think-aloud strategy. The triangular methods approach included quantitative performance measures, System Usability Scale (SUS), and qualitative thematic analysis using heuristic usability principles of usability issues.


RESULTS
Overall, radiologists achieved higher than 70% success, with favorable SUS scores, in completing the tasks for seven of the eight apps. However, task success rate and SUS score had a weak relation (r = 0.23), indicating that the perceived usability may not reflect the holistic usability of the app. Task analysis and self-report revealed 108 usability issues, which were condensed to 55 unique issues and categorized by nine usability themes and mapped to ten usability heuristics. Nonintuitive functionality (eg, nonintuitive or misleading labels) was the most frequent theme observed, leading to inefficient navigation. These usability findings were consistent with the 13 improvements the radiologists suggested.


CONCLUSIONS
This study demonstrates the feasibility of usability evaluation of radiology mobile apps and suggests potential improvements in the development of radiology mobile apps. This study also suggests that proficiency with mobile devices may not be equivalent to being an expert user, proficient in using the apps.
Department of Health Management and Informatics, University of Missouri School of Medicine, Columbia, Missouri; University of Missouri Informatics Institute, Columbia, Missouri; Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai, New York, New York. Electronic address: kimms@health.missouri.edu.
Min Soon
MS
Kim
Department of Radiology, University of Missouri, Columbia, Missouri.
Michael R
MR
Aro
Department of Radiology, University of Missouri, Columbia, Missouri.
Kraig J
KJ
Lage
Department of Radiology, University of Missouri, Columbia, Missouri.
Kevin L
KL
Ingalls
Department of Radiology, University of Missouri, Columbia, Missouri.
Vivek
V
Sindhwani
Department of Biomedical Engineering, University of Texas, Austin, Texas; Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas.
Mia K
MK
Markey
D001291:Attitude of Health Personnel; D003194:Computer-Assisted Instruction / Q000706:statistics & numerical data*; D003657:Decision Making; D020000:Decision Support Systems, Clinical / Q000706:statistics & numerical data*; D063731:Mobile Applications / Q000706:statistics & numerical data*; D011871:Radiology / Q000193:education* / Q000706:statistics & numerical data; D012986:Software Validation; D014600:Utilization Review
Usability; decision making; education; mobile app