title: Test Case Selection in Pre-Deployment Testing of Complex Clinical Decision Support Systems.
abstract: Clinical decision support (CDS) systems with complex logic are being developed. Ensuring the quality of CDS is imperative, but there is no consensus on testing standards. We tested ATHENA-HTN CDS after encoding updated hypertension guidelines into the system. A logic flow and a complexity analysis of the encoding were performed to guide testing. 100 test cases were selected to test the major pathways in the CDS logic flow, and the effectiveness of the testing was analyzed. The encoding contained 26 decision points and 3120 possible output combinations. The 100 cases selected tested all of the major pathways in the logic, but only 1% of the possible output combinations. Test case selection is one of the most challenging aspects in CDS testing and has a major impact on testing coverage. A test selection strategy should take into account the complexity of the system, identification of major logic pathways, and available resources.
authors: Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, CA;; Center for Primary Care and Outcomes Research (PCOR), Stanford University School of Medicine, Stanford, CA;
authors: Geoffrey J
authors: GJ
authors: Tso
authors: Geriatrics Research Education and Clinical Center (GRECC), VA Palo Alto Health Care System, Palo Alto, CA;
authors: Kaeli
authors: K
authors: Yuen
authors: Geriatrics Research Education and Clinical Center (GRECC), VA Palo Alto Health Care System, Palo Alto, CA;
authors: Susana
authors: S
authors: Martins
authors: Biomedical Informatics Research (BMIR), Stanford University School of Medicine, Stanford, CA;
authors: Samson W
authors: SW
authors: Tu
authors: Geriatrics Research Education and Clinical Center (GRECC), VA Palo Alto Health Care System, Palo Alto, CA;
authors: Michael
authors: M
authors: Ashcraft
authors: Cardiology, VA Palo Alto Health Care System, Palo Alto, CA;
authors: Paul
authors: P
authors: Heidenreich
authors: VA Boston Health Care System, Boston, MA;; Harvard Medical School, Boston, MA.
authors: Brian B
authors: BB
authors: Hoffman
authors: Center for Primary Care and Outcomes Research (PCOR), Stanford University School of Medicine, Stanford, CA;; Geriatrics Research Education and Clinical Center (GRECC), VA Palo Alto Health Care System, Palo Alto, CA;
authors: Mary K
authors: MK
authors: Goldstein