title: Natural Language Understanding Performance & Use Considerations in Virtual Medical Encounters.
abstract: A virtual standardized patient (VSP) prototype was tested for natural language understanding (NLU) performance. The conversational VSP was evaluated in a controlled 61 subject study over four repetitions of a patient case. The prototype achieved more than 92% appropriate response rate from naïve users on their first attempt and results were stable by their fourth case repetition. This level of performance exceeds prior efforts and is at a level comparable of accuracy as seen in human conversational patient training, with caveats. This level of performance was possible due to the use of a unified medical taxonomy underpinning that allows virtual patient language training to be applied to all cases in our system as opposed to benefiting a single patient case.
authors.affiliation: Keck School of Medicine of the University of Southern California.
authors.forename: Thomas B
authors.initials: TB
authors.lastname: Talbot
authors.affiliation: University of Southern California Institute for Creative Technologies.
authors.forename: Nicolai
authors.initials: N
authors.lastname: Kalisch
authors.affiliation: University of Southern California Institute for Creative Technologies.
authors.forename: Kelly
authors.initials: K
authors.lastname: Christoffersen
authors.affiliation: University of Southern California Institute for Creative Technologies.
authors.forename: Gale
authors.initials: G
authors.lastname: Lucas
authors.affiliation: University of Southern California Institute for Creative Technologies.
authors.forename: Eric
authors.initials: E
authors.lastname: Forbell
mesh_terms: D017408:Guidelines as Topic; D000069550:Machine Learning / Q000592:standards*; D008328:Man-Machine Systems; D009323:Natural Language Processing; D016544:Patient Simulation; D066230:Patient-Specific Modeling / Q000592:standards*; D049250:Speech Recognition Software / Q000592:standards*; D014481:United States; D014584:User-Computer Interface