title: Evaluation of symptom checkers for self diagnosis and triage: audit study.
abstract: OBJECTIVE
  To determine the diagnostic and triage accuracy of online symptom checkers (tools that use computer algorithms to help patients with self diagnosis or self triage).
  
  
  METHODS
  Audit study.
  
  
  METHODS
  Publicly available, free symptom checkers.
  
  
  METHODS
  23 symptom checkers that were in English and provided advice across a range of conditions. 45 standardized patient vignettes were compiled and equally divided into three categories of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection).
  
  
  METHODS
  For symptom checkers that provided a diagnosis, our main outcomes were whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses (n=770 standardized patient evaluations). For symptom checkers that provided a triage recommendation, our main outcomes were whether the symptom checker correctly recommended emergent care, non-emergent care, or self care (n=532 standardized patient evaluations).
  
  
  RESULTS
  The 23 symptom checkers provided the correct diagnosis first in 34% (95% confidence interval 31% to 37%) of standardized patient evaluations, listed the correct diagnosis within the top 20 diagnoses given in 58% (55% to 62%) of standardized patient evaluations, and provided the appropriate triage advice in 57% (52% to 61%) of standardized patient evaluations. Triage performance varied by urgency of condition, with appropriate triage advice provided in 80% (95% confidence interval 75% to 86%) of emergent cases, 55% (47% to 63%) of non-emergent cases, and 33% (26% to 40%) of self care cases (P<0.001). Performance on appropriate triage advice across the 23 individual symptom checkers ranged from 33% (95% confidence interval 19% to 48%) to 78% (64% to 91%) of standardized patient evaluations.
  
  
  CONCLUSIONS
  Symptom checkers had deficits in both triage and diagnosis. Triage advice from symptom checkers is generally risk averse, encouraging users to seek care for conditions where self care is reasonable.
authors.affiliation: Department of Health Care Policy, Harvard Medical School, Boston, MA 02115, USA.
authors.forename: Hannah L
authors.initials: HL
authors.lastname: Semigran
authors.affiliation: Division of General Medicine and Primary Care, Brigham and Women's Hospital & Harvard Medical School, Boston, MA, USA.
authors.forename: Jeffrey A
authors.initials: JA
authors.lastname: Linder
authors.affiliation: Division of Infectious Diseases, Boston Children's Hospital, Boston, MA, USA RAND Corporation, Boston, MA, USA.
authors.forename: Courtney
authors.initials: C
authors.lastname: Gidengil
authors.affiliation: Department of Health Care Policy, Harvard Medical School, Boston, MA 02115, USA Division of General Internal Medicine and Primary Care, Beth Israel Deaconess Medical Center, Boston, MA, USA.
authors.forename: Ateev
authors.initials: A
authors.lastname: Mehrotra
mesh_terms: D054626:Consumer Health Information / Q000592:standards*; D057220:Health Literacy / Q000706:statistics & numerical data; D006801:Humans; D057226:Information Seeking Behavior; D008485:Medical Audit; D010353:Patient Education as Topic; D011787:Quality of Health Care / Q000592:standards*; D012648:Self Care / Q000379:methods*; D014218:Triage / Q000592:standards*