title: Impact of Predicting Health Care Utilization Via Web Search Behavior: A Data-Driven Analysis.
abstract: BACKGROUND
  By recent estimates, the steady rise in health care costs has deprived more than 45 million Americans of health care services and has encouraged health care providers to better understand the key drivers of health care utilization from a population health management perspective. Prior studies suggest the feasibility of mining population-level patterns of health care resource utilization from observational analysis of Internet search logs; however, the utility of the endeavor to the various stakeholders in a health ecosystem remains unclear.
  
  
  OBJECTIVE
  The aim was to carry out a closed-loop evaluation of the utility of health care use predictions using the conversion rates of advertisements that were displayed to the predicted future utilizers as a surrogate. The statistical models to predict the probability of user's future visit to a medical facility were built using effective predictors of health care resource utilization, extracted from a deidentified dataset of geotagged mobile Internet search logs representing searches made by users of the Baidu search engine between March 2015 and May 2015.
  
  
  METHODS
  We inferred presence within the geofence of a medical facility from location and duration information from users' search logs and putatively assigned medical facility visit labels to qualifying search logs. We constructed a matrix of general, semantic, and location-based features from search logs of users that had 42 or more search days preceding a medical facility visit as well as from search logs of users that had no medical visits and trained statistical learners for predicting future medical visits. We then carried out a closed-loop evaluation of the utility of health care use predictions using the show conversion rates of advertisements displayed to the predicted future utilizers. In the context of behaviorally targeted advertising, wherein health care providers are interested in minimizing their cost per conversion, the association between show conversion rate and predicted utilization score, served as a surrogate measure of the model's utility.
  
  
  RESULTS
  We obtained the highest area under the curve (0.796) in medical visit prediction with our random forests model and daywise features. Ablating feature categories one at a time showed that the model performance worsened the most when location features were dropped. An online evaluation in which advertisements were served to users who had a high predicted probability of a future medical visit showed a 3.96% increase in the show conversion rate.
  
  
  CONCLUSIONS
  Results from our experiments done in a research setting suggest that it is possible to accurately predict future patient visits from geotagged mobile search logs. Results from the offline and online experiments on the utility of health utilization predictions suggest that such prediction can have utility for health care providers.
authors.affiliation: Biomedical Informatics Training Program, Stanford University, Stanford, CA, United States. vibhua@stanford.edu.
authors.forename: Vibhu
authors.identifier: http://orcid.org/0000-0002-6392-3924
authors.initials: V
authors.lastname: Agarwal
authors.forename: Liangliang
authors.identifier: http://orcid.org/0000-0001-7862-2414
authors.initials: L
authors.lastname: Zhang
authors.forename: Josh
authors.identifier: http://orcid.org/0000-0003-2156-8271
authors.initials: J
authors.lastname: Zhu
authors.forename: Shiyuan
authors.identifier: http://orcid.org/0000-0001-7000-7599
authors.initials: S
authors.lastname: Fang
authors.forename: Tim
authors.identifier: http://orcid.org/0000-0002-3165-2963
authors.initials: T
authors.lastname: Cheng
authors.forename: Chloe
authors.identifier: http://orcid.org/0000-0002-9897-7746
authors.initials: C
authors.lastname: Hong
authors.forename: Nigam H
authors.identifier: http://orcid.org/0000-0001-9385-7158
authors.initials: NH
authors.lastname: Shah
keywords: Internet; geotagged search logs; health care costs; health care utilization; search behavior; utility