title: Using statistical and machine learning to help institutions detect suspicious access to electronic health records.
abstract: OBJECTIVE
  To determine whether statistical and machine-learning methods, when applied to electronic health record (EHR) access data, could help identify suspicious (ie, potentially inappropriate) access to EHRs.
  
  
  METHODS
  From EHR access logs and other organizational data collected over a 2-month period, the authors extracted 26 features likely to be useful in detecting suspicious accesses. Selected events were marked as either suspicious or appropriate by privacy officers, and served as the gold standard set for model evaluation. The authors trained logistic regression (LR) and support vector machine (SVM) models on 10-fold cross-validation sets of 1291 labeled events. The authors evaluated the sensitivity of final models on an external set of 58 events that were identified as truly inappropriate and investigated independently from this study using standard operating procedures.
  
  
  RESULTS
  The area under the receiver operating characteristic curve of the models on the whole data set of 1291 events was 0.91 for LR, and 0.95 for SVM. The sensitivity of the baseline model on this set was 0.8. When the final models were evaluated on the set of 58 investigated events, all of which were determined as truly inappropriate, the sensitivity was 0 for the baseline method, 0.76 for LR, and 0.79 for SVM.
  
  
  CONCLUSIONS
  The LR and SVM models may not generalize because of interinstitutional differences in organizational structures, applications, and workflows. Nevertheless, our approach for constructing the models using statistical and machine-learning techniques can be generalized. An important limitation is the relatively small sample used for the training set due to the effort required for its construction.
  
  
  CONCLUSIONS
  The results suggest that statistical and machine-learning methods can play an important role in helping privacy officers detect suspicious accesses to EHRs.
authors.affiliation: Division of Biomedical Informatics, University of California San Diego School of Medicine, La Jolla, California 92093-0728, USA. aboxwala@ucsd.edu
authors.forename: Aziz A
authors.initials: AA
authors.lastname: Boxwala
authors.forename: Jihoon
authors.initials: J
authors.lastname: Kim
authors.forename: Janice M
authors.initials: JM
authors.lastname: Grillo
authors.forename: Lucila
authors.initials: L
authors.lastname: Ohno-Machado
mesh_terms: D001185:Artificial Intelligence; D016494:Computer Security; D057286:Electronic Health Records; D006801:Humans; D016015:Logistic Models; D008330:Management Audit / Q000379:methods*; D010865:Pilot Projects; D012680:Sensitivity and Specificity; D012986:Software Validation; D014481:United States