title: Collaborative knowledge acquisition for the design of context-aware alert systems.
abstract: OBJECTIVE
  To present a framework for combining implicit knowledge acquisition from multiple experts with machine learning and to evaluate this framework in the context of anemia alerts.
  
  
  METHODS
  Five internal medicine residents reviewed 18 anemia alerts, while 'talking aloud'. They identified features that were reviewed by two or more physicians to determine appropriate alert level, etiology and treatment recommendation. Based on these features, data were extracted from 100 randomly-selected anemia cases for a training set and an additional 82 cases for a test set. Two staff internists assigned an alert level, etiology and treatment recommendation before and after reviewing the entire electronic medical record. The training set of 118 cases (100 plus 18) and the test set of 82 cases were explored using RIDOR and JRip algorithms.
  
  
  RESULTS
  The feature set was sufficient to assess 93% of anemia cases (intraclass correlation for alert level before and after review of the records by internists 1 and 2 were 0.92 and 0.95, respectively). High-precision classifiers were constructed to identify low-level alerts (precision p=0.87, recall R=0.4), iron deficiency (p=1.0, R=0.73), and anemia associated with kidney disease (p=0.87, R=0.77).
  
  
  CONCLUSIONS
  It was possible to identify low-level alerts and several conditions commonly associated with chronic anemia. This approach may reduce the number of clinically unimportant alerts. The study was limited to anemia alerts. Furthermore, clinicians were aware of the study hypotheses potentially biasing their evaluation.
  
  
  CONCLUSIONS
  Implicit knowledge acquisition, collaborative filtering and machine learning were combined automatically to induce clinically meaningful and precise decision rules.
authors: Tel Aviv Sourasky Medical Center, Tel Aviv, Israel.
authors: Erel
authors: E
authors: Joffe
authors: Ofer
authors: O
authors: Havakuk
authors: Jorge R
authors: JR
authors: Herskovic
authors: Vimla L
authors: VL
authors: Patel
authors: Elmer Victor
authors: EV
authors: Bernstam
mesh_terms: D000740:Anemia / Q000517:prevention & control*; D001185:Artificial Intelligence; D020000:Decision Support Systems, Clinical; D003936:Diagnosis, Computer-Assisted; D057286:Electronic Health Records; D006801:Humans; D007388:Internal Medicine; D007557:Israel; D010818:Practice Patterns, Physicians'