Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance.
OBJECTIVE
To evaluate the phenotyping performance of three major electronic health record (EHR) components: International Classification of Disease (ICD) diagnosis codes, primary notes, and specific medications.


METHODS
We conducted the evaluation using de-identified Vanderbilt EHR data. We preselected ten diseases: atrial fibrillation, Alzheimer's disease, breast cancer, gout, human immunodeficiency virus infection, multiple sclerosis, Parkinson's disease, rheumatoid arthritis, and types 1 and 2 diabetes mellitus. For each disease, patients were classified into seven categories based on the presence of evidence in diagnosis codes, primary notes, and specific medications. Twenty-five patients per disease category (a total number of 175 patients for each disease, 1750 patients for all ten diseases) were randomly selected for manual chart review. Review results were used to estimate the positive predictive value (PPV), sensitivity, andF-score for each EHR component alone and in combination.


RESULTS
The PPVs of single components were inconsistent and inadequate for accurately phenotyping (0.06-0.71). Using two or more ICD codes improved the average PPV to 0.84. We observed a more stable and higher accuracy when using at least two components (mean ± standard deviation: 0.91 ± 0.08). Primary notes offered the best sensitivity (0.77). The sensitivity of ICD codes was 0.67. Again, two or more components provided a reasonably high and stable sensitivity (0.59 ± 0.16). Overall, the best performance (Fscore: 0.70 ± 0.12) was achieved by using two or more components. Although the overall performance of using ICD codes (0.67 ± 0.14) was only slightly lower than using two or more components, its PPV (0.71 ± 0.13) is substantially worse (0.91 ± 0.08).


CONCLUSIONS
Multiple EHR components provide a more consistent and higher performance than a single one for the selected phenotypes. We suggest considering multiple EHR components for future phenotyping design in order to obtain an ideal result.
Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA.
Wei-Qi
WQ
Wei
Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA.
Pedro L
PL
Teixeira
Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA.
Huan
H
Mo
Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA Department of Medicine, Vanderbilt University, Nashville, TN, USA.
Robert M
RM
Cronin
Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA Department of Medicine, Vanderbilt University, Nashville, TN, USA.
Jeremy L
JL
Warner
Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA Department of Medicine, Vanderbilt University, Nashville, TN, USA josh.denny@vanderbilt.edu.
Joshua C
JC
Denny
D000465:Algorithms; D003933:Diagnosis; D057286:Electronic Health Records; D006801:Humans; D038801:International Classification of Diseases; D008501:Medical Records, Problem-Oriented; D010641:Phenotype; D011237:Predictive Value of Tests
International Classification of Diseases; clinical notes; diagnosis codes; electronic health records; medications; phenotype; problem lists