title: Leverage hadoop framework for large scale clinical informatics applications.
abstract: In this manuscript, we present our experiences using the Apache Hadoop framework for high data volume and computationally intensive applications, and discuss some best practice guidelines in a clinical informatics setting. There are three main aspects in our approach: (a) process and integrate diverse, heterogeneous data sources using standard Hadoop programming tools and customized MapReduce programs; (b) after fine-grained aggregate results are obtained, perform data analysis using the Mahout data mining library; (c) leverage the column oriented features in HBase for patient centric modeling and complex temporal reasoning. This framework provides a scalable solution to meet the rapidly increasing, imperative "Big Data" needs of clinical and translational research. The intrinsic advantage of fault tolerance, high availability and scalability of Hadoop platform makes these applications readily deployable at the enterprise level cluster environment.
authors.affiliation: University of Illinois at Chicago, Chicago, IL.
authors.forename: Xiao
authors.initials: X
authors.lastname: Dong
authors.forename: Neil
authors.initials: N
authors.lastname: Bahroos
authors.forename: Eugene
authors.initials: E
authors.lastname: Sadhu
authors.forename: Tommie
authors.initials: T
authors.lastname: Jackson
authors.forename: Morris
authors.initials: M
authors.lastname: Chukhman
authors.forename: Robert
authors.initials: R
authors.lastname: Johnson
authors.forename: Andrew
authors.initials: A
authors.lastname: Boyd
authors.forename: Denise
authors.initials: D
authors.lastname: Hynes