Enabling big geoscience data analytics with a cloud-based, MapReduce-enabled and service-oriented workflow framework.
Geoscience observations and model simulations are generating vast amounts of multi-dimensional data. Effectively analyzing these data are essential for geoscience studies. However, the tasks are challenging for geoscientists because processing the massive amount of data is both computing and data intensive in that data analytics requires complex procedures and multiple tools. To tackle these challenges, a scientific workflow framework is proposed for big geoscience data analytics. In this framework techniques are proposed by leveraging cloud computing, MapReduce, and Service Oriented Architecture (SOA). Specifically, HBase is adopted for storing and managing big geoscience data across distributed computers. MapReduce-based algorithm framework is developed to support parallel processing of geoscience data. And service-oriented workflow architecture is built for supporting on-demand complex data analytics in the cloud environment. A proof-of-concept prototype tests the performance of the framework. Results show that this innovative framework significantly improves the efficiency of big geoscience data analytics by reducing the data processing time as well as simplifying data analytical procedures for geoscientists.
NSF Spatiotemporal Innovation Center, George Mason University, Fairfax, VA, United States of America.
Zhenlong
Z
Li
NSF Spatiotemporal Innovation Center, George Mason University, Fairfax, VA, United States of America.
Chaowei
C
Yang
NSF Spatiotemporal Innovation Center, George Mason University, Fairfax, VA, United States of America; Yunnan Provincial Geomatics Center, Yunnan Bureau of Surveying, Mapping, and GeoInformation, Kunming,Yunnan, China.
Baoxuan
B
Jin
NSF Spatiotemporal Innovation Center, George Mason University, Fairfax, VA, United States of America.
Manzhu
M
Yu
NSF Spatiotemporal Innovation Center, George Mason University, Fairfax, VA, United States of America.
Kai
K
Liu
NSF Spatiotemporal Innovation Center, George Mason University, Fairfax, VA, United States of America.
Min
M
Sun
NSF Spatiotemporal Innovation Center, George Mason University, Fairfax, VA, United States of America; Department of Computer Science, University of Texas-Austin, Austin, Texas, United States of America.
Matthew
M
Zhan
D000465:Algorithms; D000067917:Cloud Computing; D019295:Computational Biology / Q000379:methods*; D056448:Earth Sciences; D020407:Internet; D057188:Workflow