title: A genetic algorithm-based job scheduling model for big data analytics.
abstract: Big data analytics (BDA) applications are a new category of software applications that process large amounts of data using scalable parallel processing infrastructure to obtain hidden value. Hadoop is the most mature open-source big data analytics framework, which implements the MapReduce programming model to process big data with MapReduce jobs. Big data analytics jobs are often continuous and not mutually separated. The existing work mainly focuses on executing jobs in sequence, which are often inefficient and consume high energy. In this paper, we propose a genetic algorithm-based job scheduling model for big data analytics applications to improve the efficiency of big data analytics. To implement the job scheduling model, we leverage an estimation module to predict the performance of clusters when executing analytics jobs. We have evaluated the proposed job scheduling model in terms of feasibility and accuracy.
authors.affiliation: College of Computer and Communication Engineering, China University of Petroleum, Qingdao, China.
authors.forename: Qinghua
authors.initials: Q
authors.lastname: Lu
authors.affiliation: College of Computer and Communication Engineering, China University of Petroleum, Qingdao, China.
authors.forename: Shanshan
authors.initials: S
authors.lastname: Li
authors.affiliation: College of Computer and Communication Engineering, China University of Petroleum, Qingdao, China.
authors.forename: Weishan
authors.initials: W
authors.lastname: Zhang
authors.affiliation: College of Computer and Communication Engineering, China University of Petroleum, Qingdao, China.
authors.forename: Lei
authors.initials: L
authors.lastname: Zhang
keywords: Big data; Genetic algorithm; Hadoop; Job scheduling; MapReduce