title: Estimation Accuracy on Execution Time of Run-Time Tasks in a Heterogeneous Distributed Environment.
abstract: Distributed Computing has achieved tremendous development since cloud computing was proposed in 2006, and played a vital role promoting rapid growth of data collecting and analysis models, e.g., Internet of things, Cyber-Physical Systems, Big Data Analytics, etc. Hadoop has become a data convergence platform for sensor networks. As one of the core components, MapReduce facilitates allocating, processing and mining of collected large-scale data, where speculative execution strategies help solve straggler problems. However, there is still no efficient solution for accurate estimation on execution time of run-time tasks, which can affect task allocation and distribution in MapReduce. In this paper, task execution data have been collected and employed for the estimation. A two-phase regression (TPR) method is proposed to predict the finishing time of each task accurately. Detailed data of each task have drawn interests with detailed analysis report being made. According to the results, the prediction accuracy of concurrent tasks' execution time can be improved, in particular for some regular jobs.
authors: Jiangsu Collaborative Innovation Centre of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science & Technology, Nanjing 210044, China. qi.liu@nuist.edu.cn.|School of Computer & Software, Nanjing University of Information Science & Technology, Nanjing 210044, China. qi.liu@nuist.edu.cn.
authors: Qi
authors: Q
authors: Liu
authors: School of Computer & Software, Nanjing University of Information Science & Technology, Nanjing 210044, China. cwdsuzhou@nuist.edu.cn.
authors: Weidong
authors: W
authors: Cai
authors: School of Computer & Software, Nanjing University of Information Science & Technology, Nanjing 210044, China. 18751971087@163.com.
authors: Dandan
authors: D
authors: Jin
authors: Jiangsu Engineering Centre of Network Monitoring, Nanjing University of Information Science and Technology, Nanjing 210044, China. s_shenjian@126.com.
authors: Jian
authors: J
authors: Shen
authors: Jiangsu Engineering Centre of Network Monitoring, Nanjing University of Information Science and Technology, Nanjing 210044, China. wwwfzj@126.com.
authors: Zhangjie
authors: Z
authors: Fu
authors: School of Computing, Edinburgh Napier University, 10 Colinton Road, Edinburgh EH10 5DT, UK. x.liu@napier.ac.uk.
authors: Xiaodong
authors: X
authors: Liu
authors: Computer Networking and Telecommunications Research Centre, University of Salford, Salford, Greater Manchester M5 4WT, UK. n.linge@salford.ac.uk.
authors: Nigel
authors: N
authors: Linge
keywords: MapReduce; cloud computing; data analysis; data convergence; speculative execution