Stable Modeling on Resource Usage Parameters of MapReduce Application

Authors

  • Yangyuan Li Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, Hungary

Keywords:

MapReduce application, resource bottleneck, resource usage parameters, multiple linear regression, stable modeling, minimum sampling time

Abstract

Currently, Hadoop MapReduce framework has been applied to many productive fields to analyze big data. MapReduce applications based on the MapReduce programming model are used to generate and process such huge data. Due to various computational purpose, MapReduce applications have different resource requirements. For specific applications, the resource bottleneck of the cloud computing platform must inevitably impact its executive performance. Therefore, identification of the bottleneck about the allocated resource for MapReduce applications is crucially needed from the viewpoint of either cloud operators or program developers. In this paper, we model the relationship of resource usage parameters of MapReduce applications using multiple linear regression methods and investigate the minimum sampling time for stable modeling. Based on the analysis, we propose the approach which can be used to build stable performance model to expose the bottleneck resource of Hadoop platform and give the effective optimization suggestion.

Author Biography

Yangyuan Li, Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, Hungary

Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, Hungary H-1117, Magyar tudósok körútja 2, Budapest, Hungary Department of Computer Science, Xi’an Siyuan University, China 710038, Xi’an, China wlqsb@hotmail.com

Downloads

Published

2018-05-08

How to Cite

Li, Y. (2018). Stable Modeling on Resource Usage Parameters of MapReduce Application. BRAIN. Broad Research in Artificial Intelligence and Neuroscience, 9(2), pp. 45-62. Retrieved from https://lumenpublishing.com/journals/index.php/brain/article/view/2032

Publish your work at the Scientific Publishing House LUMEN

It easy with us: publish now your work, novel, research, proceeding at Lumen Scientific Publishing House

Send your manuscript right now