For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
2PTS: A Two-Phase Task Scheduling Algorithm for MapReduce
Byungnam LIM Yeeun SHIM Yon Dohn CHUNG
IEICE TRANSACTIONS on Information and Systems
Publication Date: 2016/09/01
Online ISSN: 1745-1361
Type of Manuscript: LETTER
Category: Fundamentals of Information Systems
MapReduce, task scheduling algorithm, data locality,
Full Text: PDF(220.6KB)>>
For an efficient processing of large data in a distributed system, Hadoop MapReduce performs task scheduling such that tasks are distributed with consideration of the data locality. The data locality, however, is limitedly exploited, since it is pursued one node at a time basis without considering the global optimality. In this paper, we propose a novel task scheduling algorithm that globally considers the data locality. Through experiments, we show our algorithm improves the performance of MapReduce in various situations.