For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
Dominant Fairness Fairness: Hierarchical Scheduling for Multiple Resources in Heterogeneous Datacenters
Wenzhu WANG Kun JIANG Yusong TAN Qingbo WU
IEICE TRANSACTIONS on Information and Systems
Publication Date: 2016/06/01
Online ISSN: 1745-1361
Type of Manuscript: LETTER
Category: Fundamentals of Information Systems
hierarchical scheduling, heterogeneous datacenter, multiple resources, H-DRF,
Full Text: PDF(225.2KB)>>
Hierarchical scheduling for multiple resources is partially responsible for the performance achievements in large scale datacenters. However, the latest scheduling technique, Hierarchy Dominant Resource Fairness (H-DRF), has some shortcomings in heterogeneous environments, such as starving certain jobs or unfair resource allocation. This is because a heterogeneous environment brings new challenges. In this paper, we propose a novel scheduling algorithm called Dominant Fairness Fairness (DFF). DFF tries to keep resource allocation fair, avoid job starvation, and improve system resource utilization. We implement DFF in the YARN system, a most commonly used scheduler for large scale clusters. The experimental results show that our proposed algorithm leads to higher resource utilization and better throughput than H-DRF.