Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/80372
Title: Fair Resource Allocation for Data-Intensive Computing in the Cloud
Authors: Tang, Shanjiang
Lee, Bu-Sung
He, Bingsheng
Keywords: Computer Science and Engineering
Issue Date: 2016
Source: Tang, S., Lee, B.-S., & He, B. (2016). Fair Resource Allocation for Data-Intensive Computing in the Cloud. IEEE Transactions on Services Computing, in press.
Series/Report no.: IEEE Transactions on Services Computing
Abstract: To address the computing challenge of ’big data’, a number of data-intensive computing frameworks (e.g., MapReduce, Dryad, Storm and Spark) have emerged and become popular. YARN is a de facto resource management platform that enables these frameworks running together in a shared system. However, we observe that, in cloud computing environment, the fair resource allocation policy implemented in YARN is not suitable because of its memoryless resource allocation fashion leading to violations of a number of good properties in shared computing systems. This paper attempts to address these problems for YARN. Both singlelevel and hierarchical resource allocations are considered. For single-level resource allocation, we propose a novel fair resource allocation mechanism called Long-Term Resource Fairness (LTRF) for such computing. For hierarchical resource allocation, we propose Hierarchical Long-Term Resource Fairness (H-LTRF) by extending LTRF. We show that both LTRF and H-LTRF can address these fairness problems of current resource allocation policy and are thus suitable for cloud computing. Finally, we have developed LTYARN by implementing LTRF and H-LTRF in YARN, and our experiments show that it leads to a better resource fairness than existing fair schedulers of YARN.
URI: https://hdl.handle.net/10356/80372
http://hdl.handle.net/10220/40473
ISSN: 1939-1374
DOI: 10.1109/TSC.2016.2531698
Rights: © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: [http://dx.doi.org/10.1109/TSC.2016.2531698].
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Journal Articles

Files in This Item:
File Description SizeFormat 
Fair Resource Allocation for Data-Intensive Computing in the Cloud.pdf1.69 MBAdobe PDFThumbnail
View/Open

SCOPUSTM   
Citations

12
checked on Jul 16, 2020

WEB OF SCIENCETM
Citations 20

11
checked on Oct 13, 2020

Page view(s) 20

261
checked on Oct 19, 2020

Download(s) 20

298
checked on Oct 19, 2020

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.