Please use this identifier to cite or link to this item:
Title: Fair Resource Allocation for Data-Intensive Computing in the Cloud
Authors: Tang, Shanjiang
Lee, Bu-Sung
He, Bingsheng
Keywords: Computer Science and Engineering
Issue Date: 2016
Source: Tang, S., Lee, B.-S., & He, B. (2016). Fair Resource Allocation for Data-Intensive Computing in the Cloud. IEEE Transactions on Services Computing, in press.
Series/Report no.: IEEE Transactions on Services Computing
Abstract: To address the computing challenge of ’big data’, a number of data-intensive computing frameworks (e.g., MapReduce, Dryad, Storm and Spark) have emerged and become popular. YARN is a de facto resource management platform that enables these frameworks running together in a shared system. However, we observe that, in cloud computing environment, the fair resource allocation policy implemented in YARN is not suitable because of its memoryless resource allocation fashion leading to violations of a number of good properties in shared computing systems. This paper attempts to address these problems for YARN. Both singlelevel and hierarchical resource allocations are considered. For single-level resource allocation, we propose a novel fair resource allocation mechanism called Long-Term Resource Fairness (LTRF) for such computing. For hierarchical resource allocation, we propose Hierarchical Long-Term Resource Fairness (H-LTRF) by extending LTRF. We show that both LTRF and H-LTRF can address these fairness problems of current resource allocation policy and are thus suitable for cloud computing. Finally, we have developed LTYARN by implementing LTRF and H-LTRF in YARN, and our experiments show that it leads to a better resource fairness than existing fair schedulers of YARN.
ISSN: 1939-1374
DOI: 10.1109/TSC.2016.2531698
Schools: School of Computer Engineering 
Rights: © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: [].
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Journal Articles

Files in This Item:
File Description SizeFormat 
Fair Resource Allocation for Data-Intensive Computing in the Cloud.pdf1.69 MBAdobe PDFThumbnail

Citations 10

Updated on Sep 26, 2023

Web of ScienceTM
Citations 10

Updated on Sep 25, 2023

Page view(s) 50

Updated on Sep 29, 2023

Download(s) 10

Updated on Sep 29, 2023

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.