Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/142090
Title: Long-term multi-resource fairness for pay-as-you use computing systems
Authors: Tang, Shanjiang
Niu, Zhaojie
He, Bingsheng
Lee, Bu-Sung
Yu, Ce
Keywords: Engineering::Computer science and engineering
Issue Date: 2018
Source: Tang, S., Niu, Z., He, B., Lee, B.-S., & Yu, C. (2018). Long-term multi-resource fairness for pay-as-you use computing systems. IEEE Transactions on Parallel and Distributed Systems, 29(5), 1147-1160. doi:10.1109/tpds.2017.2788880
Journal: IEEE Transactions on Parallel and Distributed Systems
Abstract: Many current computing systems such as clouds and supercomputers charge users for their resource usages. A user's demand is often changing over time, indicating that it is difficult to keep the high resource utilization all the time for cost efficiency. Resource sharing is a classical and effective approach for high resource utilization. In view of the heterogeneous resource demands of users' workloads, multi-resource allocation fairness is a must for resource sharing in such pay-as-you-use computing systems. However, we find that, existing multi-resource fair policies such as Dominant Resource Fairness (DRF), implemented in currently popular resource management systems such as Apache YARN [4] and Mesos [23] , are not suitable for the pay-as-you-use computing systems. We show that this is because of their memoryless characteristic that can cause the following problems in the pay-as-you-use computing systems: 1). users can get resource benefits by cheating; 2). users might not be able to get the total amount of resources that they are entitled to in terms of their resource contributions. In this paper, we propose a new policy called H-MRF, which generalizes DRF and Asset Fairness with the long-term notion. We show that it can address these problems and is suitable for pay-as-you-use computing systems. We have implemented it into YARN by developing a prototype called MRYARN. Finally, we evaluate H-MRF using both testbed and simulated experiments. The experimental results show that there are about 1.1 ∼ 1.5 sharing benefit degrees and 1.2 × ∼ 1.8 × performance improvement for users with H-MRF, better than existing fair schedulers.
URI: https://hdl.handle.net/10356/142090
ISSN: 1045-9219
DOI: 10.1109/TPDS.2017.2788880
Schools: School of Computer Science and Engineering 
Rights: © 2017 IEEE. All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Journal Articles

SCOPUSTM   
Citations 20

17
Updated on Mar 25, 2024

Web of ScienceTM
Citations 20

13
Updated on Oct 27, 2023

Page view(s)

176
Updated on Mar 28, 2024

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.