Please use this identifier to cite or link to this item:
https://hdl.handle.net/10356/142090
Title: | Long-term multi-resource fairness for pay-as-you use computing systems | Authors: | Tang, Shanjiang Niu, Zhaojie He, Bingsheng Lee, Bu-Sung Yu, Ce |
Keywords: | Engineering::Computer science and engineering | Issue Date: | 2018 | Source: | Tang, S., Niu, Z., He, B., Lee, B.-S., & Yu, C. (2018). Long-term multi-resource fairness for pay-as-you use computing systems. IEEE Transactions on Parallel and Distributed Systems, 29(5), 1147-1160. doi:10.1109/tpds.2017.2788880 | Journal: | IEEE Transactions on Parallel and Distributed Systems | Abstract: | Many current computing systems such as clouds and supercomputers charge users for their resource usages. A user's demand is often changing over time, indicating that it is difficult to keep the high resource utilization all the time for cost efficiency. Resource sharing is a classical and effective approach for high resource utilization. In view of the heterogeneous resource demands of users' workloads, multi-resource allocation fairness is a must for resource sharing in such pay-as-you-use computing systems. However, we find that, existing multi-resource fair policies such as Dominant Resource Fairness (DRF), implemented in currently popular resource management systems such as Apache YARN [4] and Mesos [23] , are not suitable for the pay-as-you-use computing systems. We show that this is because of their memoryless characteristic that can cause the following problems in the pay-as-you-use computing systems: 1). users can get resource benefits by cheating; 2). users might not be able to get the total amount of resources that they are entitled to in terms of their resource contributions. In this paper, we propose a new policy called H-MRF, which generalizes DRF and Asset Fairness with the long-term notion. We show that it can address these problems and is suitable for pay-as-you-use computing systems. We have implemented it into YARN by developing a prototype called MRYARN. Finally, we evaluate H-MRF using both testbed and simulated experiments. The experimental results show that there are about 1.1 ∼ 1.5 sharing benefit degrees and 1.2 × ∼ 1.8 × performance improvement for users with H-MRF, better than existing fair schedulers. | URI: | https://hdl.handle.net/10356/142090 | ISSN: | 1045-9219 | DOI: | 10.1109/TPDS.2017.2788880 | Schools: | School of Computer Science and Engineering | Rights: | © 2017 IEEE. All rights reserved. | Fulltext Permission: | none | Fulltext Availability: | No Fulltext |
Appears in Collections: | SCSE Journal Articles |
SCOPUSTM
Citations
20
17
Updated on Mar 25, 2024
Web of ScienceTM
Citations
20
13
Updated on Oct 27, 2023
Page view(s)
176
Updated on Mar 28, 2024
Google ScholarTM
Check
Altmetric
Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.