Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/80385
Full metadata record
DC FieldValueLanguage
dc.contributor.authorTang, Shanjiangen
dc.contributor.authorLee, Bu-Sungen
dc.contributor.authorHe, Bingshengen
dc.date.accessioned2016-06-13T06:29:20Zen
dc.date.accessioned2019-12-06T13:48:22Z-
dc.date.available2016-06-13T06:29:20Zen
dc.date.available2019-12-06T13:48:22Z-
dc.date.copyright2016en
dc.date.issued2016en
dc.identifier.citationTang, S., Lee, B.-S., & He, B. (2016). Dynamic Job Ordering and Slot Configurations for MapReduce Workloads. IEEE Transactions on Services Computing, 9(1), 4-17.en
dc.identifier.issn1939-1374en
dc.identifier.urihttps://hdl.handle.net/10356/80385-
dc.description.abstractMapReduce is a popular parallel computing paradigm for large-scale data processing in clusters and data centers. A MapReduce workload generally contains a set of jobs, each of which consists of multiple map tasks followed by multiple reduce tasks. Due to 1) that map tasks can only run in map slots and reduce tasks can only run in reduce slots, and 2) the general execution constraints that map tasks are executed before reduce tasks, different job execution orders and map/reduce slot configurations for a MapReduce workload have significantly different performance and system utilization. This paper proposes two classes of algorithms to minimize the makespan and the total completion time for an offline MapReduce workload. Our first class of algorithms focuses on the job ordering optimization for a MapReduce workload under a given map/reduce slot configuration. In contrast, our second class of algorithms considers the scenario that we can perform optimization for map/reduce slot configuration for a MapReduce workload. We perform simulations as well as experiments on Amazon EC2 and show that our proposed algorithms produce results that are up to 15 ~ 80 percent better than currently unoptimized Hadoop, leading to significant reductions in running time in practice.en
dc.language.isoenen
dc.relation.ispartofseriesIEEE Transactions on Services Computingen
dc.rights© 2016 IEEE.en
dc.subjectflow-shopsen
dc.subjectscheduling algorithmen
dc.subjectjob orderingen
dc.subjectMapReduceen
dc.subjectHadoopen
dc.titleDynamic Job Ordering and Slot Configurations for MapReduce Workloadsen
dc.typeJournal Articleen
dc.contributor.schoolSchool of Computer Engineeringen
dc.identifier.doi10.1109/TSC.2015.2426186en
dc.identifier.rims187084en
item.fulltextNo Fulltext-
item.grantfulltextnone-
Appears in Collections:SCSE Journal Articles

SCOPUSTM   
Citations 20

30
Updated on Jul 8, 2024

Web of ScienceTM
Citations 20

20
Updated on Oct 28, 2023

Page view(s) 20

737
Updated on Jul 13, 2024

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.