Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/101493
Title: A map-reduce based framework for heterogeneous processing element cluster environments
Authors: Tan, Yu Shyang
Lee, Bu-Sung
He, Bingsheng
Campbell, Roy H.
Keywords: DRNTU::Engineering::Computer science and engineering
Issue Date: 2012
Source: Tan, Y. S., Lee, B.-S., He, B., & Campbell, R. H. (2012). A Map-Reduce Based Framework for Heterogeneous Processing Element Cluster Environments. 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 57-64.
Abstract: In this paper, we present our design of a Processing Element (PE) Aware MapReduce base framework, Pamar. Pamar is designed for supporting distributed computing on clusters where node PE configurations are asymmetric on different nodes. Pamar's main goal is to allow users to seamlessly utilize different kinds of processing elements (e.g., CPUs or GPUs) collaboratively for large scale data processing. To show proof of concept, we have incorporated our designs into the Hadoop framework and tested it on cluster environments having asymmetric node PE configurations. We demonstrate Pamar's ability to identify PEs available on each node and match-make user jobs with nodes, base on job PE requirements. Pamar allows users to easily parallelize applications across large datasets and at the same time utilizes different PEs for processing different classes of functions efficiently. The experiments show improvement in job queue completion time with Pamar over clusters with asymmetric nodes as compared to clusters with symmetric nodes.
URI: https://hdl.handle.net/10356/101493
http://hdl.handle.net/10220/16728
DOI: http://dx.doi.org/10.1109/CCGrid.2012.35
Rights: © 2012 IEEE
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Conference Papers

Google ScholarTM

Check

Altmetric

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.