Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/96035
Title: Processing moldable tasks on the grid : late job binding with lightweight user-level overlay
Authors: Mościcki, J. T.
Lamanna, M.
Sloot, Peter M. A.
Bubak, Marian
Issue Date: 2011
Source: Mościcki, J. T., Lamanna, M., Bubak, M., & Sloot, Peter M. A. (2011). Processing moldable tasks on the grid : late job binding with lightweight user-level overlay. Future generation computer systems, 27(6), 725-736.
Series/Report no.: Future generation computer systems
Abstract: Independent observations and everyday user experience indicate that performance and reliability of large grid infrastructures may suffer from large and unpredictable variations. In this paper we study the impact of the job queuing time on processing of moldable tasks which are commonly found in large-scale production grids. We use the mean value and variance of makespan as the quality of service indicators. We develop a general task processing model to provide a quantitative comparison between two models: early and late job binding in a user-level overlay applied to the EGEE Grid infrastructure. We find that the late-binding model effectively defines a transformation of the distribution of makespan according to the Central Limit Theorem. As demonstrated by Monte Carlo simulations using real job traces, this transformation allows to substantially reduce the mean value and variance of makespan. For certain classes of applications task granularity may be adjusted such that a speedup of an order of magnitude or more may be achieved. We use this result to propose a general strategy for managing access to resources and optimization of workload based on Ganga and DIANE user-level overlay tools. Key features of this approach include: a late-binding scheduler, an ability to interface to a wide range of distributed systems, an ability to extend and customize the system to cover application-specific scheduling and processing patterns and finally, ease of use and lightweight deployment in the user space. We discuss the impact of this approach for some practical applications where efficient processing of many tasks is required to solve scientific problems.
URI: https://hdl.handle.net/10356/96035
http://hdl.handle.net/10220/10124
DOI: 10.1016/j.future.2011.02.002
Rights: © 2011 Elsevier B.V.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Journal Articles

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.