Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/73967
Title: Visualisation and management tool for supercomputer resource
Authors: Shao, Yiyang
Keywords: DRNTU::Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
Issue Date: 2018
Abstract: The machine learning, especially deep learning has gained an astonishing amount of popularity over the last few years. Because of the success of machine learning in many other fields. The interest of applying machine learning to the job scheduling of high performance supercomputer has been raised. The reason is that current scheduling schemes are mostly heuristic based approach with various strategies such as backfilling and they are relatively fixed and not flexible enough to handle the changing characteristics of large amount of jobs belonging to different types. Therefore, a more dynamic job scheduler is needed. In this report, two of the recent machine learning approaches are examined on real world workload from National Supercomputer Center of Singapore(NSCC). One of them used a nonlinear regression machine learning strategy. The other one made use of the popular deep reinforcement learning technique. The nonlinear regression functions performed fairly well on the real world workload, showing its robustness and generalization ability. The deep reinforcement learning model could not easily adapt the real world workload due to its complicated architecture and difficulty and time consuming in training. Its performance on NSCC real world data is not ideal, either. Therefore, the nonlinear regression method is preferred to be further improved and adapted for production use.
URI: http://hdl.handle.net/10356/73967
Rights: Nanyang Technological University
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Student Reports (FYP/IA/PA/PI)

Files in This Item:
File Description SizeFormat 
FYP_report.pdf
  Restricted Access
2.19 MBAdobe PDFView/Open

Google ScholarTM

Check

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.