Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/149382
Title: Performance profiling and optimizations in distributed deep learning frameworks
Authors: Zhang, Jiarui
Keywords: Engineering::Electrical and electronic engineering
Issue Date: 2021
Publisher: Nanyang Technological University
Source: Zhang, J. (2021). Performance profiling and optimizations in distributed deep learning frameworks. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/149382
Project: B3137-201
Abstract: Deep learning has been a very popular topic in Artificial Intelligent industry these years and can be applied to many fields, such as computer vision, natural language processing and so forth. However, training a deep learning model usually takes lots of time. It is necessary to identify the bottleneck of the deep learning process and implement optimizations on them to improve the training efficiency, especially the training speed. Usually, optimizations are implemented in two aspects: data processing and model training. In this work, multiple optimization methods are studied and conducted to check their corresponding effect. Regarding data processing, optimizations such as parallelization of multiple transforming processes, dataset caching, prefetching of data samples are implemented. Regarding training, data parallelism of distributed training is especially studied, and two current popular frameworks are utilized to achieve it. Experiments are conducted to compare the two frameworks and analyze possible influencing factors’ effect on the training speed.
URI: https://hdl.handle.net/10356/149382
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:EEE Student Reports (FYP/IA/PA/PI)

Files in This Item:
File Description SizeFormat 
ZhangJiarui FYP Report.pdf
  Restricted Access
1.92 MBAdobe PDFView/Open

Page view(s)

143
Updated on Dec 9, 2022

Download(s)

3
Updated on Dec 9, 2022

Google ScholarTM

Check

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.