Please use this identifier to cite or link to this item:
Title: Evaluating the merits of ranking in structured network pruning
Authors: Sharma, Kuldeep
Ramakrishnan, Nirmala
Prakash, Alok
Lam, Siew-Kei
Srikanthan, Thambipillai
Keywords: Engineering::Computer science and engineering
Issue Date: 2020
Source: Sharma, K., Ramakrishnan, N., Prakash, A., Lam, S. & Srikanthan, T. (2020). Evaluating the merits of ranking in structured network pruning. IEEE International Conference on Distributed Computing Systems (ICDCS), 2020-November, 1389-1396.
Abstract: Pruning of channels in trained deep neural networks has been widely used to implement efficient DNNs that can be deployed on embedded/mobile devices. Majority of existing techniques employ criteria-based sorting of the channels to preserve salient channels during pruning as well as to automatically determine the pruned network architecture. However, recent studies on widely used DNNs, such as VGG-16, have shown that selecting and preserving salient channels using pruning criteria is not necessary since the plasticity of the network allows the accuracy to be recovered through fine-tuning. In this work, we further explore the value of the ranking criteria in pruning to show that if channels are removed gradually and iteratively, alternating with fine-tuning on the target dataset, ranking criteria are indeed not necessary to select redundant channels. Experimental results confirm that even a random selection of channels for pruning leads to similar performance (accuracy). In addition, we demonstrate that even a simple pruning technique that uniformly removes channels from all layers in the network, performs similar to existing ranking criteria-based approaches, while leading to lower inference time (GFLOPs). Our extensive evaluations include the context of embedded implementations of DNNs - specifically, on small networks such as SqueezeNet and at aggressive pruning percentages. We leverage these insights, to propose a GFLOPs-aware iterative pruning strategy that does not rely on any ranking criteria and yet can further lead to lower inference time by 15% without sacrificing accuracy.
ISBN: 9781728170022
DOI: 10.1109/ICDCS47774.2020.00183
Rights: © 2020 Institute of Electrical and Electronics Engineers (IEEE). All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Conference Papers

Page view(s)

Updated on Jul 25, 2021

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.