Loss-Driven Channel Pruning of Convolutional Neural Networks

Xin LONG  Xiangrong ZENG  Chen CHEN  Huaxin XIAO  Maojun ZHANG  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E103-D   No.5   pp.1190-1194
Publication Date: 2020/05/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2019EDL8200
Type of Manuscript: LETTER
Category: Artificial Intelligence, Data Mining
Keyword: 
channel pruning,  convolutional neural networks,  Taylor expansion,  fine-tuning,  iterative pruning,  

Full Text: PDF(491.3KB)>>
Buy this Article




Summary: 
The increase in computation cost and storage of convolutional neural networks (CNNs) severely hinders their applications on limited-resources devices in recent years. As a result, there is impending necessity to accelerate the networks by certain methods. In this paper, we propose a loss-driven method to prune redundant channels of CNNs. It identifies unimportant channels by using Taylor expansion technique regarding to scaling and shifting factors, and prunes those channels by fixed percentile threshold. By doing so, we obtain a compact network with less parameters and FLOPs consumption. In experimental section, we evaluate the proposed method in CIFAR datasets with several popular networks, including VGG-19, DenseNet-40 and ResNet-164, and experimental results demonstrate the proposed method is able to prune over 70% channels and parameters with no performance loss. Moreover, iterative pruning could be used to obtain more compact network.