A Spectral Clustering Based Filter-Level Pruning Method for Convolutional Neural Networks

Lianqiang LI  Jie ZHU  Ming-Ting SUN  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E102-D   No.12   pp.2624-2627
Publication Date: 2019/12/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2019EDL8118
Type of Manuscript: LETTER
Category: Artificial Intelligence, Data Mining
Keyword: 
convolutional neural network,  spectral clustering,  filter-level,  pruning,  

Full Text: PDF(954.6KB)>>
Buy this Article



Summary: 
Convolutional Neural Networks (CNNs) usually have millions or even billions of parameters, which make them hard to be deployed into mobile devices. In this work, we present a novel filter-level pruning method to alleviate this issue. More concretely, we first construct an undirected fully connected graph to represent a pre-trained CNN model. Then, we employ the spectral clustering algorithm to divide the graph into some subgraphs, which is equivalent to clustering the similar filters of the CNN into the same groups. After gaining the grouping relationships among the filters, we finally keep one filter for one group and retrain the pruned model. Compared with previous pruning methods that identify the redundant filters by heuristic ways, the proposed method can select the pruning candidates more reasonably and precisely. Experimental results also show that our proposed pruning method has significant improvements over the state-of-the-arts.