unit removing" methods on computer simulations. The results show that our fusing method considerably reduces the error increase due to the pruning, even in subminimal networks where conventional methods are ineffective. This enables to cut down the total cost of computation to reach the minmal network configuration." />


Neural Network Pruning by Fusing Hidden Layer Units

Keisuke KAMEYAMA  Yukio KOSUGI  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E74-D   No.12   pp.4198-4204
Publication Date: 1991/12/25
Online ISSN: 
DOI: 
Print ISSN: 0916-8532
Type of Manuscript: LETTER
Category: Bio-Cybernetics
Keyword: 


Full Text: PDF>>
Buy this Article




Summary: 
Neural network pruning is a technique to obtain a fully functional subset of a redundant network for the efficiency of computation. A new method to prune a redundant three-layered neural network by means of neural element fusion" is introduced. In contrast to conventional pruning techniques that remove unimportant portions of the network, our method fuses a pair of hidden layer units so that features accumulated in both units are preserved as possible. The pair of hidden layer units to be fused is chosen by evaluating a firing similarity. This similarity measure also informs when the pruning should be stopped. The fusing method was compared with well known unit removing" methods on computer simulations. The results show that our fusing method considerably reduces the error increase due to the pruning, even in subminimal networks where conventional methods are ineffective. This enables to cut down the total cost of computation to reach the minmal network configuration.