A Modified Information Criterion for Automatic Model and Parameter Selection in Neural Network Learning

Sumio WATANABE  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E78-D   No.4   pp.490-499
Publication Date: 1995/04/25
Online ISSN: 
DOI: 
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Bio-Cybernetics and Neurocomputing
Keyword: 
information criterion,  AIC,  MDL,  weight pruning,  prediction error,  generalized learning,  

Full Text: PDF>>
Buy this Article



Summary: 
This paper proposes a practical training algorithm for artificial neural networks, by which both the optimally pruned model and the optimally trained parameter for the minimum prediction error can be found simultaneously. In the proposed algorithm, the conventional information criterion is modified into a differentiable function of weight parameters, and then it is minimized while being controlled back to the conventional form. Since this method has several theoretical problems, its effectiveness is examined by computer simulations and by an application to practical ultrasonic image reconstruction.