IEICE Trans - A Modified Information Criterion for Automatic Model and Parameter Selection in Neural Network Learning

A Modified Information Criterion for Automatic Model and Parameter Selection in Neural Network Learning

Sumio WATANABE

Publication
IEICE TRANSACTIONS on Information and Systems Vol.E78-D No.4 pp.490-499
Publication Date: 1995/04/25
Online ISSN:
DOI:
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Bio-Cybernetics and Neurocomputing
Keyword:
information criterion, AIC, MDL, weight pruning, prediction error, generalized learning,

Full Text: PDF>>

Summary:
This paper proposes a practical training algorithm for artificial neural networks, by which both the optimally pruned model and the optimally trained parameter for the minimum prediction error can be found simultaneously. In the proposed algorithm, the conventional information criterion is modified into a differentiable function of weight parameters, and then it is minimized while being controlled back to the conventional form. Since this method has several theoretical problems, its effectiveness is examined by computer simulations and by an application to practical ultrasonic image reconstruction.