Incremental Construction of Projection Generalizing Neural Networks

Masashi SUGIYAMA  Hidemitsu OGAWA  

IEICE TRANSACTIONS on Information and Systems   Vol.E85-D   No.9   pp.1433-1442
Publication Date: 2002/09/01
Online ISSN: 
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Biocybernetics, Neurocomputing
generalization capability,  incremental learning,  pseudo biorthogonal basis (PBOB),  projection learning,  projection generalizing neural network,  

Full Text: PDF>>
Buy this Article

In many practical situations in NN learning, training examples tend to be supplied one by one. In such situations, incremental learning seems more natural than batch learning in view of the learning methods of human beings. In this paper, we propose an incremental learning method in neural networks under the projection learning criterion. Although projection learning is a linear learning method, achieving the above goal is not straightforward since it involves redundant expressions of functions with over-complete bases, which is essentially related to pseudo biorthogonal bases (or frames). The proposed method provides exactly the same learning result as that obtained by batch learning. It is theoretically shown that the proposed method is more efficient in computation than batch learning.