For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
Differential and Algebraic Geometry of Multilayer Perceptrons
Shun-ichi AMARI Tomoko OZEKI
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences
Publication Date: 2001/01/01
Print ISSN: 0916-8508
Type of Manuscript: INVITED PAPER (Special Section on the 10th Anniversary of the IEICE Transactions of Fundamentals: "Last Decade and 21st Century")
information geometry, multilayer perceptron, singularities, learning, natural gradient,
Full Text: PDF(385.7KB)>>
Information geometry is applied to the manifold of neural networks called multilayer perceptrons. It is important to study a total family of networks as a geometrical manifold, because learning is represented by a trajectory in such a space. The manifold of perceptrons has a rich differential-geometrical structure represented by a Riemannian metric and singularities. An efficient learning method is proposed by using it. The parameter space of perceptrons includes a lot of algebraic singularities, which affect trajectories of learning. Such singularities are studied by using simple models. This poses an interesting problem of statistical inference and learning in hierarchical models including singularities.