A Mathematical Solution to a Network Designing Problem

Yoshikane TAKAHASHI  

IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E78-A   No.10   pp.1381-1411
Publication Date: 1995/10/25
Online ISSN: 
Print ISSN: 0916-8508
Type of Manuscript: PAPER
Category: Neural Networks
neural networks,  network designing,  network training,  minimum intermediate units,  computational complexity,  rate distortion theory,  linear algebra,  

Full Text: PDF>>
Buy this Article

One of the major open issues in neural network research includes a Network Designing Problem (NDP): find a polynomial-time procedure that produces minimal structures (the minimum intermediate size, thresholds and synapse weights) of multilayer threshold feed-forward networks so that they can yield outputs consistent with given sample sets of input-output data. The NDP includes as a sub-problem a Network Training Problem (NTP) where the intermediate size is given. The NTP has been studied mainly by use of iterative algorithms of network training. This paper, making use of both rate distortion theory in information theory and linear algebra, solves the NDP mathematically rigorously. On the basis of this mathematical solution, it furthermore develops a mathematical solution Procedure to the NDP that computes the minimal structure straightforwardly from the sample set. The Procedure precisely attains the minimum intermediate size, although its computational time complexity can be of non-polynomial order at worst cases. The paper also refers to a polynomial-time shortcut to the Procedure for practical use that can reach an approximately minimum intermediate size with its error measurable. The shortcut, when the intermediate size is pre-specified, reduces to a promising alternative as well to current network training algorithms to the NTP.