
For FullText PDF, please login, if you are a member of IEICE,
or go to Pay Per View on menu list, if you are a nonmember of IEICE.

A Mathematical Solution to a Network Designing Problem
Yoshikane TAKAHASHI
Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences
Vol.E78A
No.10
pp.13811411 Publication Date: 1995/10/25
Online ISSN:
DOI:
Print ISSN: 09168508 Type of Manuscript: PAPER Category: Neural Networks Keyword: neural networks, network designing, network training, minimum intermediate units, computational complexity, rate distortion theory, linear algebra,
Full Text: PDF>>
Summary:
One of the major open issues in neural network research includes a Network Designing Problem (NDP): find a polynomialtime procedure that produces minimal structures (the minimum intermediate size, thresholds and synapse weights) of multilayer threshold feedforward networks so that they can yield outputs consistent with given sample sets of inputoutput data. The NDP includes as a subproblem a Network Training Problem (NTP) where the intermediate size is given. The NTP has been studied mainly by use of iterative algorithms of network training. This paper, making use of both rate distortion theory in information theory and linear algebra, solves the NDP mathematically rigorously. On the basis of this mathematical solution, it furthermore develops a mathematical solution Procedure to the NDP that computes the minimal structure straightforwardly from the sample set. The Procedure precisely attains the minimum intermediate size, although its computational time complexity can be of nonpolynomial order at worst cases. The paper also refers to a polynomialtime shortcut to the Procedure for practical use that can reach an approximately minimum intermediate size with its error measurable. The shortcut, when the intermediate size is prespecified, reduces to a promising alternative as well to current network training algorithms to the NTP.

