
For FullText PDF, please login, if you are a member of IEICE,
or go to Pay Per View on menu list, if you are a nonmember of IEICE.

Fast Convergent GeneticType Search for MultiLayered Network
ShuHung LEUNG Andrew LUK SinChun NG
Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences
Vol.E77A
No.9
pp.14841492 Publication Date: 1994/09/25 Online ISSN:
DOI: Print ISSN: 09168508 Type of Manuscript: Special Section PAPER (Special Section of Papers Selected from the 8th Digital Signal Processing Symposium) Category: Neural Networks Keyword: backpropagation, genetic algorithm, weight evolution, multilayered neural network,
Full Text: PDF(771.2KB)>>
Summary:
The classical supervised learning algorithms for optimizing multilayered feedforward neural networks, such at the original backpropagation algorithm, suffer from several weaknesses. First, they have the possibility of being trapped at local minima during learning, which may lead to failure in finding the global optimal solution. Second, the convergence rate is typically too slow even if the learning can be achieved. This paper introduces a new learning algorithm which employs a genetictype search during the learning phase of backpropagation algorithm so that the above problems can be overcome. The basic idea is to evolve the network weights in a controlled manner so as to jump to the regions of smaller mean squared error whenever the backpropagation stops at a local minimum. By this, the local minima can always be escaped and a much faster learning with global optimal solution can be achieved. A mathematical framework on the weight evolution of the new algorithm in also presented in this paper, which gives a careful analysis on the requirements of weight evolution (or perturbation) during learning in order to achieve a better error performance in the weights between different hidden layers. Simulation results on three typical problems including XOR, 3bit parity and the counting problem are described to illustrate the fast learning behaviour and the global search capability of the new algorithm in improving the performance of backpropagated network.

