For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
Fast Convergent Genetic-Type Search for Multi-Layered Network
Shu-Hung LEUNG Andrew LUK Sin-Chun NG
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences
Publication Date: 1994/09/25
Print ISSN: 0916-8508
Type of Manuscript: Special Section PAPER (Special Section of Papers Selected from the 8th Digital Signal Processing Symposium)
Category: Neural Networks
back-propagation, genetic algorithm, weight evolution, multi-layered neural network,
Full Text: PDF>>
The classical supervised learning algorithms for optimizing multi-layered feedforward neural networks, such at the original back-propagation algorithm, suffer from several weaknesses. First, they have the possibility of being trapped at local minima during learning, which may lead to failure in finding the global optimal solution. Second, the convergence rate is typically too slow even if the learning can be achieved. This paper introduces a new learning algorithm which employs a genetic-type search during the learning phase of back-propagation algorithm so that the above problems can be overcome. The basic idea is to evolve the network weights in a controlled manner so as to jump to the regions of smaller mean squared error whenever the back-propagation stops at a local minimum. By this, the local minima can always be escaped and a much faster learning with global optimal solution can be achieved. A mathematical framework on the weight evolution of the new algorithm in also presented in this paper, which gives a careful analysis on the requirements of weight evolution (or perturbation) during learning in order to achieve a better error performance in the weights between different hidden layers. Simulation results on three typical problems including XOR, 3-bit parity and the counting problem are described to illustrate the fast learning behaviour and the global search capability of the new algorithm in improving the performance of back-propagated network.