Fast Convergent Genetic-Type Search for Multi-Layered Network

Shu-Hung LEUNG  Andrew LUK  Sin-Chun NG  

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E77-A   No.9   pp.1484-1492
Publication Date: 1994/09/25
Online ISSN: 
DOI: 
Print ISSN: 0916-8508
Type of Manuscript: Special Section PAPER (Special Section of Papers Selected from the 8th Digital Signal Processing Symposium)
Category: Neural Networks
Keyword: 
back-propagation,  genetic algorithm,  weight evolution,  multi-layered neural network,  

Full Text: PDF(771.2KB)>>
Buy this Article



Summary: 
The classical supervised learning algorithms for optimizing multi-layered feedforward neural networks, such at the original back-propagation algorithm, suffer from several weaknesses. First, they have the possibility of being trapped at local minima during learning, which may lead to failure in finding the global optimal solution. Second, the convergence rate is typically too slow even if the learning can be achieved. This paper introduces a new learning algorithm which employs a genetic-type search during the learning phase of back-propagation algorithm so that the above problems can be overcome. The basic idea is to evolve the network weights in a controlled manner so as to jump to the regions of smaller mean squared error whenever the back-propagation stops at a local minimum. By this, the local minima can always be escaped and a much faster learning with global optimal solution can be achieved. A mathematical framework on the weight evolution of the new algorithm in also presented in this paper, which gives a careful analysis on the requirements of weight evolution (or perturbation) during learning in order to achieve a better error performance in the weights between different hidden layers. Simulation results on three typical problems including XOR, 3-bit parity and the counting problem are described to illustrate the fast learning behaviour and the global search capability of the new algorithm in improving the performance of back-propagated network.