On the Activation Function and Fault Tolerance in Feedforward Neural Networks

Nait Charif HAMMADI  Hideo ITO  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E81-D   No.1   pp.66-72
Publication Date: 1998/01/25
Online ISSN: 
DOI: 
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Fault Tolerant Computing
Keyword: 
feedforward neural network,  XOR problem,  critical weights,  fault tolerance,  generalization ability,  

Full Text: PDF>>
Buy this Article




Summary: 
Considering the pattern classification/recognition tasks, the influence of the activation function on fault tolerance property of feedforward neural networks is empirically investigated. The simulation results show that the activation function largely influences the fault tolerance and the generalization property of neural networks. It is found that, neural networks with symmetric sigmoid activation function are largely fault tolerant than the networks with asymmetric sigmoid function. However the close relation between the fault tolerance and the generalization property was not observed and the networks with asymmetric activation function slightly generalize better than the networks with the symmetric activation function. First, the influence of the activation function on fault tolerance property of neural networks is investigated on the XOR problem, then the results are generalized by evaluating the fault tolerance property of different NNs implementing different benchmark problems.