Fault tolerance in feedforward neural networks using minimax optimization

Abstract

Neural computing is increasingly being proposed as a viable solution to various computational problems by emulating principles underlying the human brain. Neural networks, as parallel systems composed of numerous simple processing elements, offer advantages such as fault tolerance through distributed processing. In such systems, the failure of individual units has minimal impact on overall performance. Although neural networks are believed to be inherently fault tolerant due to their architecture, this property heavily depends on the training algorithm used. Most neural networks rely on variants of the back-propagation algorithm, which does not always yield fault-tolerant networks. This thesis introduces techniques to embed fault tolerance into feedforward neural networks, resulting in more robust systems capable of tolerating loss of node weights. The fault tolerance problem is formulated as a constrained minimization problem and addressed using two methods:

Minimax Optimization: The problem is transformed into a sequence of unconstrained, continuously differentiable functions, solved using efficient gradient-based methods with successive algorithmic improvements. l?l_\inftyl??-Norm Approximation: The objective function is approximated using the l?l_\inftyl??-norm and solved as an unconstrained minimization problem.

Networks trained using these methods demonstrate an acceptable degree of partial fault tolerance, enhancing their reliability in practical applications.