| dc.description.abstract | The thesis deals with the problem of pattern classification for which new algorithms
using three different neural network models have been proposed: (i) Recursive
(ii) Feedforward; and (iii) Self-Organizing Networks. These models are
‘enhanced’ to provide a more efficient (in terms of speed, robustness and size of
the network) solution to the classification problem.
As far as the first neural network model (namely, recursive) is concerned, it is
well known that it achieves pattern classification through a process called ‘pattern
association’. In order to improve the performance of recursive networks for pattern
association, we have considered the following problems in their design.
— Given a set of patterns in the form of bipolar vectors, how to design a
recursive network that acts as an associative memory for these patterns,
with basins of attraction as large as possible?
— When all the given patterns cannot be accommodated in the network,
how to design the network such that maximal number of patterns are
stored in the network with basins of attraction as large as possible?
We propose two methods to solve them:
Development of optimal learning algorithms by showing the equivalence
between the learning in recursive networks and that in the (2-state) Perceptron.
With this equivalence, optimal learning algorithms for the Perceptron
can be directly applied to the problem of optimal learning in the
recursive neural network models. In addition to the existing learning algorithms
for the perceptron, we propose two other algorithms for optimal
learning in perceptron, based on Thermal Perceptron Learning Rule (TPLR)
[1], and compare them, as applied to the problem of recursive network
design.
Modification of the dynamics of the network by using a finite number of the
previous state values at each node. It has been shown, empirically, that the
new network has larger regions of attraction around the equilibrium points.
Convergence conditions for the network have also been established.
The second model of networks (namely, feedforward) is the most widely used
network architecture for pattern classification. The input to these networks is
in the form of real-valued feature vectors. In the design of such networks, the
basic issue is to determine the appropriate network size for a given problem: the
network size should be sufficiently large to be able to realize the required input-output
mapping, but should be small enough to give good generalization results.
In the literature, constructive learning algorithms offer a solution to this problem.
These algorithms, instead of training a network of fixed 0, start with a network
of a small size, and add neurons as and when required.
We develop new constructive learning algorithms for the construction of feedforward
neural networks whose size is near-optimal:
Modified Upstart and Tower algorithms which use the TPLR for training
the hidden neurons; and
A modification of the TPLR, called Biased TPLR (BTPLR) for training
the hidden neurons in the design of the networks using the paradigm of
Sequential Learning [2].
An advantage of the latter (namely, BTPLR) is that it can be applied to the
construction of networks where the hidden neurons have activation functions other
than the threshold functions, such as the window and cluster activation functions.
For window neurons, the BTPLR gives results marginally superior to those of [3].
We demonstrate, by simulation studies, the superiority of the proposed algorithms
(in terms of network complexity, conceptual simplicity, and ease in implementation)
over similar algorithms of the literature. Amongst the algorithms
for networks with threshold neurons, the Sequential Learning Algorithm (SLA)
with BTPLR gives networks with the smallest number of hidden neurons, and
is faster than the SLA of [2]. The networks generated by the Tower Algorithm
also give a performance comparable to the algorithms based on SLA, except when
applied to the problem of random Boolean mappings.
After some minor modifications, all the proposed algorithms can be used for
problems with real-valued data and with multiple classes also.
Finally, based on the third model (namely, self-organization), a novel method
has been proposed for the classification of patterns given in the form of 2-D binary
images. For each of the exemplar patterns, a network of neurons is constructed,
with its neurons arranged in exactly the same way as the corresponding exemplar.
Each network is mapped onto the given test pattern using a self-organization
scheme. Some measures of mapping are proposed based on which classification is
carried out. The technique has been applied to the problem of object recognition.
The results obtained show its efficacy and robustness.
The thesis is organized as follows: Chapter 1 gives an introduction to the
three neural network models discussed in the thesis. Techniques for improving
the performance of the recursive networks for pattern association are discussed
in Chapter 2. In Chapter 3, we propose new constructive learning algorithms
for the design of feedforward neural networks for pattern classification. A new
self-organizing architecture for pattern recognition is proposed in Chapter 4. The
thesis ends with conclusions in Chapter 5. | |
