Pattern recognition techniques based on self-Organization and learning vector Quantization

Abstract

Autom atic Recognition of Patterns (APR) by a machine is perhaps the most challenging problem in Artificial Intelligence. The leitmotiv for the design of such a machine comes from the human visual system which is endowed with astonishing versatility, and constitutes the ultimate physical (albeit neural) realization of a pattern recognition system whose performance is not affected by geometric {shift, scale and rotation) transformations of patterns, like characters in various styles and sizes. The performance of standard pattern recognition techniques found in the literature stands no comparison with that of the human visual system. Naturally, attem pts are being made to design an Artificial Neural Network (ANN) for imitating human vision. The thesis deals with some aspects of the application of ANN*s for pattern recognition. Based on the neuro-piiysiological finding.s, it is now known that the human visual system has a hicnu'cliical structiu*c, in which simpl(' patt(u'u featui'o extraction in the early layers is followed by integration, in the higher layers, into more comi>licated versions. Tliis structure, which was invoked by Fukushima 7] [8] [9] [10] in liis Neocognitron (NC) to imitate human vision, acts as the motivation for our work. On the bcisis of extensive simulation studies on the NC, we have come out with significant modifications to the NC with respect to (i) its inhibition strategy and when to use it ; (it) extraction o f prim itive fea tu res ; (Hi) responses o f simple and com plex cells ; (iv) architectm^e ; and (v) training. It turns out that these improvements to the NC are not still adequate to deal with problems of pattern recognition. In an attem pt to create an ANN to recognize scaled and rotated patterns, and to improve its performance, a new approach for pattern classification has been proposed. Tliis is motivated by the innate comparison / correspondence characteristic of the human visual response. The crucial observation here is that humans are guided by the possibility of a direct correspondence between the exemplars and the test pattern, and, subsequently, by the amount of deformation an exemplar has to undergo to fit the test pattern to be able to classify the latter. Self-organizing networks are used in a novel way to carry out the deformation of the exemplars to fit the test pattern, thereby classifying it. A b s tra c t In contrast with the above (where a correspondence between the test pattern and each of the exemplars is established), we have also come out with a new method for encoding patterns to facilitate recognition. This entails overlaying the pattern on a radial-grid of appropriate resolution (size) to get the training feature array (which is invariant .to some limited deformations of the pattern) as the resulting 2-D array of cells. Two types of feature arrays can be generated : • Number of the corner and middle points of the line approximated patterns in each sector (Type I) ; and • Number of points in each sector of the radial grid (Type II). When either of the two types of feature arrays is used as inputs to a Self- Organizing Neural Network (SONN), and, after training, the neurons are labeled appropriately, it is found that the plot of the labels of the array of neurons exliibits clusters. Tliis demonstrates the aptness of the encoding techniques (and any choice of arrays, Type I or Type II) for the problem under consideration. These feature arrays are classified by a Multi-Layer-Perceptron(MLP) using Backpropagation. However, it is noticed that when Type II feature arrays are used, the performance of MLP is better than when Type I feature arrays are used. Since the MLP takes a long time to train, a network using Learning Vector Quantization (LVQ) for training, is suggested. This network takes a short time to train but yields a larger network compared to MLP. However, the accuracies of M LP and LVQ are comparable. In view of the fact that Type II are better feature arrays compared to Type I, the LVQ and further techniques use only Type II arrays. In an attempt to further increase the speed of training, and improve the performance of the ANN, two constructive techniques based on LVQ are suggested. These techniques start with a small network for recognition, and grow the network till all the training patterns are classifieds correctly.- This strategy results in smaller networks than is possible with the application of LVQ