Data Mining - Mehmed Kantardzic [130]
1. a set of neurons that have the same structure and that are connected with initially randomly selected weights; therefore, the neurons respond differently to a given set of input samples;
2. a limit value that is determined on the strength of each neuron; and
3. a mechanism that permits the neurons to compete for the right to respond to a given subset of inputs, such that only one output neuron is active at a time. The neuron that wins the competition is called winner-take-all neuron.
In the simplest form of competitive learning, an ANN has a single layer of output neurons, each of which is fully connected to the input nodes. The network may include feedback connections among the neurons, as indicated in Figure 7.12. In the network architecture described herein, the feedback connections perform lateral inhibition, with each neuron tending to inhibit the neuron to which it is laterally connected. In contrast, the feedforward synaptic connections in the network of Figure 7.12 are all excitatory.
Figure 7.12. A graph of a simple competitive network architecture.
For a neuron k to be the winning neuron, its net value netk for a specified input sample X = {x1, x2, … , xn} must be the largest among all the neurons in the network. The output signal yk of the winning neuron k is set equal to one; the outputs of all other neurons that lose the competition are set equal to 0. We thus write
where the induced local value netk represents the combined action of all the forward and feedback inputs to neuron k.
Let wkj denote the synaptic weights connecting input node j to neuron k. A neuron then learns by shifting synaptic weights from its inactive input nodes to its active input nodes. If a particular neuron wins the competition, each input node of that neuron relinquishes some proportion of its synaptic weight, and the weight relinquished is then distributed among the active input nodes. According to the standard, competitive-learning rule, the change Δwkj applied to synaptic weight wkj is defined by
where η is the learning-rate parameter. The rule has the overall effect of moving the synaptic weights of the winning neuron toward the input pattern X. We may use the geometric analogy represented in Figure 7.13 to illustrate the essence of competitive learning.
Figure 7.13. Geometric interpretation of competitive learning. (a) Initial state of the network; (b) final state of the network.
Each output neuron discovers a cluster of input samples by moving its synaptic weights to the center of gravity of the discovered cluster. Figure 7.13 illustrates the ability of a neural network to perform clustering through competitive learning. During the competitive-learning process, similar samples are grouped by the network and represented by a single artificial neuron at the output. This grouping, based on data correlation, is done automatically. For this function to be performed in a stable way, however, the input samples must fall into sufficiently distinct groups. Otherwise, the network may be unstable.
Competitive (or winner-take-all) neural networks are often used to cluster input data where the number of output clusters is given in advance. Well-known examples of ANNs used for clustering based on unsupervised inductive learning include Kohonen’s learning vector quantization (LVQ), self-organizing map (SOM), and networks based on adaptive-resonance theory models. Since the competitive network discussed in this chapter is very closely related to the Hamming networks, it is worth reviewing the key concepts associated with this general and very important class of ANNs. The Hamming network consists of two layers. The first layer is a standard, feedforward layer, and it performs a correlation between the input vector and the preprocessed