Data Mining - Mehmed Kantardzic [119]
2. Learning from Examples. An ANN modifies its interconnection weights by applying a set of training or learning samples. The final effects of a learning process are tuned parameters of a network (the parameters are distributed through the main components of the established model), and they represent implicitly stored knowledge for the problem at hand.
3. Adaptivity. An ANN has a built-in capability to adapt its interconnection weights to changes in the surrounding environment. In particular, an ANN trained to operate in a specific environment can be easily retrained to deal with changes in its environmental conditions. Moreover, when it is operating in a nonstationary environment, an ANN can be designed to adopt its parameters in real time.
4. Evidential Response. In the context of data classification, an ANN can be designed to provide information not only about which particular class to select for a given sample, but also about confidence in the decision made. This latter information may be used to reject ambiguous data, should they arise, and thereby improve the classification performance or performances of the other tasks modeled by the network.
5. Fault Tolerance. An ANN has the potential to be inherently fault-tolerant, or capable of robust computation. Its performances do not degrade significantly under adverse operating conditions such as disconnection of neurons, and noisy or missing data. There is some empirical evidence for robust computation, but usually it is uncontrolled.
6. Uniformity of Analysis and Design. Basically, ANNs enjoy universality as information processors. The same principles, notation, and steps in methodology are used in all domains involving application of ANNs.
To explain a classification of different types of ANNs and their basic principles it is necessary to introduce an elementary component of every ANN. This simple processing unit is called an artificial neuron.
7.1 MODEL OF AN ARTIFICIAL NEURON
An artificial neuron is an information-processing unit that is fundamental to the operation of an ANN. The block diagram (Fig. 7.1), which is a model of an artificial neuron, shows that it consists of three basic elements:
1. A set of connecting links from different inputs xi (or synapses), each of which is characterized by a weight or strength wki. The first index refers to the neuron in question and the second index refers to the input of the synapse to which the weight refers. In general, the weights of an artificial neuron may lie in a range that includes negative as well as positive values.
2. An adder for summing the input signals xi weighted by the respective synaptic strengths wki. The operation described here constitutes a linear combiner.
3. An activation function f for limiting the amplitude of the output yk of a neuron.
Figure 7.1. Model of an artificial neuron.
The model of the neuron given in Figure 7.1 also includes an externally applied bias, denoted by bk. The bias has the effect of increasing or lowering the net input of the activation function, depending on whether it is positive or negative.
In mathematical terms, an artificial neuron is an abstract model of a natural neuron, and its processing capabilities are formalized using the following notation. First, there are several inputs xi, i = 1, … , m. Each input xi is multiplied by the corresponding weight wki where k is the index of a given neuron in an ANN. The weights simulate the biological synaptic strengths in a natural neuron. The weighted sum of products xi wki for i = 1, … , m is usually denoted as net in the ANN literature:
Using adopted notation for wk0 = bk and default input x0 = 1, a new uniform version of net summation will be
The same sum can be expressed in vector notation as a scalar product of two m-dimensional vectors:
where
X = {x0, x1, x2, … , xm}
W