Artificial Neural Network

Learning Objective
The fundamentals of artificial neural network.
The evolution of neural network.
Comparison between biological neuron and artificial
neuron.

Artificial Neural Network
An efficient information processing system which
resembles in characteristics with a biological NN.
Consists of processing elements known as neurons.
Information is contained in the form of weights
associated with each connection link.
ANN is having the ability to learn, recall and
generalize training patterns.
Each neuron has an internal state i.e. activation or
activity level of neuron, which is the function of the
inputs the neuron inputs.


Architecture of Simple ANN

x
1
w
1

y
x
2
w
2

Y
in
=x
1
w
1
+x
2
w
2
y=f(y
in
)
X
1
Y
X
2
Biological Neural Network
Human brain consists of a huge number of neurons
approximately 10
11.










Soma or Cell Body: where cell nucleus is located.
Dendrites: where the nerve is connected to the cell body.
Axon: which carries the impulses of the neuron

Terminology relationships b/w biological and
artificial neurons
Biological Neuron Artificial Neuron
Cell Neuron
Dendrites Weights or interconnections
Soma Net Input
Axon Output
Brain vs. Computer
Speed
Processing
Size and Complexity
Storage Capacity (memory)
Tolerance
Control Mechanism
Evolution of Neural Network

Basic Models of Neural Network
Three basic entities
The models synaptic interconnection.
The training or learning rules adopted for updating and
adjusting the connection weights.
Their activation functions
Connections
There exist five basic types of neuron connection
architectures:
Single Layer feed forward network
Multilayer feed forward network
Single node with its own feedback
Single layer recurrent network
Multilayer recurrent network.

Learning
Parameter Learning
Structure Learning

Supervised Learning
Unsupervised Learning
Reinforcement Learning

Activation Functions
Identity Function: f(x)=x for all x
Binary Step Function:
f(x)=1 if x>=t
=0 if x<t
Bipolar Step Function:
f(x)=1 if x>=t
=-1 if x<t
Sigmoidal Function
Ramp Functions
f(x)= 1 if x>1
x if o<=x<=1
0 if x<0

Important Terminologies of ANN
Weights
Bias
Threshold
Learning Rate
Momentum Factor:
Vigilance Parameter: control the degree of similarity
required for patterns to be assigned to the same
cluster unit.
Ex: For the network shown in figure,
calculate the net input to the output neuron.

0.3
0.2

0.5 0.1 y

-0.3
0.6
x1
y
x3
x2
Ex. Calculate the net input for the network shown in
figure with bias included in the network

0.2 0.45
0.3

y
0.7
0.6
x1
y
x2
Ex.: Obtain the output of the neuron y for the network shown in figure using
activation functions as:
1. binary sigmoidal
2. bipolar sigmodial

0.8 0.35
0.1

0.6 0.3 y

-0.2
0.4