Artificial Neural Networks

- Introduction By Bhavik R Prajapati

History

1943: McCullochPitts neuron

Started the field

1962: Rosenblatts perceptron

Learned its own weight values; convergence proof

1969: Minsky & Papert book on perceptrons

Proved limitations of single-layer perceptron networks

1982: Hopfield and convergence in symmetric networks

Introduced energy-function concept

1986: Backpropagation of errors

Method for training multilayer networks

Present: Probabilistic interpretations, Bayesian and spiking networks

Basic concept of Neural

Networks
Neural network simplified model of biological neuron
system
Its parallel distributed processing system made up of
highly interconnected neural computing element that
have ability to learn by acquire knowledge and make it
available for use
Various learning mechanics exit to enable NN acquire
knowledge i.e.learning as TRAINING and ability solve
problem using acuired knowledge as INFERENCE
Base on learning mechanism classified into various type

 NNs imitations of central

nervous system

Human brain termed as

neurons which perform
computation such as
cognition, logical inference ,
pattern recognition etc

Hence simplified imitation of computing by neurons of brain

has been termed as Artificial neural system or artificial
neural networks

Human brain
Brain contain about 10^10 basic units
called as NEURONS
Each NEURON turn connect to about
10^4 other NEURONS
A Neuron is cell receive electro-chem
signal from its various source and turn
respond by transmitting electrical
signal impulse to other Neurons
The fastest neuron switching times are
known to be on the order of 10-3 sec.

Neuron
synapse

 A Neuron is composed of nucleus-a cell body known as SOMA

Long irregularly shaped filament attached to soma are known
DENDRITIES
DENDRITIES act as input channel ,all i/p from other neurons
arrive through the dendrites
One soma connected each other by link is the AXON
AXON appear on output cells are non linear threshold device
produce voltage pulse called SPIKE or ACTION POTENTIAL
Axon terminates in a specialized contact called SYNAPSE

Model of Artificial Neuron

Every component of the model bears direct analogy
to the actual constituents of a biological neuron so
term as artificial neuron
Its model which forms the basis of artificial Neural
network

 The choice of activation function determines the neuron model

Examples:
Thresholding function:
Signum function:
sigmoid function :
Hyperbolic tangent function:

NEURAL NETWORK
ARCHITECTURE
Architecture inspire by structure of
cerrebral cortex of brain
ANN define as data processing system
consisting of large no.of processing
element(artificial neuron)
ANN structure represented using
DIRECT GRAPH
A graph is ordered 2-tuple (V,E)
V:vertices E:edges

 Each EDGE is assigned ORIENTATION

Direct graph also call as DIGRAPH

NN classified base on learning

Method
Single layer feed forward network
Multilayer feed forward network
Recurrent network

Single layer feed forward network

Double layer feed forward network

Recurrent network

Characteristic of Neural network

Learning Method

ADALINE Network
MADALINE Network
Perceptron Network

ADALINE Network
ADALINE : Adaptive Linear Neural Element
This network framed by Bernard widrow of
stanford uni.by using Supervised learning
Only one output neuron and o/p value are
bipolar(-1 or +1)

 Supervised learning algorithm is same as

preceptron learning algorithm
Least mean square(LMS) or Delta rule

Used in all HIGH SPEED MODEMS &

TELEPHONE SWITCHING SYSTEM to cancel
echo in long distance communication ckt

MADALINE Network
MADALINE: Many ADALINE Network
Combining no. of ADALINE with many
layer span

 Learning rule adopted by MADALINE

network : MADALINE adaptation
Rule( MR )
Form of supervised learning method
Objective : to adjust weight such that
error is minimum for current training
pattern but with little damage to
previous training pattern

PERCEPTRON Network
Its computational model of Ratina of
eye so named as PERCEPTRON
Rosenblatt give first Perceptron
model for ANN
Contain 3 units:
1.SENSORY UNIT
2.ASSOCIATION
UNIT
3.RESPONSE
UNIT

Architecture of back propagation

network
Multilayer feed forward network with
back propagation sometimes also
known as multilayer perceptron
Detailed model of Back propagation
Generalized learning rule for back
propagation(delta)

Architecture of back propagation

network
Rosenblatts perceptron was introduced and
limitation regarding to solution of linearity
of inseparable(non linear separate) problem
For solving linearly inseprable problem
Require one or more than perceptron
Each set up identify by small linearity
separable section on input
Then combining their output with each
other perceptron produced final output

solution

Non linear activation

operator

Single layer artificial neural

Network

Multi layer Perceptron