Neural Networks

Artificial Neural Networks

Information processing paradigm inspired
by biological neural system
Interconnected
elements(neurons)
solve a problem

working

Applications:
Pattern recognition
Data classification

processing
in union to

Use of Neural Networks

Remarkable abilities
Derive meaning from complicated or
imprecise data
Extract patterns from complex data
which is

impossible
computers

by

human

brain

and

Use of Neural Networks

Trained NN becomes an EXPERT
An EXPERT can be used to solve
new situations
and answer what if questions

NN vs. Conventional
Computers

Conventional Computers

Use an Algorithmic approach

Specific instructions to solve a
problem should be known
These instructions are converted to
high level programming language or
machine code that the computer can
understand.

NN vs. Conventional
Computers

Conventional Computers

They are totally predictable and if

anything goes wrong is due to the
hardware or software
Capability of conventional computers are
restricted to solving problems whose
solution is known
More useful if it could do things which we
dont know how to do

NN vs. Conventional
Computers
Neural Networks
They process information in the same way
as the human brain
NN learn through example
They can be programmed to perform any
task
Examples used to train the NN has to be
selected carefully

NN vs. Conventional
Computers
Disadvantage
NN finds out the solution for a
problem by itself,
hence the results are unpredictable

NN vs. Conventional
Computers
NN and conventional computers are
not in competition they are
complement to each other
Combination of these two approaches
can perform at a maximum efficiency
(Conventional computers supervise NN)

How Human brain

learns

Dendrites
Axon
Synapse

Retrieve the signal from

Pass the electrical activ
Coverts the activity to e

Human Neurons to
Artificial Neurons

Simple Neuron
ANN has many inputs and one output
Two modes of operation
Training mode Trained to fire or not
for particular e.g.s
User mode for a taught input gives
the associated output, else determines
the output based on
Firing
Rule.

Simple Neuron

Firing Rules
Firing rule determines whether the
neuron should fire or not fire
for any input pattern
Relates to all input pattern, which is
not used in the training
Firing rule is based on Hamming
Distance

Training and Using the

Neuron
Training Mode
Collection of training patterns for a node is
used.
Set of patterns which cause it to fire (1)
Set of fire which does not cause it to fire (0)
User mode
Patterns not used in the training will fire the
neuron based on the Nearest pattern
either to the 1 state or 0 state
If there is a tie then the output remains
undefined

Example
3-input neuron (X1,X2,X3)
Output is 1 for 111 and 101
Output is 0 for 000 and 001
Truth table Before applying firing rule

Firing Rule
Input pattern is 010
Differs from 000 by 1 element
001 by 2 elements
101 by 3 elements
Nearest pattern is 000 which belongs to
0
Hence output for the pattern 010 will be
0
Input pattern is 011
Equally distant from the two taught patterns
Hence output is undefined

Applying Firing Rule

Generalization of the Neuron

Pattern Recognition
One of the most important application
of NN is pattern recognition

Example
The network is trained to recognize
the patterns
T - all black

H - all white

Truth tables after

generalization
0 for black, 1 for white
Top neuron

Middle
neuron
Bottom
neuron

Examples of pattern
recognition using the truth
table
Close to T

Close to H

Close to H & T

Complicated Neuron
McCulloch and Pitts model (MCP).
The inputs are weighted, based on the
effect that each input has on decision
making
Weight is a number which is multiplied with
the input to give Weighted Input
Weighted inputs are added and it exceeds
a threshold value the neuron fires. In other
case it does not fire

Complicated Neuron

athematical terms, the neuron fires if and only

X1W1 + X2W2 + X3W3 + ... > T

Architectures of Neural
Networks
Feed Forward Networks
Feed Backward Networks

Feed Forward Networks

Allow signals to travel one way only; from
input to output.
There is no feedback (loops) i.e. the
output of any layer does not affect that
same layer.
Extensively used in pattern recognition.
Also referred to as bottom-up or top-down.

Feed Forward Networks

Feed Backward
Networks

Signals travel in both directions

introducing loops in the network.

by

Very powerful and can get extremely

complicated.
Feedback networks are dynamic; their
'state' is changing continuously until they
reach an equilibrium point.
Also referred to as interactive or recurrent

Feed Backward
Networks

The Learning Process

Patterns
detected
paradigms

using

two

Associative Mapping - network stores

the relationships among patterns
Regularity Detection - the response
of each unit has a particular
'meaning'

Weight Matrix
Every
neural
network
possess
knowledge which is contained in the
values of the connections weights
Two
major
categories
of
neural
networks, based on the Weight matrix
Fixed networks in which the weights
cannot be changed, ie dW/dt=0
Adaptive networks which are able to
change their weights, ie dW/dt not= 0.

Transfer Function
The behaviour of an ANN (Artificial
Neural Network) depends
Weights
Input-Output
function
(transfer
function)
Three categories of Transfer Function
Linear (or ramp)
Threshold
Sigmoid

Transfer Function
Linear units, the output activity is
proportional to the total weighted output.
Threshold units, the output depends on
whether the total input is greater than or
less than some threshold value.
Sigmoid units, the output varies
continuously but not linearly as the input
changes.

Applications of Neural
Networks
Neural networks in medicine
Modelling and Diagnosing
Cardiovascular System
Electronic noses
Instant Physician

the