HAND GESTURE RECOGNITION using Matlab

Guide : Mr. S Baskar Project Members : 1026 Tarun Singh 1027 Sunil Kumar 1030 Atul Kumar Yadav

Artificial Intelligence PL57

Introduction
2

Hand Gesture Recognition is very promising area in Pattern Recognition, it includes both Vision Based Gesture Recognition as well as Sensor Based Recognition.

Hand Gesture Recognition

Objectives
3

Build System to recognize American Sign Language (ASL) Gestures Show corresponding Latin Letter for the Gesture

Hand Gesture Recognition

Domain Explanation
4

American Sign Language is a Hand Gesture Language which is used by Deaf & Dumb people through out the world. It contains different gestures for each letter of the English Alphabet.

Hand Gesture Recognition

Data Collection
5

Source of Data
Internet Images

from Camera

Hand Gesture Recognition

ALGORITHM USED
Feed Forward Neural Network

Hand Gesture Recognition

Feed Forward Neural Network

7

Hand Gesture Recognition

Feed Forward Neural Network

8

Initialization
Output
Blame

Initialize Weights initialize bias

randomly

and

Adjust Weight

Hand Gesture Recognition

Feed Forward Neural Network

9

Initialization

Output
Blame Adjust Weight

Calculate Output for the entire network using Activation Function

Output=A( wk*Ik + bias )

Hand Gesture Recognition

Feed Forward Neural Network

10

Initialization

Calculate Error for Neurons

Sum

Output

Error
Adjust Weight

Square Error Mean Square Error

Calculate Blame for the each neuron

Hand Gesture Recognition

Feed Forward Neural Network

11

Initialization

Output
Blame

Adjust Weights using Back Propagation and Gradient Descent

Wij = Wij + r * Ei * Aj ( Ij ) * Oi r Ej A j Ij Oj Learning Rate Blame for Neuron J Derivative of Neuron Js Activation Function Input in previous step Output in previous step

Adjust Weight

Hand Gesture Recognition

12

Working of Software

Hand Gesture Recognition

Flow Chart
13

Training Set Images & Target Values

Preprocess Images

Train Neural Network

Test Image

Preprocess Image

Matched Output

Hand Gesture Recognition

Pre-Processing
14

Capture
RGB 2 GRAY Resize Contrast Reset X-Y Gradient Orienting

Images were taken from each specified folders using loop

FeatureVector
Hand Gesture Recognition

Pre-Processing
15

Capture

RGB 2 GRAY
Resize Contrast Reset X-Y Gradient Orienting

Each and every image was converted into grayscale using rgb2gray function

FeatureVector
Hand Gesture Recognition

Pre-Processing
16

Capture RGB 2 GRAY

Resize
Contrast Reset
X-Y Gradient Orienting FeatureVector

Images of different sizes were converted to common size of 80x60

Hand Gesture Recognition

Pre-Processing
17

Capture

Darken Background

RGB 2 GRAY
Resize

Contrast Reset
X-Y Gradient Orienting FeatureVector Hand Gesture Recognition

Pre-Processing
18

Capture

Edge Detection
Edges

RGB 2 GRAY
Resize Contrast Reset

in X-Direction Edges in Y-Direction

X-Y Gradient
Orienting

FeatureVector

Hand Gesture Recognition

Pre-Processing
19

Capture RGB 2 GRAY Resize Contrast Reset X-Y Gradient

Find Edge Directions Orientation lies between -180 to 180 degrees

Orienting
FeatureVector

Hand Gesture Recognition

Pre-Processing
20

Capture RGB 2 GRAY Resize Contrast Reset X-Y Gradient Orienting

Finally Image is converted in Column Vector of Degrees

FeatureVector
Hand Gesture Recognition

Training Neural Network

21

Following Code was used to train Neural Network

net=newff (minmax(x),[30 target_neurons],{'logsig','logsig'},'traingdx ); net=train(net,x,t);

Hand Gesture Recognition

Training Neural Network

22

Number of Neural
Input

Layer : 4800 Hidden Layer : 30 Target Layer : Depends of diff Categories

Activation Function
Sigmoid
Hand Gesture Recognition

Training Neural Network

23

Default Learning Parameters

Learning

Rate Momentum Factor Number of Epoch

: : :

0.1 0.9 1000

Bias

0 or 1

Hand Gesture Recognition

Testing
24

Testing is simple, Image is taken from Camera and then pre-processed and given to neural network. On the basis of output vector, Target image is shown.
sim(net,test)

Hand Gesture Recognition

25

THANK YOU

Hand Gesture Recognition