Professional Documents
Culture Documents
Introduction
Gaming (Kinect)
Research Objectives
To detect and recognize harmful activities of individuals of interest from a set/pair of surveillance cameras at long range.
Motivation: Monitoring a crowded environment and locating suspicious activities by security personnel
Security personnel creates a temporary signature of people in the scene (type of clothing, the shape etc..)
Locates the individual with suspicious action and then observes him closely of what he is doing( from the joints movements etc)
Introduction
To have an automated system to perform these tasks, there are 4 different entities
Automatic Pedestrian Unique ID tagger
Seeing what action each one does : walking, running, bending etc..
Eg: Bending down to place a suitcase or pick up a box or tying his shoe lace etc..
Motivation
Should not depend on the initialization of action/gait cycle states (starting/ending points of a
an action cycle)
Should be invariant to speed of motion
Applications
Air crew hand gesture recognition for autonomous visual control of fighter jet
Decision to follow a person based on activity in surveillance.
Video
Feature
Extraction
Action
Segmentation
Action Model
Database
Action
Classification
Action Segmentation:- Segmenting out action instances consistent with the train set
Action Learning and Classification:- Learn statistical models to classify new feature
observations ( based on PCA-Generalized Regression Neural Networks)
HOF
(N)
Input Frame
Masked Region
Optical Flow
R-Transform
Hierarchical
Histogram of
Oriented Flow
+
Quantized
Local Binary
Pattern
Optical Flow
HOF
(N/2)
HOF
(N/2)
HOF
(N/2)
Feature Fusion
Mag/Dir
Feature Fusion
Action
Feature
HOF
(N/2)
Feature Selection
Feature Set
3-Level HHOF ( 140 elements) , 2-Level LBFP ( 295 elements) , 2-level R-Transform
(180) : Total Feature Set
Over fitting of regression model for each action class and tuned more to irrelevant and
redundant feature elements and thus lower accuracy.
Algorithm(Training / Testing)
RESULTS
Weizmann dataset
Weizmann Dataset
Testing strategy:- Leave 10 out (corresponding to one person)
Partial Sequence :- 15 frames with overlap of 10 frames
Legs
Occluded
With dog
Normal
Walk
Knees Up
With
Briefcase
Limping
With Pole
Moonwalk
With Skirt
Test Seq
1st Best
2nd Best
Swinging a
bag
3.094 3.939
Carrying a
briefcase
2.170 3.641
Walking
with a dog
2.338 3.824
Knees Up
3.270 4.091
Limping
Man
2.922 3.821
2.132 3.663
Occluded
Legs
2.594 2.624
Normal
Walk
2.624 3.633
2.945 3.880
2.159 3.540
Median to
all actions
6 human actions
25 subjects
4 different scenarios
600 sequence divided into 2391 subsequences
Low res : 160 120 at 25 fps
11/10/2014
Binu M Nair
14
Tested on 8 actions : dive, golf swing, lift, ride, run, skate, swing and walk
Tested on window size of 15 frames with overlap of 10.
11/10/2014
Binu M Nair
17
Challenges
Different resolutions
across cameras.
Different kinds of
features.
Thank You
Questions?