UNIVERSITI KUALA LUMPUR

MALAYSIA FRANCE INSTITUTE INDUSTRIAL AUTOMATION AND ROBOTICS

LAB REPORT: ENHANCEMENT AND SEGMENTATION

A REAL-WORLD SYSTEM FOR HUMAN MOTION DETECTION AND TRACKING USING MATLAB

IMAGE PROCESSING FSB33503

GROUP MEMBER : ABDUL SYAHIID BIN MOHD RASIDI 50207211669. MOHD LUQMAN HAQEEM BIN NAZAM 50207211579. MUHAMMAD HILMAN BIN NAIM 50207211526. MADAM NORAZLIN BINTI IBRAHIM

LECTURER

Contents
1.0 2.0 2.1 2.2 2.3 3.0 4.0 4.1 4.2 4.3 5.0 6.0 7.0 7.1 7.2 8.0 9.0 INTRODUCTION ................................................................................................................................. 4 PROBLEM STATEMENT ...................................................................................................................... 5 AUTOMATED SURVEILLANCE FOR SECURITY. ............................................................................... 5 HUMAN INTERACTION FOR MOBILE ROBOTICS. .......................................................................... 5 SAFETY DEVICES FOR PEDESTRIAN DETECTION ON MOTOR VEHICLES. ....................................... 5 OBJECTIVES ....................................................................................................................................... 5 METHODOLOGY ................................................................................................................................ 6 FIRST STEP ..................................................................................................................................... 6 SECOND STEP ................................................................................................................................ 6 THIRD STEP .................................................................................................................................... 6 BLOCK DIAGRAM OF THE HUMAN MOTION DETECTION SYSTEM ................................................... 7 FLOWCHART FOR MATLAB APPLICATION. ........................................................................................ 8 RESULT ............................................................................................................................................ 10 FRAMES BETWEEN 200 AND 320 (WALKING) ............................................................................. 11 FRAMES BETWEEN 500 AND 600 (RUNNING)............................................................................. 11 MATLAB CODING ............................................................................................................................ 12 CONCLUSION ................................................................................................................................... 13

ABSTRACT
In this report we present an operational computer vision system for real-time detection and tracking of human motion. The system captures monocular video of a scene and identies those moving objects which are characteristically human. This serves as both a proof of concept and a verication of other existing algorithms for human motion detection. An approach to statistical modeling of motion developed by us (ABDUL SYAHIID BIN MOHD RASIDI, MOHD LUQMAN HAQEEM BIN NAZAM, MUHAMMAD HILMAN BIN NAIM ) is coupled with a preprocessing stage of image segmentation and point feature tracking. This design allows a system that is robust with respect to occlusion, clutter, and extraneous motion. The results of experiments with the system indicate the ability to minimize both false detections and missed detections.

1.0 INTRODUCTION
Human motion activity is currently one of the most active research topics in computer vision. This strong interest is driven by the wide spectrum of promising applications in many areas such as virtual reality, smart surveillance and perceptual interface. These activities are described by analyzing or classifying the motion of human target in a video stream. One of the most active area is activity understanding from video imagery. Understanding activities involve being able to detect and classify targets of interest and analyze what they are doing. Human motion classification is one such research area.

Moreover, in recent years the research activities in the area of machine vision have been intensified further as the result of its applications being extended toward video representation and coding purpose.

There are a number of approaches involving shape, motion, and statistical analysis proposed over the years. Two of these analytical approaches; enhancement and segmentation are discussed in this lab report.

2.0 PROBLEM STATEMENT

Some examples of applications that could be realized with reliable human motion detection and tracking are:

2.1

Automated surveillance for security-conscious venues such as airports, casinos, museums, and government installations: Intelligent software could monitor security cameras and detect suspicious behavior. Furthermore, human operators could search archived video for classes of activity that they specify without requiring manual viewing of each sequence. Having automated surveillance vastly increases the productivity of the human operator and increases coverage of the surveillance.

2.2

Human interaction for mobile robotics: Autonomous mobile robots in the workplace or home could interact more seamlessly with the humans in their environment if they could reliably detect their presence. For example, robots to assist the elderly would know when assistance is needed based on the motion of a person.

2.3

Safety devices for pedestrian detection on motor vehicles: Intelligent software on a camera-equipped car could detect pedestrians and warn the driver.

3.0 OBJECTIVES

Differentiate motions between walking and running High-level image processing (eg. Subtraction)

4.0 METHODOLOGY
4.1 1ST STEP 4.1.1 Calculate the frame difference mask by thresholding the difference between two consecutive input frames. At the same time, the background difference mask is generated by comparing the current input image and the background image stored in the background buffer.

4.1.2

4.2

2ND STEP 4.2.1 According to the frame difference mask of past several frames, pixels which are not moving for a long time are considered as reliable background in the background registration. This step maintains an up-to-date background buffer as well as a background registration mask indicating whether the background information of a pixel is available or not.

4.2.2

4.3

3RD STEP 4.3.1 An initial object mask is constructed from the background difference mask and the frame difference mask. If the background registration mask indicates that the background information of a pixel is available, the background difference mask is used as the initial object mask. Otherwise, the value in the frame difference mask is copied to the object mask.

4.3.2

4.3.3

5.0 BLOCK DIAGRAM OF THE HUMAN MOTION DETECTION SYSTEM

Figure 1: The software block diagram of the human motion detection system

6.0 FLOWCHART FOR MATLAB APPLICATION.

START

Read AVI and set to variable Count frames of video AVI Read 100th frame of video AVI

Convert background video from RGB to Gray

Read threshold of 1st frame in Gray Convert to BW with threshold

Looping in the interval of 20. Starting from 200 to 600

Read videoavi from 1 to 600 frames in 5 frame interval

Save frames as larijalan

Show mov1

Convert mov1 to gray

Convert to BW

Difference between first frame (bgk) and moving frames (movbw) in BW

Sum of differences

Add 1 to looping on every loop repeat

Show (diff)
Filtering Segmentation: Erosion and Dilation Sharpening

Imerode (diff,se1)

Imdilate (erodedBW,se2)

Show: Imerode (diff,se1),Imdilate (erodedBW,se2)

Imerode (diff,se1)

Show graph of diffsum; Total of white pixels over time

END

7.0 RESULT
Differences between walking and running are in terms of frame counts, from the graph we can deduce that from frame 200 to frame 400, there was a peak that translates into movement from frame-to-frame shown above, this motion detected as walking. Meanwhile, frame 500 to 600 shown a peak of the same magnitude but on shorter time frame thus we can deduce that motion as running. So, walking spans 200 frames whereas running spans only 100 frame thus such motions can be identified from this graph.

ORIGINAL

ERODED

ERODED AND DILATED

SHARPENED IMAGE

DIFFERENCE

Figure 2: Matlab screenshot

FUNCTION DIFFERENCE SHARPENED ERODED DILATED

ELEMENTS SUBTRACTION(RGBGRAY,IM2BW) REPLICATE DISK DIAMOND

VALUE 0.5 DEFAULT 5 10

7.1

FRAMES BETWEEN 200 AND 320 (WALKING)

7.2

FRAMES BETWEEN 500 AND 600 (RUNNING)

8.0 MATLAB CODING

% clear workspace and command window clear all clc % Read AVI and set to variable (videoavi) videoavi = VideoReader('D:\botak.avi'); % Count frames of videoavi nFrames = videoavi.NumberOfFrames; % bgk:background % Read 100th frame of videoavi bgk1 = read(videoavi,1); bgk2 = read(videoavi,100); bgk3 = read(videoavi,500); bgk=bgk1+bgk2+bgk3; % Convert from RGB to Gray bgk = rgb2gray(bgk); % Read threshold of 1st frame in gray grylvl = graythresh(bgk); % Convert to BW with threshold bgk = im2bw(bgk,0.8); % % % % k x y z = x : y : z is the initial frame position is the frame intervals is the final frame position

loop1=1; % Looping in the interval of 20; starting from 200 to 600 for k = 1 : 10: 611 % Read videoavi from 1 to 600 frames in 5 frame interval mov1 = read(videoavi, k); % Read videoavi from 1 to 600 frames in 5 frame interval mov2 = read(videoavi, k+1); % Save frames as larijalan % filname1=strcat('C:\Users\hiidr_000\Google Drive\Image Processing\Botak\videoavi\Frame_',int2str(k)); % filname1=strcat(filname1,'.jpg'); % imwrite(mov1,filname1); % Show mov1 figure(1),subplot(321),imshow(mov1),title('Original'); % Convert mov1 to gray movgray = rgb2gray(mov1); % Convert to BW movbw = im2bw(movgray,0.2); % Difference between first frame (bgk) and moving frames (movbw) in BW diff = bgk-movbw; % Sum of differences diffsum(loop1)=sum(sum(diff)); % Add 1 to looping on every loop repeat loop1=loop1+1; % Show (diff) subplot(322),imshow(diff),title('Difference');

H = fspecial('unsharp'); sharpened = imfilter(diff,H,'replicate'); subplot(323);imshow(sharpened); title('Sharpened Image');

% Segmentation. Erosion and Dilation se1 = strel('disk',5); se2 = strel('diamond',10); erodedBW = imerode(sharpened,se1); subplot(324),imshow(erodedBW),title('Eroded'); dileratedBW = imdilate(erodedBW, se2); subplot(325),imshow(dileratedBW),title('Eroded then Dilated');

end % Show graph of diffsum; Total of white pixels over time % figure(2),subplot(221),plot(diffsum); % figure(2),subplot(222),plot(dileratedBWsum);

9.0 CONCLUSION
In this project, we implement a moving segmentation algorithm based on image change detection for the system. A background registration technique is used to construct reliable background information from the video sequence. Then, each incoming frame is compared with the background image. If the luminance value of a pixel differ significantly from the background image, the pixel is marked as moving object; otherwise, the pixel is regarded as background. Finally, a post-processing step is used to remove noise regions and produce a more smooth shape boundary. My results are not as good as the ones shown in this paper. But the basic ability of change detection and segmentation is essentially demonstrated through the Matlab implementation. However, after this work, I think the algorithm has many limitations and better techniques should be included in order to make the system robust.