Lecture 4

Background Subtraction
October 29, 2012

Copyright@ www.vns-group.com

This Lecture Covers:

1 Moving Object Detection

Introduction Challenges
2 Basic Methods

Background as Average Image Detecting Foreground Objects Advantages vs. Disadvantages

3 More Advanced Methods

Mixture of Gaussian Other Methods

4 Assignment

OpenCV Background Subtraction Algorithm

Motivation

1 2 3

plays an important in many computer vision applications in most cases, objects are the most interest, not the scene less processing costs, computational time, etc

Extract Objects of Interest

1 2

in general, the camera is still and the background is static the moving target can be extracted by subtracting the current image from the background scene

foreground image = background image current image

Challenges

gradual (sunset or sunrise) or sudden (for e.g., clouds) illumination changes effects of frequent moving objects in the scene (e.g., trees) structure changes in the scene (e.g., furniture arrangements in indoor environment or new cars entering or leaving in outdoor car-park)

2 3

A good background model

1 2

should be able to adapt to illumination changes, should be able to learn from a clean or non-clean image sequences should automatically adapt to the sudden changes of the scene structure

Background Modelling

Estimate the background scene

1

assumed that it is possible to capture a clean background (without moving targets) at time t (in general, this is possible only in indoor environment, for e.g., a shopping mall before the shop opens) mean lter: the background is the mean of previous n frames median lter: the background is the median of previous n frames more complex models: to address the challenges discussed in the previous slide

2 3 4

Background as a Mean Image

This approach
1 2

models the background as the mean of previous n frames is rather fast but very memory consuming

Mathematically,

1 B= n

Ii
i=1

where B is the background image Ii for i {1, 2, , n} is an image at time i Disadvantages can not adapt to the gradual changes over time requires n M N buffer for an image size of M N

Background as a running Average

Improves the previous approach:

1 2 3

model the background as a running average a weighted average where recent frames have a higher weight no more memory requirement

Mathematically,

Bt+1 = It + (1 ) Bt
where is the learning rate

Running Gaussian Average

Probability Distribution Function: PDF

1 2 3 4 5

Pnder(1997) t Guassian distribution over the histogram B = N(, ) where mut = It + (1 ) t t = (It t )2 + (1 ) t2

Detecting Foreground Objects

Pixel Classication
1

a pixel is classied as a foreground pixel if I( x, y , t) B( x, y , t) > th where th is a global threshold

otherwise, it is classied as a background pixel.

Challenges: th is a global threshold (the same for all pixels) how to choose th th is not a function of time

Advantages vs. Disadvantages

Advantages Easy to implement Less computational cost (fast - real time) Shortcomings High memory requirements (need to keep n number of frames in

the buffer) for mean image

depends only on the previous n frames each pixel is considered independently no spatial correlation is used

Mixture of Gaussian

Motivation different background objects are likely to appear at the same

pixel location
requires to adapt to the permanent scene change a single background model is not adequate Solution proposed by Stauffer and Grimson in 1998 Mixture of K Gaussians (i , i , i ) to cope with multiple

background objects
each Gaussian distribution describes one of the background

model or foreground object

Mixture of Gaussian-Cont:
Algorithm Overview values of a particular pixel is modeled as a mixture of adaptive

Gaussians
for instance, the history at a pixel (x0 , y0 ) is given as

{I1 , I2 , , It },
this history is modeled as a mixture of K Gaussians:
K

P(It ) =
i=1

i,t N(i,t , i,t )

where N(i,t , i,t ) = 1

1/2 i,t (2pi)D/2

exp (It i,t )T 1 (It i,t ) i,t

the number of distributions K is pre-dened (usually from 3 to 5) and

K i=1

i = 1

Mixture of Gaussian-Cont:

Segmenting the foreground weights are updated at every new frame update the model N(i,t , i,t ) if it matches the current value

(those at distance

2.5).

MOG models both the foreground and background. How to segment? 1 rank models by i /|i | 2 a model is background if it occurs frequently (high i ) and does not vary much (low i ).
b

B = arg min
b k =1

k > T

Other state-of-the-art methods:

1

Kernel Density Estimation

non-parametric model proposed by Ahmed Elgammal, David Harwood and Larry Davis in

2000
http://www.cs.rutgers.edu/~elgammal/Research/BGS/

research_bgs.htm

Mean-shift based Estimation

a gradient-ascent method, iterative and hence, slow proposed by M. Piccardi and T. Jan in 2004 http://epress.lib.uts.edu.au/research/bitstream/handle/

10453/2494/2004001173.pdf?sequence=1

Eigen-Backgrounds and its variations

A Bayesian Computer Vision System for Modeling Human

Interactions (IEEE)
proposed by Nuria Oliver, Barbara Rosario, Alex P. Pentland 2000

Assignment: Install OpenCV in your computer. make yourself familiar with OpenCV. What is OpenCV? OpenCV (Open Source Computer Vision) is a library of

programming functions for real time computer vision.

It is free for both academic and commercial use C++, C, Python and soon Java interfaces running on Windows,

Linux, Android and Mac.

opencvlibrary/files/opencv-win/2.4.0/ Books to Read:

Download Link: http://sourceforge.net/projects/

Learning OpenCV: Computer Vision with the OpenCV Library OpenCV 2 Computer Vision Application Programming Cookbook

Assignment: thousands codes available. need to know which one to choose and how to use https://sites.google.com/site/backgroundsubtraction/

available-implementation/ statistical-background-modeling For instance, OpenCV - cvbgfg-gaussmix.cpp