Effects of Post-processing on Background Subtraction Algorithms

Donovan Parks

Outline

What is background subtraction? Project motivation

How is BGS performed and what makes it difficult? Project goals and results
Concluding remarks

What is background subtraction?

Real-time method for identifying moving foreground objects within a video

Project motivation

BGS is an important low-level step in many computer vision applications:

Video surveillance Traffic monitoring FG/BG segmentation

My interest is in using BGS to extract human silhouettes for pose estimation How good are the obtained silhouettes in unconstrained environment?

Images from: Sminchisescu and Telea, Human Pose Estimation from Silhouettes, 2002.

How is BGS performed?

Static frame differencing

BG model = first frame of video

BG Differencing Input Stream

Threshold

BG Model Output Masks

What makes BGS difficult?

Moving background elements:

Adaptive, statistical BG models

Input Stream BG Differencing Mean Variance Update BG Model BG Model Threshold Output Masks

0.03

0.03

0.03

0.025

0.025

0.025

0.02

0.02

0.02

Probability

Probability

0.015

0.015

Probability

0.015

0.01

0.01

0.01

0.005

0.005

0.005

50

100 150 Component Value

200

250

50

100 150 Component Value

200

250

50

100 150 Component Value

200

250

Gaussian Pixel Model

What makes BGS difficult?

Shadows:

Shadow removal

Shadows have little effect on chromaticity, but reduce luminosity

Output Masks BG Differencing Mean Variance Threshold Shadow Removal

Input Stream

Update BG Model BG Model

What makes BGS difficult?

Ghosting:

Ghost detection via optical flow

Low optical flow = ghost!

Input Stream BG Differencing Mean Variance Update BG Model BG Model Connected Components Optical Flow Test Output Masks Threshold Shadow Removal

What else makes BGS difficult?

FG/BG blending

Project goals

Evaluate a selection of state-of-the-art background subtraction algorithms

Considering 10 algorithms in all

Analyze how post-processing influences the performance of these algorithms

Shadow removal Optical flow testing Morphological cleaning Area thresholding

Initial results

Precision

Recall

Example of shadow removal

Example of cleaned results

Conclusions

Many factors which make BGS difficult Post-processing can significantly improve results Results not as clean as more computationally expensive approaches

Questions?

Thank you.