Video Matting

Ehsan Shahrian, Brian Price

Video Matting
- Difference in input

Video Matting
- Difference in input

Video Matting
- Coherence

Video Matting
- Introduction

Most video methods are extensions

of image methods
Common extensions
Propagate/generate trimap
Compute matte using temporal
information
Refine matte using temporal information

Video Matting
- Realtime Video Matting

Video Input

Automatic binary
segmentation
and boundary
dilation

Single-frame
image matting

Result

Gong et al. Realtime video matting using multichannel Poisson equations.

Graphics Interface 2010.
6

Video Matting
- Video matting via opacity propagation

Tang et al. Video matting via opacity propagation. Visual Computer, 2012.
Tang et al. Temporally consistent video matting based on bilayer segmentation. ICME 2010.
7

Video Matting
- Video matting via opacity propagation

3D matting Laplacian

Tang et al. Video matting via opacity propagation. Visual Computer, 2012.
Tang et al. Temporally consistent video matting based on bilayer segmentation. ICME 2010.
8

Video Matting
- Bayesian Video Matting

Apostoloff et al. Bayesian video matting using Learnt Image Priors. CVPR 2004.
9

Video Matting
- Bayesian Video Matting

Apostoloff et al. Bayesian video matting using Learnt Image Priors. CVPR 2004.

10

Video Matting
- Video Matting of Complex Scenes

Chuang et al. Video Matting of Complex Scenes. TOG 2002.

11

Video Matting
- Multiframe nonlocal matting Laplacian

Choi et al. Video matting using multiframe nonlocal matting Laplacian. ECCV 2012.

12

Video Matting
- Multiframe nonlocal matting Laplacian

Spatiotemporal Refinement
Nonlocal mean regularization

Choi et al. Video matting using multiframe nonlocal matting Laplacian. ECCV 2012.

13

Video Matting
- Motion-Aware KNN Laplacian

Li et al. Motion-aware KNN Laplacian for video matting. ICCV 2013.

14

Video Matting
- Temporally Coherent Video Matting

Lee et al. Temporally coherent video matting. Graphical Models 2010.

15

Video Matting
- Video Snapcut

Temporal coherencyColor coherence Smoothness

16

Video Matting
Toward Temporally Coherent Video Matting

X. Bai, J. Wang, and D. Simons, Towards temporally-coherent video

matting, Computer Vision/Computer Graphics Collaboration
Techniques, pp. 63-74, 2011

17

Video Matting
- Temporally Coherent and Spatially Accurate

18

Video Matting
- Temporally Coherent and Spatially Accurate
Sampling strategy

19

Video Matting
- Temporally Coherent and Spatially Accurate
Sampling strategy

20

Video Matting
- Temporally Coherent and Spatially Accurate

Selection of best (F,B) pair

Frame t

Frame t+1

Estimated matte Warped matte

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, High accuracy optical flow estimation based on
a theory for warping," pp. 25-36, 2004.
21

Video Matting
- Temporally Coherent and Spatially Accurate
Post processing

Frames

Conventional Laplacian Temporal Laplacian

22

Video Matting
- Temporally Coherent and Spatially Accurate

23

Video Matting
- Multiple video streams

Joshi et al. Natural Video Matting using Camera Arrays.

Siggraph 2006.

McGuire et al. Defocus Video Matting.

Siggraph 2005.
24

Demo
- Adobe AfterEffects Refine Edge

26