ICT Junior Camp

2007 @ Mahidol
University
Image Matting and Composition

Dr. Rawesak Tanawongsuwan

ccrtw@mahidol.ac.th
1
Outline
• Image Composition
• Matting Process
• Blue Screen Matting
• Environment/Shadow Matting
• Recent Matting Techniques
(Knockout/Bayesian/Grabcut/Poisson)
• Interactive Matting Techniques (Robust
matting/Easy matting/Closed-form matting)
• Flash Matting
• Video Matting (Video cutout/Video
defocus/Camera Array matting)

2
Digital Image Composition

3
Digital Image Composition

Matte

4
How does Superman fly?

5
Pulling a matte
Problem Definition:
• The separation of an image into
• A foreground object image
• A background image
• An alpha matte

Hard problem

6
Matte creation
• From digitized images
– Blue-screen matting
– Video or chroma-keying
– Image processing

7
Blue Screen

8
Nothing special about blue

9
Digital Matting Equation
• For each pixel
C = αF + (1- α)B

• C: Rendered Color
• F: Foreground color
• B: Background color
α: Amount of light passing through the
foreground – called the Alpha-Channel

10
Challenges to Matting &
Compositing

• Extracting the Matte (alpha

• channel) from natural scenes
• Handling the occluding contour?
• Foreground has color of
background
• Refracted and reflected light
• …

11
What about semi-transparent
mattes?

12
We want a semi-transparent
matte

13
Solution…

14
More example…

15
Problems with Matting

• Images do not look realistic

• Lack of Refracted Light
• Lack of Reflected Light

16
Environment Matting and
Composition

17
Process of environment
matting

• Learn how light is reflected and refracted from

a backdrop and from various sidedrops

18
Process of environment
matting

19
Results of environment
matting

20
Results of environment
matting

21
Movies of environment
matting

22
Shadow matting and
compositing

23
Shadow matting and
compositing

24
Shadow matting
• Show clip ShadowMattingDivXS.avi

25
Natural image matting

• Beyasian approach

26
Natural image matting

27
Level of user interaction

Trimap

GrabCut
28
Robust matting tool

• Demo

29
Interactive matting
• Easy matting

30
Interactive matting

• Closed form matting

31
Interactive matting

• Soft Scissors

32
Flash matting
• Extract mattes using a pair of
flash/no-flash images

33
Image Matting is Difficult

34
Image Matting is Easy with
FLASH

35
Flash Matting

36
Flash matting

• Show clip FlashMatting.wmv

37
Video

• Let’s understand the nature of video clip

• A sequence of image frames

• Show videoCubism.avi

38
Video matting

• Easier or more difficult?

• Issues…

• Show VideoCutout.avi

39
Special hardware/cameras

• Need more than a single camera

• Defocus video matting

• Multiple cameras matting

40
Defocus video : Capture Hardware

41
Capture Hardware

42
 Preview: Input

Everything in Focus Actor in Focus Background in Focus

IP IF IB
43
Defocus video

• Show clip DefocusMatting.avi

44
Multi-cameras system

45
Using a Camera Array

46
4500

2500

500

47
48
Multi-camera system

• Automatic matte extraction

• Toward real-time performance

• Show clip CameraArrayMatting.mp4

49