Guided Image Filtering

Kaiming He
Jian Sun
Xiaoou Tang

The Chinese University of Hong Kong

Microsoft Research Asia
The Chinese University of Hong Kong

Introduction
Edge-preserving filtering
An important topic in computer vision
Denoising, image smoothing/sharpening, texture decomposition, HDR
compression, image abstraction, optical flow estimation, image superresolution, feature smoothing

Existing methods
Weighted Least Square [Lagendijk et al. 1988]
Anisotropic diffusion [Perona and Malik 1990]
Bilateral filter [Aurich and Weule 95], [Tomasi and Manduchi 98]
Digital TV (Total Variation) filter [Chan et al. 2001]

Introduction
Bilateral filter

qi

W ( p) p

jN ( i )

ij

spatial Gs(xi-xj)

input p

bilateral
W=GsGr
range Gr(pi-pj)

output q

Introduction
Joint bilateral filter [Petschnigg et al. 2004]

qi

W ( I ) p

jN ( i )

ij

bilateral filter: I=p

spatial Gs(xi-xj)

input p

bilateral
W=GsGr

output q

range Gr(Ii-Ij)

E.g. p: noisy / chrominance channel

I: flash / luminance channel
guide I

Introduction
Advantages of bilateral filtering
Preserve edges in the smoothing process
Simple and intuitive
Non-iterative

Introduction
Problems in bilateral filtering
Complexity
Brute-force: O(r2)
Distributive histogram: O(logr) [Weiss 06]
Bilateral grid: band-dependent [Paris and Durand 06], [Chen et al. 07]
Integral histogram: O(1) [Porikli 08], [Yang et al. 09]

Approximate
(quantized)

Introduction
Problems in bilateral filtering
Complexity

Example: detail enhancement

Gradient distortion

gradient
reversal

Preserves edges,
but not gradients

gradient
reversal

input

enhanced

Introduction
Our target - to design a new filter
Edge-preserving filtering
Non-iterative
O(1) time, fast and non-approximate
No gradient distortion

Advantages of bilateral filter

Overcome bilateral filters

problems

Guided filter
qi pi ni

min (aI i b pi ) 2 a 2
( a ,b )

ni - noise / texture
Linear regression
input p

qi aI i

output q

qi aI i b

cov( I , p )
a
var( I )
b p aI

guide I
Bilateral/joint bilateral filter does
not have this linear model

Definition

Guided filter
Extend to the entire image
In all local windows k ,compute
the linear coefficients

cov k ( I , p )
ak
vark ( I )
bk pk aI k

Compute the average of akIi+bk in

all k that covers pixel qi

(a I

k |i

k i

ai I i bi

qi
2

qi

bk )

Definition

Guided filter

cov k ( I , p )
ak
vark ( I )

Parameters
Window radius r

bk pk aI k

Regularization

qi

(a I

k |i

k i

ai I i bi

qi
2

2r
3

bk )

Guided filter: smoothing

a cascade of
mean filters

cov( I , p )
var( I )

var(I )
cov( I , p )

b p aI

input p

qi a I i b p

output q

var(I )
guide I

a0
b p

r : determines
band-width
(like s in BF)

Guided filter: edge-preserving

qi a I i b

qi a I i I i a b
: degree of

cov( I , p )
a

var(I )

edge-preserving
(like r in BF)

qi

I i

guide I

output q

Example edge-preserving smoothing

input &
guide

guided
filter
(let I=p)
r=4, =0.12

r=4, =0.22

r=4, =0.42

s=4, r=0.1

s=4, r=0.2

s=4, r=0.4

bilateral
filter

 Our target - to design a new filter

Edge-preserving filtering
Non-iterative
O(1) time, fast and non-approximate
No gradient distortion

Advantages of bilateral filter

Overcome bilateral filters

problems

Definition

Complexity
mean, var, cov in all local windows

cov k ( I , p )
ak
vark ( I )

Integral images [Franklin 1984]

bk pk aI k

O(1) time independent of r

qi ai I i bi

Non-approximate

O(1) bilateral
(32-bin, 40ms/M)
[Porikli 08]

O(1) bilateral
(64-bin, 80ms/M)

O(1) guided
(exact, 80ms/M)

Gradient Preserving
bilateral filter

input

guided filter

filtered

q a I
large
fluctuation
detail
(input - filtered)

enhanced
(detail * 5 + input)

gradient
reversal

Example detail enhancement

gradient
reversal

bilateral filter

input (I=p)

bilateral filter
s=16, r=0.1

guided filter

guided filter
r=16, =0.12

Example detail enhancement

gradient
reversal

bilateral filter

input (I=p)

bilateral filter
s=16, r=0.1

guided filter

guided filter
r=16, =0.12

Example HDR compression

gradient
reversal

bilateral filter

keep
anti-aliased

guided filter

input HDR

bilateral filter
s=15, r=0.12

guided filter
r=15, =0.122

Example flash/no-flash denoising

gradient
reversal

input p
(no-flash)

joint bilateral filter

s=8, r=0.02

joint bilateral

guide I
(flash)

guided filter
r=8, =0.022

guided filter

Beyond smoothing
Applications: feathering/matting, haze removal

very small
guide I

preserve most
gradients

output q

q a I
input p

Example feathering

guide I
(size 3000x2000)

Example feathering

filter input p (binary segmentation)

Example feathering

filter output q (alpha matte)

Example feathering

guide I

filter input p

filter output q
0.3s
image size 6M

matting Laplacian
[Levin et al. 06]
2 min

Example haze removal

guide I

filter input p
(dark channel prior
[He et al. 09])

filter output q

Example haze removal

guide I

guided filter
(<0.1s, 600x400p)

global optimization
(10s)

Limitation
What is an edge inherently ambiguous, context-dependent
weaker
edge

halo

halo

stronger
texture

Input

Bilateral filter
s=16, r=0.4

Guided filter
r=16, =0.42

Conclusion
We go from BF to GF
Edge-preserving filtering
Non-iterative
O(1) time, fast, accurate
Gradient preserving
More generic than smoothing

Thank you!