You are on page 1of 39

Splines: A unifying framework

for image processing


Michael Unser
Biomedical Imaging Group
EPFL, Lausanne
Switzerland

Plenary talk, ICIP 2005, Genoa, September 2005

Splines: A unifying framework


Linking the continuous and the discrete ..

4
3
2

"Sampling and acquisition

Splines

Splines: A unifying framework


Linking the discrete and the continuous ..

4
3
2

"Sampling and acquisition

Geometric processing
Feature extraction
PDEs ....

Splines

"Algorithm design
Think analog, act digital

Multiresolution

Wavelets

"Multi-scale approaches

OUTLINE

The basic atoms: B-splines

Spline-based image processing

Interpolation vs. approximation


Fast algorithms
Applications

Further perspectives

Splines and wavelet theory


Splines and fractals

Splines: definition

Splines: definition

Definition: A function s(x) is a polynomial spline of degree n with knots

< xk < xk+1 < iff it satisfies the following two properties:

Piecewise polynomial:

s(x) is a polynomial of degree n within each interval [xk , xk+1 );

2
1

Higher-order continuity:

s(x), s(1) (x), , s(n1) (x) are continuous at the knots xk .

Effective degrees of freedom per segment:

n+1
(polynomial coefficients)

n
(constraints)

3
2
1

Cardinal splines = unit spacing and infinite number of knots


The right framework for signal processing
5

THE BASIC ATOMS: B-SPLINES

Polynomial B-splines

B-spline representation

Differential properties

Dilation properties

Multidimensional B-splines

PolynomialB-spline
B-splines
basis functions
Causal B-spline of degree n

B-spline of degree n

1, x [0, 1)
B-spline basis functions
0
0
0
n
0
+
(x) = +
+
B-spline
+
(x)basis functions
+
(x) =
0, otherwise.
!
"#of degree
$ n functions
Causal B-spline
B-spline
basis
Causal B-spline
n

(n +of1)degree
times

Causal n
B-spline of
degree
n
1, x [0, 1)

0
0
0
0
1) 2
Key
properties
(x) = 1, x [0,
0+ (x),
0+ 0+
n+ (x) =
0+
1
+ (x) = +! + "#
+ (x),
+ (x)
0, otherwise.
$
=
1, nx0, [0,otherwise.
1)
"# 0 polynomial
$
support:
shortest
spline
of degree
0 !
0
n Compact
0
times
+ (x) = + + (n + 1)+
(x)
+ (x) =
(n"#+ 1) times
0, otherwise.
!
$
Positivity
Key
Keyproperties
properties
(npolynomial
+ 1) times
Key
properties
Piecewise
Compact support: shortest polynomial spline of degree n
Key properties
shortest
Smoothness:
Holder
continuous
of order
n of degree n
Compact
support:
polynomial
spline
Positivity
Compact
support: shortest polynomial spline of degree n
Positivity
Symmetric
of degree n
PiecewiseB-spline
polynomial
Positivity
Piecewise polynomial
(
)
n+1
n Smoothness:
n

continuous of order n
(x) polynomial
= + xH+older
1
2
Piecewise

Smoothness: Holder
continuous of order n
B-spline
Smoothness:
Holder
continuous
of order nn
Symmetric
of degree
Symmetric
SymmetricB-spline
B-splines
of degree
n
(
)
n+1
n
n
(
)
1
Symmetric
B-spline
of
degree
n

(x)
=

x
+
n+12
n
n +
(x) = (+ x + ) 2
n
n (x) = +
x + n+1
2
1

!2

!1

Theorem: Every cardinal spline, s(x)

V , has a unique and stable

representation in terms of its B-spline expansion

"

B-spline
representation
s(x) =
c[k] (x k)
n
+

kZ

B-spline representation

Theorem (Schoenberg, 1946)

Every cardinal polynomial spline, s(x), has a unique and stable representation in
terms of its B-spline expansion

s(x) =

"

c[k]

kZ

n
+
(x

k)

V =

Derivative operator:

s(x) 1: Dn+1
{s(x)}
=
Basis
functions
0.8

d
D =0.6 dx

"

kZ

a[k](x k)

0.4

discrete signal
8 Every cardinal
Theorem:
0.2spline,
(B-spline coefficients)

analog signal

s(x) V , has a unique and sta

representation in terms of its B-spline expansion

s(x) =

Cubic spline (n=3)

"

kZ

3
2

n
c[k] +
(x k)

In modern terminology:

n
{+
(x k)}kZ forms a Riesz basis.
8
8

TheDifferential
lego revisited
operators
"Continous

operator

d
D{} =
dx

Dm {}

Construction

d
D{} =
dx
Discrete

+ {}

m
+ {}

(j)m

of the B-spline of

(j)m

D {}
m

operator
F

degreeStep
0 function:

1 ej

(1 ej )m
x0+ = D1 {(x)}

The lego
j

e
0
0
0
1 0
0
1

(x)
=
x

(x

1)
=

Step function: x+ = D {(x)}


+
+
+
+ x+
F
m
+ {}

(1 ej )m
n

+
0
n+1 (n+1)
n
+
(x) = x0+ (x 1)0+ = 1+ D1 {(x)}
+
(x)
=

D
{(x)}
=
d
F
D{} =

j+4
n+1 dx
n
!
"
n+1

x
j
+
+
n+1 (n+1)
n
1

e
(1 e
+ (x) = + D
{(x)} =
n

+ () =
=
n!
F
j
(j
j
"Fourier !
domain
formula
"

{}

e
n+1
+ j n+1
j
operator (finite
1

e
(1

e Discrete
)
m1
" difference)
m
m
n

S
,

f
(x)
=

D
f
+ () =
=
+
+
j
n+1
4
j
(j)
1e
0

+ () =
m n
m nm
D

(x)
=

(x)
m1
+
+ +
"
m
m
j
f S , + f (x) = + D f (x)

+ {}

n
Dm +
(x)

nm
m

(x)
+ +

Continuous operator (derivative)

d
D{} =
dx

j
B-splines:
differential
interpretation
Differential operators
"Continuous

Dm {}

operators

Discrete

(j)m

operators

d
F
F
j

j
D{} =
Derivatives
Finite
differences

{}

e
+
dx
Differential operators
F
F
m
j m
m
m

{}

(1

e
)
D {}

(j)
+

x0+ = D1 {(x)} (Step function) ! 1 ej "n+1


j n+1
(1

e
)
d
F
F
n
j

+ ()

j =
=0=
+ {}0
1 1
1 e D{}
0
n+1
j
(j)
+ (x) = construction
D
{(x)}
=
x

(x

1)
dx
B-spline
1
+
+
+
F
j m
F
m
m
{}

(1

e
)Dm {} n+1
n
+

(j)
+ x+
n+1 (n+1)
n+1
n
Green
function
of
D

(x)
=

D
{(x)}
=
+
" +
j n+1
n!
e
(1 ej )n+1
n+1
D
{(x)} = (x) =
!
"
F
=
n+1
j
j {}
j n+1
n+1
1
2
3
4
5

e
(1

e
)1 e
+

(j)
4
n

+ () =
=
n+1
F
m
j
(j)
{}

(1 ej )m One-sided power function


"Fourier domain formula
m1
m

f
(x)
=

! + j "n+1
+

(impulse respo

One-sided power function:

f S ,
D f (x)
j n+1
1

e
(1

e
)
n

nm
n =
m
=
+ ()
D
+
(x) =4 m

(x)
+
+
j
(j)n+1
"

(x) = D

xn ,
xn+ =
0,

x0
x<0

10
3

B-splines: Dilation properties


m-scale relation
m-scale relation

m-scale relation

Dilation by a factor m
"m1
#n+1
Dilation by a factor m
Dilation
by
a
factor
m
!
!
1
! n
n
n
n
k
"
n
n
n

(x/m)
=
h
[k]
(x

k)
with
H
(z)
=
z
m
+
hm
n
+ (x/m) =
m [k]+ (x k) with !
m
1
n
n
n
n
kZ
k=0
+ (x/m) =
hm [k] (x k) with Hm (z) = n
kZ
m
"m1
#n+1
kZ
!
1 constantk
Piecewise
n
Hm (z) = n
z case (n = 0)
m
Piecewise
constant case (n = 0)
0
1 =
1 1 1+ k=0
H1 (z)
z11 + z (m1)
(Moving sum filter)
m

0 0)
1
(m1)
Piecewise constant case (nH=
(z)
=
1
+
z
+

z
(Moving sum filter)
m
Dyadic case (m = 2)
0
(m1)
%
$ sum filter)
%
Hm
(z) = 1 $
+ z 1 +

z
(Moving
n+1
n+1
1
!
Dyadic
)
1+z
1 casen(m
+ 1= 2k
n
H2 (z) = 2
= n
z % (Binomial filter) $
$
%
Applications: fast2spline-based2 algorithmsk
n+1
1 n+1
!
1+z
1
n+1
n k=0
k
H
(z)
=
2
=
z
(Bin
2
Zooming
n
2
2
k
k=0

Smoothing

Multi-scale processing
Wavelet transform

5
5
5

11

Dyadic case: Wavelets


Dyadic case: wavelets
Dilation by a factor ofDyadic
2
case: wavelets
!
n
n
n

(x/2)
=
h
[k]
+
2 of 2
+ (x k)
Dilation
by a factor
!
kZ
n
n
+ (x/2) =
hn2 [k]+
(x k)
Binomial filter
kZ
$
%
n+1
!
"
#n+1
1
1
n+1
n
1
Binomial
filter
H2 (z) = n 1 + z
= n
z k
2"
2 n+1
k
#
"
#
1 n+1
k=0 n + 1
1+z
1 !
n
k
H
(z)
=
2
=
z
2
Example:
piecewise
2 linear splines
2n
k
k=0

Example: piecewise linear splines

2
1

17

12

B-spline representation of images


B-spline representation of images
Symmetric,
Symmetric,
tensor-product
B-splines
tensor-product
B-splines

n (x1 , , xd ) = n (x1 ) n (xd )


Multidimensional spline functions
! representation of images
B-spline
s(x1 , , xd ) =
c[k1 , , kd ] n (x1 k1 , , xd kd )
(k1 ,kd )Zd B-splines
Symmetric, tensor-product
n
n
n (x1 , , xd ) = +
(x1 ) +
(xd )

Multidimensional

spline function

Multidimensional spline functions


!
s(x1 , , xd ) =
c[k1 , , kd ] n (x1 k1 , , xd kd )
(k1 ,kd )Zd

continuous-space image

15

image array
(B-spline coefficients)

Compactly supported
basis functions

13

SPLINE-BASED IMAGE PROCESSING

Spline fitting: overview

Designing simple, efficient algorithms

interpolation
approximation
B-spline interpolation
Fast multi-scale processing

Applications

14

Spline fitting: Overview


Spline fitting
s(x) =

"B-spline

c[k] n (x k)

representation: s(x) =

kZ

kZ

Spline fitting

c[k] n (x k)

Spline fitting
Spline
Splinefitting
fitting
Goal: Determine
c[k] !
such that
s(x)
is a good
representation
our signal
Goal:
Determine
c[k] such
that s(x) isof
a good
representation
!
!
n
s(x) =
c[k]
s(x)
c[k]
s(x)=(x
= k)
c[k]nn(x(xk)k)
Discrete input f [k]
c[k]
=
f
[k]
Discrete inputsuch
f [k] that s(x)|
c[k]
such
that s(x)|x=k =
x=k
kZ
kZ
kZ
"Exact fit: interpolation (reversible)
2
Analog
input
f (x)
c[k]
such
that
min
"f

s
Goal: input
Determine
c[k]
such that
s(x)
is
a that
good
representation
of
Determine
c[k]
s(x)
isisamin
of
our
signal
Goal:
Determine
c[k]such
such
that
s(x)
agood
good
representation
of
our
signal
Analog
fGoal:
(x)
c[k]
such
that
"frepresentation
our
s"signal
L2
sV
a
sVa
Discrete inputDiscrete
fDiscrete
[k] input
c[k]
such
that f
s(x)|
= s(x)|
f+s(x)|
[k]
c[k]
that
[k]
inputf
f[k]
[k] discrete
c[k]
such
that
[k] that
x=k
x=k
x=k==f f
Noisy
input
[k]such
=
s(k)
n[k]
such

Noisy discrete input f [k]


= s(k) +"n[k]
such that
Interpolation
2# +
22
$
algorithm
Analog"input fAnalog
(x)
such
"f that
that
s"min
fc[k]
c[k]
such
"f

s"
Analoginput
input
f(x)
(x) # !
c[k]that min
such
min
"f

s"
L2
LL
22

+
2
m
2
sV
sV
sV
a
aa
min
|f
[k]

s(k)|
+

|D
s(x)|
dx
2sW m
m
2
min
|finput
[k]discrete

s(k)|
+f+
|D++such
s(x)|
dx such
2

kZ
Noisy
discrete
f
[k]
=
s(k)
n[k]
that
m
Noisy
input
[k]
=
s(k)
n[k]
that
Noisy
discrete
input
f
[k]
=
s(k)
n[k]
such
that
sW2

"Regularized
splines
"kZ fit: smoothing
$
""
$$
#
#
#
+
+
+
!
!
!2
2|D
22
2 m s(x)|2 dx mm
minm
|fmin
[k]

s(k)|
+

|f
[k]

s(k)|
+

|D
s(x)|
dx
min
|f
[k]

s(k)|
+

|D
s(x)|
dx
m
m
sW2
sW
sW

22

kZ
kZ
kZ
"Least squares approximation: spline projectors
"Generalized

13

sampling theory
13

"Multi-scale

approximation (resizing, pyramids, wavelets)


13

1313

s(x) =

c[k] n (x k)

Spline fitt
Regularized
fit:
Smoothing
splines
Goal: Determine c[k] such that s(x) is a good representation of our signal
kZ

Spline fitting
s(x) =
c[k] n (x k)
!
n that s(x)|
Discrete
input f [k]
c[k]
such
s(x)
=
c[k]
(x k)
"B-spline
representation:
kZ x=k = f [k]

Determine c[k]
c[k] =kZ
"f, (/a
Goal:
k)#
suchsuch
that that s(x) is a goo
"Smoothing splines
Goal: Determine c[k] such that s(x) is a good representation
min $f s$2L2
f [k]
c[k]
such
sVa
Discrete, noisy input: Discrete input
Smoothing such that s(x)|
Discrete input f [k]
c[k]
x=k =
Noisy discrete input f [k] = s(k) + n[k] Analog such
that
inputalgorithm
f (x)
c[k]
such t
"
Analog input
f+
(x)
c[k] $
such that min "f s
#
!
sVa
2
m discrete
2
Noisy
input
f
[k]
=
s(k)
+ n[k]
min
|f
[k]

s(k)|
+

|D
s(x)|
dx
Theorem:The
The solution (amongallallfunctions)
functions)ofofthe
thesmoothing
smoothing
splineproblem
problem
Theorem:
spline
"
sW2m solution (among Noisy
discrete
input
f
[k]
=
s(k)
+
n[k]
such
that
#

+
kZ
!
!
$
"
$ m
!
$
# +
2
#
#
"
minm m
|f2 [k] +
s(k)| +
|D
! +
"
2
2
m
2
m s(x)|
2 dx
min
|f [k]
[k]
min
s(k)|2 +
+
|f [k]sW
|D
2

s(k)|
+

|D
s(x)|
dx
min
|f
s(k)|
|D
s(x)|
dx
kZ
m
s(x)
m
Analog input f (x)

sW2

kZ
kZ

sW2

kZ

13 its coefficients c[k] = h f [k]


cardinalspline
splineofofdegree
degree2m
2m11. .ItsIncoefficients
addition,
isisaacardinal
c[k] = h f [k] canbe
can be obtained
by digital
suitable
digitalof
filtering
of the
input samples
obtained
by suitable
filtering
the input
samples
f [k]. f [k].

Specialcase:
case:the
thedraftmans
draftmans
spline
Special
case:
spline
"Special
the
draftmans
spline

13
13

Theminimum
minimumcurvature
curvatureinterpolant
interpolantisisobtained
obtainedbybysetting
settingm
m==22and
and
00. .
The
cubicspline
spline! !
ItItisisaacubic

16

Smoothing splines: Example

Spline

s(x) =

kZ

c[k] n (x k)

Goal: Determine c[k] such that s(x) is a


Discrete input f [k]

c[k]

Smoothing with increasing values of

"Efficient

implementation: separable, recursive filtering

17

a
kZ

Spline fitting
Spline
fitting
Spline
space at scale a
n
!
!
Rescaled
basis
function:

n
!
a(
n
s(x) = s(x)
c[k]
k) (x k)
= (x
c[k]
"
kZ
n(x)
such
th
V = s(x) =Dual B-spline:
c[k] n (x
ak)
: c[k
kZ

Least squares fit: Multi-scale approximation


a

kZ
Goal: Determine
such that
a good
of our
signalof our signal
Spline
space
at scale
a s(x)
Goal: c[k]
Determine
c[k]
suchisthat
s(x) representation
is a good representation

a=1
a=2
Spline space at scale a
$x%
n
n
! input
#suchx=k
Rescaled
function:
[k]
Discrete
f [k] input f [k]
c[k]
such that s(x)|
=
f [k]
Discrete
c[k]
thatbasis
s(x)|
=
f
a (x) :=
a
x=k
"
Minimum
error
spline
app
n
5n (x), n
VaSpline
= s(x)
=
c[k]
(x
ak) :ac[k] "2Dual B-spline: 1an (x)
space
at
scale
2 such
3 that
4 #
aa
Spline space
at
scale
a
a
"
#
"
#
Continuous-space
input
f
(x)
kZ
!
! $ % a=1
n
a=2
n: c[k] "
c[k]
(x

ak)
V
=
s(x)
=
x
n
n
a
2
s(x)
=a:= c[k]a (x ak) : c[k] "2 such that min %f s%2L
a =
Rescaled basis Vfunction:
a (x)
a
sVa
kZ
kZ
Minimum error spline approximation at
n
n
n$ %

$=x %[k]
Spline
fitting
Dual B-spline: a (x) such thatn#a (x), n
(x
x ak)$
f
[k]
=
s(k)
+
n[k]
a
n
n
Rescaled basis
function:

(x)
:=

2
4
Rescaled
basis
function:
(x)
:=

a aContinuous-space
!a
a
input f (x)
c[k] = #f
n
n(xat
n n a
s(x)
=suchthat
c[k]
scale
k)
Minimum
errorDual
spline
approximation

n
n = [k]
Dual B-spline:
anB-spline:
(x)
#

(x),

ak)$
a (x) such
thata#(x
a (x),

min
ak)$%f=[k]
a
2
a (x
such
that
s%
kZ
L2
n a

Minimum
error
approximation
at
scale

sV
a
Continuous-space
input fspline
(x)error
c[k] =
#f, at
ascale
( ak)$
Minimum Minimum
error
approximation
a
spline
approximation
Goal: Determine c[k] such that s(x) isata scale
good a
representation of our signal
f
[k]
=
s(k)
+
n[k]
2
n

such
that
min
%f

s%
n ( ak)$
(=#f,
ak)$
such that such that
Continuous-space
input
f
(x)
c[k]
=
#f,

L
Continuous-space
c[k]

2input f (x)
a
a
sVa Discrete input f [k]
c[k]
such that s(x)|x=k = f [k]
Sampling
2
2
min %f min
s%L2%f s%L
2
algorithm
f [k]sV
= as(k) +sV
n[k]
2
a
Analog input f (x)
c[k]
such that min "f s"L2
sVa

Noisy discrete input f [k]

= s(k)19+ n[k]
such that
"
$
#
+
!
20 s(k)|2 +
minm
|f [k]
|Dm s(x)|2 dx
19

sW2

kZ

20

kZ

Spline

kZ

$
%
x
Rescaled basis function: an (x) := nn a
approximation: LS
resizing
Rescaled
basis function: a (x) :=
n
n
n (x) suchthat

Dual B-spline:
#

(x),

(x
n
n

a
a
a
Dual B-spline: (x) such that # (
a

a=1

a
=
2
a=1
a=2
Minimum error
spline
approximation
at
sca
Minimum error spline approximat

Continuous-space input f (x)


c[k]
Continuous-space input f (x)
such that

n
= #f, c[k
a

2
min
%f

s%
L2 %f s%2
such
that
min
sVa
L2
sVa

f [k] = s(k)f [k]


+ n[k]
= s(k) + n[k]

Orthogonal projection onto Va (cubic spline)


Orthogonal projection onto Va (cubic splin

a = 1 10
a = 1 10

20

19

Even though splines are quite


sophisticated mathematically,
... they can be implemented
simply and efficiently !

20

B-spline interpolation made simple


B-spline interpolation

Discrete B-spline kernels

4/6

!n/2"

zB-spline
1/6
1/6
interpolation
= (x)|x=k

B1n (z)
=
n (k)z k
k=!n/2"
!n/2"
!
z
#
n interpolation n " k
B-spline
bn1 [k] = n (x)|!

B
(z)
=

(k)z
1
f (x)|x=k = x=kc[l] n (k l) =1(bn1 c) [k] c[k] = (bn1 ) f [k]
k=!n/2"
!n/2"
!
kZ
z
n
k
bn1 [k] =!
n (x)|x=k

B1nn(z) =

(k)z
n
n 1
2
f [k]
=
c[l]

(x

l)|
=
(b

c)
[k]

c[k]
=
(b
f [k]
B-spline
interpolation
B-spline
interpolation
6
(1

)
1
1)
x=k
z
n 1
k=!n/2"
(b1 ) kZ
[k]

=
z + 4 + z 1
(1 z)(1 z 1 ) !n/2"
!
!n/2"
!
1
n
n z
n
2 n = (b ) !
z
n
n
n
k
f [k]
=
c[l]
(x

l)|
=
(b

c)
[k]

c[k]
f [k]
n
n
n
k
6
(1

)
1
1 n(k)z
z
x=k
1
b
[k]
=

(x)|
B
(z)
=
1
n 1
b
[k]
=

(x)|
B
(z)
=

(k)z
x=k
x=k
1
1
1
1
=
(b1 ) [k]

kZ
1
1 )
(1 z)(1 zk=!n/2"
1 z 1
1 z z + 4 + z
k=!n/2"
2
! !
6n
(1

)
z1
n11
(symmetric
exponential)
1
n 1
n
n
nf [k]
(b1 ) [k] f
[k] =f [k] c[l]
c[l]
(x1=
l)|(xx=k
= x=k
(bn1 =c)(b[k]

c[k]
=
(b
)

l)|

c)
[k]

c[k]
=
(b
)
f [k
1
1 1 )
1
1
z
+
4
+
z
(1

z)(1

z
1 z
1 z kZ kZ

bn1 [k]

B-spline interpolation: inverse filter solution

Efficient recursive implementation

1
1
2
2
6
(1

)
6
(1

)
z
Cascade
of
first
order
recursive
filters
1
z
1
n
n
1 z 1
1
z
= 1 =
(b
)
[k]
7
(b
[k]
1
1)
1
z+4+
z 4 + z(1 z)(1
z 1) z 1 )
z+
(1
z)(1
1
1
1
1
1 z11 z 1
1 z17 z
causal

anti-causal
7

21

Generic C-code

(splines of any degree n)

Main recursion
void

ConvertToInterpolationCoefficients (
double c[ ], long DataLength, double z[ ], long NbPoles, double Tolerance)
{double Lambda = 1.0; long n, k;
if (DataLength == 1L) return;
for (k = 0L; k < NbPoles; k++) Lambda = Lambda * (1.0 - z[k]) * (1.0 - 1.0 / z[k]);
for (n = 0L; n < DataLength; n++) c[n] *= Lambda;
for (k = 0L; k < NbPoles; k++) {
c[0] = InitialCausalCoefficient(c, DataLength, z[k], Tolerance);
for (n = 1L; n < DataLength; n++) c[n] += z[k] * c[n - 1L];
c[DataLength - 1L] = (z[k] / (z[k] * z[k] - 1.0))

* (z[k] * c[DataLength - 2L] + c[DataLength - 1L]);
for (n = DataLength - 2L; 0 <= n; n--) c[n] = z[k] * (c[n + 1L]- c[n]); }
}

Initialization
double InitialCausalCoefficient (
double c[ ], long DataLength, double z, double Tolerance)
{ double Sum, zn, z2n, iz; long n, Horizon;
Horizon = (long)ceil(log(Tolerance) / log(fabs(z)));
if (DataLength < Horizon) Horizon = DataLength;
zn = z; Sum = c[0];
for (n = 1L; n < Horizon; n++) {Sum += zn * c[n]; zn *= z;}
return(Sum);
}

22

Spline interpolation
Equivalent forms of spline representation
!

kZ

n! "

s(x)
c[k] (x k)
n =
c[k] (x k) =
s(k)
kZ

kZ

!"

# n (bn1 )1 [k]
= n 1 s(k)
(b1 ) [k] (x k)
kZ

n=
s(k)int (x

kZ

s(k)nint (x k)
k)

n (x k)

kZ

Cardinal (or fundamental) spline


nint (x) =

n 1
! n (x) =
(b
[k]
nint1
n
1)
(b1 ) [k] (x k)
kZ
kZ

n (x k)

-5 -4 -3 -2 -1

7
7
Finite-cost implementation
of an infinite impulse response
interpolator !

23

Limiting behavior
Spline interpolator

1
2

Impulse response

nint (x)

Frequency response

H () =
n

sin(/2)
/2

"n+1

1
B1n (ej )

0.5

"n+1

sin(/2)
1
F
n
n
to the
H sinc-interpolator
() =
The cardinal splineinterpolators
converge
(ideal filter) as then j
int (x)
/2
B1 (e )0.5
degree
to infinity:
goes
Asymptotic

property

1.5

#$
lim nint (x) = sinc(x),
lim H n () = rect
n The cardinal spline interpolators
n
converge to2
the sinc-interpolator (ideal filter) as the
(in all Lpdegree
-norms)goes to infinity:

#$
lim H n () = rect
n
2

lim nint (x) = sinc(x),

(in all Lp -norms)

(in all Lp-norms )

(Aldroubi et al., Sig. Proc., 1992)


7

Includes Shannons theory as a particular case !


24
7

Geometric transformation
of images
Geometric transformation

Geometric transformation
Geometric transformation
2D separable model
k1!
+n+1 l1 +n+1
!
n
n
k1!
+n+1 l1 +n+1
f (x,!
y) =
c[k,
l]

(x

l)

(y l)
n k1 +n+1 l1n
f (x, y) =
c[k,k=k
l] 1 (xl=l
1 l) +n+1
(y l)
!
!
c[k, l] n (x l) n (y l)
k=k1
l=l1 f (x, y) =

l]

f [k, l]
c[k, l]
f [k, l]

k=k1

(x, y)

c[k, l]
(x, y)
c[k, l]

2D filtering
(separable)

Applications

l=l1

(x, y)
2D re-sampling

5
5
5

zooming, rotation, re-sizing, re-formatting, warping

25

Cubic spline coefficients in 2D

Geometric
Geometric transformation
Digital filter

f (x, y) =

k1!
+n+1 l1 +n+1
!
k=k1

f [k, l]

c[k, l]

Pixel values f [k, l]

l=l1

(recursive,
separable)

f (x, y) =

c[k, l] (x l) (y l)
n

f [k, l]
(x, y)

k1!
+n+1 l1 +n
!
k=k1

c[k, l]

Pixel values f [k, l]


B-spline coefficients c[k, l]

B-spline coefficients c[k, l]

26

l=l

Interpolation benchmark
Cumulative rotation experiment: the best algorithm wins !

Bilinear

Truncated
Truncatedsinc
sinc

Windowed-sinc

Cubic spline

Cubic
Cubic
spline
spline
27

High-quality image interpolation


Splines: best cost-performance tradeoff

Lena 256 x 256, rotation 15 x 24, central 128 x 128 SNR (dB)

35

Bspline(6)
Bspline(5)

30
Bspline(4)
Bspline(3)

Bspline(2)
25

Meijering(7) [1999]

Schaum(2) [1993]
Keys [1981]
Schaum(3) [1993]

Meijering(5) [1999]

Dodgson [1997]

Demo

20
Linear
Sinc Hamming(4)
Nearest-neighbor
German [1997]
15
0.0

0.2

0.4

0.6
0.8
Execution time (s rot-1)

1.0

1.2

1.4

Thvenaz et al., Handbook of Medical Image Processing, 2000


28

Fast multi-scale filtering


n
Three alternative
methods
for
the
fast
computation
of:
f
(x)

Three alternative methods for the fast computation(x/a)


of: f (x)

Three alternative methods for the fast evaluation ofn

n (x/a)

n
n
Three
alternative
methods
Three
for
the
alternative
fast
computation
methods
for
of:
the
f
(x)
fast

computation

(x/a)
of:
f
(x)

Three alternative methods


Three
for the
alternative
fast computation
methods for
of: the
f (x)
fast computation
(x/a) of: f (x) (
O(N ) complexity

O(N ) complexity
O(N
) complexity
O(N
) complexity
O(N
) complexity
O(N
) complexity
1) Pyramid
or
tree
Pyramid
and
treealgorithms
algorithms
Pyramid
andalgorithms
tree
algorithms
Pyramid
and
tree
Pyramid
and tree algorithms
i

1
1
1
1
Pyramid and tree algorithms
a = 2 Pyramid and tree algorithms
n
ation of: f (x) a=
(x/a)
i =i 2i
i i
a
n
2
a
=
2
n
Three
alternative
methods
for
the
fast
computation
of:
f
(x)

(x/a)
a
=
2
a
=
2
H2 (z)
2
n nn
n n
H
(z)

2
H
2 2 a = m Z+
H
(z)

2(z)
O(N
) 2complexity
H2 (z)
2 2H
2 filtering (iterative
Recursive
moving average)
2 (z)

+ +
+m
Recursive
filtering
(iterative
Recursive
moving
filtering
average)
(iterative
a
=
moving
m

Z
average)
a
=
Pyramid
and
tree
algorithms
Recursive
filtering
(iterative
Recursive
moving
filtering
average)
(iterative
a
=
moving
m

Z
average)
a
=
m
Recursive
filtering
(iterative
moving
average)
a
=
m

Z
sm [k]Binomial
= smfilter
[k 1] + f [k] f [k m]
(Unser et al., IEEE Trans. PAMI, 1993)
i [k] = sm [k 1] +sfm[k]
s
[k]

=
f
s[k
[k

1]
f[kf
m
mf
a=
2
sm [k] =integration
sm [k 1] +
sfm[k]
[k]
=
s[kmm]

[kdifferences
m]
+
1]f+[k]f [k]
[km]
m]
Differential:
and
weighted

sm [k] = sm [k 1] + f [k] f [k m]
2) Recursive
filtering
(iterated
moving
average)
n
Differential:
integration
Differential:
and
weighted
integration
differences
andand
weighted
differences
+
H
2
integration
Differential:
and weighted
integration
differences
weighted
differences
a 2(z)
R Differential:
Differential:
integration and weighted differences
+

average) a =amRZ
+ +
!Rx+ +
a

1 a = m
1 1Z+ 1
1
1
Recursive
filtering
(iterative
moving
average)
a

R
a

R
Integral (or primitive):
F (x) = f (t)dt = D {f (x)}
+
aR
! x! x
!
1 1 x ! x
1 1
Integral
(or
primitive):
F
Integral
(x)
=
(or
primitive):
f
(t)dt
=
F
(x)
D
=
{f
(x)}
f
(t)dt
=
D
sFinite
=Integral
sm [k (or

1]
+ f [k]
fa(x)
[k
m]
primitive):
FIntegral
=
(or=
primitive):
f
(t)dt
=
F
(x)
D
=
{f
(x)}
f
{f (x)}
1
1(t)dt
1 =1 D{f
1 (x)}
!
m [k] difference

with
step
a:

{f
(x}
f
(x)

f
(x

a)

x
1
Integral
(or
primitive):
F
(x)
=
f
(t)dt
=
D
{f
(x)}
d differences Finite0difference with step

aFFinite
: a
difference

{f
with
=
fD
step
a: (x)}
fa
{f

a)
= f=(x)
f(xf
1with
1
adifference
Differential:
integration
and
weighted
differences
difference
withFinite
:

{f
=(x)
fstep
(x)
:(xaf
(xa(x}
{f
(x}
a)
f (x)
(xa)
a)
f (x)
Finite
(x/a)
= F (x)
step
(x
a)a(x}
= (x}

{f
+

(Unser et al., IEEE Trans. Sig. Proc, 1994)


0 difference
0 :0
1 1
1 1
with
step
+ Finite
a
0
1{f
1
1 {f
1(x)}
fR(x)

(x/a)
=
F
(x)
f
(x)

(x
(x/a)

a)
=

F
(x)
D
F
(x
(x)}

a)
=

D
f (x) + + (x/a) = F (x)
f (x)+F(x
(x/a)

a)
=

F
(x)
D

{f
F
(x
(x)}

a)
=

D
{f (x)}
a
a
29
+
a
a

a {f (x} = f (x) f (x a)
1

Recursive filtering (iterative moving average) a = m Z


Pyramid and tree algorithms n
2
smPyramid
[k]i = smand
[k tree
1] +algorithms
f [k] f H
[k2(z)
m]
a = 2i
a=
2
Recursive
filtering (iterative moving average) a =
Differential:
integration
and
weighted
differences
H2nn(z)
2
H
2
sm [k] = sm [k n1] + f [k] f [k m]
a 2(z)
R+
+
ternative
methods
forO(N)
the fast
computation
of:
f
(x)

(x/a)
Challenge:
evaluation
of
Recursive
filtering
(iterative
moving
average)
a
=
m

Z
! xmoving average)
+
Recursive
filtering
(iterative
a
=
m

Z
1
Differential:
Integral (or primitive): F (x) =
f (t)dt = Dintegration
{f (x)} and weighted differences
sm [k] = sm [k 1] + f [k] f [k m]
omplexity
+
[k]
=
s
[k

1]
+
f
[k]

f
[k

m]
m
m
a

R
3) sDifferential
approach
Finite difference with step a: a {f (x} = f (x) f (x a)
!x
Differential:
integration and weighted differences
mid and tree
algorithms
1
Differential:
integration
and
weighted
differences
0
1 1
Integral
(or
primitive):
F
(x)
=
f
(t)dt
=
D
{f (x)
f (x) ++ (x/a) = F (x) F (x a) = a D {f (x)}

a R+
aR
step a: a {f (x} = f (x) f (x a
! xFinite difference with
1
Integral (or primitive): F (x) = !
=
D
x f (t)dt
17 0
1{f (x)}
Integral
f (t)dt
= D ={fF(x)}
2 (or primitive): F (x) =
f (x)
+ (x/a)
(x) F (x a) = 1a D1 {f (x)}
Finite difference with step a: a {f (x} = f (x) f (x a)
+
Finite-difference
with moving
step a: average)
(x)m

f (x
a)
a {f (x)} =af =
rsive filtering
(iterative
Z
0
1 1
17
f (x) +
(x/a)
=
F
(x)

F
(x

a)
=

0
1D 1{f (x)}
a
f (x) + (x/a) = F (x) F (x a) = a D {f (x)}
= sm [k 1] + f [k] f [k m]
multiple
weighted
Generalization

Fast multi-scale filtering (Contd)

17
26

ential: integration and weighted differences


f (x)

n
+
(x/a)

1
n+1 (n+1)

D
{f (x)}
n
a
a

integration

Running sums

differences

Interpolation + filtering

n The integral
(n+1)
Principle:
ofn+1
a
spline
of degree
n is a spline of degree n + 1.
f (x)
+
(x/a) !
=xa1n
D
{f (x)}
1
a

(or primitive):

F (x) =

f (t)dt = D

{f (x)}

Principle: The integral of a spline of degree n is a spline of degree n + 1.


ference with step a: a {f (x} = f (x) f (x a)
0

30

Application: Image resizing


"Resizing

algorithm

"Interpolation
"

n=1

"scaling=

70%
Text

SNR=22.94 dB

Spline projector

Application: Image resizing (LS)


"Resizing

algorithm

"Orthogonal

projector

"

n=1

"scaling=

70%
Text

SNR=28.359 dB
+ 5.419 dB
(Munoz et al., IEEE Trans. Imag. Proc, 2001)

Splines: More applications

Sampling and interpolation

Feature extraction

Contours, ridges
Differential geometry
Image pyramids
Shape and active contour models

Image matching

Interpolation, re-sampling, grid conversion


Image reconstruction
Geometric correction

Stereo
Image registration
(multi-modal, rigid body or elastic)

Motion analysis

Optical flow
33

SPLINES: FURTHER PERSPECTIVES


Fractional

B-splines

tional B-splines
Fractional B-splines

0
+
(x) = + x0+

+
(x)

0
+
(x) = + x0+

+1
+ x+

( + 1)=

+
(x)

1 ej
1 ej
F j

F
+1
+ x+

( + 1)

"+1

1e !
"
j +1
1e
F
j
j

x , x 0
x , x0

ded power
function:
x
=
One-sidedpower
power function:
function:
x =
One-sided
+
0, + x<0,0 x < 0

Properties
&
'
'
1

(x

k)
is
a
valid
Riez
basis
for

<

+
2
Riez basis for < 2
+ (x k) kZ is a valid kZ
1
2
1 +2
Convolution
1 property:
2 + 1
+
+ 2 = +
nvolution property: + + = +

perties

(Unser & Blu, SIAM Rev, 2000)

34

FURTHER PERSPECTIVES
Splines

and wavelet theory

(Unser and Blu, IEEE-SP, 2003)

Factorization of any scaling function (or wavelet) of order :

(x) =

1
+

"

0 (x)

where +
with = 1: regular, B-spline part
distribution
Factorization of any scaling
function (or wavelet) of order :
"!
"

S
distributional
part
B-spline: 0explains: irregular.
all

1
(x)
= +
fundamental
properties

0 (x)

Fractional splines are the optimal scaling


functions
for the estimation
of fractalirregular
processes
function
regular part
part

2H+1
where
+
withspectral
=
1: regular,
B-spline
part motion)
with
1/
decay
(fractional
Brownian

0 and
S " : irregular.
distributional part
Splines
fractals

Splines are the optimal functions for the estimation of fractal processes
with 1/ 2H+1 spectral decay (fractional Brownian motion)
35

Splines: The key to wavelet theory


Sobolev smoothness

s L2

B-spline factorization:

= +1 0

Approximation order:

f Pa f

L2

= O(a )

Multi-scale differentiator

() ( j) , 0

general case: n < n + 1


compact support:

= n + 1 (Strang-Fix, 1971)

Polynomial reproduction
degree:

n = 1

c
Vanishing moments:
p
x
(x)dx = 0, p = 0,, n

(Unser and Blu, IEEE-SP, 2003)


36

CONCLUSION

Distinctive features of splines

Splines and image processing

Simple to manipulate
Smooth and well-behaved
Excellent approximation properties
Multiresolution properties
Fundamental nature (Green functions of derivative operators)

A story of avoidance and, more recently, love


Best cost/performance tradeoff
Many applications

Unifying signal processing formulation

Tools: digital filters, convolution operators


Efficient recursive-filtering solutions
Flexibility: piecewise-constant to bandlimited
37

Acknowledgments
Many thanks to

Dr. Thierry Blu


Prof. Akram Aldroubi
Prof. Murray Eden
Dr. Philippe Thvenaz
Annette Unser, Artist

+ many other researchers


and graduate students

38

The end: Thank you!

Spline tutorial

Spline and wavelets

M. Unser, T. Blu, "Wavelet Theory Demystified," IEEE Trans. on


Signal Processing, vol. 51, no. 2, pp. 470-483, 2003.

Smoothing splines and stochastic formulation

M. Unser, "Splines: A Perfect Fit for Signal and Image


Processing," IEEE Signal Processing Magazine, vol. 16, no. 6, pp.
22-38, 1999.

M. Unser, T. Blu, "Generalized Smoothing Splines and the Optimal


Discretization of the Wiener Filter," IEEE Trans. Signal
Processing, vol. 53, no. 6, pp. 2146-2159, June 2005.

Preprints and demos: http://bigwww.epfl.ch/


39

You might also like