Miti Part

Table of Content
Digital Image Processing Image Transform (Fourier) / Quantization / Quality

O. Le Meur
olemeur@irisa.fr
http://www.irisa.fr/temics/staff/lemeur/
Univ. of Rennes 1
December 6, 2010
Table of Content
1 2 3 4 5
Introduction Image transformation Fourier transformation Time sampling Discrete Fourier Transform
6 7 8
Scalar Quantization Distortion/quality assessment Conclusion
Introduction
Introduction Image transformation Fourier transformation Time sampling Discrete Fourier Transform Scalar Quantization Distortion/quality assessment Conclusion
1 2 3 4 5
6 7 8
Introduction
What is a signal? This is a function of one or more variables (time, distance, temperature...).
A signal carries information!!

Signal processing is required to obtain a more desirable form. Types of signals: Natural (lighting) or synthetic (from a lab); one dimension or more; Continuous/discrete.
Introduction
Classication of signals
Continuous-time vs Discrete-time signals: Analog vs Digital:
A continuous-time signal contains a value for all real numbers along the time axis; A discrete-time signal only has values at equally spaced intervals along the time axis. Analog signal corresponds to a continuous set of possible function values; Digital signal corresponds to a discrete set of possible function values. The past, present and future of a deterministic signal are known with certainty (its
Deterministic vs Random Signals:
values can be determined by a mathematical expression, rule or table). Otherwise, it is called Random and its properties are explained by statistical techniques.
Periodic vs Aperiodic: periodic signals repeat with some period T ; Finite vs Innite length.
Introduction
Classication of signals
Sampling = discrete time (or space)

6
Quantization = discrete amplitude
Introduction
There are many dierent ways to describe or represent a signal mathematically. Most of the time a linear combination of elementary time functions. These functions are called basis functions. Let the set of basis functions be dened by: {p }, (dierent posssibilities for the range of p )
x (t )
an n (t )
Notations and reminder
a continuous signal x (.); a discrete signal x [.]; a matrix A.
Notations and reminder

(s);
f is the frequency (Hz); w = 2f is the pulsation (radian/s); T x Lp (R) means that

+
1 = f
is the period
For instance, the space L2 (R) is + composed of the functions x with a nite energy |x (t )|2 dt < +;
|x (t )|p dt < +.
Euler's formulas: exp (j ) = cos + j sin ;
exp(j ) = cos + j sin exp(j ) = cos j sin exp(j ) + exp(j ) = 2 cos exp(j ) exp(j ) = 2j sin
From Wikepedia.
9
Reminder
x sinc (x ) = sin ; x
Inner product:
x, y x, y
=
+
x (t ) y (t )dt x (n) y (n)
n=
Condition of orthogonality for real basis function: t2 0 n (t )m (t )dt = m t1
n=m n=m
where m is a constant (m = 1, orthonormal). Condition of orthogonality for complex basis function: t2 0 n=m n (t ) m (t )dt = m n = m t1 where m is a constant (m = 1, orthonormal).
10
Image transformation
Presentation 3 kinds of transformation Point to point transformation Local to point transformation Global to point transformation
1 2
Introduction Image transformation Presentation 3 kinds of transformation Point to point transformation Local to point transformation Global to point transformation Fourier transformation Time sampling Discrete Fourier Transform
6 7 8
3 4 5
11
What is a transformation? T
im x y
[ , ] IM [u , v ]
im is the original image; IM is the transformed image; x , y (or u, v ) represents the spatial coordinates of a pixel.
Goal of a transformation The goal of a transformation is to get a new representation of the incoming picture. This new representation can be more convenient for a particular application or can ease the extraction of particular properties of the picture.
12
There exist 3 types of transformation: Point to point transformation: The output value at a specic coordinate is dependent only on one input value but not necessarily at the same coordinate; Local to point transformation: The output value at a specic coordinate is dependent on the input values in the neighborhood of that same coordinate; Global to point transformation: The output value at a specic coordinate is dependent on all the values in the input image.
Note that the complexity increases with the size of the considered neighborhood...
13
Point to point transformation
Geometric transformation such as rotation, scaling... Example of rotation:
Scalar quantization LUT (Look-Up-Table)
Remark: a point to point transformation doesn't require extra memory...

14
Example of local to point transformation
Note that local to point transformation is also called neighborhood operators. Examples of local to point transformation: Linear transformation: For instance, a tranverse lter (FIR) IM [x , y ] = i j h(i , j )im[x i , y j ], where h is the 2D impulse response. Non-linear transformation such as adaptive linear ltering and rank ltering (median)... Morphological ltering (erosion, dilatation...) based on the denition of a structuring element. Remark: Non linear lters are not reversible in general.
15
Global to point transformation
One of the most important transformation is the Fourier transform that gives a frequential representation of the signal. Fourier series (development in real term): Every signal x (x L2 (t1 , t1 + T )) can be decomposed into a linear combination of sinusoidal and cosinusoidal component functions.
x (t )
b0 +
n =1
an cos (2nt ) +
n =1
bn sin(2nt )
an and bn are used to weight the inuence of each frequency component.
16
Fourier series (development in complex term):
x (t )
with,
Cn exp(j 2n t ) T n=
+
Cn
Remark:
T t1
t 1 +T
x (t )exp(j 2n t )dt T
an +jbn , n 1 ; C = C ; jbn Cn = an n n 2 , n 1 ; Cn = 2 Note that if the function to be represented is also T -periodic, then t1 is an arbitrary choice. Two popular choices are t1 = 0, and t1 = T /2.
17
18
Fourier transformation
Denition Trivial properties Parseval Formula Properties Some examples
1 2 3
Introduction Image transformation Fourier transformation Denition Trivial properties Parseval Formula Properties Some examples Time sampling Discrete Fourier Transform
6 7 8
4 5
19
Fourier transformation in L2 (R)

x t
( ) X (f )
Inverse Fourier Transform:

+
x t
( )
X f
( )exp (j 2 ft )df
Fourier Transform:
+
X f
( )
x t exp
( )
(j 2 ft )dt
provided that x L2 (R) and X L2 (R).

20
From the output of the Fourier transform, we dene: The frequency spectrum: Real (X (f )) + jImg ((X (f ))) The Fourier transform of a function produces a frequency spectrum which contains all of the information about the original signal, but in a dierent form. Magnitude spectrum: |Real (X (f )) + jImg (X (f ))|
Img (X (f )) Phase spectrum: Arctg Real (X (f ))

Power spectrum: Real (X (f ))2 + Img (X (f ))2
21
Linearity:
ax (t ) ax1 (t ) + bx2 (t )
aX (f ) aX1 (f ) + bX2 (f )
F Complex conjugate: x (t ) X (f )
x (t ) x (t )
=
+
X (f )exp(j 2ft )df
X (f )exp(j 2ft )df
F F In the same way, x (t ) X (f ) and x (t ) X (f ).
22
Hermitian symmetry: if x (t ) R, we deduce X (f ) = X (f )
X (f ) X (f ) X (f )
= =
+ +
x (t )exp(j 2ft )dt x (t )exp(j 2ft )dt
x (t )exp(j 2ft )dt
Given that x (t ) R, we have x (t ) = x (t ) that implies X (f ) = X (f ).
23
Note that, as we use the space L2 (R), the inner product of x L2 (R) and y L2 (R) is given by:
x, y
The norm is then given by:
+
x (t )y (t )dt
|x (t )|2 dt
x
=
2=
x, x
+
|x (t )|2 dt
x (t )x (t )dt x (t ) X (f )
+ +
=
+
X (f )exp(j 2ft )df dt x (t )exp(j 2ft )dt df
=
+
=
+
|x (t )|2 dt
X (f )X (f )df
|X (f )|2 df
24
|x (t )|
+
dt
|X (f )|
df
This formula (Parseval's theorem or energy conservation) proves that the energy is conserved by the Fourier transform. Loosely, the sum (or integral) of the square of a function is equal to the sum (or integral) of the square of its transform.
25
F Translation in time/space domain: x (t t0 ) exp(j 2t0 f )X (f )
x (t t0 ) x (t t0 ) x (t t0 )
+ +
x (t t0 )exp(j 2tf )dt x (k )exp(j 2(k + t0 )f )dk

+
k = t t0

F F
exp(j 2t0 f )
x (k )exp(j 2fk )dk
Displacement in time or space induces a phase shift proportional to frequency and to the amount of displacement.
F Frequency shift: x (t )exp (j 2f0 t ) X ( f f0 )
Displacement in frequency multiplies the time/space function by a unit phasor which has angle proportional to time/space and to the amount of displacement. Amplitude modulation.
26
F Convolution in time/space domain: x1 (t ) x2 (t ) X1 (f ) X2 (f ) + + +
x1 (t ) x2 (t ) x1 (t ) x2 (t ) x1 (t ) x2 (t )
x1 ( )x2 (t )d exp(j 2ft )dt

+
x1 ( )
x2 (t )exp(j 2ft )dt d

X2 (f )
Translation
x1 ( )exp(j 2f )d
F Convolution in frequency domain: x1 (t ) x2 (t ) X1 (f ) X2 (f )
These properties mean that if two functions are multiplied in one domain, then their Fourier transforms are convolved in the other domain.
27
F Time and frequency scaling: x (t )
||
f) X(
x (t ) x (t ) x (t )
+ +
x (t )exp(j 2tf )dt x (k )exp(j 2k f ) dk

+
k = t

F F
| |
1
| |
x (k )exp(j 2k f )dk
Multiplication of the scale of the time/space reference frame changes by the factor scales the frequency axis of the spectrum of the function.
All the previous properties are what makes Fourier transforms so useful and practical.
28
Translated Dirac, x (t ) = (t T )
x (t ) = Arect (t |T )
X (f ) X (f )
x (t )exp(j 2tf )dt
exp(j 2Tf )
X (f )
x (t )exp(j 2tf )dt exp(j 2tf )dt
+T 2
If T = 0, X (f ) = 1. The Fourier transform is constant whatever the frequency. x (t ) = 1, X (f ) = (f )

29
X (f )
A [exp(jfT ) exp(jfT )] j 2f ATsinc (fT )
T 2
Ideal low-pass lter in frequency domain: X (f ) = rect (f | F 2 ) selects low F F frequencies over 4 , 4 . The impluse response of the lter is calculated with the inverse Fourier integral:
x (t )
X (f )exp(j 2ft )df exp(j 2ft )df
= =
+F 4
sin x (t )
= =
1 exp(j 2 F t ) exp(j 2 F t ) j 2t 4 4 exp(j ) exp(j ) 2j F sinc ( tF ) 2 2
F 4
30
31
Time sampling
What is the objective? Reminder on Dirac Sampling Sampling Theorem In practice
1 2 3 4
Introduction Image transformation Fourier transformation Time sampling What is the objective? Reminder on Dirac Sampling Sampling Theorem In practice Discrete Fourier Transform
6 7 8
32
Time sampling
To be more exible and to gain in accuracy... Digital algorithms are now ... everywhere from television, CD, embedded electronic (car...). Whether sound recordings or images, most discrete signals are obtained by sampling an analog signal. But, be careful, we have to follow some constraints in order to be able to reconstruct the analog signal. The simplest way to discretize an analog signal x is to record its sample values at intervals T : x (t ) can be approximated by {x (nT )}nZ .
33
Denition
Diracs are essential to make the transition from functions of real variable to discrete sequences. A Dirac can be dened as:
+
(t )x (t )dt = x (0)
(t ) =
0
+
+
if t = 0 for t = 0
(t )dt = 1.
Remark: { : (t )} Z is an orthonormal basis:

(t n), (t m) =
0 1
if m = n for m = n .
34
Properties of a Dirac
(t ) Scaling and symmetry: (t ) = || +
(t )dt (t )
= = (t ) ||
(k )
dk
| |
Note that if = 1, (t ) = (t ). Product: x (t ) (t T ) = x (T ) (t T ) Convolution: x (t ) (t T )
x (t ) (t T )
=
+
x ( )(t T )d x ( )( (t T ))d
x (t T )
The eect of convolving with a translated Dirac is to time-delay x (t ), by the same amount (copy of the signal x (t ) at the position T ).
35
Dirac comb: x (t ) =
nZ (t nT ). Notation: T (t )
The product of a comb with a signal x (t ) conducts to a sampling:
The convolution of a comb with a signal x (t ) conducts to a periodization of x (t ):
36
FT of a Dirac
F Dirac: x (t ) = (t ), (t ) 1, constant function whatever the frequency
X (f ) X (f )
(t )exp (j 2 tf )dt
exp(j 20f ) = 1
F Translated Dirac: x (t ) = (t T ), (t T ) exp(j 2Tf )
X (f ) X (f )
(t T )exp (j 2 tf )dt
exp(j 2Tf )
F Constant signal: x (t ) = 1, 1 (f ). The FT of a constant signal is a weight at the null frequency.
37
Dirac comb: x (t ) = nZ (t nT ) This is a periodic function, since x (t + T ) = x (t ), t . Its Fourier serie is given by:
x (t ) Cn x (t )
n= T 1 2
Cn exp(j 2n t ) T
(t )exp (j 2 n
T
1
T 2 +
nt )dt = 1 T T
T n=
exp(j 2n t ) T exp(j 2n t ) T n= T
1
+
By using the Fourier Trasnform,

F (x (t )) F (x (t )) F (x (t )) (t nT ) = F
T n= T n= T nZ
1 1
+
exp(j 2n t ) T
+
=
F
exp(j 2t (f
nZ
(f
n) T
n ))dt T
Sampling
Sampling a signal is equivalent to multiplying it by a grid of impulses.
39
Sampling
Let x (t ) be a continous function, its sampled version y (t ) is given by:
y (t )
= = =
x ( t ) T ( t ) x (t ) (t nT )
nZ nZ
x (t )(t nT ) x (nT )(t nT )
nZ
y (t )
1 ). where T (t ) is the sampling function (sampling frequency T
40
Fourier Transform of a sampled signal

Let x (t ) be a continous function, its sampled version y (t ), y (t ) = x (t ) T (t )
y (t )
F (y (t )) F (y (t )) F (y (t )) F (y (t )) F (y (t ))
x (t )
nZ
(t nT ) (t nT ) (t nT ) (f
F x (t )
nZ nZ nZ
X (f ) F
X (f ) T T
1 1
nZ nZ
n ) T
= =
X (f ) (f X (f
n ) T
n ) T
n . For n = 0, the spectrum is the primary spectrum and X (f ) and all its replicas at positions T for n = 0 we have copies of the spectrum.
41
The Fourier representation of the sampled function is a superposition of the Fourier transform

Let x (t ) be a continous function, its sampled version y (t ), y (t ) = x (t ) T (t ) Let X (f ) be the Fourier transform of x (t ) and Y (f ) the Fourier transform of its sampled version y (t ). 1 n
Y (f ) =
T nZ
X (f
The sampling operation replicates the primary spectrum periodically in the Fourier domain.
42
43
Aliasing eect... We won't be able to recover the original signal with a good quality. The transformation is no more inversible! The sampling frequency is too small.
44
Shannon-Nyquist theorem
To ensure a perfect reconstruction of a signal x (t ) (to avoid aliasing eect), the sampling frequency fe must verify:
f
e > 2 fmax
where fmax is the maximal frequency of the spectrum X (f ).

The periodization doesn't provoke an aliasing eet.
45
Whittaker-Shannon theorem
If x (t ) is bandlimited to a frequency range [fmax , fmax ], that is, its Fourier transform X (f ) is zero for |f | > fmax , then x (t ) can be completely specied by samples taken at the Nyquist sample rate of 2fmax .
46
Whittaker-Shannon theorem
To recover the signal x (t ), we can use a low-pass lter in frequency domain. For instance, the ideal low-pass lter in the frequency domain is given by G (f ) = Te rect (f /fe )
X (f ) x (t )
= =
Y (f ) rect (f /fe )
+
k =
x (kTe )sinc ((t kTe )fe )
47
The sampling of a physical signal always induces an aliasing eect, even with an ideal sampler. Paley-Wiener theorem indeed showed that there is no bandlimited signal having a nite energy... In many cases, the spectrum of the signal is not perfectly known and can be corrupted by noise. It is thus required to lter the analog signal before the sampling operation.
1 2 3
Pre-ltering to obtain a bandlimited signal; Sampling by checking the value of the sampling frequency; Low-pass ltering to retrieve the signal.
48
Discrete Fourier Transform
Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples
1 2 3 4 5
Introduction Image transformation Fourier transformation Time sampling Discrete Fourier Transform Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples
6 7 8
49
Denition of 1D DFT
The discrete Fourier transform (DFT) is the transform of a nite sequence. Since the time-domain is discrete the spectrum is periodic The DFT of a sequence of N complex numbers x0 , . . . , xN 1 is given by: N 1 x [n]exp (j 2 nk ), k = 0, . . . , N 1 1 X [k ] = N n =0 N
X [k ] = x , ek , where ek (n) = exp(j 2N nk ). The family ek (n) = exp (j 2 N nk ) 0k <N is an orthogonal basis of the space of signals of period N .
The IDFT of a sequence of N complex numbers X0 , . . . , XN 1 is given by: 1 2 x [n ] = N n=0 X [k ]exp (j N nk ), n = 0, . . . , N 1 The DFT computes the X [k ] from the x [n], while the IDFT shows how to compute the x [n] as a sum of sinusoidal components X [k ]exp ( j 2 N kn) with frequency k /N cycles per sample.
50
Denition of 1D DFT
X [k ]
N n =0
1 N 1
x [n]exp(j 2 nk ) N
1 can be removed or replaced by 1 . In the rst case, the The weighting coecient N N 1 whereas in the second case it is equal weighting coecient of the IDFT is equal to N 1 . to N
What is important is that DFT IDFT = Id .
51
Discrete bi-dimensional Fourier transformation

im k l
[ , ] is a discrete picture having a size
Inverse Fourier Transform:

N 1 M 1
IM u v exp j
M.
im k l
[ , ]
=
u=0 v =0
[ , ]
( 2 (
k N
l M
))
Fourier Transform:
IM u v
[ , ]
1
NM
N 1 M 1
im k l exp
[ , ]
k =0 l =0
(j 2 (
k N
l M
))
u v
The unit is cycles per unit length (pel 1 ).

52
u , [0, 1]; = 0, . . . , N 1, k = N k v , [0, 1]. = 0, . . . , M 1, l = M l
Discrete bi-dimensional Fourier transformation
A frequential decomposition in terms of exponential basis images ...

im k l
[ , ]
im k l
[ , ]
[0, 0] exp (j 2 (0 k + 0 l )) + [0, 1] exp (j 2 (0 k + l )) + IM [1, 0] exp (j 2 (k + 0 l )) + . . . = IM [0, 0] + IM [0, 1] (cos (2 l ) + jsin (2 l )) + IM [1, 0] (cos (2 k ) + jsin (2 k )) + IM [1, 1] (cos (2 (k + l )) + jsin (2 (k + l ))) + . . . =
IM IM
IM [u , v ] are the weighting coecients of the spatial frequencies present in the signal.
53
Some remarks:
1 A picture having a size of N M is a linear combination of N M exponential basis images!!!! 2 The DFT of a picture yields a complex picture of the same size;
54
Some remarks:
3 IM [u , v ] and im[k , l ] are periodic of innite extent. Period = N for k , u . Period = M for l , v :
The image imt is bandlimited and is spatially sampled, for example, with a rectangular array of N M pels:
im[k , l ]
N 1 M 1
=
i =0 j =0
imt (k , l )(k iTK )(l jTL )
55
Some remarks:
56
Translation
Translation in the frequency domain

im k l exp
[ , ]
2
N
(u0 k + v0 l )
DF T
IM u
[ u0 , v v0 ]
Translation in the spatial domain

im k
[ k0 , l l 0 ]
DF T
IM u v exp
[ , ]
2
N
(k0 u + l0 v )
The multiplication by an exponential function in a representation modies the origin in the other representation.
57
Average value
im
1 N 1 M 1
NM
im k l
[ , ] (j 2 (
k N l M
IM u v
[ , ] [0, 0]
1 1
k =0 l =0 N 1 M 1
im k l exp
[ , ]
))
NM
k =0 l =0 N 1 M 1
im k l
IM
[ , ]
NM
k =0 l =0
im
1
NM
IM
[0, 0]
IM
[0, 0] is proportional to the average value of the picture.
58
Separable transformation = separable basis decomposition

The 2D DFT can be computed as the separable product of two one-dimensional discrete Fourier transforms. N 1 M 1 IM [u, v ] = 1 im[k , l ]exp(j 2( k u + l v )) N M NM k =0 l =0
k u + l v )) Initialy the family ek ,l [u , v ] = exp (j 2( N is used, but we M 0k <N ,0l <M can also use the family {ek (u )el (v )}0k <N ,0l <M .
IM [u, v ] IM [u, v ] IM [u, v ]

{el }0l <M .
1 N 1
= =
exp(j 2 k u) MDFT1D (im[k , l ]) N N k =0 NDFT1D MDFT1D (im[k , l ])
1 N 1
exp(j 2 k u) N N k =0
1 M 1
M l =0
im[k , l ]exp(j 2 l v ) M
We start by computing the Fourier coecients on the image rows int the basis A second transform is applied on the columns of the transformed images in the basis {ek }0k <N .
60

For a picture having a size N N , there are three ways to compute its DFT: DFT 1D: N 2 multiplications and N (N 1) additions, O(N 2 )
x [n]exp(j 2 nk ) N n =0 N DFT 2D: N 2 N 2 multiplications and N 2 (N 1)2 additions, O(N 4 ) X [k ]

=
1 N 1
im[k , l ]exp(j 2( k u + l v )) N k =0 l =0 N N DFT 2D separable transform: 2N N 2 multiplications and 2N N (N 1)2 additions, O(N 3 ) IM [u, v ]
=
1 N 1 N 1
61
Examples
Remark: the spatial frequency amplitude spectra of natural images are well described by a power law X (f ) = fk . Estimated average values of are ranged over 0.9 and 1.2. The power always falls with spatial frequency.
From E.P. Simoncelli and B.A.Olhausen, Natural Image Statistics and Neural Representation, Annun. Rev. Neuroscience, 2001. 62
Power spectrum of a natural image (solid line) compared with f12 (dashed line).
Examples
From R.M. Balboa, N.M. Grzywacz, Power spectra and distribution of contrasts of natural images from dierent habitats, Vision Research, 43, pp. 2527-2537, 2003.
The spectra may be more or less anisotropic.
63
Examples
From left hand-side to right: original picture, spectrum, contour lines.
Remark: For display purpose, a logarithm is applied on the spectrum.
64
Scalar Quantization
Principle Uniform quantization Optimal quantization, Lloyd-Max algorithm Examples: uniform, semi-uniform and optimal quantizer
1 2 3 4 5
Scalar Quantization Principle Uniform quantization Optimal quantization, Lloyd-Max algorithm Examples: uniform, semi-uniform and optimal quantizer Distortion/quality assessment Conclusion
7 8
65
Principle
The quantization is a process to represent a large set of values with a smaller set.
Scalar quantization:
Q:X x
X C
C = {yi , i = 1, 2, ...N } Q ( x ) = yi
N is the number of quantization level;
could be continue (ex: R) or discret; is always discret (codebook,dictionnary); card (X ) > card (C ); As x = Q (x ), we will lost some information (lossy compression).
66
Uniform quantization
In the uniform quantization, the quantization step size is xed, no matter what the signal amplitude is. Denition: output y The quantization thresholds are uniformly distributed: i {1, 2, ..., N }, ti ti +1 = {yi }, representative levels The output values are the center of the quantization interval: t +t i {1, 2, ..., N }, yi = i 2i +1 input x t 1 t2 t3 t4 t 6 t7 t8 t9 Example of the nearest neighborhood quantizer (see on the left): {ti }, decision levels x + 0 .5 . Q (x ) = y2 y1
67
Uniform quantization
A mid-step quantizer Zero is one of the representative levels yk
A uniform quantization is completely dened by the number of levels, the quantization step and if it is a mid-step or mid-riser quantizer.
output y
A mid-riser quantizer Zero is one of the decision levels tk
output y
t1 t2 t3 t 4 y2 y1
t6 t7 t 8 t 9
input x
t1 t2 t 3 t4 y2 y1
t6 t7 t 8 t9
input x
Usually, a mid-riser quantizer is used if the number of representative levels is even and a mid-step quantizer if the number of level is odd.
68
Uniform quantization with dead zone

Uniform quantization output y
Uniform quantization with dead zone output y
t1 t2 t3 t 4 y2 y1
t6 t7 t 8 t 9
input x
t1 t2 t 3 t4 y2 y1
t6 t7 t 8 t9
input x
Deadzone
Interest: To remove small coecients by favouring the zero value. Increase the coding eciency with a small visual impact.
69
Example of an uniform quantization

Example (Original picture quantized to 8, 7, 6, 4 bits/pixels)
(a) Original (8 bits per pixel)
(b) 7 bpp ( = 2)
(c) 6 bpp ( = 4)
(d) 4 bpp ( = 8)
70
Optimal quantization
Optimal quantization An optimal quantization of a random variable X having a probability distribution p (x ) is obtained by a quantizer that minimises a given metric: Linf -norm: D = max |X Q (X )|; L1 -norm: D = E [|X Q (X )|]; L2 -norm: D = E (X Q (X ))2 called the Mean-Square Error (MSE). This is the most used. Considering the MSE, we have: max p (x )(x Q (x ))2 dx ; if the random variable X is continue: D = xxmin n if the random variable X is discret: D = k =1 p (xk )(xk Q (xk ))2 .
71
Example
Example (Quantization error) Hypothesis: Uniform quantization with a quantization step ; p(x ) is a uniform probability distribution of a random variable X ; N is the number of representative levels. xmax xmin
xmin
xmax xmax xmin N
Quantization step: =
1 1 Probability distribution: p (x ) = xmax xmin = N
72
Example
Example (Quantization error)
= = = =
xmax xmin
p(x )(x Q (x ))2 dx

/2
N N
2
/2 /2
p(x )(x 0)2 dx , 0 the mid-point

1
/2 N
x 2 dx
12
73
Optimal quantizer in a nutshell
Uniform quantizer is not optimal if the source is not uniformly distributed. Optimal quantizer To nd the decision levels {ti } and the representative levels {yi } to minimize the distortion D . To reduce the MSE, the idea is to decrease the bin's size when the probability of occurrence is high and to increase the bin's size when the probability is low. For N representative levels and with a probability density p (x ), the distortion is given by:
xmax
xmin N 1 tk +1 k =1 t k
p(x )(x Q (x ))2 dx p(x )(x yk )2 dx
74
Optimal quantizer in a nutshell
The optimal {ti } and {yi } satisfy:

D ti
=0
D and yi = 0
Lloyd-Max quantizer y +y D = 0 ti = i 2 i +1 . ti ti is the midpoint of yi and yi +1 . ti ti 1 p (x )xdx D = 0 y = . ti i yi ti 1 p (x )dx yi is the centroid of the interval [ti 1 , ti ]. given the {ti }, we can nd the corresponding optimal {yi }. given the {yi }, we can nd the corresponding optimal {ti }. How can we nd the optimal {ti } and {yi } simultaneously?
75
Lloyd-Max algorithm in a nutshell
The Lloyd-Max algorithm is an algorithm for nding the representative levels {yi } and the decision levels {ti } to meet the previous conditions, with no prior knowledge. Principle of the iterative process
1 The iterative process starts for (0) ( 0) levels y1 , ..., yN .
k = 0 with a set of initial values for the representative

(k ) +y
y 2 New values for decision levels are determined ti(k +1) = i

3 Compute the distortion
4 Depending on the stopping criteria ( (k ) < ), stop the process or update the ti(k +1) xp(x )dx (k +1) t 1 representative levels yi(k +1) = i and go back to step 2. ti(k +1) p(x )dx (k +1) ti 1
D (k ) and the relative errors (k ) ;
(k ) i +1
76
Uniform, Semi-uniform and optimal quantizer

Suppose we have the following 1D discrete signal:
= {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}
Example (Uniform quantizer: N = 4, Mid-riser and = 2) output y ti T = {t0 = 0, t1 = 2, t2 = 4, t3 = 6, t4 = 8} ri R = {r0 = 1, r1 = 3, r2 = 5, r3 = 7} 7

5 3 1 0 2 4 6 8 Check the denition of a uniform quantizer...
input x
Quantized vector X = {1, 1, 3, 3, 1, 3, 5, 3, 5, 7, 7} (MSE = 0.42)

77
Uniform, semi-uniform and optimal quantizer

= {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}
Example (Semi-uniform quantizer: N = 4, Mid-riser and = 2) output y ti T = {t0 = 0, t1 = 2, t2 = 4, t3 = 6, t4 = 8} ri R = {r0 = 2/3, r1 = 3.25, r2 = 4.75, r3 = 7.5} 7
5 3 1 0 2 4 6 8
input x
Quantized vector X = {2/3, 2/3, 3.25, 3.25, 2/3, 3.25, 4.75, 3.25, 4.75, 7.5, 7.5} (MSE = 0.31)
78
Uniform, semi-uniform and optimal quantizer

= {0, 0.01, 2.8, 3.4, 1.99, 3.6, 5, 3.2, 4.5, 7.1, 7.9}
Example (Lloyd-Max Algorithm) Given that T = {0, 1.5, 3.87, 6.125, 8} and R = {0.005, 2.998, 4.75, 7.5}, the quantized vector is X = {0.005, 0.005, 2.998, 2.998, 2.998, 2.998, 4.75, 2.998, 4.75, 7.5, 7.5} (MSE=0.18).
output y
Uniform Semi-uniform Optimal 7 5 3 1 0
79
xi
input x
Uniform vs optimal quantizer

Example (N=5)
(a) Original
(b) Uniform
(c) Optimal
(d) Histo.
80
(e) Decision levels
(f) Decision levels
Distortion/quality assessment
Taxonomy Signal delity Perceptual metric Examples
1 2 3 4 5
6 7
Scalar Quantization Distortion/quality assessment Taxonomy Signal delity Perceptual metric Examples Conclusion
81
Taxonomy
Distortion/quality metrics can be divided into 3 categories:

1 2 3
Full-Reference metrics (FR) for which both the original and the distorted images are required (benchmark, compression) ; Reduced-Reference metrics (RR) for which a description of the original and the distorted image is required (network monitoring); No-Reference (NR) metrics for which the original image is not required (network monitoring).
Each category can be divided into two subcategories: metrics based on signal delity and metrics based on properties on the human visual system.
82
Peak Signal to Noise Ratio (PSNR)

The PSNR is the most popular quality metric. This simple metric just calculates the mathematical dierence between each pixel of the degraded image and the original image. PSNR: Let I and D the original and impaired images, respectively. These images having a size of M pixels are coded with n bits. n 1)2 ) PSNR = 10log10 ((2MSE dB, with the Mean Squared Error MSE =
( x ,y ) (
I (x ,y )D (x ,y ))2 . M
A high value indicates that the amount of impairment is small. A small value indicates that there is a strong degradation.
83

The PSNR is not always well correlated with the human judgment (MOS Mean Opinion Score). The reason is simple: this metric does not take into account the properties of the human visual system. Example
(a)
(b)
(c) Original
(d) Original+uniform noise
(a) and (b) from [?]: impact of Gabor patch on our perception.
84

Example (These three pictures have the same PSNR...)
(a) Original
(b) Contrast stretched
(c) Blur
85
(d) JPEG
Metric based on the error visibility
For this type of metric, the behavior of the visual cells are simulated: I Perceptual Color Space PSD CSF Masking
Pooling
Quality
Perceptual Color Space
PSD
CSF
Masking
PSD: Perceptual Subband Decomposition (Wavelet, Gabor, Fourier); CSF: Contrast Sensitivity Function.
86
Metric based on the error visibility
Example VDP (Visible Dierences Predictor) [Daly,93]:
WQA (Wavelet-based Quality Assessment) [Ninassi et al.,08a]
VQM (Video Quality Model) [Pinson et al.,04]...

87
Metric based on the structural similarity

SSIM standing for Structural Similarity index. Image degradations are considered here as perceived structural information loss instead of perceived errors [Wang et al.,04a]. Let I and D the original and the degraded images, respectively.
S (x , y ) = l (x , y )
c (x , y )
s (x , y )
(1)
The luminance comparison measure l (x , y ) = The contrast comparison measure c (x , y ) =
2x y 2 + 2 x y
xy x y
2x y 2 2 x + y
The structural comparison measure s (x , y ) =
SSIM (x , y ) =
(2x y + C1 )(2xy + C2 ) 2 2 2 (2 x + y + C1 )(x + y + C2 )
SSIM 1, the best quality and SSIM 0 indicates a poor quality.

88
Metric based on the structural similarity
All pictures have the same MSE. (a) Original image; (b) Contrast-stretched image, MSSIM = 0.9168; (c) Mean-shifted image, MSSIM = 0.99; (d) JPEG, MSSIM = 0.6949; (e) Blurred image, MSSIM = 0.7052; (f) Impluse noise, MSSIM = 0.7748. Extracted from [Wang et al.,04a].
89
Example of distortion maps

Example
(a) Original
(b) Degraded
(c) MSE
90
(d) WQA
(e) SSIM
Distortion/quality assessment
1 2 3 4 5
6 7 8
91
Conclusion
Image Transformation: global to point; Fourier transformation; Time sampling with Shannon's theorem; Scalar quantization, a lossy process; Quality and distortion.
92
Suggestion for further reading... S. Daly. The visible dierences predictor: An algorithm for the assessment of image delity. Digital Images and Human Vision, pp. 179-206, 1993, MIT Press. A. Ninassi, O. Le Meur, P. Le Callet, and D. Barba. On the performance of human visual system based image quality assessment metric using wavelet domain. Proc. SPIE Human Vision and Electronic Imaging XIII., Vol. 6806, pp. 680610-12, 2008. M. Pinson and S. Wolf. A new standardized method for objectively measuring video quality. IEEE Trans. Broadcasting, Vol. 50, N. 3, pp. 312-322, 2004. Z. Wang, A. C. Bovik, H.R. Sheikh and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Trans. on Image Processing, Vol. 13, pp. 600-612, 2004.
92

Miti Part

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Miti Part

Uploaded by

Copyright:

Available Formats

Table of Content

Digital Image Processing Image Transform (Fourier) / Quantization / Quality

Scalar Quantization Distortion/quality assessment Conclusion

Scalar Quantization Distortion/quality assessment Conclusion

A signal carries information!!

Deterministic vs Random Signals:

Sampling = discrete time (or space)

Quantization = discrete amplitude

Notations and reminder

a continuous signal x (.); a discrete signal x [.]; a matrix A.

Notations and reminder

f is the frequency (Hz); w = 2f is the pulsation (radian/s); T x Lp (R) means that

Euler's formulas: exp (j ) = cos + j sin ;

x (t ) y (t )dt x (n) y (n)

Condition of orthogonality for real basis function: t2 0 n (t )m (t )dt = m t1

Scalar Quantization Distortion/quality assessment Conclusion

Point to point transformation

Geometric transformation such as rotation, scaling... Example of rotation:

Scalar quantization LUT (Look-Up-Table)

Remark: a point to point transformation doesn't require extra memory...

Example of local to point transformation

Global to point transformation

an and bn are used to weight the inuence of each frequency component.

Global to point transformation

Fourier series (development in complex term):

Global to point transformation

Denition Trivial properties Parseval Formula Properties Some examples

Scalar Quantization Distortion/quality assessment Conclusion

Denition Trivial properties Parseval Formula Properties Some examples

Fourier transformation in L2 (R)

Inverse Fourier Transform:

provided that x L2 (R) and X L2 (R).

Denition Trivial properties Parseval Formula Properties Some examples

Img (X (f )) Phase spectrum: Arctg Real (X (f ))

Denition Trivial properties Parseval Formula Properties Some examples

X (f )exp(j 2ft )df

X (f )exp(j 2ft )df

F F In the same way, x (t ) X (f ) and x (t ) X (f ).

Denition Trivial properties Parseval Formula Properties Some examples

Hermitian symmetry: if x (t ) R, we deduce X (f ) = X (f )

x (t )exp(j 2ft )dt x (t )exp(j 2ft )dt

x (t )exp(j 2ft )dt

Given that x (t ) R, we have x (t ) = x (t ) that implies X (f ) = X (f ).

Denition Trivial properties Parseval Formula Properties Some examples

X (f )exp(j 2ft )df dt x (t )exp(j 2ft )dt df

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

F Translation in time/space domain: x (t t0 ) exp(j 2t0 f )X (f )

x (t t0 )exp(j 2tf )dt x (k )exp(j 2(k + t0 )f )dk

x (k )exp(j 2fk )dk

Denition Trivial properties Parseval Formula Properties Some examples

F Convolution in time/space domain: x1 (t ) x2 (t ) X1 (f ) X2 (f ) + + +

x1 ( )x2 (t )d exp(j 2ft )dt

x2 (t )exp(j 2ft )dt d

F Convolution in frequency domain: x1 (t ) x2 (t ) X1 (f ) X2 (f )

Denition Trivial properties Parseval Formula Properties Some examples

F Time and frequency scaling: x (t )

x (t )exp(j 2tf )dt x (k )exp(j 2k f ) dk

Denition Trivial properties Parseval Formula Properties Some examples

x (t )exp(j 2tf )dt

x (t )exp(j 2tf )dt exp(j 2tf )dt

If T = 0, X (f ) = 1. The Fourier transform is constant whatever the frequency. x (t ) = 1, X (f ) = (f )

A [exp(jfT ) exp(jfT )] j 2f ATsinc (fT )

an and bn are used to weight the inuence of each frequency component.

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition Trivial properties Parseval Formula Properties Some examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples

Denition of 1D DFT Denition of 2D DFT Remarks Properties Separable transformation Examples