04 Quantization

Quantization
Heejune AHN
Embedded Communications Laboratory
Seoul National Univ. of Technology
Fall 2013
Last updated 2013. 9. 24
Heejune AHN: Image and Video Compression p. 2
Outline
Quantization Concept
Quantizer Design
Quantizer in Video Coding
Vector Quantizer
Rate-distortion optimization Concept
Lagrangian Method
Appendix
Detail for Lloyd-max Quantization Algorithm
1. Quantization
Reduce representation set
N
large
points ([0 ~ N
large
]) to N
small
points ([0, N
small
])
Inevitable when Analog to Digital Conversion
Not Reversible !

Original
(digital/analog)

ADC/Quant

Quanitzed
Levels

ReScale

Scalar Quantizer
Quantize a single value (single input & single output)
In contrast to VQ (Vector Quantizer, to be discussed later)
Mapping function
{[x
i-1
, x
i
) => y
i
}
Decision levels to
reconstruction level
Quantizer
input
Quantizer
output
x
i-1
x
i

y
i

Output : {y
i
}
Input : x
Q
q
Q
-1
x
q
x
Quantizers Effects
+ Bit Rate Reduction
Bits required to represent the values: (E1bits=> E2 bits)
N
large
(= 2
E1
)points ([0 ~ N]) to N
small
(= 2
E2
)points ([0, N
small
])
Bit reduction = E2 - E1 = log
2
(N
Large
/N
small
)
- Distortion
e = y
i
- x (in [x
i-1
, x
i
))
MSE (mean squared error)

Trade-off problem

| | | | ( )
1 2
1 2
1 1
2
1 2 1 2
0 0
1 2
1

MSE= , ,
N N
n n
x n n x n n
N N

= =


2. Quantization Design
Distortion
Quantization error = difference original (x) and quantized (y
i
)
Distortion Measure

Quantizer design parameters
Step size (N)
decision level (x
i
)
reconstruction level (y
i
)
Distortion depends on Input signal distribution (f(x))
Optimization Problem
How to set the ranges and reconstruction values, i.e., {y
i
& [x
i-1
,
x
i
)} for i =0, N, so that the resulting MSE is minimized.

}
=

=
N
i
x
x
i
i
i
X
dx x f y x D
1

2
1
) ( ) (
Optimal Quantizer
Solutions
Note performance depends on the input statistics.

Simpler design
Uniform (Fixed quant step) quantizer
With/Without dead-zone near zero

Lloyd-Max algorithm
a optimal quantizer design algorithm
More levels for dense region, Less levels for sparse region.
(non-linear quantizer)

Uniform Quantizers
Properties
x
i
and y
i
are spaced evenly (so, uniform), with space (step size)
Mid-tread (odd # of recon. levels) vs Mid-rise (no zero recon. )
mid-tread is more popular than mid-rise.

2 3 Input
-3 -2 -
Reconstruction
3.5
2.5
1.5
0.5
-0.5
-1.5
-2.5
-3.5
Uniform Midrise Quantizer
-2.5 -1.5 -0.5
Reconstruction
3
2

-
-2
-3
Uniform Midtread Quantizer
0.5 1.5 2.5 Input
What will be the mathematical representation for quant/inverse?

Uniform Quantizer Performance
x
A A
A 2
1
) (x f
X
For Uniform dist. Input, Uniform quantizer is optimal
Can you prove this?
Uniform Input X ~ U(-A,A)

Error signal e = y x ~ U(-/2, /2)
Signal Power (variance)

MSE

SNR

Note
If one bit added, is halved, noise variance reduces to 1/4, SNR
increases by 6 dB.
SNR does not depends on input range, MSE does.

2 , 2
) 2 ( 3
1 2
12
1
12
2 2
2
2
2
2
r
x
r
r
N
A
N
A
D = =
|
.
|
\
|
=
A
=

o
bit dB
dB r
D
dB SNR
r
x
/ 6
) ( 02 . 6 ) 2 ( log 10 log 10 ) (
2
2

= = =
o
3
2
2
A
x
= o
Non-uniform Quantizer
Issue
Many data are non-uniformly distributed and even unbounded
Concept
General Intuition: Dense Levels for Dense Input region
Boundness: Use saturation Post filter for unbounded input
Generally no closed-form expression for MSE.
Range and Reconstruction Level is cross-related
Lloyd-Max Algorithm
Algorithm
Iteration algorithm with decision level (x
i
) and recon. level (y
i
)
Run until code book does not change any more
Step 1: find Near Region R
i
((x
i-1 ,
x
i
])
R
i
= {x | d(x,y
i
) <= d(x,y
j
), for all j =\= I }
Step 2: find level y
i

y
i
= E[X| x in R
i
]

C
m C
m+1
{R
i
}
m
centroid
calculation

range
update

}
=
+
= =
L
k
t
t
x
k
k
k dx x f y x y x E
1
2 2
1
) ( ) ( ] ) ' [( c
pdf input on the depending ] | [
) (
) (
2
) (
1
1

1
T e = =
+
=
}
}
+
+
k
t
t
x
t
t
x
k
k k
k
x x E
dx x f
dx u xf
y
y y
t
k
k
k
k
where T
k
is the k th interval [t
k
,t
k+1
)
0 =
c
c
=
c
c
k k
y t
c c
Lloyd-Max Algorithm
Minimize mean squared error.

Example
Quantization and Quality (Lloyd-Max applied)
Original (8bpp) vs 6 bpp (spatial domain)
High Density Quantization
,
x
o << A
i
R
A
conditional pdf of
in the interval
i i i
R R y x
1
in ) (
A
~
0 ] [ ] [
1
~ =
N r r
R P R P
| |
12 12
) (
1
) (
] ) [( ) ( ] )
[( ] )
[(
2 2 /
2 /
2
2
2 2 2
A
=
A
=
A
~
= = =
}

A
A
i
i
i
i
i
i i i
R P de e R P
R y X E R P R X X E E X X E D
(
A A
2
,
2
x
) (x f
X
High Quality
Condition
Observation
Slow change (Constant) over the quantization region
Approximate with Uniform distribution
Quantization in Video Coding
UTQ (uniform Threshold Quantization) often is used
Difficult in adaptive design (a LM quantizer design)
Video quantizers are high quality ones
Quantization in VC
Input signal: quantize the (DCT) coefficients
Parameters
q (): quantization scale factor ( 2 ~ 62, usu. q = 2QUANT, 1 ~31 )
Index (Level) : index for the reconstruction values
Note: index, not reconstruction values are transmitted
E.g.
Index: I (n,m) = floor(F(n,m)/q) = floor(F(n,m)/2QUANT)
i.e. [i-1, i) *q => i
Reconstruction: F(n.m) = (I(n,m) + ) x q = (2*I(n,m) +1)xQUANT
i.e. i => (i +1/2) q (i.e, middle of range [i -1, i)*q)
Note: with QUANT, no 1/2 in expression.

Uniform Threshold Quantizer
Threshold (dead zone) for reducing dct coeff.
Intra DC (zero threshold), Intra AC, inter DC, AC (non-zero)
2*Q(UANT) = q
Quantizer
input
Quantizer
output
q/2
-q/2
q
2q
-q
-2q
-q
-2q
-3q
2q
3q
q
3q
-3q
Quantizer
input

Quantizer
output
2Q
3Q
-2Q
-3Q
-Q
-3Q
-5Q
3Q
5Q
q
5Q
-5Q
AC (With Threshold)
DC (without Threshold, q = 8 )
dead zone
3. Quantization in Video Coding
Applied on the Transform Coefficients
Key Throttle for Bit-Rate and Quality
Very Important Trade-off in Video coding

Estimation
(MC, IntraP)

X
orig

Transform
(e.g. DCT)
e = X
orig
- X
est
Y = T(e)

Quantizati
on
Entrophy
Coding
Y
q
= Q(Y)

VLC(Y
q)
Example #1

Example #2: Comparison with Coarse and Fine Quant
See Textbook Table 7.4
185 3 1 1 -3 2 -1 0
1 1 -1 0 -1 0 0 1
0 0 1 0 -1 0 0 0
1 1 0 -1 0 0 0 -1
0 0 1 0 0 0 -1 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0

Q(8)
1480 26.0 9.5 8.9 -26.4 15.1 -8.1 0.3
11.0 8.3 -8.2 3.8 -8.4 -6.0 -2.8 10.6
-5.5 4.5 9.0 5.3 -8.0 4.0 -5.1 4.9
10.7 9.8 4.9 -8.3 -2.1 -1.9 2.8 -8.1
1.6 1.4 8.2 4.3 3.4 4.1 -7.9 1.0
-4.5 -5.0 -6.4 4.1 -4.4 1.8 -3.2 2.1
5.9 5.8 2.4 2.8 -2.0 5.9 3.2 1.1
-3.0 2.5 -1.0 0.7 4.1 -6.1 6.0 5.7

Zig-zag scan
Transformed 8x8 block
DCT Coef. Quantization
Human visual weight
coarse quant to High frequency components
NDZ-UTQ to Intra DC, DZ-UTQ to all others
Intra block Case
I (n,m) = round(F(n,m)/(q*W(n,n))/8 )
F(n.m) = (I(n,m) + ) x q x W(n,m)/8
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 34 38
22 22 26 27 29 34 37 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 58
26 27 29 34 38 46 56 69
27 29 35 38 46 56 69 83
intra
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
inter
Quantizer Example
MPEG-4 example
Inverse Quantization only Defined
if LEVEL = 0, REC = 0
Else If QUANT is ODD
|REC| = QUANT *(2 |LEVEL| +1)
Else
|REC| = QUANT *(2|LEVEL| +1) -1
+ 1 : Dead-zone
(sorry, I cannot explain why odd/even)
Quantization is not defined
Why?

Examples
Q= 4

31
23
15
7
-7
-15
-23
-31
27
19
11
0
-11
-19
-27
3
Len = 14 > 8
34
26
18
10
-10
-18
-26
-34
27
19
11
0
-11
-19
-27
Len = 14 > 8
Un-biased

biased

4. Vector Quantizater
Vector Quantizer
Use statistical correlation of N dimension Input

Vector Quantizer Design
Generalized Loyd-Max Algorithm

Why Not Vector Quantization in Video Standards
Transform Coding utilizes the Correlation, the remaining signal does
not has Correlation much.

Performance of VQ
Embedded quantizers
Motivation: scalability decoding of compressed
bitstreams at different rates (with different qualities)
Nested quantization intervals

In general, only one quantizer can be optimum
(exception: uniform quantizers)
Q0
Q1
Q2
x
Rate Distortion Theory
Rate vs Distortion
Inverse relation between Information/Data Rate and Distortion
Revisit to a uniform distributed r.v. with a uniform quantizer

Plot it !

2 , 2
) 2 ( 3
1 2
12
1
12
2 2
2
2
2
2
r
x
r
r
N
A
N
A
D = =
|
.
|
\
|
=
A
=

o
Exponential decreasing
Proportional to Complexity of
source in statistics
Rate Distortion Theory
Rate-distortion Function and Theroy
Inverse relation between Information/Data Rate and Distortion
Example with Gaussian R.V.
3.0
2.0
1.0
0.1 0.2 0.4 1
R D
I ( )
( )
D / o
2
0.1 0.5 1.0
) (
) (
R D
I
R
2
o
R D
D
D
D
I ( )
( )
log ,
,

=
s s
>
1
2
0
0
2
2
2
o
o
o
Rate-Distortion Optimization
Motivation
Image coding
Condition: we can use 10KBytes for a
whole picture
Each part has different complexity (i.e,
simpler and complex, variance)
Which parts I have to assign more bits?

Analogy to school score
Students are evaluated sum of Math and
English grade.
I gets 90pts in Math, 50pts in Eng.
Which subject I have to study?
(natural assumption: 90 to 100 is much
harder than 50 to 60)

simpler blocks
More complex blocks
Lagrange multiplier
Constrained Optimization Problem
Minimize distortion with bits no larger than max bits

Cannot use partial differential for minima and maxima

Lagrange Multiplier technique
Insert one more imaginary variable (called Lagrange)
We have multivariable minimization problem
max
R R subject to )} ( min{ < R D
} ) ( min{ R R D J + =
Side note: why not partial differential
Simple example
min subject to

Solution
By substitution
minimum at x = y = 1/sqrt(2)

Wrong Solution
With complex constraints, we cannot use substitution of variable
method
We cannot change partial different with ordinary difference
If we check,

x = y = 0 ?
2 2
y x +
1
1
2 2
) , ( y x y x f + =
1 ) , ( = + = y x y x
0 2
) (
2 2
= =
c
+ c
x
x
y x
0 2
) (
2 2
= =
c
+ c
y
y
y x
An Example: Lagrange Multiplier
Optimization Problem
multiple independent Gaussian variables

Lagrange multiplier optimization
B b
M
i
i
=
s
) (
1
i h
b
J
B b b D J
i
b
i
i
M
i
i
i
i
= =
c
c
+
=

2 2
1
2 ) 2 ln ( 2 0
) ( ) (
o
1 , 2 2 ) ( ), ( ) (
2 2 2 2
1
> = =

=
h h h b D b D B D
i i
b
i
b
i i i i
M
i
i i
o o
i C h
i
b
i
=

for ] ) 2 ln 2 [( 2
1 2 2
o
Solution
Multiplying M equations, and take M-square

Lagrange value

Bit-allocation

Large variance, then more bit allocation
But the increment is logarithmic (not linear)

C
b
M
b
M
b M
i
i
i
i
i
=

|
|
.
|
\
|

|
.
|
\
|

(
(
[
2 2
2
2
1
2
1
2
2 2 2 o
M
b
b
i
M
i
i

[
=
|
|
.
|
\
|
= ,
1
2 2
o
|
|
.
|
\
|
+ =
2
2
2
log
2
1

0
o
i
i
b b
b
M
i
i
b
h h
2
1
2 2 2
2 2 ln 2 2 2 ln 2

|
|
.
|
\
|
= =
[
o
Appendix
Lloyd-Max Algorithm Details
Lloyd-Max scalar quantizer
Problem : For a signal x with given PDF find a quantizer with
M representative levels such that

( )
X
f x

Solution : Lloyd-Max quantizer
[Lloyd,1957] [Max,1960]
M-1 decision thresholds exactly
half-way between representative
levels.
M representative levels in the
centroid of the PDF between two
successive decision thresholds.
Necessary (but not sufficient)
conditions
( )
1
1
1
1
1, 2, , -1
2
( )
0,1, , -1
( )
q
q
q
q
q q q
t
X
t
q
t
X
t
t x x q M
x f x dx
x q M
f x dx
+
+
= + = .
= = .
}
}
( )
2
min. d MSE E X X
(
= =
(

Iterative Lloyd-Max quantizer design
1. Guess initial set of representative levels
2. Calculate decision thresholds

3. Calculate new representative levels

4. Repeat 2. and 3. until no further distortion reduction
0,1, 2, , -1
q
x q M = .
1
1
( )
0,1, , -1
( )
q
q
q
q
t
X
t
q
t
X
t
x f x dx
x q M
f x dx
+
+
= = .
}
}
( )
1
1
1, 2, , -1
2
q q q
t x x q M
= + = .
Example of use of the Lloyd algorithm (I)
X zero-mean, unit-variance Gaussian r.v.
Design scalar quantizer with 4 quantization indices with
minimum expected distortion D*
Optimum quantizer, obtained with the Lloyd algorithm
Decision thresholds -0.98, 0, 0.98
Representative levels 1.51, -0.45, 0.45, 1.51
D*=0.12=9.30 dB

Boundary
Reconstruction

Example of use of the Lloyd algorithm (II)
Convergence
Initial quantizer A:
decision thresholds 3, 0 3
Initial quantizer B:
decision thresholds , 0,

After 6 iterations, in both cases, (D-D*)/D*<1%

5 10 15
-6
-4
-2
0
2
4
6
Q
u
a
n
t
i
z
a
t
i
o
n

F
u
n
c
t
i
o
n
Iteration Number
0 5 10 15
0
0.2
0.4
D
0 5 10 15
0
5
10
S
N
R
O
U
T

[
d
B
]
Iteration Number
SNR
OUT

final
= 9.2978
5 10 15
-6
-4
-2
0
2
4
6
Q
u
a
n
t
i
z
a
t
i
o
n

F
u
n
c
t
i
o
n
Iteration Number
0 5 10 15
0.1
0.15
0.2
D
0 5 10 15
6
8
10
S
N
R
O
U
T

[
d
B
]
Iteration Number
SNR
OUT

final
= 9.298
Example of use of the Lloyd algorithm (III)
X zero-mean, unit-variance Laplacian r.v.
Design scalar quantizer with 4 quantization indices with
minimum expected distortion D*
Optimum quantizer, obtained with the Lloyd algorithm
Decision thresholds -1.13, 0, 1.13
Representative levels -1.83, -0.42, 0.42, 1.83
D*=0.18=7.54 dB

Threshold
Representative

Example of use of the Lloyd algorithm (IV)
Convergence
Initial quantizer A,
decision thresholds 3, 0 3
Initial quantizer B,
decision thresholds , 0,

After 6 iterations, in both cases, (D-D*)/D*<1%

0 5 10 15
0
0.2
0.4
D
0 5 10 15
4
6
8
S
N
R

[
d
B
]
Iteration Number
SNR
final
= 7.5415
2 4 6 8 10 12 14 16
-10
-5
0
5
10
Q
u
a
n
t
i
z
a
t
i
o
n

F
u
n
c
t
i
o
n
Iteration Number
2 4 6 8 10 12 14 16
-10
-5
0
5
10
Q
u
a
n
t
i
z
a
t
i
o
n

F
u
n
c
t
i
o
n
Iteration Number
0 5 10 15
0
0.2
0.4
D
0 5 10 15
4
6
8
S
N
R

[
d
B
]
Iteration Number
SNR
final
= 7.5415
Lloyd algorithm with training data
1. Guess initial set of representative levels
2. Assign each sample x
i
in training set T to closest representative

3. Calculate new representative levels

4. Repeat 2. and 3. until no further distortion reduction
; 0,1, 2, , -1
q
x q M = .
x
1
0,1, , -1
q
q
B
q
x x q M
B
e
= = .
q
x
( ) { }
: 0,1, 2, , -1
q
B x Q x q q M = e = = . T
Lloyd-Max quantizer properties
Zero-mean quantization error

Quantization error and reconstruction decorrelated

Variance subtraction property

0 E X X
(
=

( )

0 E X X X
(
=

( )
2
2 2

X
X
E X X o o
(
=
(

04 Quantization

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

04 Quantization

Uploaded by

Copyright:

Available Formats

Quantization

Heejune AHN: Image and Video Compression p. 6

You might also like