Heejune AHN
Embedded Communications Laboratory
Seoul National Univ. of Technology
Fall 2013
Last updated 2013. 9. 24
Heejune AHN: Image and Video Compression p. 2
Outline
Quantization Concept
Quantizer Design
Quantizer in Video Coding
Vector Quantizer
Ratedistortion optimization Concept
Lagrangian Method
Appendix
Detail for Lloydmax Quantization Algorithm
Heejune AHN: Image and Video Compression p. 3
1. Quantization
Reduce representation set
N
large
points ([0 ~ N
large
]) to N
small
points ([0, N
small
])
Inevitable when Analog to Digital Conversion
Not Reversible !
Original
(digital/analog)
ADC/Quant
Quanitzed
Levels
ReScale
Heejune AHN: Image and Video Compression p. 4
Scalar Quantizer
Quantize a single value (single input & single output)
In contrast to VQ (Vector Quantizer, to be discussed later)
Mapping function
{[x
i1
, x
i
) => y
i
}
Decision levels to
reconstruction level
Quantizer
input
Quantizer
output
x
i1
x
i
y
i
Output : {y
i
}
Input : x
Q
q
Q
1
x
q
x
Heejune AHN: Image and Video Compression p. 5
Quantizers Effects
+ Bit Rate Reduction
Bits required to represent the values: (E1bits=> E2 bits)
N
large
(= 2
E1
)points ([0 ~ N]) to N
small
(= 2
E2
)points ([0, N
small
])
Bit reduction = E2  E1 = log
2
(N
Large
/N
small
)
 Distortion
e = y
i
 x (in [x
i1
, x
i
))
MSE (mean squared error)
Tradeoff problem
    ( )
1 2
1 2
1 1
2
1 2 1 2
0 0
1 2
1
MSE= , ,
N N
n n
x n n x n n
N N
= =
}
=
=
N
i
x
x
i
i
i
X
dx x f y x D
1
2
1
) ( ) (
Heejune AHN: Image and Video Compression p. 7
Optimal Quantizer
Solutions
Note performance depends on the input statistics.
Simpler design
Uniform (Fixed quant step) quantizer
With/Without deadzone near zero
LloydMax algorithm
a optimal quantizer design algorithm
More levels for dense region, Less levels for sparse region.
(nonlinear quantizer)
Heejune AHN: Image and Video Compression p. 8
Uniform Quantizers
Properties
x
i
and y
i
are spaced evenly (so, uniform), with space (step size)
Midtread (odd # of recon. levels) vs Midrise (no zero recon. )
midtread is more popular than midrise.
2 3 Input
3 2 
Reconstruction
3.5
2.5
1.5
0.5
0.5
1.5
2.5
3.5
Uniform Midrise Quantizer
2.5 1.5 0.5
Reconstruction
3
2

2
3
Uniform Midtread Quantizer
0.5 1.5 2.5 Input
What will be the mathematical representation for quant/inverse?
Heejune AHN: Image and Video Compression p. 9
Heejune AHN: Image and Video Compression p. 10
Uniform Quantizer Performance
x
A A
A 2
1
) (x f
X
For Uniform dist. Input, Uniform quantizer is optimal
Can you prove this?
Uniform Input X ~ U(A,A)
Error signal e = y x ~ U(/2, /2)
Heejune AHN: Image and Video Compression p. 11
Signal Power (variance)
MSE
SNR
Note
If one bit added, is halved, noise variance reduces to 1/4, SNR
increases by 6 dB.
SNR does not depends on input range, MSE does.
2 , 2
) 2 ( 3
1 2
12
1
12
2 2
2
2
2
2
r
x
r
r
N
A
N
A
D = =

.

\

=
A
=
o
bit dB
dB r
D
dB SNR
r
x
/ 6
) ( 02 . 6 ) 2 ( log 10 log 10 ) (
2
2
= = =
o
3
2
2
A
x
= o
Heejune AHN: Image and Video Compression p. 12
Nonuniform Quantizer
Issue
Many data are nonuniformly distributed and even unbounded
Concept
General Intuition: Dense Levels for Dense Input region
Boundness: Use saturation Post filter for unbounded input
Generally no closedform expression for MSE.
Range and Reconstruction Level is crossrelated
Heejune AHN: Image and Video Compression p. 13
LloydMax Algorithm
Algorithm
Iteration algorithm with decision level (x
i
) and recon. level (y
i
)
Run until code book does not change any more
Step 1: find Near Region R
i
((x
i1 ,
x
i
])
R
i
= {x  d(x,y
i
) <= d(x,y
j
), for all j =\= I }
Step 2: find level y
i
y
i
= E[X x in R
i
]
C
m C
m+1
{R
i
}
m
centroid
calculation
range
update
Heejune AHN: Image and Video Compression p. 14
}
=
+
= =
L
k
t
t
x
k
k
k dx x f y x y x E
1
2 2
1
) ( ) ( ] ) ' [( c
pdf input on the depending ]  [
) (
) (
2
) (
1
1
1
T e = =
+
=
}
}
+
+
k
t
t
x
t
t
x
k
k k
k
x x E
dx x f
dx u xf
y
y y
t
k
k
k
k
where T
k
is the k th interval [t
k
,t
k+1
)
0 =
c
c
=
c
c
k k
y t
c c
LloydMax Algorithm
Minimize mean squared error.
Heejune AHN: Image and Video Compression p. 15
Example
Quantization and Quality (LloydMax applied)
Original (8bpp) vs 6 bpp (spatial domain)
Heejune AHN: Image and Video Compression p. 16
Original (8bpp) vs 4 bpp (spatial domain)
Heejune AHN: Image and Video Compression p. 17
Original (8bpp) vs 2 bpp (spatial domain)
Heejune AHN: Image and Video Compression p. 18
High Density Quantization
,
x
o << A
i
R
A
conditional pdf of
in the interval
i i i
R R y x
1
in ) (
A
~
0 ] [ ] [
1
~ =
N r r
R P R P
 
12 12
) (
1
) (
] ) [( ) ( ] )
[( ] )
[(
2 2 /
2 /
2
2
2 2 2
A
=
A
=
A
~
= = =
}
A
A
i
i
i
i
i
i i i
R P de e R P
R y X E R P R X X E E X X E D
(
A A
2
,
2
x
) (x f
X
High Quality
Condition
Observation
Slow change (Constant) over the quantization region
Approximate with Uniform distribution
Heejune AHN: Image and Video Compression p. 19
Quantization in Video Coding
UTQ (uniform Threshold Quantization) often is used
Difficult in adaptive design (a LM quantizer design)
Video quantizers are high quality ones
Quantization in VC
Input signal: quantize the (DCT) coefficients
Parameters
q (): quantization scale factor ( 2 ~ 62, usu. q = 2QUANT, 1 ~31 )
Index (Level) : index for the reconstruction values
Note: index, not reconstruction values are transmitted
E.g.
Index: I (n,m) = floor(F(n,m)/q) = floor(F(n,m)/2QUANT)
i.e. [i1, i) *q => i
Reconstruction: F(n.m) = (I(n,m) + ) x q = (2*I(n,m) +1)xQUANT
i.e. i => (i +1/2) q (i.e, middle of range [i 1, i)*q)
Note: with QUANT, no 1/2 in expression.
Heejune AHN: Image and Video Compression p. 20
Uniform Threshold Quantizer
Threshold (dead zone) for reducing dct coeff.
Intra DC (zero threshold), Intra AC, inter DC, AC (nonzero)
2*Q(UANT) = q
Quantizer
input
Quantizer
output
q/2
q/2
q
2q
q
2q
q
2q
3q
2q
3q
q
3q
3q
Quantizer
input
Quantizer
output
2Q
3Q
2Q
3Q
Q
3Q
5Q
3Q
5Q
q
5Q
5Q
AC (With Threshold)
DC (without Threshold, q = 8 )
dead zone
Heejune AHN: Image and Video Compression p. 21
3. Quantization in Video Coding
Applied on the Transform Coefficients
Key Throttle for BitRate and Quality
Very Important Tradeoff in Video coding
Estimation
(MC, IntraP)
X
orig
Transform
(e.g. DCT)
e = X
orig
 X
est
Y = T(e)
Quantizati
on
Entrophy
Coding
Y
q
= Q(Y)
VLC(Y
q)
Heejune AHN: Image and Video Compression p. 22
Example #1
Example #2: Comparison with Coarse and Fine Quant
See Textbook Table 7.4
185 3 1 1 3 2 1 0
1 1 1 0 1 0 0 1
0 0 1 0 1 0 0 0
1 1 0 1 0 0 0 1
0 0 1 0 0 0 1 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Q(8)
1480 26.0 9.5 8.9 26.4 15.1 8.1 0.3
11.0 8.3 8.2 3.8 8.4 6.0 2.8 10.6
5.5 4.5 9.0 5.3 8.0 4.0 5.1 4.9
10.7 9.8 4.9 8.3 2.1 1.9 2.8 8.1
1.6 1.4 8.2 4.3 3.4 4.1 7.9 1.0
4.5 5.0 6.4 4.1 4.4 1.8 3.2 2.1
5.9 5.8 2.4 2.8 2.0 5.9 3.2 1.1
3.0 2.5 1.0 0.7 4.1 6.1 6.0 5.7
Zigzag scan
Transformed 8x8 block
Heejune AHN: Image and Video Compression p. 23
DCT Coef. Quantization
Human visual weight
coarse quant to High frequency components
NDZUTQ to Intra DC, DZUTQ to all others
Intra block Case
I (n,m) = round(F(n,m)/(q*W(n,n))/8 )
F(n.m) = (I(n,m) + ) x q x W(n,m)/8
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 34 38
22 22 26 27 29 34 37 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 58
26 27 29 34 38 46 56 69
27 29 35 38 46 56 69 83
intra
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
16 16 16 16 16 16 16 16
inter
Heejune AHN: Image and Video Compression p. 24
Quantizer Example
MPEG4 example
Inverse Quantization only Defined
if LEVEL = 0, REC = 0
Else If QUANT is ODD
REC = QUANT *(2 LEVEL +1)
Else
REC = QUANT *(2LEVEL +1) 1
+ 1 : Deadzone
(sorry, I cannot explain why odd/even)
Quantization is not defined
Why?
Examples
Q= 4
31
23
15
7
7
15
23
31
27
19
11
0
11
19
27
3
Len = 14 > 8
34
26
18
10
10
18
26
34
27
19
11
0
11
19
27
Len = 14 > 8
Unbiased
biased
Heejune AHN: Image and Video Compression p. 25
4. Vector Quantizater
Vector Quantizer
Use statistical correlation of N dimension Input
Heejune AHN: Image and Video Compression p. 26
Vector Quantizer Design
Generalized LoydMax Algorithm
Why Not Vector Quantization in Video Standards
Transform Coding utilizes the Correlation, the remaining signal does
not has Correlation much.
Heejune AHN: Image and Video Compression p. 27
Heejune AHN: Image and Video Compression p. 28
Performance of VQ
Heejune AHN: Image and Video Compression p. 29
Embedded quantizers
Motivation: scalability decoding of compressed
bitstreams at different rates (with different qualities)
Nested quantization intervals
In general, only one quantizer can be optimum
(exception: uniform quantizers)
Q0
Q1
Q2
x
Heejune AHN: Image and Video Compression p. 30
Rate Distortion Theory
Rate vs Distortion
Inverse relation between Information/Data Rate and Distortion
Revisit to a uniform distributed r.v. with a uniform quantizer
Plot it !
2 , 2
) 2 ( 3
1 2
12
1
12
2 2
2
2
2
2
r
x
r
r
N
A
N
A
D = =

.

\

=
A
=
o
Exponential decreasing
Proportional to Complexity of
source in statistics
Heejune AHN: Image and Video Compression p. 31
Rate Distortion Theory
Ratedistortion Function and Theroy
Inverse relation between Information/Data Rate and Distortion
Example with Gaussian R.V.
3.0
2.0
1.0
0.1 0.2 0.4 1
R D
I ( )
( )
D / o
2
0.1 0.5 1.0
) (
) (
R D
I
R
2
o
R D
D
D
D
I ( )
( )
log ,
,
=
s s
>
1
2
0
0
2
2
2
o
o
o
Heejune AHN: Image and Video Compression p. 32
RateDistortion Optimization
Motivation
Image coding
Condition: we can use 10KBytes for a
whole picture
Each part has different complexity (i.e,
simpler and complex, variance)
Which parts I have to assign more bits?
Analogy to school score
Students are evaluated sum of Math and
English grade.
I gets 90pts in Math, 50pts in Eng.
Which subject I have to study?
(natural assumption: 90 to 100 is much
harder than 50 to 60)
simpler blocks
More complex blocks
Heejune AHN: Image and Video Compression p. 33
Lagrange multiplier
Constrained Optimization Problem
Minimize distortion with bits no larger than max bits
Cannot use partial differential for minima and maxima
Lagrange Multiplier technique
Insert one more imaginary variable (called Lagrange)
We have multivariable minimization problem
max
R R subject to )} ( min{ < R D
} ) ( min{ R R D J + =
Heejune AHN: Image and Video Compression p. 34
Side note: why not partial differential
Simple example
min subject to
Solution
By substitution
minimum at x = y = 1/sqrt(2)
Wrong Solution
With complex constraints, we cannot use substitution of variable
method
We cannot change partial different with ordinary difference
If we check,
x = y = 0 ?
2 2
y x +
1
1
2 2
) , ( y x y x f + =
1 ) , ( = + = y x y x
0 2
) (
2 2
= =
c
+ c
x
x
y x
0 2
) (
2 2
= =
c
+ c
y
y
y x
Heejune AHN: Image and Video Compression p. 35
An Example: Lagrange Multiplier
Optimization Problem
multiple independent Gaussian variables
Lagrange multiplier optimization
B b
M
i
i
=
s
) (
1
i h
b
J
B b b D J
i
b
i
i
M
i
i
i
i
= =
c
c
+
=
2 2
1
2 ) 2 ln ( 2 0
) ( ) (
o
1 , 2 2 ) ( ), ( ) (
2 2 2 2
1
> = =
=
h h h b D b D B D
i i
b
i
b
i i i i
M
i
i i
o o
i C h
i
b
i
=
for ] ) 2 ln 2 [( 2
1 2 2
o
Heejune AHN: Image and Video Compression p. 36
Solution
Multiplying M equations, and take Msquare
Lagrange value
Bitallocation
Large variance, then more bit allocation
But the increment is logarithmic (not linear)
C
b
M
b
M
b M
i
i
i
i
i
=


.

\


.

\

(
(
[
2 2
2
2
1
2
1
2
2 2 2 o
M
b
b
i
M
i
i
[
=


.

\

= ,
1
2 2
o


.

\

+ =
2
2
2
log
2
1
0
o
i
i
b b
b
M
i
i
b
h h
2
1
2 2 2
2 2 ln 2 2 2 ln 2


.

\

= =
[
o
Appendix
LloydMax Algorithm Details
Heejune AHN: Image and Video Compression p. 38
LloydMax scalar quantizer
Problem : For a signal x with given PDF find a quantizer with
M representative levels such that
( )
X
f x
Solution : LloydMax quantizer
[Lloyd,1957] [Max,1960]
M1 decision thresholds exactly
halfway between representative
levels.
M representative levels in the
centroid of the PDF between two
successive decision thresholds.
Necessary (but not sufficient)
conditions
( )
1
1
1
1
1, 2, , 1
2
( )
0,1, , 1
( )
q
q
q
q
q q q
t
X
t
q
t
X
t
t x x q M
x f x dx
x q M
f x dx
+
+
= + = .
= = .
}
}
( )
2
min. d MSE E X X
(
= =
(
Heejune AHN: Image and Video Compression p. 39
Iterative LloydMax quantizer design
1. Guess initial set of representative levels
2. Calculate decision thresholds
3. Calculate new representative levels
4. Repeat 2. and 3. until no further distortion reduction
0,1, 2, , 1
q
x q M = .
1
1
( )
0,1, , 1
( )
q
q
q
q
t
X
t
q
t
X
t
x f x dx
x q M
f x dx
+
+
= = .
}
}
( )
1
1
1, 2, , 1
2
q q q
t x x q M
= + = .
Heejune AHN: Image and Video Compression p. 40
Example of use of the Lloyd algorithm (I)
X zeromean, unitvariance Gaussian r.v.
Design scalar quantizer with 4 quantization indices with
minimum expected distortion D*
Optimum quantizer, obtained with the Lloyd algorithm
Decision thresholds 0.98, 0, 0.98
Representative levels 1.51, 0.45, 0.45, 1.51
D*=0.12=9.30 dB
Boundary
Reconstruction
Heejune AHN: Image and Video Compression p. 41
Example of use of the Lloyd algorithm (II)
Convergence
Initial quantizer A:
decision thresholds 3, 0 3
Initial quantizer B:
decision thresholds , 0,
After 6 iterations, in both cases, (DD*)/D*<1%
5 10 15
6
4
2
0
2
4
6
Q
u
a
n
t
i
z
a
t
i
o
n
F
u
n
c
t
i
o
n
Iteration Number
0 5 10 15
0
0.2
0.4
D
0 5 10 15
0
5
10
S
N
R
O
U
T
[
d
B
]
Iteration Number
SNR
OUT
final
= 9.2978
5 10 15
6
4
2
0
2
4
6
Q
u
a
n
t
i
z
a
t
i
o
n
F
u
n
c
t
i
o
n
Iteration Number
0 5 10 15
0.1
0.15
0.2
D
0 5 10 15
6
8
10
S
N
R
O
U
T
[
d
B
]
Iteration Number
SNR
OUT
final
= 9.298
Heejune AHN: Image and Video Compression p. 42
Example of use of the Lloyd algorithm (III)
X zeromean, unitvariance Laplacian r.v.
Design scalar quantizer with 4 quantization indices with
minimum expected distortion D*
Optimum quantizer, obtained with the Lloyd algorithm
Decision thresholds 1.13, 0, 1.13
Representative levels 1.83, 0.42, 0.42, 1.83
D*=0.18=7.54 dB
Threshold
Representative
Heejune AHN: Image and Video Compression p. 43
Example of use of the Lloyd algorithm (IV)
Convergence
Initial quantizer A,
decision thresholds 3, 0 3
Initial quantizer B,
decision thresholds , 0,
After 6 iterations, in both cases, (DD*)/D*<1%
0 5 10 15
0
0.2
0.4
D
0 5 10 15
4
6
8
S
N
R
[
d
B
]
Iteration Number
SNR
final
= 7.5415
2 4 6 8 10 12 14 16
10
5
0
5
10
Q
u
a
n
t
i
z
a
t
i
o
n
F
u
n
c
t
i
o
n
Iteration Number
2 4 6 8 10 12 14 16
10
5
0
5
10
Q
u
a
n
t
i
z
a
t
i
o
n
F
u
n
c
t
i
o
n
Iteration Number
0 5 10 15
0
0.2
0.4
D
0 5 10 15
4
6
8
S
N
R
[
d
B
]
Iteration Number
SNR
final
= 7.5415
Heejune AHN: Image and Video Compression p. 44
Lloyd algorithm with training data
1. Guess initial set of representative levels
2. Assign each sample x
i
in training set T to closest representative
3. Calculate new representative levels
4. Repeat 2. and 3. until no further distortion reduction
; 0,1, 2, , 1
q
x q M = .
x
1
0,1, , 1
q
q
B
q
x x q M
B
e
= = .
q
x
( ) { }
: 0,1, 2, , 1
q
B x Q x q q M = e = = . T
Heejune AHN: Image and Video Compression p. 45
LloydMax quantizer properties
Zeromean quantization error
Quantization error and reconstruction decorrelated
Variance subtraction property
0 E X X
(
=
( )
0 E X X X
(
=
( )
2
2 2
X
X
E X X o o
(
=
(