Transform Coding

Why Transform?
Purpose of transform

Remove correlation between input samples Transform most energy of an input block into a few coefficients Small coefficients can be discarded by quantization without too much impact to reconstruction quality

Block transform vs. global transform

Block transform: DCT Global transform: Wavelet

Block Transform
Divide input data into blocks Encode each block separately (sometimes with information from neighboring blocks) Most DCT-based image/video/audio coding standards Reduce delay and the complexity (e.g. memory requirement)

2-D DCT Example

Apply 8x8 DCT to each 8x8 block Histograms of source and DCT coefficients

Matrix Representation of Transform

Linear transform is an N N matrix: yN1=TNN* xN1 Inverse Transform: x=T-1*y

Orthogonal Transform: T1 = TT

T*TT= I

For orthogonal transform: Rows/Cols are orthogonal to each others

Optimum Transform
If Ax = x,

: Eigenvalues of A x: Eigenvectors of A

If A symmetric A has N orthogonal eigenvectors xi:

Let U = [x1, x2, , xN] U1AU= =diag(1, 2,, N) Matrix A can be diagonalized by its eigenvectors. Usually R xx R xx is not diagonal since data are correlated If we choose T= R xx Ryy= T R xx T Karhunen-Love Transform (KLT) Different components of Y are uncorrelated. Desired for compression.

KLT is not practical

Need to estimate Rxx Finding eigenvectors are time consuming Need to send T as side information Fast implementation of the KLT is generally not available

Find a nice approximation of the KLT for a given Rxx.

Discrete Cosine Transform (DCT)

AR(1) signal
x(n) = x(n 1) + e(n)
A good approximate for natural image 0.9<<0.1

DCT is the approximation of the KLT of AR(1) signal when its correlation coefficient is close to 1 (e.g. 0.95) DCT Definition

2-D Separable DCT

Apply T to each row Then apply T to each column

2-D 8-point DCT Example

JPEG
Joint Photographic Experts Group
ISO/IEC JTC1 SC29 WG1 Formed in 1986 by ISO and CCITT (ITU-T) Became International Standard (IS) in 1991 ISO/IEC IS 10918-1 (ITU-T T.81): Requirements and guidelines ISO/IEC IS 10918-2 (ITU-T T.83): Compliance testing ISO/IEC IS 10918-3 (ITU-T T.84): Extensions

Digital Coding of Continuous-Tone Still Images (grayscale or color)

Compression ratio 10 to 50; 0.5 to 2 bpp.

Baseline JPEG
8x8

DCT

DCT

JPEG Quantization
Uniform mid-tread quantizer Larger step sizes for chroma components Different coefficients have different step sizes
Smaller sizes for low frequency coefficients (more bits) Larger sizes for low frequency coefficients (less bits) Human visual system is not sensitive to error in high frequency.

Entropy Coding
Zig-Zag Scanning

First DCT Coefficient: DC

Others (63 Coefficients): AC

Entropy Coding

DCT

DC Prediction
DC Coefficients: Average of a block DC of neighboring blocks are still similar to each others: redundancy The redundancy can be removed by differential coding:
e(n) = DC(n) DC(n-1)

Only encode the prediction error e(n)

JPEG for Color Image

Color converter: RGB to YUV Level offset: subtract 2^(N-1). N: bits / pixel. Quantization: Different step size for different coeffs

DC: Predict from DC of previous block AC:

Zigzag scan to get 1-D data Run-level: joint coding of non-zero coeffs and number of zeros before it.

Quality Factor
Actual step size: Scale the basic table by a quality factor. Actual Q table = scaling x Basic Q table:
Q: quality factor 50: scaling = 50 / quality; quality factor > 50: scaling = 2 - quality / 50;

Rate Control
Controlling Quality
Q: Quality Factor (0<Q<100) Higher Q Better Quality

Controlling Bit-rate
Use a lookup-table and trade-off Lower Q for more complex picture Higher Q for simple picture

JPEG Example

JPEG Quantization

JPEG Example

JPEG Implementation

JPEG Implementation

JPEG 2000
Image Coding System (JTC 1.29.14, ISO 15444)

Goals
Low bit-rate compression e.g., below 0.25 bpp for highly detailed gray-level images Lossless and lossy compression in a single bitstream Large images (More than 64K by 64K) Single decompression architecture Transmission in noisy environments Computer generated imagery Compound documents: bi-level and gray-scale

JPEG Implementation

JPEG Implementation

JPEG Implementation

JPEG Implementation

JPEG Types
Baseline JPEG Sequential JPEG Progressive JPEG
Lossless Lossy

Progressive JPEG
Baseline JPEG encodes the image block by block:
Decoder has to wait till the end to decode and display the entire image.

Progressive: Coding DCT coefficients in multiple scans:

The first scan generates a low-quality version of the entire image Subsequent scans refine the entire image gradually.

Two procedures defined in JPEG:

Spectral selection:
* Divide all DCT coefficients into several bands (low, middle, high frequency subbands) * Bands are coded into separate scans

Successive approximation:
* Send MSB of all coefficients first. * Send lower significant bits in subsequent scans.

Objective Quality Measure

PSNR (dB): Peak Signal-to-Noise-Ratio For 8-bit Images of size M x N:

Objective Quality for Lena

JPEG 2000
Image Coding System (JTC 1.29.14, ISO 15444)

Goals
Low bit-rate compression e.g., below 0.25 bpp for highly detailed gray-level images Lossless and lossy compression in a single bitstream Large images (More than 64K by 64K) Single decompression architecture Transmission in noisy environments Computer generated imagery Compound documents: bi-level and gray-scale

Objective Quality Measure

PSNR (dB): Peak Signal-to-Noise-Ratio For 8-bit Images of size M x N: