You are on page 1of 37

EMBEDDED ZEROTREE WAVELET COMPRESSION

Neyre Tekbyk

And

Hakan evki Tozkoparan





Undergraduate Project Report
submitted in partial fulfillment of
the requirements for the
degree of Bachelor of Science (B.S.)





in

Electrical and Electronic Engineering Department
Eastern Mediterranean University



June 2005

2




Approval of the Electrical and Electronic Engineering Department



______________________________

Prof. Dr. Dervi Z. Deniz
Chairman





This is to certify that we have read this thesis and that in our opinion it is fully adequate,
in cope and quality, as an Undergraduate Project.




______________________________
...
Supervisor



Members of the examining committee

Name Signature

1. Prof. Dr. Haluk Tosun ...

2. Asst. Prof. Dr. Hseyin Bilgekul ..
.
3. Asst. Prof Dr. Hasan Abou Rajab ...

4. Asst. Prof. Dr. Erhan Ince ..

5. Asst. Prof. Dr. Aykut Hocann ..



Date: ...

3




ABSTRACT

EMBEDDED ZEROTREE WAVELET COMPRESSION

by

Neyre Tekbyk and Hakan evki Tozkoparan

Electrical and Electronic Engineering Department
Eastern Mediterranean University


Supervisor: Asst. Prof. Dr. Erhan A. NCE

Keywords: EZW, Successive Approximation Quantization, Entropy Coding,
Discrete Wavelet Transform

Image compression is very important in many applications, especially for progressive
transmission, image browsing and multimedia applications. The whole aim is to obtain
the best image quality and yet occupy less space. Embedded zerotree wavelet
compression (EZW) is a kind of image compression that can realize this goal. EZW
algorithm is fairly general and performs remarkably well with most types of images.
Also, it is applicable to transmission over a noisy channel. This project will focus on
embedded zerotree wavelet (EZW) compression for images. The objective is to explain
the idea of EZW compression, design a matlab program which will apply an embedded
zerotree wavelet compression algorithm on an image. Eventually, we should achieve the
best image quality for a given bit rate in such a way that all encodings of the same image
at lower bit rates are embedded in the beginning of the bit stream for the target bit rate.
The report will introduce the concept of zerotree and show how zerotree coding can
efficiently encode[9] a significance map of wavelet coefficients by predicting the absence
of significant information across scales. We will reveal how successive approximation is
used in conjunction with zerotree coding and arithmetic coding to accomplish efficient
embedded coding. Finally, the MATLAB program that we are going to write, will enable
embedded zerotree wavelet algorithm. So, we will be able to apply this algorithm on an
image to see its effects by achieving some experimental results for various rates.

4




Acknowledgments



We would like to thank our supervisor, Asst. Prof. Dr. Erhan Ince, for his invaluable
guidance and support throughout the entire process of this work. His valuable critics were
largely responsible for the success of the final version. We would like to thank all our
families for their love and caring at all times. The number of people that have contributed
to our both personal and scientific life during the last four years is too high to list them
explicitly, so rather than risking forgetting anybody, we would just like to thank all of
you.

5



Table of Contents


ABSTRACT....................................................................................................................... 3
ACKNOWLEDGMENTS ................................................................................................ 4
TABLE OF CONTENTS.................................................................................................. 5
LIST OF FIGURES .......................................................................................................... 6
LIST OF TABLES ............................................................................................................ 7
LIST OF EQUATIONS................................................................................................ 8
1. INTRODUCTION......................................................................................................... 9
2. EMBEDDED ZEROTREE WAVELET CONCEPTS ............................................ 11
2.1 EMBEDDED CODING ................................................................................................. 11
2.2 WAVELET TRANSFORM............................................................................................ 12
2.2.1 Continuous Wavelet Transform....................................................................... 15
2.2.2 Discrete Wavelet Transform............................................................................ 16
2.2.3 DWT and Filter Banks ..................................................................................... 19
2.3 ZEROTREES OF WAVELET COEFFICIENTS................................................................... 20
2.3.1 Significance Map Encoding ............................................................................. 20
2.3.2 Compression of Significance Maps Using Zerotrees of Wavelet Coefficients 23
2.4 SUCCESSIVE APPROXIMATION ENTROPY-CODED QUANTIZATION .............................. 27
2.5 AN EXAMPLE ABOUT EZW......................................................................................... 29
3. EXPERIMENTAL RESULTS....................................................................................32
4. CONCLUSION ............................................................................................................33
5. FUTURE WORK.........................................................................................................34
6. REFERENCES.35


6



List of Figures

Figure 1. Embedded Zerotree Wavelet Compression Chart ........................................................ 10
Figure 2. Embedded Coding Scheme........................................................................................... 12
Figure 3. Demonstration of (a) a Wave and (b) a Wavelet .......................................................... 13
Figure 4. Wavelet Transform Diagram........................................................................................ 15
Figure 5. First Stage of a discrete wavelet transform................................................................... 16
Figure 6. A two scale wavelet decomposition.............................................................................. 17
Figure 7. 2 scale wavelet decomposition (matlab simulation) ..................................................... 17
Figure 8. 3 scale wavelet decomposition ..................................................................................... 18
Figure 9. 3 scale wavelet decomposition (matlab simulation) ..................................................... 18
Figure 10. Original picture after reconstruction............................................................................ 19
Figure 11. Successive lowpass and highpass filtering ................................................................. 19
Figure 12. Reconstruction of the original signal from Wavelet Coefficients .............................. 20
Figure 13. Quantized Coefficients and Significance Map ........................................................... 21
Figure 14. Low bit rate image coder ............................................................................................ 22
Figure 15. Zig-zag manner Scanning order................................................................................. 25

Figure 16. Morton Scan order ...................................................................................................... 25
Figure 17. Flow chart for encoding a coefficient of the significant map..................................... 25
Figure 18. Example of 3 scale wavelet Transform of an 8 x 8 image.......................................... 28



7



List of Tables

Table 1. First Dominant pass at threshold T = 32........................................................................ 30
Table 2. Results of the Example................................................................................................... 31
Table 3. Experimental Results For bird.gif.. ............................................................................ 33





































8




List of Equations


Figure 1. Wavelet Transform Formula................................................. 14
Figure 2. Wavelet Reconstruction Formula.. ....................................... 14
Figure 3. Continuous Wavelet Transform Formula. .................................... 15
Figure 4. Simple Representation of the formula of CWT.. .......................................... 15
Figure 5. Entropy of the symbols H.. ............................................... 22
Figure 6. Cost of encoding the bit stream. ........................................... 23






























9







1. Introduction

The embedded zerotree wavelet algorithm (EZW) [1] is an effective image
compression algorithm. This new technique produces a fully embedded bit stream for
image coding. Also, the compression performance of this algorithm is competitive with
virtually all known techniques. Furthermore, this technique requires definitely no
training, no pre-stored codebooks or tables and requires no preceding knowledge of the
image source. The EZW algorithm is based on four principal concepts: a discrete wavelet
transform or hierarchical sub-band decomposition, prediction of the absence of
significant information across scales, entropy coded successive-approximation
quantization and universal lossless data compression which is accomplished via adaptive
arithmetic coding. The EZW algorithm includes: a discrete wavelet transform (DWT) [4],
zerotree coding, successive approximation and a priorization protocol which helps us to
determine the order of importance due to various characteristics. In addition, adaptive
arithmetic coding is used to achieve a fast and efficient method for entropy coding. We
want to call attention to the following sentence. This algorithm runs consecutively and
stops whenever a target bit rate or a target distortion is met. Certainly, one of the most
significant parts of this algorithm is the encoding process. As mentioned above, in our
case, discrete wavelet transform is used. The original image is passed through discrete
wavelet transformation to produce transform coefficients. The aim of this transformation
is to produce de-correlated coefficients and remove dependencies between samples as
much as possible. Besides, the more significant bits of precision of most coefficients are
allowed by this transformation to be efficiently encoded as part of exponentially growing
zerotrees. This transformation is considered to be lossless. The produced transform
coefficients are then quantized to produce a stream of symbols which will be used in
compression. Almost all of the information loss occurs in the quantization[10] stage. For
embedded coding, the successive approximation quantization is used which allows the
coding of multiple significance maps using zerotrees, and allows the encoding and

10



decoding to stop at any point. Therefore, in quantization stage the threshold value is
arranged to determine significance. The coding process allows the entropy coder to
incorporate learning into the bit stream itself. As a final part of encoding, the data
compression stage takes the stream of symbols and attempts to represent the data stream
with no loss as efficiently as possible. In decoding operation converse of these concepts
will be applied. In the decoding process after receiving the compressed image data,
entropy decoding, de-quantization and inverse DWT will be applied consecutively. In the
decoding operation, each decoded symbol refines and reduces the width of the
uncertainty interval in which the true value of the coefficient may occur. A brief literature
search has only found two published results where authors generate an actual bit stream
that claims higher Peak Signal to Noise Ratio (PSNR) performance at rates between 0.25
and 1 bit/pixel, the latter of which is a variation of the EZW. The performance of the
EZW coder was also compared widely available version of JPEG. It is seen that JPEG
does not allow the user to select a target bit rate.

EZW algorithm is an evident advantage. The user can choose a bit rate and encode[3] the
image to exactly the wanted bit rate. Below you will find the step-by-step explanations on
how one can apply the EZW coding algorithm:

Figure 1. Embedded Zerotree Wavelet Compression Chart

11
2. Embedded Zero Tree Wavelet Concepts

2.1 Embedded Coding

The embedded zerotree wavelet algorithm (EZW) is a simple [1] and remarkably
effective, image compression [5] algorithm. This coding [12] algorithm producing an
embedded code has the property that the bits in the bit stream are generated in order of
importance. This property allows all the low rate codes to be included at the beginning of the
bit stream. The embedded code represents a sequence of binary decisions that distinguish an
image from the null image, or all gray, image. Using an embedded code, an encoder can
terminate the encoding at any point and allows a target rate or distortion metric to be met
exactly. When the target is met the encoding simply stops. The embedded coder can cease at
any time and provide the best representation of an image achievable within its framework.
The binary coded stream can realize progressive transmission using multi-threshold
embedded zerotree wavelet coding. Therefore, the coding rate and distortion can be
controlled accurately. According to the order of importance of the bit plane, the most
important bit plane is coded first. The coding can end at any time when the bit budget is
reached or compression ratio is attained. When transmission occurs, the most important bit
plane is transmitted first, and the least important is transmitted last. Since the information
which is needed to represent an image coded at some rate always contains the needed
information for the same image coded at lower rates, it is possible to choose a fixed target bit
rate. Hence, the decoder can cease decoding the bit stream at any point, and simulate an
image coded at a lower rate corresponding to the truncated bit stream. This means that, the
decoder can interrupt the decoding process at any point in the bit stream, and still reconstruct
the image. Thus, a compression scheme which generates an embedded code can start sending
over the network the coarser version of the image first. Then, it continues with the
progressive transmission of the refinement details.
The schematic diagram of an embedded image coding scheme that employs the image
transform, can be seen in Figure 2. The image coder constitutes of three main blocks which
are: transform, quantizer, and entropy coder [7]. The purpose of an image transform (T)
operated on an image (I) is to decorrelate the image data. The output C of the transform can
be attained or calculated as C = TI.

12

Figure 2. Embedded Coding Scheme

In the case of the DWT (image transform), C is the subband decomposed version of the
original image. It consists of the wavelet coefficients at different scales, and the lowest
frequency contents of the image at the coarsest scale. The image I can be perfectly recovered
by operating the inverse transform on C. The next block in Figure is a quantizer. It quantizes
the transformed image into a sequence of integers. Output of the quantizer is coded using an
entropy coding scheme.
In addition, the EZW encoder is based on progressive encoding which is also known
as embedded encoding to compress an image into a bit stream with increasing accuracy. This
means that when more bits are added to the stream, as a result, the decoded image will
contain more detail. Every digit which is added increases the accuracy of the number,
however we can stop at any accuracy we like. It can be said, embedded coding is similar in a
way to the binary finite precision representations of real numbers. All real numbers can be
represented by a string of binary digits. For each digit added to the right, more precision is
added so that the decoded image will have more detail.
It is known that, typical image coders require extensive training for both quantization
and generation of nonadaptive entropy codes. However, the embedded coder attempts to be
universal by incorporating all learning process in to the bit stream.

2.2 Wavelet Transform

The Wavelet Transform enables a time-frequency representation of the signal. It was
developed to overcome the short coming of the Short Time Fourier Transform (STFT), which
can also be used to analyze non-stationary signals. While STFT gives a constant resolution at
all frequencies, the Wavelet Transform uses multi-resolution technique by which different
frequencies are analyzed with different resolutions.
A wave is an oscillating function of time or space and is periodic. In contrast, wavelets are
localized waves. They have their energy concentrated in time or space and are suited to

13
analysis of transient signals. While Fourier Transform and STFT use waves to analyze
signals, the Wavelet Transform uses wavelets of finite energy. A demonstration of a wave
and wavelet can be seen below in Figure 3.


(a) (b)
Figure 3. Demonstration of (a) a Wave and (b) a Wavelet [16]

The wavelet analysis is done similar to the STFT analysis. The signal to be analyzed is
multiplied with a wavelet function just as it is multiplied with a window function in STFT,
and then the transform is computed for each segment generated. However, unlike STFT, in
Wavelet Transform, the width of the wavelet function changes with each spectral component.
The Wavelet Transform, at high frequencies, gives good time resolution and poor frequency
resolution, while at low frequencies, the Wavelet Transform gives good frequency resolution
and poor time resolution.

Wavelet analysis [2] is a technique to transform an array of N numbers from their actual
numerical values to an array of N wavelet coefficients. Each wavelet coefficient represents
the closeness of the fit (or correlation) between the wavelet function at a particular size and a
particular location within the data array. By varying the size of the wavelet function (usually
in powers-of-two) and shifting the wavelet so it covers the entire array, you can build up a
picture of the overall match between the wavelet function and your data array. Since the
wavelet functions are compact (hence the term wavelet), the wavelet coefficients only
measure the variations around a small region of the data array. This property makes wavelet
analysis very useful for signal or image processing; the "localized" nature of the wavelet
transform allows you to easily pick out features in your data such as spikes (for example,
noise or discontinuities), discrete objects (in, for example, astronomical images or satellite
photos), edges of objects, etc.

In general wavelets are functions that can be used to efficiently represent other functions.
Wavelet representation is a recent technique and connected to image representation. Simply
we can say wavelets allow complex filters to be constructed for digital data which can

14
remove or enhance selected parts of the signal. If we compare Wavelet Transform with the
Fourier transform (FT), FT is less useful in analyzing non-stationary data, where there is no
repetition within the region sampled. Wavelet transforms (of which there are, at least
formally, an infinite number) allow the components of a non-stationary signal to be analyzed.
Wavelets also allow filters to be constructed for stationary and non-stationary signals.
The wavelet Transform formula could be seen below in Equation 1:

( ) ( ) 0 ,
1
, >
(

|
.
|

\
|
=


s dt
s
t
h
s
t x s X



Equation 1. Wavelet Transform Formula

Where; s = scale , = time shift

For reconstruction formula can be seen below in Equation 2.

2
0
1
) , (
1
) (
s
d ds
s
t
h
s
s X t x
C

|
.
|

\
|
=




Equation 2. Wavelet Reconstruction Formula

Where h() is called the mother wavelet.

Wavelet based lossy compression techniques have three steps in general [1];
(i)Transform: data are first transformed into wavelet domain. This step is invertible.
(ii)Quantization: the wavelet coefficients are quantized to a finite alphabet. This step is not
invertible, thus introduces the so called quantization noise.
(iii)Entropy Coding: the resulting symbols after quantization are further entropy coded to
reduce the bit rate. This step is also invertible.

For a Wavelet Transform a simple diagram is as shown below in Figure 4.


15

Figure 4. Wavelet Transform Diagram[15]

Now we are going to explain two wavelet[8] transforms called;

a) Continuous Wavelet Transform

b) Discrete Wavelet Transform

2.2.1 Continuous Wavelet Transform

The Continuous Wavelet Transform can be found from the below equation, where x(t)
is the signal to be analyzed where (t) is the mother wavelet. All the wavelet functions used
in the transformation are derived from the mother wavelet through shifting and scaling.
Continuous wavelet transform formula is as shown in Equation 3.

dt
s
t
t x
s
s X
WT


= ) ( . ) (
1
) , (
*


Equation 3. Continuous Wavelet Transform Formula

Simply we can show as a simple version as below in Equation 4,


= dt t position scale t f position scale C ) , , ( ) ( ) , (

Equation 4. Simple Representation of the formula of CWT

The mother wavelet used to generate all the basis functions is designed based on some
desired characteristics associated with that function. The translation parameter relates to the
location of the wavelet function as it is shifted through the signal. Thus, it corresponds to the
time information in the Wavelet Transform. The scale parameter s is defined as | 1/frequency
| that corresponds to frequency information. Scaling either expands or compresses a signal.
Large scales, low frequencies, expands the signal and provide detailed information hidden in
the signal, while small scales, high frequencies, compress the signal and provide global

16
information about the signal. The above analysis becomes very useful as in most practical
applications, high frequencies, low scales do not last for a long duration while low
frequencies, high scales usually last for entire duration of the signal.



2.2.2 Discrete Wavelet Transform

In the project we used discrete wavelet transform while compiling our matlab
program. [4]. In CWT, the signals are analyzed using a set of basis functions which relate to
each other by simple scaling and translation. In the case of DWT, a time-scale representation
of the digital signal is obtained using digital filtering techniques.
The signal to be analyzed is passed through filters with different cutoff frequencies at
different scales. Discrete wavelet transform is defined by a square matrix of filter
coefficients, transforming an array into a new array, usually of the same length. If correctly
constructed, the matrix is orthogonal, and in this case not only the transform but also its
inverse can be easily implemented. The wavelet transform resembles the Fourier transform in
many respects. Also its identical to a hierarchical sub-band system, where the sub-bands are
logarithmically spaced in frequency and represent octave-band decomposition. To begin the
decomposition, the image is divided into four sub-bands and critically sub-sampled as shown
below in Figure 5.

Figure 5. First stage of a discrete wavelet transform

Each coefficient represents a spatial area corresponding to approximately a (2 2) area of the
original image. The low frequencies represent a bandwidth approximately corresponding to
w, which can take values between 0 and /2, whereas the high frequencies represent the band
from /2 to . The four sub-bands arise from separable application of vertical and horizontal
filters. The sub-bands labeled LH1, HL1, and HH1 represent the finest scale wavelet

17
coefficients. To obtain the next coarser scale of wavelet coefficients, the sub-band LL1 is
further decomposed and critically sampled as shown below in Figure 6. with the output we
get from the matlab program by using the dwt2 function.



Figure 6. A two scale wavelet decomposition


Figure 7. Two scale wavelet decomposition (matlab simulation)





18

Figure 8. Three scale wavelet decomposition


Figure 9. Three scale wavelet decomposition (matlab simulation)


In the project we used 3 scale wavelet decomposition and 8 x 8 analysis of the matrixes. After
finishing the encoding in the very last step after the decoding operations we used wavelet
reconstruction or synthesis using the idwt2 function to get the original picture as below in
Figure 10.



19

Figure 10. Original Picture after reconstruction

2.2.3 DWT and Filter Banks

Filters are one of the most widely used signal processing functions. Wavelets can be
realized by iteration of filters with rescaling. The resolution of the signal, which is a measure
of the amount of detail that are found in the signal and the scale is determined by upsampling
and downsampling operations.
The DWT is computed by successive lowpass and highpass filtering of the discrete time-
domain signal as shown in the figure below in Figure 11.. This is called the Mallat algorithm
or Mallat-tree decomposition. In the figure, the signal is denoted by the sequence x[n], where
n is an integer. The low pass filter is denoted by G
0
and the high pass filter is denoted by H
0
.
At each level, the high pass filter produces detail information, d[n], while the low pass filter
associated with scaling function produces coarse approximations, a[n].



Figure 11. Successive lowpass and highpass filtering[16]


20
At each decomposition level, the half band filters produce signals spanning only half
the frequency band. This doubles the frequency resolution as the uncertainity in frequency is
reduced by half. In accordance with Nyquists rule if the original signal has a highest
frequency of , which requires a sampling frequency of 2 radians, then it now has a highest
frequency of /2 radians. It can now be sampled at a frequency of radians thus discarding
half the samples with no loss of information. This decimation by 2 halves the time resolution
as the entire signal is now represented by only half the number of samples. Thus, while the
half band low pass filtering removes half of the frequencies and thus halves the resolution,
the decimation by 2 doubles the scale. The filtering and decimation process is continued until
the desired level is reached. The maximum number of levels depends on the length of the
signal. The DWT of the original signal is then obtained by linking all the coefficients, a[n]
and d[n], starting from the last level of decomposition.

Figure 12. Reconstruction of the original signal from Wavelet Coefficients[16]

The above Figure 12. shows the reconstruction of the original signal from the wavelet
coefficients. Basically, the reconstruction is the reverse process of decomposition. The
approximation and detail coefficients at every level are upsampled by two, passed through the
low pass and high pass synthesis filters and then added. This process is continued through the
same number of levels as in the decomposition process to obtain and Hthe original signal.
The Mallat algorithm works equally well if the analysis filters, G
0
and H
0
, are exchanged with
the synthesis filters, G
1 1
.

2.3 Zerotrees of Wavelet Coefficients

2.3.1 Significance Map Encoding
The EZW algorithm recognized that a significant fraction of the total bits needed to
code an image were required to code position information, which EZW called significance

21
maps. A significance map is a binary function whose value determines whether each
coefficient is significant or not significant. If a coefficient is not significant, it is assumed to
quantize to zero. Hence, a decoder knows the significance map needs no further information
about that coefficient. Otherwise, the coefficient is quantized to a non-zero value. The EZW
algorithm determined a very efficient way to code significance maps by coding the location
of the zeros. It was proven experimentally that zeros could be predicted very precisely across
different scales in the wavelet transform. The EZW algorithm is based on the hypothesis that
if a wavelet coefficient at a coarse scale is insignificant with respect to a given threshold T,
then all wavelet coefficients of the same orientation in the same spatial location at finer
scales are likely to be insignificant with respect to given threshold.
For each threshold T, the positions of the significant, and insignificant coefficients,
are indicated in significance maps. The values of "0" indicate the positions of the
insignificant coefficients, whereas the values of "1" are indicating the positions of the
significant coefficients. A sample for both quantized coefficients and a significance map is
shown in figure. A modality to encode the significance maps is the Zerotree Representation.
It allows us to predict insignificant coefficients across the scales efficiently. By using this
technique, the cost of encoding the significance maps is reduced by grouping the insignificant
coefficients in trees growing exponentially across the scales, and by coding these coefficients
with zerotree symbols.



Quantized Coefficients Significance Map


Figure 13. Quantized Coefficients and Significance Map

The coding of the significance map which means the positions of those coefficients
that is going to be transmitted as nonzero values is one of the important views of low bit rate
64 56 48 32 24 16 0 0
56 48 40 24 16 23 0 0
40 40 30 24 16 8 0 8
32 32 32 24 24 16 0 0
24 24 16 8 0 0 8 0
16 16 8 0 0 8 0 0
0 0 0 8 0 0 0 0
0 0 0 0 0 0 0 0
1 1 1 1 1 1 0 0
1 1 1 1 1 1 0 0
1 1 1 1 1 1 0 1
1 1 1 1 1 1 0 0
1 1 1 1 0 0 1 0
1 1 1 0 0 1 0 0
0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0

22
image coding. After quantization followed by entropy coding, the zero symbol which occurs
with the highest probability, should be extremely high in order to attain very low bit rates.
Thus, a large fraction of the bit budget is spent to encode the significance map. This results in
an important progress in encoding the significance map, and a higher efficiency in
compression.
To understand the importance of significance map encoding, a typical transform
coding system may be considered. This consideration is required to provide a person with a
sense of the relative coding costs of the position information which is contained in the
significance map relative to sign and amplitude information. A typical low bit-rate image
coder has three basic components: a transformation, a quantization and data compression. A
typical low bit rate image coder can be seen in Figure 14.

Figure 14. Low bit rate image coder

Transformation is required to produce transform coefficients, and to produce
coefficients that are decorrelated. Most of the time, it is considered to be lossless.
Quantization produces stream of symbols by quantizing the transform coefficients. Each of
these stream of symbols corresponds to an index of a particular quantization bin. Finally, the
aim of the data compression stage is to losslessly represent the data stream by taking the
stream of symbols. The aim of quantization is to make the entropy of the resulting
distribution of bin indexes small enough so that the symbols can be entropy coded at some
target low bit rate. For each nonzero index encoded, the entropy code will need at least one
bit for the sign. An entropy code can be designed based on modeling probabilities of bin
indices. Thus, the entropy of the symbols H is expressed as:

[ ]
HNZ
p p p p p H + + = 1 ) 1 ( ) 1 ( ) 1 (
log log
2 2

Equation 5. Entropy of the symbols H

In this equation, p is the probability that a transform coefficient is quantized to zero,
whereas HNZ represents the conditional entropy of the absolute values of the quantized
coefficients conditioned on them being zero. Besides, the first two terms of this sum

23
represents the first order binary entropy of the significance map. Therefore, the true cost of
encoding the actual bit stream becomes:
Values Nonzero of Cost Map ce Significan of Cost Cost Total + =
Equation 6. Cost of encoding the bit stream

Hence, the conclusion is that the cost of determining the positions of the several
significant coefficients represents an important portion of the bit budget at low rates. As the
rate decreases, the cost of determining the positions has a probability to become an increasing
fraction of the total cost.

2.3.2 Compression of Significance Maps Using Zerotrees of Wavelet Coefficients
For improvement of the compression of significance maps of wavelet coefficients, a
data structure called zerotree[13] is defined. A zerotree is a quad-tree of which all nodes are
equal to or smaller than the root. The tree is coded with a single symbol. Then, it is
reconstructed by the decoder as a quadtree filled with zeroes. The zerotree is based on a
hypothesis which says if a wavelet coefficient at a coarse scale is insignificant with respect
to a threshold T, then all wavelet coefficients of the same orientation in the same spatial
location at the finer scale are likely to be insignificant with respect to the same threshold.
An Embedded Zerotree Wavelet (EZW) uses a series of decreasing thresholds and
compares the wavelet coefficients with those thresholds. If the magnitude of a coefficient is
smaller than a given threshold T, the node is called insignificant or 0 with respect to given
threshold. Otherwise, the coefficient is significant or 1. The EZW uses four symbols to
represent a tree node: POS (positive significant), NEG (negative significant), IZ (isolated
zero node) and ZTR (zerotree) instead of 0 and 1. Embedded zerotree wavelet coding can be
seen below which is done in 5 steps.

Initialization:

))) (max( 2 (log
0
2
coeffs floor
T =
k=0
Dominant List = All Coefficients
Subordinate List = []

SignificantMap
for each coefficient x in the Dominant List
if | x |
k
T
if x > 0
set symbol POS

24
else
set symbol NEG
else if x is non-root part of a zerotree
set symbol ZTD(ZeroTree Descendant)
if x is zerotree root
set symbol ZTR
otherwise
set symbol IZ

Dominantpass
If symbol(x) is POS or NEG(it is significant)
Put symbol(x) on the Subordinate List
Remove x from the Dominant List

SubordinatePass
For each entry symbol(x) in Subordinate List
if value(x) is the element of Bottom Half of [
k
T ,
k
T 2 ]
output 0
else
output 1

Update
=
+1 k
T
2
k
T

k=k + 1

Go to Significance Map

Furthermore, in a hierarchical subband system (except the highest frequency
subbands), every coefficient at a specific scale can be related to a set of coefficients at the
next finer scale of similar orientation. The coefficient at the coarse scale is called the parent,
and all coefficients corresponding to the same spatial location at the next finer scale of
similar orientation are called children. Similarly, we can define the concepts as descendants
and ancestors. For a given parent, the set of all coefficients at all finer scales of similar
orientation corresponding to the same location are called descendants. Similarly, for a given
child, the set of coefficients at all coarser scales of similar orientation corresponding to the
same location are called ancestors. It is known that, every coefficient is a parent for the
previous related coefficients and the last coefficient in the lowest resolution is the parent of
the whole image. If a parent is insignificant in a low-resolution image (when compared with
threshold), than we can assume that all its children are also insignificant (when compared
with threshold).When an insignificant parent node is detected, most probably the descendents
are also insignificant. Thus, most of the time, a zerotree significance map symbol is
generated. However, because p which is the probability of this event, is close to 1, its
information content which is p log p, is very small. So most of the time, a very small

25
amount of information is transmitted, and this keeps the average bitrate required for the
significance map small.
In the wavelet decomposition, coefficients that are spatially related across scale can
be compactly described using tree structures. With the exception of the low resolution
approximation (LLK) and the highest frequency bands (HL1, LH1, and HH1 ), each
coefficient (parent) at level j of the decomposition correlates to 4 coefficients (children) at
level j 1 of the decomposition which are at the same frequency orientation. For the LLK
band, each parent coefficient correlates with 3 children coefficients, one in the HLK, one in
the LH K , and one in the HH K bands. For example, level HH3 includes the children of HH2.
Also, level HH2 in turn is the parent to the sub band level HH1. In scanning order of the sub-
bands for encoding a significance map, parents must be scanned before the children and also
all positions in a given sub-band are scanned before the scan moves to the next sub-band. A
sample of the scanning order can be seen below in figure, the scanning order is in [6] Zigzag
manner. In our project, we used Morton scan order. Morton scan order is shown in below.


Figure 15. Zig-zag Manner Scanning Order [1]


Figure 22. Morton Scan Order [3]

Given a threshold T to determine whether or not a coefficient is significant, a
coefficient is part of a zero-tree if it is zero or insignificant with respect to the threshold T.

26
Also, it is an element of zerotree, if all of its descendants are zero or insignificant with
respect to the threshold T. If a coefficient is not a part of another zero-tree starting at a
coarser scale, then it is also a zero-tree root. Hence, for a given threshold, any wavelet
coefficient could be represented in one of the four data types. As mentioned before, these are
POS, NEG, ZTR, and IZ, corresponding to positive coefficients, negative coefficients,
zerotree roots,
and isolated zeros, respectively. Zerotree coding is lossless and zerotree symbols are assigned
to wavelet coefficients as the coefficient file is scanned. If a coefficient is found to be zero or
insignificant, then corresponding coefficients in lower resolutions are checked. If all of these
children are zero, this coefficient is coded as a (zerotree symbol) ZTR. Besides, if one or
more of the children of an insignificant node is significant, that means a symbol for isolated
zero (IZ) is transmitted. The probability of this event is lower most of the times but it is
necessary to be able to avoid losing significant information down the tree.
In this manner the significance map can be efficient represented as a string of symbols
from a 4-symbol alphabet (POS, NEG, ZTR, and IZ) which is then entropy coded. A sample
coefficient decision procedure and the encoding process of a coefficient of the significance
map can be seen below in Figure 17.




Figure 17. Flow chart for encoding a coefficient of the significant map

27


If a significant coefficient is detected or found out, the coefficient is moved to the end
of the list of significant coefficients, and the node value is zeroed out in the wavelet tree.
Also, the zerotree symbols can be transmitted many times for a given coefficient, until it rises
above the sinking threshold that it will be tagged as a significant coefficient. After this point,
zerotree information is not going to be transmitted for this coefficient anymore. These trees
of zero coefficients can be coded by only coding the root coefficient of the tree. This provides
the compression that none of the coefficients below the root need to be coded. In general,
zerotree coders combine the idea of cross-band correlation with the notion of coding zeros
jointly, for generation of very powerful compression algorithms. Aside from the nice energy
compaction properties of the wavelet transform, this coder attains its compression ratios by
joint coding of zeros.


2.4 Successive Approximation Entropy-Coded Quantization

The successive approximation quantization gives an important characteristic to those
algorithms: they are all embedded, that is, the bit stream they generate for one rate contains
the bit streams for all achievable lower rater. Shapiro characterizes an embedded code by
defining two properties:

a) When coding the same data with different rates, the two resulting
representations must match exactly for the extent of the smaller one. In other
words, the smaller one must be reproduced exactly in the beginning of the
larger one. This way, the coded representation for a given data rate contains all
the representations for smaller data rates. Shortly, the representations get more
precise as we add more symbols to them.
b) It should obtain a good representation for a given data rate.

To perform the embedded coding in the program we applied successive-approximation
quantization (SAQ). As will be seen, SAQ is related to bit-plane encoding of the magnitudes.
The SAQ sequentially applies a sequence of thresholds to determine significance, where the
thresholds are chosen so that T(i) = T (i-1)/2. The initial threshold T(0) is chosen so that |X(j)|
< 2T(0) for all transform coefficients Xj. In the program, during the encoding and decoding,
two separate lists of wavelet coefficients are present. Dominant list contains the coordinates
of the coefficients that have not yet been found to be significant in the same order as the

28
initial scan. This scan is such that the sub-bands are ordered, and with each sub-band, the set
of coefficients are ordered. The subordinate list contains the magnitudes of those coefficients
that have been found to be significant and for each threshold, each list is scanned once.
During a dominant pass, coefficients with coordinates are compared to the threshold Ti
to determine their significance, and if significant, their sign. This significance map then later
zerotree coded. Each time a coefficient is encoded as significant as positive or negative then
its magnitude is appended to the subordinate list, and the coefficient in the wavelet transform
array is set to zero so that the significant coefficient does not prevent the occurrence of a
zerotree on future dominant passes at smaller thresholds.
Then a dominant pass is continued with subordinate pass is performed on the
subordinate list, which contains all pixel values previously found to be significant, in which
all coefficients on the subordinate list are scanned and the specifications of the magnitudes
available to the decoder are refined to an additional bit of precision. For each magnitude on
the subordinate list, this refinement can be encoded using a binary alphabet with a 1
symbol indicating that the true value falls in the upper half of the old uncertainty interval and
a 0 symbol indicating the lower half. The string of symbols from this binary alphabet that is
generated during a subordinate pass is then entropy coded. After the completion of a
subordinate pass the magnitudes on the subordinate list are sorted in decreasing magnitude.
The process continues to alternate between dominant passes and subordinate passes where the
threshold is halved before each dominant pass.
Thus, the algorithm for the successive quantization for embedded zero tree encoding
procedure can be seen below :

1. Choose a high quantisation step T (half the max value)
2. Until finest quantisation step, repeat
2.1 Quantise wavelet coefficients using T
ie. if (w(x,y)<T) w'(x,y)=0 ; else w'(x,y)=1;
Creates a binary image of the most significant wavelet coefficients
2.2 Encode binary image (w') using zero-trees and output
(This is the highest order bits of the remaining data)
2.3 Subtract the encoded data from the remaining data
w(x,y)=w(x,y) w'(x,y)*T
2.4 half the quantisation step size
T=T/2
3. End

29


In the decoding operation, by reading the subordinate symbol corresponding to a
significant pixel and knowing the threshold, the decoder is able to determine the range in
which the pixel lies and reconstructs the pixel value to the midpoint of that range, each
decoded symbol which in both during a dominant and a subordinate pass, refines and reduces
the width of the uncertainty interval in which the true value of the coefficient may occur. The
reconstruction value used can be anywhere in that uncertainty interval. The encoding stops
when some target stopping condition is met, such as when the bit budget is exhausted. The
encoding can cease at any time and the resulting bit stream contains all lower rate encodings.
Terminating the decoding of an embedded bit stream at a specific point in the bit stream
produces exactly the same image that would have resulted had that point been the initial
target rate. This ability to cease encoding or decoding anywhere is extremely useful in
systems that are either rate or distortion constrained.


2.5 An Example About EZW
A 3-scale wavelet transform of an 8 x 8 image is given below [1]. We are going to discuss
about only LL2 section for simplification.

63 -34 49 10 7 13 -12 7
-31 23 14 -13 3 4 6 -1
15 14 3 -12 5 -7 3 9
-9 -7 -14 8 4 -2 3 2
-5 9 -1 47 4 6 -2 2
3 0 -3 2 3 -2 0 4
2 -3 6 -4 3 6 3 6
5 11 5 6 0 3 -4 4

Figure 18. Example of 3-scale Wavelet Transform of an 8 x 8 Image


We will choose our initial threshold as T
o
=32, The table below, Table 1. ,shows the
processing on the first dominant pass.





30

Subband Coefficient
Value
Symbol Reconstruction
Value
LL3 63 POS 48
HL3 -34 NEG -48
LH3 -31 IZ 0
HH3 23 ZTR 0

Table 1. First Dominant Pass at Threshold T = 32


The top left corner coefficient 63 is the biggest value among all coefficients and represents
the DC coefficient. According to the largest coefficient magnitude, the first dominant pass
threshold T can be any number between 31.5 and 63. We Choose T0 = 32. The lowest
frequency coefficient with magnitude 63 is greater than the threshold and meaning a positive
symbol. At the decoder end, this coefficient will be decoded to the center value of the
uncertainty interval [32, 64), which is 48. Next, according to the scan order, the coefficient -
34 is to be processed absolute value is taking in order to process. Its magnitude is also greater
than 32 and then putting a negative symbol behind. It will be decoded to -48 in the
reconstruction of the image. The coefficient of LH3 is -31, which is insignificant with respect
to the threshold 32. But it has a descendant of magnitude 47 in LH1. Therefore, -31 is an
isolated zero. So far, the 1st dominant pass has completed the process on the finest scale and
is going to move to the 2nd scale. At HL2 subband, except 49 is a positive significance, all
other three values are zerotree root. At LH2 subband, all coefficients are insignificant.
However, 14 has a significant child. Therefore, 14 is an isolated zero, like -31, and all other
three are zerotree roots. Coefficients in HH2 subband are all zerotree roots since themselves
and their descendants are all insignificant. Then we move to the finest scale. Both HL1 and
HH1 have all zeros. At LH1 subband, except 47 is a positive significant, all others are zeros.
This completes the 1st dominant pass of the sample 88 image. After the 1st dominant pass,
the 1st subordinate pass is applied to the significance map. The subordinate pass only looks at
the nonzero values and refine them. The uncertainty intervals are refined as [32, 48) and [48,
64). Any significant coefficients greater than 48 will be encoded as symbol 1, and as
symbol 0 if lying between 32 13 and 48. Each symbol 1 will later be decoded as 56,
which is the center of the interval [48, 64), while symbol 0 will later be decoded as 40, the
center of the interva l [32, 48). During the 2nd dominant pass, the threshold value is set to be
16 since all the coefficients to be processed are those insignificant ones after the 1st dominant

31
pass, which are between [0, 32). The procedure is the same as the 1st dominant pass and
results are shown in below. The 2nd dominant pass processes all the significant coefficients
after the 2nd dominant pass, including those from the 1st dominant pass. The uncertainty
intervals are refined as [16, 24), [24, 32), [32, 40), [40, 48), [48, 56) and [56, 64). In other
words, uncertainty intervals starts from the threshold value, and has interval length 16/2 = 8.
Results can be seen on following page in Table 2.


Table 2. Results of the example

Subband
Coefficient
Value
1
st
Dominant
Pass
(T=32)
1
st
Subordinate
Pass Interval
center (56 40)
2
nd

Dominant
Pass
(T=16)

2
nd
Subordinate Pass
(60 52 44 36 28 20)
LL3 63 POS 48 1 56 0 60
HL3 -34 NEG -48 0 40 0 36
LH3 -31 IZ 0 0 NEG -24 28
HH3 23 ZTR 0 0 POS 24 20
HL2 49 POS 48 1 56 0 52
HL2 10 ZTR 0 0 ZTR 0 0
HL2 14 ZTR 0 0 ZTR 0 0
HL2 -13 ZTR 0 0 ZTR 0 0
LH2 15 ZTR 0 0 ZTR 0 0
LH2 14 IZ 0 0 ZTR 0 0
LH2 -9 ZTR 0 0 ZTR 0 0
LH2 -7 ZTR 0 0 ZTR 0 0
HL1 7 Z 0 0 ZTR 0 0
HL1 13 Z 0 0 ZTR 0 0
HL1 3 Z 0 0 ZTR 0 0
HL1 4 Z 0 0 ZTR 0 0
HL1 -1 Z 0 0 ZTR 0 0
HL1 47 POS 48 0 40 0 44
HL1 -3 Z 0 0 ZTR 0 0
HL1 -2 Z 0 0 ZTR 0 0

32
2. Experimental Results
We applied the embedded zero-tree wavelet algorithm (EZW) on the bird image by adjusting
the Threshold and Last step level values. Below you will find results for the reconstructed
images at different compression rates. It is clear that as the number of significant coefficients
kept increases the compression decreases and hence the PSNR improves.

Original image

reconstructed image

Threshold to stop at : 8
Last Step Level : 4
Sum : 4710
PSNR : + 28.82 dB
Compression : 13.91 : 1

Original image

reconstructed image

Threshold to stop at : 4
Last Step Level : 5
Sum : 8065
PSNR : +32.2597 dB
Compression : 8.13 : 1

33
Original image

reconstructed image

Threshold to stop at : 1
Last Step Level : 7
Sum : 27017
PSNR : +39.5650 dB
Compression : 2.43:1


Original image

reconstructed image

Threshold to stop at : 1
Last Step Level : 8
Sum : 42274
PSNR : +42.9387dB
Compression : 1.55 : 1
Table .3 Experimental results for bird.gif





34
4. Conclusion
This project is about Embedded Zerotree Wavelet (EZW) algorithm which is
considerably effective and important compression algorithm. This algorithm is devised by
Shapiro[1] in 1993, and it is an image compression algorithm that the bits in the bit stream
are generated in order of importance. The algorithm is based on the selection of significant
coefficients with respect to given thresholds. In this project, we tried to apply the embedded
zero tree wavelet compression on an image. In our program, we used an encoder to encode
the image by using three level decomposition, with discrete wavelet transform, and then
quantized the image. We checked the importance of coefficients by comparing with the
threshold which is done in the beginning of each dominant pass and taking the values which
are greater than the threshold. Then we used the decoding algorithms, and we used inverse
discrete wavelet transform to get the reconstructed image.
In our project, we tried our best to understand and improve EZW algorithm to be able
to apply it for various images. We wrote this report according to our matlab program. The
Matlab program that we have is the improved version of the matlab program which was
compiled by Xiangang Li. The main purpose of the program was to encode and decode the
only one 8 x 8 image matrix. After we had analyzed the program, we improved the program for
various of scaled images and added discrete wavelet transform algorithm (DWT) and inverse discrete
wavelet transform (IDWT) algorithm, to make the program as told in Shapiros [1]. We added
Discrete Wavelet transform and Inverse Discrete Wavelet Transform to the matlab program for
decomposition. Also, we adjusted the old program so that now, we are able to use this
algorithm for 256 x 256 images or different sized images. The advantages of this program are
that, it yields a fully embedded bit stream, providing competitive compression performance,
more compression in the first steps, and we had a precise rate controlling by changing the
threshold and last step values. Also the picture is getting more clear as we continue changing
the steps of the program.
While compiling the Matlab code, first we came across with some difficulties in the
analysis of the program. Then, we had some extra difficulties while we were trying to
develop our matlab program. However, these difficulties made us do more research and force
ourselves to do our best. Also, this project made us to learn and search the required matlab
codes needed for the program and helped us to improve our research abilities in the specified
manner. This project has taught us many valuable things, and we have gained significant
experience about EZW. It also showed us the importance of working in groups for future
projects since we will all be forced to work in groups to achieve the projects, and it showed
us how people can work together more efficiently and effectively for a project. Furthermore,

35
we learned the importance of timing and systematical work, which helps making the project
to yield in a successful way.
Finally, we want to say that this project is the result of work done by two Electrical and
Electronics Engineering students, because we worked and studied very hard to accomplish
our task by doing this project. To sum up, we have learned the importance of EZW and its
application of encoding, decoding and the compression of the image that we have done in this
project. EZW is one of the best compression algorithms and can be used in various fields
which can form one of the basis of image compression algorithms in the future.
5. Future Work
EZW is a remarkably effective compression algorithm. Therefore this algorithm can
be beneficial for many areas. We have chosen to apply this algorithm to different sized
images to encode, compress, decode, and reconstruct them without using extra codings such
as Arithmetic coding or Huffman coding. However, in future this project can be improved by
adding Arithmetic coding or Huffman coding to attain better compression results to achieve
high compression ratios. In addition to this, instead of DWT, different transforms can be used
for this project to obtain better results or to see the differences between these applications.
Also, an additional algorithm can be added to our matlab code to send an image through
noisy channel and receive images without noise. The program can be changed or written
again so that if a noisy image comes to the decoder, decoder will be able to do denoising and
reconstruct the image without having noise on it.
Furthermore, this project can be improved that it can be applied for video
compression using EZW. Because a video consists of various images taken sequentially, this
algorithm can be used to compress each frame or image of a video. Hence, the video
compression can be accomplished. Additionally, by using this project people can change this
algorithm to apply it to 3D images. In future, compression speed can also be increased. In this
situation, people can send their images faster via channels. Sending and receiving images
faster is very important according to different areas. For example, EZW can be used in
medicine or medical applications to send the images of the patients to other doctors quickly.
Thus, they can find quick solutions for the benefit of their patients. Finally, EZW algorithm
can be used in cell phones to send large sized images or even videos via cell phone by
compressing them.





36




References

[1] Shapiro, J. M. R. B., 1993, Embedded Image Coding Using Zerotrees of Wavelet
Coefficients, IEEE Transactions on Signal Processing, vol. 41, no. 12

[2] Nasir, M. R., Rolan, G. W., Franois, G. M., Ronald R. C., December 2003,
Adaptive Wavelet Packet Basis Selection for Zerotree Image Coding, IEEE
Transactions on Signal Processing, vol. 12, no. 12

[3] C, Valens. (1999-2004). EZW encoding. Retrieved October 2, 2004 from the World
Wide Web: http://perso.wanadoo.fr/polyvalens/clemens/ezw/ezw.html

[4] Fowler, J. E. (5/2003). Embedded Wavelet-Based Image Compression: State of the
Art. Retrieved October 2, 2004 from the World Wide Web: http://www.extenza-
eps.com/extenza/loadPDFInit?objectIDvalue:22708

[5] Averbuck, A. Wavelet Compression. Retrieved December 12, 2004 from the World
Wide Web: www.cs.tau.ac.il/~amir1/COMPRESSION/
PDF_SLIDES/wavelet_compression.pdf

[6] Xu, X. Embedded Zero Tree as Image Coding. Retrieved December 12, 2004 from
the World Wide Web: http://www.uoguelph.ca/~xux/courses/ENGG6560.pdf

[7] Kumar, S.(2001-2003). Entropy Coding. Retrieved October 2, 2004 from the
World Wide Web:
http://www.google.com.tr/search?q=cache:5Q6N4lkKlaUJ:www.debugmode.com/im
agecmp/entrcode.htm

[8] Hong, D., Barrett, M., Xiao, P. Wavelets and Bi-orthogonal Wavelets for Image
Compression: Retrieved June 16, 2005 from the World Wide Web:
http://www.etsu.edu/math/hong/ps/HongBXiao.pdf


[9] Valens, C. (1999). Embedded Zerotree Wavelet Encoding: Retrieved June 16, 2005
from the World Wide Web: http://www.mindless.com

[10] Jin, L. Jay, K. (August 15, 1996). Embedded Wavelet Coder with Multistage
Vector Quantization: Retrieved June 16, 2005 from the World Wide Web:
http://viola.usc.edu/newextra/Publication/ PDF/selected/1997_JIST_Liang.pdf

[11] Geoffrey M. D., A Wavelet Based Analysis of Fractal Image Compression, To
Appear in IEEE Transactions on Image Processing, 1997.


37
[12] Geoffrey M. D., Wavelet-based Image Coding an Overview: Retrieved June 16,
2005 from the World Wide Web: http:// www.geoffdavis.net/papers/accsc.pdf

[13] Sun, H., Zhang, Y. (December 12, 2003). Embedded Zerotree Wavelet Image Codec:
Retrieved June 16, 2005 from the World Wide Web:
http://www.cae.wisc.edu/~ece533/project/f03/sunzhang.pdf

[14] Torrence, C., Compo, G. A practical Guide to Wavelet Analysis: Retrieved June 16,
2005 from the World Wide Web: http://paos.colorado.edu/research/wavelets/

[15] Nachtergaele, L., Vanhoof, B., Peon, M., Lafruit, G., Bormans, J., Bolsens, I.
Implementation of a scalable MPEG-4 wavelet-based visual texture compression
system. : Retrieved June 16, 2005 from the World Wide Web:
http://www.hwswworld.com/downloads/y13/19_3.PDF

[16] Sripathi, D., Efficient Implementations of Discrete Wavelet Transforms Using
FPGAs: Retrieved October 18, 2003 from the World Wide Web:
http://etd.lib.fsu.edu/theses/available/etd-11242003-185039/

You might also like