I

I
m
m
a
a
g
g
e
e
C
C
o
o
d
d
i
i
n
n
g
g
u
u
s
s
i
i
n
n
g
g
E
E
Z
Z
W
W

a
a
n
n
d
d
Q
Q
M
M
c
c
o
o
d
d
e
e
r
r



E EC CE E 5 53 33 3 P Pr ro oj je ec ct t R Re ep po or rt t




A Ah hu uj ja a, , A Al lo ok k
+ +

S Si in ng gh h, , A Aa ar rt ti i
+ +






+
(50% contribution by each member)

A Ab bs st tr ra ac ct t
This project involves Matlab implementation of the Embedded Zerotree Wavelet (EZW)
coding algorithm followed by a QM coder. The Embedded Zerotree Wavelet (EZW)
algorithm is a wavelet-based image compression algorithm that yields a fully embedded code
as well as remarkable compression efficiency. An embedded code has the property that bits
in the bit stream are generated in order of importance, hence the encoding can be truncated at
any point to meet a specified bit rate. The output of the algorithm is a stream of symbols that
can be further compressed using an adaptive binary arithmetic coder like the QM coder.




O Ov ve er rv vi ie ew w
This report starts with a brief statement of the work performed the objectives, material
studied and implemented. The main part of the report is divided into two broad sections
covering the concepts behind EZW algorithm and the QM coder. Specifically we have tried
to present issues that we found intricate, in a simpler manner and also tried to emphasize
issues that are not clearly stated in the references. The next section contains the results
followed by conclusion and discussion.








S St ta at te em me en nt t o of f W Wo or rk k
This project is somewhere between an implementation and study project. The project was
started with an aim to understand and implement the EZW coding algorithm proposed by J.
M. Shapiro (1). We tried to understand the scope of improvement in the EZW algorithm (2)
and better ways to implement it eg. using SPIHT (3).
Also we tried to focus on investigating adaptive arithmetic coders, specifically Q and QM
coders that implemented finite precision representation for the interval size as well as
provided computational simplicity. Starting with papers on adaptive binary arithmetic coder
(7) to Q coders and its adaptive probability estimation (5), (6), we finally arrived at the QM
coder (4), which is used in the JPEG standard for image compression. Due to the many
improved features of the QM coder (as discussed later), we decided to implement it in
Matlab. Results of a test sequence provided in (4) were reproduced.

Finally, the QM-coder was interfaced and used alongwith the EZW coder.






E EZ ZW W b ba as se ed d I Im ma ag ge e C Co od di in ng g
The Embedded Zerotree Wavelet (EZW) algorithm is based on four key concepts
(1) A discrete wavelet transform or hierarchical subband decomposition.
(2) Prediction of absence of significant information in finer level wavelet coefficients
based on coarser level coefficients.
(3) Entropy-coded successive-approximation quantization.
(4) Lossless data compression based on an adaptive arithmetic coder. (In this project, the
coder used is the QM coder and we devote a separate section to it.)

D Di is sc cr re et t e e W Wa av ve el le et t T Tr ra an ns sf f o or rm m is carried out by decomposing the image into four subbands
(LL, LH, HL and HH) using separable wavelet filters and critically subsampling the output.
The next coarser level of coefficients are obtained by decomposing the low frequency
subband LL.

Fig. A two- level subband decomposition (1)
The goal of the transform is to produce coefficients that are decorrelated. This results in
concentration of energy in a few coefficients while most are insignificant enough to be
discarded, thus offering opportunity for compression.

S Si ig gn ni if f i ic ca an nc ce e M Ma ap p E En nc co od di in ng g The coefficients are compared to a threshold to determine if
they are significant enough. The zerotree is based on the hypothesis that if a wavelet
coefficient is insignificant with respect to a threshold T, then all wavelet coefficients of the
same orientation in the same spatial location at finer scales are likely to be insignificant with
respect to T.
More specifically, in hierarchical subband decomposition, with the exception of the highest
frequency subbands, every coefficient at the given level (parent) can be related to a set of
coefficients (children)at next finer level of similar orientation and spatial location. For a
given parent, the set of coefficients at all finer scales of similar orientation corresponding to
the same location are called descendants.

Fig. Parent-child dependencies (1)
As can be seen each parent has three children except for the lowest frequency subband where
the relationship is defined so that each parent node (in LL
3
) has three children (one each in
HL
3
, LH
3
and HH
3
). Scanning of the coefficients is performed so that no child is scanned
before its parent.
Given a threshold level T to determine whether or not a coefficient is significant, a
coefficient is an element of a zerotree for threshold T if itself and all of its descendents are
insignificant wrt T. An element of a zerotree is a zerotree root if it is not the descendent of a
previously found zerotree root. Thus, a zerotree root indicates that the insignificance of the
coefficients at all finer levels is completely predictable.
Thus, the output of the EZW coder consists of the following symbols:
1. Zerotree root, Z.
2. Isolated zero, I. (means that coefficient is insignificant but has some significant
descendents).
3. Positive significant, P.
4. Negative significant, N.
Two more symbols L and H are generated which are discussed in the next section.

S Su uc cc ce es ss si iv ve e- -A Ap pp pr ro ox xi im ma at t i io on n E En nt t r ro op py y- -C Co od de ed d Q Qu ua an nt t i iz za at t i io on n is applied to achieve embedded
coding. The encoding can be stopped when a specified bit-rate is met. Successive
approximation quantization determines significance by starting with an initial threshold T
0
,
chosen so that all transform coefficients are < 2T
0
. Successive thresholds are obtained as T
i
=
T
i-1
/2. The encoding (and decoding) involves two main passes
1. Dominant pass During a dominant pass, coefficients that have not yet been found to
be significant are compared against the threshold, T
i
. The scanning order follows
treating parents before children, eg. raster scan, however those coefficients are
omitted which are descendents of a zerotree root found before. If the coefficient is
found significant, it is appended to the subordinate list, and is set to zero in the
wavelet transform array to prevent it from effecting future dominant passes at smaller
thresholds. Coefficients found insignificant are coded as either zerotree root or
isolated zero based on the previous discussion.
2. Subordinate pass follows the dominant pass and outputs an L or H which further
refines the specification of the magnitude of the coefficient put out to the decoder. i.e.
it designates whether the coefficient is greater than T
i
/2 (H) or less (L). Thus it
reduces the uncertainty interval of the magnitude of the coefficient (or equivalently
the quantizer step-size) by half. In the decoder, the reconstruction value used can be
anywhere in that uncertainty interval. The magnitudes on the subordinate list are then
sorted in decreasing order of magnitude.
The process continues to alternate between dominant and subordinate passes where the
threshold is halved T
i-1
=T
i
/2 before each dominant pass.











Q QM M c co od de er r
The QM coder is a binary arithmetic coder, implying that it codes a stream of only two
symbols 0 and 1. A source with mult iple symbols can also be coded by decomposing each
symbol using a binary decision tree.
Advantages of a binary arithmetic coder lie in that it generates compressed data as a finite-
precision fraction which identifies an interval on the number line. Each symbol is encoded or
decoded on the basis of a probability estimate which determines where the binary arithmetic
coder splits the interval into two subintervals. The current symbol determines which
subinterval becomes the new interval. When the size of the new interval drops below a
minimum value, renormalization shifts the precision until it is greater than or equal to the
minimum size. With each shift, a bit is produced for the compressed data stream.

I In np pu ut t b bi in na ar ry y a al lp ph ha ab be et t was generated using a tree of binary decisions. As noted previously
the output of the EZW code was a stream of symbols containing the symbols: P, N, Z,
I, H and L. The following binary decision tree was used based on the probability of
occurrence of these symbols.

L
0 1
H
0 1
0 1
Z 0 1
I 0 1
P N
Symbol stream
S Sy ym mb bo ol l o or rd de er ri in ng g a an nd d i in nt t e er rv va al l s su ub bd di iv vi is si io on n When the QM coder codes a binary decision (0
or 1), it does not directly assign intervals to these symbols. Rather, it assigns intervals to the
more probable symbol (MPS) and the less probable symbol (LPS) such that the LPS
subinterval is always above the MPS subinterval.

If the interval is A and the LPS probability estimate is Qe, the MPS probability estimate
should ideally be (1-Qe). The lengths of the respective subintervals are then A Qe and
( ) A Qe 1 . In the QM coder, code stream C points to the bottom of the current interval so
that we need to add to the code stream only when an LPS occurs.

C Co od di in ng g a a s sy ym mb bo ol l changes the interval and code stream as follows:
After MPS: ( ) Qe A A C = 1 unchanged is

After LPS: ( ) Qe A A Qe A C C = + = 1


R Re en no or rm ma al l i iz za at t i io on n is done by doubling (shifting left) A, and in accordance C, each time the
interval falls below some minimum value (0.75). This enables finite precision representation
C+A
C+A.(1-Q
e
)
C
A.Q
e

A.(1-Q
e
)
LPS
MPS
for A by confining A within the limits 0.75-1.5 (since 1.5 is double of 0.75) and also aids to
avoid the multiplication A.Qe as explained below.

E El li im mi in na at t i io on n o of f m mu ul lt t i ip pl li ic ca at t i io on ns s b by y a ap pp pr ro ox xi im ma at t i io on n We desire to have A 1, so that A.Qe
Qe. Keeping A bounded in the range 0.75 = A <1.5 achieves this.
After MPS:


( ) Qe A Qe A A Qe A A
C
= = 1
unchanged is


After LPS:


( ) C C A Qe C A A Qe C A Qe
A A Qe Qe
= + = + +
=
1


Important note: Since after every LPS occurrence A Qe < 0.5, according to the discussion
on renormalization, a renormalization occurs everytime an LPS occurs.

I In nt t e eg ge er r r re ep pr re es se en nt t a at t i io on n In the representation chosen for QM-coder, the upper bound for
interval A (1.5) is defined as X 10000 and lower bound (0.75) as X 8000. (The register for
A is 16 bit and is initialized as X 0000 (equivalent to X 10000). This makes multiplication
by shifts easier). Thus the correspondence between decimal representation at implementation
is given by (1/0.75)(X 8000).

C Co on nd di it t i io on na al l e ex xc ch ha an ng ge e When Qe is large, the size of the MPS subinterval may become less
than the size of the LPS subinterval i.e. A-Qe < Qe. To avoid the conflict between the relation
of probability and subinterval size for a symbol, the assignment of LPS and MPS to the two
intervals is interchanged. Since A-Qe < Qe < 0.5, both subintervals are less than 0.5 and
renormalization must occur. Thus conditional exchange is always done after a
renormalization.

A Ad da ap pt t i iv ve e P Pr ro ob ba ab bi il li it t y y E Es st t i im ma at t i io on n The estimation process is based on a form of approximate
counting in which the interval register normalization is used to estimate the MPS and LPS
symbol counts. Whenever a renormalization occurs i.e. after every LPS and after every MPS
that requires renormalization, a new probability estimate is obtained from a look-up table that
provides a bigger Qe value when LPS renormalization occurs and a smaller Qe value when a
MPS renormalization occurs. Thus, the distribution of probability estimates gets centered
about the desired value.


Markov-chain modeling of probability estimation The estimation process can be regarded
as a markov-chain where each state represents one probability estimate (and also contains the
sense of MPS) i.e. each state S can be thought of as a structure that contains a Qe(S) and a
MPS(S). An index Index(S) points to the current state. Index(S) is initialized to the first state
in the look- up table which corresponds to a Qe(S) ~ 0.5 (i.e. assuming initially that both
symbols are almost equally probable). The state machine has mirror symmetry about the
change in the sense of MPS. The look-up table used for the QM-coder estimation state
machine thus consists of the state index, associated Qe, NMPS (Next state index after MPS
renormalization), NLPS (Next state index after LPS) and Switch (if 1 indicates MPS sense
needs to be switched).





Fig. Markov-chain modeling of probability estimation


D Di if f f f e er re en nc ce es s b be et t w we ee en n t t h he e Q Q a an nd d Q QM M c co od de er r: :
1. Interval subdivision is improved in the QM coder by introducing conditional
exchange where the sense of MPS is reversed if the MPS subinterval becomes less
than the LPS subinterval. This leads to better compression.
2. QM coder allows carry to be resolved completely before transmitting next byte thus
preventing carry propagation problems.
3. The QM coder uses an initial (fast attack states) to arrive quickly at approximately the
right probability estimate and has more states in the probability estimation machine
thus is more responsive to unstable statistics.






Qe(S
j
)
MPS(S
j
)
Qe(S
k
)
MPS(S
k
)
Qe(S
i
)
MPS(S
i
)
Index(S)
MPS
renormalization
Qe(S
i
) < Qe(S
j
) < Qe(S
k
)
LPS
R Re es su ul lt ts s
The EZW algorithm was implemented and it can be seen how Wavelet transform
concentrates all the energy in few significant coefficients. This feature is exploited by the
zerotree coding to discard the insignificant coefficients and achieve good compression.
The QM coder was also tested using the sequence given in (4) and detailed results compared
with the one given. The QM coder works as a perfect lossless coder.
The following compression ratios were obtained using a combination of the EZW and QM
coder:
Image Bits needed for image
transmission
Size x 8
compressed stream
length (output of QM
coder)
Compression ratio

Fruits.png 512x512x8 = 2097152 129472 16.2
Lena.bmp 512x512x8 = 2097152 138600 15.1
Barbara.png 512x512x8 = 2097152 196789 10.6

Clearly, lesser are the details the more is the compression efficiency since fewer coefficients
are required to capture those details.
Reconstruction results on images however could not be obtained due to some sign
manipulation problem in the EZW decoder. However the magnitudes of the reconstructed
coefficients agreed pretty well with the original coefficients.



C Co on nc cl lu us si io on n
This project was a good learning experience about the power of wavelet coding-based
compression techniques for image compression. We were able to understand and reproduce
the results in references (1) and (4) by implementing EZW and QM coder in Matlab.

However, given the time, many concepts could not be explored and would form topic of
future work. Investigating the effect of different wavelet filters on compression and
reconstruction as well as the concept of embedded bit-coding to achieve the desired bit-rate
are some of these. Further there are many other coders like MQ etc. that were not looked at
and would be interesting to study and compare.


























R Re ef fe er re en nc ce es s
E EZ ZW W: :
1. Embedded Image Coding using Zerotrees of Wavelet Coefficients, J. M. Shapiro, IEEE
trans. on Sig. Proc., vol. 41, No. 12, Dec 1993, pp. 3445-3462.
2. http://perso.wanadoo.fr/polyvalens/clemens/ezw/ezw.html
3. A New, Fast, and Efficient Image Codec Based on Set Partitioning in Hierarchical
Trees, A. Said and W. A. Pearlman, IEEE Trans. Circuits Syst. Video Technol., vol. 6,
June 1996, pp. 243-249.
Q QM M- - c co od de er r: :
4. JPEG Still Image Data Compression Standard, W. B. Pennebaker & J. L. Mitchell.
5. An overview of the basic principles of the Q-coder adaptive binary arithmetic coder,
W. B. Pennebaker, et al., IBM J. Res. Develop., vol. 32, No. 6, Nov. 1988, pp. 717-725.
6. Probability estimation for the Q-coder, W. B. Pennebaker and J. L. Mitchell, IBM J.
Res. Develop., vol. 32, No. 6, Nov. 1988, pp. 737-751.
7. An introduction to arithmetic coding, Glen G. Langdon, IBM J. Res. Develop., vol. 28,
No. 2, March 1984, pp. 135-149.



S Su um mm ma ar ry y o of f C Co od de e
E EZ ZW W e en nc co od de er r : :
Files required: EZW.m
dominant_pass.m
subordinate_pass.m
process_LL_element.m
process_element.m
zerotree.m
decodeEZW.m
Q QM M c co od de er r : :
Files required: Arithmetic_coder.m
Code_MPS.m
Code_LPS.m
makestring.m
Renorm.m
Update_Index_S_after_LPS.m
Update_Index_S_after_MPS.m
byteout.m
Arithmetic_decoder.m
Cond_MPS_exchange_S.m
Cond_LPS_exchange_S.m
Renorm_d.m
byte_in.m