www.open.ac.

uk

SCALABLE
MULTIVIEW IMAGING
BY:
MOBOLAJI AKINOLA

SUPERVISORS:
PROF. LAURENCE DOOLEY
DR. PATRICK WONG ©2008 by Mobolaji Akinola
BACKGROUND
• Massive interest in the use of imaging
technologies due to reduction in the price of
digital cameras/camcorders and increase in
computer processing power.
• Emerging new applications such as: mobile video
conferencing, object tracking and image
recognition, HDTV, 3-D D television (3DTV), free
viewpoint TV and Holographic projections.
• Proliferation and increase use in traditional
applications areas such as police and traffic
surveillance turning our society to a Big Brother
society. ` ©2008 by Mobolaji Akinola
BACKGROUND….. Cont’d.
• Convergence of various Imaging technologies
such as: computer vision ( image segmentation
and recognition), computer graphics (image
modeling and animations), next generation
communications (internet streaming, cognitive
radio) and distributed video sensors.
• Emergence and rapid evolution of various video
coding standards which includes ITU-T
ITU and
MPEG.
• Enabling various devices with different
capabilities and competing resources scalable
and flexible access to video streams coded as
©2008 by Mobolaji Akinola

complex data sets.

CHALLENGES
• The challenge of change in paradigm from more
pixels and higher resolution to more views which
is made possible by array of cheap cameras.
• The challenge to continue to search for higher
efficiency coding methods due to more media
being needed for new applications such as 3-D3
television (3DTV), free viewpoint TV and
Holographic projections. The bit rate for reality is
1.36 X 1012 bits/second [1] calculated by Holman
• The challenge of making codes and systems
scalable to accommodate the different devices
that capture, process, stream and render video in
©2008 by Mobolaji Akinola

terms of capacity and power.

CHALLENGES….Cont’d
• The challenge of reducing the complexity of the
encoder and balancing the work load between
encoder and decoder so that devices with
resource constraints can effectively encode. One
area of current research is distributed video
coding.
• The challenge to continue to improve Standards
so that current further research is geared towards
compatible frameworks. The trend even for
standards to converge, e.g. Joint Multi-view
Multi
Video Model (JMVM), a standard being worked
upon by both ITU-T T and MPEG.
©2008 by Mobolaji Akinola
QUESTIONS ARRISING FROM CHALLENGES

• Can more efficient codes be written to meet the

demands for higher resolution and reality media?
–Are
Are current metrics optimal?
–Can
Can other types of metrics be defined? E.g.
perceptual quality indices
• How to reduce encoder complexity and/or
balance the encoder-decoder
decoder computational
overhead?
Introduction of methods that are
–Introduction
computationally less complex?
– Introducing scalable/selectable modes of
operation depending on application
©2008 by Mobolaji Akinola
QUESTIONS ARRISING FROM CHALLENGES……. Cont’d

• How to increase decoder flexibility and/or balance

the encoder-decoder
decoder computational overhead?
–The
The possible use of Distributed video
coding techniques?
– Will performance levels ever approach
conventional coding standards?
–Possible
Possible hybrid solutions with aspects of
both co
Can decoder request only amounts of data
–Can
it needs to decode particular scenes,
thereby decoders become more
active/cognitive?
©2008 by Mobolaji Akinola
DISTRIBUTED VIDEO CODING
• INTRODUCTION TO DVC
– In today’s conventional coding framework, the
encoder has the task to exploit both the temporal and
spatial redundancies.
– The exploitation of these redundancies involves both
motion estimation and inter-frame
inter predictive coding.
– As a consequence encoders have much higher
computational complexity than decoder.
– Encoders are typically 5 to 10 times more complex
than decoders [2].
– The DVC paradigm proposed to allow decoder take on
some of the computationally intensive encoder work.
©2008 by Mobolaji Akinola
DISTRIBUTED VIDEO CODING…. Cont’d
• THEORETICAL FOUNDATION FOR DVC
A. Slepian-Wolf Coding
• If X and Y be two correlated memoryless sources, losslessly
encoded
–If
If they communicate, well known Shannon theory gives
joint entropy = H(X,Y)
–Slepian
Slepian and Wolf established surprisingly in 1997 that if
the sources do not communicate, but jointly decoded a
higher entropy can even be achieved i.e. ≥ H(X,Y) [4]
–Proof
Proof is by random binning which is non constructive.

Fig.1 Distributed compression of two

statistically dependent random
processes, X and Y. The decoder
jointly decodes X and Y and thus
exploits mutual dependence [3]

©2008 by Mobolaji Akinola

DISTRIBUTED VIDEO CODING…. Cont’d
B. Wyner-Ziv Coding
• X and Y two correlated sources coding problem considered
w.r.t. a fidelity criterion, they established in 1976
–The rate-distortion
distortion (RD) function R* X/Y(D) for the case
where SI Y is perfectly known to the decoder only.
–For
For Gaussian sources and a mean square error distortion
measure, there is no rate loss w.r.t. joint encoding and
joint decoding.
–The
The information theoretical bounds for lossy compression
with SI at the decoder.
–AA distortion is acceptable, which tends to zero for the
gaussian case. Theoretical proof of rate loss is 0.5 b/s [6]
Fig.2 Lossy compression of
random sequence X using
statistically related SI Y. [3]

©2008 by Mobolaji Akinola

DISTRIBUTED VIDEO CODING…. Cont’d
• PRACTICAL IMPLEMENTATION
A. Slepian-Wolf Coding
• Though theory dates back to 70s, only recently in the 90s
that emerging applications motivated serious attempts at
implementation, e.g. [5]. The implementation is akin to
lossless channel coding
–If
If X & Y are similar binary sequences, then error
sequence will mostly be 0s except for few 1s.
–Therefore
Therefore channel coding is applied and only parity bits
are transmitted.
–Decoder
Decoder uses SI to perform error correcting decoding.
–Most
Most distributed source coding today are derived from
proven channel coding ideas.
–Turbo
Turbo codes, Iterative channel codes and Low density
parity-check
check codes (LPDC) codes have been used.
©2008 by Mobolaji Akinola
DISTRIBUTED VIDEO CODING…. Cont’d
B. Practical Wyner-Ziv
Ziv Coding
• Like S & W coding , Serous implementation efforts at
implementing Wyner-Ziv
Ziv theory was only recently undertaken
–Practical
Practical code construction rely on a quantizer (source
code) followed by a S&W coder (channel code).
–The
The quantizer partitions the continuous valued source
space into cells.
–A
A codeword is associated with each cell therefore
constructing a source codebook.
–The
The source codebook is used by S&W coder and channel
coded, transmitting only the parity bits w.r.t available SI
–Under
Under ideal Gaussian assumptions the the W & Z limits
can be achieved with nested lattice quantizers

©2008 by Mobolaji Akinola

DISTRIBUTED VIDEO CODING…. Cont’d
DVC potential Benefits
• Low Complexity Video Encoding
–Conventional
Conventional video systems are asymmetric having
highly complex encoders which is suitable for present day
video broadcasting. Or retrieval from powerful servers
–However,
However, large deployment of mobile devices induces
need for structure where video is encoded on low cost
devices but decoded on powerful platforms
• Research Space
– The area is still evolving and there are many problems to
be solved
–There
There is a significant gap in performance of DVC codecs
relative to conventional decoders
–DVC
DVC research have concentrated on Mono-view
Mono codecs
(intra-view) and Multi-view
view DVC (inter-view)
(inter is neglected.
©2008 by Mobolaji Akinola
DISTRIBUTED VIDEO CODING…. Cont’d
Short Term Goals
• Conventional Research Video Acquisition
–Need
Need to acquire the videos usually used and mentioned in
research papers
–This
This will enable us to try to replicate research work so far
done and compare our results with reported results.
• Implementation of DVC codecs in Existing Format
– Step up effort to acquire codes and codecs existing
–Break
Break them down and try out improvements
• Acquire our own video sequences
• Target a quality conference
• Gather material for journal publication

©2008 by Mobolaji Akinola

REFERENCES
• [1] T. Holman, “The bit rate of reality [picture/sound reproduction]”
Intl. Conference on consumer electronics, ICCE 2000.
• [2] Christine Guillermot et all, “Distributed Mono-view
Mono and multi-
view video coding, IEEE Signal Processing Magazine 2007.
• [3] Bernd Girod et all, “Distributed Video Coding”, IEEE special
issue on advances in Video coding and delivery 2005.
• [4] Slepian and Wolf, “Noiseless coding of correlated information
sources”, IEEE Trans. Inform. Theory 1973.
• [5] Pradhan and Ramchandran, “Distributed source coding using
syndromes (DISCUS): Design and construction”, IEEE data comp.
conf. 1999
• [6] R. Zamir, “The rate loss in Wyner-Ziv
Wyner problem”, IEEE
transactions on information theory 1996.

©2008 by Mobolaji Akinola