H63DCM

DIGITAL COMMUNICATIONS
1
 Lecture 2&3
2

Lecture 2 & 3 : Digital Encoding Techniques

Sampling and Quantisation
Quantisation Noise
Companding

H63DCM
 Types of Sources
3

Source with finite alphabet set (text):

Example: Roman (ABCXYZ + punctuation marks), Greek
(), Chinese, Arabic
ASCII is widely used to encode Roman characters (7-bits, 8-bits)
and UNICODE (for Chinese, eg) 16-bits.
Each ASCII character is chosen from a set of 27=128 (or 28=256)
characters for ASCII and 216=65536 for Unicode16.
Source with infinite alphabet set:
Example: sound, image, movie (sound+image)
Encoding usually textify such sources
Typical specifications:
Sound: 16bits x 2channels (stereo) @ 44.1kHz
Image: 3,12, 24bits for 3 channels (Red, Green, Blue)
(i.e., 1,4,8-bit colour depth per channel of RGB)

H63DCM
 Source Encoding
4

Sound: Discretize continuous waveform in TIME

Image: Discretize visual perception in SPACE
Requires an ADC, an electronic integrated circuit which
converts continuous signals to discrete digital numbers
Sound Source Image Source
y-axis: Resolution

y-axis: Color Depth

e.g., 216 levels

e.g., 212 levels

x-axis: TIME x-axis: SPACE

Encoding transforms data from one representation to another

Digital Audio uses waveform encoding: Pulse-Code Modulation
(PCM), i.e., sampling and quantizing performed by an Analog-to-
Digital-Converter
H63DCM
 Impulse Sampling
5

Impulse sampling or ideal sampling is the process of

multiplying a signal x(t) by a train of impulses,

Ts (t ) (t nT )
n
s

The resulting sampled waveform xs(t) is:

xs (t ) x(t ) (t nTs ) x(nT ) (t nT )
s s
n n

x(t) xs(t) sampled signal:

analog
input discrete-time
signal continuous amplitude

H63DCM
 F.T. of Sampled Signal
6

The Fourier Transform of a signal x(t) impulse sampled at a rate

of fs is:

X s ( f ) fs X ( f f n)
n
s

Where fs is the sample frequency/rate:

1
fs
Ts
The F.T. of the sampled waveform consists of a train
of spectral copies of the original waveforms Fourier Transform.
The spectral copies are centered around integer multiples of
the sample frequency (harmonics).
The copies are weighted by amount fs

H63DCM
 F.T. of Sampled Signal
7

H63DCM
 F.T. of Sampled Signal
8

If the F.T. of the original signal looks like:

X(f)

-B 0 B f

Then the F.T. of the sampled signal is:

height fs
Xs(f)

... ...
-fs 0 fs f

Here we have assumed that fs2B

H63DCM
 Recovering the Analog Signal
9

The original analog signal is recovered from the sampled signal by

using an ideal low-pass filter or Digital-to-Analog converter:

... ...
fs fs f
-fs 0 fs
2 2

If fs2B then we recover x(t) exactly:

X(f)

-B 0 B f

H63DCM
 Undersampling and Aliasing
10

If the waveform is undersampled (i.e. fs<2B) then there will be

spectral overlap in the sampled signal:

The signal at the output of the DAC will be different than the original
analog signal:

Aliasing has occurred!

H63DCM
 Undersampling and Aliasing
11

Due to undersampling
Appear in the frequency band between ( f s f m ) and f m

H63DCM
 Nyquist Sampling Rate
12

The analog sound wave is sampled at its predefined input frequency

Nyquist sampling criterion: minimum rate of sampling without loss of information
is twice the bandwidth (highest frequency component) of input waveform
fsamp > 2 * fmax (frequency)

i.e., Tsamp < 0.5 * Tmax (time)

where Tmax = period of the highest frequency component

If Nyquist criterion not met, aliasing happens. To avoid aliasing, which is a result of
frequency components higher than half the sampling rate, perform first filtering of
these frequency components.

H63DCM
 Nyquist Sampling Rate & Aliasing
13

Nyquist Theorem:
We can digitally represent only frequencies up to half the
sampling rate.

Example:
CD: fsamp=44,100 Hz
Nyquist Frequency = fsamp/2 = 22,050 Hz

Example:
FM:fsamp=22,050 Hz
Nyquist Frequency = fsamp/2 = 11,025 Hz

H63DCM
 Nyquist Sampling Rate & Aliasing
14

Frequencies above Nyquist frequency "fold over" to sound like

lower frequencies.
This foldover is called aliasing.

Aliased frequency f in range [fsamp/2, fsamp] becomes f':

f' = |f - fsamp|

H63DCM
 Nyquist Sampling Rate & Aliasing
15

f' = |f - fsamp|

Example:

fsamp = 20,000 Hz
Nyquist Frequency = 10,000 Hz
f = 12,000 Hz --> f' = 8,000 Hz
f = 18,000 Hz --> f' = 2,000 Hz
f = 20,000 Hz --> f' = 0 Hz

H63DCM
 Nyquist Sampling Rate & Aliasing
16
Original (blue) sine wave is @ 5000Hz

H63DCM
 Nyquist Sampling Rate & Aliasing
17

The frequency of the aliased signal is the difference between the

signal frequency and the sampling rate.
E.g., a 2 kHz sine wave being sampled at 1.5 kHz would be
reconstructed as a 500 Hz sine wave.

Use of an anti-aliasing filter to avoid aliasing:

the input to an ADC must be low-pass filtered to remove
frequencies above half the sampling rate.

Aliasing is USEFUL in providing simultaneous down-mixing of a

band-limited high frequency signal in the frequency mixer circuit.

H63DCM
 Example
18

Example: Music Example

We demonstrate aliasing (due to undersampling) for a short piece of music:
For these examples, we first resampled each piece of music to 48kHz (from 44.1kHz);
note that this step does not add any information to the signal, but merely simplifies the
process of down-sampling.
Then, we down-sampled the piece of music to 8kHz, 4kHz and 2kHz in two different ways:
(1) with low-pass filtering prior to down-sampling and
(2) without low-pass filtering.
The examples with low-pass filtering are referred to as anti-aliased, since the low-pass
filtering prior to down-sampling ensures that no high-frequency aliases appear in the
reconstructed signal.
Listening to these two different versions at each lower sampling frequency, we can
definitely hear distortion (aliasing) in the examples where no low-pass filtering was
applied prior to down-sampling, while such distortion is not evident in the anti-aliased
(low-pass filtered) examples.

H63DCM
 Example
19

SAMPLING RATE Anti-Aliased Aliased

48 kHz Music48kHz.wav N/A
8 kHz Music08kHzAA.wav Music08kHz.wav
4 kHz Music04kHzAA.wav Music04kHz.wav
2 kHz Music02kHzAA.wav Music02kHz.wav

H63DCM
 Example Problem
20

The bandwidth of a speech signal is from 50 Hz through to 4 kHz and that of

a music signal is from 20 Hz through to 18 kHz. You want to digitize these
signals using the Nyquist criterion.

a. What is the bit rate produced for the speech signal if 12 bits are used per
sample?
b. Perform the same for the music signal when 16 bits per sample are used.
c. How many mega bytes of storage do you need for 30 minutes of
stereophonic music ?

Answer:
a. Speech signal must be sampled at 8 kHz. Therefore its bit rate = 12 * 8 = 96
kbps
b. Music signal must be sampled at 36 kHz. Therefore its bit rate = 16 * 36 = 576
kbps
c. 30 minute of mono music requires 30*60*576K = 1036 Mbits = 130 Mbytes
For stereophonic music, space required = 130*2 = 260 Mbytes
H63DCM
 Conclusion Sampling Theorem
21

The original signal can, in principle, be recovered from a Nyquist or an oversampled

signal simply by low pass filtering to remove the higher frequency bands.
Oversampling, of course, usually leads to the production of more binary digits and so
is generally to be avoided. However, if Nyquist sampling is used, it can be difficult to
design a filter with a sufficiently sharp cut off to efficiently exclude the higher
frequency bands, so often some small amount of oversampling is used. For example
in many telephone systems, the voice signals are first bandlimited to frequencies
below 3,400 Hz and then sampled at a sampling frequency of 8kHz.

It should be noted that the sampling theorem (and hence the ability to faithfully
recover a continuous signal from a set a samples) assumes that the samples (although
quantised in time) are not quantised in amplitude. Hence for the reconstructed signal
to be as close as possible to the original, the quantisation must have sufficient levels
so as to cause only a minimum of error. The level of error introduced by quantisation
is discussed next.

H63DCM
 Quantisation
22

A quantiser takes a sample defined on a continuous range

of values, and maps it to one of a set of discrete values.
The quantiser is completely specified by its quantisation
levels:
Example: = {-3,-1,+1,+3}

The quantiser rounds the continuous value to the

closest quantisation level.
Examples: 1.5 1, 2.9 3, -2.3 -3, etc.

H63DCM
 Quantisation
23

Quantization is:
An irreversible process.
A source of information loss.
A critical stage in image and video compression.
It has significant impact on:
The distortion of reconstructed image and video
The bit rate of the encoder .

H63DCM
 Quantisation: Decisions
24

Resolution
How many bits should we use ?
Step Size
How to spread the resulting quantization levels

Quantization noise
How efficient can this process be ?
How much noise we insert to the quantized signal ?
SNR, MSE ....

H63DCM
 Rate Distortion Theory
25
Ratedistortion theory is a major branch of information theory which
provides the theoretical foundations for lossy data compression.

Addresses the problem of determining the minimal number of bits per

symbol, as measured by the rate R, that should be communicated over a
channel so that the source (input signal) can be approximately
reconstructed at the receiver (output signal) without exceeding a given
maximum distortion D.

Audio, speech, image, and video compression techniques have

transforms, quantization, and bit-rate allocation procedures that
capitalize on the general shape of ratedistortion functions.

Ratedistortion theory was created by Claude Shannon in his

foundational work on information theory
H63DCM
 Rate Distortion Theory
26

Rate:
It is the number of bits per data sample to be stored
or transmitted.

Distortion:
It is defined as the variance of the difference
between input and output.
Measured by:
Hamming distance
Squared error

H63DCM
 Rate Distortion Theory
27

Rate distortion theory deals with the

problem of representing information
allowing a distortion

idea: less exact representation needs less bits

H63DCM
 Distortion Measure
28

Lower the bit-rate R by allowing some acceptable distortion

D of the signal

H63DCM
 Example
Mouse: Original 29Picture

H63DCM
 Connecting rate-distortion theory to
channel capacity
31
Suppose we want to transmit information about a source to the user with a
distortion not exceeding D:
o Ratedistortion theory tells us that at least R(D) bits/symbol of
information from the source must reach the user.
o We also know from Shannon's channel coding theorem that if the source
entropy is H bits/symbol, and the channel capacity is RC; thus RC < H
o Therefore, H RC bits/symbol will be not used when transmitting this
information over the given channel.
o For the user to have any hope of reconstructing with a maximum
distortion D, we must impose the requirement that the information lost
in transmission does not exceed the maximum tolerable loss of H R(D)
bits/symbol.
o This means that the channel capacity must be at least as large as R(D).

H63DCM
 Designing the quantiser
32
Dec region #4
1. Decide how many Upper Boundary #4
quantization levels, i.e.,
resolution of quantizer 10 11
Lower Boundary
2. Assign codewords to each #4
level Dec region #3 Upper Boundary
#3
(Gray code yields lower bit 11 10 Lower Boundary
error rates) #3
3. Define decision regions Upper Boundary
01 01 #2
with their boundaries
Dec region #2 Lower Boundary
midway between #2
quantization levels (for Upper Boundary
extremities, the boundaries 00 00 #1

are infinity)
Gray Binary Dec region #1
Lower Boundary
#1
H63DCM
 Uniform Scalar Quantization
33

The endpoints of partitions of equally spaced intervals in the

input values of a uniform scalar quantizer are called decision
boundaries
The output value for each interval is the midpoint of the
interval
The length of each interval is called the step size
A UQT can be midrise or midtread

H63DCM
 Uniform Scalar Quantization
34

H63DCM
 Uniform Scalar Quantisation
35

Midtread quantiser
Has zero as one of its output values
Has an odd number of output values
Midrise quantiser
Has a partition interval that brackets zero
Has an even number of output values

H63DCM
 Uniform Scalar Quantisation
36

We want to minimize the distortion for a given input source with

a desired number of output values
Do this by adjusting the step size to match the input
statistics
Let B = {b0, b1, , bM } be the set of decision boundaries
Let Y = {y1, y2, , yM } be the set of reconstruction or
output values

H63DCM
 Uniform Scalar Quantisation
37

Assume the input is uniformly distributed in the interval

[-Xmax, Xmax]
Then the rate of the quantizer is
R = log2 M
R is the number of bits needed to code the M output values
The step size is given by
= 2Xmax/M

H63DCM
 Using the Quantiser
38

Sample the analog waveform

Depending on decision region of
the sampled waveform at time k,
map the signal to its nearest level
(nearest neighbour quantisation)
The more decision regions, i.e., After quantization
high resolution, the smaller the
quantisation noise (error as a result
of quantisation).

H63DCM
 Example: Uniform Scalar Quantisation
39

For the following sequence {1.2,-0.2,-0.5,0.4,0.89,1.3},

quantise it using a uniform midrise quantiser in the range of (-
1.5,1.5) with 4 levels, and write the quantised sequence.

H63DCM
 Uniform vs. Nonuniform Quantisers
40
A quantizer can be completely specified by a list of
quantization levels.
~
X {~
x1 , ~
x2 , , ~
xL }
This means the boundaries (endpoints) of the quantization
regions do not need to be separately specified.
Why?

A quantizer can be uniform or nonuniform:

Uniform:
~
xk ~
xk 1 const

Optimal if X has uniform pdf.

Otherwise, it is nonuniform
Optimal if X has pdf other than unifom.
H63DCM
 Nonuniform Quantisation
41

Many signals such as speech have

Output signal
~
x 6
a nonuniform distribution.
The amplitude is more likely to
4 be close to zero than to be at a
high level.
2
Nonuniform quantizers have
unequally spaced levels
-8 -6 -4 -2 2 4 6 8
The spacing can be chosen to
-2
Input signal optimize the SNR for a
x
particular type of signal.
-4

-6

Example Nonuniform 3 bit quantizer

H63DCM
 Nonuniform Quantisation: Non-linear ADC
42

Companding (COMPressing and exPANDING):

to reduce no of bits required in a converter while
achieving an equivalent dynamic range or signal-to-
quantization-noise ratio.
In order to improve the resolution of weak signals within a
converter (and hence enhance the signal-to-quantization-
noise ratio), the weak signals need to be enlarged, or the
quantization step size decreased.
Strong signals can be reduced without compromising
(significantly) the signal-to-quantization-noise ratio.
Used widely in audio ADCs.

H63DCM
 Companding : u-law
43

mu-law (also "u-law") encoding is a form of logarithmic

quantization. It is based on the observation that many signals
are statistically more likely to be near a low signal level than a
high signal level. Therefore, it makes more sense to have more
quantization points near a low level than a high level.
Typically, linear samples of 14 to 16 bits are companded to 8
bits.
Most telephone quality codecs (eg Sparcstation's audio codec) use mu-
law encoding.
Adopted in North America, Japan and Australia

where u = 255 if 8-bit (US standard) OR 2n -1 if n-bit.

H63DCM
 Companding : A-law
44
Similar to u-law companding, this is used predominantly in the European
countries

where A is the compression parameter (typically 87.7)

Figure adopted from Wikipedia

H63DCM
 Quantisation

Quantisation Noise
Pulse Code Modulation

H63DCM
 Quantisation Error

The quantisation error is /2e /2

H63DCM
 Quantisation Error

SNR due to quantization errors

/2 1 2
/2
e 2
E[e ] e p(e)de e de
2 2 2
/ 2 / 2 12

1 X max
S 2 M 2 2

N q rms e2 12

M2 3 2Xmax=M

2 2
 Pulse Code Modulation

Each quantization level is expressed as a codeword

M-level quantizer consists of M codewords
A codeword of M-level quantizer is represented by m-
bit binary digits.
m log 2 M
PCM
Encoding of each quantized sample into a digital
word

H63DCM
 Pulse Code Modulation

H63DCM
 Example

x(t) is a sum of 3 sinusoids whereby each takes the form an sin(2tfn) where n=1,2, and
3. The parameters of the three sinusoids have values:

a1= 1 V; a2 = 0.33V and a3 = 0.2 V;

f1= 500 Hz, f2= 1500 Hz and f3 = 2500 Hz
The resulting waveform evaluated from t = 0 to t = 2ms is plotted below.

H63DCM
 Example

Design a 3-bit uniform quantiser for x(t), whereby the ADC has knowledge of the
amplitudes an of the waveform. State clearly the maximum and minimum voltages, the
decision regions, the output voltage of the quantiser and the boundaries of the quantiser.

t(ms) x(t) Quantizer Output Squared Error (V2)

Voltage (V)
0
0.2
0.4
0.6
0.8
1.0

H63DCM