King Fahd University of Petroleum and Minerals

Electrical Engineering Department

Dr. Suhail Al-Dharrab

Office: 59-0012-05
Office Hours: Sun, Tue: 13:00-13:50
Mon,Wed: 11:00-11:50 or by appointment
E-mail: suhaild@kfupm.edu.sa, Tel: 8210

EE576 ErrorControlCoding(142)

Lec01

Lecture Outline
Basics of error control coding
Coding gain
Hamming bound
Gilbert bound

EE576 ErrorControlCoding(142)

Lec01

Communication System

EE576 ErrorControlCoding(142)

Lec01

Error Control Coding

What is error control?
Detecting/Correcting errors in digital data
How?
By adding redundancy to the information sequence (encoding)
and utilize it to catch the errors (decoding)
Why?
Enhance the reliability of the system
4

EE576 ErrorControlCoding(142)

Lec01

Binary Repetition Codes

Ex: Consider a (3,1) Binary Repetition Code over Binary
Symmetric Channel (BSC)

Coding Rule: 0 => 000

1 => 111
If received sequence: 011. What was transmitted?
Scenario A: 111 with one error in 1st location
Scenario B: 000 with two errors in 2nd & 3rd locations.
Maximum Likelihood Decoding (MLD):
Scenario A
1
Scenario B
1
Scenario A
Scenario B
(for
Decoding decision: 011 => 111
5

EE576 ErrorControlCoding(142)

0.5)
Lec01

Post-Decoding Probability of Error

(3,1) repetition code can correct single errors.
3
2

Block Error Probability

3
3

1
3 1

In general for a -error correcting code:

1
Bit error probability

[for the (3,1) code,

Improvement: For a BSC with

10 ,

]
10 .

Cost: Expansion in bandwidth.

6

EE576 ErrorControlCoding(142)

Lec01

Coding Gain
Coding gain is the reduction in / required to achieve a
specific BER for a coded system compared to uncoded
system.
For fair comparison,
/
of the coded system must be
scaled by the rate of the code (or equivalently reduced by
10 log 1/ dB), to ensure the same energy per information
bit for both systems.
Ex: For DPSK
0.5 exp
When coding is introduced,

/
/

.
drops by a factor of

We should calculate
0.5 exp
/
, then evaluate
as a function of . Now it is fair to compare
and .
7

EE576 ErrorControlCoding(142)

Lec01

 Plot the curve of the uncoded

system (A)
Shift it to the right by 10log 1/ .
(B). This is the BER of the coded
system before decoding, at the
same energy per information bit.
Use the values of BER on (B) to
find error rate after decoding (C).
Compare A and C for code gain.
Asymptotic gain: gain at large
/
4-5 dB gain is typical. Some codes
provide up to 9 dB.

BER

Graphical Illustration

EE576 ErrorControlCoding(142)

B
G A
Eb/N0

Lec01

Hamming Distance
Def: The Hamming distance between two codewords
, denoted by

and

, is the number of components (bits) at

which they differ.

011,000

011,111

Therefore 011 is closer to 111.

Maximum Likelihood Decoding (MLD) reduces to Minimum
Distance Decoding.
9

EE576 ErrorControlCoding(142)

Lec01

Geometrical Illustration
000

001
010

011

101

100

110
10

EE576 ErrorControlCoding(142)

111
Lec01

Error Correction and Detection

Consider a code consisting of two codewords with
Hamming distance .

How many errors can be detected?

How many errors can be corrected?
Number of errors that can be detected

Number of errors that can be corrected

In other words, for -error correction, we must have
2
11

EE576 ErrorControlCoding(142)

Lec01

Error Correction and Detection

Ex: A binary code of two codewords and

It can correct two errors

or
It can detect four errors
or
It can correct one error and detect two more errors.
In general:
12

1
EE576 ErrorControlCoding(142)

Lec01

Minimum Distance of a Code

Def: The minimum distance of a code C is the minimum
Hamming distance between any two different codewords.
min

and in

A code with minimum distance

can correct all error

patterns up to and including -error patterns, where

2

It may be able to correct some higher error patterns, but not

all.
13

EE576 ErrorControlCoding(142)

Lec01

(7,4) Binary Code

No. Message Codeword

0
0000
0000000
1
1000
1101000
2
0100
0110100
3
1100
1011100
4
0010
1110010
5
1010
0011010
6
0110
1000110
7
1110
0101110

14

No. Message Codeword

8
0001
1010001
9
1001
0111001
10
0101
1100101
11
1101
0001101
12
0011
0100011
13
1011
1001011
14
0111
0010111
15
1111
1111111

EE576 ErrorControlCoding(142)

Lec01

Coding: Gain and Cost

Given an

code.

- Gain is proportional to the error correction capability, .

- Cost is proportional to the number of check digits,
Given a sequence of

information digits, it is desired to

add as few check digits ( ) as possible to correct as many

errors ( ) as possible.
What is the relation between these code parameters?
15

EE576 ErrorControlCoding(142)

Lec01

Hamming Bound
For an

code, there are 2 codewords and 2 possible

received words.
Think of the 2 codewords as centers of spheres in an dimensional space.
All received words that differ from codeword
positions lie within the sphere

of center

in

or less

and radius .

For the code to be -error correcting (i.e. all t-error pattern

for any codeword transmitted can be corrected), all
spheres
16

1, . . , 2 , must be disjoint.
EE576 ErrorControlCoding(142)

Lec01

Hamming Bound
In other words, when a codeword is selected, none of the
-bit sequences that differ from that codeword by

or less

locations can be selected as a codeword.

Consider the all-zero codeword. The number of words that
differ from this codeword by locations (bits) is
The total number of words in any sphere (including the
codeword at the center) is
1
17

EE576 ErrorControlCoding(142)

Lec01

Hamming Bound
The total number of -bit sequences that must be available
for the code to be a -error correcting code is
2
But, the total number of sequences is 2n. Therefore,
2

2
or
2

18

EE576 ErrorControlCoding(142)

Lec01

Hamming Bound
This bound is known as the Hamming (sphere-packing)
Bound. It provides a necessary, but not a sufficient,
condition for the existence of an
,
-error correcting
code.
Ex: Is it theoretically possible to design a (10,7) single-error
correcting code?
10
10
1 10 11 2
0
1
It is not possible.
A code for which the equality is satisfied is called a perfect
code.

19

EE576 ErrorControlCoding(142)

Lec01

Gilbert Bound
It provides the sufficient condition for the existence of a
, code with
.
1
2

Ex: What are the values of that are guaranteed to exist for a
single-error-correcting code of
10.
Gilbert bound requires
5. A (10,5) code with
3
does exist.
Hamming bound requires
6. A (10,7) code with
3
does not exist. The (10,6) code with
3, according to
the two bounds,may or may not exist. It is found to exist.
20

EE576 ErrorControlCoding(142)

Lec01

The Encoding Problem

How to select 2k codewords of the code C from the 2n
sequences such that some specified (or possibly the
maximum possible) minimum distance of the code is
guaranteed?
Ex: How were the 16 codewords of the (7,4) code
constructed? Exhaustive search is impossible, except for very
short codes (small k and n).
How would the 2k n-bit codewords be assigned to the 2k kbit information sequences?
Are we going to store the whole table of 2k(n+k) entries?!
A constructive procedure for encoding is necessary.
21

EE576 ErrorControlCoding(142)

Lec01

The Decoding Problem

Standard Array
000000011010000110100101110011100100011010100011001011101010001011100111001010001101010001110010110010111 1111111
00000011101001011010110111011110011001101110001110101111101000001110001100100000110001000101001010 00101101111110
00000101101010011011010111101110000001100010001000101100101001101110111100111000111101000011001001 00101011111101
00001001101100011000010110001110110001111010000100101010101010101111011100001000100101001111001111 00100111111011
0001000110000001111001010100111101000100101001110010011010110010`10001110110100001010101011100001100111111110111
00100001111000010010010011001100010000101010101100111110100000101010011110101001110101100111011011 00001111101111
01000001001000001010011111001010010011101011001100001110111000100110011000101010110100000111101011 01101111011111
10000000101000111010000111000110010101101000001101101110001000111110010100101100110111000110001011 10101110111111

Exhaustive decoding is impossible!!

Well-constructed decoding methods are required.

22

EE576 ErrorControlCoding(142)

Lec01

The Course is About

Design/Find good codes, driven by the Noisy Channel
Coding Theorem

There exist codes of rate R that have arbitrarily small decoding

error probability (provided that R is less than the channel
capacity)
Propose encoder and decoder circuits and algorithms
Evaluate the performance of the code
23

EE576 ErrorControlCoding(142)

Lec01