Error Control Coding

Saswat Chakrabarti
GS Sanyal School of Telecommunications,
IIT Kharagpur

02/01/15

SCK, IIT Kharagpur

Some milestones in the history of

Error Control Coding
1948-49: C.E. Shannons pioneering papers on A
Mathematical Theory of Communication
1950: R. W. Hamming reports Hamming Code
1954-55: P. Elias introduces convolutional codes
1959-60: R. C. Bose and D. K. Ray-Chaudhuri and A.
Hocquenghem independently invent BCH Codes
1960: I. S. Reed and G. Solomon describe ReedSolomon (RS) codes
1966: G. D. Forney explains concatenated coding
1967: A. J. Viterbi introduces a Maximum Likelihood
(ML) decoding algorithm for convolutional codes
02/01/15

SCK, IIT Kharagpur

Some more recent

milestones
1977: Voyger deep space mission uses a RS-Conv.
Concatenated codec
1980: Standard for compact disc including RS code
formulated
1982: Trellis Coding (TCM) described by G. Ungerboeck
1993: C. Berrou, A. Glavieux and P. Thitimazshima
formulate Turbo Codes
1995: D. J. MacKay and R. M. Neal rediscover LDPC
codes long after R. G. Gallager [1962]
1998: S. Alamouti describes Space-time coding
02/01/15

SCK, IIT Kharagpur

Reference
Shu Lin and D. J. Costello, Jr., Error Control Coding, 2nd
Ed., Pearson Prentice Hall, 2004
R. G. Gallager, Information Theory and Reliable
Communications, John Wiley, New York, 1968
R. E. Blahut, Theory and Practice of Error Control
Codes, Addison Wesley, 1983
T. K. Moon, Error Correction Coding, mathematical
methods and algorithms, Wiley India Ed., 2005
W. W. Peterson and E. J. Weldon, Jr., Error Correcting
Codes, 2nd Ed., MIT Press Cambridge, 1972
02/01/15

SCK, IIT Kharagpur

Block diagram of a typical data transmission system

Information
source

Source
encoder

Channel
encoder

Channel
(storage
medium)

Noise

Destination

02/01/15

Source
decoder

Channel
decoder

SCK, IIT Kharagpur

Modulator

Demodulator

Modulation Channel
Propagation
Channel

Receiver

Transmitter

Coder

Modem

RF/IF
Stages

RF/IF
Stages

Modem

Decoder

Radio
Channel

Discrete / Coding Channel

02/01/15

SCK, IIT Kharagpur

Means of Error Control

Error Control Coding
(Channel Coding)

ARQ
(Auto Repeat reQuest)

02/01/15

Forward Error
Correction
(FEC)

SCK, IIT Kharagpur

Hybrid ARQ

A Classification of FEC Codes

FEC Codes

Tree
(Convolutional)

Block

Non-Binary

Binary

Systematic

Systematic

Nonsystematic

Nonsystematic

02/01/15

SCK, IIT Kharagpur

Non-Binary

Binary

Systematic

Systematic

Nonsystematic

Nonsystematic

A binary block code with k = 4 and n = 7

02/01/15

Messages

Codewords

(0000)

(0000000)

(1000)

(1101000)

(0100)

(0110100)

(1100)

(1011100)

(0010)

(1110010)

(1010)

(0011010)

(0110)

(1000110)

(1110)

0101110)
SCK, IIT Kharagpur

02/01/15

Messages

Codewords

(0001)

(1010001)

(1001)

(0111001)

(0101)

(1100101)

(1101)

(0001101)

(0011)

(0100011)

(1011)

(1001011)

(0111)

(0010111)

(1111)

(1111111)
SCK, IIT Kharagpur

10

Classification of Decoding techniques for FEC Codes:

Decoding Techniques

Minimum Dist Dec

Syndrome
Decoding

02/01/15

Trellis
Decoding

Bounded Distance
Decoding
Sequential
Decoding
Majority Logic
Decoding & threshold decoding
SCK, IIT Kharagpur

11

Classification of Syndrome Decoding techniques

Syndrome Decoding
Usually for linear
block codes

List /
Table look
up

Systematic
Search
Error
trapping

02/01/15

Step
by step

Algebraic

Meggitt

SCK, IIT Kharagpur

12

Memoryless Channels
A memoryless channel is one in which the
output rn at the n-th symbol time depends only on
the input at time n.
Given the input at time n, the output at time n is
statistically independent of the outputs at other
times.
Additive Gaussian channel and Binary
Symmetric Channel (BSC) may be viewed as
memoryless channels.
02/01/15

SCK, IIT Kharagpur

13

q1
1 q1

s1

1- q2

s1

q2
q1 << q2
Sate s1
1 p1

q1 << q2
p1

Sate s2
1 p2
p2

p1
1

1 p1

p2
1

1 p2

A Simplified model of a channel with memory.

02/01/15

SCK, IIT Kharagpur

14

Introduction to Linear Algebra

Modern algebraic theory classifies many arithmetic
systems according to their mathematical
strength. For example,
Group: A set of mathematical objects (elements)
that can be added and subtracted.
Ring: A set of mathematical objects that can
be added, subtracted and multiplied.
Field: A set of mathematical objects that can be
added, subtracted, multiplied and divided.

02/01/15

SCK, IIT Kharagpur

15

Introduction to Linear Algebra

Ex. of a Field [GF (2)]: Simplest one with 2
elements (say, 0 & 1).
0+0=0
0+1=1
1+0=1
1+1=0
(Addition in GF (2) is the
modulo-2 addition or
EXOR addition)
02/01/15

0.0=0
0.1=0
1.0=0
1.1=1
(Multiplication in GF (2)
is modulo-2 multiplication
or AND operation )
SCK, IIT Kharagpur

16

Introduction to Linear Algebra

Further, 1 + 1 = 0 - 1 = 1 subtraction
& 1. 1 = 1 1%1 = 1 division
Note: Most of the techniques of linear algebra
(e.g. matrix operations) can be justified / or will
have an analogous operation in a finite field.
So, a basic understanding of finite fields is
important for learning FEC techniques.

02/01/15

SCK, IIT Kharagpur

17

Introduction to Linear Algebra

Definition (Group): A group G is a set of
elements together with an operation on pairs of
elements in the set (denoted by * ) satisfying the
following four properties:
1. Closure: For every a and b in the set,
c = a * b is also in the set.
2. Associativety: For every a, b, c in the set
a * (b*c) = ( a * b) *c
02/01/15

SCK, IIT Kharagpur

18

Introduction to Linear Algebra

3. Identity: There is an element e in G called the
identity element that satisfies
a * e = e * a = a, for every a in the set.
4. Inverses: If a is in the set, then there is some
element b in the set called an inverse of a such
that, a * b = b * a = e, for all a & b in G.
Some groups satisfy the additional property of
commutativity, i.e. a * b = b * a, for any a & b
Such a group is called a commutative group or an
abelian group (after Niels Abel, 1802 1829)
02/01/15

SCK, IIT Kharagpur

19

Introduction to Linear Algebra

If G has finite no. of elements, it is a finite group.
Some conventional symbols for abelian groups:
as
*
+ (addition)
e
0 (Zero)
inverse of a
-a
[a * b = b * a = e a + (-a)=(-a)+ a =0 ]
02/01/15

SCK, IIT Kharagpur

20

Introduction to Linear Algebra

(OR) *
e

. (multiplication)
1 (one)

inverse of a

a-1

[ a * b = b * a = e a . a-1 = a-1.a = 1]
Theorem #1 (Group 1): In every group, the
identity element is unique. Also, the inverse of
each group element is unique, and, (a -1)-1= a
[OR, for + and - representation, - (-a) = a]
02/01/15

SCK, IIT Kharagpur

21

Introduction to Linear Algebra

Proof: Let, e & e be two possible identity elements
e = e * e = e * e = e. (proved)
Next, let b & b are the inverses of a.
a * b = e (by identity property).
Now, b = b *e = b * (a * b) = (b*a)* b (Associative)
= e * b = b
Lastly, a-1* a = a* a-1 = 1.
So, a is an inverse of a-1. As we just showed that
the inverses are unique, (a-1)-1= a.
02/01/15

SCK, IIT Kharagpur

22

Introduction to Linear Algebra

Some examples of Groups:
Infinite groups:
a)Integers under the operation of addition (+ ve, 0,
- ve; identity element : 0)
b)Positive rational numbers under the operation of
multiplication.
c) Set of 2 2 real valued matrices under matrix
addition [ Abelian group ? ]
02/01/15

SCK, IIT Kharagpur

23

Finite groups:
a) {0, 1} under EXOR operation
b) {0, 1, .., 8, 9} under modulo-10 addition.
Example of a finite non-abelian group:
transformation (rotation & reflection) of an
equilateral triangle:
Let us define the following six transformations:
1 = (ABC ABC) [no change]
a = (ABC BCA) [anti-clockwise rotation by120]
02/01/15

SCK, IIT Kharagpur

24

b = (ABC CAB)

[clockwise rotation by120]

c = (ABC ACB) [Reflection about bisector of <A]

d = (ABC CBA) [Reflection about bisector of <B]
e = (ABC BAC) [Reflection about bisector of <C]
Let, the group (G, *) be defined as,

G ={1, a, b, c, d, e}
& x * y is a group element that
denotes the transformation one

gets by performing sequentially first the

02/01/15

SCK, IIT Kharagpur

25

transformation denoted by y and then the

transformation denoted by x.
a*d
= (ABC BCA) * (ABC CBA)
= (ABC BAC) = e.
Now, verify that, d * a = c
Similarly, a*c c * a

02/01/15

SCK, IIT Kharagpur

26

Table for x * y:
x\y 1

[Note: every element appears once in each column & once in

each row. This always happens in a finite group]
02/01/15

SCK, IIT Kharagpur

27

Q: Do permutations on n letters form a group?

Subgroup: If G is a group & H is a subset of G,
then H is called a subgroup w.r.t the restriction of *
(operator) to H.
It implies that a non empty set H is a subgroup of
G if it is shown that the closure & inverse properties of
H are valid. The other properties of associativity &
identity are in herited from the group G.
02/01/15

SCK, IIT Kharagpur

28

Ex. In the set (G) of integers under addition, the set

of even integers (or the set of multiples of 3) is a
subgroup H.
To form a subgroup H of a finite group G:
Take

any element h from G & form a

sequence of elements

02/01/15

h, h* h,

h*h*h, h*h*h*h,

h3

h2

h4
SCK, IIT Kharagpur

29

Eventually, it may be found that, hc = 1; when c is

called the order of the element h.
Cyclic Group: A group that consists of all the
powers of one of its elements is called a cyclic
group.
Coset Decomposition of a finite group G:
Let H be a subgroup of G & its elements be
h1, h2, h3, .. with h1 as the identity element.
02/01/15

SCK, IIT Kharagpur

30

Now, construct an array as follows:

h1 = 1

h2

h3

h4

hn

g 2 * h1 = g 2

g 2 * h2

g2 * h 3

g2 * h4 g2 * hn.

g3 * h1 = g3

g 3 * h2

g3 * h 3

g3 *h4

g 3 * hn

..............................................................................................
gm * h1 = gm

g m * h2

gm * h 3

gm * h4 .. gm * hn

g2 : an element of G not considered earlier.

Stop when all the group elements appear somewhere in the
array; it has to stop as G is finite. The coset decomposition
is02/01/15
always rectangular.

SCK, IIT Kharagpur

31

Theorem #2: Every element of G appears once

& only once in a coset decomposition of G.
Proof: Every element appears at least once
(otherwise the process does not stop).
Now, suppose, two elements in the same row g i *
hj & gi * hk are equal. Then multiplying each with
gi1 gives hi = hk, which his contradictory. So, an
element can not occur twice in the same row.
02/01/15

SCK, IIT Kharagpur

32

Further, suppose two elements in different rows, g i

*hj and gk * hl, are equal and that k<i.
Now multiplying on the right by hj1. We get,
gi = gk * hl * hj1.
Then, gi is in the k th co-set because hl *hj1 is in
the subgroup. This is contradictory again. So, an
element cannot appear in two different rows.
Corollary: if H is a subgroup of G, then the no. of
elements in H divides the no. of elements in G.
02/01/15

SCK, IIT Kharagpur

33

That is,
(order of H).(No.of cosets of G w.r.t H)=(order of G)
Theorem #3: The order of a finite group is divisible
by the order of any of its elements.
RINGS:
Definition: A ring R is a set of elements with two
operations defined : the first is called addition (+) &
the second is called multiplication (denoted by
juxtaposition). Further, the following axioms are
satisfied:
02/01/15

SCK, IIT Kharagpur

34

(i) R is an abelian group under addition (+).

(ii) Closure: For any a, b in R, the product ab is in
R.
(iii) Associative law: a (bc) = (ab)c
(iv) Distributive law: a(b + c) = ab + ac
(b + c) a = ba + ca
Note: + operation is always commutative in a
Ring, but . may not be. If multiplication
operation is also commutative it is a
commutative ring.
02/01/15

SCK, IIT Kharagpur

35

Theorem #4: For any two elements a, b in a ring R,

(i) a 0 = 0 a = 0
(ii) a (-b) = (-a) b = -(ab)
Proof : (i) a 0 = a(0 + 0) = a 0 + a 0
a0 a0 = a.0 + a 0 a 0
or 0 = a 0.
Proof of (ii) is left as an exercise.

If a Ring has an identity element over multiplication, it is a

ring with identity, i.e. a = a.1 = 1.a, for all a
02/01/15

SCK, IIT Kharagpur

36

In R, every element has an inverse over +, but

may not have inverses over (.). However, in R
with identity, if a b = 1, then b is called as the
right inverse of a.
Theorem #5: In a ring with identity,
(i) The identity is unique
(ii) If an element a has both a right inverse b and a
left inverse c, then b = c.
02/01/15

SCK, IIT Kharagpur

37

In this case the element a is said to have an

inverse (denoted by a-1). The inverse in unique.
Proof: Same as Theorem #1, left as an exercise.
An element with an inverse is called a unit. The
set of all units is closed under multiplication (if a
& b are units, c = ab has inverse c1 = b-1 a-1)

02/01/15

SCK, IIT Kharagpur

38

Theorem #6
(i) Under ring multiplication, the set of units of a
ring forms a group.
(ii) If c = ab & c is a unit, then a has a right
inverse & b has a left inverse.
Some examples of rings:
(a) The set of all real numbers under the usual
addition & multiplication (commutative ring with identity,
every non zero element is a unit).

02/01/15

SCK, IIT Kharagpur

39

(b)The set of all integers ( +, -, 0) under usual + &

(commutative rig with identity; only units are 1)

(c) Set of all n n matrices with real elements

under matrix + & matrix (non commutative ring with
identity as the n n identity matrix).

(d) Set of all polynomials in x with real valued

coefficients under polynomial + & . (commutative
ring with identity; the identity is the zero-degree polynomial p(x) = 1)

02/01/15

SCK, IIT Kharagpur

40

FIELDS:
Definition: A field F is a set that has two operations
defined on its elements: addition and
multiplication, such that the following axioms are
satisfied:
(i) The set is an abelian group under addition.
(ii) The field is closed under multiplication and the
set of non zero (i.e. except the identity element of
addition) elements is an abelian group under
multiplication.
02/01/15

SCK, IIT Kharagpur

41

(iii) The distributive law (a + b) c = ac + bc holds for

all a, b, c in the field.
Convention: a b = a + (-b) ; a/b = b-1a

Examples of fields:
(a) R: the set of real numbers
(b) C : the set of complex numbers
(c) Q: the set of rational numbers.
A field with q elements, if it exists, is called a finite field
or Galois Field, GF(q), (after Evariste Galois)
02/01/15

SCK, IIT Kharagpur

42

The smallest field: GF(2) : {0, 1},

+

Subtraction & division are defined implicitly in a field by the

addition & multiplication tables.
02/01/15

SCK, IIT Kharagpur

43

GF (3) = {0, 1, 2} with the following operations:

+

0
1
2

0
1
2

1
2
0

2
0
1

0
1
2

0
0
0

0
1
2

0
2
1

Definition: Let F be a field. A subset of F is called a

subfield if it is a field under the inherited addition &
multiplication. The original field F is then called an
extension field of the subfield.
02/01/15

SCK, IIT Kharagpur

44

Theorem #7: In any field, if ab = ac & a 0, Then b = c.

Proof: (multiply by a-1)
GALOIS FIELD (Properties, computations,
polynomials)
Our study of Galois field is based on (a) integer
ring and (b) polynomial ring.

02/01/15

SCK, IIT Kharagpur

45

The Integer Ring: As noted earlier, the set of

integers ( + ve, - ve & zero) forms a ring under usual
operations of addition & multiplication.
Let Z : The integer ring.
Though division is not possible in general in the ring
of integers Z, the concept of division is present in
terms of two weaker operations of cancellation and
division with remainder.
02/01/15

SCK, IIT Kharagpur

46

Definition: An integral domain is a commutative ring

in which b = c whenever ab = ac and a is nonzero.
[i.e. the cancellation property is valid].

Note that, Z is an integral domain.

Now, we say, that the integer s is divisible by the
integer r (or r divides s or, r is a factor of s) if,
ra = s, for some integer a.

02/01/15

SCK, IIT Kharagpur

47

 Prime integer: A + ve integer p > 1 that is divisible

only by p or 1.
Composite: nonprime integer >1.
GCD (r,s): Greatest common division of r & s is the
largest positive integer that divides both of them.
LCM (r, s): Least common multiple of two integers
r & s is the smallest positive integer that is divisible
by both of them.
Two integers are relatively prime if their GCD is 1.
02/01/15

SCK, IIT Kharagpur

48

Theorem #8 [Division Algorithm]:

For every pair of integers c & d with d
nonzero, there is a unique pair of integers Q (the
quotient) and s (the remainder) such that, c = dQ +
s, where 0 s < |d|.
Usually, the remainder s is of more interest
than the quotient Q.
s = Rd[c] s is the remainder or residue of c
when divided by d.
02/01/15

SCK, IIT Kharagpur

49

Another notation:
s c (mod d) s is congruent to c, modulo-d.
In this case, s need not be less than d.
The following theorem greatly simplifies the operations
needed to find the remainder.

Theorem #9 [On Remainder]

(i) Rd [ a + b] = Rd {Rd [a] + Rd[b]}
(ii)Rd [ a.b] = Rd {Rd [a]. Rd [b]}
Ex: Let, a = 100, b = 15; d = 3. Then, R3 [115] =1 and R3 [1500]= 0
02/01/15

SCK, IIT Kharagpur

50

Using the division algorithm, we can find the GCD of two

integers:

Ex: To find GCD (814, 187)

814 = 4 187+ 66

[As the GCD divides both 814 & 187; it

also divides 66]

187 = 2 66 + 55

[Same argument & hence the GCD also

divides 55]

66 = 1 55 + 11

[Same argument & hence the GCD also

divides 55]

02/01/15

SCK, IIT Kharagpur

51

55 = 5 11 + 0 [ 11 divides 55 & hence 66 & hence 187 & finally 814].

It is interesting to note:
11 = GCD = 66 1 55
= 66 1 (187 2 66) = 3 66 1 187
= 3 (814 4 187) 1 187
= 3 814 13 187
The GCD can be expressed as a linear combination of 814
& 187 with coefficients from the integer ring.

02/01/15

SCK, IIT Kharagpur

52

Theorem #10(Euclidean Algorithm for integers):

Given two distinct nonzero integers r and s,
their GCD can be computed by an iterative
application of the division algorithm. Suppose that
r<s and both are positive; then the algorithm is,
s = Q1r + r1
r = Q2r1 + r2
r1 = Q3 r2 + r3
...

02/01/15

SCK, IIT Kharagpur

53

rn1 = Qn+1 rn, where the process stops when a

remainder of zero is obtained. The last nonzero
remainder, rn, is the GCD.
Corollary (Euclidean algorithm for integer
ring):
For any integers r and s, there exist
integers a and b such that
GCD (r, s) = a.r + b.s
02/01/15

SCK, IIT Kharagpur

54

Proof: The last remainder rn is the GCD (r,s). Using

the set of equations, eliminate all other remainders.
This results in a linear combination of r and s with
integer coefficients.
Problems: (a) Use the Euclidean algorithm to find GCD (157,
308).
(b) Find integers A and B that satisfy GCD (1573, 308) = 1573
A + 308B.

02/01/15

SCK, IIT Kharagpur

55

Definition (Quotient Ring):

Let q be a positive integer. The quotient ring,
called the ring of integers modulo q and denoted by
Z/(q), is the set { 0, 1, ., q1} with addition and
multiplication defined by,
a + b = Rq [a + b]
a .b = Rq [ab]
Elements 0, .., q 1 appear in both Z & Z/(q).
02/01/15

SCK, IIT Kharagpur

56

Note: Two elements a & b of Z that map in to the same

element of Z/(q) ore congruent modulo q, and a + b = m.q,
where m is an integer.

Theorem #11 (Finite Field over integer ring):

The quotient ring Z/(q) is a field if and only if
q is a prime integer.
Proof: Let q be a prime. To prove that the ring Z/(q)
is a field, we must show that every nonzero element
has a multiplicative inverse.
02/01/15

SCK, IIT Kharagpur

57

Let s be a zero element of the ring. Then 1

s q -1
Now, as q is a prime, GCD (s,q) = 1.
1 = aq + bs for some integers a & b.
1 = Rq[1] = Rq [aq + bs]
= Rq {Rq[aq] + Rq[bs]}
= Rq {0 + Rq[bs]} = Rq[bs]
= Rq{Rq[b] . Rq[s]}
= Rq{Rq[b] . s} [ 1SCK,
IITs Kharagpur
q 1]

02/01/15

58

 We see that s has a multiplicative inverse Rq[b]

under modulo q multiplication.
Now, let q be a composite & q = rs.
In this case, if the ring Z/(q) is a field, then the
element r has a multiplicative inverse r -1.
Hence, s = Rq[s] = Rq [r 1 rs] = Rq[r 1 q] = 0
But, s 0, & hence there is a contradiction.
So, the ring is not a field if q is a composite.
If Z/(q) is a field, it is also denoted as GF(q).
02/01/15

SCK, IIT Kharagpur

59

Polynomial Rings:
The mathematical expression
f(x) = fn-1 xn-1 + fn-2 xn-2 + + f1x + f0 is called a
polynomial over a field GF(q) if the coefficients f n-1,
fn-2, ., f1, f0 are elements of GF(q) (called scalars)
and the indices and exponents are integers.
x is an indeterminate here.
The zero polynomial is f(x) = 0.
02/01/15

SCK, IIT Kharagpur

60

A monic polynomial is a polynomial with the

leading coefficient fn-1 = 1.

Two polynomials are equal if fis are equal for
each i.
The degree of a polynomial f(x), denoted as deg
f(x) is the index of the leading coefficient f n-1 (or, the
exponent of the leading indeterminate x)
By convention, deg. of the zero polynomial is -
and the zero degree polynomial is f(x) = f 0.
02/01/15

SCK, IIT Kharagpur

61

 The set of all polynomials over GF(q) forms a

ring if addition and multiplication are defined as
usual addition & multiplication of polynomials.
This ring is denoted as GF(q)[x]
Addition: The sum of two polynomials f(x) and g(x)
in GF(q)[x] is another polynomial in the same ring.

f x g x fi gi xi
i 0

, indices larger then the

higher degree of f(x) & g(x) are all zeros.

02/01/15

SCK, IIT Kharagpur

62

Ex: over GF(2):

x3 x 2 1 x 2 1 x3 x 2 x 2 1 1 x3

Multiplication:

f x g x

f
g
x
;2

j i j
j 0

j 0

j 0

j 0

f j g0 j x f j g1 j x f j g 2 j n f j gi j xi
0

j 0

f 0 g 0 f 0 g1 f1g0 x f0 g 2 f1g1 f 2 g 0 x 2 ....... f j g i j x i

j 0

02/01/15

SCK, IIT Kharagpur

63

Ex;. Over GF(2):

x3 x 2 1 x 2 x 1 x5 x 1

A polynomial s(x) is said to be divisible by the

polynomial r(x) if s(x) = r(x).a(x) where a(x) is a

polynomial.

Irreducible Polynomial: A polynomial p(x) that

is divisible only by .p(x) or where is an

arbitrary field element in GF(q).
02/01/15

SCK, IIT Kharagpur

64

Prime polynomial: A monic irreducible

polynomial of degree at least 1.

GCD [r(x), s(x)] is the monic polynomial of the

largest degree that divides both r(x) and s(x).

LCM [r(x), s(x)] is the monic polynomial of the

smallest degree that is divisible by both r(x) and

s(x).

Two polynomials are said to be relatively prime

if their GCD is 1.
02/01/15

SCK, IIT Kharagpur

65

If r(x) divides s(x) and is also divisible by s(x),

then, r(x) = s(x), where is a field element of

GF(q).
[Let r(x) = s(x).a(x) & s(x) = r(x).b(x)
r(x) = r(x).b(x).a(x)
Now, equating the degrees of both sides, we see
that the b(x).a(x) is a zero degree polynomial i.e.
they are scales.]
02/01/15

SCK, IIT Kharagpur

66

Formal derivative of a polynomial:

Defn: If r(x) = rn-1 xn-1 + rn-2 xn-2 + + r1x + r0 is a
polynomial over GF(q), its formal derivative r(x) is a
polynomial given by,
r(x) =((n 1)) rn-1 xn-2 + ((n-2)rn-2 xn-3 + + r1,
where the coefficients ((i)) are called integers of the
field GF(q) and are given by,
((i)) = 1 + 1 + + 1 in GF(q) of i terms [i.e.
reduce the number to modulo q]
02/01/15

SCK, IIT Kharagpur

67

It may be noted that the following property holds for

formal derivative of polynomials:
[r(x) s(x)] = rs(x) + r(x) s(x).
Division Algorithm for Polynomials:
Theorem #12 [Division Algorithm for polynomials]:
For every pair of polynomials c(x) and d(x) with d(x)
0, there is a unique pair of polynomials Q(x), the
quotient polynomial, and s(x), the remainder
02/01/15

SCK, IIT Kharagpur

68

s(x), such that,

c(x) = d(x).Q(x) + s(x)
And deg. s(x) < deg. d(x)
Proof: A quotient polynomial and a remainder
polynomial can be found by long division of
polynomials.
Now to show their uniqueness let,
c(x) = d(x).Q1(x) + s1(x) = d(x).Q2 (x) + s2(x).
02/01/15

SCK, IIT Kharagpur

69

Then,

d x Q1 x Q2 x
1 4 4 44 2 4 4 4 43
If non zero, deg is deg d(x)

s2 x s1 x
1 44 2 4 43

If non zero, degree < deg d(x)

Hence, contradiction

Following our previous notation, the remainder or

residue polynomial s(x) is written as,
s(x) = Rd(x) [c(x)]
02/01/15

SCK, IIT Kharagpur

70

The congruence:
s(x) = c(x) (mod d(x)) [i.e. both s(x) & c(x)
have common remainder when divided by d(x)]
The following theorem helps in finding the
remainder polynomial:
Theorem #13 [On remainder polynomial]:
If d(x) is a multiple of g(x), then for any a(x),
Rg(x) [a(x)] = Rg(x) [Rd(x)[a(x)]]
02/01/15

SCK, IIT Kharagpur

71

Proof: Let, d(x) = g(x).h(x) for some h(x).

Now, a(x) = Q1(x) d(x) + Rd(x) [a(x)]
= Q1(x) g(x) h(x) + Rd(x) [a(x)] (1)
Now, the remainder polynomial can be expressed
as, [when divided by g(x)]

Rd x a x Q2 x g x

Rg x Rd x a x
1 4 4 4 2 4 4 43

Degree less than that of g(x)

.. (2)
02/01/15

SCK, IIT Kharagpur

72

Again, if a(x) is directly divided by g(x), we can

write,
A(x) = Q(x) + R g(x) [a(x)]

.. (3)

Now, the division algorithm says that the remainder

polynomial of degree less than that of g(x) is unique
& from equation (2) & (3), we get,
Rg(x) [a(x)] = Rg(x) [Rd(x) [a(x)]]
02/01/15

SCK, IIT Kharagpur

73

Theorem #14 [on properties of remainder

polynomial]:
(i) Rd(x) [a(x) +b(x)] = Rd(x) [a(x)] + Rd(x) [b(x)]
(ii) Rd(x) [a(x) . b(x)] = Rd(x) {Rd(x) [a(x)] . Rd(x) [b(x)]}
Proof: (Use division algorithm on both sides & equate the
remainder)

02/01/15

SCK, IIT Kharagpur

74

Theorem #15 [Euclidean Algorithm for Polynomials]:

Given two polynomials r(x) & s(x) over GF(q), their
GCD can be computed by iterative application of the
division algorithm. If deg. s(x) deg. r(x) 0, this means,
s(x) = Q1(x) r(x) + r1(x)
r(x) = Q2(x) r1(x) + r2(x)
r1(x) = Q3(x) r2(x) + r3(x)
..
rn-2(x) = Qn(x) rn-1(x) + rn(x)
rn-1(x) = Qn+1 (x) rn(x)
where the process stops when a zero remainder is obtained.
02/01/15

SCK, IIT Kharagpur

75

Then, rn(x) = .GCD [r(x), s(x)], where is a scalar.

Proof: (Similar to the proof of the theorem for integers)
As, by definition, GCD [r(x),s(x)] divides both the
dividend & divisor, it divides the remainder r1(x). By extending
the same argument, GCD[r(x),s(x)] divides rn(x).
Now, starting with the bottom equation: rn(x) divides
the divisor and remainder and then divides the dividend.
{Note: rn(x) divides rn-1(x), rn-2(x),, r(x) and s(x). So, rn(x) also divides
the GCD[r(x),s(x)] }

02/01/15

SCK, IIT Kharagpur

76

As rn(x) both divides and is divided by

GCD[r(x),s(x)], the theorem follows.
Corollary to Euclidean Algorithm:
GCD [r(x), s(x)] = a(x).r(x) + b(x).s(x) where
a(x) & b(x) are polynomials over GF(q).
Proof: Can be obtained easily be eliminating the intermediate
remainder polynomials from the Euclidean algorithm.
02/01/15

SCK, IIT Kharagpur

77

Problems:
1 Over GF(2), let p1(x) = x3 +1, and let p2(x) = x4 + x3 + x2+1
(a)

Find GCD [p1(x), p2(x)]

(b)

Find A(x) and B (x) that satisfy

GCD [p1(x), p2(x)] = A(x) p1(x) + B(x)p2(x)

2 (a) How many distinct second-degree monic polynomials
of the form x2 + ax + b, b 0 are there over GF(16) ?

02/01/15

SCK, IIT Kharagpur

78

3. (b) How many distinct polynomials of the form (x -)

(x - ), , 0 are there over GF(16)?
(c) How many second degree prime polynomials
are there over GF(16)?
4. How many distinct monic polynomials over
GF(2) divide x6 1 ?
5. Construct GF(5) by constructing an addition
table and a multiplication table.
6. Construct addition and multiplication tables for GF(8) and
GF(9)
02/01/15

SCK, IIT Kharagpur

79

Zero of a polynomial:
If p(x) is a polynomial over GF(q), is an element of
GF(q) and if p() = 0, then is called a zero of the
polynomial p(x) or a root of the equation p(x) = 0.
A polynomial need not have a zero in its own field

Theorem #16 [On zero of a polynomial]:

A polynomial p(x) has field element as a
zero if and only if (x - ) is a factor of p(x). Further,
at most n field elements are zeros of a polynomial
p(x) of degree n.

02/01/15

SCK, IIT Kharagpur

80

Proof: Using the division algorithm, p(x) = (x - ) Q(x) + s(x),

where the deg. s(x) <1.
That is, s(x) is a field element, say, s0.
Hence,
0= p() = ( - ) Q()+s0 & hence, s(x)= s0 = 0
Conversely, if (x -) is a factor, then p(x) = (x - ) Q(x) and p() =
( - ) Q() = 0 & thus is a zero of p(x).
Now, factorize p(x) is a field element & prime polynomials. The
deg of p(x) = sum of the degrees of the prime factors and one
such prime factor exists for each zero.
Hence, there are at most n zeros.

02/01/15

SCK, IIT Kharagpur

81

Finite fields based on Polynomial Rings:

Construction is somewhat similar to what has been
discussed in the context of integer rings.
Definition [Ring of polynomials modulo p(x)]:

For any monic polynomial p(x) with non zero

degree over a field F, the ring of polynomials modulo
p(x) is the set of all polynomials with degree smaller
than that of p(x), together with polynomial addition
and polynomial multiplication modulo p(x).
This ring is conventionally denoted as:
F[x]/p(x)

02/01/15

SCK, IIT Kharagpur

82

Any element r(x) of F[x] (Here F[x] is the ring of

polynomials defined over field F) can be mapped
into the ring of polynomials modulo p(x)
i.e. F[x]/p(x) as follows:
r(x) Rp(x)[r(x)]
Two elements a(x) & b(x) of F[x]/p(x) are called
congruent:
a(x) b(x) [modulo p(x)]
In such a case,
b(x) = a(x) + Q(x)p(x), for some polynomial
Q(x)
02/01/15

SCK, IIT Kharagpur

83

Example: Let,

p(x) = x3 + 1

Then,GF(2) [x]/p(x) = GF(2)[x]/(x3 + 1) has the

following elements:
{0, 1, x, x + 1, x2, x2 + 1, x2 + x, x2 + x + 1}
Ex. of multiplication:
(x2 + 1). x2 = R x + 1[(x2 + 1). x2]
3

= R x + 1[x.(x3 + 1)+x2+x] = x2+x

3

02/01/15

SCK, IIT Kharagpur

84

Theorem #17 [Finite field over polynomial

Ring]:
The ring of polynomials modulo a monic
polynomial p(x) is a field if and only if p(x) is a
prime polynomial.
Proof: Let, p(x) be a prime polynomial. To prove that
the ring is a field, we have to establish that every non
zero element has a multiplicative inverse.
02/01/15

SCK, IIT Kharagpur

85

Let, s(x) be a non zero element of the ring.

Then, deg. s(x) < deg. p(x).
Further, GCD[s(x), p(x)] = 1 and hence,
1 = a(x).p(x) + b(x).s(x), for some polynomials a(x) and
b(x).
1 = Rp(x) [1] = Rp(x) [a(x) p(x) + b(x)s(x)]
= Rp(x) [Rp(x) [b(x)]. Rp(x) [s(x)]]
= Rp(x) [Rp(x) [b(x)].s(x)]
= (Rp(x) [b(x)]).s(x) [mod p(x)]
We see that Rp(x) [b(x)] is a multiplicative inverse of s(x)
in
the ring of polynomialsSCK,
modulo
p(x).
02/01/15
IIT Kharagpur
86

Now, suppose that p(x) [whose degree is at least

2, as lower deg. polynomials are prime] is not prime.
Then,
p(x) = r(x).s(x), say for some r(x) and s(x), each
of deg. at least 1. & defined over the ring of polynomial.
If the ring is a field, then r(x) has an inverse polynomial r -1(x).
Hence,
s(x) = Rp(x)[s(x)] = Rp(x)[r-1(x). r(x). s(x)] = Rp(x)[r-1(x). p(x)] = 0

But s(x) 0 & hence a contradiction. So, the polynomial

ring is not a field if p(x) is not a prime polynomial.
02/01/15

SCK, IIT Kharagpur

87

we can construct a finite field [GF(qn)], whose

elements are represented by polynomials over
GF(q) of degree less than n.
What is the no of elements of the finite field so constructed?

Ans: [GF(qn)].
Ex: Construct GF(23) from GF(2) using the prime
polynomial p(x) = x3 + x + 1.

(Hint. Identify the elements &

construct the tables for addition & multiplication).

02/01/15

SCK, IIT Kharagpur

88

Primitive Field Element

Definition :
A primitive field element of GF(q) is an
element such that every field element
(except zero) can be expressed as a power of
.
Ex: In GF(5), 21 = 2, 22 = 4, 23 = 3, 24 = 1.

Primitive elements are very useful for

constructing fields because, if one primitive
element is found, the multiplication table can
be constructed easily.
02/01/15

SCK, IIT Kharagpur

89

Theorem #18 [On Field Elements]

Let, 1, 2, ., q

denote the non zero field

elements of GF(q). Then,

xq-1 1 = (x - 1) (x - 2) (x - q-1)
Proof: The set of non zero elements of GF(q) is a
finite group under the operation of multiplication.
Now, let be any non zero element of GF(q) and
h is its order under multiplication, i.e.
h = 1.
02/01/15

SCK, IIT Kharagpur

90

Now, remembering that the no of elements in a

finite group (i.e. the order of the group) is divisible
by the order of any element of the group, we can
say that h divides (q 1).

q-1= (h) (q-1)/h =1

So, is a zero of the polynomial (xq-1 1) and

hence, considering all i s,
xq-1 1 = (x - 1) (x - 2) . (x - q-1)

02/01/15

SCK, IIT Kharagpur

91

Theorem #19 [Cyclic group property of nonzero elements of GF(q)]:

The group of non-zero elements of GF(q)
under multiplication is a cyclic group (Remember: A
cyclic group contains all the powers of one of its elements).
Proof: Omitted. However, we note that, if (q 1) is a prime, the proof is
obvious as every element, except 1, has order (q1). So, every element
is primitive, if (q 1) is a prime.

02/01/15

SCK, IIT Kharagpur

92

Theorem #20 [ Existence of a primitive element in

GF(q)]:
Every Galois field has a primitive element.
Proof: The previous theorem stated that the non-zero
elements of GF(q) form a cyclic group (i.e. all powers of an
element of the group exist).
If is that element, then, the non zero elements of
GF(q) can be expressed as, , 2, 3, 4, .., q-1, [as there
are (q 1) elements].
So, there is one element whose order is q 1, i.e. q-1
= 1. Thus is the primitive element of GF(q).
02/01/15

SCK, IIT Kharagpur

93

The order of every non zero element of GF(q)

divides (q 1).
When constructing an extension field as
a set of polynomials, it is usually convenient if
the polynomial p(x) corresponds to a primitive
element of the field for x = .
This is done by choosing a special polynomial
the primitive polynomial.

02/01/15

SCK, IIT Kharagpur

94

Definition [Primitive polynomial]:

A primitive polynomial p(x) over GF(q) is a
prime polynomial over GF(q) with the property that
in the extension field constructed modulo p(x), the
field element represented by x is primitive.
Primitive polynomial of every degree exists
over every GF.
A primitive polynomial is a prime polynomial
having the primitive element as its zero.
02/01/15

SCK, IIT Kharagpur

95

THE STRUCTURE OF FINITE FIELDS:

Definition (Characteristic): The number of elements
in the smallest subfield of GF(q) is called the
characteristic of GF(q).
Theorem #21 [on the characteristic of a Galois
Field]:
Each Galois field contains a unique smallest
subfield, which has a prime number of elements.
02/01/15

SCK, IIT Kharagpur

96

Proof: (Omit)The field contains 0 & 1. To define the

sub field, consider the subset G = {0, 1, 1+1, 1 +
1+1, ..} denoting these by {0, 1, 2, 3, }. This is
a cyclic sub group under addition and it must
contain a finite, say p no. of elements.
We have to show that p is a prime & G =
GF(p). In G, + is modulo p (as it is a cyclic gr.
Under +). Further, . = ( 1 + 1 + .+ 1) = +
.. + , where there are copies of in the sum.
02/01/15

SCK, IIT Kharagpur

97

Hence, multiplication is also modulo p.

Moreover, each element has an inverse
under . Because, the sequence , 2 , 3 , ., is
a cyclic subgroup of G. It contains 1 so that = 1
for some in G.
Thus, the subset G contains the identity
element, is closed under addition & multiplication
and contains all inverse under + & ..
02/01/15

SCK, IIT Kharagpur

98

Hence, it is a subfield & has mod. p arithmetic.

From our theorem on the quotient ring we say that
p has to be a prime.
Definition (minimal polynomial):
If GF(Q) is an extension field of GF(q) and
is an element of GF(Q), then the prime polynomial
f(x) of the smallest degree over GF(q) with f() = 0
is called the minimal polynomial of over GF(q).
02/01/15

SCK, IIT Kharagpur

99

Theorem #22 [On minimal polynomial]:

Every element of GF(Q) has a unique
minimal polynomial over GF(q).
Further, if has the minimal polynomial f(x)
and a polynomial g(x) has as a zero, then f(x)
divides g(x).
Proof: (Omit) We note that, is always a zero of
(xQ x), which is a polynomial over GF(q).
02/01/15

SCK, IIT Kharagpur

100

Now, with the help of unique factorization theorem,

xQ x = f1(x). f2(x).. fk(x).
Now, if is a zero of the left side, then it must be a
zero of some term on the right side (of only one term,
because, over the extension field GF(Q), the prime terms
can be further factored into linear and constant term).
To prove the second part of the theorem, let us write,
g(x) = f(x).h(x) + s(x), where deg. s(x) < deg. f(x) and hence
s(x) can not have as a zero.
But, 0 = g() = f().h() + s() = s().
s(x) must be zero and the theorem is proved.
02/01/15

SCK, IIT Kharagpur

101

Theorem #23 [on the expression of a field

element]:
Let be a primitive element in GF(Q), an
extension field of GF(q) and m be the degree of
f(x), the minimal polynomial of over GF(q). Then,
the number of elements in the field GF(Q) is, Q = q m
and each element can be written as,
= am-1 m-1 + am-2 m-2 + + a1 + a0, where, am1

, am-2, ., a1, a0 are elements of GF(q).

02/01/15

SCK, IIT Kharagpur

102

Proof: We note that any element may be written

in the form
= am-1 m-1 + am-2 m-2 + . + a1 + a0, is an
element of GF(Q) [as is the primitive element]
Further, this expression is unique; because if
= bm-1 m-1 + bm-2 m-2 + . + b1 + b0, then,
0 = (am-1 bm-1)

m-1

+ (am-2 bm-2) m-2 +

+ (a1 b1) + (a0 b0) & hence, is a zero of a

polynomial of degree (m 1)
02/01/15

SCK, IIT Kharagpur

103

This is contrary to the definition of n. As there are

qm such , Q is at least as large as qm.
On the other hand, we know that every non
zero field element can be expressed as a power of
. However if f(x) is the minimal polynomial of ,
f() = 0. hence,
m + fm-1 m-1 + . + f1 + f0 = 0
Or,
02/01/15

m = - fm-1 m-1 - . f1 - f0.

SCK, IIT Kharagpur

104

Using this relationship, any power of can be

reduced to a linear combination of ( m-1, m-2, ..,
, 0).
For example,
m+1 = m.
= -fm-1. m -fm-2 m-1 - - f1 2 f0
= -fm-1(-fm-1 m-1 - fm-2 m-2 - .. - f1 - f0
= - fm-2 m-1 - fm-3 m-2 - . f1 2 f0
02/01/15

SCK, IIT Kharagpur

105

Hence, every element of GF(Q) can be expressed

as a district linear combination of

m-1

m-2

, ., 0.

So, Q is not larger than qm & the theorem is proved.

Corollary (on the no. of elements):
Every Galois field has pm elements for some
positive integer m & prime p.
Proof: Every GF has a subfield with p elements to
which the previous theorem applies.
02/01/15

SCK, IIT Kharagpur

106

The previous theorem helps us to associate a

polynomial of deg (m-1) with each field element
simply by replacing by x.
These polynomials may be regarded as field
elements; may be added & multiplied modulo the
minimal polynomial f(x).
This is just the field we would obtain from the
theorem on finite fields over polynomial ring using
f(x) as the prime polynomial.
02/01/15

SCK, IIT Kharagpur

107

So,

each

GF

can

be

constructed

by

polynomial arithmetic modulo a prime polynomial

and the no of elements is a prime power.
The following theorem establishes that for every
prime p and positive integer m, there is a GF of
polynomials.

02/01/15

SCK, IIT Kharagpur

108

Theorem #24 (Algebraic property of the field

elements in term f the characteristic):

Let, GF(q) have characteristic p. Then for

any positive integer m and for any elements
and in GF(q),
( )pm = pm pm
Proof: Suppose the theorem is true for m = 1. Then
( )p = p p
02/01/15

SCK, IIT Kharagpur

109

This can be raised to the p th power.

(( )p)p = (p p)p = p2 p2
Repeating this (m 1) times, we get,
( )pm = pm pm
So, it is necessary to prove the theorem for m = 1.
Now, by binomial theorem,

02/01/15

Ci
p

p i

i 0

SCK, IIT Kharagpur

110

Now,

Ci

p
i p i

is a prime number.

i p i

p. p 1

or 1 i (p 1) and p

So, the denominator divides p 1

for each i

(1 i (p 1)) . hence pCi is a multiple of p.

pCi = 0 (mod p) for 1 i (p 1).
In GF(q), when the integer arithmetic is
modulo p,

02/01/15

SCK, IIT Kharagpur

111

Ci
p

p i

i 0

.
0

p 0

0 0 .... .
p

Now, if p = 2, 2= 2 = - 2 . & if p is an odd

prime,
p
p
p
p
p & hence, in general,

p

02/01/15

pm

pm

pm

SCK, IIT Kharagpur

112

Theorem #25 (On the no. of elements of the smallest

splitting field):

Let m be a positive integer and p a prime.

Then the smallest splitting field of the polynomial
g(x) = xpm - x regarded as a polynomial over
GF(p) and has pm elements.

02/01/15

SCK, IIT Kharagpur

113

Corollary (Existence of a GF for every m & p):

For every prime p & positive integer m, there
is a Galois field with pm elements.
Finally, even if q is not a prime but a prime power
(say, pn) then GF (qm) can be constructed as an
extension field of GF(q).

02/01/15

SCK, IIT Kharagpur

114

The Chinese Remainder Theorems [Fast

Algorithm]:
It is possible to uniquely determine a nonnegative integer given only its remainders module
residues with respect to modules of each of several
integers, provided that the integer is known to be
smaller than the product of the modulation
Chinese remainder theorem.
02/01/15

SCK, IIT Kharagpur

115

Explanation: Let the nodulation be, m 0 = 3, m1 = 4

& m2 = 5 and let .
k

M mi m0 m1m2 60
i 0

Given an integer C, let, Ci = Rmi [c]

The Chinese remainder theorem says that there is
a one-to-one map between the to possible values
of C and the sixty values that the vector of
residues (C0, C1, C2) can take on.
02/01/15

SCK, IIT Kharagpur

116

0 C1 < 3, 0 C2 <4
0 C3 <5
Suppose, C0 = 2, C1 = 1 & C2 = 2
For C0 =2, the values of C that we should consider
are,{2, 5, 8, 11, 14, 17, 20, 23, 26, ...}
Similarly for C1 = 1, the considerable values of C
are { 1, 5, 9, 13, 17, 21,25,29,..}& for C2 = 2,
{2,7,12,17,22,27,32, ..}
We
unique
soli. For C is 17.
02/01/15observe that the SCK,
IIT Kharagpur

117

First theorem for integer ring (1st Chinese

Remainder Theorem) :
Given a set of integers m0, m1, .., mk that
are pair wise relatively prime and a set of integers
C0, C1, ., Ck with Ci <mi, then the system of
equations
Ci = c(mod mi), i = 0, , k has almost one
solution for C in the interval
02/01/15

SCK, IIT Kharagpur

118

0 C mi
i 0

Proof: Suppose that c & C are solutions in this

interval. Then

&

C = Q im i + C i

0ik

C = Qimi +Ci

0ik

So, (C - C) is a multiple of mi for each i. Then, (C

k

- C) is also a multiple of mi as mi-s are relatively

prime.
02/01/15

i 0

SCK, IIT Kharagpur

119

But, by defu.
k

i 0

i 0

mi C C mi

So, the only possibility is that, C - C = 0 &

hence (C = C)

(Proved)

The second, accompanying Chinese Remainder

theorem shows a method to find the solution of the
system of congruences, i.e.
Ci = Rmi [C] = C(mod mi)
02/01/15

SCK, IIT Kharagpur

120

This is done using the corollary of the Euclidean

algorithm.
k

Let us define M mr and M i M .

m
r 0

Then, we note that, GCD (Mi, mi) = 1.

So, using the corollary of Euclidean algorithm, we
can say that these exist integers Ni & ni (for each i)
with NiMi + nimi = 1,
02/01/15

i = 0, ., k.
SCK, IIT Kharagpur

121

28. (Second) Chinese Remainder Theorem for

integers [Th. 2.8.2; Bla85]:
k

Let M m be a product of relatively prime

r
r 0

integers; let,

Mi M

mi

and for each i, Let N i satisfy

NiMi + nimi = 1. Then system of congruences

Ci = C(mod. mi), i = 0, , k is uniquely solved by
k

C Ci N i M i mod M
i 0

02/01/15

SCK, IIT Kharagpur

122

Proof: The uniqueness of a solution has already

been proved by the previous theorem. So, it is
sufficient to show that this C solves the specified
system of congruence.
Now,
k

C Cr N r M r C0 N 0 M 0 C1N1M 1 ....... Cr N r M r
r 0

We note that mi divides Mr, for r i.

02/01/15

SCK, IIT Kharagpur

123

So,

C Cr N r M r mod M
r 0

Ci Ni M i mod mi

Further, as N M n m 1 , N M 1 mod m
i i
i
i i
i i
[Hence proved]
C = Ci ( mod mi)
Considering the previous example:
M = 60, M0 = 20, M1 = 15 & M2 = 12.
Further,
02/01/15

1 = (-1) M0 + 7m0
1 = (-1) M1 + 4 m1

SCK, IIT Kharagpur

124

&

1 = (-2) M2 + 5 m2

N0M0 = -20, NiMi = -15, N2M2 = -24.

C = -20 C0 15 C1 24 C2 (mod to)
If in particular, C0 = 2, C1 = 1 & C2 = 2,
C = -103 (mod to)
= 17.
To find C = ab,
Let, for each i, ai = Rmi [a], bi = Rmi [b] & ci = Rmi[c]
Then, for 0 i k,
C
i = aibi (mod mi)
02/01/15
SCK, IIT Kharagpur
125

Chinese Remainder Theorems over Ring of

polynomials:
First theorem [Th. 2.8.3]: Given a set of polynomials
m(0)(x), m(1)(x),., m(k)(x) that are pair wise relatively
prime and a set of polynomials C(0)(x), C(1)(x), ., C(k)
(x) with deg C(i)(x) deg. M(i) (x), then the system of
equations
C(i)(x) = C(x) (mod m(i)(x)),

i = 0, , k. has at

most one solution for C(x) satisfying

02/01/15

SCK, IIT Kharagpur

126

i
deg C ( x ) deg m x
i 0

Proof: Similar to the analogous theorem over

integrity.
Let C(x) & C(x) be two solution.
C(x) = Q(i)(x) m(i)(x) + C(i)(x)
&

C(x) = Q(i)(x) m(i)(x) + C(i)(x)

k

So, C(x) - C(x) is a multiple of m i x . But, deg

i 0
C(x) - C(x) is less then the degree
02/01/15

SCK, IIT Kharagpur

127

of m i x . So, C(x) - C(x) = 0

[Hence proved]

i 0

Second Theorem:
k

Let, M x m r x be a product of relatively

r 0

prime polynomial; let, and N(i)(x) satisfy

N(i)(x) M(i)(x) + n(i)(x) m(i)(x) = 1. Then the system of
congruences C(i)(x) = C(x) (mod m(i)(x)) i = 0, ., k
is uniquely solved by
02/01/15

SCK, IIT Kharagpur

128

i
i
i
C x C x N x M x mod M x
i 0

Proof: The uniqueness is already proved by the

previous theorem.
So, we have to show that C(x) satisfies every
congruence.
For this, we observe in a similar way as before,
C(x) = C(i)(x) N(i)(x) M(i)(x) (mod m(i)(x)).
Because, M(r)(x) has m(i)(x) as a factor for r 1.
02/01/15

SCK, IIT Kharagpur

129

Further, N(i)(x) M(i)(x) + n(i)(x) m(i)(x) = 1

So, N(i)(x) M(i)(x) = 1 (mod m(i)(x)).
C(x) = C(i)(x) (mod m(i) (x)) for 0 i k
[Hence proved].
BCH CODES
A large class of Multiple over connecting linear,
Relock, cyclic codes.
02/01/15

SCK, IIT Kharagpur

130

Attractive because, (i) good codes exist for

moderate block length (though, asymptotically, t/n is
poor,) (ii) relatively simple encoding & decoding, (iii)
contains a very important subclass of R-S codes,
(iv) well structured & hence, its study forms a basis
for good underestimating of algebraic codes.
Definition of BCH Codes: Let q and m be given
and let be any element of GF(qm) of
02/01/15

SCK, IIT Kharagpur

131

order n (i.e. n = 1). Then for any positive integer

t and any integer j0, the corresponding BCH code
is the cyclic code of block length n with the
generator polynomial g(x) = LCM { fj0(x), fj0 +1(x),
fj0+2t 1(x)}
When fj(x) is the minimal polynomial of j [over
GF(q)].

Usually, one chooses j0 = 1, which leads to the

smallest
degree of g(x).
02/01/15
SCK, IIT Kharagpur

132

Further, a practical code requires a large block

length n (remember Shannon?) & hence, is

chosen as the primitive element, whose order is the
largest.
For such a selection of , the BCH code is called
a primitive BCH code & in this case, n = q m 1
(the order of the primitive element).
Let,
n 1
r(x) = Received polynomial = r x

i
i 0

02/01/15

SCK, IIT Kharagpur

i
133

c(x) = code word polynomial = cixi

e(x) = error polynomial =

n 1

i
e
x
i
i 0

r(x) = c(x) + e(x).

Now, C(x) is a multiple of g(x)
So, r(j) = c(j) + e(j)

= e(j), if j is a zero of c(x), i.e. a zero of g(x)

n 1

i
e

r(j) = e(j) = i j ,
i 0

js that are zero of g(x).

02/01/15

j = 1, 2, , r for all

SCK, IIT Kharagpur

134

This is a set of r equation involving the error pattern

& not the codewords.
To solve these equation, the syndromes are
defined as,
Sj = r(j), j = 1, 2, , r 2t 1 or j0 +1, j0 +2,
.., r j0 + 2t 1.
& Degree of g(x) = n k mt for a binary BCH code
Designed distance = 2t + 1 & dmin (2t + 1) or d*.
02/01/15

SCK, IIT Kharagpur

135

Representations of GF(16) as an extension field of

GF(2): The primitive polynomial: p(z) = z 4 + z +1
Exponential
Notation

Polynomial
Notation

Binary
Notation

Decimal
Notation

Minimal
Polynomial

0000

0001

x +1

0010

x4 + x +1

z2

0100

x4 + x + 1

z3

1000

x4 + x3 + x2 + x + 1

02/01/15

SCK, IIT Kharagpur

136

Class Test-I; 40 minutes; 13.02.2007

Q #1. a) Construct a group G of integers with ten elements.
b) Verify whether the group is cyclic.
c) Identify a subgroup S with 3 or more elements and
obtain coset decomposition of G.
Q #2. Prove that the identity element is unique for a ring with
identity.
Q #3. a) Prove that the quotient ring Z/(q) is a field if and only if q
is a prime integer.
b) Construct Z/(7) by identifying its elements and operations.
Now, identify all the groups and subgroups that can also be
defined with the above elements and operations.
02/01/15

SCK, IIT Kharagpur

137

02/01/15

SCK, IIT Kharagpur

138

02/01/15

SCK, IIT Kharagpur

139

Theorem (On a cyclic code):

In the ring of polynomials GF(q) [x]/(xn1), a subset is
a cyclic code if and only if it satisfies the following
two properties:
1. is a subgroup of GF(q)[x]/(xn 1) under addition
2. If c(x) and a(x) GF(q)[x]/(xn 1), then

Rx n 1 a x c x

a(x): Takes care of end-around shifting & addition.

02/01/15

SCK, IIT Kharagpur

140

Generator polynomial g(x):

The unique non-zero monic polynomial (in ) of
smallest degree is called the generator polynomial
of and is denoted by g(x)
g(x) is of degree(n k)

02/01/15

SCK, IIT Kharagpur

141

Theorem #32 (On the property of the generator

polynomial):
A cyclic code consists of all multiples of the
generator polynomial g(x) by polynomials of degree
( k 1) or less.
[ k: number of information symbols / block]

02/01/15

SCK, IIT Kharagpur

142

Proof: All such polynomials must be in the code by the

earlier theorem which defines a cyclic code (as g(x) is in
the code).
Now, if any polynomial c(x) is in the code, then,
c(x) = Q(x) g(x) + s(x) where deg. s(x) < deg. g(x) = n k & s(x) =
c(x) Q(x)g(x) which happens to be a codeword because, both the
terms on the RHS are codeword polynomials & the code is linear.
But deg. of s(x) < (n k), which is the smallest degree of
any non zero codeword polynomial.

Hence, s(x) = 0 & c(x) = Q(x) g(x)

02/01/15

SCK, IIT Kharagpur

143

Theorem #33 [g(x) divides xn 1]:

There is a cyclic code of block length n with
generator polynomial g(x) if and only if g(x) divides (x n
1). [ Proof omitted ]
Following this theorem,
Xn 1 = g(x) h(x) parity check polynomial.
Further,

Rx n 1 h x e x

as, h(x) c(x) = h(x) g(x) a(x) = (xn 1) a(x) some

polynomials.
02/01/15

SCK, IIT Kharagpur

144

 If m(x) is information polynomial of degree (k -1)

or less,
c(x) = m(x) g(x) non systematic form
xn k m(x) + re(x), systematic form, where,
re(x) = Remainder polynomial
= - Rg(x) [xn-k m(x)]
02/01/15

SCK, IIT Kharagpur

145

BCH CODES (Bose-Chaudhuri_Hocquenghem)

A large class of multiple error correcting, linear,

block, cyclic codes.
Attractive because, (i) good codes exist for
moderate block length (though, asymptotically, t/n is
poor,) (ii) relatively simple encoding & decoding, (iii)
contains a very important subclass of R-S codes,
(iv) well structured & hence, its study forms a basis
for good underestimating of algebraic codes.
02/01/15

SCK, IIT Kharagpur

146

Definition [of BCH Codes]: Let q and m be given

and let be any element of GF(qm) of order n (i.e.
n = 1). Then for any positive integer t and any
integer j0, the corresponding BCH code is the
cyclic code of block length n with the generator
polynomial
g(x) = LCM { fj0(x), fj0 +1(x), fj0+2t 1(x)}
where fj (x) is the minimal polynomial of j .
02/01/15

SCK, IIT Kharagpur

147

 Usually, one

chooses j0 = 1, which leads to the smallest

degree of g(x).
Further, a practical code requires a large block length
n (Shanons Theorem?) & hence, is chosen as the
primitive element, whose order is the largest.
For such a selection of , the BCH code is called a
primitive BCH code & in this case, n = qm 1 (the order
of the primitive element).

02/01/15

SCK, IIT Kharagpur

148

Let,

r(x) = Received polynomial = rixi

n 1
i i
iei x
i 0

c(x) = code word polynomial = c

x

e(x) = error polynomial = eixi

r(x) = c(x) + e(x).
Now, c(x) is a multiple of g(x)
1
So, r(j) = c(j) +ne(
j) i

ei j

= e(j), if i j 0is a zero of c(x), i.e. a zero of g(x)

r(j) = e(j) =

j = 1, 2, , r for all

SCK, IIT Kharagpur

js that are zero of g(x).
02/01/15

149

This is a set of r equations involving the error

pattern & not the codewords.
To solve these equations, the syndromes are
defined as,
Sj = r(j), j = 1, 2, , r 2t or j0, j0 +1, j0 +2,
.., r j0 + 2t 1.
Degree of g(x) = n k m.t for a binary BCH code
Designed distance = 2t + 1 & dmin (2t + 1) or d*.
02/01/15

SCK, IIT Kharagpur

150

Representations of GF(24) as an extension field of

GF(2): The primitive polynomial: p(z) = z 4 + z +1
Exponential
Notation

Polynomial
Notation

Binary
Notation

Decimal
Notation

Minimal
Polynomial

0000

0001

x +1

0010

x4 + x +1

z2

0100

x4 + x + 1

z3

1000

x4 + x3 + x2 + x + 1

02/01/15

SCK, IIT Kharagpur

151

Exponential
Notation

Polynomial
Notation

Binary
Notation

Decimal
Notation

Minimal
Polynomial

z +1

0011

x4 + x + 1

z2 +z

0110

x2 + x + 1

z3 + z2

1100

12

x4 + x3 + x2 + x + 1

z3 + z +1

1011

11

x4 + x3 + 1

z2 + 1

0101

x4 + x + 1

z3 + z

1010

10

x4 + x3 + x2 + x + 1

10

z2 + z +1

0111

x2 + x + 1

11

z3 + z2 +z

1110

14

x4 + x3 + 1

02/01/15

SCK, IIT Kharagpur

152

Exponential
Notation

12

Polynomial
Notation

Binary
Notation

Decimal
Notation

Minimal
Polynomial

z3 + z2 + z +1 1111

15

x4 + x3 + x2 + x + 1

13

z3 + z2 +1

1101

13

x4 + x3 + 1

14

z3 + 1

1001

x4 + x3 + 1

15 = 1
Note: , 2, ( 2)2 = 4,

all have the same

12

all have the same

minimal polynomial x4 + x + 1.
Similarly 3, ( 3)2 = 6,

minimal polynomial x4 + x3 + x2 + x + 1
02/01/15

SCK, IIT Kharagpur

153

Theorem #34 (On the property of minimal

polynomial):
If f(x) is the minimal polynomial over GF(q) of
[(an element of GF(qm)], then f(x) is also the
minimal polynomial of q.
Definition (Conjugates):
Two elements of GF(qm) that share the same
minimal

polynomial

over

GF(q)

are

called

conjugates [w.r.t. GF(q)].

02/01/15

SCK, IIT Kharagpur

154

The Peterson Gorenstein Zierler Decoder: (PGZ)

Any technique for decoding a cyclic code can be
used to decode BCH codes but all of them are not
equally good for practical implementation.
The PGZ algorithm is easier to understand and it
provides the basis for the more efficient decoding
schemes.
For the sake of simplicity, let, j0 = 1.
02/01/15

SCK, IIT Kharagpur

155

So, g(x) = LCM [f1(x), f2(x), f2t (x)]

Where fj(x) is the minimal polynomial of j . is any
element of GF(qm) may or may not be primitive.
So, n = code length = order of over GF(qm).
t = error correcting capability of the code.
k = no. of message symbols; dictated by m, na
and t.
Now, the error polynomial may be written as, e(x) =
en-1 xn 1 + en 2 xn-2 + + e1x +e0.
02/01/15

SCK, IIT Kharagpur

156

Here, at most t coefficients are non-zero.

Let, the actual no. of errors in a received word be
.
The decoding algorithm will work if 0 t. If the
errors are at locations i1, i2, i, the error
polynomial can also be written as,
E(x) = ei1 xi1 + ei2 xi2 + + ei xi

, unknown.

S1 = r() = c() + e() = e()

= ei1 i1 + ei2 i2 + . + ei i

02/01/15

SCK, IIT Kharagpur

157

We re-write the equation as

S1 = Y1X1 + Y2X2 + + YX where
Yl = eil is the error magnitude (1 l ) and Xl = il
is the error location number.
Similarly, we can define the other syndromes Sj,
for 1 j 2t as
Sj = r(x = i) = c(i) + e(i) = e(i)
Then we have the following set of 2t simultaneous
02/01/15

SCK, IIT Kharagpur

158

equations in the unknown error location X1, X2,

X and the unknown error magnitudes Y1, Y2, , Y.
S1 = Y1X1 + Y2X2 + .. + YX
S2 = Y1X12 +Y2X22 + .. + YX2
..
S2t = Y1X12t + Y2X22t + + YX2t
This set of 2t simultaneous nonlinear equations must
have at least one solution because of the way the
syndromes have been defined.
02/01/15

SCK, IIT Kharagpur

159

 The solution is unique.

To solve these equations, an intermediate
polynomial (x), called the error locator polynomial
is defined as,
(x) = x + -1 x-1 + -2 x-2 + .. + 1x + 1
(B)
(x) has zeros at the inverse error locations X l 1
for l = 1, 2, , . i.e.
(x) = (1 x X1) (1 x X2) (1 x X).
02/01/15

SCK, IIT Kharagpur

160

Now, the immediate aim is to find the coefficients of

(x), i.e. 1, 2, from the knowledge of 2t
syndromes.
Multiplying both side of (B) by YlXlj+, we get,
YlXl j+ (1 + 1x + 2x2 + .. + -1 x-1 + x) =
YlXl j+.(x)
Now, let x = Xl 1. So,
YlXl j+ (1 + 1Xl1 + 2Xl-2 + .. + -1 Xl-(-1) + Xl - ) = 0
02/01/15

SCK, IIT Kharagpur

161

Or,
Yl [ Xlj+ + 1Xlj + -1 + 2Xlj+ - 2 + + -1 Xl j +1 + Xlj] = 0

This relationship is valid for each l( 1 l )

and each j.
So,

j
j 1
j 2
j 1
j

Y
X

.....

X
l l
1 l
2 l
1 l
l 0
l 1

or,

Yl X l
l 1

Sj+
02/01/15

1 Yl X l
l 1

j 1

..... Y1 X l j 0
l 1

Sj for t & 1j

Sj+-1
SCK, IIT Kharagpur

162

So, we have a set of equations relating the i-s

with Sj-s:
1S j+-1 + 2S j + -2 + . + Sj = -Sj+
j = 1, ..,
This is a set of linear equations relating the
syndromes with the coefficients of (x). It is easier
to solve them.
Written in a matrix form:
02/01/15

SCK, IIT Kharagpur

163

 S1 S2 S3 ......... S 1
S S S ......... S
3
4

2
S3

M
S
1 4

S 1
S4 S5 ......... S 1 S 2

M M M
M
M
S 1 S 2 .........S 2 2 S 2 1
4 4 4 4 42 4 4 4 4 4 43

1
2

M

1

S 1
S
2

S 3

S
2

The i -s may be found by matrix inversion if the Smatrix is non singular. It is really so if t.
02/01/15

SCK, IIT Kharagpur

164

Theorem #35 [On Vander monde matrix]: The

Vander monde matrix defined as any matrix of the
form
1

X1

X 12

X 13

L L 1

X2L L X

2
X 22 L L X

3
X 23 L L X

1
X 1 1 X 2 1 L L X

02/01/15

SCK, IIT Kharagpur

165

has a non zero determinant if and only if all of the

Xi for i = 1, 2, .., are distinct.
Proof:

Not included but can be given using the principle of

induction.

Theorem #36 [On non-singularity of syndrome

matrix]:
The matrix of syndromes

02/01/15

SCK, IIT Kharagpur

166

 S1

S2

S2
M
M

S3

M
S 1

S 1

M
S2 1

is non singular if is equal to , the actual no. of

errors. The matrix is singular if >.
Proof: Omitted
02/01/15

SCK, IIT Kharagpur

167

So, by matrix inversion, we can find 1, 2, ..,

by setting = t first and checking each time
whether det (M) 0.
Next we find zeros of (x) to find the error location
X1, X2, , X.
If the code is non-binary, write the set of 2t linear
equations in Yi-s:
S1 = Y1X1 + Y2X2 + . + YX
02/01/15

SCK, IIT Kharagpur

168

S2 = Y1X12 + Y2X22 + .. + YX2

S2t = Y1X12t + Y2X22t + . + YX2t.
The first equation can be solved for error
magnitudes if the determinant of the matrix of
coefficients is non zero.
Now, we see that,
X1

X2L

det X12

X1

X 22 L

02/01/15

X 2 L

SCK, IIT Kharagpur

X
2
X

X
169

X1

X1 X 2 ..... X det X12

X1 1

1L
X2L
X 22 L
X 2 1 L

X
X2

X 1

The last matrix is a Vander monde matrix; Xi-s are

non zero and distinct.
So, the corresponding determinant is non zero.
02/01/15

SCK, IIT Kharagpur

170

We now have an outline of the PGZ decoding

algorithm:
Find the syndromes
Find the value of starting with a trial value of = t and
then reducing it in steps of 1.
Now we can determine the coefficients of the error
locator polynomial by a matrix inversion
Next, we can determine the error locations by a search
procedure (Chien Search)
02/01/15

SCK, IIT Kharagpur

171

The flow chart for the PGZ decoder

Enter r(x)
Compute syndromes
Sj = r( j), j = 1,2, .,
2t
=t

det [M] = 0

Yes

-1

No

Find error location Xl = (l = 1, 2, ..,

By finding zeros of (x)

Clrien
search

(x) = r(x) + (x)

Halt.

02/01/15

SCK, IIT Kharagpur

172

Horners rule in Chien search:

() = ( ((( + -1) + -2) + -3) + ..
+ 0)
It needs only multiplications and additions to
compute ().

02/01/15

SCK, IIT Kharagpur

173

Logic Circuits for Finite Field Arithmetic

Scalar:

h.

Function of a single input variable

Multiplies i/p with a fixed element of GF(q)

02/01/15

SCK, IIT Kharagpur

174

Adder:

Multiplier:

02/01/15

Functions of two inputs

from GF(q).
For the binary case, they
are EX-OR gate & AND
gates respectively.

SCK, IIT Kharagpur

175

n stage shift register:

Each stage contains one element of GF(q) (at the

output)
Hexadecimal shift registers made from binary
components:

Serial:
02/01/15

SCK, IIT Kharagpur

176

Parallel
Addition of two field elements:

+
serial
02/01/15

SCK, IIT Kharagpur

177

+
+
+
+
Parallel
02/01/15

SCK, IIT Kharagpur

178

Multiplication by a constant field element over

GF(24):
Let, = z3 & = 3z3 + 2z2 + 1z + 0
= 3z6 + 2z5 + 1z4 + 0z3
= (3 + 0).z3 + (3 + 2).z2 + (2 + 1).z+ 1 [reduced
with the help of the primitive polynomial p(z) = z 4 + z
+1]

02/01/15

SCK, IIT Kharagpur

179

+
3

Parallel

2
1

02/01/15

SCK, IIT Kharagpur

180

3 2 1 0
+
Serial

+
Z

Z4

This ckt. Divides by the

fixed polynomial z4 +z+1

02/01/15

Z3

SCK, IIT Kharagpur

* This ckt performs the

two steps of
multiplicities

181

Shift Register Circuits (digital filters) useful in the

construction of encoders / decoders:
(i)
x

x1

x2

xn-1

xn

Cyclically shifts a polynomial of degree (n 1).

It computes x.v(x) (mod xn 1) in one shift

02/01/15

SCK, IIT Kharagpur

182

(ii) A general LFSR (Linear Feedback Shift Register)

-1

h1

h2

h3

h4

pL-1

p j hi p j i ,

hL - 1
p1

pL-2

hL
pj -s

p0
., p3, p2,p1,p0

jL

i 1

Load p0 to pL-1 & get pj, j L. (unending)

02/01/15

SCK, IIT Kharagpur

183

(iii) An autoregressive filter or a recursive filter (a

variant of the LFSR):
Load the shift register stage with p0, p1, p2, ., p L-1.
Now, feed a0, a1, a2, .. to get the following sequence:

-1

pL-1

h1

h2

h3

pL-2

hL - 1

hL
p1

ak, ak-1, ., a1, a0

L

p j hi p j i a j L

p0
., p3, p2,p1,p0

jL

i 1

02/01/15

SCK, IIT Kharagpur

184

(iv) Linear Feed forward Shift Register (a non

recursive filter or an FIR filter):

b0, b1,b2,.,bL+k-1, bL+k

+
gL-1

+
gL-2

gL-3

02/01/15

g1

g0

gL-4

gL

a0, a1,a2,.,ak-1, ak
(input in reverse order)

L stage
SCK, IIT Kharagpur

185

Linear Feed forward Shift Register

An externally generated sequence is used as

input to the shift register
Let the tap weights of the feed forward shift
register be denoted by:
g(x) = gLxL + gL-1 xL-1 + .. + g1x + g0
Let the input and output sequences be,
a(x) = akxk + + a1x + a0
&
02/01/15

b(x) = bk+L x k+L + . + b1x + b0

SCK, IIT Kharagpur

186

Linear Feed forward Shift Register

Then the shift register operation generates,

b(x) = g(x).a(x) or

b j gi a j i , a n 0 0 j L k
i 0

Note: (a) The shift register initially contains zero and

(b) An input a0 is followed by L zeros.
Because of this multiplication property, the circuit is
also called a multiply-by g(x) circuit.
It may be noted that in this scheme the contents of the
shift register are not altered.
02/01/15

SCK, IIT Kharagpur

187

An Example
Ex #1: Let, g(x) = x8 + x7 + x4 + x2 + x + 1 over GF(2).
b(x) = a(x).g(x)
+

x
Input
a(x)

x7

x4

x2

b(x)

1
a(x)

An alternative configuration where contents of shift register

are altered:
b(x)
+

1
a(x)
02/01/15

x2

x4

x7

x8

b(x) = a(x).g(x)
SCK, IIT Kharagpur

188

(v) Circuit to divide an arbitrary polynomial a(x) by a

fixed polynomial g(x):

Let us assume a monic polynomial as the

divisor polynomial:
g(x) = xn-k + gn-k-1 xn-k-1 + + g1x +g0

an1 x an2 an1 g nk 1 x ...

k 1

nk

x gnk 1 x

n k 1

n 1

k 2

n2

... g1 x g 0 an1 x an2 x .... a1 x g 0

an1 x an2 an1ank 1
n 1

02/01/15

SCK, IIT Kharagpur

x n 2

...
189

Recursively, let,
Q(r) (x): quotient polynomial at the r-th
recursion [with, Q(0)(x) = 0]
R(r)(x): remainder polynomial at the r-th
recursion [R(0)(x) = a(x)]

R ( x) R
(r )

Then,

( r 1)

( r 1) k r
nr

( x) R

.g ( x )

Q ( r ) ( x) Q ( r 1) ( x) Rn( rr1) x k r
and
Rn rr1

:Coeff. of xn-r in the remainder polynomial R(r-1)(x)

02/01/15

SCK, IIT Kharagpur

190

g0

+
a(x) ., an-2, an-1

02/01/15

g1

gn-k-1

g2

Initially loaded with

zero.

SCK, IIT Kharagpur

-1

+
bk-21, bk-1.

191

 From (n-k) th to n-th shift, the quotient passes out of

the shift register and the remainder is left in the shift
register.
This configuration of the divide-by-g(x) ckt modifies
the content of the shift register.
Further, additions are needed to internal stages of
the shift register.

02/01/15

SCK, IIT Kharagpur

192

Ex. Let, g(x) = x8 + x7 + x4 + x2 + x + 1. over GF(2).

Input

Output

+
a(x)

+
x0

02/01/15

+
x2

+
x4

SCK, IIT Kharagpur

+
x7

193

An alternative configuration of divide-by-g(x) ckt where

internal addition is not needed:

The idea is to delay the subtractions until all

the subtractions in the same column can be done
at the same time. Following the idea, one can
modify the previously written iterative expressions
as:

R(r)(x) = a(x) Q(r)(x) g(x)

And hence,
n 1

Rn r an r g n r i Qi
r 1

r 1

i 1

02/01/15

SCK, IIT Kharagpur

194

Further,

r 1

k r
x

R
nr x
r 1

Schematic of divide-by-g(x) circuit using (n-k) stage

shift register:
+
-1
a(x)
... an-2, an-1

gn-k-1

+
gn-k-2

+
gn-k-4

+
g1

g0

+
Quotient during shifts
from (n-k) to n

Down for shifts from

(n+1) to (n+k)
02/01/15

SCK, IIT Kharagpur

195

Ex: g(x) = x8 + x7 + x4 + x2 + x + 1
+

a(x)

+
R

02/01/15

SCK, IIT Kharagpur

196

Shift register encoder:

Ex. of a Hamming (15,11) nonsystematic code: g(x) = x4 + x + 1

The codeword c(x) = m(x)g(x).

+
x4

x0

c(x)

m(x)
15 bit codeword

11 bit int. 4 bit pad

Encoder g(x)
02/01/15

SCK, IIT Kharagpur

197

The corresponding systematic coder:

In systematic form,
c(x) = xn-k m(x) + re(x) when re(x) = -Rg(x) [xn-k m(x)].
Realization: a) conceptually straight forward but
computationally inefficient.

02/01/15

SCK, IIT Kharagpur

198

Open on last four

bits
m(x)
+

Up for last 4 bits

(n-k) zeros
padded at the
end of m(x)

C(x)
Down for first 11 bits
(after 4 initial shifts)

Totally, 4 + 11 + 4 = 19 shifts are needed to encode.

Multiplication by x4 is implicit in the timing of the circuit.
The division operation does not begin till the first four bits are in
position in the register. So, an additional 4 bit buffer is used to
ensure that the first bit is sent to the channel just when the first
step of the division occurs.
02/01/15

SCK, IIT Kharagpur

199

After eleven iteration of division operation, the

remainder is left in the divide-by-g(x) circuit to be
shifted out to the channel.
During these last 4 shifts, the feedback path is
broken.
So, in all, it takes 19 shits to generate a 15 bit
codeword.

02/01/15

SCK, IIT Kharagpur

200

Realization: b)
Simplifications possible in the above realization:
Observe that the last four bits of x4m(x) are always
zero as the information is only 11 bits long. So, the last
four bits need not be added to the remainder.
The incoming information bits do not immediately
enter the divide-by-g(x) circuit, but are entered at the
right time to form the feedback signal. The following
modified realization does the same thing as in
realization a) but in 15 clock cycles.
02/01/15

SCK, IIT Kharagpur

201

Open on last four

bits

Up for last 4 bits

Down for first 11 bits

m(x)

This scheme requires only 11 + 4 = 15 shifts &

hence is faster.

02/01/15

SCK, IIT Kharagpur

202

Switch
1
g1

r0

g2

r1

g n-k-1

r2

rn-k-1

+
V(X)

n-k shift registers

m(X)

Fig: Encoding with an (n-k)stage shift register

02/01/15

SCK, IIT Kharagpur

203

A syndrome decoder for a nonsystematic Hamming

(15, 11) code:
This is a single-error correcting code
The received polynomial r(x) = c(x) + e(x)
s(x) = syndrome polynomial (degree 3)
= Rg(x) [r(x)] = Rg(x) [c(x) + e(x)]
= Rg(x) { c(x)} + Rg(x) {e(x)} = Rg(x) [e(x)].
The most likely error patterns for all possible
syndrome patterns are stored in a look up table.

02/01/15

SCK, IIT Kharagpur

204

r(x)

16 word by 15 bit ROM

c(x)
15 bit shift Reg.
15 bit shift Reg.

( x) is the quotient polynomial when

m
c$( x) is divided by g(x).
02/01/15

SCK, IIT Kharagpur

m(x)

205

Reed Solomon (RS) Codes

Linear, block, cyclic nonbinary codes: i) systematic
(common) ii) nonsystematic.
Multiple random as well as burst error correcting
capability.
A popular subset of the non-binary BCH codes in
which the symbol field GF(qm1 ) and the error locator
field GF(qm2 ) are the same, i.e. m1 = m2=1
A primitive RS code is characterized by the generator
polynomial:
g(x) =
element of GF(q).
02/01/15

d-1

i
(x-
), where is the primitive
i=1

SCK, IIT Kharagpur

206

 Note: The minimal polynomial over GF(q) of an

element, say , in the same field GF(q) is f(x) = (x-)

A Reed-Solomon Code is a maximum distance

separable (MDS) code.

d = 2t +1 = n-k+1 = d*; i.e. n-k = 2.t

[ Singleton Bound: d* n- k+1]
No. of coded symbols in a codeword n = q 1;
No. of information symbols in a codeword k = n 2t.
The dual of a Reed-Solomon code is also an RS code.
(31, 15) (31,16) RS code. So, a large no of codes are
available.
02/01/15

SCK, IIT Kharagpur

207

Berlekamp- Massey Algorithm

Most of the computation required to decode
BCH codes using PGZ algorithm are due to the
solution of the matrix equation.:
S1

S2 .. S

S2

S3 .. S+1

-1

S+1 .. S2+1

02/01/15

SCK, IIT Kharagpur

-S +1
=

-S +2
-S2
208

 No. of computation needed for matrix inversion 3

B-M algorithm bypasses this matrix inversion by
viewing the problem as that of an LFSR synthesis
problem:
Let, (x) be known. Then, the first row of the matrix
equation defines S+1 in term of S1, S2, S and the
coefficients of (x)
Similarly, the second row defines S+2 in term of S1, S2,
S+1 and the coefficients of (x) and so on.
This sequential observation may be summarized by the
following equation:

Sj = - i=1 iSj-i,
02/01/15

j =+1, , 2.
SCK, IIT Kharagpur

209

The following LFSR does this job if initially loaded with

S1, S2, . S:

+
0=1

-1
Sj-1

-2

--1

Sj= -

i=1

iSj-i

Sj-2 . Sj-+1 Sj-

..S3S2S1

Our problem is to find the lowest degree (x) [corresponding

to the least-weight error pattern] which can be used as tap
weights to generate the 2t syndromes.
02/01/15

SCK, IIT Kharagpur

210

 To design the required LFSR, we must determine (a)

the shift register length L and (b) the feedback
connection polynomial (x) = x + -1 x-1 + .. + 1x +
1, where by design (x) L (Some rightmost stages may
not be tapped).
The design procedure is inductive.
For each r, starting with r =1, we design an LFSR for
generating the first r syndromes. Let the minimum length
shift register (MLSR) producing S1, S2, ., Sr be denoted
by (Lr, (r)(x)).
This register need not be unique. Several choices
may exist but all will have the same length.
02/01/15

SCK, IIT Kharagpur

211

At the start of r-th iteration, we have

already constructed a list of LFSR-s till the (r-1)-th iteration:
(L1, (1)(x))
(L2, (2)(x))
.
.
(Lr-1, (r-1)(x))
The BM algorithm finds a way to compute a new
shortest length SR (Lr, (r)(x)) which generates S1, S2, .., Sr
by (i) using the most recent SR or (ii) by modifying the tap
weights. or (iii) by increasing the length & modifying the tap
weights.
02/01/15

SCK, IIT Kharagpur

212

 At r-th iteration, compute the next output of (r-1)

-th shift register:

r = -

n-1
j=1

(r-1)

Sr-j

(Many terms are zero here as the upper limit has been
chosen as n-1 for convenience)

Let us now define the r-th discrepancy as:

r = Sr - r = Sr +
n-1

02/01/15

j=0

(r-1)

n-1
j=1

(r-1)

IIT Kharagpur
SSCK,
r-j

Sr-j
213

(r)

(r-1)

Case-(i) If r = 0, set (Lr, (x)) = (Lr-1 ,

(x))

Case-(ii) & (iii) Otherwise, modify the taps as follows:

(r)

(r-1)

(x) = (x) + Axl(m-1)(x) where A is a field

element, l is a positive integer and (m-1)(x) is one of
the SR polynomials appearing earlier in the list.
Now, recompute the r-th discrepancy (say r):
n-1

r=
=
02/01/15

j=0

n-1

n-1

j(r) .Sr-j =

j=0j

(r-1)

j=0
n-1

Sr-j + A.

{j(r-1) + A.xl.j(m-1) (x)}.Sr-j

(m-1)

Sr-j-l
j=0 j

SCK, IIT Kharagpur

214

Now, if we choose m < r such that m 0 and

choose l= r - m and A= - r/m, then see that,
r = r (r/ m). m

= 0.

So, the new SR will generate S1, S2, , Sr-1, Sr.

Now, to specify m, which gives rise to lowest
degree (x), choose m as the most recent iteration
at which Lm>Lm-1. (Note: m 0 when Lm > Lm-1).

02/01/15

SCK, IIT Kharagpur

215

Forney Algorithm
(x) = x + -1x-1 + . + 1x + 1.

= l=1 (1-xXl)
Let us define a syndrome polynomial S(x) as:
S(x)=

2t

j=1 Sjx =
j

2t

j j

Y
X
j=1
i=0
i ix

Also define an error evaluator polynomial (x) as,

(x) = S(x).(x) (mod x2t).

02/01/15

SCK, IIT Kharagpur

216

Theorem # (On the expression of the error evaluator

polynomial (x)):

The error evaluator polynomial can be written as,

(x) = x. i=1 YiXi li (1-xXl)

Proof:

(x) = S(x).(x) (mod x2t)

= [

2t

j j

Y
X
j=1
i=1 i i .x ] [

2t

2t
(1-xX
)]
(mod
x
)
i=1
l
j-1

2t
=
{Y
X
x[(1-xX
).
(X
x)
].
(1-xX
)}
(mod
x
)
i i
i
j=1IIT Kharagpur
i
li
l
02/01/15 i=1
SCK,
217

Now,
(1-xXi)

2t

j-1

j=1

(Xix) =(1-xXi){1+xXi+ (xXi) + (xXi) +..

+ (xXi)2t-1}
2t

= 1- (xXi) = 1- x2t Xi2t

(x) = { i=1YiXix(1-x2tXi2t)} li(1-xXl) (mod x2t)

= x. i=1 YiXili (1-xXl)}

02/01/15

SCK, IIT Kharagpur

Hence proved.

218

Theorem (The Forney Algorithm): The error

magnitudes are given by,
Yl =

(Xl-1)
ji (1-XjXl-1)

= -

(Xl -1)
Xl -1 (Xl -1)

Proof:
(x) = x [Y1X1(1-xX2)(1-xX3).(1-xX) + Y2X2(1-xX1)
(1-xX3) .. (1-xX) + + YlXl (1-xX1) (1-xX2) .
(1-xXl-1) (1-xXl+1) (1-xX) + ... + YX(1-xX1)
(1-xX2) (1-xX3) . (1-xX-1)]
(x = Xl-1) = Xl-1 [ 0 + 0 +. + YlXl (1-X-1lX1) (1-XSCK, IIT Kharagpur
219
102/01/15
-1
-1
-1
lX2) (1-X lXl-1) (1- X lXl+1).(1-X lX)]

= Xl-1 [YlXl jl (1-Xl-1 Xj)]

= Yl jl (1-XjXl-1)

i.e. Yl= [(Xl-1) ]/ [jl (1-XjXl-1)]

Further,
-1

-2

(x)=. x + (-1) -1x

=-

i=1

+ .. +2 2x + 1

Xi ji (1-xXj)

(Xl-1) = - Xl jl(1 XjXl-1)

jl(1-XjXl-1) = -Xl-1(Xl-1)
02/01/15

SCK, IIT Kharagpur

Yl= - [(Xl ) ]/ Xl (Xl )

-1

-1

-1

Hence Proved.

220

Berlekamp Massey Algorithm:

Initialize: (x)=0; r=0

L =0; B (x)=1
r r+1
Compute error in next syndrome
L

r = Sr +
Does current shift register
design produce next
syndrome?
02/01/15

j=1

jSr-j =

r=0?

j=0

jSr-j

YES (taps are OK)

NO (taps must be corrected)

SCK, IIT Kharagpur

221

Compute new connection polynomial for

which r = 0; T(x) = (x) - rx B(x)
Must shift register
be lengthened?

2L r-1
?

NO

(x) T(x)

YES
-1

B(x)r (x) : Store old shift register

after normalizing
(x) T(x)

: Update shift register

L r L

: Update length.

02/01/15

SCK, IIT Kharagpur

B(x) x B(x)

222

YES

NO

Halt- more
than t errors.

02/01/15

deg (x) = L
?

r = 2t ?

NO

YES

Proceed to next
step (i.e Forney
Algorithm)

SCK, IIT Kharagpur

223

Major steps for fast decoding of BCH Codes:

Enter v(x)
Compute syndromes: Sj = v(j);

j= 1,2,.2t.

Find (x) using the Berlekamp-Massey Algorithm

Find error location Xl by finding zeros of (x);
l = 1, 2, ., (chien search algorithm)

02/01/15

SCK, IIT Kharagpur

224

Forney Algorithm to compute error values:

(x) = S(x)(x) (mod x2t)

(x) = j=1 (jj)xj-1

= - i=1 Xiji (1-xXj)

- (Xl-1)
and Yl =
Xl-1(Xl-1)
(Xl-1)
=
jl (1-XjXl-1)
l= 1, 2, .,
Correct the received word v(x)
02/01/15

SCK, IIT Kharagpur

225

Multi-stage Coding
Schemes

02/01/15

SCK, IIT Kharagpur

226

SUPER CHANNEL
DATA

OUTER
CODER
SOURCE
(n, r)

INNER
CODER
(N, R)

CHANNEL

SUPER CODER (n N , r R)

INNER
DE
CODER

OUTER
DE
CODER

DATA
SINK

SUPER DECODER

TWO STAGE CODE CONCATENATION SCHEME

02/01/15
GSSST, IITKGP

SCK, IIT Kharagpur

S. CHAKRABARTI

FEC

227

Convolutional Coding

02/01/15

SCK, IIT Kharagpur

228

m = m1,m2, ., mi, ..

....

kK
kK stage shift
register

Input sequence
(shifted in k at a time)

n modulo-2
adders

Codeword sequence U = U1,U2, , Ui,

Where Ui = u1i, , uji,, uni
= i-th codeword branch
uji = j-th binary code symbol
of branch word Ui
02/01/15

Convolutional Encoder with constraint length K and rate k/n

SCK, IIT Kharagpur

229

u1

First code
symbol
Output branch
word

Input bit
m

u2

Second code
symbol

Convolutional encoder (rate= , K = 3)

02/01/15

SCK, IIT Kharagpur

230

(2,1,3) Convolutional Encoder

(.101101)

s0

02/01/15

s1

SCK, IIT Kharagpur

(0)

(0)

(1)

=(v0 v0 , v1(0) v1(0),..

(1)

231

State Diagram of (2,1,3) Convolutional Code

10
d

11

01

10

01

0
1

00
11
a

02/01/15

10

11
S0 S1

01

00
00
SCK, IIT Kharagpur

232

Tree Diagram of (2,1,3) Convolutional Code

00
00

11

00
00

00

01
11

10

01

01

10

11
00

11

00

02/01/15

01

SCK, IIT Kharagpur

01

10

10

11

233

Trellis Diagram of (2,1,3) Convolutional Code

00

00

(a) 00

00

00
11

11

11

11

11

(b) 01

11

11

11
00

00
10

10

00

00

10

10

(c) 10
01
11

11

(d) 11
(0)

(1)

(2)

(3)

01

01
11

11
(4)

(5)

State transitions
02/01/15

SCK, IIT Kharagpur

234

Viterbi Decoding Algorithm

02/01/15

SCK, IIT Kharagpur

235

Decoder trellis diagram (rate = 1/2, K=3)

02/01/15

SCK, IIT Kharagpur

236

Add-compare-select computations in Viterbi decoding

02/01/15

SCK, IIT Kharagpur

237

The Viterbi Algorithm

convolutional code:
Step 1.
metric for
path (the

for

(n,

k,

K)

Beginning at time unit t = 1, compute the partial

the single path entering each state. Store the
survivor) and its metric for each state.

Step 2.
Increase t by 1. Compute the partial metric for
all 2k paths
entering a state by adding the branch metric
entering that state to the metric of the connecting survivor at
the previous time unit.
For each state, compare the metrics of all 2k paths
entering that state, select the path with the largest metric (the
survivor), store
it along with its metric, and eliminate all
other paths.
The basic computation performed by the Viterbi algorithm is
the add, compare, select (ACS) operation of step 2. As a
02/01/15
SCK, IIT Kharagpur
238
practical matter, the information sequence corresponding to

Likelihood of s2

Likelihood of s1

p(z|s2)

p(z|s1)

z(T)
8-level soft
decision

000 001 010 011 100 101 110 111

0
02/01/15

Fig: Hard and soft decisions

SCK, IIT Kharagpur

2-level hard decision

239

Uncoded 4-ary PAM

Rate 2/3 coded 8-ary PAM

Uncoded 4-ary PSK

Uncoded 16-ary QAM

Rate 2/3 coded 8-ary PSK

Rate 4/5 coded 32-ary QAM

Fig: Increase of signal set size for trellis-coded modulation

02/01/15

SCK, IIT Kharagpur

240

First
data
bit

m1

u1 First coded bit

u2 Second coded bit

Second
data bit

m2

u3 Third coded bit

Fig: Rate 2/3 Convolutional Encoder

02/01/15

SCK, IIT Kharagpur

241

Branch word
u1 u2 u3
ti
State a = 00

ti+1

000
100
011

11
1

b = 10

01
1

111
0
00
0
10

010
110

c = 01

d = 11

02/01/15

010
110

001

10
1

101
00
1

SCK, IIT Kharagpur

Fig: Trellis diagram (rate 2/3

code)
242

Fig : Ungerboeck partitioning of an 8-PSK signal set

02/01/15

SCK, IIT Kharagpur

243

Fig: Ungerboeck partitioning of 16-QAM signals

02/01/15

SCK, IIT Kharagpur

244

Fig: Eight-state trellis diagram for coded 8-PSK

02/01/15

SCK, IIT Kharagpur

245

Fig: Bandwidth-efficiency plane

02/01/15

SCK, IIT Kharagpur

246

Parallel concatenation of two RSC encoders.

02/01/15

SCK, IIT Kharagpur

247

Feedback (Turbo) Decoder

02/01/15

SCK, IIT Kharagpur

248

Error Performance of some

FEC Codes

02/01/15

SCK, IIT Kharagpur

249

Decoded BER vs input BER

for the (15,11) BCH code;
1) experimental (HDD),
2) analytical (HDD),
3) experimental (SDD), and
4) analytical(SDD).

GSSST, IITKGP
02/01/15

S. CHAKRABARTI
SCK, IIT Kharagpur

FEC

250

Post decoding BER vs Eb/No

for the (15,11) BCH code;
1) HDD,
2) SDD, and
3) uncoded system performance.

P
O
S
T
D
E
C
O
D
I
N
G
B
E
R

GSSST, IITKGP
02/01/15

S. CHAKRABARTI
SCK, IIT Kharagpur

FEC

251

State Diagram for the rate half non systematic convolutional code of
constraint length = 5

02/01/15

GSSST, IITKGP

SCK, IIT Kharagpur

S. CHAKRABARTI

252

FEC

Viterbi decoded BER vs input BER

for the rate half convolutional
codes;
1) = 3 (HDD),
2) = 5 (HDD),
3) = 3 (SDD), and
4) = 5 (SDD).

GSSST, IITKGP
02/01/15

S. CHAKRABARTI
SCK, IIT Kharagpur

FEC

253

Viterbi decoded BER vs Eb/No

for the rate half convolutional
codes;
1) = 3 (HDD),
2) = 5 (HDD),
3) = 3 (SDD),
4) = 5 (Transfer function bound)
and
5) Uncoded system.

GSSST, IITKGP
02/01/15

P
O
S
T
D
E
C
O
D
I
N
G
B
E
R

S. CHAKRABARTI
SCK, IIT Kharagpur

FEC

254

Decoder over loading rate of

the sgt Viterbi decoder ( = 5);
1) Input BER = 0.13 and
2) Input BER = 0.10

02/01/15
GSSST, IITKGP

SCK, IIT Kharagpur

S. CHAKRABARTI

FEC

255

Measured performance of the (255,233) R S and (K) = 7 convolution code,

concatenated together

02/01/15
GSSST, IITKGP

SCK, IIT Kharagpur

S. CHAKRABARTI

256
FEC

Thank You

02/01/15

SCK, IIT Kharagpur

257