6.

897 Algorithmi Introdu tion to Coding Theory November 14, 2001

Le ture 16
Le turer: Madhu Sudan S ribe: Chris Peikert

Today's material is on:

 Low Density Parity Che k (LDPC) Codes
 Linear-time De odable Codes
The goal is to have very simple de oding algorithms. Re all that a linear ode is spe ied by an
n  (n k) matrix H su h that C is a odeword i CH = 0. Gallager's observation about linear odes
was that the parity he k matrix gives some additional information about where the errors o urred.
Suppose (CH )j = 1; this indi ates that the j th onstraint is not satised, so some bit of C orresponding
to a 1 in the j th olumn of H is wrong. Then if H has very few 1s in all of its olumns, we get a lot of
information about where an error lies in C . Gallager showed, by the probabilisti method, that one an
approa h the Gilbert-Varshamov bound on the distan e of su h a ode by randomly pla ing a onstant
number of 1s per olumn. Exer ise: re onstru t Gallager's proof (don't try this alone!).
Sipser and Spielman '97 show how to hoose H onstru tively, and argue the distan e of its asso iated
ode ombinatori ally. They view H as the adja en y matrix of a bipartite graph B = (L; R; E ) where
jLj = n, jRj = m = n k. The left verti es L represent the oordinates of odewords, and the right
verti es R represent the onstraints. A onstraint is \satised" if the ex lusive-or of its adja ent odeword
oordinates is 0 (this orresponds exa tly to matrix multipli ation in GF2 ). Then the number of 1s in
ea h olumn of H is small i the degrees of the right verti es is bounded from above. We will require
an \almost random" stru ture to the graph; this is known as the \expander" property.
Denition 1 (bounded, regular, expander) A bipartite graph B = (L; R; E ), jLj = n is:
 ( ; d)-bounded if its left verti es have degree at most and its right verti es have degree at most d,
 ( ; d)-regular if equality holds above,
 an (; Æ)-expander if 8S  L, jS j  Æn, the neighborhood of S , (S ), is large: (S )  jS j.
Dene CB , the linear ode derived from the graph B , naturally. Observe that 0 is a odeword of CB ,
and that the rate of CB = k=n  (n m)=n (the inequality follows from the fa t that some onstraints
may be linearly dependent, whi h an only in rease the rate). In parti ular, if B is ( ; d)-regular, then
n = dm ) m=n = =d ) rate  1 =d. The distan e of CB is given by the following lemma:
Lemma 2 If B is an (; Æ)-expander and is ( ; d)-bounded, then the distan e of CB is at least Æn,
provided 2 > .
Before proving the lemma, let's onsider its onsequen es. In order for it to be useful,  will need
to be very large. By boundedness of degrees, we must have j (S )j  jS j; the lemma requires that
j (S )j  jS j=2. Fortunately, su h expanders do exist.
Now we'll sket h a proof of Lemma 2, and ll in the holes as they appear. For any S  L, partition
(S ) into sets A; B  R, su h that A ontains the verti es of (S ) whi h have exa tly 1 neighbor in
S , and B is the remainder of (S ). Be ause CB is linear, it is suÆ ient to show that it ontains no
odeword with Hamming weight < Æn. Let S be the oordinates at whi h a ve tor is set to 1. We will
show that if a ve tor has fewer than Æn 1s, some onstraint is unsatised. It will be suÆ ient to prove
that A 6= ; when we x any jS j < Æn, be ause every onstraint in A is unsatised.
Lemma 3 If B = (L; R; E ) is a ( ; d)-bounded, (; Æ)-expander, then for any S  L, jS j < Æn, and A
dened above, jAj  (2 )jS j.

16-1
Proof of Lemma 3 First, (S ) = jAj + jB j > jS j by the expander property. Let e be the number
of edges in ident to S . Then we have jAj + 2jB j  e  jS j. By multiplying and subtra tion, we get
jAj  (2 )jS j, as desired.
We now give a simple, linear-time de oding algorithm for this ode: while there exists a oordinate
su h that more than half of its neighbors are unsatised onstraints, ip that oordinate.
To prove the orre tness of this algorithm, it is enough to analyze how it de odes a orrupted zero
odeword. It is lear that the algorithm requires no more than m = n k iterations, be ause the number
of satised onstraints de reases with ea h iteration. From now on assume that  > 3 =4, and say the
input ve tor has  n 1s. We prove that for suitable , the algorithm will terminate with the zero word.
Lemma 4 If 0 < jS j  Æn, then there is some j 2 S su h that j has more than =2 of its neighbors in
A.
Lemma 5 The number of 1s in the ve tor is always  Æn (for suitably hosen , to be determined).
>From these two laims, orre tness follows: jS j > 0 implies that the algorithm has not terminated.
And be ause jS j  Æn at all times, the algorithm terminates with jS j = 0. We now prove the lemmas.
Proof of Lemma 4 By Lemma 3, jAj > (3 =2 )jS j  jS j=2 so some j 2 S must have at least =2
neighbors in A.
Proof of Lemma 5 At the start, the number of unsatised onstraints is  n. So jAj  n
throughout the algorithm. Then by Lemma 3 we have

jS j  2jAj  2  n
so jS j  Æn (  = Æ(2 )= .
Note: we have proven a linear bound on the number of iterations. In fa t, with suitable data
stru tures, ea h iteration an be made to run in O(1) time.
Caveats:
 There is no known way of en oding other than by matrix multipli ation, in O(n2 ) time.
 There are no known deterministi onstru tions (at press time...) of expanders with a good enough
expansion fa tor ( > 3 =4). But even so, this is a non onstru tive ode with an expli it de oding
algorithm, whi h is good.
 Variants of this algorithm work, requiring a smaller expansion fa tor.
To make improvements, Lemma 3 is the bottlene k; we want to redu e the ratio. We would like
to dene A su h that its elements are those \ onstraints" (however dened) for whi h at most
of
its asso iated oordinates are \wrong." But we an just view this as an error- orre ting ode having
distan e
. Then ea h onstraint is \my neighbors should make a odeword of some other ode." With
this restri tion, we get jAj  ((
+ 1) )jS j, and the analysis follows from there.

16-2