Implementing the Belief Propagation Algorithm in MATLAB

Bjrn S. Rer o u Christopher M. Kellett Technical Report Version as of November 13, 2008

We provide some example Matlab code as a supplement to the paper [6]. This technical report is not intended as a standalone introduction to the belief propagation algorithm, but instead only aims to provide some technical material, which didnt t into the paper.

1 Introduction
For an excellent introduction and mathematical treatise of modern iterative decoding theory, we refer to [4]. Worth mentioning is also the survey paper [2] on factor graphs and the sum-product algorithm, the superclass that contains belief propagation. This current technical note provides Matlab code to implement the dynamical system formulation of the belief propagation algorithm and a few related concepts, as detailed in [6]. More conventional implementations that is, from a coding perspective exist and some are publicly available [3]. The Matlab code examples detailed in this report can be found, along with the most up-to-date version of this report itself, at [5]. Our presentation diers also in another aspect from the standard ones: Unlike the information theory convention, where messages and codewords are represented by row vectors, we throughout use column vectors as this is standard in dynamical systems. Of course this does not lead to dierences other than representational ones. This report is organized as follows: In Section 2 we give a simple example on how one can generate a very basic random parity-check matrix and compute a corresponding generator matrix. Section 3 details the channel transmission and Section 4 provides code to implement the belief propagation algorithm as a dynamical system. The output trajectories obtained using this Matlab code can then be plotted using the routine in

School of Electrical Engineering and Computer Science, The University of Newcastle, Australia, Bjoern.Rueffer@Newcastle.edu.au, Chris.Kellett@newcastle.edu.au

Section 5. In Section 6 we provide a more advanced method for generating parity-check matrices with prescribed degree distribution.

2 Parity-Check and generator matrices

A parity-check matrix is any matrix H Fmn . Throughout we assume that n = m + k, 2 n, m, k > 0 and that H has full rank, i.e., rank m. If H does not have full rank, then rows can be removed until it does, thereby increasing k and decreasing m accordingly. To generate a parity-check matrix for a repeat-n code in canonical form, one could use the following Matlab statement:
n=10; m=n1; H=spalloc(n1,n,2(n1)); for i=1:n1, H(i,i)=1; H(i,i+1)=1; end

A simple random parity-check matrix can be generated using the code in Listing 1, and code for generating more involved parity-check matrices is given in Section 6. Using only Gauss elimination and possibly by swapping columns, H can be brought into the form QHP = Im A , (1) where the invertible matrix Q Fmm encodes the steps of the Gauss elimination 2 (swapping and adding rows of H), P Fnn is a permutation matrix to encode the 2 swapping of columns in H, A Fmk , and Im is the m m identity matrix. 2 A generator matrix for H is a matrix G Fnk such that HG = 0. According to (1) 2 we can take G to be A G=P , (2) Ik since QHG = Im A P 1 P A = 2A = 0 Ik (in Fm ) . 2

Now G maps message vectors m Fk to codewords c = Gm Fn , i.e., to elements 2 2 of the null-space C = CH = {x Fn : Hx = 0} of H, which is also termed the set of 2 codewords or just the code. The MATLAB code examples in Listings 1 and 2 can be used to generate a very basic parity-check matrix and to obtain a generator matrix from a given parity-check matrix. Listing 1: A crude way to obtain a simple parity-check matrix, by just specifying the dimensions m and n and a density of non-zero elements in H of at least d (0, 1).
1 2 3 4 5 6 7 8

function H = g e n e r a t e H (m, n , d )
% H = GENERATE H ( m , n , d ) % % generate a m by n by the parity check matrix , where zero the and density one ) is % influenced parameter d ( between

H=sparse (m, n ) ; H=mod(H, 2 ) ;

9 10 11 12

while not ( a l l (sum(H,1) >=2) && a l l (sum(H,2) >=2)) , H=H+abs ( sprand (m, n , d )) >0; H=mod(H, 2 ) ; end

Listing 2: A code example to construct a generator matrix for a given parity-check matrix. Some consistency checks (e.g., to see if n > m) are omitted.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

function [G]= g e n e r a t o r m a t r i x (H ) ;
% function % H compute [ G] = g e n e r a t o r m a t r i x ( H ) ; a generator matrix G given a sparse parity check matrix

Hp=H; [m, n]= s i z e (Hp ) ;

% s u p p o s e m<n !

colperm=1:n ; f o r j =1:m,
% find row to put as new row j

i=min( find (Hp( j :m, j ) ) ) ; i f isempty ( i ) ;

% do some column swapping !

k=min(max( find (Hp( j , : ) ) , j ) ) ; i f isempty ( k ) , disp ( [ problem in row num2str ( j , 0 ) ] ) ; continue ; end temp = Hp ( : , j ) ; Hp ( : , j )=Hp ( : , k ) ; Hp ( : , k)=temp ; temp=colperm ( k ) ; colperm ( k)=colperm ( j ) ; colperm ( j )=temp ; end
% swap rows % adjust indices !

i=i+j 1; i f ( i = j ) , temp = Hp( j , : ) ; Hp( j , : ) = Hp( i , : ) ; Hp( i , : ) = temp ; end % i f

% clear out rest of column

K= find (Hp ( : , j ) ) ; K= K( find (K= j ) ) ; i f isempty (K) ,

41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

t 1=f u l l (Hp( j , : ) ) ; f o r k=K , t 2=f u l l (Hp( k , : ) ) ; temp=xor ( t1 , t 2 ) ; Hp( k , : ) = sparse ( temp ) ; end end end % f o r
% now Hp = [ I d m A ]

A = Hp ( : ,m+1:n ) ;
%c o m p u t e G

[ b , invperm ]= sort ( colperm ) ; G = [A; speye ( nm ) ] ; G=G( invperm , : ) ;

% consistency check : mod ( HG , 2 ) should give all zero matrix

end % f u n c t i o n

3 Transmission trough an AWGN channel

Now that we can generate codewords for given messages, we still have to map these to channel symbols, transmit via an Additive White Gaussian Noise (AWGN) channel and compute log-likelihood ratios (LLRs) afterwards. This is the aim of this section. An AWGN channel takes as input a real number x and outputs x+z, where z is drawn from a N (0, 2 ) normal distribution. Here is a channel parameter. Transmission of a vector x Rn means to consecutively transmit its components. It is assumed that the noise samples aecting each component are all independent.

3.1 Binary phase shift keying (BPSK)

A given codeword x C Fn can be mapped to a vector x Rn via 2 xi xi = (1)xi . This procedure is commonly referred to as BPSK. (3)

3.2 The transmission step

We have to x a channel parameter > 0. In Matlab, the transmission step is now very easy, we just use the assignment y = tilde x + randn(n,1)sigma2, to compute the vector of received channel symbols y Rn .

3.3 Computing LLRs

The log-likelihood ratios for each bit are given by ui = log pYi |Xi (yi |0) , pYi |Xi (yi |1) (4)

where pYi |Xi is the density of the conditional probability (xi , yi ) P (Yi yi |Xi = xi ). To compute ui , we actually have to know , or at least make a guess about . The guessing step, which we will omit here, is called estimation. For simplicity we assume that the receiver knows . Substituting the density formulas for the N (1, 2 ) and N (1, 2 ) distributions into (4) we obtain 4yi (5) ui = 2 . 2 So at this stage we can encode a k-bit message to an n-bit codeword, transmit it through a noisy channel (using BPSK) and compute a-priori log-likelihood ratios.

4 The Belief Propagation (BP) algorithm

A detailed descriptionof BP and motivation why this algorithm should do what it does, can be found in [4]. The implementation presented here is based on the article [6], which also contains a rather condensed introduction to iterative decoding using belief propagation. Listing 3 sets up the matrices B and P as well as the structure needed for the operator S, which is given in Listing 4. Listing 5 implements the BP algorithm as a dynamical system. The iterate BP(T,u) takes a number of iterations T to perform and a vector of input LLRs u Rn as arguments. The output is a real n (T + 1) matrix containing the trajectories of the output LLRs. Listing 3: Initialization for the main program: Compute the matrices B,P and the structure for the operator S, which is here also encoded via a matrix (S )
1 2 3 4 5 6 7 8 9 10 11 12 13 14

function [ ] =H2DS(H)
% H2DS ( H ) % % generate matrices f o r dynamical system a s s o c i a t e d to H, i . e . , P , % S , and q , which a r e s t o r e d a s g l o b a l v a r i a b l e s , a s w e l l a s m and % n, the dimensions of H

global B P S [m, n]= s i z e (H ) ; q=nnz (H ) ;

% calculate % needed for

q m n

the

amount

of

nonzero

elements

these

matrices :

15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

P=s p a l l o c ( q , q , ( sum(H, 2 ) 1 ) sum(H, 2 ) ) ; S =s p a l l o c ( q , q , ( sum(H,1) 1) sum(H, 1 ) ) ;

% find the matrix P

k =0; f o r j =1:n , I=find (H( : , j ) ) ; f o r x =1: length ( I ) , f o r y=x +1: length ( I ) , P( k+x , k+y )=1; P( k+y , k+x )=1; end end k=k+length ( I ) ; end
% find S ( structure for the nonlinearity S)

k =0; f o r i =1:m, J=find (H( i , : ) ) ; f o r x =1: length ( J ) , f o r y=x +1: length ( J ) , S ( k+x , k+y )=1; S ( k+y , k+x )=1; end end k=k+length ( J ) ; end
% compute matrix B

B=s p a l l o c ( q , n , q ) ; b =[]; f o r k =1:m, b =[ b find (H( k , : ) ) ] ; end B=sparse ( [ 1 : q ] , b , ones ( q , 1 ) , q , n ) ;

Listing 4: The implementation of the nonlinearity S : Rq Rq

1 2 3 4 5 6 7 8 9 10 11

function y = S ( x )
% S the % % y=S ( x ) % formula % row in nonlinearity is to the the vector in t h e BP f e e d b a c k by system the to atanh the ith

obtained of S x

applying

those global

indices matrix

corresponding

global S

y=ones ( q , 1 ) ; f o r i =1: q ,

12 13 14 15 16

f o r j=find ( S ( i , : ) ) , y ( i ) = y ( i ) tanh ( x ( j ) / 2 ) ; end end y=2atanh ( y ) ;

Listing 5: A function to calculate output trajectories for the dynamical system mimicking BP, where u Rn is the vector of a-priori LLRs and T N is the number of iterations to make
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

function y = i t e r a t e B P (T, u )
% y = i t e r a t e B P (T , u ) This f u n c t i o n implements % system ; the output t r a j e c t o r y i s returned for % input u. The initial state is always zero . t h e BP d y n a m i c a l f i n a l t i m e T and

global B P S x1 x2 x1 x2

q n m

k=zeros ( q , 1 ) ; k=zeros ( q , 1 ) ; k 1=zeros ( q , 1 ) ; k 1=zeros ( q , 1 ) ;

y=zeros ( n , T+1); f o r t =1:T, x 1 k 1=P x 2 k+Bu ; x 2 k 1=S ( x 1 k ) ; y ( : , t )=B x 2 k + u ; x 1 k=x 1 k 1 ; x 2 k=x 2 k 1 ; end y ( : , T+1)=B x 2 k + u ;

5 Plotting output trajectories

To visualize the output of the function iterate BP() from the previous section, we provide here a routine to generate a plot in the avor of Figure 2 in [6], see Listing 6 Listing 6: Plotting output trajectories in color or monochrome
1 2 3 4 5 6 7 8

function p l o t B P o u t p u t ( y , mono , f i l e n a m e )
% p l o t B P o u t p u t ( y , mono , f i l e n a m e ) P l o t % If mono is supplied ( regardless of its a given output then trajectory . in value ) , plot

% monochrome , o t h e r w i s e i n c o l o r . I f i n a d d i t i o n a f i l e n a m e i s % s u p p l i e d , s a v e t h e f i g u r e t o t h a t f i l e ( i n EPS f o r m a t ) . S a v i n g % to a file implies that the plot will be monochrome .

global n

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

T=s i z e ( y ) ; T=T(2) 1; clf ; i f nargin >1, plot ( 0 : T, y , ko - ) % m o n o c h r o m e else plot ( 0 : T, y , o- ) % c o l o r end grid on ; axis ( [ 0 T min(min( y )) .5 max(max( y ) ) + . 5 ] ) LEGEND= [ ] ; f o r k =1:n , LEGEND=[LEGEND; s t r c a t ( output , num2str ( k ) ) ] ; end legend (LEGEND) xlabel ( time ) ; ylabel ( LLR ) ; i f nargin==3, i f isstr ( filename ) , s a v e a s ( gcf , f i l e n a m e , eps ) ; end end

6 More parity-check matrices

In this section we rst introduce so-called regular parity check matrices. These are a special case of parity-check matrices with a prescribed degree distribution pair. By the degree distribution we actually refer to the bipartite undirected graph dened by a parity-check matrix, the so-called Tanner or factor graph, cf. [4]. In fact, it is mostly the degree distribution pair, that determines how the majority of the possible choices of parity-check matrices for that pair and a given code length perform [4, p.94], at least for very large block length. One distinguishes the factor or check node distribution and the variable or bit node distribution . Together these form a degree distribution pair (, ), given by the polynomials (x) =
i

i xi1 i xi1 ,
i

(6) (7)

(x) =

cf. [4, p.79], where i is the fraction of edges that connect to a variable node of degree i and i is the fraction of edges that connect to a check node of degree i.

6.1 Regular parity-check matrices

Now for a given pair of positive integers (l, r), a regular (l, r)-code or a regular (l, r) parity check matrix has l = 1, i = 0 for i = l and r = 1 and i = 0 for i = r. That is, the degree is the same for all variable nodes, namely l, and the same for all check nodes, namely r. The interesting thing about regular codes is, that for a xed pair (l, r) the number q of non-zero entries in H scales linearly with n. Note that m must satisfy l n = r m, so that m is determined by the triple (l, r, n).

6.2 Capacity achieving degree distributions

It has been shown, that certain degree distribution pairs lead to codes with rates performing extremely close to Shannons channel capacity, the theoretical limit. Such degree distributions can be found in [1]. There a code length of n = 107 has been used, with a randomly constructed parity-check matrix (avoiding short cycles). At a maximal variable degree of 200 and an average check node degree of 12, it could be shown that the code performed within 0.04 dB of the Shannon limit at a bit error rate of 10 6. It is reported that in this setting for a successful decoding an average of 8001100 iterations were needed. Please note that the provided Matlab code has only been tested with much shorter block length, e.g., n = 100. To eciently simulate large block-lengths, some numerical improvements should be taken, as indicated in [1].

6.3 Generating parity-check matrices with pre-described degree distributions

In Listing 7 we provide an example Matlab program to generate a parity-check matrix for a given degree distribution pair. Listing 7: Generate a parity-check matrix with prescribed degree distribution pair.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

function [ H, f i n a l c o l u m n w e i g h t s , f i n a l r o w w e i g h t s ] = . . . MacKayNealCreateCode ( n , r , v , h )
% % MacKay % % Date % % Inputs : % % % Outputs : % % % % Sample p a r a m e t e r s f o r r a t e 1/2 , l e n g t h 100 , (3 ,6) r e g u l a r code % n = 100; % r = 0.5; % Code % Code length rate Neal algorithm 3 for constructing 2008 code rate (r), (h) (H) , generated column column weight polynomial p a r i t y check matrix

created : code (v) ,

November (n) ,

length row

weight x n

polynomial

n (1 r ) and row

p a r i t y check

matrix

weight

polynomials

18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66

% v = [0 % h = [0

1.0]; 1.0];

% Column % Row

distributions as

as

polynomial

0 0 0 0

distribution

polynomial

% % Initialisation

m = f l o o r ( n(1 r ) ) ; H = zeros ( [ m, n ] ) ; alpha = [ ] ;

% alpha % each

will column

contain

the

column

weight

for

f o r i = 1 : length ( v ) for j = 1 : ( floor ( v ( i )n ) ) % a l w a y s u n d e r f i l l and t h e n add e x t r a s

% later

alpha = [ alpha i ] ; end end while ( length ( a l p h a ) = n ) % f i l l alpha = [ alpha i ] ; end

out

alpha

to

the

appropriate

length

beta = [ ] ; % b e t a w i l l c o n t a i n t h e row w e i g h t f o r e a c h row f o r i =1: length ( h ) f o r j =1:( f l o o r ( h ( i ) m) ) % a l w a y s u n d e r f i l l a n d t h e n a d d e x t r a s

% later

beta = [ beta , i ] ; end end while ( length ( beta ) = m) beta = [ beta i ] ; end
% Construction

fill

out

beta

to

the

appropriate

% length

for i = 1 : n
% construct column i

c = []; b e t a t e m p = beta ; for j = 1 : alpha ( i ) temp row = r a n d i n t ( 1 , 1 , [ 1 ,m ] ) ;

% We r u l e out a choosing the in same to we up row twice with for a one 10. column row the by % indicating selection beta temp We a l s o w e i g h t IF correct one too

% s e l e c t a row t h a t h a s y e t % POSSIBLE . However , s i n c e % column weight , we will end

equal i t s i n s i s t on with some

desired getting rows

having

% many e n t r i e s . C est la guerre . The a c t u a l row w e i g h t s t h u s % c o n s t r u c t e d are c a l c u l a t e d below . You s h o u l d c h e c k t h a t % they re not too far off ...

while ( ( ( b e t a t e m p ( temp row ) == 0 ) && . . . (max( b e t a t e m p ) > 0 ) ) | | . . . ( ( b e t a t e m p ( temp row ) <= 1)))

10

67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106

temp row = mod( temp row +1,m)+1; end c = [ c temp row ] ; b e t a t e m p ( temp row ) = 10; end
% decrement entries in beta

f o r k = 1 : length ( c ) beta ( c ( k ) ) = beta ( c ( k )) 1; end

% populate H

for j = 1 : alpha ( i ) H( c ( j ) , i ) = 1 ; end end

% Calculate actual column distribution

c o l u m n w e i g h t s = H ones (m, 1 ) ; f o r i = 1 :max( c o l u m n w e i g h t s ) c ou nt = 0 ; f o r j = 1 : length ( c o l u m n w e i g h t s ) i f ( c o l u m n w e i g h t s ( j )== i ) c ou n t = c ou n t + 1 ; end end f i n a l c o l u m n w e i g h t s ( i ) = c ou n t / length ( c o l u m n w e i g h t s ) ; end

% Calculate actual row weights

r o w w e i g h t s = H ones ( n , 1 ) ; f o r i = 1 :max( r o w w e i g h t s ) c ou nt = 0 ; f o r j = 1 : length ( r o w w e i g h t s ) i f ( r o w w e i g h t s ( j )== i ) cou nt = cou nt + 1 ; end end f i n a l r o w w e i g h t s ( i ) = coun t / length ( r o w w e i g h t s ) ; end

7 Example
In Listing 8 we show some exemplary use of the provided Matlab functions. Suppose we would like to generate a simple random 10 15 parity-check matrix, see what it looks like, and compute a generator matrix. Here we have n = 15, m = 10, and hence k = 5. In Figure 7 we see the structure of these two matrices.

11

Now let us generate a random message vector m Fn , encode this message to a 2 codeword x, and transmit it via an AWGN channel with standard deviation = 0.5. Now we could base a hard decision on this vector u and see if that yields a codeword, i.e., in Matlab we could try any(mod(H(double(u<0)),2)), which should return 1 if we have a codeword. We may nd at this stage that we do not have found a valid codeword yet. So let us perform belief propagation decoding, using the dynamical system implementation given in Section 4. Listing 8: Generating a parity-check and a generator matrix as in Section 7. Exercise the encoding, channel transmission and decoding.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % Setup a repeat 10 c o d e . % % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

n=10; m=n1; H=s p a l l o c ( n1,n , 2 ( n 1 ) ) ; f o r i =1:n1, H( i , i )=1; H( i , i +1)=1; end G=g e n e r a t o r m a t r i x (H ) ; spy (H) % s e e what H l o o k s spy (G) % s e e what G l o o k s H2DS(H ) ; % s e t u p f o r t h e BP a l g o r i t h m

like like

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % Alternatively : % setup a random l e n g t h 15 code of dimension 5.

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

H=g e n e r a t e H ( 1 0 , 1 5 , . 0 5 ) ; % 1 5 1 0 i s 5 a s we a l l k n o w
%% % at this s t a g e H might this step not have full rank , in that case we just % repeat %%

global H spy (H) % s e e what H l o o k s s a v e a s ( gcf , H.eps , eps ) ; G=g e n e r a t o r m a t r i x (H ) ; spy (G) % s e e what G l o o k s s a v e a s ( gcf , G.eps , eps ) ; H2DS(H ) ; % s e t u p f o r t h e BP a l g o r i t h m

like

like

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

12

38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86

% % Now g e n e r a t e , encode and channel code a random message % % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

[m, n]= s i z e (H ) ; k=nm; message=d o u b l e ( rand ( k , 1 ) > 0 . 5 ) ; % a k v e c t o r o f 0 s a n d 1 s x=G message ; % x i s now a n n v e c t o r o f 0 s a n d 1 s ( a c o d e w o r d ) x t i l d e=b p s k ( x ) ; % m a k e t h a t 1 s a n d 1 s ( c o d e w o r d i n c h a n n e l s y m b o l s ) i s c o d e w o r d ( h a r d d e c i s i o n ( x t i l d e ) ) % e v e r y t h i n g OK u p t o h e r e ?

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % Simulate transmission through a n AWGN c h a n n e l . Actually , we to % repeat as often as needed u n t i l the channel c o r r u p t s the % t r a n s m i t t e d c o d e w o r d , s o t h a t we w i l l h a v e t h e o p p o r t u n i t y % employ % % % REPEAT FROM HERE TO VIEW PLOTS OF DIFFERENT TRAJECTORIES % % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% the iterative decoder .

sigma = 0 . 5 ; % t h e c h a n n e l p a r a m e t e r
% generate a channel output which at first is not a codeword :

y=x t i l d e+randn ( n , 1 ) sigma 2 ; % t h e c h a n n e l a d d s t h e n o i s e TRIES=100; t =0; while i s c o d e w o r d ( h a r d d e c i s i o n ( y ) ) && t<TRIES , y=x t i l d e+randn ( n , 1 ) sigma ; t=t +1; end

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % % Compute the a p r i o r i of log l i k e l i h o o d ratios (u) in and perfrom T

% iterations

t h e BP a l g o r i t h m

( actually ,

our

implementation ,

% we d o T HALF i t e r a t i o n s . % % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

T=50; % ( h a l f ) i t e r a t i o n s

for

BP t o

perform

u=4y / ( 2 sigma 2 ) ; % c o m p u t e a p o s t e r i o r i LLRs i f any (mod(H d o u b l e ( y < 0 ) , 2 ) ) , y already represents a codeword

13

87 88 89 90 91 92 93 94 95 96 97 98

clf ; else performing BP Y=i t e r a t e B P (T, u ) ; % t h a t m e a n s T ( h a l f ) i t e r a t i o n s f o r k =1:T, i f any (mod(H d o u b l e (Y( : , k ) < 0 ) , 2 ) ) , s t r c a t ( found a codeword at iteration _ , num2str ( k ) ) break ; end end plot ( 0 : T, Y, o- ) end

0 1 2 3

8
4 5

10
6 7 8 9 10 11 0 2 4 6 8 nz = 60 10 12 14 16

12

14

16 0 2 nz = 29 4 6

Figure 1: The graphical output of Listing 8: H on the left and G on the right.

8 Remarks on numerical precision

In the graphical output generated in the previous section we observe that some trajectories seem to cease to exist and then possibly reappear later on. This eect is caused by round-o errors due to the numerical precision of the computations in Matlab. For very large arguments the tanh-function returns 1, and atanh(1) = . In theory though, nite arguments always give nite outputs, but number representation limits Matlab in actually producing accurate results after sometimes only a few iterations. Countermeasures employed, e.g., in [3], include to apply some saturation function after each iteration to the log-likelihood ratios, so that they do not grow unboundedly.

14

References
[1] S.-Y. Chung, G. D. Forney, T. J. Richardson, and R. L. Urbanke. On the design of low-density parity-check codes within 0.0045 dB of the Shannon limit. IEEE Communications Letters, 5(2):5860, Feb. 2001. [2] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger. Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47(2):498519, 2001. [3] R. M. Neal. Software for Low Density Parity Check (LDPC) codes. online, 2001. http://www.cs.toronto.edu/ radford/software-online.html. [4] T. Richardson and R. Urbanke. Modern Coding Theory. Cambridge University Press, New York, 2008. [5] B. S. Rer and C. M. Kellett. u Example Matlab code to implement Belief Propagation as a Dynamical System, Nov. 2008. Available at http://sigpromu.org/systemanalysis. [6] B. S. Rer, C. M. Kellett, P. M. Dower, and S. R. Weller. Belief Propagation as u a Dynamical System: The Linear Case and Open Problems. Submitted, Nov. 2008. Preprint available at http://sigpromu.org/systemanalysis/publications.html.

15