Irregular Low-Density Parity-Check Lattices

Ihn-Jung Baik and Sae-Young Chung

School of EECS, KAIST, Daejeon, Korea,
Email: ihn jung@kaist.ac.kr, sychung@ee.kaist.ac.kr
AbstractWe construct lattices with high coding gains based
on nested low-density parity-check (LDPC) codes by Construc-
tion D

. We generalize the LDPC lattices [1] to have irregular

degrees and call them irregular LDPC lattices. To construct good
irregular LDPC lattices, we optimize the degree distributions
by using density evolution and a modied sequential quadratic
programming (SQP) and show our optimized irregular LDPC
lattice has a threshold 0.48dB from the capacity, which is
about 0.6dB better than the regular one in [1]. To construct
a Tanner graph corresponding to LDPC lattices, we generalize
the progressive-edge growth (PEG) algorithm and show a lower
bound on the girth of the graph. Simulation is performed using
the sum-product algorithm. Also, we compare the performance
of joint decoding and multi-stage decoding [2] and show the
advantage of joint decoding.
I. INTRODUCTION
Lattices are useful in many coding applications. In [3], lat-
tice encoding and decoding was shown to achieve the additive
white Gaussian noise (AWGN) channel capacity. However,
such achievability proofs rely on random construction of
lattices and it is not clear how to construct a good lattice
with low complexity. In this paper, we focus on constructing
good lattices to achieve such a goal.
Sadeghi et al. [1] proposed regular LDPC lattices based
on lattice construction D

, a multi-level construction of lat-

tices using linear codes [4], evaluated their performance, and
showed their codes perform within 3dB of the Shannon limit
at the symbol error rate of 10
6
. In this paper, we generalize
the approach in [1] to construct irregular LDPC lattices and
evaluate their performance. An irregular LDPC lattice can
have multiple different degrees for non-binary variable and
check nodes and different edge types [5]. By using the density
evolution [6] for the non-binary variable and check nodes
with multiple edge types and applying a modied SQP [7],
we optimize the degree distributions for the irregular LDPC
lattices. The optimized irregular LDPC lattice has a threshold
0.48dB from capacity, which is about 0.6dB better than the
regular one in [1].
We construct a parity-check matrix for our lattice by gener-
alizing the PEG algorithm [8] [1] that constructs a parity-check
matrix in an edge-by-edge manner. By using this algorithm,
we can get a lower bound on the girth of the constructed
graph. For the decoding, we use the sum-product algorithm
for jointly decoding multiple levels of our LDPC lattices.
Our simulation result also shows that the performance of our
optimized irregular LDPC lattice is about 0.56dB better than
that of the regular one in [1] when the block length is 5000.
We also consider a multi-stage decoding [2] that decodes
each level of LDPC lattices sequentially by subtracting previ-
ously decoded layers, i.e. successive interference cancelation
(SIC). Since each layer is a binary LDPC code, it can simplify
decoding at the expense of degraded performance over the
joint decoding of multiple levels described above. For the
multi-stage decoding, we try two possible decoding orders
and calculate the thresholds for decoding the most signicant
bit (MSB) and the least signicant bit (LSB) using density
evolution and compare their performances.
II. IRREGULAR LDPC LATTICES
A. Lattices
A lattice is a discrete subgroup of R
n
that spans the real
vector space R
n
. The fundamental Voronoi region [2] V of
is a set of minimum-energy points a V in its coset + a
such that every point x R
n
can be uniquely represented as
x = + r, where , r V. This requires we break ties
arbitrarily when deciding the set of minimum energy points.
The lattice signal-to-noise ratio, SNR
L
, for the AWGN channel
is dened in [9] as
SNR
L
=
det()
2/n
n
2
V
2/n
n
, (1)
where det() is the volume of the fundamental Voronoi
region,
2
is the noise variance and V
n
is the volume of a
n-dimensional sphere with radius 1. This is asymptotically
equal to the volume-to-noise ratio [2]
det()
2/n
2e
2
since nV
2/n
2e as n .
B. Construction D

Construction D

is a multi-level construction of lattices

using nested linear codes, which converts a set of parity-
checks to congruences for a lattice. Let C
0
, C
1
, , C

be
binary linear codes satisfying C
0
C
1
C

. Each
code C
i
has parameters [n, k
i
], where n is the block length
and k
i
is the length of information for C
i
. Code C
i
has a
parity-check matrix having linearly independent parity-check
row vectors h
1
, h
2
, . . . , h
r
i
with components 0 or 1, where
r
i
= n k
i
and r
1
= 0. We dene the parity-check matrix
H of the lattice constructed by Construction D

as
H = [h
1
; . . . ; h
r
0
; 2h
r
0
+1
; . . . ; 2h
r
1
; . . . ; 2

h
r
1
+1
; . . . ; 2

h
r

].
(a) Two-level regular (b) Two-level irregular
(c) General irregular
Fig. 1. Parity-check matrix of LDPC lattices
The components of the parity-check matrix H of are not
binary anymore and their values depend on the level. We can
get the following relationship of the parity-check matrix H
and a lattice point x .
Hx
T
0 mod 2
+1
, (2)
where ( + 1) is the number of levels in H. The components
of the j-th row of H, i.e. 2
i
h
j
consist of 0 or 2
i
for all
r
i1
+ 1 j r
i
, where i = 0, 1, . . . , .
For example, we consider a 3-level lattice, i.e. = 2. Codes
C
0
, C
1
and C
2
are nested and their parity-check matrices are
{1010}, {1010, 1100, 0110} and {1010, 1100, 0110, 1001}, re-
spectively. If
H =
_

_
1 0 1 0
2 2 0 0
0 2 2 0
4 0 0 4
_

_
, (3)
then x belongs to if and only if Hx
T
0 mod 2
3
. As an
example, (4,0,4,0) and (6,2,2,0) are valid codewords for the
above parity-check matrix.
C. Irregular LDPC lattices
If the parity-check matrix H of a lattice has few non-
zero elements, then we call it a LDPC lattice. Regular LDPC
lattices were introduced in [1], where the number of non-
zero elements in each row is the same for all rows and the
number of non-zero elements in each column is the same in
all columns. In this section, we generalize it to irregular LDPC
lattices. The parity-check matrix of a general irregular LDPC
lattice is shown in Fig. 1(c), which consists of submatrices
H
ij
that are regular, i.e. the number of non-zero elements is
the same in all rows and the same in all columns. To dene
irregular LDPC lattices, we introduce the following notations.
The polynomial representation of the degree distribution for
variable nodes is
(x
0
, x
1
, , x

) =

j0,j1, ,j

j
0
,j
1
, ,j

x
j
0
0
x
j
1
1
x
j

, (4)
where j
i
, i {0, 1, . . . , } is the number of non-zero elements
in the part of a column of H corresponding to the (i + 1)-th
level and
j
0
,j
1
, ,j

is the number of columns in H that have

(j
0
, j
1
, , j

) non-zero elements in the corresponding levels.

For a column in the submatrix (H
T
0j
H
T
1j
. . . H
T
j
)
T
of H, we
set j
i
= d
i,j
s
, i {0, 1, . . . , }, where d
i,j
s
is the variable
degree of H
ij
. Similarly, the polynomial representation of the
degree distribution for check nodes is
P(x
0
, x
1
, , x

) =

i=0

k
P
i,k
x
k
i
, (5)
where P
i,k
is the number of rows in the (i + 1)-th level
that have k non-zero elements. We assume that check nodes
in the same level have the same degree motivated by the
fact that right-regular distributions are often close to optimal.
Therefore, P
i,j
=

k
P
i,k
for some j. As an example, for the
parity-check matrix (3), the polynomial representations are
(x
0
, x
1
, x
2
) =
1,1,1
x
0
x
1
x
2
+
0,2,0
x
2
1
+
1,1,0
x
0
x
1
+
0,0,1
x
2
= x
0
x
1
x
2
+ x
2
1
+ x
0
x
1
+ x
2
P(x
o
, x
1
, x
2
) = P
0,2
x
2
0
+ P
1,2
x
2
1
+ P
2,2
x
2
2
= x
2
0
+ 2x
2
1
+ x
2
2
The size of submatrix H
ij
is P
i

d
0,j
s
,d
1,j
s
, ,d
,j
s
, where P
i
is

k
P
i,k
. Then, the variable degree of (H
T
0j
H
T
1j
. . . H
T
j
)
T
is
d
j
s
=

i=0
d
i,j
s
and the check degree of H
i
= (H
i0
, . . . , H
it
)
is d
i
c
=

t
j=0
d
i,j
c
, where d
i,j
c
is the check degree of H
ij
.
Also, r
i
=

i
j=0
P
j
for i {0, 1, , } and from this we
get the volume of a Voronoi region as [1]
det() = 2

i=0
ri
. (6)
We know that in LDPC codes if there are variable nodes
with degree 1, there would be a large error oor. Thus, we
consider a parity-check matrix of an irregular LDPC lattice
with variable degrees d
0,j
s
for all j {0, 1, . . . , t} larger than
or equal to 2. The rate of the code is given by
R
total
= + 1

i
R
i
=

i
R
i
,
(7)
where R
i
=

(j
0
,j
1
,...,j

)

(j
0
,j
1
,...,j

)
k
i
nd
i
c
+ R
i1
and R
i
= 1
R
i
, i {0, 1, . . . , } is the rate for the (i + 1)-th level and
R
1
is 0.
Parity-check matrices for two-level regular and irregular
LDPC lattices are shown in Fig. 1(a) and (b), respectively.
The parity-check matrix of two-level lattices considered in [1]
consists of two submatices H
0
and H
1
. The number of non-
zero elements in each column is the same and the number of
Fig. 2. Multi-edge type Tanner graph for irregular LDPC lattice
non-zero elements in each row is the same. Unlike the regular
case, the parity-check matrix of a two-level irregular LDPC
lattice shown in Fig. 1(b) can have different number of non-
zero elements in each column. Fig. 2 shows an example of the
Tanner graph corresponding to the parity-check matrix of Fig.
1(b), where d
0
c
= 4, d
1
c
= 3 and d
0,0
s
= 2, d
0,1
s
= 2, d
0,2
s
=
3, d
1,0
s
= 0, d
1,1
s
= 1, d
1,2
s
= 2. In the Tanner graph,
0
and

1
denote random permutations.
D. Generalized Progressive Edge Growth (PEG) Algorithm
To construct nested LDPC codes that generate an irregular
LDPC lattice, we generalize the PEG algorithm [8] [1]. The
generalization is trivial, so we do not describe the algorithm.
The girth of a graph is the length of the shortest cycle
contained in the graph. Let g
i
denote the minimum girth of
the (i+1)-th level of the code. Then, the girth g of the Tanner
graph of is min{g
0
, g
1
, . . . , g

} and the girth of each level

has the relationship g
0
g
1
g

. A lower bound
on the girth of Tanner graph which is generated by the PEG
algorithm is [8]
g 2
_
_

_
log
_
r

d
max
c

r

d
max
c
d
max
s
r

+ 1
_
log(d
max
s
1)(d
max
c
1)
1

_
+ 2
_
_
, (8)
where d
max
s
and d
max
c
are the maximum variable node degree
and check node degree, respectively and the base of log is 2.
E. Decoding Algorithm
For the decoding, we apply the sum-product algorithm
(SPA) [10], which can do symbol-by-symbol maximum a
posteriori decoding if there is no cycle. In SPA, decoding
is performed by exchanging messages between variable and
check nodes iteratively. To compute the initial messages from
the channel, we use the following.
p(y
i
|x
i
) =
5

k=5
exp
_

(y
i
x
i
k)
2
2
2
_
, (9)
where is a normalization constant, y
i
is the i-th component
of the received vector y,
2
is the noise variance per dimen-
sion, and = 2
+1
. At variable and check nodes, the SPA
operates as follows.
Fig. 3. Optimized parity-check matrix form
1) At variable nodes, a likelihood p(y
i
|x
i
) is used for
decoding under the assumption that all codewords are equally
likely and independent channel observations.
p
M
s
i
ch
j
(x
i
= u)
= p(y
i
|x
i
= u)

k{k|h
k,i
=0,k=j}
p
M
ch
k
s
i
(x
i
= u),
(10)
where h
k,i
is the (k, i)-th element of the parity-check matrix
and M
s
i
ch
j
and M
ch
k
s
i
denote the message from the i-th
variable node to the j-th check node and the message from
the k-th check node to the i-th variable node, respectively and
u is one of {0, 1, . . . , 1}.
2) At check nodes in the l-th level, the following operation
is performed.
p
M
ch
j
s
i
(x
i
= u)
=

k{k|uk mod 2
l+2
}
p

M
ch
j
s
i
(x
i
= k)
2
l1
,
(11)
where
p

M
ch
j
s
i
(x
i
= u)
=

k{k|h
j,k
=0,k=i}
p
M
s
k
ch
j
(x
k
= u
k
),
where the summation is over u
k
s such that u +

k
u
k

0 mod 2
+1
.
3) Finally, at variable nodes, the message is decoded as
follows.
x
i
= arg max
u
{p(x
i
= u)}
= arg max
u
_
_
_
p(y
i
|x
i
= u)

k{k|h
k,i
=0}
p(x
i
= u)
_
_
_
.
(12)
The number of operations (addition and multiplication) in each
iteration of sum-product algorithm is

(j
0
,j
1
,...,j

(j
0
,j
1
,...,j

)
2
+1
_

l=0
j
l
1
_

l=0
j
l
+

l=0

i
P
l,i
(2
+1
)
i1
(i 1) 2
l+2
,
(13)
TABLE I
OPTIMIZED DEGREE DISTRIBUTIONS

i
0
,i
1
/ns optimized value
i
0
,i
1
/ns optimized value

2,0
/n 0.42469
4,1
/n 0.04760

2,1
/n 0.16578
4,2
/n 0.06271

3,0
/n 0.23295
7,0
/n 0.00914

3,2
/n 0.00937
7,2
/n 0.04776
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
10
6
10
5
10
4
10
3
10
2
10
1
10
0
SNR [dB]
P
r
o
b
a
b
ilit
y

o
f

s
y
m
b
o
l
e
r
r
o
r


Regular simulation
Irregular simulation
Regular threshold
Irregular threshold
Fig. 4. Threshold of irregular and regular LDPC lattices and simulation
result for them when the block length is 5000. The Shannon limit is 0 dB.
where d
si
is the degree of variable node s
i
and d
c
k
is the
degree of check node c
k
.
F. Optimization
For two-level irregular LDPC lattices, we optimize the de-
gree distributions by generalizing density evolution for multi-
edge type LDPC codes [5] to irregular LDPC lattices. Without
loss of generality, we assume the all-zero codeword is sent.
The algorithm used for non-linear optimization is SQP [7],
which is a generalization of Newtons method. We set the
check node degree to d
0
c
= 5 for the rst level and d
1
c
= 3
for the second level. These check node degrees are optimally
chosen among many candidates. As shown in Fig. 3, we
optimize the set of degree distributions
_
2,0
n
,
2,1
n
, . . . ,
7,2
n
_
.
Rates of the rst and the second levels for the optimized degree
distributions are 0.45 and 0.3, respectively. We constraint the
total rate R
total
to be 0.75. The optimized degree distributions
are shown in Table I.
G. Comparison of simulation results between irregular and
regular LDPC lattices
Based on the optimized degree distributions in Table I,
we compare the thresholds of the two-level irregular LDPC
lattice and the (2,3;4)-regular LDPC lattice [1]. As shown in
Fig. 4, the threshold of the regular LDPC is 1.08dB and that
of the irregular LDPC is 0.48dB showing a gain of 0.6 dB
over the regular one. The curves in Fig. 4 are the simulated
error performance for the two codes when n = 5000 and
the maximum number of iteration is 40. Both curves are
about 0.7dB away from their corresponding thresholds at the
probability of symbol error of 10
6
. The lower bound of the
girth of our optimized irregular LDPC lattice is 6 and that of
the regular one is 10. Despite this, our irregular lattice has a
better threshold.
H. Joint vs. multi-stage decoding
In this section, we compare the performance of joint decod-
ing and multi-stage decoding. In joint decoding, we decode all
levels jointly as was done in Section II. E.
Multi-stage decoding with SIC for a lattice can be done
similarly as in [2]. Since each layer of our irregular lattice
is a binary LDPC code, decoding of such codes is simpler.
Therefore, the multi-stage decoding can be a good alterna-
tive for practical applications. Furthermore, ideal multi-stage
decoding with SIC is without loss of optimality [2].
A lattice point x for our irregular LDPC lattice can
be uniquely written as
x = x

+ 2x
1
+ + 2

x
0
+ 2
+1
w, (14)
where x
i
, i = 0, 1, . . . , is a length-n vector with binary
{0, 1} elements such that x
0
and x

correspond to the MSB

and the LSB, respectively and w is any lattice point in the n-
dimensional lattice Z
n
. Since we focus on sending information
only through {x
0
, x
1
, . . . , x

}, we will not consider w from

now on.
(14) can be considered as a superposition coding, i.e., we
can consider x
i
s are separately coded and then they are added.
Figure 5 shows the encoding x = x

+ 2x
1
+ + 2

x
0
and multi-stage decoding with SIC as was done in [2], where
z N(0,
2
I) denotes the AWGN with covariance matrix

2
I and y is the received vector. Due to the presence of w, we
assume y = (x+z) mod 2
+1
Z
n
. There can be two decoding
orders, i.e., LSB rst and MSB rst. Figure 5 is show the
former. We describe the two scenarios below.
We rst consider decoding the LSB rst as shown in
Figure 5, i.e., the decoding order is given by x

x
1

x
0
. For every decoding stage, we assume that the
previously decoded blocks are removed from the received
vector, i.e. SIC. Since x satises (2), we get
(H
T
0
H
T
1
. . . H
T

)
T
(x

+ 2x
1
+ + 2

x
0
)
T
0 mod 2
+1
.
(15)
When decoding x
i
, we assume that x

, x
1
, . . . , x
i+1
have
been perfectly decoded as x

, x
1
, . . . , x
i+1
without errors
and removed from the received vector. By taking mod 2
i+1
on both sides of (15), we get
_
H
T
0
(2
1
H
T
1
) . . . (2
i
H
T
i
)

T
x
i
T
0 mod 2 (16)
for i {0, 1, . . . , }, which can be used to decode x
i
. Note
that we only need mod 2 operations in (16), which means the
codes at each level are binary LDPC codes and we can use
conventional decoding algorithms for such codes. Note that
taking mod2
i+1
on both sides of (15) would be without
loss of optimality in the limit of large dimension (Lemma 8
in [2]). To see this, consider a multiple-access channel (MAC)
Y = (

i=0
2
i
X
i
+ Z) mod 2
+1
, where X
i
s are binary
and Z is any additive noise. Due to symmetry, the optimal
input distributions for this channel are given by X
i
{
1
2
,
1
2
}
Fig. 5. Superposition coding and multi-stage decoding with SIC.
for all i. Now, we get the following chain rule:
I(X
0
, . . . , X

; Y ) =

i=0
I(X
i
; Y |X

, X
1
, . . . , X
+1i
).
(17)
Due to the presence of X
1i
, . . . , X
0
, we can replace X
i
by X
i
mod 2
i+1
in (17), i.e.,
I(X
i
; Y |X

, . . . , X
+1i
)
= I(X
i
mod 2
i+1
; Y |X

, . . . , X
+1i
). (18)
There can be a loss in performing such an operation (taking
mod 2
i+1
on both sides of (15)) if n is nite. Since n is
typically large, such a loss can be very small.
We now consider decoding the MSB rst, i.e., the decoding
order is given by x
0
x
1
x

. Ideally, this is also

without loss of optimality (p. 828 [2]), which can be seen from
the following chain rule:
I(X
0
, . . . , X

; Y ) =

i=0
I(X
i
; Y |X
0
, X
1
, . . . , X
i1
). (19)
Assuming x
0
, x
1
, . . . , x
i1
have been decoded successfully
and subtracted from the received vector, we can use the
following for decoding x
i
, i {0, 1, . . . , }.
_
H
T
0
H
T
1
. . . H
T
i
_
T
(x

+ 2x
1
+ + 2
i
x
i
)
T
0 mod 2
+1
.
(20)
For our optimized two-level codes, let th
0
and th
1
denote
the thresholds for decoding the MSB and the LSB, respec-
tively. Let us rst consider the case of decoding LSB rst.
When computing th
0
, we assume hypothetically that decoding
of LSB is successful. Therefore, th
0
would be lower than the
threshold for joint decoding. The thresholds (SNR
L
in dB)
are given in Table II. We also show the Shannon limits for
the LSB and MSB for both irregular and regular cases. We
observe that th
0
and th
1
are about 0.4 1 dB from the limit.
Note that the performance of the overall multi-stage decoding
will be determined by the worst of the two threshold, which is
th
1
in these examples. The reason for the better performance
of the regular LDPC lattice in the case of LSB-rst multi-
stage decoding is because the optimized degree distributions
TABLE II
THRESHOLD (SNR
L
IN DB) COMPARISON OF THE JOINT DECODING AND
MULTI-STAGE DECODING (THE VALUES IN PARENTHESIS ARE THE
SHANNON LIMITS.)
h
h
h
h
h
h
h
h
h
h
hh
Decoding algorithm
Lattice
Irregular Regular
Joint decoding 0.48 1.08
SIC th
0
= -1.19 (-1.87) th
0
= 0.47 (-1.51)
(LSB MSB) th
1
= 3.44 (2.98) th
1
= 3.37 (2.54)
SIC th
0
= 1.93 th
0
= 4.53
(MSB LSB) th
1
= -3.09 th
1
= -51.87
used in Table I are optimized for joint decoding not for the
multi-stage decoding. In the case of decoding MSB rst, the
bottleneck is the MSB and the calculated thresholds are also
given in Table II.
ACKNOWLEDGMENT
This work is in part supported by KOSEF (R01-2006-000-
11112-0).
REFERENCES
[1] M. R. Sadeghi, A. H. Banihashemi, and D. Panario, Low-density parity-
check lattices: construction and decoding analysis, IEEE Trans. Inf.
Theory, vol. 52, pp. 44814495, Oct. 2006.
[2] G. D. Forney, M. D. Trott, and S.-Y. Chung, Sphere-bound-acheiving
coset codes and multilevel coset codes, IEEE Trans. Inf. Theory, vol. 46,
May 2000.
[3] U. Erez and R. Zamir, Achieving
1
2
log(1 + SNR) on the AWGN
channel with lattice encoding and decoding, IEEE Trans. Inf. Theory,
vol. 50, pp. 22932314, Oct. 2004.
[4] J. H. Conway and N. J. A. Sloane, Sphere Packings, Lattices and Groups.
New York: Springer, 1998.
[5] T. J. Richardson and R. L. Urbanke, Multi-edge type LDPC codes,
submitted to IEEE Trans. Inf. Theory.
[6] T. Richardson, A. Shokrollahi, and R. Urbanke, Design of provably
good low-density parity-check codes, in IEEE Int. Symp. Inform. Theory
(ISIT), Sorrento, Italy, 2000.
[7] M. J. D. Powell, A fast algorithm for nonlinearly constrained optimiza-
tion calculations, Numerial Analysis, ed. G. A. Watson, Lecture Notes
in Mathematics, vol. 630, 1978.
[8] X. Y. Hu, E. Eleftheriou, and D. H. Arnold, Regular and irregular
progressive edge-growth Tanner graphs, IEEE Trans. Inf. Theory,
vol. 51, pp. 386398, Jan. 2005.
[9] V. Tarokh and A. Vardy, Universal bound on the performance of lattice
codes, IEEE Trans. Inf. Theory, vol. 45, pp. 670681, Mar. 1999.
[10] F. R. Kschischang, L. J. Frey, and H. A. Loeliger, Factor graphs and the
sum-product algorithm, IEEE Trans. Inf. Theory, vol. 47, pp. 498519,
Feb. 2001.