Professional Documents
Culture Documents
and Implementations
Module 1: LDPC Decoding
Ned Varnica
varnica@gmail.com
1
Overview
2
Error Correction Codes (ECC)
3
Error Correcting Codes (ECC)
Parity bits
User message bits (redundancy)
0000 0000000
1000 1101000
0100 0110100
1100 1011100 0010000
0010 1110010
1010 0011010 0000000
0110 1000110 4-bit message space 7-bit codeword space
1110 0101110
0001 1010001
1001 0111001
7-bit word space
0101 1100101
1101 0001101
0011 0100011
1011 1001011
0111 0010111
1111 1111111
4-bit message 7-bit codeword
Rate = 4 / 7
4
Linear Block Codes
Block Codes
• User data is divided into blocks (units) of length K bits/symbols
u1 u2 u3 u4 u5 ...
• Example:
– in Flash Devices user block length K = 2Kbytes or 4Kbytes is typical
– code rate R = K / N is usually ~0.9 and higher
5
Generator Matrix and Parity Check Matrix
g 00 g 01 ... g 0, N −1
g g11 ... g 1, N −1 encoding
G=
10
→ v = u ⋅ G
... ... ... ...
g K −1, 0 g K −1,1 ... g K −1, N −1 Codeword User message
6
Example
Encoding
1 1 0 1 0 0 0
0 1 1 0 1 0 0 u = (1 1 0 1)
G =
1 1 1 0 0 1 0 v = u ⋅ G = (0 0 0 1 1 0 1)
1 0 1 0 0 0 1
v ⋅ HT = 0
Decoding
1 0 0 1 0 1 1
vˆ = (0 0 0 1 1 0 0)
H = 0 1 0 1 1 1 0
vˆ ⋅ H T = (1 0 1)
0 0 1 0 1 1 1
7
Low-Density Parity-Check (LDPC) Codes
David MacKay
8
LDPC Codes
1 1 0 0 ... 0 1
0 0 0 1 ... 0 0
H = 0 0 1 0 ... 0 1
... ... ... ... ... ... ...
1 0 0 0 ... 1 0 Parity check equation
9
ECC Decoder Classification:
Hard vs Soft Decision Decoding
10
Hard vs. Soft Decoder Classification
Hard decoders only take hard decisions (bits) as the input
010101110 010101010
decoder
EDC BCH
encoder encoder
11
Hard vs. Soft Decoder Classification
Error-and-Erasure decoder is a variant of soft information decoder: in addition
to hard decisions, it takes erasure flag as an input
010101**0 010101010
decoder
EDC encoder
encoder
decisions {1,0,*}
“error-and-
EDC Front End
erasure”
decoder / Detection
decoder
12
Hard vs. Soft Decoder Classification
13
Hard vs. Soft Decoder Classification
• Each valid 11-bit SPC codeword c=(c0,c1,…c10) has the sum (mod 2) of all the
bits equal to 0
• The received vector does not satisfy SPC code constraint, indicating to the
decoder that there are errors present in the codeword
• Furthermore, assume that channel detector provides bit level reliability metric in
the form of probability (confidence) in the received value being correct
• From the soft information it follows that bit c4 is least reliable and should be
flipped to bring the received codeword in compliance with code constraint
14
Obtaining Hard or Soft Information
from Flash Devices
15
One Read: Hard Information
Decision bin Decision bin Decision Decision Decision bin Decision bin
A1 B1 bin C1 bin C0 B0 A0
-2
10
SFR
Hard, single
pass decoder
-3
10
Hard input,
soft ITR decoder
-4
10
Soft input,
Soft ITR decoder
7.6 7.8 8 8.2 8.4 8.6 8.8 9 9.2 9.4 9.6 9.8
SNR [dB]
Raw BER
18
Decoding LDPC Codes
19
Representation on Bi-Partite (Tanner) Graphs
0 0 0 0 0 parity check constraints
check nodes
variable nodes
0 1 0 1 1 0 encoded bits
Each bit “1” in the parity check matrix is represented by an edge between
corresponding variable node (column) and check node (row)
0 1 0 0
1 0
1 1 0 0
1 0
H = 0 1 1 0
0 1
1 0 0 0
1 0
0 0 1 1 1 1
20
Hard Decision Decoding: Bit-Flipping Decoder
End
First
Second
of step
firststep
step
01 10 1
0 01 0
Valid
codeword
0
1 10 0 1 1 0
TheThe
second
Examine
left-most
bit from
number
bit Flip
is
the
the
the
left
of
Flip
only
second
unsatisfied
isthe
the
bitleft-most
that
only
bit has
from
bit
check
that
bit
2the
unsatisfied
neighbors
has
left2 unsatisfied
check
for each
neighbors
check
bit neighbors
21
Bit-Flipping Decoder Progress on a Large
LDPC Code
Decoder starts with a relatively large number of errors
As decoder progresses, some bits are flipped to their correct values
Syndrome weight improves
• As this happens, it becomes easier to identify the bits that are erroneous and
to flip the remaining error bits to actual (i.e. written / transmitted) values
Number
of bit errors 91
50
21
10
6 4 2 0
Iteration
Syndrome
weight
310
136
60
26 14 10 6 0
1 2 3 4 5 6 7 Iteration
22
Soft Information Representation
The information used in soft LDPC decoder represents bit reliability metric,
LLR (log-likelihood-ratio)
P (bi = 0)
LLR (bi ) = log
P(bi = 1)
The following chart shows how to convert LLRs to probabilities (and vice
versa)
23
Soft Information Representation
Bit LLR>0 implies bit=0 is more likely, while LLR<0 implies bit=1 is more likely
0.9
0.8
0.7
0.6
P(bl=1)
P (bi = 0) 0.5
0.4
0.3
0.2
0.1
0
-100 -80 -60 -40 -20 0 20 40 60 80 100
LLR
24
Soft Message Passing Decoder
Bit nodes
(columns of H) n = 0 n=1 n=2 n=3 n=4 n=5 n=6
25
Soft LDPC Decoder
There are four types of messages
• Message from the channel to the n-th bit node L n
• Message from n-th bit node to the m-th check node Q n( i−) > m
• Message from the m-th check node to the n-th bit node R ( i )
m −>n
• Overall reliability information for n-th bit-node at the end of iteration Pn( i )
Q 3(i−)>1
P 6(i )
Channel
ChannelDetector
Detector
26
Soft LDPC Decoder (cont.)
27
Soft LDPC Decoder (cont.)
(i )
Bits-to-checks pass: Qn −> m : n-th bit node sums up all the information it has
received at the end of last iteration, except the message that came from m-th
check node, and sends it to m-th check node
• At the beginning of iterative decoding all R messages are initialized to zero
R 0i −1>3
− i −1
R 2 −>3
∑
m '≠1
L3
Channel
ChannelDetector
Detector
28
Soft LDPC Decoder (cont.)
Checks-to-bits pass:
• Check node has to receive the messages from all participating bit nodes
before it can start sending messages back
• Least reliable of the incoming extrinsic messages determines magnitude of
check-to-bit message. Sign is determined so that modulo 2 sum is satisfied
m m
n1 n2 n3 n1 n2 n3
29
Soft LDPC Decoder (cont.)
At the end of each iteration, the bit node computes overall reliability information
by summing up ALL the incoming messages
Pn( i ) = Ln + ∑ Rm(i−>
)
n
m
Pn(i ) ’s are then quantized to obtain hard decision values for each bit
) 1, if Pn < 0
(i )
xn =
0, else
30
LDPC Decoder Error Correction: Example 1
-12 +7 +4
+4 -11
-9 +7 +4 +10
-4 -10
-7 +4 +4
+4 -4 -7 -4
31
LDPC Decoder Error Correction: Example 1
-4 -10
-7 +4 +4
+4 -4 -7 -4
P: -5 +3 -8 -20 +3 +6 -7
⌢
HD: 1 0 1 1 0 0 1
32
LDPC Decoder Error Correction: Example 2
-12 +7 -4
-4 -11
-9 -7 -4 +10
+4 -10
+7 -4 -4
+4 +4 -7 +4
33
LDPC Decoder Error Correction: Example 2
-12 +7 -4
-21 -11
-9 -7 -7 +10
+7 -10
+7 -7 -4
+7 +9 -7 +4
34
LDPC Decoder Error Correction: Example 2
+7 -10
+7 -7 -4
+7 +9 -7 +4
HD: 1 0 1 1 0 0 1
35
Sum-Product and Min-Sum Decoders
Instead of these update rules, simple approximate rules have been devised: The
rules require only computing minimum messages at each check node
• In order to make approximation work, it is necessary/critical to utilize
scaling/offsetting of messages from check to bit nodes
This algorithm is widely known as min-sum with scaling/offset and is often
choice of implementation in Hardware
m m
Q ni1−> m = −10
n1 n2 n3 n1 n2 n3
36
Histogram of LLRs on Large LDPC Codes
Decision bin Decision bin Decision Decision Decision bin Decision bin
A1 B1 bin C1 bin C0 B0 A0
900 Number
of bit errors 442
294
224
166
104
53
2 0
Iteration
Syndrome
weight
0
-60 240
874
Iteration counter
420
7
6
5
4
3
2
1
0 322
266
196
116
8 0
1 2 3 4 5 6 7 Iteration
Performance / Complexity Trade-Offs
The choice of number of iterations is typically made
with consideration of the following parameters:
Throughput / Latency
SNR performance (Capacity gain)
SNR
Implementation Complexity performance
Implementation
Complexity
(Capacity gain)
Power Consumption
0
10 Throughput Power
& Latency Consumption
System
-1 Performance
10
-2
10
SFR
SNR [dB]
Raw BER
39
Code Design, Code Performance Characteristics
and Efficient Hardware
40
Quasi-Cyclic LDPC Codes
41
Layered / Flooding Decoder
Both of these decoders benefit from structured matrices that naturally allow for
parallel processing of a portion of the matrix, i.e. parallel processing of some
number of rows / columns in the matrix
42
LDPC Iterative Decoder
Performance Characteristics
43
RS-ECC Performance Characterization
Sector failure rate of RS ECC keeps falling at exponential rate with SNR increase
• No flattening of SFR vs. SNR curve is observed at higher SNR’s
44
LDPC Decoder Performance Characterization
45
LDPC ITR Decoder Error Rate Characteristics
Waterfall region
• BER/SFR drops rapidly with small change in SNR
Error Floor (EF) region (High SNR region)
• BER/SFR drop is much slower
Specific structures in the LDPC code graph lead to decoding errors at high SNRs
• structures known as Near-Codewords (trapping sets) are dominant in the EF region
LDPC RS/BCH
LDPC system
operating region
waterfall region
SyER/SFR
SNR [dB]
46
Mitigating Error Floor of LDPC Code
Retry mode performance of Iterative code; Rate .89; 0.5K sector size;
0
10
Bit-true Simulator
Code design can be tailored to
On-The-Fly Error Floor achieve the error floor bellow
Error Floor After Retry
-5
HER requirements
10
-15
10
-20
10
17 17.5 18 18.5 19
SNR[dB]
47
Summary
Iterative LDPC codes can enable FLASH industry to hit new capacity milestones
Types of Decoders:
• Hard: Bit-Flipping Decoders
• Soft: Sum-Product (Min-Sum) Decoders
Soft message passing decoders offer large SNR gains – this translates to
capacity gains
48
APPENDIX
49
LDPC Iterative Error Rate Characteristics
Iterative decoder gets trapped into one of NCW’s, and is unable to come out of
this steady state (or oscillating state)
• Even if decoder has time to run 100’s of iterations, it would not be able to
come out of the trapping set
50
Code Selection
SFR profile
as a function of code/decoder selection
0
10
lower error floor
higher error floor
Optimizing code/decoder selection
-2
10 based on the performance at low
SNR’s only may lead to impractical
-4 code selection.
10
-6
10
between performance at low SNR’s,
-8
defect correction, and error floor
10
(performance at high SNR’s)
-10
10
-12
10
17 17.5 18 18.5 19
SNR
51
Mis-Correction in LDPC Decoding
52
Minimum Distance of a Code
The minimum of all the distance between any two code words is called the
minimum distance of the code, denoted by dmin
dmin
53
Decoder Miscorrection
Received sequence
transmitted code word
Decoder Error
54
Iterative Error Rate Characteristics
Production grade devices will operate in the Error Floor region (High SNR
region)
mis-correction
SNR [dB]
55