Submitted to IEEE Transa

tions on Information Theory.

1

Analyti Expressions for the Bit Error Probabilities of

Rate 1/2 Memory 2 Convolutional En oders1
M. Lentmaier, D. V. Truha hev, and K. Sh. Zigangirov

Department of Information Te hnology

Lund University, P.O. Box 118, SE-22100 Lund, Sweden
Email: {mi hael,dimitri,kamil}it.lth.se

Abstra t
Analyti expressions for the exa t bit error probabilities of rate R = 1=2, memory m = 2 onvolutional
en oders are derived for a maximum likelihood (ML) de oder and transmission over the binary symmetri
hannel (BSC). The resulting expressions are rational fun tions of the rossover probability of the BSC.
In addition to lassi al non-systemati en oders without feedba k we onsider also re ursive systemati
en oders, whi h be ame espe ially important as omponent en oders in on atenated oding s hemes. To
attest the validity of the results, they are ompared to omputer simulations. Based on the presented
te hnique also the bit error probability and the probability distribution of the output log-likelihood ratios
of the Max-Log-MAP algorithm are derived in analyti form.

I. Introdu tion

Convolutional odes an be found in many appli ations. Their spe ial stru ture makes
them simple to en ode and the Viterbi algorithm [1℄[2℄ provides a very eÆ ient method
for maximum-likelihood (ML) de oding. For very good hannels the performan e of ML
de oded onvolutional odes an be estimated quite a urately with union bound te h-
niques. However, it gets more and more diÆ ult to obtain tight bounds on the error
probability when the rossover probability p of the hannel gets larger. But this region
is parti ularly interesting in iterative de oding s hemes, where several simple omponent
de oders together try to rea h the orre ting apability of a large overall ode. Therefore
it an be of great theoreti al value to get some insight into the de oders and analyze the
de oding performan e even for onvolutional odes with small memory.
That nite Markov hain theory an be used for the error probability al ulation of a
onvolutional ode has been observed by Morrissey already in the late 1960's. He al ulated
1 This work was supported in part by Swedish Resear h Coun il for Engineering S ien es (Grant 01-3123).

O tober 21, 2002 DRAFT

 2

the exa t bit error probability of systemati en oders without feedba k for a suboptimum
feedba k de oder [3℄. For a memory m = 1 en oder it has been demonstrated that this
approa h an be extended to ML de oding [4℄. A drawba k of Morrissey's model is that
the omplexity grows not only with the ode memory but also with the de oding delay.
A method to al ulate the event error probability of onvolutional odes was developed
by S halkwijk et. al. [5℄. Their analysis is based on a de oder operating in the syndrome
former trellis, whi h an be shown to be equivalent to a onventional Viterbi de oder
[6℄. For ertain en oders this de oding method redu es the number of possible metri
values, whi h allows savings in hardware implementations. The Viterbi algorithm is not
only appli able for hannel de oding but also for sour e en oding. The sour e en oding
performan e of binary onvolutional odes was investigated by Calderbank et. al. [7℄,
also based on some Markov hain approa h. The te hnique from [7℄ was then used by
Best et al. [8℄ to evaluate the exa t bit error probability, Pb , for onvolutional odes with
Viterbi de oding. An expli it analyti expression of Pb has been derived in [8℄ for a simple
memory m = 1 onvolutional en oder without feedba k. For odes with memory m = 2
and m = 3 only numeri al results have been presented. In the present paper we al ulate
and make available orresponding analyti expressions for a memory m = 2 onvolutional
ode, using the ideas from [8℄.
Let us onsider a binary, rate R = 1=2 onvolutional ode. The information sequen es
u = : : : u 1 u0 u1 u2 : : : , ut 2 GF (2), are mapped by a onvolutional en oder into ode se-
quen es v = : : : v 1 v0 v1 v2 : : : , where v t = (vt(1) vt(2) ) and vt(1) ; vt(2) 2 GF(2). The mapping
an be des ribed by a generator matrix G(D) = (g1 (D); g2(D)), where g1 (D) and g2 (D),
in general, are rational transfer fun tions. Then the D-transform, v (D), of an arbitrary
ode sequen e v an be written as the produ t v (D) = u(D)G(D), where u(D) is the
D-transform of an input information sequen e u. The ode sequen es are transmitted
over a binary symmetri hannel (BSC) with rossover probability p. At the re eiver a
maximum-likelihood (ML) de oder, based on the Viterbi algorithm, uses the re eived se-
quen e r = : : : r 1 r0 r1r2 : : : , rt = (rt(1) rt(2) ), rt(1) ; rt(2) 2 GF(2), to make an estimate u^
of the information sequen e u. We are interested in the probability P (^ut 6= ut ) that an
information symbol at some arbitrary time t is de oded erroneously. Sin e we assume
innite sequen es this probability is independent of the time t and will be denoted by

O tober 21, 2002 DRAFT

 3

Pb . Wei et. al. [9℄ showed how the te hnique in [8℄ an be extended to take into a ount
transmission in blo k format or nite ba k-sear h limit in the de oder. In the rst ase
P (^ut 6= ut ) will depend on the time index t and Pb from the innite ase will form an
upper bound. In the se ond ase P (^ut 6= ut ) will depend on the ba k-sear h depth. For
ea h of these parameters the bit error probability has to be al ulated individually. All
results that we dis uss in the present paper an also be extended to these ases.
We onsider the memory m = 2 onvolutional ode, whi h is generated by the generator
matrix G(D) = (1 + D + D2 ; 1 + D2 ).2 A onvolutional ode an be en oded by dierent
generator matri es, and ea h generator matrix an be realized in many ways. Throughout
this paper we assume that the en oder is realized in ontroller anoni al form [10℄. While
the distan e properties and the burst (event) error probability are mere ode properties,
the bit error probability, Pb , does depend on the mapping from the information sequen es
to the ode sequen es and, hen e, is an en oder property. We investigate three dierent
en oders for the ode, des ribed by the generator matri es
   

1 + D2 1 + D + D2
GA (D) = 1 + D + D2 ; 1 + D2 G B ( D ) = 1; G C (D ) = ;1 :
1 + D + D2 1 + D2
(1)
GA (D) denes a non-systemati polynomial en oder, GB (D) and GC (D) dene systemati
en oders with feedba k, often referred to as re ursive systemati en oders. If the order of
the olumns in the generator matri es, i.e. g1 (D) and g2 (D), is hanged, one obtains an
equivalent ode and the bit error probability Pb is un hanged.
The remainder of this paper is organized as follows. In Se tion II we introdu e a
Markov hain to derive the probability distribution of the umulative Viterbi metri s.
Although the prin ipal steps in this se tion are already known from [8℄, the ontent and
our notation will be essential for the understanding of the following se tions. In Se tion III
we show how the bit error probability an be derived for a polynomial en oder. When the
maximum-likelihood path is not unique the bit error probability will also depend on the
rule the de oder uses to hoose among dierent survivor paths. We analyze and ompare
two dierent de oding rules in Se tions III-A and III-B. The previous investigations in
[3℄[4℄[5℄[8℄ were restri ted to polynomial en oders without feedba k. In Se tion IV we
2 sin e the generator polynomials an be represented by the o tal numbers 7 and 5 this ode is sometimes referred
to as (7,5) ode.
O tober 21, 2002 DRAFT
 4

illustrate how the proposed model an be extended to also take into a ount re ursive
systemati en oders with feedba k, whi h play an important role in iterative de oding
s hemes. Then we demonstrate in Se tion V that the rst de oding rule, investigated in
Se tion III-A, is losely related to ommon soft-output de oders and present a way to
derive the bit error probability and the probability distribution of the soft-output values
of the Max-Log-MAP de oder [11℄. The al ulated expressions are reported and dis ussed
in Se tion VI and on lusions are nally given in Se tion VII.

II. The Cumulative Viterbi Metri s and their Probability Distribution

When a sequen e r is re eived from the hannel, the ML de oder sele ts as its estimate
v^ a ode sequen e v whi h maximizes P (rjv ), i.e.
2
YY 2
XX
v^ = arg max P (rjv ) = arg max P (rt jvt ) = arg max
(i) (i)
log P (rt(i) jvt(i) ) : (2)
v v v
t i=1 t i=1

For the BSC with rossover probability p this an be written as

q
v^ = arg min d(r; v ) log +  = arg min d(r; v ); (3)
v p v

where q = 1 p,  is a onstant and d(
;
) denotes the Hamming distan e between two
sequen es. In order to nd the ode sequen e v^ orresponding to the minimal metri

 = (r) = min
v
d(v ; r) ; (4)

the Viterbi algorithm re ursively al ulates umulative metri values by moving forward
through the en oder trellis. Let t ( ) denote the minimal umulative metri up to time t
of all trellis paths that pass at that time through en oder state  . The omponents of the
ve tor t+1 = (t+1 (0); t+1 (1); : : : ; t+1 (2m 1)) an be al ulated from the orrespond-
ing ve tor t of the previous time instant by

t+1 ( ) = min

0
0 (t ( ) +
0 ; (r t )) ;
2S
8  2 S def
= f0; : : : ; 2m 1g ; (5)

where
0 ; (rt ) is the metri in rement orresponding to the trellis transition3 from state
t =  0 to t+1 =  . Sin e the minimization problem is not ae ted by subtra tion of
3 The in rement is dened to be
 ; (r t ) = 1 if there is no transition between  0 and  .
0

O tober 21, 2002 DRAFT

 5

t rt = (1; 0) t+1
00 0
t (0) = 0
11 -1
11
t (1) = 1 0
00 0
10 0
t (2) = 1 -1
01
01 2
t (3) = 1
10 -2
Fig. 1. The trellis se tion at time t, onsidered in Example 1.

an arbitrary value from all omponents of t , we an redu e the number of variables by

onsidering the normalized metri ve tor

t = ( t (1); : : : ; t (2m 1)) def

= (t (1) t (0); : : : ; t (2m 1) t (0)) (6)

instead of t and keeping in mind that t (0) = t (0) t (0) = 0 a ording to this
denition. For given rt and t at time t the omponents of t+1 at the following time
instant are determined by (5) and (6).
Example 1: Let us determine t+1 , if t = ( 1; 1; 1) and rt = (1; 0). Figure 1 illus-
trates this situation in the trellis se tion at time t. The in rement values
0 ; (rt ) are
equal to 0, 1, 1, and 0 for the transitions labeled by ode symbols 00, 01, 10, and 11,
respe tively. The minimal values into the states t+1 = 0; 1; 2; 3 are 1, 0, 1, and 2,
respe tively, resulting in t+1 = (1; 0; 1). 
A key observation is, that the number of dierent metri ve tors t that an o ur at
arbitrary times t when pro eeding through the trellis is nite. For the simplest onvolu-
tional ode with memory m = 1, whi h is studied in [8℄, there are only ve dierent t ,
namely the s alar values 2; 1; 0; 1; 2. In ase of the onsidered rate R = 1=2 ode with
memory m = 2, t has three omponents taking values between 3 and 3. The total
number of dierent t is equal to # = 31. Be ause of the linearity of the ode the set of
metri ve tors is independent of the transmitted sequen e.
In the following analysis we assume, without loss of generality, that the all-zero sequen e
v = 0 is transmitted. A ording to the hannel model, the elements of the re eived se-

O tober 21, 2002 DRAFT

 6

quen e r are Bernoulli distributed random variables. As a onsequen e, also the ompo-
nents of t are random variables. To obtain the probability distribution of the umulative
Viterbi metri s t at some arbitrary time t we an, like in [8℄, onstru t a Markov hain
that has the dierent possible values of t , denoted by 1 , 2 ,. . . , # , as states. This
Markov hain is des ribed by a transition matrix M , where the element mj;k at row j and
olumn k is equal to the probability to ome from state t 1 = j to state t = k . This
probability, whi h is independent of the time t, is given by
X
P ( t = k j t 1 = j ) = P ( t = k ; r t 1 j t 1 = j ) ; (7)
rt 1

where P ( t = k ; rt 1 j t 1 = j ) is equal to P (rt 1 ) if j ; rt 1 ! k and equal to zero

otherwise. The solution to the linear equation system

 = M T  ; 1 + 2 +


+ # = 1 ; (8)

where i = P ( t = i ), i = 1; : : : ; # , gives the steady-state probabilities of the Markov

hain and, hen e, the probability distribution of t at arbitrary time t. The Markov hain
does not depend on the information mapping in the en oder trellis. Therefore the system
has to be solved only on e for the three en oders given in (1).

III. Bit Error Probability Cal ulation for Polynomial En oders

For en oders with polynomial generator matri es the set of en oder states at time t + 1
an be partitioned into two subsets, S0 and S1 , orresponding to information symbol
ut = 0 and ut = 1, respe tively. In a realization in ontroller anoni al form this follows
immediately from the fa t that the input is fed dire tly into the shift register dening the
en oder states. Consequently, the bit error probability Pb = P (^ut 6= ut ) is equal to the
probability, that the ML de oder hooses a ode sequen e whose path in the trellis passes
at time t + 1 through a state belonging to the wrong subset. For the en oder des ribed
by GA (D), this is the ase if the minimal metri , given in (4), orresponds to a path
passing through a state t+1 2 S1 = f2; 3g.
An ML de oder needs to onsider omplete path metri s to be able to sele t the estimate
ode sequen e v^. The umulative metri values t ( ) take into a ount only the part of
the ode sequen es up to time t. For this reason we introdu e orresponding values ~t ( )
O tober 21, 2002 DRAFT
 7

for the remaining part of the paths after time t, and obtain
def
t ( ) = t ( ) + ~t ( ) = vmin d(v ( 1;t 1℄ ; r( 1;t 1℄) + vmin d(v [t;1) ; r[t;1))  ; (9)
:t = :t =

where  denotes an oset aused by the normalization in (6). Then t ( ) is the minimal
omplete metri of all paths that pass at time t through en oder state t =  . The value
~t ( ), denoting the minimal metri of all paths that start at time t in state t =  , an
be al ulated by a ba kward re ursion, equivalent to (5) and (6) in reverse dire tion. The
probability distribution of ~ t = (~ t (1); ~t(2); : : : ; ~t (2m 1)) an be obtained with the
method des ribed for t in Se tion II if the transition matrix M is transposed. In ase
of the onsidered memory m = 2 ode we an make use of some symmetry properties:
moving in reverse dire tion through the en oder state diagram is equivalent to ex hanging
states  = 1 and  = 2. Hen e,



P ~ t = (~ 1 ; ~2 ; ~3) = P t = (~ 2 ; ~1 ; ~3 ) ; (10)

i.e. the distribution of ~ t follows dire tly from the distribution of t , derived in the previous
se tion, and does not require any additional al ulations. It an be seen in (9) that the
ve tors t and ~ t are ompletely independent, as they result from non-interse ting parts
of the ode sequen es. To obtain the distribution of t we an go through all ombinations
of t and ~ t and ompute
X
P( t = )= P ( t = )P ( ~t = ~ ) : (11)
+ ~=

The bit error probability, Pb , an be obtained as sum of P ( t+1 = ) over those

that indi ate an error. A bit error at time t o urs if the de oder sele ts a path passing
through t+1 2 S1 , whi h is the ase if min( t+1 (0) = 0; t+1 (1)) > min( t+1 (2); t+1 (3)).
However, if there exist several survivor paths with minimal metri , it an happen that they
disagree about symbol ut . If min( t+1 (0); t+1 (1)) = min( t+1 (2); t+1 (3)) the o urren e
of a bit error depends on the path sele tion. In pra ti e it an be useful to hoose ertain
deterministi survivor sele tion rules in order to simplify the de oder [6℄. On the other
hand su h de oding rules typi ally make the error probability dependent on the transmit-
ted ode sequen e v , whi h makes an analysis of the average error probability diÆ ult. In
this paper we onsider two dierent de oding rules to treat multiple survivor paths. Both
O tober 21, 2002 DRAFT
 8

methods involve randomness to assure that the bit error probability is independent of the
transmitted sequen e.

A. De oding Rule 1: Tra ing Ba k All Survivors

In the following we investigate a de oding rule that diers in the survivor sele tion from
a standard Viterbi de oder but has the advantage that it is simple to analyze. It will be
shown in Se tion V that it also establishes a link between onventional Viterbi de oders
and soft-output de oders [11℄ [12℄, whi h are ommon in iterative de oding s hemes.
Consider a de oding algorithm, that tra es ba k all survivors, i.e. all trellis paths with
maximum likelihood. The number of en oder states in the trellis, passed by the surviving
paths, will be dierent at dierent time instan es. To de ode information symbol ut , the
de oder pi ks, with equal probability, one of these survivor states at time t + 1. Then the
estimate u^t is made a ording to the information symbol value that orresponds to the
surviving bran h into this state. If two bran hes ompete one is sele ted at random.
The bit error probability of this de oding algorithm an easily be obtained from the
probability distribution of t+1 , given by (11). Let M( t+1 ) = arg min 
( t+1 ( )) denote
the set of en oder states for whi h t+1 ( ) is minimal and Me ( t+1 ) = M( t+1 ) \ S1
denote the subset of M( t+1 ) whi h orresponds to a bit error at time t. For the en oder
des ribed by GA (D), Me ( t+1 ) = M( t+1 )\f2; 3g. Then the fra tion ( t+1 ) = jM e ( t+1 )j
jM( t+1)j
is the probability that a bit error o urs onditioned on the metri ve tor t+1 , and the
bit error probability be omes
X
Pb =  ( )P ( t+1 = ): (12)

Even though the trellis bran hes, sele ted by the onsidered de oder at ea h time t,
always lie on some ML path, they not ne essarily form one ontinuous trellis path. Stri tly
speaking, the algorithm denes thus a symbol-by-symbol de oder rather than a sequen e
de oder. Sin e this an only happen if the survivor is not unique, we still onsider this
de oder as a Viterbi (ML) de oder.

B. De oding Rule 2: Tra ing Ba k One Survivor

Let us now onsider a de oding rule, whi h is more diÆ ult to analyze but ommonly
used in pra ti e. The de oder tra es ba k only one survivor path. At ea h trellis node
O tober 21, 2002 DRAFT
 9

where the path splits into two possible dire tions, one of them is sele ted at random. This
rule is sometimes alled the oin- ip tie-breaking rule. It is worth to mention, that a
de oder based on this rule is dierent from a de oder that hooses among all maximum
likelihood sequen es with equal probability.
The analysis of this algorithm would be the same as for de oding rule 1, if the sele ted
survivor path would pass at time t + 1 with equal probability through all surviving states,
i.e. en oder states  with minimal t+1 ( ). In reality these states are passed with dierent
probabilities, depending on the path sele tion rule, and to determine if a bit error o urred
at time t, it is no longer suÆ ient, like in Se tion III-A, to observe the ve tor t+1 . Some
additional knowledge is required about the survivor path, sele ted by the de oder. Re all,
that a bit error at time t o urs, if the sele ted survivor path passes through a state
t+1 2 S1 . Hen e, Pb an be expressed as
X XX
Pb = P (ts+1 =  ) = P (ts+1 =  j t+1 = )P ( t+1 = ) ; (13)
2S1 2S1

where P (ts =  ) denotes the probability, that the sele ted survivor path passes at time t
through state  . The onditioned probabilities P (ts+1 j t+1 ) an be al ulated re ursively.
For any  = 0; : : : ; 2m 1 and i , i = 1; : : : ; # , we an write
X
P (s =  j  = i ) = P (s =  j  = i ; r )P (r ) : (14)
r
The probability P (s =  j  ; r ) an be expressed as a fun tion of the probabilities
P (s +1 j  +1 ). As mentioned before, for given r and  at time  > t, the ve tor  +1 is
determined by (5) and (6).
Example 2: Consider the ase  = ( 1; 1; 1), r = (1; 0) and  +1 = (1; 0; 1),
whi h is illustrated in Figure 2. The sele ted survivor path will pass one of the transitions
( ;  +1 ) = (1; 0), (1; 2), (2; 1), (3; 1), (3; 3). Observe, that there is a tie at state  +1 = 1.
From this follow the probabilities
P (s = 0j  ; r ) = 0
P (s = 1j  ; r ) = P (s +1 = 0j  +1 ) + P (s +1 = 2j  +1 )
1
P (s = 2j  ; r ) = P (s +1 = 1j  +1 )
2
1
P (s = 3j  ; r ) = P (s +1 = 1j  +1 ) + P (s +1 = 3j  +1 ) :
2
O tober 21, 2002 DRAFT
 10
r = (1; 0)
  +1
00 1
 (0) = 0  +1 (0) = 0
11 0
11
 (1) = 1 1  +1 (1) = 1
00 1
10 1
 (2) = 1 0  +1 (2) = 0
01
01 3
 (3) = 1 10 -1  +1 (3) = 1

Fig. 2. The survivor bran hes at time  , onsidered in Example 2.


Corresponding expressions an be obtained for other  , r and  +1 and substituted into
(14). We will obtain a system of equations of the form
m 1 #
2X X
P ( s
 =  j  = ) =
k
0 ; j P (s +1 =  0 j  +1 = j ) ; (15)
0 =0 j =1

where  = 0; : : : ; 2m 1, k = 1; : : : ; # and the oeÆ ients 0 ; j are onstants. The

system des ribes a ba kward re ursion for the onditioned probabilities. Due to the fa t
that we operate with innite sequen es the onsidered probabilities are independent of
time, i.e. P (s j  ) = P (s +1 j  +1 ) for any  . In parti ular, when  = t + 1, we an solve
the equation system (15), together with the onditions
m 1
2X
P (ts+1 =  j t+1 = i ) = 1; i = 1; : : : ; # ; (16)
=0

to obtain the probabilities P (ts+1 j t+1 ). The bit error probability follows nally from (8)
and (13).
The equation system is a generalization of equations (14) and (15) in [8℄ for a memory
m = 1 en oder. It's nature is similar to that of system (8) for the steady state distribution
of the Markov hain in Se tion II, but (15), (16) are more diÆ ult to solve (124 instead of
31 independent equations for the memory m = 2 ode). Equations (15) and (16) do not
depend on the information mapping in the en oder trellis and are therefore invariant over
the three en oders given in (1).
O tober 21, 2002 DRAFT
 11

IV. Extension to En oders with Feedba k

When the generator matrix is not polynomial but ontains rational generators, like
GB (D) and GC (D), there exists a feedba k loop in the realization and as a onsequen e
the information bit does not dire tly enter the shift register. Then the set of en oder states
annot be partitioned into subsets orresponding to ut = 1 and ut = 0, as we assumed in
Se tion III, and it is no longer possible to determine if a bit error o urs by only knowing
the state t+1 whi h is passed by the sele ted survivor path.
Assume we know rt and t and al ulate the resulting ve tor t+1 using (5) and (6).
Let us now introdu e an auxiliary ve tor it+1 = (it+1 (0); it+1 (1); : : : ; it+1 (2m 1)), where
ea h element it+1 ( ) 2 f0; 1; g denotes the value of information symbol ut orresponding
to the surviving bran h into state t+1 =  . If there is a tie, this value might not be
spe ied uniquely. In this ase it+1 ( ) is set to ''. Sin e for polynomial en oders the
information bit ut determines the state t+1 the information assignment is onstant and
has the form it+1 = (0; : : : ; 0; 1; : : : ; 1). For en oders with feedba k it+1 is dierent for
dierent transitions t ; rt ! t+1 . The probabilities P ( t+1 ; it+1 j t ) an be obtained in
a similar way as before ( f. (7)). Then we an al ulate the joint probability for the pair
t+1 ; it+1 ,
X
P ( t+1 ; it+1 ) = P ( t+1 ; it+1 j t )P ( t ) ; (17)
t

where t is distributed a ording to the steady distribution, following from (8). In this
manner the states of the Markov hain are, at time t + 1, split a ording to the dierent
possible information assignments it+1 . Like before, we an ombine the pre eding and
forth oming parts of the trellis paths and obtain
X
P( t+1 = ; it+1 ) = P ( t+1 = ; it+1 )P (~ t = ~ ) ; (18)
+~ =

and the bit error probability Pb is equal to the sum of P ( t+1 = ; it+1 = i) over those
and i that indi ate an error.
Let us rst onsider the de oding rule from Se tion III-A. We dene the sets M and Me
in the same way as before, however, Me depends now on both and i, i.e. Me = Me ( ; i).
Furthermore, we introdu e the subset M ( ; i) of M( ), whi h orresponds to states 
with symbols i that, due to ties, are unspe ied and marked by ''. Then the probability
O tober 21, 2002 DRAFT
 12

that a bit error o urs, onditioned on t+1 = and it+1 = i, is equal to the fra tion
( ; i) = jMe j+1
jMj
=2jM j
, and the bit error probability is given by
X
Pb = ( ; i)P ( t+1 = ; it+1 = i) : (19)
;i

The same ideas an be applied to the se ond de oding rule from Se tion III-B. We
obtain
X
Pb = P (^ut 6= ut j t+1 = ; it+1 = i)P ( t+1 = ; it+1 = i) ; (20)
;i
where
X 1 X
P (^ut 6= ut j t+1 ; it+1 ) = P (ts+1 =  j t+1 ) + P (ts+1 =  j t+1 ) ; (21)
2S1 (it+1 )
2 2S (i
t+1 )

and S1 (i), S (i) denote the sets of en oder states  for whi h i( ) = 1 and i( ) = ,
respe tively.
These derivations of the bit error probability Pb for onvolutional en oders with rational
generator matri es in lude the polynomial en oders, onsidered in Se tion III, as a spe ial
ase.

V. Soft-Output De oders

In situations where the de oder output is further pro essed, whi h is the ase in iterative
de oding s hemes, the performan e an essentially be improved with an algorithm that
delivers soft output information. The optimal reliability values are obtained by a posteriori
probability (APP) de oders, whi h are usually realized by a forward-ba kward (BCJR)
algorithm, proposed by Bahl et.al. [14℄. Unfortunately, the de oding omplexity of APP
de oders is mu h larger than for Viterbi de oding. Be ause of this fa t, suboptimal soft
output de oding algorithms are very often used in pra ti e. One su h algorithm is the
Max-Log-MAP algorithm [11℄, whi h originally was introdu ed as an approximation to
the BCJR algorithm. On the other hand, this algorithm an be shown to be equivalent
to the modied soft-output Viterbi algorithm (SOVA) [12℄, a modi ation of the Viterbi
algorithm that delivers soft-output values.
Aside from the redu ed omplexity ompared to optimal APP de oders, the similarity
to the Viterbi algorithm makes the Max-Log-MAP algorithm also simpler to analyze. For

O tober 21, 2002 DRAFT

 13

an analysis of the iterative de oding performan e of turbo odes, given in [15℄, a max-path
algorithm was onsidered, whi h delivers the same output as the Max-Log-MAP algorithm,
but operates with probabilities instead of log-likelihood ratios.
Using the results from Se tion III-A, it is possible to derive the probability distribution
of the soft-output values of the max-path (Max-Log-MAP) algorithm. The max-path
de oding algorithm al ulates for ea h time t the probability of the most probable trellis
path, orresponding to ut = 0, and the probability of the most probable trellis path
orresponding to ut = 1. The de ision on symbol ut is then made in favor of the more
probable trellis path. Let us express the output of the algorithm in terms of the log-
likelihood ratios
maxv:ut=0 P (rjv )
t = log = max log P (rjv ) max log P (rjv ) : (22)
maxv:ut=1 P (rjv ) v:ut=0 v :ut =1

In the Max-Log-MAP algorithm these log-likelihood ratios t are eÆ iently omputed in

the logarithmi domain. The log-likelihood ratio t , given in (22), an be expressed in
terms of the ve tors t+1 and it+1 ( f. Se tion IV),

t = vmax log P (rjv ) vmax log P (rjv ) = min t+1 ( ) min t+1 ( ) ;
:u =0
t :u =1 t 2S0 (it+1 )[S (it+1 ) 2S1 (it+1 )[S (it+1 )
(23)
where S0 (i), S1 (i) and S (i) denote the sets of en oder states  for whi h i( ) = 0,
i( ) = 1 and i( ) = , respe tively4 . With (23) and (18) the probability distribution of
the soft-output values of the max-path (Max-Log-MAP) algorithm be omes
X
P (t = ) = P( t+1 = ; it+1 = i) ; (24)

( ;i)=

where

( ; i) = min ( ) min ( ) : (25)
2S0 (i)[S (i) 2S1 (i)[S (i)

From (22) follows dire tly, that the hard de ision output of this algorithm is the same as
for the standard Viterbi algorithm, provided that the maximum-likelihood path is unique.
If there exist several survivor paths that disagree upon symbol ut , the log-likelihood ratio
is equal to t = 0 (minimal reliability). In this ase, a de ision is usually made for u^t = 0
4 Equation (23) is true if the base of the logarithm equals q=p. Otherwise a onstant fa tor will appear in the
term on the right hand side.

O tober 21, 2002 DRAFT

 14

and u^t = 1 with equal probability, sin e t = 0 delivers no information about the symbol
ut .
The bit error probability Pb of the max-path (Max-Log-MAP) de oder an be al ulated
with equation (19), when ( ; i) is set to
8
>
>1; if
( ; i) < 0
>
>
<
( ; i) = 1=2; if
( ; i) = 0 (26)
>>
>
>
:0; otherwise.
Surprisingly, the resulting bit error probability is slightly higher than the ones of the
Viterbi algorithm, given in (12),(13),(19) and (20). Consequently, for transmission over
the BSC, the max-path (Max-Log-MAP) de oder is worse than the Viterbi de oder in
terms of the bit error probability. This ee t, whi h is hara teristi for a dis rete hannel,
is theoreti ally interesting and an be onrmed by omputer simulations.

VI. Results and Dis ussion

Applying the methods des ribed in Se tions II{IV we evaluated the bit error probabili-
ties as fun tions of the rossover probability p of the BSC for the en oders with generator
matri es GA,GB and GC , given in (1). We derived the probabilities for the two de oding
rules, introdu ed in Se tion III-A and III-B, and for the Max-Log-MAP de oder. The
resulting expressions are rational fun tions in the variable p. Figure 3 shows the urves of
these exa t bit error probabilities for all three en oders. Results of omputer simulations
with the se ond ( oin- ip) de oding rule are also presented and mat h perfe tly with the
orresponding urves. Through the omplete range of the hannel rossover probability p
the se ond de oding rule results in a slightly better bit error performan e than the rst
de oding rule or the Max-Log-MAP de oder. However, the dieren e is so small that the
urves are diÆ ult to distinguish in the gure. We want to stress on e more, that all the
onsidered de oding rules ensure that the derived probabilities are valid independently
of the transmitted sequen es, even though the all-zero sequen e is used as referen e in
the analysis. The dependen e of the performan e on the de oding rules raises the still
unsolved question how to nd an optimal survivor sele tion rule, giving the best possible
result. Note also, that the Viterbi algorithm in general is not optimal in the sense of

O tober 21, 2002 DRAFT

 15

−1
10

−2
10
Pb

B
PSfrag repla ements 10
−3

−4
10 A
C
−5
10 −1
10
p
Fig. 3. The exa t bit error probabilities Pb for the en oders with generator matri es GA , GB and GC for
the rst (dashed) and se ond (solid) de oding rule and the Max-Log-MAP de oder (dotted). The points
marked by dots orrespond to simulation results.

minimizing the bit error probability.

P
The bit error probabilities an be expressed as Pb = f (p)=h(p), where f (p) = k fk pk
P
and h(p) = k hk pk are polynomials that are relatively prime, i.e. g d(f (p); h(p)) = 1.
For the non-systemati en oder with generator matrix GA (D) and the rst de oding rule
(the simplest ase), the oeÆ ients of these two polynomials are listed in Table I. The
high degree of the polynomials and the large values of the oeÆ ients make the resulting
formulas diÆ ult to tabulate. Nevertheless, for the onsidered memory m = 2 ode the
derived expressions an easily be represented and manipulated by means of mathemati al
omputer tools like, e.g., Maple or Mathemati a. For this reason, we de ided to make the
evaluated expressions available online in ele troni format5, instead of listing them in form
of additional tables.
A series expansion of Pb up to any required degree an be obtained from the evaluated
5 The al ulated results an be found at www.it.lth.se/Mi hael/Exa tErrorProbability.

O tober 21, 2002 DRAFT

 16

k fk hk k fk hk
0 6 35 -909044077180391296 313229432403674880
1 -72 36 2417988471991882560 -275042638680130944
2 768 37 -5330207094877951104 -153021977548253952
3 265 -4872 38 10295173207183127104 1410859165841021568
4 -351 27084 39 -17867563322317690624 -4097687548943430144
5 -1981 -114288 40 28247006369981566976 8851064408146699776
6 43537 426132 41 -41013528951384722432 -16114717950327183360
7 -285281 - 1347828 42 54981081370326788864 25848346636450576896
8 1299366 3538206 43 -68285806284195823616 -37298529692365197312
9 - 4959709 -7283652 44 78751150567728106496 48973288757912229888
10 17059854 7979868 45 -84449551149776898048 -58907722481064726528
11 -58312320 23650476 46 84269456602031430656 65181387751386765312
12 221005066 -209292570 47 -78263767733638082560 -66509732270121345024
13 -920731628 967965564 48 67632157238602244096 62665601363281059840
14 3823571126 -3557957562 49 -54343652041446424576 -54544451406933688320
15 - 14727932596 11231234940 50 40557700180785000448 43846560611492241408
16 51078738901 -30859274556 51 -28072204850764808192 -32523561374632574976
17 -157943395547 72470887548 52 17985550123648270336 22227746940363177984
18 434026993772 -135511955922 53 -10640924019658096640 -13967785331540951040
19 -1057453043540 146188834860 54 5796710625752629248 8048655559373586432
20 2268958822042 247858143180 55 -2897387685058379776 -4238512129990066176
21 -4182839660275 -2238249813240 56 1323178513955356672 2031363725532463104
22 5987394598045 9187203123972 57 -549289584889954304 -881523356935913472
23 -2925500798458 -29356207455480 58 205999648093765632 344231188706623488
24 - 24406816376597 80346002993016 59 -69266281384116224 -120038930264358912
25 143941851696404 - 192777484284072 60 20686532953243648 37029049345769472
26 -563599817927379 400411866456582 61 -5422810214170624 -9984956763734016
27 1832188505707872 -676541843021712 62 1228899082043392 2317917521707008
28 -5164854573769100 722404687735464 63 -235938273820672 -453967607758848
29 12680943146670664 556270890548112 64 37324322766848 72954817806336
30 -26697619822281292 -6051612119253816 65 -4671850676224 -9238474653696
31 45968866059359024 21818446713585504 66 433858805760 864496386048
32 -55089943581085088 -57753599060602272 67 -26575110144 -53150220288
33 2281961809544704 124682392316170368 68 805306368 1610612736
34 237382529550233168 -222544666659852960
TABLE I
The oeffi ients of the polynomials f (p) and h(p) in the numerator and denominator of

Pb (p) of en oder A.

formulas, whi h an be useful if one is only interested in lower ranges of the hannel
rossover probability p, where the terms of lower order dominate. The rst few oeÆ ients
of the series expansion an be found in Table II. The oeÆ ient in the lowest order term
of the series expansion determines the asymptoti behavior when p tends to zero. For very
small p it is known that the union bound
1 X
X 1
Pb < i n(w; i) Pw ; (27)
w=dfree i=1

delivers a good approximation of the bit error probability. Here dfree denotes the free
distan e of the ode, n(w; i) denotes the number of paths with total weight w and infor-
mation weight i, and Pw is the pairwise error probability, that a path with weight w is
O tober 21, 2002 DRAFT
 17

de oding rule 1
265 p3 943 p4 651 p5 42277 p6 456331 p7


A 6 + 2 2 2 6 +O p8

979 p3 2653 4 34499 5 61079 6 188707 7

B 12 + 12 p 12 p 4 p + 6 p +O p8

851 3 1013 4 8649 5 64407 6 57935 7

C 12 p + 4 p 4 p 4 p + 6 p +O p8

de oding rule 2

3519 p4 14351 p5 1267079 p6 31646405 p7

A 44 p3 + 8 32 64 512 +O p8

163 3 365 4 24045 5 1557571 6 23008183 7

B 2 p + 2 p 8 p 128 p + 512 p +O p8

141 p3 1739 p4 71899 p5 1717003 p6 2635041 p7

C 2 + 8 32 128 + 128 +O p8

Max-Log-MAP de oder
89 p3 753 5 151489 7


A 2 + 490 p4 2 p 22210 p6 2 p +O p8

35057 p6 33637 p7


B 82 p3 + 262 p4 2743 p5 2 + 2 +O p8

143 p3 4049 5


C 2 + 301 p4 2 p 19226 p6 614 p7 + O p8

TABLE II
Series expansion of the bit error probabilities for the different en oders and de oding

rules.

more probable than the transmitted all-zero path. It is ommon to upper bound the pair-
p
wise error probability by Pw < W w with the Bhatta haryya parameter W = 4p(1 p).
Using the weight enumerator fun tion of the en oder the union bound (27) an then be
expressed in a losed form (Viterbi bound) [16℄. A tightened version of this bound an
be obtained by making use of the fa t that P2i 1 = P2i , i  1 (van de Meeberg bound)
[17℄. For the onsidered ode, only paths of weight w = dfree = 5 and w = 6 ontribute to
the leading oeÆ ient of the series expansion. Therefore, to estimate this oeÆ ient the
exa t pairwise error probability Pw an be used instead of the Bhatta haryya bound. The
estimates resulting from the dierent variants of the union bound (27) are shown in Ta-
ble III. A omparison of Table II and Table III shows, that the union bounds are suÆ ient
for an estimation of the order of the leading term but do not give the exa t value of the
oeÆ ient itself. It an also be seen that the van de Meeberg bound and even more the

O tober 21, 2002 DRAFT

 18

exa t Pw van de Meeberg Viterbi

A 50p3 + O(p4 ) 320p3 + O(p4 ) 32p5=2 + 256p3 + O(p7=2 )
B 90p3 + O(p4 ) 576p3 + O(p4 ) 96p5=2 + 384p3 + O(p7=2 )
C 80p3 + O(p4 ) 512p3 + O(p4 ) 64p5=2 + 384p3 + O(p7=2 )
TABLE III
Leading oeffi ients of union bound estimates of the bit error probabilities for the

different en oders.

Viterbi bound suer from not using the exa t values of the pairwise error probabilities Pw .
In the region of high rossover probabilities p it is mu h harder to nd tight bounds on
the performan e of a onvolutional ode [18℄. Observe that in this region the polynomial
en oder, whi h performs best for low values of p, is worse than the re ursive systemati
en oders. It has been shown in [19℄ that also in the general ase, as the hannel gets
worse, re ursive systemati en oders outperform polynomial en oders. This fa t makes
the re ursive systemati en oders very attra tive in iterative de oding s hemes, where the
omponent odes have to ope with hannels lose to or even beyond their apa ity limit.
In prin iple all the methods an be applied to odes with higher memory. Unfortunately,
the omplexity of the al ulations grows dramati ally with the memory of the omponent
ode and we were not able to derive orresponding results for odes with m  3. For
typi al examples of en oders with memory m = 3 and m = 4 the number of Markov hain
states be omes # = 471 and # = 25641. Another extension would be to investigate
soft-de ision de oding by onsidering a dis rete memoryless hannel (DMC) with more
than two output values. But also in this ase the number of metri ve tors will in rease
dramati ally. A method to analyze a memory m = 2 en oder for a ontinuous additive
white Gaussian noise (AWGN) hannel was proposed in [20℄. The evaluation of the bit error
probabilities, whi h is also based on a forward and ba kward re ursion, an unfortunately
only be performed numeri ally and also requires quantization for the representation of the
underlying probability density fun tions.
Finally we dis uss some extension of the onsidered model that we see as an interest-
ing topi for forth oming investigations. When reliability information is passed from one
omponent de oder to another in an iterative de oding s heme, it is important that the

O tober 21, 2002 DRAFT

 19

de oding algorithm an make use of a priori information about the symbols to de ode.
Consider a systemati en oder and assume that for information symbols the a priori prob-
ability pa = P (ut = 1) is known. To take this information into a ount, equation (3) an
be modied to
qq q
v^ = arg min dinf (r; v ) log a + dpar (r; v ) log ; (28)
v pa p p
where qa = 1 pa , and dinf , dpar denote the Hamming distan es ounting only information
or parity- he k symbols, respe tively. This orresponds to a modi ation of the Viterbi
de oder su h that the metri in rement for information symbols is larger than for parity-
he k symbols. If pa is known, in prin iple the error probability of this algorithm an be
al ulated with the same model as for the analysis of the usual Viterbi de oder. Unfor-
tunately, the number of dierent Markov hain states will depend on the ratio between
log(qa q )=(pa p) and log q=p, whi h makes a general al ulation of Pb (pa ; p) as a fun tion in
two variables impossible, at least with this simple variant of the onsidered model. If this
ratio is not rational it is not lear that the number of states # is nite, and hen e even
a numeri al al ulation might not be possible for arbitrary ombinations pa and p. For
an exa t analysis of iterative de oding it would be very interesting to nd some modi a-
tion of this method that ir umvents this problem. We see this as a dire tion for future
resear h.

VII. Con lusion

We have derived analyti expressions for the bit error probabilities of dierent rate
R = 1=2 memory m = 2 onvolutional en oders. The results for lassi al en oders with-
out feedba k were ompared to re ursive systemati en oders. All onsidered en oders
generate the same onvolutional ode but lead to dierent bit error probabilities. The bit
error probability does also depend on the rule the de oder uses to sele t among multiple
survivor paths. We onsidered two dierent de oding rules. Additionally we derived the
bit error probability and soft-output value distribution of the Max-Log-MAP de oder. All
these results are expressed as rational fun tions of the rossover probability of the BSC.
The resulting expressions are made available online and an be used as a referen e.

O tober 21, 2002 DRAFT

 20

Referen es

[1℄ A. J. Viterbi, \Error bounds for onvolutional odes and an asymptoti ally optimum de oding algorithm",
IEEE Trans. Inform. Theory , vol. IT{13, no. 2, pp. 260{269, Apr. 1967.
[2℄ G. D. Forney, Jr., \The Viterbi algorithm", Pro . IEEE , vol. 61, pp. 268{278, 1973.
[3℄ T. N. Morrissey, Jr., \Analysis of de oders for onvolutional odes by sto hasti sequential ma hine methods",
IEEE Trans. Inform. Theory ,vol. IT-16, pp. 460{469, July 1970.
[4℄ T. N. Morrissey, JR., \A Markovian analysis of Viterbi de oders for onvolutional odes", in Pro . Nat. Ele -

troni s Conf. , pp. 303{307, O t. 1969.

[5℄ J. P. M. S halkwijk, K. A. Post, and J. P. J. C. Aarts, \On a method of al ulating the event error probability
of onvolutional odes with maximum mikelihood de oding", IEEE Trans. Inform. Theory , vol. IT-25, no. 6,
pp. 737{743, Nov. 1979.
[6℄ J. P. M. S halkwijk, A. J. Vin k, and K. A. Post, \Syndrome de oding of binary rate k/n odes", IEEE

Trans. Inform. Theory , vol. IT-24, no. 5, pp. 552{562, Sept. 1978.
[7℄ A. R. Calderbank, P. C. Fishburn, A. Rabinovi h, \Covering properties of onvolutional odes and asso iated
latti es", IEEE Trans. Inform. Theory , vol. IT-41, no. 3, pp. 732{746, May 1995.
[8℄ M.R. Best, M.V. Burnashev, Y. Yanni k, A. Rabinovi h, P.C. Fishburn, A.R. Calderbank, and D.J. Costello,
Jr., \On a te hnique to al ulate the exa t performan e of a onvolutional ode", IEEE Trans. Inform. Theory ,
vol. IT{41, no. 2, pp. 441{447, Mar h 1995.
[9℄ L. Wei, T. Aulin, and H. Qi, \On the ee t of trun ation length on the exa t performan e of a onvolutional
ode", IEEE Trans. on Inform. Theory , vol. 43, no. 5, pp. 1678{1681, Sept. 1997.
[10℄ R. Johannesson and K. Sh. Zigangirov, Fundamentals of Convolutional Coding , IEEE Press, Pis ataway, N.J.,
1999.
[11℄ P. Robertson, E. Villebrun, and P. Hoeher, \A omparison of optimal and sub-optimal de oding algorithms
in the log-domain", Pro . ICC , Seattle, WA, pp. 1009{1013, June 1995.
[12℄ M. Fossorier, F. Burkert, S. Lin, and J. Hagenauer, \On the equivalen e between SOVA and Max-Log-MAP
de odings", IEEE Communi ations Letters , vol. 2, pp. 137{139, May 1998.
[13℄ J. Hagenauer and P. Hoeher, \A Viterbi algorithm with soft-de ision outputs and its appli ations", Pro .

Globe om , Dallas, TX, Nov. 1989, pp. 1680{1686.

[14℄ L. Bahl, J. Co ke, F. Jelinek, and J. Raviv, \Optimal de oding of linear odes for minimizing symbol error
rate", IEEE Trans. Inform. Theory , vol. IT{20, no. 1, pp. 284{287, Mar h 1974.
[15℄ D. Truha hev, M. Lentmaier, and K. Sh. Zigangirov. \Some results on erning design and de oding
of turbo odes", Problemy Pereda hi Informatsii , 37(3):6{23, July{September 2001. (available online at
http://www.it.lth.se/Dmitri/Publi ations).
[16℄ A. J. Viterbi and J. K. Omura, Prin iples of Digital Communi ation and Coding. M Graw Hill, 1979.
[17℄ L. Van de Meeberg, \A tightened upper bound on the error probability of binary onvolutional odes with
Viterbi de oding", IEEE Trans. Inform. Theory , vol. 20, pp. 389{391, May 1974.
[18℄ D. Truha hev, K. Engdahl, and K. Sh. Zigangirov, \New de oding error probability bounds for xed onvo-
lutional odes used in BSC", in Pro . Workshop on Coding and Cryptography , Paris, Fran e, Jan. 2001.
[19℄ R. Johannesson, J. L. Massey, and P. Staahl, \Systemati bits are better and no buts about it", vol. 20,
pp. 389{391, May 1974. in Codes, Graphs, and Systems: A Celebration of the Life and Career of G. David

Forney, Jr. on the O asion of His Sixtieth Birthday , Kluwer, 2000.

[20℄ H. Yoshikawa, I. Oka, C. Fujiwara, and Y. Daido, \Exa t analysis of bit error probability for 4-state soft
de ision Viterbi de oding", IEICE Trans. Fundamentals , vol. E85{A, no. 10, pp. 2263{2266, O t. 2002.
O tober 21, 2002 DRAFT