Parallel Additive Gaussian Channels

Let us assume that we have N parallel one-dimensional channels disturbed

by noise sources with variances σ12 , · · · , σN
2
.
N (0, σ12 )

x1
+ y1

N (0, σN
2
)

xN + yN

Energy Constraint: The total input energy is constrained on average

to E the total average energy per channel use:
N
X N
X
x2n = En = E
n=1 n=1

The capacity of these parallel channels is achieved by the following input

energy distribution:

σn2 + En = µ; σn2 < µ

En = 0; σn2 ≥ µ
P
where µ is the Lagrange multiplier chosen such that n En = E.
Then the capacity of this set of parallel channels is given by the following
Theorem:
   
1N
X En XN 1 µ
C= log 1 + 2  = log  2 
n=1 2 σn n=1 2 σn

Source: [1], Section 7.5, pp. 343 ff.

EE 7950: Statistical Communication Theory 2

Waterfilling Theorem:

Proof: Let x = [x1 , · · · , xN ] and y = [y1 , · · · , yN ] and consider

(1) N
X
I(x; y) ≤ I(xn ; yn ) (1) independent xn
n=1
 
(2) 1 X En
≤ log 1 + 2  (2) Gaussian distributed xn
n=1 2 σn
| {z }
f (E)
Since equality can be achieved in both inequalities above the
next step is to find the maximizing energy distribution E =
[E1 , · · · , EN ]. We make use of Theorem 1 and identify the fol-
lowing sufficient and necessary conditions:
∂f (E)
≤ λ
∂En
1
≤ λ
2(σn2 + En )
1
σn2 + En ≤ =µ
2λ
Q.E.D.
This theorem is called the Waterfilling Theorem. Its functioning can be
visualized in the following figure, where the power levels are in black:

Power level:Differnce µ − σn2

2
level: σN

Channels 1 through N
EE 7950: Statistical Communication Theory 3

Correlated Parallel Channels (MIMO)

Correlated channels arise from e.g., multiple antenna channels using Nt

transmit antennas and Nr receive antennas:

h11
x1 y1
x2 y2

hij
xNt yNr
hNt Nr
This channel is a multiple-input multiple-output (MIMO) channel
described by the matrix equation:

y = Hx + n

• The transmitted signals xn are complex signals, as are the channel gains
hij and the received signals yn .
• The noise is complex additive Gaussian noise with variance N0 (that is
N0 /2 in each dimension).
• The path gains hij are complex gain coefficients modeling a random
phase shift and a channel gain. Often these are modeled as Rayleigh
random variables modeling a scattering-rich or mobile radio transmission
environment.

MIMO Rayleigh Channel: The hij are modeled as i.i.d. (or correlated)
complex Gaussian random variables with variance 1/2 in each dimension.
EE 7950: Statistical Communication Theory 4

Channel Decomposition

The correlated MIMO channel can be decomposed via the singular value
decomposition SVD:

H = U DV +
(if r < t)

where U and V are unitary matrices, i.e., U U + = I, and V V + = I.

The matrix D contains the singular values of H, which are the square
roots of the eignevalues of HH + and H + H. If H is square, the singular
values are identical to the square of the eigenvalues of H.

The channel equation can now be written in an equivalent form:

If Nt > Nr only the first Nr signals of
y = Hx + n
x̃ will be received.
= U DV + x + n If Nr > Nt the Nr − Nt “bottom”
U + y = ỹ = D x̃ + ñ channels will carry no signal.

This leads to parallel Gaussian channels ỹn = dn x̃n + n˜n

d1 N (0, σ 2 )

x̃1 + ỹ1
+

dN N (0, σ 2 ) m = min(Nt , Nr )

x̃m + ỹm
+
EE 7950: Statistical Communication Theory 5

MIMO Capacity

The multiplicative factors dn can be eliminated by multiplying the re-

ceived signal y with D −1 . This leads back exactly to the parallel channel
problem, and the capacity of the MIMO channel is determined by the
waterfilling theorem:
   
N
X

d2n En  X N

d2n µ 
C= log 1 + = log
n=1 2σn2 n=1 2σn2

Note: The channels are complex, and hence there is no factor 1/2 and the
variance is 2σn2 .
This capacity is achieved with the waterfilling power allocation:
2σn2
+ En = µ; σn2 < µ
d2n
En = 0; σn2 ≥ µ

Optimal System: These considerations lead to the following optimal

signalling strategy:
1. Perform the SVD of the channel H → U , V , D.
2. Multiply the input signal vector x̃ with V . This is matrix processing.
3. Multiply the output signal y with the matrix processor U + .
4. Use each channel with signal-to-noise ratio d2n En /(2σn2 ) independently.

x̃ V x y U+ ỹ

Matrix Processor Matrix Processor

MIMO Channel
EE 7950: Statistical Communication Theory 6

Symmetric MIMO Capacity

Drawback: the channel H needs to be known at both the transmitter

and the receiver so the SVD can be computed.
Fact: Channel knowledge is not typically available at the transmitter,
and the only choice we have is to distribute the energy uniformly over all
component channels. This leads to the Symmetric Capacity:

   
N
X

d2n E  N
Y

d2n E 
C= log 1 + = log 1+
n=1 2Nt σ 2 n=1 2Nt σ 2

Noting that the d2n are the eigenvalues of HH + , the above formula can be
Q
written in terms of matrix eigenvalues, using the fact det(M ) = λ(M ),
Q
and det(I + M ) = (1 + λ(M )):

  Ã !
N
Y d2n E  E
C = log 1+ 2
= log det I N r
+ 2
HH +
n=1 2Nt σ 2Nt σ
à !
E +
= log det I Nr + H H
2Nt σ 2

Discussion:
The capacity of a MIMO channel is goverend by the singular values of H,
or in the symmetrical case by its eigenvalues. The eigenvalues determine
the channels gains of the independent parallel channels.
• Channel H needs to be known at the receiver → Channel Estimation
We will study the behavior of channel eigenvalues later in this course.
EE 7950: Statistical Communication Theory 7

MIMO Fading Channels

Assumed that the channel H is known at the receiver

I(x; (y, H)) = I(x; H) + I(x; y|H)

= I(x; y|H)
= EH [I(x; y|H = H 0 )]

We need to average the mutual information over all channel realizations.

I(x; y|H) is maximized if x is

circularly symmetric complex Gaussian with covariance Q, and
" Ã !#
E
I(x; (y, H)) = EH log det I r + 2
HQH +
2Nt σ
" Ã !#
E + +
I(x; (y, H)) = EH log det I r + (HU )D(U H )
2Nt σ 2
• The spectral decomposition of Q = U DU + produces an equivalent
channel H̃ = HU with the same statistics, hence the maximizing
Q is diagonal.
• Furthermore, concavity of the function log det() shows that Q = I,
hence the maximizing Q is a multiple of the identity

Capacity of the MIMO Rayleigh Channel:

" Ã !#
E
C = EH log det I r + 2
HH +
2Nt σ

• By the law of large numbers:

à !
Nt →∞ E
HH +
−→ Nt I r and C = r log 1 + 2
2σ
EE 7950: Statistical Communication Theory 8

Evaluation of the Capacity Formula

Following Telatar [2] define the random matrix

(
HH + if Nr < Nt
W =
H +H if Nr ≥ Nt
W is an m × m; m = min(r, t) non-negative definite matrix with real,
non-negative eigenvalues µn = d2n
The capacity can be written in terms of these eigenvalues:
 Ã ! for r = t symmetric chan-
m
X E
C = E{µn }  log 1 + µn  nels, let {λn } be the eigen-
2Nt σ 2
n=1
values of H, then µn = λ2n

• The eigenvalues of W follow a Wishart distribution:

m
Y Y
p(µ1 , · · · , µm ) = α e−µi µn−m
i (µi − µj )2
i=1  i<j 

1 ··· 1 
m 
· · · µm  n = max(Nr , Nt )
Y  µ1 
p(µi , · · · , µm ) = α e−µi µn−m
i det 
 .
.. .. ;

 . 
i=1  
m−1
µ1 · · · µm
m−1

• C therefore depends only on the distribution of a single eigenvalue:

" Ã !# Z ∞ Ã !
µE µE
C = mE{µ} log 1 + = mp(µ) log 1 + dµ
2Nt σ 2 0 2Nt σ 2
This integration evaluates to
Z ∞ Ã ! m−1
µE X k!
C= log 1 + [Ln−m (µ)]2 µn−m e−µ dµ
0 2Nt σ 2 k=1 (k + n − m)! k

k
−x n−m+k
• Ln−m
k
d
(x) = k!1 ex dx k (e x ) is the Laguerre polynomial
of order k
EE 7950: Statistical Communication Theory 9

Large Systems

As the number of antennas Nt → ∞, Nr → ∞, the number of eigenvalues

N (µ) → ∞, and the capacity formula

 Ã ! Ã !
m
X µE Z ∞ µEm
C = E{µi }  log 1 +  → m log 1 + dFµ (µ)
i=1 2Nt σ 2 0 2Nt σ 2

where Fµ (µ) is the cumulative distribution (CDF) of the eigenvalues

of W , which becomes continuous as m, n → ∞

For random matrices like W , a general result states

 và !à !


 1uu µ+ µ−
dFµ (µ) 
 t −1 1− ; for µ ∈ [µ− , µ+ ]
→ 2π µ µ
dµ 

 
0 otherwise
√
• n
m → τ ≥ 1, and µ± = ( τ ± 1)2 .

In the limit, the capacity of the Rayleigh MIMO is given by

à !v
uà !à !
C 1 Z d+ µEm u µ+ µ−
= log 1 + t −1 1− dµ
m 2π d− 2Nt σ 2 µ µ

References
[1] R.G. Gallager, Information Theory and Reliable Communication,
John Wiley & Sons, Inc., New-York, 1968.
[2] I.E. Telatar, “Capacity of mulit-antenna Gaussian channels”, Eur.
Trans. Telecom., Vol. 10, pp. 585–595, Nov. 1999.