Ahmad Ghasemi Mostafa Abdollahi Ajast Paper-4

International Journal of Advances in Science and Technology,
Vol. 3, No.4, 2011

Applications of Linear Algebra in Communication

Ahmad Ghasemi
1
, Mostafa Abdollahi
2
, Foad Ghasemi
3
and Asadollah Salimi Dehkodi
4
1,2,3,4
Department of Electrical Engineering, Boroujen Branch, Eslamic Azad University, Boroujen, Iran
ahmadqassemi@gmail.com, Mostafa.Abdollahi.Bastaki@gmail.com, zzfqassemi@yahoo.com,
asad_salimy@yahoo.com

Abstract

Linear algebra has important role in engineering, but many people whom work in science and
technology dont sufficient information about using it in them field, in this literature we try
somewhat introduce applications of linear algebra in engineering, particularly in communication
and relevancy its concepts to phrases as Fourier Transform, least squares, transfer functions,
matched filter and etc to show its important role.

Some other mathematical subjects that are important for electrical engineering students include
probability and stochastic processes, statistics, and discrete mathematics.

Keywords: Linear algebra, communication, vectors, matched filter, singular value decomposition

1. Introduction

Linear algebra (LA) has important role in engineering, but many people whom work in
science and technology dont sufficient information about using it in them field, in this
literature we try to introduce applications of linear algebra in engineering, particularly in
communication and relevancy its concepts to phrases as Fourier Transform, least squares,
transfer functions, matched filter and etc to show its important role. The aim of this paper is
showing important role of LA with examples in communication.
This paper contains five part, first introduction, second basic concepts, third applications,
forth conclusion and future research directions and finally references.

2. Basic concepts

In this part, we study the topics in linear algebra that will be needed in the rest of the
literature.
We begin by discussing the building blocks of linear algebra: matrices and vectors.
Definition: a matrix is any rectangular array of numbers [1].
If a matrix A has m rows and n columns, we call A an m by n matrix. We refer to m by n as
the order of the matrix. A typical m by n matrix A may be written as

(
(
(
(
=
mn m m
n
n
a a a
a a a
a a a
A
2 1
2 22 21
1 12 11

(1)

The number in the ith row and jth column of A is called the ijth element of A and is written
a
ij
. Two matrices A=[a
ij
] and B=[b
ij
] are equal if only if A and B are of the same order and for
all I and j, a
ij
= b
ij
.
Definition: a matrix is any matrix with only one column (that is, any m1 matrix) may be
thought of as a column vector. R
m
will denote the set of all m-dimensional column vectors.
October Issue Page 23 of 85 ISSN 2229 5216
Vol. 3, No.4, 2011

In analogous fashion, we can think of any vector with only one row (a 1 n matrix) as a row
vector. The dimension of a row vector is the number of columns in the vector.
The Scalar Product of Two Vectors: An important result of multiplying two vectors is the
scalar product. To define the scalar product of two vectors, suppose we have a row vector u =
[u
1
u
2
u
n
] and a column vector v = [v
1
v
2
v
n
]
T
of the same dimension. The scalar product of
u and v (written u. v) is the number u
1
v
1
+ u
2
v
2
+ + u
n
v
n
.
Note that two vectors are perpendicular if and only if their scalar product equals 0. Thus, the
vectors [1 -1] and [1 1] are perpendicular.
We note that u . v = ||u||||v|| cosu, where ||u|| is the length of the vector u and u is the angle
between the vectors u and v.
The Transpose of a Matrix: Given any m n matrix, A=[a
ij
], the transpose of A (written A
T
)
is the n m matrix, A
T
=[a
ji
], Thus, A
T
is obtained from A by letting row 1 of A be column 1 of
A
T
, letting row 2 of A be column 2 of A
T
, and so on. For any matrix A, (A
T
)
T
= A.
Matrix Multiplication: Given two matrices A and B, the matrix product of A and B (written
AB) is defined if and only if Number of columns in A = number of rows in B. For the moment,
assume that for some positive integer r, A has r columns and B has r rows. Then for some m and
n, A is an m r matrix and B is an r n matrix [1].
Properties of Matrix Multiplication: To close this section, we discuss some important
properties of matrix multiplication. In what follows, we assume that all matrix products are
defined as following [1].

a) Row i of AB=(row i of A)B
b) Column j of AB=A(column j of B)
c) Matrix multiplication is associative. That is, A(BC)=(AB)C
d) Matrix multiplication is distributive. That is, A(B+C)=AB+AC and (B+C)D=BD+CD.

Euclidean Vector Norm: For a vector x
n1
, the Euclidean norm of x is defined as follows:

a)
= =
=
n
i
T
i
x x x x
1
2 / 1 2
) ( || || whenever
1
e
n
R x
b)
x x x x
n
i
i
*
1
2 / 1 2
) || || ( || || = =

=
whenever
1
e
n
C x
General Vector Norms: A norm for a real or complex vector space V is a function ||*||
mapping V into R that satisfies the following conditions [1].

a) ||x|| > 0 and ||x|| = 0 x = 0
b) ||x|| = || ||x|| for all scalars
c) ||x + y|| ||x|| + ||y||

General Matrix Norms: A matrix norm is a function ||*|| from the set of all complex
matrices (of all finite orders) into R that satisfies the following properties:

a) ||A|| > 0 and ||A|| = 0 A = 0
b) ||A|| = || ||A|| for all scalers
c) ||A + B|| ||A|| + ||B||

Linear : y = mx + c generalizes to vector equation, y = Mx + c ( y, x, c are vectors, M
= matrix)
Quadratic forms: Quadratic expressions in 1 variable: x
2
. Vector expression: x
T
x (
projection!), Quadratic forms generalize this, by allowing a linear transformation A as well
Multivariable quadratic expression: x
2
+ 2xy + y
2
Captured by a symmetric matrix A, and
quadratic form: x
T
Ax
Singular value decomposition (SVD): for each A R
mn
of rank r, there are orthogonal
matrices U
mm
, V
nn
and a diagonal D
rr
= diag (
1
,
2
,,
r
) such that
Vol. 3, No.4, 2011

T
n m
V
D
U A
|
|
.
|
\
|
=
0 0
0

(2)

With
1

2

r
> 0, the
i
s are called the nonzero singular values of A. when r < p =
min{m,n}, A is said to have p r additional zero singular values. The above factorization is
called a singular value decomposition of A, and the columns in U and V are called left -hand and
right-hand singular vectors for A, respectively [1].
Trace: the trace of an nn matrix A = [a
ij
] is defined the sum of the entries lying on the main
diagonal of A. That is, = + + + =
=
n
i ii nn
a a a a A tr
1 22 11
) (

3. Applications

Signal space: The distance in signal space is measure by calculating the norm: Norm
between two signals

|| ) ( ) ( ||
,
t y t x d
y x
=

(3)

We refer to the norm between two signals as the Euclidean distance between two signals.

This is what Fourier transform does projects a function onto a infinite number of orthogonal
basis functions: e
jw
or e
j2n
, and adds the results up (to get an equivalent representation in the
frequency domain).
CDMA codes are orthogonal, and projecting the composite received signal on each code
helps extract the symbol transmitted on that code.
Each code is an orthogonal basis vector, then signals sent are orthogonal. In figure 1 we
show orthogonal projections in CDMA [1].

Figure 1. CDMA orthogonal code
Linear channel codes (e.g.: Hamming, Reed-Solomon, BCH) can be viewed as k-dimensional
vector sub-spaces of a larger N-dimensional space. Then k-data bits can therefore be protected
with N-k parity bits.
This is linear algebra in action: design an appropriate k-dimensional vector sub-space out of
an N-dimensional vector space, In figure 2 we show error forward coding in Reed-Solomon
(RS).

Detection in AWGN noise: Pick the closest signal vector
Vol. 3, No.4, 2011

Figure 2. error forward coding in Reed-Solomon (RS)
Determinant criterion for space time code design good code exploiting time diversity
should maximize the minimum product distance between code words.
Coding gain determined by min of determinant over code words.
The error vector e of x is orthogonal to x. i.e. Inner product of them is zero, e
T
x = 0.
From this idea use to:
a) Find a least-squares line that minimizes the sum of squared error (i.e. min Ee
T
e)
b) Detection under AWGN noise to get the test statistic.
Matched filter is the filter that maximizes the signal-to-noise ratio it can be shown that it also
minimizes the BER: it is a simple projection operation [1].
Now show Matched Filter (MF) receiver as pictorially

Figure 3. Matched Filter (MF) receiver
If the receiver were to sample this signal at the correct times, the resulting binary message
would have a lot of bit errors.
a) Consider the received signal as a vector r, and the transmitted signal vector as s
b) MF projects the r onto signal space spanned by s (matches it)

Figure 4. MF projects the r onto signal space spanned by s
Filtered signal can now be safely sampled by the receiver at the correct sampling instants,
resulting in a correct interpretation of the binary message.
Gaussian vector formula has a quadratic form term in its exponent: exp [-0.5 (x -)
T
K
-1
(x -
)]

a) Similar to 1-variable gaussian: exp(-0.5(x -)
2
/o
2
)
b) K
-1
(inverse covariance matrix) instead of 1/ o
2

c) Quadratic form involving (x -) instead of (x -)
2

Vol. 3, No.4, 2011

(A
T
A) or (A
*
A) will appear often in communications math (MIMO). They will also appear in
SVD (singular value decomposition). The pseudo inverse (A
T
A)
-1
A
T
will appear in decorrelator
receivers for MIMO.
SVD for MIMO Channels: for channel in MIMO we have structure as y = Hx + w. now
SVD of H is H = UV
*
, then HH*=U
t
U
*
and H*H=V
t
V
*
.

a) Represent input in terms of a coordinate system defined by the columns of V (V*x)
b) Represent output in terms of a coordinate system defined by the columns of U ( U*y)
c) Then the input-output relationship is very simple (diagonal, i.e. scaling by singular
values)

Discrete Cosine Transform [2]: if x
C
denote DCT of vector x, then

x C x DCT x x
N c
DCT
= A ) (
(4)
]
2
) 1 )( 1 2 (
cos[ ) ( ] [
N
i j
i C C
ij N
t
=
(5)
=
=
W O
N
i
N
i C
.
2
1
1
) (

(6)

Now introduces a class of rectangular matrices with two advantages. They are simple, and
they are important. They are known as incidence matrices, and every entry is 1, -1, or 0. What is
remarkable is that the same is true of L and U and the basis vectors for the four subspaces.
Those subspaces play a central role in network theory and graph theory. The incidence matrix
comes directly from a graph, and we begin with a specific exampleafter emphasizing that the
word "graph" does not refer to the graph of a function (like a parabola for y = x
2
). There is a
second meaning, completely different, which is closer to computer science than to calculus, and
it is easy to explain.
A graph has two ingredients: a set of vertices or nodes, and a set of arcs or edges that
connect them. The graph in Fig. 5 has 4 nodes and 5 edges. It does not have an edge between
every pair of nodes; that is not required (and edges from a node to itself are forbidden). It is like
a road map, with cities as nodes and roads as edges. Ours is a directed graph, because each edge
has an arrow to indicate its direction.
The edge-node incidence matrix is 5 by 4; we denote it by A. It has a row for every edge, to
indicate the two nodes connected by the edge. If the edge goes from node j to node k, then that
row has -1 in column j and +1 in column k. The incidence matrix is printed next to the graph.

Figure 5. A directed graph and its edge-node incidence matrix
Row 1 shows the edge from node 1 to node 2. Row 5 comes from the fifth edge, from node 1
to node 4.
Notice what happens to the columns. The third column gives information about node 3, it
tells which edges enter and leave. Edges 2 and 3 go in, edge 4 goes out. A is sometimes called
the connectivity matrix, or the topology matrix, and it normally has more rows than columns.
When the graph has m edges and n nodes, A is m by n. Its transpose is the node-edge incidence
matrix.
Vol. 3, No.4, 2011

We start with the null space of A. Is there a combination of the columns that gives zero?
Normally the answer comes from elimination, but here it comes at a glance. The columns add up
to the zero column. Therefore the null space contains the vector of l's; if x = (1, 1, 1, 1) then Ax
= 0. The equation Ax = b does not have a unique solution (if it has a solution at all). Any
"constant vector" x = (c, c, c, c) can be added to any solution of Ax = b, and we still have a
solution.
This has a meaning if we think of the components x
1
, x
2
, x
3
, x
4
as the potentials at the nodes.
The vector Ax then gives the potential differences. There are five components of Ax (the first is
x
2
- x
1
from the 1 in the first row of A) and they give the differences in potential across the
five edges. The equation Ax = b therefore asks: Given the differences b
1
, ..., b
5
find the actual
potentials x
1
,
..., x
4
. But that is impossible to do! We can raise or lower all the potentials by the
same constant c, and the differences will not change-confirming that x = (c, c, c, c) is in the null
space of A. In fact those are the only vectors in the null space, since Ax = 0 means equal
potentials across every edge. The null space of this incidence matrix is 1-dimensional. Now we
determine the other three subspaces.
Column space: For which differences b
1
,..., b
5
can we solve Ax = b? To find a direct test,
look back at the matrix. The sum of rows 1 and 2 is row 3. On the right side we need b
1
+ b
2
=
b
3
, or no solution is possible. Similarly the sum of rows 3 and 4 is row 5. Therefore the right
side must satisfy b
3
+ b
4
= b
5
, in order for elimination to arrive at 0 = 0. To repeat, if b is in the
column space then b
1
+ b
2
- b
3
= 0 and b
3
+ b
4
- b
5
= 0.
Continuing the search, we also find that rows 1, 2, and 4 add to row 5. But this is nothing
new; adding the above equations already produces b
1
+ b
2
+ b
4
= b
5
. There are two conditions on
the five components, because the column space has dimension 3 = 5 - 2. Those conditions
would be found more systematically by elimination, but here they must have a meaning on the
graph.
The rule is that potential differences around a loop must add to zero. The differences around
the upper loop are b
1
b
2
, and -b
3
(the minus sign is required by the direction of the arrow). To
circle the loop and arrive back at the same potential, we need b
1
+ b
2
- b
3
= 0. Equivalently, the
potential differences must satisfy (x
2
- x
1
) + (x
1
- x
3
) = (x
2
- x
3
). Similarly the requirement b
3
+
b
4
- b
5
= 0 comes from the lower loop. Notice that the columns of A satisfy these two
requirements, they must, because Ax = b is solvable exactly when b is in the column space.
There are three independent columns and the rank is r = 3.
Left null space: What combinations of the rows give a zero row? That is also answered by
the loops! The vectors that satisfy y
T
A = 0 are y
1
T
= [1 1 -1 0 0] and y
2
T
= [0 0 1 1 -1]. Each loop
produces a vector y in the left null space. The component +1 or -1 indicates whether the edge
arrow has the same direction as the loop arrow. The combinations of y
1
and y
2
are also in the left
null space, in fact y
1
+ y
2
= (1, 1, 0, 1, -1) gives the loop around the outside of the graph.
You see that the column space and left null space are closely related. When the left null
space contains y
1
= (1, 1, -1, 0, 0), the vectors in the column space satisfy b
1
+ b
2
- b
3
= 0. This
illustrates the rule y
T
b = 0, soon to become part two of the "fundamental theorem of linear
algebra". We hold back on the general case, and identify this specific case as a law of network
theoryknown as Kirchhoff's voltage law.
The vectors in the left null space correspond to loops in the graph. The test for b to be in the
column space is Kirchhoffs Voltage Law: The sum of potential differences around a loop must
be zero.
Row space: That leaves one more subspace to be given a meaning in terms of the graph. The
row space contains vectors in 4-dimensional space, but not all vectors; its dimension is only r =
3. We could look to elimination to find three independent rows, or we could look to the graph.
The first three rows are dependent (because row 1 + row 2 = row 3) but rows 1, 2, 4 are
independent. Rows correspond to edges, and the rows are independent provided the edges
contain no loops. Rows 1, 2, 4 are a basis, but what do their combinations look like? In each
row the entries add to zero. Therefore any combination will have that same property. If f = (f
1
,
f
2
, f
3
, f
4
) is a linear combination of the rows, then f
1
+ f
2
+ f
3
+ f
4
= 0. That is the test for f to be
Vol. 3, No.4, 2011

in the row space. Looking back, there has to be a connection with the vector x = (1, 1, 1, l)in the
null space. Those four ls in above equation cannot be a coincidence: if f is in the row space and
x is in the null space then f
T
x = 0.
Again that illustrates the fundamental theorem of linear algebra. And again it comes from a
basic law of network theory, which now is Kirchhoff's current law. The total flow into every
node is zero. The numbers f
1
, f
2
, f
3
, f
4
are current sources at the nodes. The source f
1
must
balance - y
1
- y
3
- y
5
, which is the flow leaving node 1 along edges 1, 3, 5. That is the first
equation in A
T
y = f. Similarly at the other three nodes-conservation of charge requires that "flow
in = flow out". The beautiful thing is that the transpose of A is exactly the right matrix for the
current law.
The system A
T
y =f is solvable when f is in the column space of A
T
, which is the row space of
A: The four equations A
T
y = f, from the four nodes of the graph, express Kirchhoffs Current
Law: The net current into every node is zero.
This law can only be satisfied if the total current entering the nodes from outside is f
1
+ f
2
+
f
3
+ f
4
= 0.
If f = 0 then Kirchhoff's current law is A
T
y = 0. It is satisfied by any current that goes around
a loop. Thus the loops give the vectors y in the null space of A
T
[3].
SVD for image processing: Suppose a satellite takes a picture, and wants to send it to earth.
The picture may contain 1000 by 1000 "pixels"little squares each with a definite color. We
can code the colors, in a range between black and white, and send back 1,000,000 numbers. It is
better to find the essential information in the 1000 by 1000 matrix, and send only that.
Suppose we know the SVD. The key is in the singular values (in ). Typicall y, some are
significant and others are extremely small. If we keep 60 and throw away 940, then we send
only the corresponding 60 columns of Q
1
and Q
2
. The other 940 columns are multiplied in
Q
1
Q
2
T
by the small n's that are being ignored. In fact, we can do the matrix multiplication as
columns times rows:

+ + + =
T
r r r
T T T
v u v u v u Q Q o o o
2 2 2 1 1 1 2 1

(7)

Here the us are columns of Q
1
and the vs are columns of Q
2
{ v
1
T
is the first row of Q
2
T
).
Any matrix is the sum of r matrices of rank one. If only 60 terms are kept, we send 60 times
2000 numbers instead of a million.
The pictures are really striking, as more and more singular values are included. At first you
see nothing, and suddenly you recognize everything [4].
Other applications of LA are circuit analysis, channel desi gn, etc.
Cognitive Multiple-input Multiple-output (MIMO) Channel: if we consider a secondary
system that both CR transmitter and CR receiver are equipped with multiple antennas. The
received signal at the CR receiver is given by

n Hx y + = (8)

where x is the
T
n dimensional signal vector transmitted by the CR transmitter,

H is the
T R
n n full rank channel matrix between the transmitter and its intended receiver,

y is the
R
n dimensional noise plus interference vector at the CR receiver. It is proposed to use a set
of power and interference constraints to realize several practical considerations for the CR,
including but not limited to the following:

a) Maximum transmit power for the transmitter:

t
P Q Tr s ) (
(9)
Vol. 3, No.4, 2011

Q is the covariance matrix of the CR transmitter and
t
P is its own transmit power
constraint.
b) Null power constraints:

0 = Q U
H

(10)
Where U is a matrix whose columns represent the spatial and/or the frequency
directions the CR transmitter is not allowed to transmit.
c) Average power constraints:

ave H
P QG G Tr s ) (
(11)
Where the range space of the matrix G identifies the subspace where the interference
level should be kept under the required power constraint,
ave
P .
d) Peak power constraints:

peak H
P QG G s ) (
max

(12)
Where (.)
max
denotes the maximum eigen value of a matrix and
peak
P is the
corresponding maximum peak power constraint.
With above constraints, different transmit schemes can be designed to maximize the
achievable rate of the secondary link [5].

|| || log
2
H
HQH I R + =

(13)

Blind Channel Estimation: In some application user need to acquire the information of
different channels, among different methods, the idea of effective interference channel is
introduced in [6]. When two users take turn to transmit to each other, the central transmitter
takes samples from both users over frequency band. Define the covariance matrix due to only
signals from users as following [6].

2 2 2 2 1 1 1 1
G S G G S G Q
H H
s
o o + =

(14)

Where
i
G is the channel matrix from the i th user to the transmitter,
i
S is the covariance
matrix of the i th user, and
i
o is the expected value of the portion of instants that the i th user
is transmitting. From the sample covariance matrix, the effective channel, which is defined as

2 / 1 '
) (
s
H
eff
Q G =

(15)

Where
'
s
Q is the estimated version of
s
Q , can be obtained. Then the transmit covariance
matrix of the transmitter

H
tr tr tr
A A S =

(16)

Should, ideally, be designed such that

0 =
tr eff
A G
(17)

Vol. 3, No.4, 2011

4. Conclusions and Future research directions

In this paper, we provided a comprehensive overview of linear algebra, and its applications
to the research in the communication. We explained the fundamental concepts and properties,
and provided a detailed discussion about the methodologies on how to apply the linear algebra
in communication as MIMO, coding, and other fields.
In the future work we want to investigate roles of LA in other fields of electrical engineering
as control engineering and power engineering.

5. References

[1] C. D. Meyer, Matrix Analysis and Applied Linear Algebra, Siam, USA, 2000
[2] G. Strang, Linear Algebra and Its Applications, video lectures
[3] K. R. Rao, P. Yip, V. Britanak, Discrete cosine transform: Algorithms, Advantages, Applications,
Academic Press, USA, 1990
[4] G. Strang, Linear Algebra and Its Applications, 3thedi.,Thomson Learning, USA, 1988
[5] R. Zhang, Y. C. Liang, Exploiting multi-antennas for opportunistic spectrum sharing in cognitive
radio network, IEEE Journal on Selected Topics in Signal Processing, vol. 2, no. 1, pp.88-102,
Feb. 2008.
[6] R. Zhang, F. Gao, Y. C. Liang, Cognitive beam forming made practical: effective interference
channel and learning-throughput trade-off, IEEE Transaction on Selected Area in
Communication, vol. 58, no. 2, pp.706-718, Feb. 2010.


Ahmad Ghasemi Mostafa Abdollahi Ajast Paper-4

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ahmad Ghasemi Mostafa Abdollahi Ajast Paper-4

Uploaded by

Copyright:

Available Formats

International Journal of Advances in Science and Technology,

Vol. 3, No.4, 2011

You might also like