Teachers Training Programme-2011

Prepared by Dr. G.S.Rao

P
a
g
e
1

3. Systems of Linear Equations
A linear equation in the variables
n
x x x ,..., ,
2 1
over the real field R is an equation that can be
written in the form
b x a x a x a
n n
= + + + ....
2 2 1 1
(1)
Where b and the coefficients
n
a a a ,..., ,
2 1
are given real numbers
A solution to the linear equation is a sequence of numbers
n
s s s ,..., ,
2 1
which has the property that
the linear equation is satisfied when
n n
s x s x s x = = = ,..., ,
2 2 1 1
.
A system of m linear equations in n unknowns,
n
x x x ,..., ,
2 1
, i.e.,
m n mn m m
n n
n n
b x a x a x a
b x a x a x a
b x a x a x a
= + + +
= + + +
= + + +
...
...
...
2 2 1 1
1 2 2 22 1 21
1 1 2 12 1 11
M
(2)
In matrix notation, the linear system (2) can be written as B AX = where
(
(
(
(
(
(
(

=
(
(
(
(
(
(
(

=
n mn m m
n
n
x
x
x
X
a a a
a a a
a a a
A
.
.
.
,
...
.
.
.
...
...
2
1
2 1
2 22 21
1 12 11
and
(
(
(
(
(
(
(

=
m
b
b
b
B
.
.
.
2
1

We call A the coefficient matrix of the system (2), and the matrix
(
(
(
(
(

m mn m m
n
n
b
b
b
a a a
a a a
a a a
M
L
M M M
L
L
2
1
2 1
2 22 21
1 12 11
is called the augmented matrix of the system.
For example for the non homogeneous linear system
5 3 3 2
4 2
2 3
3 2 1
3 2
3 2 1
=
=
= +
x x x
x x
x x x
(3)
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
2

The matrix
(
(
(

=
3 3 2
2 1 0
1 3 1
A is the coefficient matrix and
(
(
(

5 3 3 2
4 2 1 0
2 1 3 1
is the
augmented matrix
A solution of a linear system in n-unknowns
n
x x x ,..., ,
2 1
is an n-tuple ) ,..., , (
2 1 n
s s s of real
number that makes each of the equations in the system a true statement when the values
n
s s s ,..., ,
2 1
are substituted for
n
x x x ,..., ,
2 1
respectively. The set of all possible solution is called
the solution set of the linear system
If the linear system has no solution, it is said to be inconsistent; if it has a solution, it is called
consistent.
If 0 ...
2 1
= = = =
m
b b b , then the system of linear equation is called a homogenous system
Note that 0 ...
2 1
= = = =
m
x x x is always a solution to a homogenous system; it is called a trivial
solution. Otherwise, it is called a nontrivial solution.
If 0
i
b for some i, then the system (2) is called non-homogeneous.
A system of linear equations has either
1. no solution, or
2. exactly one solution, or
3. Infinitely many solutions.










Inconsistent Consistent
Unique Solutions More than
one solution
No solution
System of Linear equation
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
3

Geometrically Interpretation of solution sets for linear system:
Let us consider the system of two equations

1 2 12 1 11
b x a x a = +

2 2 22 1 21
b x a x a = +
Geometrically, solutions to each equation are represented by lines. A simultaneous solution
corresponds to a point of intersections. These there are three possibilities
1. The two lines are coincident (the same line) so there are infinitely many solutions.
2. The two lines are parallel, so there are no solution
3. The two lines intersect at a single point, so there is a unique solution.







Coincident lines definitely parallel lines intersecting lines
many solution No solution unique solution.
Solving a linear system:
This is the process of finding the solutions of a linear system. We first see the technique of
elimination. The basic operations that are used to produce an equivalent system of linear
equations (We say that two linear systems are equivalent if they have the same solution set.),
which is easier to solve, are the following:
1. Replace one equation by the sum of itself and a multiple of another equation.
2. Interchange two equations
3. Multiply all the terms in an equation by a non-zero constant.
We illustrate this technique by using the following example.


Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
4

Example 1 Solve the system

5 3 3 2
4 2
2 3
3 2 1
3 2
3 2 1
=
=
= +
x x x
x x
x x x

Solution: We perform the elimination procedure with and without matrix notation of the system,
and put the results side by side for comparison:

5 3 3 2
4 2
2 3
3 2 1
3 2
3 2 1
=
=
= +
x x x
x x
x x x

(
(
(

5 3 3 2
4 2 1 0
2 1 3 1

We keep x
1
in the first equation and eliminate it from the other equations. For this replace the
third equation by the sum of itself and two times equation 1.

[ ]
[ ]
9 5 3
5 3 3 2
4 2 6 2
] 3 . [
: 3 .
: 1 . . 2
3 2
3 2 1
3 2 1
=
=
= +
+
x x
x x x
x x x
eq New
eq
eq

We write the new equation in place of the original third equation:

9 5 3
4 2
2 3
3 2
3 2
3 2 1
=
=
= +
x x
x x
x x x

(
(
(

9 5 3 0
4 2 1 0
2 1 3 1

Next use the x
2
in equation 2 to eliminate 3x
2
in equation 3.

[ ]
[ ]
3
5 5 3
12 6 3
] 3 . [
: 3 .
: 2 . . 3
3
3 2
3 2
=
=
= +
+

x
x x
x x
eq New
eq
eq

The resulting equivalent system is:

3
4 2
2 3
3
3 2
3 2 1
=
=
= +
x
x x
x x x

(
(
(

3 1 0 0
4 2 1 0
2 1 3 1

Now we dominate the 2x
3
term form equation 2. For this we use x
3
in equation 3.

[ ]
[ ]
3
4 2
6 2
] 3 . [
: 2 .
: 3 . . 2
3
3 2
3
=
=
=
+
x
x x
x
eq New
eq
eq

Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
5

From this we get

3
2
2 3
3
2
3 2 1
=
=
= +
x
x
x x x

(
(
(

3 1 0 0
2 0 1 0
2 1 3 1

Again by using the x
3
term in equation 3, we eliminate the x
3
term in equation 1.
[ ]
[ ]
1 3
2 3
3
] 1 . [
: 1 .
: 3 . . 1
2 1
3 2 1
3
= +
= +
=
+
x x
x x x
x
eq New
eq
eq

Thus we get the system

3
2
1 3
3
2
2 1
=
=
= +
x
x
x x

(
(
(

3 1 0 0
2 0 1 0
1 0 3 1

Finally, we eliminate the
2
3x term in equation 1. We use the x
2
term in equation 2 to eliminate
the
2
3x term above it.
[ ]
[ ]
5
2 3
6 3
] 1 . [
: 1 .
: 2 . . 3
1
2 1
2
=
= +
=
+

x
x x
x
eq New
eq
eq

So we have an equivalent system (to the original system) that is easier to solve.
(
(
(

=
=
=
3 1 0 0
2 0 1 0
5 0 0 1
3
2
5
3
2
1
x
x
x

Thus the system has only one solution, namely (5, -2, -3).
To verify that (5, -2, -3) is a solution, substitute these values in to the left side of the original
system, and compute:
5 + 3(-2) - (-3) = 5 6 + 3 = 2
-2 - 2(-3) = -2 + 6 = 4
-2(5) - 3(-2) - 3(-3) = -10 + 6 + 9 = 5
It is a solution, as it satisfies all the equation in the given system (3)
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
6

Example1 illustrates how operations in a linear system correspond to operations on the
appropriate rows of the augmented matrix. The three basic operations listed earlier correspond to
the following operations on the augmented matrix. We call them elementary row (column)
operations.
1. (Replacement) Replace one row (column), say i, by the sum of itself and a multiple
of another row ( column), say j, abbreviated as ) (
i j i i j i
C C k C R R k R + +
2. (Interchange) Interchange two rows (columns) and (say i and j). This is abbreviated as
) (
j i j i
C k C R k R .
3. (Scaling) Multiply all entries in a row (column), say i, nonzero constant (scalar)k,
abbreviated as ) (
i i i i
C k C R k R
Definition: These three matrix operations defined above are called elementary row (column)
transformations or operations
Note: These row operations can be applied to any matrix, not merely to one that arises as the
augmented matrix of a linear system.
Definition: We say that two matrices are row (column) equivalent if one is obtained from the
other by a finite sequence of elementary row (column) operations. We can write it A B.
Notation:
1. Let us denote an elementary row (column) operation by, and the reverse operation
-1
.
2. If A B, then there exist elementary row operations
1
,
2

n
such that

n

n-1

2

1
A = B
Remark: It is important to note that row operations are reversible. i.e. the inverse of elementary
row (column) operation is an elementary row ( column) operation.
(i) If two rows are interchanged, they can be returned to their original positions by
another interchange. i.e,
= interchanging rows i and j then
-1
= interchanging rows j and i
(ii) If a row is scaled by a nonzero constant k, then multiplying the new row by
k
1

produces the original row. i.e.,
= multiplying row i by a nonzero k , then
-1
= multiplying row i by a nonzero
k
1

(iii) Finally, consider a replacement operation involving two rows, say rows i and j,
and suppose k times row i is added to row j to produce a new row j. To
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
7

reverse this operation, add k times row i to new row j and obtain the original
row j. i.e.,
= adding k-times row j to row i, then
-1
= adding k times row j to row i
Therefore, we conclude that the inverse of an elementary row operation is an elementary row
operation (transformation)
3.1 Proposition: Row equivalent ( ) is an equivalence relation.
Proof: proof follows from the above remark
3.2 Theorem: If the augmented matrices of two linear systems are row equivalent, then the two
systems have the same solution set. (OR)
Let A and B be the augmented matrices of two systems in n unknowns. If A B, then the two
systems have the same solutions.
Proof: proof follows from the elementary row operations
Thus to solve a linear system we first perform appropriate row operations on the
augmented matrix of the system to obtain the augmented matrix of an equivalent system of linear
equations which is easier to solve and use back substitution on the resulting new system. This
method can also be used to answer questions about existence and uniqueness of a solution.
Example 2 Determine if the following system is consistent. If so how many solutions does it
have?

1 2
2 5 5
3
3 2 1
3 2 1
3 2 1
= +
= +
= +
x x x
x x x
x x x

Solution: The augmented matrix is
(
(
(

=
1 1 1 2
2 5 5 1
3 1 1 1
A
Let us perform a finite sequence of elementary row operations on the augmented matrix.
3 1 3 2 1 2
2
1 1 1 2
1 6 6 0
3 1 1 1
1
1 1 1 2
2 5 5 1
3 1 1 1
~
R R R R R R A +
(
(
(

+
(
(
(

=
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
8

(
(
(
(

(
(
(

2
9
0 0 0
1 6 6 0
3 1 1 1
2
1
5 3 3 0
1 6 6 0
3 1 1 1
3 2 3
R R R
The corresponding linear system is the last matrix is

2
9
0
1 6 6
3
3 2
3 2 1

=
=
= +
x x
x x x
(*)
But the last equation
2
9
. 0 . 0 . 0
3 2 1

= + + x x x is never true. That is there are no values

3 2 1
, , x x x that satisfy the new system (*). Since (*) and the original linear system have the same
solution set, the original system is inconsistent (has no solution).
Echelon Form of a Matrix
An mxn matrix A is said to be in reduced row echelon form if it satisfies the following
properties:
(a) All zero rows if there are any; appear at the bottom of the matrix.
(b) The first leading entry from the left of a nonzero row is unity (the number 1). This entry is
called a leading entry of its row.
(c) For each nonzero row, the leading entry appears to the right and below any leading entries in
the preceding rows.
(d) If a column contains a leading entry, then all other entries in that column are zero.
An mxn matrix satisfying properties, (a), (b) and (c) is said to be in row echelon form.
Typical Form of a reduced row-echelon matrix
|
|
|
|
|
|

\
|
0
*
*
*
*
0
1
0
0
0
0 0 0 0 0
0 0 0 0 0
* 1 0 0 0
* 0 1 0 0
* 0 0 * 1

Informal way: Any matrix is a row-echelon form if all non-zero entries appear in a stair case
shaped region in the upper hand portion of the matrix.
3.3 Theorem: Every matrix can be transform into reduced row echelon form by using
elementary row operation i.e., every matrix is row-equivalent to a reduced row echelon matrix.
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
9

Proof: omitted
Remark: The reduced row echelon form of a matrix is unique. i.e., if different sequence of
elementary row operations are used to transform a matrix into matrices Q and Q
1
in reduce row
echelon form then Q = Q
1
.
Note: (reduced) row echelon form is an important role to determine the solution of the system of
linear equations.
Example: We do row operations on matrix below to convert to rref (reduced row echelon form)

Multiply row 1 by 2 and add to row 2 i.e,
1 2 2
2 R R R +

Multiply row 1 by -3 and add to row 3 i.e.,
1 3 3
) 3 ( R R R +

Multiply row 2 by 1 and add to row 3 i.e.,
1 3 3
. 1 R R R +

We are now in ref (row echelon form), we continue:
Multiply row 2 by -2 and add to row 1 i.e.,
1 1 1
) 2 ( R R R +

The matrix is now in rref.


Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
1
0

Rank of a Matrix:
Definition: Let A = ) (
ij
a be an m n matrix with elements in the field F. The row vectors
R
1
, R
2
R
m
of A belong to (F) V
n
. The subspace W of (F) V
n
spanned by the row vectors of A
is called the row space of A.
Similarly, we can define the column space.
Recall:
Let
(
(
(
(

=
mn m m
n
n
a a a
a a a
a a a
A
L
M O M M
L
L
2 1
2 22 21
1 12 11
.
The ith row of A is
[ ] , , , 2 , 1 ,
2 1
m i a a a R
in i i i
K L = = ,
and the jth column of A is
. , , 2 , 1 ,
2
1
n j
a
a
a
C
mj
j
j
j
K
M
=
(
(
(
(
(

=
Definition of row space and column space:
{ }
m
R R R span , , ,
2 1
K , which is a vector space under standard matrix addition and scalar
multiplication, is referred to as the row space.
Similarly, { }
n
C C C span , , ,
2 1
K which is also a vector space under standard matrix addition and
scalar multiplication, is referred to as the column space.
Important Result:
3.4 Theorem: If A and B are two n m row equivalent matrices, then the row spaces of A and B
are equal.
The following theorem will be useful for investigating linear independence of vectors
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
1
1

3.5 Theorem: The non-zero row vectors of a reduced row-echelon matrix A are linearly
independent, and hence, form a basis for the row space of A
3.6 Corollary: If A is any matrix and R
A

is a reduced row-echelon matrix of A. Then non zero
row vectors of R
A
constitute a basis for the row space of A
In view of above corollary the row reduction of a matrix gives a systematic procedure for finding
a basis for the space spanned by a given set of vectors in (F) V
n

How to find the bases of the row and column spaces:
Suppose A is a n m matrix. Then, the bases of the row and column spaces can be found via the
following steps.
Step 1: Transform the matrix A to the matrix in reduced row echelon form.
Step 2:
The nonzero rows of the matrix in rref a basis of the row space of A.
The columns corresponding to the ones containing the leading 1s form a basis.
For example, if 6 = n and the reduced row echelon matrix is
(
(
(
(
(
(
(
(

0 0 0 0 0 0
0 0 0 0 0 0
1 0 0 0
1 0 0
1
M M M M M M
,
then the 1st, the 3rd, and the 4th columns contain a leading 1 and thus
4 3 1
C , C , C form a basis
of the column space of A.
Note: To find the basis of the column space is to find to basis for the vector space
{ }
n
C C C span , , ,
2 1
K .
Example: Let
(
(
(
(

=
3 4 0 2 1
3 2 7 3 2
4 1 8 2 3
4 3 0 2 1
A .
Find the bases of the row and column spaces of A.
Solution: Step 1: Transform the matrix A to the matrix in reduced row echelon form,
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
1
2

(
(
(
(

(
(
(
(

=
0 0 0 0 0
1 1 0 0 0
1 0 1 1 0
1 0 2 0 1
3 4 0 2 1
3 2 7 3 2
4 1 8 2 3
4 3 0 2 1
form echelon row reduced in
A

Step 2:
The basis for the row space is
[ ] [ ] [ ] { } 1 1 0 0 0 , 1 0 1 1 0 , 1 0 2 0 1
The columns corresponding to the ones containing the leading 1s are the 1st, the 2nd, and
the 4th columns. Thus,

(
(
(
(

(
(
(
(

(
(
(
(

4
2
1
3
,
2
3
2
2
,
1
2
3
1
form a basis of the column space.
Definition of row rank and column rank:
Definition: The dimension of the row space of A is called the row rank of A and the dimension
of the column space of A is called the column rank of A.
Example (continue): Since the basis of the row space of A is
[ ] [ ] [ ] { } 1 1 0 0 0 , 1 0 1 1 0 , 1 0 2 0 1 ,
the dimension of the row space is 3 and the row rank of A is 3. Similarly,

(
(
(
(

(
(
(
(

(
(
(
(

4
2
1
3
,
2
3
2
2
,
1
2
3
1

is the basis of the column space of A. Thus, the dimension of the column space is 3 and the
column rank of A is 3. (Note that row rank = column rank)
3.7 Corollary: The rank of a matrix A is equal to the number of non-zero rows in reduced row-
echelon matrix row equivalent to A.
3.8 Corollary: If A is nn matrix of rank n, the reduced row echelon matrix row equivalent to A
is the identity matrix I
n
.
Proof: This follows from above corollary and the definition of rrem

Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
1
3

The Solution Space of a Homogenous System:
Recall that a system of linear equations can be put into the matrix form
AX = b (1)
Where A = ) (
ij
a be an m n matrix with elements in the field F and

) ( 1 ,
.
. ) ( 1 ,
.
.
2
1
2
1
enries scalar vector column m
b
b
b
b and entires unknow vector column n
x
x
x
X
m n

|
|
|
|
|
|

\
|
=
|
|
|
|
|
|

\
|
=

Definition: When b = 0, the system (1) is called homogeneous i.e., AX =0.
3.9 Theorem: Let A = ) (
ij
a be an m n matrix over the field F. Then the set of all solutions of
the homogenous system of linear equations AX = 0 (null matrix) with coefficient matrix A, is a
subspace of (F) V
n
.
Definition: The sub space S of V
n
(F) is called the solution space of the system. It is also called
the null space of the matrix A.
3.10 Theorem: The solution space S of a homogenous system AX = 0 has dimension n-r, where
n is the number of unknowns, and r is the rank of the coefficient matrix A.
Proof: omitted
3.11 Corollary: A homogenous system Ax = 0 in n unknowns has only the zero solution X = 0
iff rank A = n
Example:
3.12 Lemma: Let A be an m n matrix and X an n 1 matrix, then AX is a linear combination of
the column vectors.
Remark:
1. Let A be an m n matrix and X an n 1 matrix over the field. Now, we can easily
identify X with the vector of (F) V
n
having the same coordinates relative to the standard
basis. It clearly makes no difference to the algebra of vectors in (F) V
n
whether we write
these vectors as rows or columns, and since we want to consider vectors of the form AX,
this necessitates writing X as a column vector. In spite of this, we still consider X as a
vector in (F) V
n
.
Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
1
4

With this understanding, we can consider the following linear transformation.
2. Let (F) V (F) V : T
m n
be a mapping defined by T(X) = AX, where A is m n matrix
T is well defined : because A is m n matrix and X is n 1, AX is m 1 vector belong
to (F) V
m

T is a linear transformation : if X
1
, X
2
(F) V
n
and a, b F, then
T (aX
1
+ bX
2
) = a T(X
1
) b T(X
2
)
It clearly makes no difference
3.13 Lemma: Let (F) V (F) V : T
m n
be a linear transformation, defined T(X) = AX, where A is
m n matrix and X is n 1. Prove that ImT is a subspace of (F) V
m
spanned by column vectors
Definitions: Let A be an m n matrix, we can define the following three spaces.
(i) The row space of A, the subspace V of (F) V
n
spanned by the row vectors of A.
(ii) The column space of A, the subspace W of (F) V
m
spanned by the column
vectors of A. This subspace is identified with the space ImT = {AX: X (F) V
n
}.
(iii) The null space N is the solution space of the homogenous system AX = 0.
3,14 Theorem (Dimension Theorem): For any m n matrix A with column space W and null
space N, then dim N + dim W = n
Proof: omitted
3.15 Corollary: For any m n matrix A, the dimension of the column space is equal to the
dimension of the row space.
3.16 Corollary: If A is a square matrix, the column vectors of A are linearly independent iff the
row vectors of A are linearly independent.
Definition: The dimension of the null space N of a matrix A is called the nullity of A.
Remark: Since by corollary, (3.16 ), dim W = rank A, therefore theorem (3.14 ) can be restated
as
For any m n matrix A, rank A + nullity A = n
Example:



Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
1
5

Non-homogeneous System:
We know that a system AX = 0 of homogeneous linear equations in n unknown has n-r linearly
independent solutions vectors where r = rank A. Moreover, we know how to find these solutions
by reduction of A to row echelon form as described in previous section. We have also seen that
the same reduction process applied to the augmented matrix can be used to find the solutions of a
non-homogeneous system when these exist
Definition: A non-homogeneous system of linear equations of the form AX = b, where b 0
We say that the system is consistent iff it has a solution.
The augmented matrix of the system AX = b is written as ( A|b) or [A | b]
3.17 Theorem: Let AX = b be a system of linear equations. Then the system is consistent if and
only if rank (A) = rank (A |b). i.e., rank of the coefficient matrix = rank of augmented matrix.
3.18 Corollary: A homogeneous system is always consistent
Example:
We now prove a theorem which gives the relation between the solution of the system Ax = b and
Ax = 0
3.19 Theorem:
(i) If S is a fixed solution vector of AX = b, and T is a solution of Ax = 0, then S + T is a
solution vector of AX = b.
(ii) Every solution vector of AX = b is of the form S+T, where S is a particular solution
vector of AX = b and T is a solution of AX = 0
Example:
3.20 Theorem: If A is an n n matrix, the following three statements are equivalent
(i) The rank of A is n
(ii) The system AX = b has a unique solution
(iii) The system of AX = 0 has only trivial solution 0 ...
2 1
= = = =
n
x x x




Teachers Training Programme-2011
Prepared by Dr. G.S.Rao
P
a
g
e
1
6

Nonsingular Matrices and Matrix Inversion:
Definition: An n n matrix A is said to be non-singular if rank A = n and singular if
rank A < n.
Definition: If A is an n n matrix, an n n matrix B such that AB = BA = I is called the inverse
of A. We write
-1
A B = or dually
-1
B A = , where I is an nn identity matrix.
We will discuss the following properties without proof
3.21 Properties:
1. An n n matrix has an inverse if and only if it is non-singular i.e.,
A has inverse if and only if rank A = n
2. The inverse of a non-singular matrix is unique.
3.
If A is non-singular then
-1
A is non-singular.

4. If A and B are non-singular n n matrices, then AB is non-singular and
1 -1
) ( (AB)

= BA
5.
If
n
A A ... , A
2 1
are non-singular n n matrices,

1
1
!
1
1 1
2 1
.... ) ... , (A

= A A A A A
n n n
.
In particular
m
A ) ( ) (A
1 1 m
=

6.
If A is non-singular n n matrix, then A
T
is non-singular and
T
A ) ( ) (A
1 -1 T
=

7.
If A is invertible (or non-singular) n n matrix, and the matrix ( A | I) is transformed into
a matrix of the form ( B| I) by means of finite number of elementary row operations,
then
-1
A B =

Example: