1

Chapter 1

Shirley Huang
Systems of linear equations
and matrices
2
Introduction
A Linear equation



A Linear system


b x a x a x a
n n
= + +
2 2 1 1
1 1 2 12 1 11
b x a x a x a
n n
= + +
m n mn m m
b x a x a x a = + +
2 2 1 1

3
Solving linear systems
Example: n=2


Two steps:

1. Elimination (equation 2 - 4 x equation 1)
-3y=-6 y=2

2. Substitution
y=2 x=-1


6 5 4
3 2
= +
= +
y x
y x
Linear system
There are 3 possibilities:

- zero solution,

- one solution,

- infinitely many solutions,
4
0
3
= +
= +
y x
y x
6 2 2
3
= +
= +
y x
y x
6 2
3
= +
= +
y x
y x
5
Linear system

An equation system that has no solution is
said to be inconsistent.
If there is at least one solution, it is called
consistent.
6
Matrix
An mn matrix is a rectangular array of numbers with m rows
and n columns. Each number in the matrix is called an entry.
For an equation system, we can write the rectangular array of
numbers. This is called augmented matrix.
(
(
(
(

m mn m m
n
n
b
b
b
a a a
a a
a a a

2
1
2 1
2 21
1 12 11
7
Elementary operations

1. An equation is swapped with another, swapping.
Interchange two rows

2. An equation has both sides multiplied by a nonzero
constant, rescaling.
Multiply a row by a nonzero constant

3. An equation is replaced by sum of itself and a multiple
of another equation, pivoting.
Add a multiple of one row to another row

8
Example






3 2
3
1
2 2 5
9 3
2 1
3 2 1
3
= +
= +
=
x x
x x x
x
1. swap equation 1 with equation3
2. multiply equation 1 by 3
3. add 1 X equation 1 to equation 2
9
Matrix form
(
(
(

3 0 2 3 / 1
2 2 5 1
9 3 0 0
10
Row echelon form
In each row, the first entry with nonzero
coefficient is the rows leading entry.
A system is in row echelon form if the leading
entry in a row is to the right of the leading entry
in the row above it and all leading entries are 1.
Echelon form: Staircase pattern
(
(
(
(

-
-
-
-
-
-
-
-
-
- -
0 0 0
0
0
0
0
0
0
(
(
(

4 1 0 0
1 3 1 0
3 1 2 1
11
Reduced Echelon form
The leading entry in each non-zero row is 1.
All other entries of the column in which the
leading entry 1 occurs are zeros
(
(
(
(

-
-
-
-
-
0 0
1
0
0
0
0
0
0
1
0
0
0
0
1
(
(
(

1
0
0
0
3
2
0
1
0
0
0
1
12
Elimination
The procedure that produces a
matrix in an echelon form is called
Gaussian elimination.



The procedure that produces a
matrix in a reduced echelon form is
called Gauss-Jordan elimination.
13
Exercise
Solve the system by Gaussian elimination and
Gaussian-Jordan elimination
10 4 7 3
1 3 2
8 2
3 2 1
3 2 1
3 2 1
= +
= +
= + +
x x x
x x x
x x x
14
Pivot variables and free variables
Pivot is the nonzero coefficient which is used to
eliminate the coefficient in the rows underneath it.
In each equation, the first variable with nonzero
coefficient is the equations leading variable.
The leading variables in an Echelon form linear
system are the pivot variables.
The non-leading variables in an Echelon form linear
system are the free variables.
15
Example:
2 3 3
6 2 4
1 4 2
4 3
4 2
4 1
= + +
= + +
= + +
x x
x x
x x
t x
t
x
t
x
t
x
=

=

=
4
3
2
1
,
3
3 2
,
4
2 6
,
2
4 1
(
(
(

2
6
1
3
2
4
3 0 0
0 4 0
0 0 2
Solution:
Free variables Leading variables
4
2 1
12
6 11
8
2 13
4
3
2
1
s
x
s
x
s
x
s x

=
+
=
+
=
=
16
Example
Solve the following system by using matrix row operations.




Echelon form: solution:


9
0
1
5
2 4
3
2

=
=
=
=
+

+
+

+
+ w
w
w
w
z
z
z
y
y
x
x
x
(
(
(
(

0
3
1
5
0 0 0 0
1 1 0 0
1 0 1 0
1 1 0 2
t w
t z
t y
x
=
=
+ =
=
3
1
1
17
Homogeneous linear systems
A linear equation system is said to be homogeneous
if the constant terms, are all zero.




is called the trivial solution of the homogeneous
system.
A homogeneous system with more unknowns than
equations has infinitely many solution (n>m).
i
b
0
1 2 12 1 11
= + +
n n
x a x a x a
0
2 2 1 1
= + +
n mn m m
x a x a x a

0 =
i
x
18
Matrix and vector
Recall: Matrix is a rectangular array of
numbers with m rows and n columns. Each
number in the matrix is an entry. The (i,j) entry
is denoted and equals the entry in the ith
row and jth column.
General form: Example:



(
(
(

=
9 8 7
6 5 4
3 2 1
A
ij
a
n m
mn m m
n
n
a a a
a a a
a a a
A

(
(
(
(

2 1
2 22 21
1 12 11
19
Some special matrices
Square matrix: a matrix with m=n

Diagonal matrix: a square matrix with all of
whose entries not on the main diagonal equal
zero

Low Triangular matrix: L,
=0 if i<j for 1im and 1jn

Upper Triangular matrix: U,
=0 if i>j for 1im and 1jn
ij
l
ij
u
(
(
(
(

nn
a
a
a

0 0
0 0
0 0
22
11
(
(
(
(

mn
n
n
a
a a
a a a

0 0
0
2 22
1 12 11
(
(
(
(

mn m m
a a a
a a
a

2 1
22 21
11
0
0 0
20
Some special matrices
Identity matrix : a diagonal matrix
I with =1


Band matrix: most entries are
zero, their nonzero entries are in
a band around the main diagonal


Partitioned matrix: a matrix can
be subdivided into smaller
matrices.

ii
i
(
(
(
(

1 0 0
0 1 0
0 0 1

(
(
(
(
(
(

mn n m
n m
a a
a
a a
a a
1 ,
, 1
22 21
12 11
0 0
0 0
0 0

D C
B A
21
Vector
A vector is a matrix with a single row or a single
column (row or column matrix).
Examples:



Pay-off of a security under 3 scenarios

| | 3 2 1 = a
(
(
(

=
6
5
4
b
(
(
(

=
1
7 . 0
2 . 1
1
a
22
Operations on matrices
Equality: A=B
Addition: A+B and subtraction: A-B
Multiplication:
1. Scalar multiple: cA
2. Product AB

23
Inner product
Row * Column




Example



| | cw bv au
w
v
u
c b a + + =
(
(
(

*
| | 32 6 * 3 5 * 2 4 * 1
6
5
4
* 3 2 1 = + + =
(
(
(

24
A matrix a vector
Ab by rows




Example




=
(
(
(

(
(
(

0
5
2
4 1 1
1 0 3
6 1 1
(
(
(
(

=
(
(
(
(

b a
b a
b a
b
a
a
a
Ab
m m

2
1
2
1
25
A matrix a vector
Ab by columns




Example




=
(
(
(

(
(
(

0
5
2
4 1 1
1 0 3
6 1 1
| |
n n
n
n
a b a b a b
b
b
b
a a a Ab

+ + + =
(
(
(
(

2 2 1 1
2
1
2 1
26
A matrix a matrix
By rows





The i, j entry of AB is the inner product of the
ith row of A and the jth column of B.

| |
2 1
2
1
22 21
12 11
22 21
12 11

=
(

= b b
a
a
b b
b b
a a
a a
AB
(

+ +
+ +
=
22 22 12 21 21 22 11 21
22 12 12 11 21 12 11 11
b a b a b a b a
b a b a b a b a

=
=
n
k
kj ik ij
b a AB
1
) (
27
matrix matrix
By columns



Note:
The parameters of A and B must be comparable!

| | | |
2 1 2 1
= = Ab Ab b b A AB
Matrix products as linear combinations
A linear combination:


where are scalars.
Ab by columns is a linear combination!

28
r r
A c A c A c + + +
2 2 1 1
r
c c c , , ,
2 1

| |
n n
n
n
a b a b a b
b
b
b
a a a Ab

+ + + =
(
(
(
(

2 2 1 1
2
1
2 1
Matrix form of a linear system
Write equation system as



which is
29
(
(
(
(

=
(
(
(
(

+ +
+ +
+ +
m n mn m m
n n
n n
b
b
b
x a x a x a
x a x a x a
x a x a x a

2
1
2 2 1 1
2 2 22 1 21
1 2 12 1 11
(
(
(
(

=
(
(
(
(

(
(
(
(

m n mn m m
n
n
b
b
b
x
x
x
a a a
a a a
a a a

2
1
2
1
2 1
2 22 21
1 12 11
b Ax=
30
Properties of matrix operation
A+B=B+A
A+(B+C)=(A+B)+C
A(BC)=(AB)C
A(BC)=ABAC
(BC)A=BACA
a(BC)=aBaC
(ab)C=aCbC
a(bC)=(ab)C
a(BC)=(aB)C=B(aC)
Identity matrix: I, AI=A and IA=A

31
Properties of matrix operations
Matrix multiplication is not commutative:
Usually ABBA

The cancellation law does not always
hold: AB=AC B=C

AB=0 A=0 or B=0

32
Example
Compute AB, 2AC, B+2C and compare AB+2AC with
(B+2C)A.

(

=
1
2
0
1
3
1
A
(
(
(

=
0
1
1
1
2
1
B
(
(
(

=
0
3
2
0
1
2
C
33
Transpose of matrix
The transpose of a matrix: interchange the rows and columns
of A:



Example:





Trace of A, tr(A) is defined as the sum of the entries on the
main diagonal of A.
ji
T
ij
A A = ) (
=
T
A
(
(
(

=
9 6 3
8 5 2
7 4 1
A
) ( ) (
T
A Tr A Tr =
34
Transpose of matrices
T T T
A B AB = ) (
A A
T T
= ) (
T T T
B A B A + = + ) (
Properties:
T T
kA kA = ) (
35
Inverse of a matrix
Inverse of A: If A is a square matrix and if there
is a matrix B such that AB=BA=I, then A is said
to be invertible and B is called an inverse of A,
denoted by . If no such matrix B can be
found , then A is said to be Singular.

1
A
36
Inverse
A 2 by 2 matrix



is invertible if ad-bc0. The inverse is given by

(

=
d c
b a
A
(
(
(

bc ad
a
bc ad
c
bc ad
b
bc ad
d
A
1
37
Matrix inverses
An invertible matrix has only one inverse

Inverse of A:
I AA A A = =
1 1
A A =
1 1
) (
1 1 1
) (

= A B AB
1 1
) ( ) (

=
T T
A A
38
Exercise
Find the and where
(

=
4 2
3 1
A
T
A
1
A
39
Powers of a matrix
If A is a square matrix and invertible

I A =
0
A AA A
n
=
n n
A A ) (
1
=
s r s r
A A A
+
=
rs s r
A A = ) (
1 1
1
) (

= A
k
kA
T T
A A ) ( ) (
1 1
=
40
Elementary matrices
Elimination: add a multiple k of equation i to equation j.
(inverse: add a multiple -k of equation i to equation j )
-Matrix: Put the number k into the (j,i) position of an identity matrix.
(inverse: Put the number -k into the (j,i) position of an I matrix )

Rescaling: k times equation i
(inverse: equation i divided by k)
-Matrix: Put the number k into the (i,i) position of an identity matrix.
(inverse: Put the number 1/k into the (i,i) position of I )

These matrices are called elementary matrices. All elementary
matrices are invertible.
41
Exercises
Find the Elementary matrices that reduce A to
echelon form where




Describe the rows of EA and Columns of AE if
(
(
(

=
10 8 4
8 5 2
3 2 1
A
(
(
(

=
1 0 0
0 1 0
0 6 1
E
42
Relationship
If A is a square matrix (n by n), then the
following statements are equivalent.

1. A is invertible
2. Ax=0 has only the trivial solution
3. The reduced form of A is I
4. A is expressible as a product of elementary matrices.

43
Using row operations to find
If A is invertible, then



Multiplying both sides by yields



A sequence of elementary row operations
reduces A to I; the same sequence of
elementary row operations turns I to .

I A E E E
k
=
1 2

1
1 2

= A I E E E
k

1
A
1
A
1
A
44
Finding
1
A
| | I A |
| |
1
|

A I
Exercise:
(
(
(

=
2 7 2
0 6 4
1 1 2
A
?
1
=

A
45
Solving linear systems
If A is an invertible matrix, then for each n by 1
matrix b, the system of equations, Ax=b has
exactly one solution.



b A x
1
=
46
Diagonal matrices
General form:
(
(
(
(

=
n
d
d
d
D

0 0
0 0
0 0
2
1
(
(
(
(
(

=
k
n
k
k
k
d
d
d
D

0 0
0 0
0 0
2
1
(
(
(
(
(
(
(
(

n
d
d
d
D
1
0 0
0
1
0
0 0
1
2
1
1

47
Triangular matrices
The transpose of a lower triangular matrix is
upper triangular.
The product of a lower triangular matrices is
lower triangular.
A triangular matrix is invertible if and only if
its diagonal entries are all nonzero.
The inverse of lower (upper) triangular matrix
is lower (upper) triangular.
48
Example

Find where
1
A
(
(
(

=
6 0 0
5 4 0
3 2 1
A
49
Symmetric
A matrix A is said to be symmetric if


If A and B are symmetric matrices,

1. A+B and A-B are symmetric
2. kA is symmetric
3. However, generally the product AB is not symmetric!
(Only when AB=BA: A and B commute!)
A A
T
=
50
Symmetric
If A is an invertible symmetric matrix, then
is symmetric.

For any matrix A, is a symmetric matrix
and square matrix.



T
AA
1
A
51
Exercise
If A is a symmetric matrix, show


is symmetric.


I A A +
1 2
3 2