1

Chapter 1. Systems of Linear Equations and Matrices

1.1

Introduction to Systems of Linear Equations

1. The equation
2x + 3y = 4
is a linear equation in the two variables x and y.
Geometrically, it represents a straight line in the xy-plane (the rectangular xy-coordinate
system).
2. The equation
x 2y + 3z = 4
is a linear equation in the three variables x, y and z.
Geometrically, it represents a plane in the rectangular xyz-coordinate system.
3. Definition.
A linear equation in n variables x1 , , xn has the form
a1 x1 + a2 x2 + + an xn = b.
The coefficients a1 , , an are real numbers,
the constant term b is a real number,
and the numbers a1 , , an are not all zero.
4. In the special case when b = 0, equation (1) takes the form
a1 x1 + a2 x2 + + an xn = 0,
and it is called a homogeneous linear equation in n variables x1 , , xn .
5. Examples of linear equation
(a) The equation
2x + 3y = 4
is a non-homogeneous linear equation.
(b) The equation
x 2y + 3z = 4
is a non-homogeneous linear equation.

(1)

2
(c) The equation
2x + 3y = 0
is a homogeneous linear equation.
Geometrically it represents a straight line which passes through the origin.
6. Examples of nonlinear equations

x 4y = 7

sin x + 5y = 6
3x + xy 7y = 8
ex + y = 4
5x + 4y 2 = 3
7. A finite set of linear equations is called a system of linear equations or, more briefly, a
linear system. The variables are called the unknowns.
8. Example. The system
2x + y = 1
x 2y = 3
is a system of two linear euqations in two variables.
9. Definition.
A system of m linear equations in n variables has the form
a11 x1 + a12 x2 + + a1n xn = b1
a21 x1 + a22 x2 + + a2n xn = b2

am1 x1 + am2 x2 + + amn xn = bm

10. A solution of a linear equation in n variables x1 , , xn is a sequence of numbers
s1 , , sn for which the substitutions
x1 = s 1 ,

xn = sn

make the equation a true statement.

This solution can be written as
(s1 , s2 , , sn )
which is called an ordered n-tuple.

3
11. Example:
For the equation
2x + 3y = 4,
(0, 4/3) is a solution,
(2, 0) is a solution,
(1, 2/3) is a solution,
.
12. A solution of a system of linear equations in n variables x1 , , xn is a sequence of
numbers s1 , , sn for which the substitutions
x1 = s 1 ,

xn = sn

make each equation in the system a true statement.

This solution can be written as
(s1 , s2 , , sn )
which is called an ordered n-tuple.
An ordered 2-tuple is also called an ordered pair.
An ordered 3-tuple is also called an ordered triple.
13. Example:
For the system
xy =3
x + y = 7,
(5, 2) is a solution,
(3, 0) is not a solution.
14. A system of linear equations is called consistent if it has at least one solution.
A system of linear equations is called inconsistent if it has no solutions.
15. For a system of linear equations, precisely one of the following is true:
(a) The system has exactly one solution.
(b) The system has infinitely many solutions.
(c) The system has no solution.
16. Example. The system
xy =3
x+y =7
has a unique solution. It is consistent.

4
17. Example. The system
x 2y = 3
2x 4y = 6
has infinitely many solutions. It is consistent.
18. Example. The system
x 2y = 3
x 2y = 4
has no solutions. It is inconsistent.
19. If a linear system has infinitely many solutions,
then the set of all solutions of the linear system is called the solution set,
and this solution set has a parametric representation.
20. Example. The system
x+y+z =1
x+y+z =2
x+y+z =3
has no solutions. It is inconsistent.
21. Example. The system
x+y =1
y+z =2
x+z =3
has exactly one solution. It is consistent.
22. Example. The system
x + y 2z = 1
x 2y + z = 1
2x + y + z = 1
has no solution. It is inconsistent.
23. Example. The system
x+y+z =1
x+y+z =2
x+yz =3
has no solution. It is inconsistent.

5
24. Example. The system
x+y =2
y+z =2
xz =0
has no solution. It is inconsistent.
25. Example. The system
x + y 2z = 1
x 2y + z = 1
has infinitely many solutions. It is consistent.
Each of the following is a solution,
, (1, 0, 0), (2, 1, 1), (3, 2, 2), .
The general solution is
{(1 + t, t, t) : t is a real number .}
26. A system of m linear equations in n variables

a11 x1 + a12 x2 + + a1n xn = b1

a21 x1 + a22 x2 + + a2n xn = b2

am1 x1 + am2 x2 + + amn xn = bm

is associated with the rectangular array of numbers
a11
a
21

am1

a12
a22

am2

a1n
a2n

amn

b1
b2

bm

called the augmented matrix of the system.

The augmented matrix of a linear system is a condensed notation of the linear system.
27. For the system of m linear equations in n variables
a11 x1 + a12 x2 + + a1n xn = b1
a21 x1 + a22 x2 + + a2n xn = b2

am1 x1 + am2 x2 + + amn xn = bm ,

6
the matrix

a11
a
21

am1

a12
a22

am2

a1n
a2n

amn

is called the coefficient matrix of the system.

28. Example.
Write the augmented matrix and the coefficient matrix of the system
x + y 2z = 1
x 2y + z = 1
2x + y + z = 1
29. The augmented matrix and the coefficient matrix are not the same.
30. Two systems of linear equations are called equivalent if they have the same solution
set.
31. Example.
The systems
xy =3
x+y =7
and
2x y = 8
x + 2y = 9
are equivalent.
32. Operations that lead to equivalent systems of equations
(a) Interchange two equations.
(b) Multiply an equation by a nonzero number.
(c) Add a multiple of an equation to another equation.
Each of the above corresponds to an elementary row operation on the augmented matrix.
33. Elementary row operations (on matrices)
(a) Interchange two rows.
(b) Multiply a row by a nonzero number.
(c) Add a multiple of a row to another row.

7
34. Notations
(a) Ri Rj means to interchange the i-th row with the j-th row.
(b) cRi Ri means to multiply the i-th row by c.
(c) Ri + cRj Ri means to add c times the j-th row to the i-th row.
35. In general, the operation cRi + Rj Ri is not an elementary row operation.
36. In general, the operation cRi + dRj Ri is not an elementary row operation.
37. For example,
The operation
2R1 R2 R1
is not an elementary row operation.
38. Example.
2x + y = 1
x 2y = 3.
Practice Problems for Section 1:
Page 9: 1, 3, 5, 7, 9, 11, 13.

8
KEY WORDS OF SECTION 1
linear equation
variables
coefficients
homogeneous linear equation
system of linear equations
a solution of a linear equation
a solution of a system of linear equations
consistent
inconsistent
three possibilities for systems of linear equations
solution set
parametric representation
augmented matrix
coefficient matrix
equivalent systems
the three types of elementary row operations

1.2

Gaussian Elimination

1. Definition. Let m and n be positive integers. An m n matrix is a rectangular array

a11
a
21

am1

a12
a22

am2

a1n
a2n

amn

in which each entry aij is a real number. An m n matrix has m rows and n columns.
2. Example.
The matrix

0 1 7
5 9 3

is a 2 3 matrix.
3. Definition. A matrix is said to be in the row-echelon form if
(a) All rows consisting entirely of zeros occur at the bottom of the matrix.
(b) For each row that does not consist entirely of zeros, the first nonzero entry is 1
(called a leading 1).
(c) For two successive (nonzero) rows, the leading 1 in the higher row is farther to the
left than the leading 1 in the lower row.
4. Definition.
A matrix in row-echelon form is in reduced row-echelon form if every column that has
a leading 1 has zeros in every position above and below its leading 1.
5. Examples.
The matrix

0 0 0
5 9 3

2 3 4
0 0 0

0 1 2
1 0 3

A=
is not in the row-echelon form.
The matrix
B=
is not in the row-echelon form.
The matrix
C=
is not in the row-echelon form.
The matrix

D=

1 2 5
0 1 7

10
is in the row-echelon form, but not in the reduced row-echelon form.
The matrix

E=

1 0 4
0 1 3

is in the row-echelon form, and is also in the reduced row-echelon form.

6. Gaussian elimination The procedure of reducing a matrix into its row echelon form.
(a) Step 1. Locate the leftmost column that does not consist entirely of zeros.
(b) Step 2. Interchange the top row with another row, if necessary, to bring a nonzero
entry to the top of the column found in Step 1.
(c) Step 3. If the entry that is now at the top of the column found in Step 1 is a,
multiply the first row by 1/a in order to introduce a leading 1.
(d) Step 4. Add suitable multiples of the top row to the rows below so that all entries
below the leading 1 become zeros.
(e) Step 5. Now cover the top row in the matrix and begin again with Step 1 applied
to the sub-matrix that remains. Continue in this way until the entire matrix is in
row echelon form.
7. Example.

0 0 2 1

2 4 10 6
2 4 5 6

8. Gauss-Jordon elimination The procedure of reducing a matrix into its reduced row
echelon form.
(a) (The forward phase) First, use the Gaussian elimination method to reduce the
matrix into its row echelon form.
(b) (The backward phase) Then, beginning with the last nonzero row and working
upward, add suitable multiples of each row to the rows above to introduce zeros
above the leading 1s.
9. Every matrix has a unique reduced row echelon form.
10. Row echelon forms of a matrix A are not unique.
11. Gauss-Jordon elimination can be used to solve linear systems.
12. Example.
x+y =1
y+z =3
x+z =4

11
13. A pivot position of a matrix A is a position where the row echelon form of A has a
leading 1.
14. A column that contains a leading 1 is called a pivot column of A.
15. Row echelon forms of a matrix A are not unique, but they must have the same pivot
positions.
16. Different sequences of elementary row operations can result in different row echelon
forms.
17. Leading variables, Free variables
In a linear system, if a variable corresponds to a pivot column of the augmented matrix,
then the variable is called a leading variable.
Variables that are not leading variables are called free variables.
18. If a linear system is consistent and has at least one free variable, then the leading
variables can be solved in terms of the free variables.
In this case, we say the expressions of the leading variables in terms of the free variables
give the general solution of the linear system.
19. Example. The system
x + y 2z = 1
x 2y + z = 1
has infinitely many solutions. It is consistent.
Each of the following is a solution,
, (1, 0, 0), (2, 1, 1), (3, 2, 2), .
The parametric representation of the solution set is
x = 1 + t, y = t, z = t.
20. Example.
x+yz =2
x+y+z =6
21. Example.
x+yz =2
x+y+z =6
x+y =3

12
22. If the last nonzero row of a row-echelon form of the augmented matrix of a linear system
is
(0 0 0 1)
then the linear system is inconsistent.
For example, the linear system
x+y =3
2x + 2y = 9.
is in consistent.
23. If a system of m linear equations in n variables is of the form
a11 x1 + a12 x2 + + a1n xn = 0
a21 x1 + a22 x2 + + a2n xn = 0

am1 x1 + am2 x2 + + amn xn = 0

then it is called a homogeneous system.
24. Theorem. A homogeneous system always has the trivial solution
x1 = 0, x2 = 0, , xn = 0.
Therefore, a homogeneous system is always consistent.
25. Theorem 1.2.1. [Free Variable Theorem for Homogeneous Systems]
If a homogeneous linear system has n unknowns, and if the reduced row echelon form
of its augmented matrix has r nonzero rows, then the system has n r free variables.
Proof.
Let A be the augmented matrix of the linear system.
Let B be the reduced row echelon form of A.
Every nonzero row of B has a leading 1.
B has r leading 1s.
A has r pivot positions.
A has r pivot columns.
The linear system has r leading variables.
The total number of variables is n.
The system has n r free variables.

13
26. Theorem 1.2.2. A homogeneous linear system with more unknowns than equations has
infinitely many solutions.
Proof.
Suppose the system has m equations and n variables.
Then n > m.
Let A be the augmented matrix of the linear system,
then A has m rows.
Let B be the reduced row echlon form of A.
then B has m rows.
Let r be the number of nonzero rows in B.
Then r m.
Obviously, we have r < n.
The system has n r free variables.
n r > 0.
The system has at least one free variable.
The system has infinitely many solutions.
27. Example.
x + y 2z = 0
x y z = 0.
Practice Problems for Section 2:
Page 22: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31.

14

1.3

Matrices and Matrix Operations

1. Definition.
A matrix is a rectangular array of numbers. The numbers in the array are called the
entries in the matrix.
We use capital letters to denote matrices.
2. The size of a matrix is described in terms of the number of rows (horizontal lines) and
columns (vertical lines) it contains.
If a matrix has m rows and n columns, then we say it is an m n matrix.
An m n matrix takes the form
a11
a

A = 21

am1

a1n
a2n

.

amn

a12
a22

am2

A compact notation is
A = [aij ]mn

or

A = [aij ].

* Example.
The matrix

0 1 7
5 9 3

is a 2 3 matrix.
3. The entry that occurs in row i and column j is called the ij-entry of the matrix.
The ij-entry of the matrix A is denoted by (A)ij or aij .
4. When discussing matrices, it is common to refer to numerical quantities as scalars.
Unless stated otherwise, scalars will be real numbers; complex scalars will be considered
later in the text.
5. A 1 n matrix is called a row vector.
An m 1 matrix is called a column vector.
* Example.
The vector
( 0 1 7 )
is a row vector.
The vector

is a column vector.

2
5

0
7

15
6. Row and column vectors are of special importance, and it is common practice to denote
them by boldface lowercase letters rather than capital letters.
a = ( a1

a2

an )

is a row vector.
b1
b2

..
b=
.
bm

is a column vector.
7. A matrix A with n rows and n columns is called a square matrix of order n, and the
entries a11 , a22 , , ann are said to be on the main diagonal of A.
* Example.
The matrix

0 1 7

2 4
1

5 2 7

is a square matrix.
8. Definition 2.
Two matrices are defined to be equal if they have the same size and their corresponding
entries are equal.
* Example.
The matrices

0 1

1
7
2 4

0 1 7
,
1 2 4


are not equal.

* Example.
The matrices

1 2
,
3 4


1 3
2 4

are not equal.

9. Definition 3.
If A and B are matrices of the same size, then the sum A + B is the matrix obtained
by adding the entries of B to the corresponding entries of A, and the difference A B
is the matrix obtained by subtracting the entries of B from the corresponding entries
of A.
Matrices of different sizes cannot be added or subtracted.

16
* Example.
Let

A=

0 1 7
,
1 2 4


B=

6 2 1
.
0 3 4


Find A + B.
10. Matrix notation.
[A + B]ij = [A]ij + [B]ij ,
[A B]ij = [A]ij [B]ij .
11. Definition 4.
If A is any matrix and c is any scalar, then the product cA is the matrix obtained by
multiplying each entry of the matrix A by c.
The matrix cA is said to be a scalar multiple of A.
12. Matrix notation.
[cA]ij = c[A]ij .
* Example.
Let

A=

0 1 7
.
1 2 4


Find 3A.
13. If
a = ( a1
is a row vector and

a2

an )

b1
b2

b = ..
.
bn

is a column vector of the same length, then

ab = a1 b1 + a2 b2 + + an bn .
* Example.
Let

a = ( 0 1 7 ) ,
Find ab.

b= 5
.
2

17
14. Definition 5.
If A is an m r matrix and B is an r n matrix, then the product AB is the m n
matrix whose entries are determined as follows:
To find the entry in row i and column j of AB, single out row i from the matrix A
and column j from the matrix B. Multiply the corresponding entries from the row and
column together, and then add up the resulting products.
15. Definition 5.
If A is an m r matrix,
and B is an r n matrix,
then AB is the m n matrix,
and the ij-entry of AB is the product of the i-th row of A and the j-th column of B.
* Example.
Let

A=

0 1 7
,
1 2 4


2 3

B = 4 7 .
0 1

Find AB.
16. A matrix can be subdivided or partitioned into smaller matrices by inserting horizontal
and vertical rules between selected rows and columns.
17. Example. Consider the matrix


A=

0 1 0 3 1
.
5 9 3 7 9


If we let

A1 =
and

A2 =

0 1
5 9

0 3 1
,
3 7 9


then
A = ( A1

A2 ) .

18. Every matrix is a row of columns.

Every matrix is a column of rows.
The matrix

A=

0 1 7
1 2 4

is a row of three two-dimensional column vectors.

Its is also a column of two three-dimensional row vectors.

18
19. Consider

If we let

a11
a

A = 21

am1

a12
a

a2 = 22 ,

am2

a11
a

a1 = 21 ,

am1

a1n
a2n

.

amn

a12
a22

am2

a1n
a

an = 2n ,

amn

then
A = ( a1

an ) .

a2

20. Definition 6. If A1 , A2 , , Ar are matrices of the same size, and if c1 , c2 , , cr are

scalars, then an expression of the form
c1 A 1 + c2 A 2 + + cr A r
is called a linear combination of A1 , A2 , , Ar with coefficients c1 , c2 , , cr .
* Example.
Let

A=

0 1 7
,
1 2 4


B=

2 3 0
.
2 7 0


Find 2A + 3B.
21. Theorem 1.3.1.
If A is an m n matrix, and if x is an n 1 column vector, then the product Ax can be
expressed as a linear combination of the column vectors of A in which the coefficients
are the entries of x.
22. Consider
a11
a

A = 21

am1

If we let

a11
a

a1 = 21 ,

am1

a1n
a2n

,

amn

a12
a22

am2

a12
a

a2 = 22 ,

am2

x1
x2

x = .. .
.
xn

a2

a1n
a

an = 2n ,

amn

then
A = ( a1

an ) ,

19
and

x1
x2

.. ,
an )
.
xn

Ax = ( a1

a2

Ax = x1 a1 + x2 a2 + + xn an .
23. Matrix notation of a linear system.
Consider the system of m linear equations in n variables
a11 x1 + a12 x2 + + a1n xn = b1
a21 x1 + a22 x2 + + a2n xn = b2

am1 x1 + am2 x2 + + amn xn = bm .

If we let

a11
a
21
A=

am1

a12
a22

am2

x1
x2

x = .. ,
.
xn

a1n
a2n

,

amn

b1
b2

b = .. ,
.
bm

then
Ax = b.
24. Example.
Consider the system
x+yz =2
x+y+z =6
x + y = 3.
25. Matrix notation of a linear system.
Consider the system of m linear equations in n variables
a11 x1 + a12 x2 + + a1n xn = b1
a21 x1 + a22 x2 + + a2n xn = b2

am1 x1 + am2 x2 + + amn xn = bm .

20
If we let

a1n
a12
a
a

a2 = 22 , , an = 2n ,


amn
am2

b1
x1
b2
x2

x = .. , b = .. ,
.
.
bm
xn

a11
a

a1 = 21 ,

am1

then
x1 a1 + x2 a2 + + xn an = b.
26. The linear system Ax = b is consistent b is a linear combination of the columns
of A.
27. Example. Consider the system
xy =1
x + y = 3.
28. Example. Consider the system
x+y =2
2x + 2y = 3.
29. Definition 7.
If A is an m n matrix, then the transpose of A, denoted by AT , is defined to be the
n m matrix that results by interchanging the rows and columns of A;
that is, the first column of AT is the first row of A, the second column of AT is the
second row of A, and so forth.
30. Matrix notation
[A]ij = [AT ]ji .
31. Example. Find the transpose of the matrix


2 5 1
0 3 7

32. Definition 8. If A is a square matrix, then the trace of A, denoted by Tr (A), is defined
to be the sum of the entries on the main diagonal of A.
The trace of A is undefined if A is not a square matrix.
33. Example. Find the trace of the matrix


2 5
0 3

Practice Problems for Section 3:

Page 35: 1, 3, 5, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29.

21

1.4. Inverses; Algebraic Properties of Matrices

1. Theorem 1.4.1. Properties of Matrix Arithmetic
Assuming that the sizes of the matrices are such that the indicated operations can be
performed, the following rules of matrix arithmetic are valid.
(a) A + B = B + A

(Commutative law for addition)

(b) A + (B + C) = (A + B) + C
(c) A(BC) = (AB)C

(Associative law for addition)

(Associative law for multiplication)

(d) A(B + C) = AB + AC

(Left distributive law)

(e) (B + C)A = BA + CA

(Right distributive law)

(f) A(B C) = AB AC
(g) (B C)A = BA CA
(h) a(B + C) = aB + aC
(i) a(B C) = aB aC
(j) (a + b)C = aC + bC
(k) (a b)C = aC bC
(l) a(bC) = (ab)C
(m) a(BC) = (aB)C = B(aC)
2. In general AB 6= BA.
(a) It may happen that AB is defined and BA is not defined.
(b) It may happen that both AB and BA are defined, but they have different sizes.
(c) It may happen that both AB and BA are defined and they have the same size,
but they are not equal.
3. Some explanation.
(a) If A is a 2 3 matrix,
B is a 3 4 matrix,
then AB is a 2 4 matrix, and BA is not defined.
(b) If A is a 2 3 matrix,
B is a 3 2 matrix,
then AB is a 2 2 matrix, and BA is a 3 3 matrix.




0 1
1 0
(c) Let A =
and B =
, then AB 6= BA.
0 0
0 0
4. Order Matters in Matrix Multiplication !

22
5. A matrix whose entries are all zero is called a zero matrix.
We will denote a zero matrix by 0 unless it is important to specify its size, in which
case we will denote the m n zero matrix by 0mn .
6. Example.


023 =

0 0 0
.
0 0 0


7. Theorem 1.4.2. Properties of Zero Matrices

If c is a scalar, and if the sizes of the matrices are such that the operations can be
performed, then:
(a) A + 0 = 0 + A = A
(b) A 0 = A
(c) A A = A + (A) = 0
(d) 0A = 0
(e) If cA = 0, then c = 0 or A = 0.
8. A Zero Product with Nonzero Factors




0 1
1 0
Let A =
and B =
, then AB = 0.
0 0
0 0
9. Failure of the Cancellation Law.
Let

A=

0 1
,
0 0


B=

1 0
,
0 0


C=

0 0
,
0 0


then
AB = AC = CB = 0.
(a) We have AB = CB, and B 6= 0.
However, A 6= C.
The right cancellation law does not hold.
(b) We have AB = AC, and A 6= 0.
However, B 6= C.
The left cancellation law does not hold.
10. A square matrix with 1s on the main diagonal and zeros elsewhere is called an identity matrix.
An identity matrix is denoted by the letter I.
If it is important to emphasize the size, we will write In for the n n identity matrix.
* For example,

1 0 0

I3 = 0 1 0 ,
0 0 1

23
11. Theorem 1.4.3.
If R is the reduced row echelon form of an n n matrix A, then either R has a row of
zeros or R is the identity matrix In .
Proof. Consider the special case 3 3.
Let A be a 3 3 matrix, and R its reduced row-echelon form.
If every row of R is a nonzero row,
then every row has a leading 1.
12. If A is an m n matrix, then
Im A = AIn = A.
* Example. Let


A=

0 1 7
,
1 2 4


Find I2 A and AI3 .

13. Definition 1.
If A is a square matrix, and if a matrix B of the same size can be found such that
AB = BA = I, then A is said to be invertible and B is called an inverse of A.
If no such matrix B can be found, then A is said to be singular.
* If we let

A=

2 1
,
5 3


B=

3 1
,
5 2


then AB = BA = I2 .
14. An invertible matrix is called a nonsingular matrix.
* A non-square matrix is singular.
There are singular square matrices and nonsingular square matrices.
15. A square matrix with a row or column of zeros is singular.
For example, the matrix


A=

0 0
5 3

is not invertible.
16. Theorem 1.4.4. [The inverse of a matrix, if exists, is unique]
If B and C are both inverses of the matrix A, then B = C.
Proof.
17. If A is invertible, then we denote the inverse of A by A1 .

24
18. Theorem 1.4.5. The matrix

a b
A=
c d
is invertible if and only if ad bc 6= 0, in which case the inverse is given by the formula


1
d b
1
A =
.
ad bc c a


Proof of if is easy.
Proof of only if will be given later.
19. If A is invertible, then the solution of Ax = b is
x = A1 b.
* Example. Solve the system
2x y = 8,
x + 2y = 9.
20. Theorem 1.4.6.
If A and B are invertible matrices with the same size, then AB is invertible and
(AB)1 = B 1 A1 .
Proof.
21. A product of any number of invertible matrices is invertible, and the inverse of the
product is the product of the inverses in the reverse order.
22. A restatement.
If A1 , A2 , , Am are invertible matrices with the same size, then A = A1 A2 Am is
invertible and
1 1
A1 = (A1 A2 Am )1 = A1
m A2 A1 .
23. If A is a square matrix, then we define the nonnegative integer powers of A to be
A0 = I
and
An = AA A [ n factors ].
And, if A is invertible, then we define the negative integer powers of A to be
An = (A1 )n = A1 A1 A1

[n factors ].

Because these definitions parallel those for real numbers, the usual laws of nonnegative
exponents hold; for example,
Ar As = Ar+s
and
(Ar )s = Ars .
Proof.

25
24. Example.
Let

A=

2 1
.
5 3


Find A3 and A2 .
25. Theorem 1.4.7. If A is invertible and n is a nonnegative integer, then:
(a) A1 is invertible and (A1 )1 = A.
(b) An is invertible and (An )1 = An = (A1 )n .
(c) kA is invertible for any nonzero scalar k, and (kA)1 = k 1 A1 .
Proof.
* For example, if A is invertible, then
A3 (A1 )3 = AAAA1 A1 A1 = AAA1 A1 = AA1 = I.
This means that (A3 )1 = (A1 )3 .
26. If A and B are matrices of the same size and AB = BA, then we say that A and B
commute.
27. Example. Let


A=

1 2
,
0 1


B=

1 3
,
0 1


then AB = BA.
28. The Square of a Matrix Sum
In real arithmetic, where we have a commutative law for multiplication, we can write
(a + b)2 = a2 + ab + ba + b2 = a2 + ab + ab + b2 = a2 + 2ab + b2 .
However, in matrix arithmetic, where we have no commutative law for multiplication,
the best we can do is to write
(A + B)2 = A2 + AB + BA + B 2 .
It is only in the special case where AB = BA that we can go a step further and write
(A + B)2 = A2 + 2AB + B 2 .
29. Matrix Polynomials
If A is a square matrix, say an n n matrix, and if
p(x) = a0 + a1 x + a2 x2 + + am xm

26
is a polynomial, then we define the n n matrix p(A) to be
p(A) = a0 In + a1 A + a2 A2 + + am Am

(1)

where In is the n n identity matrix;

that is, p(A) is obtained by substituting A for x and replacing the constant term a0 by
the matrix a0 I.
An expression of form (1) is called a matrix polynomial in A.
30. Example. Let


A=

2 1
,
5 3


and let p(x) = x2 2x + 3.

Find p(A).
31. Theorem 1.4.8. If the sizes of the matrices are such that the stated operations can be
performed, then:
(a) (AT )T = A
(b) (A + B)T = AT + B T
(c) (A B)T = AT B T
(d) (kA)T = kAT
(e) (AB)T = B T AT
32. Proof of the formula (AB)T = B T AT .
* Some preparation.
(a) If a is an n-dimensional row vector and b an n-dimensional column vector, then
ab = bT aT .
(b) If A is a matrix,
then the transpose of row i of A is the column i of AT ,
and the transpose of column j of A is the row j of AT .
Now the proof of the formula (AB)T = B T AT .

=
=
=
=
=

((AB)T )ij
(AB)ji
row j of A column i of B
transpose of column i of B
transpose of row j of A
row i of B T column j of AT
(B T AT )ij .

27
33. The transpose of a product of any number of matrices is the product of the transposes
in the reverse order; that is,
(A1 A2 An )T = ATn AT2 AT1 ,
provided that the product A1 A2 An is defined.
34. Theorem 1.4.9. If A is an invertible matrix, then AT is also invertible and
(AT )1 = (A1 )T .
Proof.
Practice problems:
Page 49: 5, 7, 9, 15, 17, 19, 21, 25, 27, 31, 33, 35, 37, 39, 41, 53, 55.

28

1.5
A1

Elementary Matrices and a Method for Finding

1. In this section we will develop an algorithm for finding the inverse of a matrix, and we
will discuss some of the basic properties of invertible matrices.
2. In Section 1.1 we defined three elementary row operations on a matrix A:
(a) Multiply a row by a nonzero constant c.
Notation: cRi Ri .
(b) Interchange two rows.
Notation: Ri Rj
(c) Add a constant c times one row to another.
Notation: Ri + cRj Ri
3. It is evident that if we let B be the matrix that results from A by performing one of the
operations in the above list, then the matrix A can be recovered from B by performing
the corresponding operation in the following list.
(a) Multiply the same row by 1/c.
(b) Interchange the same two rows.
(c) If B resulted by adding c times row r1 of A to row r2 , then add c times r2 to
row r1 .
* Example:


(a) Let A =

a b
, and let be the operation R1 R2 .
c d


a b
(b) Let A =
, and let be the operation 3R1 R1 .
c d


a b
(c) Let A =
, and let be the operation R1 + 2R2 R1 .
c d


4. Simply put, every elementary row operation has an inverse operation.

5. Inverse operations
(a) The inverse operation of Ri Rj is itself.
(b) Let c 6= 0. The inverse operation of cRi Ri is (1/c)Ri Ri .
(c) The inverse operation of Ri + cRj Ri is Ri cRj Ri .
6. It follows that if B is obtained from A by performing a sequence of elementary row
operations, then there is a second sequence of elementary row operations, which when
applied to B recovers A.

29
* For example, if the sequence of operations 1 , 2 , 3 transforms A to B, then .
7. Definition 1.
Matrices A and B are said to be row equivalent if either can be obtained from the other
by a sequence of elementary row operations.
* For example, the matrices


A=

a b
,
c d


B=

c d
a
b

are row equivalent.

* A matrix is row equivalent to every one of its row echelon forms.
Every matrix is row equivalent to its reduced row echelon form.
8. Definition 2. An n n matrix is called an elementary matrix if it can be obtained
from the identity matrix In by performing a single elementary row operation.
9. There are three types of elementary row operations, therefore there are three types of
elementary matrices.
10. Examples. The matrices


2 0
1 2
0 1
,
,
0 1
0 1
1 0
 

 

are elementary matrices.

The matrices

2 1
2 0
0 1
0 1
,
,
,
0 1
0 3
2 0
0 2
 

 

 

are not elementary matrices.

If A is an m n matrix, B is an n r matrix, and is an elementary row operation,
then
(AB) = (A)B.
11. Theorem 1.5.1. Row Operations by Matrix Multiplication
If
(a) the elementary matrix E results from performing a certain row operation on Im ,
and
(b) A is an m n matrix,
then the product EA is the matrix that results when this same row operation is performed on A.

30
* For example, if we let be the operation R1 R2 , and


E = (I2 ),

A=

a b
,
c d


then EA = .
Proof.
If E = (I), then
(A) = (IA) = (I)A = EA.
12. Let be an elementary row operation,
and let A be an m n matrix.
If E = (Im ),
then EA = (A),
or (Im )A = (A).
13. Let be an elementary row operation. Then we have

E = (Im )
EA = (A) for all matrices A that have m rows.

14. Every elementary row operation corresponds to an elementary matrix E, in such a

way that:
(a) if we perform on A, we get EA,
(b) if we perform on I, we get EI = E.
15. Similarly, every elementary column operation corresponds to an elementary matrix
E, in such a way that:
(a) if we perform on A, we get AE,
(b) if we perform on I, we get IE = E.
16. Multiplication from left = row operation.
17. Example.


1 2
3 4



R1
R2

= .

18. Multiplication from right = column operation.

19. Example.


( C1

C2 )

1 2
3 4

= .

31
20. Theorem 1.5.2. Every elementary matrix is invertible, and the inverse is also an
elementary matrix.
Proof.
Let E1 be an elementary matrix.
E1 corresponds to 1 .
1 has an inverse 2 .
2 corresponds to E2 .
2 (1 (I)) = I,
E2 E1 I = I.
E2 E1 = I.
In a similar way, E1 E2 = I.
* For example, let


E1 =

1 2
.
0 1


21. Theorem 1.5.3. Equivalent Statements (1st set)

If A is an n n matrix, then the following statements are equivalent, that is, all true
or all false.
(a) A is invertible.
(b) Ax = 0 has only the trivial solution.
(c) The reduced row echelon form of A is In .
(d) A is expressible as a product of elementary matrices.
Proof of (a) (b)
Proof of (b) (c)
Ax = 0 has only the trivial solution.
Every variable of the system Ax = 0 is a leading varaible.
Every column of A is a pivot column.
Proof of (c) (d)
Proof of (d) (a)
22. Example. Express


A=
as a product of elementary matrices.
Find the inverse of A.

1 2
3 4

32
23. Inversion Algorithm
To find the inverse of an invertible matrix A, find a sequence of elementary row operations that reduces A to the identity and then perform that same sequence of operations
on In to obtain A1 .
Proof. If
Em Em1 E2 E1 A = I,
then
Em Em1 E2 E1 I = A1 .
24. If A is an m r matrix, B is an r n matrix, and b1 , b2 , , bn are the n columns of
B, then
AB = A ( b1 b2 bn ) = ( Ab1 Ab2 Abn ) .
25. Similarly, If A is an m r matrix, B is an r n matrix, and a1 , a2 , , am are the m
row of A, then

a1 B
a1
a B
a
2
2
B =
AB =

am
am B
26. If A is an m r matrix, B is an r n matrix, and C is an r p matrix, then
A(B

C ) = ( AB

AC ) .

27. Similarly, if A is an m r matrix, B is an n r matrix, and C is an r p matrix, then



A
C=
B


AC
BC

28. If A is a square matrix and A is invertible, then

A1 ( A In ) = ( In

A1 ) .

29. Similarly, if A is a square matrix and A is invertible, then



A
A1 =
In


In
.
A1


30. Let A be a square matrix, if we perform some elementary row operations on A and get
a matrix B which contains a row of zeros, then A is singular.
Proof.
E1 E2 En A = B.
B is singular.
If A is invertible, then contradiction.
Practice Problems:
Page 58: 1, 3, 5, 7, 9, 11, 13 15 17, 19, 21, 23, 25, 29, 31, 33, 35, 37.

33

1.6

More on Linear Systems and Invertible Matrices

1. Theorem 1.6.1 A system of linear equations has zero, one, or infinitely many solutions.
There are no other possibilities.
2. If a linear system has two or more solutions, then it has infinitely many solutions.
Proof.
3. Theorem 1.6.2. If A is an invertible n n matrix, then for each n 1 matrix b, the
system of equations Ax = b has exactly one solution, namely, x = A1 b.
4. Statement. Let A and B be square matrices of the same size. If BA = I, then A is
invertible.
Proof. If Ax = 0,
x = BAx = B0 = 0.
The only solution of Ax = 0 is x = 0.
By the 1st set of equivalent statements, this means that A is invertible.
5. Theorem 1.6.3. Let A be a square matrix.
(a) If B is a square matrix satisfying BA = I, then B = A1 .
(b) If B is a square matrix satisfying AB = I, then B = A1 .
Proof of (a):
Since BA = I, A is invertible;
AA1 = A1 A = I.
AA1 = I.
BAA1 = BI.
IA1 = BI.
A1 = B.
6. Theorem 1.6.4. Equivalent Statements
If A is an n n matrix, then the following are equivalent.
(a) A is invertible.
(b) Ax = 0 has only the trivial solution.
(c) The reduced row echelon form of A is In .
(d) A is expressible as a product of elementary matrices.
(e) Ax = b is consistent for every n 1 matrix b.
(f) Ax = b has exactly one solution for every n 1 matrix b.

34
Proof.
(a) (f).
(f) (e).
(e) (a).
7. Corollary. Equivalent Statements
If A is an n n matrix, then the following are equivalent.
(a) A is not invertible.
(b) Ax = 0 has infinitely many solutions.
(c) The reduced row echelon form of A is not In .
(d) Ax = b is inconsistent for some n 1 matrix b.
8. Corollary.
Let A be an n n matrix and b an n 1 vector. If A is not invertible, then either
(a) Ax = b has infinitely many solutions, or
(b) Ax = b is inconsistent.
9. Corollary.
Let A be an n n matrix. If A is not invertible, then the last row of the row-echelon
form of A is a zero row.
* Restatement:
Let A be an n n matrix, and B a row-echelon form of A. If the last row of B is a
nonzero row, then A is invertible.
Proof. Consider the special case 3 3.
Every row of B has a leading 1.
10. Theorem 1.6.5. Let A and B be square matrices of the same size. If AB is invertible,
then A and B must also be invertible.
Proof is similar, omitted.
11. A Fundamental Problem
Let A be a fixed m n matrix. Find all m 1 matrices b such that the system of
equations Ax = b is consistent.
12. Example. Let


A=

1 2
.
3 6


Find all b such that the system of equations Ax = b is consistent.

35
13. Example. Let

1 1 0

A = 0 1 1 .
1 0 1
Find all b such that the system of equations Ax = b is consistent.
Practice Problems:
Page 65: 1, 3, 5, 7, 13, 15, 17, 19, 21.

36

1.7

Diagonal, Triangular, and Symmetric Matrices

1. Definition.
A diagonal matrix is a square matrix in which all the entries off the main diagonal are
zero.
* For example, the matrix

3 0 0

A = 0 5 0
0 0 7
is a diagonal matrix.
2. If A is a diagonal matrix, and its diagonal entries are a1 , a2 , , an , then we write
A = diag (a1 , a2 , , an )
3. Inverses of Diagonal Matrices
(a) If none of the numbers a1 , a2 , , an is zero, then
A = diag (a1 , a2 , , an )
is invertible, and
1
1
A1 = diag (a1
1 , a2 , , an ).

(b) If at least one of the numbers a1 , a2 , , an is zero, then

A = diag (a1 , a2 , , an )
is singular, for it has a zero row.
4. Powers of Diagonal Matrices
If
A = diag (a1 , a2 , , an ),
then
A2 = diag (a21 , a22 , , a2n ),
A3 = diag (a31 , a32 , , a3n ),

5. General Principles:
In words,
(a) to multiply a matrix A on the left by a diagonal matrix D, one can multiply
successive rows of A by the successive diagonal entries of D, and
(b) to multiply A on the right by D, one can multiply successive columns of A by the
successive diagonal entries of D.

37
6. For example, If A is an m 3 matrix, and B = diag (b1 , b2 , b3 ). Let the three columns
of A be a1 , a2 , a3 , then
AB = ( a1

a2

a3 ) diag (b1 , b2 , b3 ) = .

7. A square matrix in which all the entries above the main diagonal are zero is called lower
triangular.
For example, the matrix

7 0 0

A = 3 5 0
2 0 8
is lower triangular.
A square matrix in which all the entries below the main diagonal are zero is called
upper triangular.
For example, the matrix

7 3 2

B = 0 5 0
0 0 8
is upper triangular.
A matrix that is either upper triangular or lower triangular is called triangular.
8. (a) A square matrix A = [aij ] is upper triangular if and only if all entries to the left
of the main diagonal are zero;
that is, aij = 0 if i > j.
(b) A square matrix A = [aij ] is lower triangular if and only if all entries to the right
of the main diagonal are zero;
that is, aij = 0 if i < j.
(c) A square matrix A = [aij ] is upper triangular if and only if the i-th row starts
with at least i 1 zeros for every i.
(d) A square matrix A = [aij ] is lower triangular if and only if the j th column starts
with at least j 1 zeros for every j.
9. Theorem 1.7.1.
(a) The transpose of a lower triangular matrix is upper triangular, and the transpose
of an upper triangular matrix is lower triangular.
For example, if

7 0 0

A=
3 5 0 ,
2 0 8
then

7 3 2

T
A = 0 5 0
.
0 0 8

38
(b) The product of lower triangular matrices is lower triangular, and the product of
upper triangular matrices is upper triangular.
Proof.

0
a b c
a b0 c 0

0 d e 0 d0 e 0 = .
0 0 f0
0 0 f
(c) A triangular matrix is invertible if and only if its diagonal entries are all nonzero.
(c1) Let A be a triangular matrix. If none of the diagonal entries of A is zero, then A
is invertible.
(c2) If a triangular matrix is invertible, then its diagonal entries are all nonzero.
(d) The inverse of an invertible lower triangular matrix is lower triangular, and the
inverse of an invertible upper triangular matrix is upper triangular.
Proof of (c2) and (d). If

a b c

A=
0 d e
0 0 f
is invertible, then

a b c
a11

0 d e a21
0 0 f
a31
Proof of (c1).
Let

a12
a22
a32

a13
1 0 0

a23 = 0 1 0 .
a33
0 0 1

a b c

A = 0 d e .
0 0 f

Assume that a 6= 0, d 6= 0, f 6= 0.
We are going to show that A is invertible.
By the first set of equivalent statements, we need only to show that the reduced
row echelon form of A is I3 .
10. Definition 1. A square matrix A is said to be symmetric if A = AT .
11. A square matrix A = [aij ] is symmetric if and only if
(A)ij = (A)ji
for all values of i and j.
* The matrix

is symmetric.

3 0 5

0 7 8
5 8 2

39
12. Theorem 1.7.2. If A and B are symmetric matrices with the same size, and if k is any
scalar, then
(a) AT is symmetric.
(b) A + B and A B are symmetric.
(c) kA is symmetric.
13. It is not true, in general, that the product of symmetric matrices is symmetric.
14. Let

A=

1 1
,
1 1


B=

1 1
,
1 3


then AB is not symmetric.

15. Theorem 1.7.3. The product of two symmetric matrices is symmetric if and only if the
matrices commute.
Proof. Easy.
16. Theorem 1.7.4.
If A is an invertible symmetric matrix, then A1 is symmetric.
Proof. Easy.
17. Matrix products of the form AAT and AT A arise in a variety of applications.
If A is an m n matrix, then AT is an n m matrix, so the products AAT and AT A
are both square matrices the matrix AAT has size m m, and the matrix AT A has
size n n.
Such products are always symmetric since
(AAT )T = (AT )T AT = AAT
and
(AT A)T = AT (AT )T = AT A.
18. Theorem 1.7.5. If A is an invertible matrix, then AAT and AT A are also invertible.
Practice Problems:
Page 71: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31.