Outlines

Cramer’s Rule and Gauss Elimination

Mike Renfro

September 28, 2004

Mike Renfro Cramer’s Rule and Gauss Elimination

 Part I: Review of Previous Lecture
Outlines
Part II: Cramer’s Rule and Gauss Elimination

Review of Previous Lecture

Mike Renfro Cramer’s Rule and Gauss Elimination

 Part I: Review of Previous Lecture
Outlines
Part II: Cramer’s Rule and Gauss Elimination

Cramer’s Rule and Gauss Elimination

Cramer’s Rule
Introduction
Example
Advantages and Disadvantages

Gauss Elimination
Introduction and Rules
Example
Matrix Version and Example
Advantages and Disadvantages

Homework

Mike Renfro Cramer’s Rule and Gauss Elimination

 Part I

Review of Previous Lecture

Mike Renfro Cramer’s Rule and Gauss Elimination

Review of Previous Lecture

Graphical interpretation
Solvable and unsolvable problems
Linear dependence and independence
Ill-conditioning

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule
Gauss Elimination
Homework

Part II

Cramer’s Rule and Gauss Elimination

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule Introduction
Gauss Elimination Example
Homework Advantages and Disadvantages

Cramer’s Rule (1750)

A linear system of equations can be solved by using Cramer’s rule,
which for a system of 2 equations
    
a11 a12 x1 b1
=
a21 a22 x2 b2
yields
|[A1 ]| |[A2 ]|
x1 = , x2 =
|[A]| |[A]|
where
 
b1 a12
A1 =
b2 a22
 
a11 b1
A2 =
a21 b2

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule Introduction
Gauss Elimination Example
Homework Advantages and Disadvantages

Cramer’s Rule Details

For a system of n equations, Cramer’s rule requires that you

calculate n + 1 determinants of n × n matrices.
In the general case for a system of equations [A]{x} = {b},
the matrix [Ai ] is obtained by replacing the ith column of the
original [A] matrix with the contents of the {b} vector.
Each unknown variable xi is found by dividing the determinant
|[Ai ]| by the determinant of the original coefficient matrix
|[A]|.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule Introduction
Gauss Elimination Example
Homework Advantages and Disadvantages

Cramer’s Rule Example

Solve the following system of equations using Cramer’s rule:

x1 − x2 + x3 = 3
2x1 + x2 − x3 = 0
3x1 + 2x2 + 2x3 = 15

Convert the system of equations into matrix form:

    
1 −1 1  x1   3 
 2 1 −1  x2 = 0
3 2 2 x3 15
   

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule Introduction
Gauss Elimination Example
Homework Advantages and Disadvantages

Cramer’s Rule Example (continued)

     
1 −1 1  x1   3 
[A] =  2 1 −1  , {x} = x2 , {b} = 0
3 2 2 x3 15
   

Define matrices [A1 ], [A2 ], and [A3 ] as

   
3 −1 1 1 3 1
[A1 ] =  0 1 −1  , [A2 ] =  2 0 −1  ,
15 2 2 3 15 2
 
1 −1 3
[A3 ] =  2 1 0 
3 2 15

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule Introduction
Gauss Elimination Example
Homework Advantages and Disadvantages

Cramer’s Rule Example (continued)

Calculate determinants of [A], [A1 ], [A2 ], and [A3 ]:

|[A]| = 12
|[A1 ]| = 12
|[A2 ]| = 24
|[A3 ]| = 48

Unknowns x1 , x2 , and x3 are then calculated as

|[A1 ]| 12 |[A2 ]| 24 |[A3 ]| 48
x1 = = = 1, x2 = = = 2, x3 = = =4
|[A]| 12 |[A]| 12 |[A]| 12

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule Introduction
Gauss Elimination Example
Homework Advantages and Disadvantages

Cramer’s Rule Example (continued)

Be sure to double-check your answers by substituting them into

the original equations:

x1 − x2 + x3 = 1 − 2 + 4 = 3
2x1 + x2 − x3 = 2 + 2 − 4 = 0
3x1 + 2x2 + 2x3 = 3 + 4 + 8 = 15

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule Introduction
Gauss Elimination Example
Homework Advantages and Disadvantages

Cramer’s Rule Advantages/Disadvantages

Advantages
Easy to remember steps
Disadvantages
Computationally intensive compared to other methods: the
most efficient ways of calculating the determinant of an n × n
matrix require (n − 1)(n!) operations. So Cramer’s rule would
require (n − 1)((n + 1)!) total operations. For 8 equations,
that works out to 7(9!) = 2540160 operations, or around 700
hours if you can perform one operation per second.
Roundoff error may become significant on large problems with
non-integer coefficients.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Gauss Elimination

Recall the scaffolding problem from the beginning of Chapter 3. Its

matrix form was
   
1 −1 −1 0 −1 

1  T A 
 
 P1 

 0 −9 1 4 0 7 
 T B





 −5P 1



    
 0 0 1 1 −1 0 T P
   
C 2

  =
 0 0 0 −3 2 0  T   −P2 

 0
 D 
  
0 0 0 1 1  T P
  
3
 E

 
 

   
0 0 0 0 0 −4 −P3

TF
  

Notice that its coefficient matrix contains nothing but zeroes

below the diagonal. This is an example of an upper triangular
matrix, and these systems of equations are very easy to solve.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Gauss Elimination Introduction (continued)

The original system of equations on the scaffolding problem was

TA +TB −TC −TD −TF = P1

−9TB +TC +4TD +7TF = −5P1
TC +TD −TE = P2
.
−3TD +2TE = −P2
TE +TF = P3
−4TF = −P3

Notice that we can solve for TF using only the sixth equation in
the system. That is, TF = P43 . After solving for TF , we can solve
for TE using only the fifth equation. The pattern continues,
back-substituting through the system of equations until finally we
solve for TA using the first equation.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Gauss Elimination Introduction (continued)

The goal of Gauss elimination is to convert any given system of

equations into an equivalent upper triangular form. Once
converted, we can back-substitute through the equations, solving
for the unknowns algebraically.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Gauss Elimination Rules

The operations used in converting a system of equations to upper

triangular form are known as elementary operations and are:
Any equation may be multiplied by a nonzero scalar.
Any equation may be added to (or subtracted from) another
equation.
The positions of any two equations in the system may be
swapped.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Gauss Elimination Example

2x1 − x2 + x3 = 4 (1)
4x1 + 3x2 − x3 = 6 (2)
3x1 + 2x2 + 2x3 = 15 (3)
To eliminate the 4x1 term in Equation 2, multiply Equation 1 by 2
and subtract it from Equation 2. To eliminate the 3x1 term in
Equation 3, multiply Equation 1 by 23 and subtract it from
Equation 3. This gives a system of equations
2x1 − x2 + x3 = 4 (4)
5x2 − 3x3 = −2 (5)
7 1
x 2 + x3 = 9 (6)
2 2

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Gauss Elimination Example (continued)

To eliminate the 27 x2 term from Equation 6, multiply Equation 5

7
by 10 and subtract it from Equation 6. This gives a system of
equations

2x1 − x2 + x3 = 4 (7)
5x2 − 3x3 = −2 (8)
13 52
x3 = (9)
5 5

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Gauss Elimination Example (continued)

Equation 9 can easily be solved for x3 .

 
52 5
x3 = =4
5 13

Equation 8 can easily be solved for x2 , once x3 is known.

1 1
x2 = (−2 + 3x3 ) = (−2 + 3(4)) = 2
5 5
Equation 7 can easily be solved for x1 , once both x2 and x3 are
known.
1 1
x1 = (4 + x2 − x3 ) = (4 + 2 − 4) = 1
2 2

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Matrix Version of Gauss Elimination

The Gauss elimination method can be applied to a system of

equations in matrix form. Instead of eliminating terms from
equations, we’ll be replacing certain elements of the coefficient
matrix with zeroes.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Matrix Version (Step 0)

Start by defining the augmented matrix [C (0) ] for the problem:

(0) (0) (0) (0)
 
a11 a12 ··· a1n a1,n+1
 (0) (0) (0) (0)
 a21 a22 · · · a2n a2,n+1

(0)

[C ]=
 .. .. .. .. ..

 . . . . .


(0) (0) (0) (0)
an1 an2 · · · ann an,n+1

where the first n columns are the elements of the original [A]
matrix, and the last column is the elements of the original {b}
matrix.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Matrix Version (Step 1)

Zero out the first column of the [C ] matrix, rows 2 · · · n. To turn

a21 to a zero, multiply row 1 by aa11
21
, then subtract the numbers on
row 1 from row 2. To turn a31 to a zero, multiply row 1 by aa31 11
,
then subtract the numbers on row 1 from row 3. Repeat for rows
4 · · · n.
(0) (0) (0) (0)
 
a11 a12 ··· a1n a1,n+1
(1) (1) (1)
 0 a22 ··· a2n a2,n+1
 
(1)

[C ]=
 .. .. .. .. ..

 . . . . .


(1) (1) (1)
0 an2 ··· ann an,n+1

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Matrix Version (Step 2)

Zero out the second column of the [C ] matrix, rows 3 · · · n. To

turn a32 to a zero, multiply row 2 by aa22
32
, then subtract the
numbers on row 2 from row 3. To turn a42 to a zero, multiply row
2 by aa22
42
, then subtract the numbers on row 2 from row 4. Repeat
for rows 5 · · · n.
 
(0) (0) (0) (0) (0)
a11 a12 a13 · · · a1n a1,n+1
 (1) (1) (1) (1) 
 0 a22
 a13 · · · a2n a2,n+1  
(2) (2) (2)
[C (1) ] =  0 0 a33 · · · a3n a3,n+1 
 
 . .. .. .. .. .. 
 .
 . . . . . .


(2) (2) (2)
0 0 an3 · · · ann an,n+1

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Matrix Version (Step n-1)

Zero out the (n − 1)th column of the [C ] matrix, row n. To turn

an,n−1
an,n−1 to a zero, multiply row n − 1 by an−1,n−1 , then subtract the
numbers on row n − 1 from row n.
 
(0) (0) (0) (0) (0)
a11 a12 a13 · · · a1n a1,n+1
 (1) (1) (1) (1) 
 0 a22
 a13 · · · a2n a2,n+1 

(2) (2) (2)
[C (1) ] =  0 0 a33 · · · a3n a3,n+1 
 
 . .. .. .. .. .. 
 .
 . . . . . .


(n−1) (n−1)
0 0 0 · · · ann an,n+1

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Example

Solve the following system of equations with Gauss elimination:

    
2 −1 1  x1   4 
 4 3 −1  x2 = 6
3 2 2 x3 15
   

First, set up the augmented matrix [C (0) ]:

 
2 −1 1 4
[C (0) ] =  4 3 −1 6 
3 2 2 15

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Example (continued)

Step 1a: eliminate the 4 on row 2, column 1. Multiply all the

elements of row 1 by aa11
21
= 42 = 2, then subtract them from the
elements of row 2.
 
2 −1 1 4
[C ] =  4 − (2)(2) 3 − (2)(−1) −1 − (2)(1) 6 − (2)(4) 
3 2 2 15
 
2 −1 1 4
= 0
 5 −3 −2 
3 2 2 15

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Example (continued)

Step 1b: eliminate the 3 on row 3, column 1. Multiply all the

elements of row 1 by aa11
31
= 32 = 1.5, then subtract them from the
elements of row 3.
 
2 −1 1 4
(1)
[C ] =  0 5 −3 −2 
3 − (1.5)(2) 2 − (1.5)(−1) 2 − (1.5)(1) 15 − (1.5)(4)
 
2 −1 1 4
= 0
 5 −3 −2 
0 3.5 0.5 9

This completes the first elimination step.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Example (continued)

Step 2: eliminate the 3.5 on row 3, column 2. Multiply all the

elements of row 2 by aa22
32
= 3.5
5 = 0.7, then subtract them from the
elements of row 3.
 
2 −1 1 4
[C (2) ] =  0 5 −3 −2 
0 3.5 − (0.7)(5) 0.5 − (0.7)(−3) 9 − (0.7)(−2)
 
2 −1 1 4
= 0 5 −3 −2 
0 0 2.6 10.4

This completes the second elimination step.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Example (continued)

We’ve now converted the original system of equations

    
2 −1 1  x1   4 
 4 3 −1  x2 = 6
3 2 2 x3 15
   

into an equivalent upper-triangular system of equations

    
2 −1 1  x1   4 
 0 5 −3  x2 = −2
0 0 2.6 x3 10.4
   

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Example (continued)

The new system of equations can be converted back to algebraic

form as:

2x1 − x2 + x3 = 4 (10)
5x2 − 3x3 = −2 (11)
2.6x3 = 10.4 (12)

Solve Equation 12 for x3 : x3 = 10.4

2.6 = 4. Then solve Equation 11
1
for x2 : x2 = 5 (−2 + 3x3 ) = 2. Then solve Equation 10 for x1 :
x1 = 21 (4 + x2 − x3 ) = 1.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Example (continued)

Double-check the solution by substituting the values of x1 , x2 , and

x3 into the original equations:

2x1 − x2 + x3 = 2(1) − 2 + 4 =4
4x1 − 3x2 − x3 = 4(1) + 3(2) − 4 =6
3x1 + 2x2 + 2x3 = 3(1) + 2(2) + 2(4) = 15

Mike Renfro Cramer’s Rule and Gauss Elimination

 Introduction and Rules
Cramer’s Rule
Example
Gauss Elimination
Matrix Version and Example
Homework
Advantages and Disadvantages

Gauss Elimination Advantages/Disadvantages

Advantages
Much less computation required for larger problems. Gauss
3
elimination requires n3 multiplications to solve a system of n
equations. For 8 equations, this works out to around 170
operations, versus the roughly 2.5 million operations for
Cramer’s rule.
Disadvantages
Not quite as easy to remember the procedure for hand
solutions.
Roundoff error may become significant, but can be partially
mitigated by using more advanced techniques such as pivoting
or scaling.

Mike Renfro Cramer’s Rule and Gauss Elimination

 Cramer’s Rule
Gauss Elimination
Homework

Homework

Solve Problem 3.4 using: Cramer’s rule, Gauss elimination,

and MATLAB’s \ operator. Double-check your answers by
substituting them back into the original equations.

Mike Renfro Cramer’s Rule and Gauss Elimination