Solution of Non-homogeneous

Linear Equations
Class 7
Engineering 112
Engineering Methods II

Systems of Linear Equations

Homogeneous
All of the elements of
array on the right-handside of homogeneous
equations is zero.

1
0.5 x 0
3

1

4
0.75 y = 0

0.5 0.75
5 z 0

Non-Homogeneous
Some of the elements of the
array on the right-hand-side
of homogeneous equations
are not zeros.

1
0.5 x 4
3

1

4
0.75 y = 2

0.5 0.75
5 z 3

Non-homogeneous Linear
Equations
Non-homogeneous systems of linear equations are
encountered in many disciplines within engineering
and science.
A typical problem involves finding the numbers in the
array x that satisfies all equations simultaneously.

a11 a 12
a
a
21
22

M
M

a n1 a n2

L a 1n x1 y1

L a 2n x 2 y 2
=
O M M M

L a nn x n y n

Solution Methods
There are many methods for solving linear equations.

[a]{x} = {b}
Gaussian Elimination

Cholesky factors

Cramers Rule

Gauss-Seidel Iteration

Each of these methods will give the same result.

Some are more efficient and robust than others.

Cramers Rule
Refer to Class 3 for detailed discussion
3x + 4y = 9
2x 5y = 8

D1 =

8 -5

= (9)( 5) (8)(4)

= 77

3 4 x 9
2 - 5 y = 8

D1 =

3 9
2 8

= (3)(8) (2 )(9 )

=6

D=

2 -5

= 23

= (3)( 5) (2)(4)

77
= 3.35
23
6
= 0.26
y=
23

x=

Gaussian Elimination
Example Problem
x + 2y + 3z = 14
2x + 3y + 4z = 20
3x + 4y + z = 14

1 2 3 x 14
2 3 4 y = 20

3 4 1 z 14

Step 1: Replace the second equation with an equation obtained

by multiplying the first equation by 2 and adding it to the
second equation.

- 2x 4y 6z = -28
2x + 3y + 4z = 20

y 2z 8

x + 2y + 3z = 14
- y 2z = -8
3x + 4y + z = 14

Gaussian Elimination
(Example Continued)

x + 2y + 3z = 14
- y 2z = -8
3x + 4y + z = 14

1 2 3 x 14
0 - 1 - 2 y = 8

3 4 1 z 14

Step 2: Replace the third equation with an equation obtained by

multiplying the first equation by 3 and adding it to the third
equation.

- 3x 6y 9z = -42
3x + 4y + z = 14
2y 8z = 28

x + 2y + 3z = 14
- y 2z = -8
2y 8z = -28

Gaussian Elimination
(Example Continued)

x + 2y + 3z = 14
- y 2z = -8
2y 8z = -28

1 2 3 x 14
0 - 1 - 2 y = 8

0 - 2 - 8 z 28

Step 3: Replace the third equation with an equation obtained by

multiplying the second equation by 2 and adding it to the third
equation.

2y + 4z = 16
2y 8z = -28
4z = 12

x + 2y + 3z = 14
- y 2z = -8

- 4z = -12

Gaussian Elimination
(Example Continued)

x + 2y + 3z = 14
- y 2z = -8
- 4z = -12

1 2 3 x 14
0 - 1 - 2 y = 8

0 0 - 4 z 12

Step 4: Solve for z from the third equation.

- 12
z=
=3
4

Gaussian Elimination
(Example Continued)

x + 2y + 3z = 14
- y 2z = -8
- 4z = -12

1 2 3 x 14
0 - 1 - 2 y = 8

0 0 - 4 3 12

Step 5: Solve for y from the second equation.

y 6 = 8
y=2

Gaussian Elmination
(Example Continued)

x + 2y + 3z = 14
- y 2z = -8
- 4z = -12

1 2 3 x 14
0 - 1 - 2 2 = 8

0 0 - 4 3 12

Step 6: Solve for x from the first equation.

x + 4 + 9 = 14
x =1

Gaussian Elimination
(Example Continued)
Original Equations

1 2 3 x 14
2 3 4 y = 20

3 4 1 z 14
Verify Answer

1 2 3 1
2 3 4 2 =

3 4 1 3

Gaussian Elimination
(Example Continued)

Gaussian Elimination involves two steps.

First: Algebraic operations are performed to reduce
the coefficients below the main diagonal to zero (i.e.
eliminating them). This is called the Elimination
Process.
Second: The equations are successively solved starting
with the last equation which has only a diagonal term.
This process is call Back Substitution.

Gaussian Elimination and

Determinants
Original Equations

1 2 3 x 14
2 3 4 y = 20

3 4 1 z 14

Equations after
Elimination Process

1 2 3 x 14
0 - 1 - 2 2 = 8

0 0 - 4 3 12

1 2 3
1[(3)(1) (4 )(4 )]
2 3 4 = 2[(2 )(1) (3)(4 )]

1 2 3
0 - 1 - 2 = (1)( 1)( 4 )

3 4 1

=4

+ 3[(2 )(4 ) (3)(3)]

=4

-4

Gaussian Elimination and

Determinants
(Continued)
1. In class 3 we found (Cramers Rule) that a nonhomogeneous set of equations has a solution if the
determinant of the coefficient matrix is non-zero.
2. The determinant of the coefficient matrix is not
affected by the elimination process. Reference Class
3, Theorem 7.
3. The determinant of a matrix with all zeros above or
below the main diagonal is equal to the product of the
diagonal elements.

Solution Methods
There are many methods for solving linear equations.

[a]{x} = {b}
Gaussian Elimination
Cramers Rule

Cholesky factors
Gauss-Seidel

Gauss-Seidel
(Example)
1 2 3 x 14
2 3 4 y = 20

3 4 1 z 14
1 0 0 x 0 2 3 x 14
0 3 0 y + 2 0 4 y = 20

0 0 1 z 3 4 0 z 14
1 0 0 x i +1 14 0 2 3 x i
0 3 0 y = 20 2 0 4 y

i +1
i
0 0 1 z i +1 14 3 4 0 z i

Gauss-Seidel
(Example Continued)
1 x i
5 0 0 x i +1 14 0 2
0 3 0 y = 20 2 0 0.5 y
i

i +1
0 0 6 z i +1 14 1 0.5 0 z i

Step 1:
1 Assume a solution for the 1st iteration. Evaluate
the right-hand-side of the equation. Solve for the 2nd
solution.
1 1
5 0 0 x i +1 14 0 2
0 3 0 y = 20 2 0 0.5 1

i +1
0 0 6 z i +1 14 1 0.5 0 1

x i +1 2.2

y i +1 = 5.8333
z 2.0833
i +1

Gauss-Seidel
(Example Continued)
Step 2:
2 Use the previous solution to obtain an improve
another estimate for the solution.
1 2.2
5 0 0 x i +1 14 0 2
0 3 0 y = 20 2 0 0.5 5.833

i +1
0 0 6 z i +1 14 1 0.5 0 2.0833

x i +1 0.050

=
y
4
.
8528
i +1

z 1.4806
i +1

Gauss-Seidel
(Example Continued)
Step 3:
3 Use the previous solution to obtain an improve
another estimate for the solution.
1 0.050
5 0 0 x i +1 14 0 2
0 3 0 y = 20 2 0 0.5 4.8528

i +1
0 0 6 z i +1 14 1 0.5 0 1.4806

x i +1 0.5628

=
y
6
.
3866
i +1

z 1.9206
i +1

Gauss-Seidel
(Example Continued)
Step 18:
18 Use the previous solution to obtain an improve
another estimate for the solution.
1 - 0.1488
5 0 0 x i +1 14 0 2
0 3 0 y = 20 2 0 0.5 6.4630

i +1
0 0 6 z i +1 14 1 0.5 0 1.8197

x i +1 0.1491

y i +1 = 6.4626
z 1.8195
i +1

The Gauss-Seidel
method does not always
converge (i.e. never finds
the answer)

Gauss-Seidel
(Convergence)
How do I know when the Gauss-Seidel
Iteration method has converged?
Convergence Criteria
1. The absolute value of the difference between each
element of the i+1 and i iterations is less than a
small number (Convergence Tolerance).
18th Iteration

17th Iteration

Difference

0.1491

6
.
4626

1.8195

0.1488

6
.
4630

1.8197

0.0003

0
.
0004

0.0002

Gauss-Seidel
(Convergence)
2. The absolute value of the difference in the
norms of the solution vectors for the i+1 and i
iterations divided by the norm of the solution
vector for the ith iteration.
Norm i +1 Norm i

Ratio = abs
Norm i

18th Iteration

17th Iteration

0.1491

6.4626
1.8195

0.1488

6
.
4630

1.8197

Norm=6.7155052

Norm=6.7159377

Ratio=6.44e-5

Gauss-Seidel
(Example Program)

Solution Methods
There are many methods for solving linear equations.

[a]{x} = {b}
Gaussian Elimination
Cramers Rule

Cholesky factors
Gauss-Seidel

Cholesky Factors
[A]{x} = {b}

U11
0
[U ] =
M

[A] = [L][U ]
L11
L
[L] = 21
M

Ln1

0 L
L22 L
M O
Ln 2

L Lnn
0
0
0

[L]{y} = {b}
[U ]{x} = {y}

U12 L U1n
U 22 L U 2n
M O M

0 L U nn

Most Commonly Used Solution

Methods
The Cholesky factorization method is used
more often in commercial software than are the
other methods. In general it is the most
efficient and robust.
Of the other methods, Gauss-Seidel and
Gaussian elimination are used about the same.
Cramers rule is seldom used in programs
involving large numbers of equations.

MATLAB Equation Solver

The Cholesky Factorization Method is used in
MATLAB when the left-division operator is
encountered.
a=[1 2 3; 2 3 4; 3 4 1];
b=[14; 20; 14];
x=a\b

Assignment
1. Use Cramers Rule and Gaussian Elimination to solve
the following equations. Verify your answers using the
left-division operator (Cholesky Factors) in MATLAB.
2. Compute the determinant of each coefficient matrix and
verify your answers with MATLAB.
3. Enter the Gauss-Seidel program discussed in class into
MATLAB. Modify the program to create a plot that
shows the convergence ratio versus iteration number.
Use it to solve each set of equations. Turn in both the
program and your answers.

10 - 1 - 1 x1 13
1 10 1 x = 16

2
- 1 - 1 10 x 3 35

4 2 1 x 14
1 5 - 1 y = 10

1 1 8 z 20