ENGG 407 - P12 - L20 - Lecture - 03

ENGG 407 Numerical Methods in Engineering
Lecture #3 Dr. Sameh Nassar

L20 P12
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Recall: ENGG 407 W12 Topics

1. Introduction and Mathematical Background (Ch. #1, #2) 2. Roots of Nonlinear Equations (Ch. #3)
3. Linear Equations and Systems (Ch. #4)

4. Interpolation, Least-squares Estimation and Curve Fitting (Ch. #5) 5. Numerical Differentiation (Ch. #6) 6. Numerical Integration (Ch. #7) 7. Ordinary Differential Equations: Initial Value Problems (Ch. #8) 8. Ordinary Differential Equations: Boundary Value Problems (Ch. #9)
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Linear Algebraic Equations

Simultaneous linear algebraic equations:
a11 x 1 + a12 x 2 + a13 x 3 +.... + a1n x n = b1 a21 x 1 + a22 x 2 + a23 x 3 +.... + a2n x n = b 2 M M M M
as: Constant Coefficients
bs: Constants
an1 x 1 + an2 x 2 + an3 x 3 +.... + ann x n = b n

Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

In Matrix form:
[A]{x} = {B}
[A] {B} {x}
or simply
Ax = b
n*n square matrix of coefficients n*1 column vector of constants n*1 column vector of unknowns
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
a11 a 21 A = a31 M an1
a12 a22 a32 M an2
a13 a23 a33 M an3
...... ...... ...... M ......
a1n a2n a3n M ann
x1 x 2 x = x3 M xn
b1 b 2 b = b3 M bn
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Small Number of Equations

Small sets of simultaneous equations means in general that: n 3. Methods: (1) Graphical method (2) Cramers rule (3) Elimination of unknowns
Sameh Nassar
Something you know or may tried before
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Large Number of Equations

Means that we have to use Numerical Methods. Methods: (1) Gauss Elimination (2) Gauss-Jordan Elimination (3) LU Decomposition (4) Jacobi
Iterative methods Direct (Elimination) Methods
(4) Gauss-Seidel
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Graphical Method (1/2)

Case of 2 Equations (for example), i.e. n =2:
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Graphical Methods (2/2)

3 x 1 +2 x 2 = 18 x 1 +2 x 2 = 2 (1) (2)
3 x2 = x1 + 9 2 1 x2 = x1 + 1 2
Sameh Nassar
( 1a ) ( 2a )
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Cramers Rule
b1 b2 x1 = b3
a12 a22 a32 | A|
a13 a23 a33

x2 =
a11 a31
b1 b3 | A|
a13 a23 a33
a11 a21 x3 = a31
a12 a22 a32 | A|
b1 b2 b3
a21 b2
Where | | is the corresponding Determinant

Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Elimination of Unknowns
3 x 1 +2 x 2 = 18 x 1 +2 x 2 = 2
3 x2 = x1 + 9 ( 1a ) 2 3 x1 + 2( x1 + 9 ) = 2 ( 2 a ) 2
(1) (2)
x2 elimination
x1 3 x1 + 18 = 2 4 x1 = 16 ( 2b )
Sameh Nassar
x1 = 4
3 3 x2 = x1 + 9 = * 4 + 9 = 3 2 2
x1 back-substitution
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Problems with Some Linear Equations
No Solution
Infinite No. of Solutions
Singular Systems
Sameh Nassar
Ill-Conditioned Systems
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Gauss Elimination Method

Nave Gauss Elimination (or simply Gauss Elimination)
It is an extension of the method of Elimination of Unknowns to large sets of equations by developing a systematic scheme or algorithm to eliminate unknowns and then back substitute. Therefore, as in the shown case of the solution of 2 equations, the technique for n equations consists of two stages (phases): - Forward elimination of unknowns - Back substitution
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

b1 b = b2 b3
a11 a 21 a31
Sameh Nassar
a12 a22 a32
a13 M b1 a23 M b2 a33 M b3
Augmented Matrix
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

a11 a12 a13 M b1 a a a M b 2 21 22 23 a31 a32 a33 M b3
a11 a12 a13 M b1 0 a' a' M b' 22 23 2 0 a'32 a'33 M b'3
Phase 1: Forward Elimination
R2 ( R3 (
a 21 )R1 a 11 a 31 )R 1 a 11
R3 (
Sameh Nassar
a11 0 0
a12 a'22 0
a13 a'23 a"33
b1 M b'2 M b"3 M
a'32 )R 2 a'22
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Phase 2: Back Substitution
b"3 x3 = a" a"33
b'2 a'23 x 3 x2 = a'22

b1 a13 x 3 a12 x 2 x1 = a11
Sameh Nassar
a11 0 0
a12 a'22 0
a13 a'23 a"33
M b1 M b'2 M b"3
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Pivot Equation
Pivot Coefficient or Pivot Element
Division of a row Ri by its pivot element (aii , aii , etc.), is called:
Row Normalization
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

In General:
(1) Forward Elimination
a11 0 0 M 0
a12
' a22 0
a13 ......
' a23 ...... " a33 ......
a1n
' a2 n " a3n
M M M M
M 0
M 0
......
(n ...... ann1) M
b1 ' b2 " b3 M (n bn 1)
(2) Back Substitution
Upper Triangular Matrix
b xn = a
Sameh Nassar
(n - 1) n (n - 1) nn
b xi =
(i 1) i
j =i +1 (i 1) ii
(i aij 1) x j
for i = n 1, n 2,....,1
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #1
Use Gauss Elimination to solve:
Check your answers by substituting them into the original equations.

Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #1 (Sol. 1/3)
10 R 2 ( )R1 8 12 R 3 ( )R1 8
10 2 ( ) * 2 = 0.5 8 10 4 ( ) * 2 = 6.5 Sameh Nassar 8
8 2 2 M 2 10 2 4 M 4 12 2 2 M 6
2( 12 ) * 2 = 1.0 8 12 2 ( ) * 2 = 5.0 8
4(
10 ) * 2 = 6.5 8
Augmented Matrix
12 6 ( ) * 2 = 9.0 8
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
R3 (
1.0 )R 2 0.5
2 2 M 2 8 0 0.5 6.5 M 6.5 5 M 9 0 1
9(
5(
1.0 ) * 6.5 = 8 0.5
1.0 ) * 6.5 = 4 0.5
Sameh Nassar
2 2 M 2 8 0 0.5 6.5 M 6.5 0 0 8.0 M 4.0
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

4 x3 = = 0.5 8 6.5 6.5 * 0.5 x2 = = 6.5 0.5
2 2 M 2 8 0 0.5 6.5 M 6.5 0 0 8.0 M 4.0
2 (2 * 0.5) 2 * 6.5 = 1.5 x1 = 8
8(1.5) + 2(6.5) 2(0.5) = 2

Check:
10(1.5) + 2(6.5) + 4(0.5) = 4 12(1.5) + 2(6.5) + 2(0.5) = 6
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Problems Encountered (1) Round-off Errors: become important when large numbers of equations are solved. Every result is dependent on previous results. Hence, an error in the early steps will tend to propagate. (2) Ill-Conditioned Systems: are those where small changes in coefficients results in large changes in the solution. (3) Division by Zero: For example if a11 is zero, the elimination procedure fails. On the other hand, although there may be no zeros on the main diagonal in the original matrix, the elimination process may create zero aii elements on the main diagonal in the final matrix. Similar problems may occur if a11 (or any resulted aii) is close to zero.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Overcoming Encountered Problems (1/2)
(1) Use of More Significant Figures: To help in solving the problems of: Round-off errors Ill-conditioned systems (2) Pivoting: Before each row is normalized, the largest available coefficient in the column below the pivot element is determined. The rows can then be switched so that the largest element is the pivot element. This approach is called Partial Pivoting.
Ex:
Sameh Nassar
8 2 2 M 2 10 2 4 M 4 12 2 2 M 6
12 2 2 M 6 10 2 4 M 4 8 2 2 M 2
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Overcoming Encountered Problems (2/2)
(2) Partial Pivoting: Avoids division by zero Minimizes Round-off errors Serves as a partial remedy for Ill-conditioning. (3) Scaling: When the magnitude of the elements of one or more of the equations are greatly different from the magnitudes of the elements of the other equations, this leads to significant round-off errors . Scaling is accomplished by dividing the elements of each row by the largest element in the row (thus, the resulted largest element is 1.0). After scaling, pivoting is employed. (This is known as scaled pivoting).
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Scaling and Partial Pivoting (1/2)

Pivoting only,
no scaling
divide R1 by 0.5 Scaling divide R2 by 1.9 divide R3 by 1
0.1 0.3 0.5 M 0.44 0.5 1 1.9 M 0.67 0.3 0.52 1 M 0.01
0.1 0.3 0.5 M 0.44 0.5 1 1.9 M 0.67 0.3 0.52 1 M 0.01
0.5 1 1.9 M 0.67 0.1 0.3 0.5 M 0.44 0.3 0.52 1 M 0.01
0.6 1 M 0.88 0.2 0.2632 0.5263 1 M 0.3526 0.3 0.52 1 M - 0.01 0.52 1 M - 0.01 0.3 0.2632 0.5263 1 M 0.3526 0.2 0.6 1 M 0.88 0.3 0.52 1 M 0.01 0.5 1 1.9 M 0.67 0.1 0.3 0.5 M 0.44
Scaling, then pivoting using

scaled coefficients
0.6 1 M 0.88 0.2 0.2632 0.5263 1 M 0.3526 0.3 0.52 1 M - 0.01
Scaling, then pivoting 0.1 0.3 0.5 M 0.44 using original coefficients 0.5 1 1.9 M 0.67
(Recommended approach)
Sameh Nassar
0.3 0.52 1 M 0.01
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Scaling and Partial Pivoting (2/2)

As shown, scaling itself will sometimes introduce round-off errors. Thus, in most Gauss elimination applications, scaling is employed to calculate scaled values of the coefficients solely as a criterion for pivoting but the original coefficient values are retained for the actual elimination and substitution computations;
details on previous slide!

Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #2
Given the equations:
2x1 6x2 x3 = 38 3x1 x2 + 7x3 = 34 8x1 + x2 2x3 = 20

(a) Solve by Gauss elimination with partial pivoting only (i.e. no scaling). Show all steps of the computations. (b) Substitute your results into the original equations to check your answers.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

2x1 6x2 x3 = 38 3x1 x2 + 7x3 = 34 8x1 + x2 2x3 = 20
3 R2 ( ) R1 8 R3 ( 2 ) R1 8
2 6 1 M 38 3 1 7 M 34 8 1 2 M 20
Augmented Matrix
34 ( 3 ) * 20 = 26.5 8
8 1 2 M 20 3 1 7 M 34 2 6 1 M 38
2 ) *1 = -5.75 8 2 1 ( ) * 2 = 1.5 8 6(
3 1 ( ) *1 = -1.375 8 3 7 ( ) * 2 = 7.75 Sameh Nassar 8
38 (
2 ) * 20 = 43 8
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

1 2 M 20 8 0 1.375 7.75 M 26.5 0 5.75 1.5 M 43
26.5 (
1.375 ) * 43 = -16.2174 5.75
R3 (
1.375 ) R2 5.75
1 2 M 20 8 0 5.75 1.5 M 43 0 1.375 7.75 M 26.5
7.75 (
1.375 ) * 1.5 = 8.108696 5.75
Sameh Nassar
1 2 M 20 8 0 5.75 1.5 M 43 0 0 8.108696 M -16.2174
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

-16.2174 x3 = = 2 8.108696
1 2 M 20 8 0 5.75 1.5 M 43 0 0 8.108696 M -16.2174
x2 =
43 (1.5) * 2 =8 5.75
20 (2) * 2 1* 8 x1 = =4 8
2 * 4 6 * 8 ( 2) = 38
Check:
Sameh Nassar
3 * 4 8 + 7 * ( 2) = 34 8 * 4 + 8 2 * (-2) = 20
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Gauss-Jordan Elimination
Is a variation of Gauss Elimination. The major difference is that when an unknown is eliminated in the Gauss-Jordan Elimination method, it is eliminated from all other equations rather than just the subsequent ones as in Gauss elimination. In addition, all rows are normalized by dividing them by their pivot elements. Thus, the elimination step results in an Identity matrix rather than an upper triangular matrix. Hence, it is not necessary to employ back substitution.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Gauss-Jordan Elimination
Case of n = 3
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #3
Use Gauss-Jordan elimination to solve:
2x1 + x2 x3 = 1 5 x1 + 2x2 + 2x3 = 4 3x1 + x2 + x3 = 5

Do not employ pivoting. Check your answers by substituting them into the original equations.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

2x1 + x2 x3 = 1 5 x1 + 2x2 + 2x3 = 4 3 x1 + x2 + x3 = 5
2 1 1 M 1 5 2 2 M 4 3 1 1 M 5
R1/2
1 0.5 0.5 M 0.5 5 2 M 4 2 3 1 1 M 5
R2 5 * R1
R3 3 * R1
1 0.5 0.5 M 0.5 5 2 2 M 4 3 1 1 M 5
4 5 * 0.5 = 6.5
5 3 * 0.5 = 3.5
2 5 * 0.5 = -0.5
2 5 * 0.5 = 4.5
Sameh Nassar
1 3 * 0.5 = -0.5
1 3 * 0.5 = 2.5
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

1 0.5 0.5 M 0.5 0 0.5 4.5 M 6.5 0 0.5 2.5 M 3.5
R2/-0.5
1 0.5 0.5 M 0.5 0 1 9 M 13 0 0.5 2.5 M 3.5
0.5 (0.5) * 9 = 4
R1 0.5 * R2
R3 (0.5) * R2
1 0.5 0.5 M 0.5 0 1 9 M 13 0 0.5 2.5 M 3.5
0.5 0.5 *13 = 6
2.5 (0.5) * 9 = -2
3.5 (0.5) *13 = 10

Sameh Nassar
1 0 4 M 6 0 1 9 M 13 0 0 - 2 M 10
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

1 0 4 M 6 0 1 9 M 13 0 0 - 2 M 10
R1 4 * R3
R3/-2
1 0 4 M 6 0 1 9 M 13 0 0 1 M 5
R2 (9) R3
1 0 4 M 6 0 1 9 M 13 0 0 1 M 5
6 4 * 5 = 14
13 ( 9) * 5 = 32
X1 = 14 X2 = -32
Sameh Nassar
X3 = -5
1 0 0 M 14 0 1 0 M 32 0 0 1 M - 5
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Check:
2 * 14 + (-32) (-5) = 1 5* 14 + 2 * (-32) + 2 * (-5) = 4 3* 14 + (-32) + (-5) = 5
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
LU decomposition is a class of elimination methods Its major appeal is that the time-consuming elimination step can be formulated so that it involves only operations on the matrix of coefficients A. Therefore, it is will suited for situations where many right-hand-size vectors {B} (i.e. b) must be evaluated for a single value of A. LU decomposition provides an efficient means for computing the matrix inverse, i.e. A-1.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
Define: Then:
Similar to the 1st phase of Gauss elimination, consider:
Then:
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
General Forms (for computing the elements of L and U):
Computing the 1st column of L (for known u11) and 1st row of U (for known l11)
Computing the main diagonal elements of L (for known uii) and U (for known lii)
Computing the elements of the ith row of U or the jth column of L
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
General Solution (for d and x):
In General:
i 1
b1 d1 = l11
bi lij d j di =
j =1
lii
for i = 2,3,....,n
Forward Substitution
dn xn = unn
Sameh Nassar
di xi =
j = i +1
ij
xj for i = n 1, n 2,....,1
Back Substitution
uii
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
Thus, the LU decomposition method is decomposing (factoring) the coefficient matrix A into a product of 2 matrices L (lower-triangular matrix) and U (upper-triangular matrix).
Several methods can be used to obtain the corresponding L and U.
Methods to be discussed: (1) Doolittle Decomposition (all lii i.e. L main diagonal elements = 1). (2) Crouts method all uii i.e. U main diagonal elements = 1).
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
(1) Doolittle Decomposition (Factorization):
Is related to Gauss Elimination.
Case of 3 Equations (for ex.), i.e. n =3:
a11 a A 21 a31
a12 a22 a32
a13 a23 a33
a11 a12 0 a' 22 0 0
a13 a'23 a"33
U A = LU L
1 l 21 l 31
0 1 l 32
0 0 1
Using Gauss Elimination:
Sameh Nassar
a21 l21 = a11
a31 l31 = a11
a'32 l32 = a'22
Coefficients of Gauss elimination process
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
(1) Doolittle Decomposition (Factorization): cont.
d1 = b1 d 2 = b2 l21 * d1 d 3 = b3 l32 * d 2 l31 * d1
1 0 l 21 1 l31 l32
0 d1 b1 0 d 2 = b2 1 d 3 b3
a11 a12 a13 x1 d1 0 a' a' x = d 22 23 2 2 0 0 a"33 x3 d3

Sameh Nassar
x3 = d3 / a"33 x2 = (d 2 a'23 *x3 ) / a'22 x1 = (d1 a13 * x3 a12 * x2 ) / a11
Back Substitution
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
(1) Doolittle Decomposition (Factorization): cont. All lii are equal to 1.
Note that pivoting is required if any uii is zero

Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
(1) Doolittle Decomposition (Factorization): cont.
d1 = b1
d i = bi lij d j
j =1
i 1
for i = 2,3,...., n
dn xn = unn
Sameh Nassar
di xi =
j = i +1
ij
xj for i = n 1, n 2,....,1
Back Substitution
uii
Note that pivoting is required if any uii is zero
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #4
Solve the following system of equations by LU decomposition (Doolittle approach) without pivoting:
8x1 + 4 x2 x3 = 11 2x1 + 5x2 + x3 = 4 2 x1 x2 + 6 x3 = 7
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

8x1 + 4 x2 x3 = 11 2x1 + 5x2 + x3 = 4 2 x1 x2 + 6 x3 = 7
8 4 - 1 A = - 2 5 1 2 1 6
11 B = 4 7
l11 = l22 = l33 = 1
u11 = a11 = 8 u12 = a12 = 4

u13 = a13 = 1
Sameh Nassar
l21 = l31 =
a21 2 = = 0.25 u11 8 a31 2 = = 0.25 u11 8
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

u 22 = a22 l21 * u12 = 5 (0.25) * 4 = 6
u23 = a23 l21 * u13 = 1 (0.25) * 1 = 0.75
l32 = a32 l31 * u12 1 0.25 * 4 = = 0.33333 u 22 6
u33 = a33 l31 * u13 l32 * u 23 = 6 0.25 * 1 (0.33333) * 0.75 = 6.5
L
Sameh Nassar
0 0 1 0.25 1 0 0.25 0.33333 1
8 4 - 1 0 6 0.75 0 0 6.5
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
0 1 0.25 1 0.25 0.33333
0d1 11 0d2 = 4 1d3 7
d1 = 11 d 2 = 4 ( 0.25) *11 = 6.75 d 3 = 7 (-0.33333) * 6.75 0.25 *11 = 6.5
8 4 - 1 x1 11 0 6 0.75 x = 6.75 2 0 0 6.5 x3 6.5

Sameh Nassar
x3 = 6.5 / 6.5 = 1.0 x2 = (6.75 0.75 *1.0) / 6 = 1.0 x1 = (11 (1) *1.0 4 *1.0) / 8 = 1.0
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
(2) Crouts Method: All uii are equal to 1.
Note that pivoting is required if any lii is zero

Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
LU Decomposition
(2) Crouts Method: cont.
b1 d1 = l11
bi lij d j di =
j =1
i 1
lii
for i = 2,3,...., n
xn = d n
Sameh Nassar
xi = d i
j = i +1
ij
xj
for i = n 1, n 2,....,1
Back Substitution
Note that pivoting is required if any lii is zero
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #5
Solve the following system of equations by LU decomposition (Crouts approach) without pivoting:
8x1 + 4 x2 x3 = 11 2x1 + 5x2 + x3 = 4 2 x1 x2 + 6 x3 = 7
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

8x1 + 4 x2 x3 = 11 2x1 + 5x2 + x3 = 4 2 x1 x2 + 6 x3 = 7
8 4 - 1 A = - 2 5 1 2 1 6
11 B = 4 7
u11 = u22 = u33 = 1
l11 = a11 = 8 l21 = a21 = 2

l31 = a31 = 2
Sameh Nassar
u12 = u13 =
a12 4 = = 0.5 l11 8 a13 1 = = 0.125 l11 8
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

l22 = a22 l21 * u12 = 5 (2) * 0.5 = 6
u 23 = a23 l21 * u13 1 (2) * 0.125 = = 0.125 l22 6
l32 = a32 l31 * u12 = 1 2 * 0.5 = 2

l33 = a33 l31 * u13 l32 * u23 = 6 2 * 0.125 (2) * 0.125 = 6.5
L
Sameh Nassar
0 0 8 2 6 0 2 2 6.5
1 0.5 - 0.125 0 1 0.125 0 0 1
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
8 0 0 d1 11 2 6 0 d2 = 4 2 2 6.5d 7 3
d1 = 11 / 8 = 1.375 d 2 = [4 ( 2) *1.375] / 6 = 1.125 d 3 = [7 2 *1.375 (2) *1.125] / 6.5 = 1
1 0.5 - 0.125 x1 1.375 0 1 0.125 x2 = 1.125 0 0 1 x3 1

Sameh Nassar
x3 = 1.0 x2 = (1.125 0.125 *1.0) = 1.0 x1 = 1.375 (0.125) *1 0.50 *1 = 1.0

Same results as Doolittle (Ex. #4)
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
The Matrix Inverse

Why the Matrix Inverse is needed for solving systems of linear equations?
A.x = b
By definition:
x = A -1 .b
where I is the identity matrix
Note that the condition for the existence of A-1 is that: |A| # 0.
a11 a21 a 31
Sameh Nassar
a12 a22 a32

A
a13 z11 a23 z 21 a33 z31
z12 z 22 z32
A -1
z13 1 0 0 z 23 = 0 1 0 z33 0 0 1 I
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Matrix Inverse using LU Decomposition

The matrix inverse can be obtained using LU Decomposition
Using LU Decomposition
(where M is an intermediate Matrix)
z11 A 1 = z 21 z 31
z12 z 22 z32
z13 z 23 z33
m11 M = m21 m31
m12 m22 m32
m13 m23 m33
d1 m13 m11 m12 The vectors: m21 , m22 and m23 will be obtained as the d 2 vector. d3 m33 m31 m32
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Doolittle LU Decomposition:
1 0 l 21 1 l31 l32
m11 = I11
0 m11 0 m21 1 m31
m12 m22 m32
m13 1 0 0 I 11 m23 = 0 1 0 = I 21 I 31 m33 0 0 1
I 12 I 22 I 32
I 13 I 23 I 33
LM
m12 = I12
=
m22 = I 22 l21 * m12
=
m13 = I13
m21 = I 21 l21 * m11
m23 = I 23 l21 * m13
m31 = I 31 l32 * m21 l31 * m11 m32 = I 32 l32 * m22 l31 * m11 m33 = I 33 l32 * m23 l31 * m13
Sameh Nassar
Compute M
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Doolittle LU Decomposition:
a11 0 0
a12 a' 22 0
a13 z11 a' 23 z 21 a"33 z31

UA-1
z12 z 22 z32
z13 m11 z 23 = m21 m31 z33

=
m12 m22 m32

M
z33 = m33 / a"33
m13 m23 m33
z31 = m31 / a"33 z21 = (m21 a'23*z31) / a'22
z32 = m32 / a"33 z22 = (m22 a'23*z32 ) / a'22
z23 = (m23 a'23*z33) / a'22
z11 = (m11 a13 * z31 a12 * z21) / a11 z12 = (m12 a13 * z32 a12 * z22 ) / a11 z13 = (m13 a13 * z33 a12 * z23) / a11
Sameh Nassar
Back Substitution
Compute A-1
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #6
Solve the following system of equations using the matrix inverse approach (LU by Doolittle approach):
8x1 + 4 x2 x3 = 11 2x1 + 5x2 + x3 = 4 2 x1 x2 + 6 x3 = 7

- Use the L and U matrices obtained from Example #4. - When obtaining A-1 , check it by verifying [A][A-1] = [I].
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

From Example #4
8 4 - 1 A = - 2 5 1 2 1 6
11 b = 4 7
0 0 1 L = 0.25 1 0 0.25 0.33333 1
Sameh Nassar
8 4 - 1 U = 0 6 0.75 0 0 6.5
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

0 0 m11 1 0.25 1 0 m21 0.25 0.33333 1 m31 m12 m22 m32 m13 1 0 0 m23 = 0 1 0 0 0 1 m33
LM
m11 = 1 m21 = 0 (0.25) *1 = 0.25 m31 = 0 ( 0.33333) * 0.25 0.25 *1 = -0.16667
m12 = 0 m22 = 1 (-0.25) * 0 = 1 m32 = 0 ( 0.33333) *1 0.25 * 0 = 0.33333

m13 = 0 m23 = 0 ( 0.25) * 0 = 0
Sameh Nassar 33
1 0 0 M = 0.25 1 0 0.16667 0.33333 1
m = 1 (0.33333) * 0 0.25 * 0 = 1
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

8 4 - 1 z11 0 6 0.75 z 21 0 0 6.5 z31 z12 z 22 z32 z13 1 0 0 = 0.25 z 23 1 0 0.16667 0.33333 1 z33
= M
UA-1 z31 = 0.16667/6.5 = -0.025641
z 21 = (0.25 0.75 * -0.02564)/6 = 0.044872 z11 = (1 ( 1) * -0.02564 4 * 0.04487)/8 = 0.099359

z 32 = 0.33333/6. 5 = 0.051282 z 22 = (1 0.75 * 0.051282)/ 6 = 0.160256 z12 = ( 0 ( 1) * 0.051282 4 * 0.160256)/ 8 = -0 .073718
z 33 = 1/6.5 = 0.153846 z 23 = (0 0.75 * 0.153846)/ 6 = -0 .019231

13 Sameh Nassar
z = (0 ( 1) * 0.153846 4 * -0 .01923 )/8 = 0.028846
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

0.099359 0.073718 0.028846 A1 = 0.044872 0.160256 0.019231 0.025641 0.051282 0.153846
Check:
8 4 - 1 0.099359 0.073718 0.028846 1 0 0 AA1 = - 2 5 1 0.044872 0.160256 0.019231 = 0 1 0 2 1 6 0.025641 0.051282 0.153846 0 0 1
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

x = A -1 .b
0.099359 0.073718 0.028846 A1 = 0.044872 0.160256 0.019231 0.025641 0.051282 0.153846
11 x1 b = 4 x = x2 7 x3
x1 0.099359 0.073718 0.028846 11 x = x2 = 0.044872 0.160256 0.019231. 4 x3 0.025641 0.051282 0.153846 7
0 .099359 * 11 0 .073718 * 4 + 0 .028846 * 7 1 = 0 .044872 * 11 + 0 .160256 * 4 0 .019231 * 7 = 1 0 .025641 * 11 + 0 .051282 * 4 + 0 .153846 * 7 1

Sameh Nassar
Same results as Ex. #4
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Matrix Inverse using Gauss-Jordan Elimination

By definition: where I is the identity matrix
a11 a 21 a31
a12 a22 a32
a13 z11 a23 z 21 a33 z31
z12 z 22
A
a11 a 21 a31 a12 a22 a32 A a13 z11 1 a23 z 21 = 0 a33 z31 0 x1 = b 1
z13 1 0 0 z 23 = 0 1 0 z32 z33 0 0 1 I A -1
Hence, Gauss-Jordan elimination (for example) can be used to solve 3 systems of linear equations of the forms (i.e. applying it 3 times in this case):
a11 a 21 a31 a12 a22 a32 A a13 z12 0 a23 z 22 = 1 a33 z32 0 x2 = b 2 a11 a 21 a31 a12 a22 a32 A a13 z13 0 a23 z 23 = 0 a33 z33 1 x3 = b 3
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Recall: Solving Systems of Linear Equations (SLEs)
Methods: (1) Gauss Elimination (2) Gauss-Jordan Elimination (3) LU Decomposition (4) Jacobi
Iterative methods Direct (Elimination) Methods
(4) Gauss-Seidel
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Iterative Methods for Solving SLEs

Called iterative methods, and hence, are similar in spirit to the iterative techniques of root finding (Ch. 3), specifically the fixed-point iteration technique. Employ initial guesses and then iterates to obtain refined estimates of the solution. These methods are particularly suitable for large number of equations where elimination methods can be subject to round-off errors. However, there are certain instances where the method will not converge to the correct answer.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

b1 a12 x2 a13 x3 x1 = a11

b2 a21 x1 a23 x3 x2 = a22
b3 a31 x1 a32 x2 x3 = a33

Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

(1) Gauss-Seidel Method:
(2) Jacobi Method:
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Gauss-Seidel Method
| |=
j ai
x i , j x i , j1 x i, j
100%
1st Iteration
where: i = 1, 2,., n j = 1, 2,. is the iteration number
Note that the unknowns are updated sequentially in every iteration using the most updated value for each unknown.
Sameh Nassar
2nd Iteration
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Jacobi Method
| |=
j ai
x i , j x i , j1 x i, j
100%
1st Iteration
where: i = 1, 2,., n j = 1, 2,. is the iteration number
Note that the unknowns are updated only once (i.e. all together) in every iteration using the values for all unknowns from the previous iteration.
Sameh Nassar
2nd Iteration
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Convergence Criterion
a11 a 21 A = a31 M an1 a12 a22 a32 M an2 a13 a23 a33 M an3 ...... a1n ...... a2n ...... a3n M M ...... ann
| aii |> | aij |

j =1 j i
Condition: the absolute value of the main diagonal element must be greater than the sum of the absolute values of the off-diagonal elements for each row. Such systems are called Diagonally Dominant. This condition is not necessary for convergence but it guarantees convergence.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #7
Use the Gauss-Seidel method to solve the following system of equations to a tolerance of s = 5%:
- 3x1 + x2 + 12 x3 = 50 6 x1 x2 x3 = 3 6 x1 + 9x2 + x3 = 40
Start with initial values of zero for all unknowns.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

- 3x1 + x2 + 12 x3 = 50 6 x1 x2 x3 = 3 6 x1 + 9x2 + x3 = 40
6 x1 x2 x3 = 3 6 x1 + 9x2 + x3 = 40 - 3x1 + x2 + 12 x3 = 50
Diagonally Dominant
3 + x2 + x3 x1 = 6
40 6 x1 x3 x2 = 9 50 + 3 x1 x2 x3 = 12
x10 = x20 = x30 = 0
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

1st Iteration
x11 = x 21 = 3+ 0+ 0 = 0.5 6 40 6 * 0.5 0 = 4.11111 9
| 1 1 |= a | 1 2 |= a | 1 3 |= a 0.5 0 100% = 100% 0.5 4.11111 0 100% = 100% 4.11111 3.949074 0 100% = 100% 3.949074
50 + 3 * 0.5 4.11111 x 31 = = 3.949074 12
2nd Iteration
x12 = x 22 = 3 + 4.11111+ 3.949074 = 1.843364 6 40 6 *1.843364 3.949074 = 2.776749 9
2 | a1 |=
1.843364 0.5 100% = 72.88% 1.843364 2.776749 - 4.11111 100% = 48.05% 2.776749 4.396112 - 3.949074 100% = 10.17% 4.396112
2 | a 2 |=
50 + 3 *1.843364 2.776749 x 32 = = 4.396112 12

Sameh Nassar
2 | a 3 |=
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

iteration 1 unknown x1 x2 x3 x1 x2 x3 x1 x2 x3 x1 x2 x3 value 0.5 4.111111 3.949074 1.843364 2.776749 4.396112 1.695477 2.82567 4.355063 1.696789 2.829356 4.355084
a
100.00% 100.00% 100.00% 72.88% 48.05% 10.17% 8.72% 1.73% 0.94% 0.08% 0.13% 0.00%
maximum a
100.00% 72.88%
8.72%
4
Sameh Nassar
0.13%
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #8
Use the Jacobi method to solve the following system of equations:
- 3x1 + x2 + 12 x3 = 50 6 x1 x2 x3 = 3 6 x1 + 9x2 + x3 = 40
Start with initial values of zero for all unknowns and apply 2 iterations only.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

- 3x1 + x2 + 12 x3 = 50 6 x1 x2 x3 = 3 6 x1 + 9x2 + x3 = 40
6 x1 x2 x3 = 3 6 x1 + 9x2 + x3 = 40 - 3x1 + x2 + 12 x3 = 50
Diagonally Dominant
3 + x2 + x3 x1 = 6
40 6 x1 x3 x2 = 9 50 + 3 x1 x2 x3 = 12
x10 = x20 = x30 = 0
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

1st Iteration
3+ 0+ 0 x11 = = 0.5 6 40 6 * 0 0 x 21 = = 4.444444 9 50 + 3 * 0 0 x 31 = = 4.166667 12
| 1 1 |= a | 1 2 |= a | 1 3 |= a 0.5 0 100% = 100% 0.5 4.444444 0 100% = 100% 4.444444 4.166667 0 100% = 100% 4.166667
2nd Iteration
x12 = 3 + 4.444444 + 4.166667 = 1.935185 6 40 6 * 0.5 4.166667 = 3.648148 x 22 = 9 50 + 3 * 0.5 4.444444 x 32 = = 3.921296 12
2 | a1 |=
1.935185 0.5 100% = 74.16% 1.935185 3.648148 - 4.444444 100% = 21.83% 3.648148 3.921296 - 4.166667 100% = 6.26% 3.921296
2 | a 2 |=
2 | a 3 |=
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
True Errors and Residuals

For a system of linear equations Ax = b with true solution vector xtrue solution and numerical approximate solution vector xnumerical solution , the following error vectors are defined: True Error Vector e: e = xtrue solution - xnumerical solution
Residual Vector r (or simply Residuals): r = Axtrue solution - Axnumerical solution = b - Axnumerical solution Similar to what discussed before for single variables (Lecture #2), r can be estimated while e is usually unknown. However, r does not really indicate how small the errors in the solution are and hence a small residual vector r does not guarantee a small true error vector e.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
True Error and Residual Vectors

Assume we have:
Assume the following 2 numerical solution vectors were obtained:
Lets now obtain the true error vectors and the corresponding residual vectors of the 2 cases (numerical solutions) and then compare the results.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
True Error and Residual Vectors

e is sensitive to the errors in the numerical solution
r is not sensitive to the errors in the numerical solution

Note that in case (1), the residuals (r1) have the same order of magnitude as the true errors (e1), while in case (2) they are not. This is a typical example of an ill-conditioned coefficient matrix A.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Norms
Based on the discussions in previous slides, a more accurate estimate of the error in a numerical solution (of SLEs) should be obtained. In order to relate the magnitude of the residuals to the magnitude of the true errors, and also to determine the bounds for these errors, we are going to use quantities that measure the magnitude of matrices and vectors. These quantities are called norms. A norm is a non-negative real number assigned to a matrix or a vector. The norm of a vector is denoted by |||| ||||. The norm of a matrix A is denoted by || A]|| (or simply ||A|| ). ||[ ||
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Vector Norms
Categories (Types):
Properties:
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Matrix Norms
Categories (Types):
Euclidean
(m x n matrix)
2-norm
||[A]|| || || =
Where is an eigenvector of the matrix A corresponding to an eigenvalue (discussed later!)

Sameh Nassar
Additional Property:
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Error Bounds of SLEs Numerical Solutions

For a system of linear equations Ax = b with true solution vector xT and numerical approximate solution vector xN , the following terms (using norms) are defined: Relative Error Re: Re = , e = xT - xN (true error)
Relative Residual Rr: Rr =
, r = b - AxN (residual)
Thus, a major objective here is to use Rr (that can be estimated from the numerical solution) to obtain or estimate an upper and lower bounds of Re (recall that Re itself is unknown).
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Error Bounds of SLEs Numerical Solutions
Lower Bound
The term
Upper Bound
is called the Condition Number of A.
Sameh Nassar
Lower Bound
Upper Bound
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Condition Number of a Matrix

Properties:
(I is the identity matrix)
If Cond(A) is large (i.e. >> 1), a small Rr will not guarantee a small Re (i.e. small changes in b or the elements of A may result in large changes in x). In this case, the SLE (and hence A) is ill-conditioned. If Cond(A) is small (i.e. ~ 1), a small Rr will indicate a small Re (i.e. small changes in b or the elements of A will result in small changes in x). In this case, the SLE (and hence A) is well-conditioned.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Eigenvalue Problems
Known also as characteristic-value problems. An eigenvalue is a characteristic value of a square matrix (n x n). Background #1: If A is an n x n matrix, usually there is no obvious geometric relationship between a vector x and its image Ax (i.e. when A is multiplied by x). However, there are some nonzero vectors x that A is mapping into scalar multiples of themselves. These types of vectors arise in several engineering applications that involve vibrations, electrical systems, quantum mechanics, mechanical stress and elasticity.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Eigenvalue Problems
Ax is a scalar multiple of x No Geometric Relation between x and Ax

Sameh Nassar
x is an eigenvector
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Eigenvalue Problems
In the latter case, Ax = x, where is a scalar. The scalar is called an eigenvalue (or characteristic value) of A and x is said to be an eigenvector corresponding to . Background #2: (a) The system of equations: A.x = b is called non-homogeneous (b) The system of equations: A.x = 0 is called homogeneous For case (a), a unique solution of x can be obtained if the equations are linearly independent (i.e. |A| 0). For case (b), nontrivial solutions (any solutions rather than x = 0) can be obtained, however, they are generally not unique (i.e. several solutions of x are available).
Sameh Nassar
Why?
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Eigenvalues & Eigenvectors x

The non-unique solution for x is obtained since several available relationships can be established between the x-vector elements that can be satisfied by various combinations of the x-values. To obtain the relations of the x-values, the eigenvalues of A are obtained first, where the general form of the equations will be:
A.x = .x
A.x - .I.x = 0
[A - .I].x = 0
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Eigenvalues & Eigenvectors x

Hence, the unknown parameter (eigenvalue) should be obtained and then the relations for x-values can be evaluated. Condition to obtain for nontrivial solutions of x: |A - .I| = 0 The values of can be obtained by expanding the determinant |A - .I| and equalize to zero, which will yield a polynomial in the unknown . The values of are the roots of the polynomial (any method of solving SLEs can be used). Then, the eigenvector x can be evaluated for each value of . For any constant c, cx is also an eigenvector of A (therefore, the eigenvectors x are normalized to be unit vectors, i.e. || ||2= 1. ||x||
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Example #9
Find the eigenvalues of the given matrix:
3 2 A= 0 1
Then, evaluate the corresponding eigenvectors.
Sameh Nassar
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Eigenvalues: 3 2 A= 1 0
|A - .I| = 0
3 2 =0 0 1
(3 ) * (2 *1) = 0
2 + 3 + 2 = 0
3 1 = 2
3 32 4 *1* 2 = 2 *1
1 = 1
Sameh Nassar
2 = 2
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Eigenvectors: [A - .I].x = 0 = 1 = -1:
3 2 x1 0 1 x 2 = 0 = 0 0
2 x1 3 (1) = 0 0 (1)x2 1
Use ||x||2= 1 || ||
2 2 x1 1 1 x2 = 0
x12 + ( x1 ) 2 = 1
x2 = x1
x + x =1
2 1 2 2
Sameh Nassar
1 x1 = 2
1 x2 = 2
1 2 x1 = 1 2
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar

Eigenvectors: = 2 = -2:
2 x1 3 (2) = 0 0 (2)x2 1
Use || ||2= 1 ||x||
1 2 x1 = 0 1 2 x2
1 2 x + ( x1 ) = 1 2
2 1
1 x2 = x1 2
x + x =1
2 1 2 2
2 x1 = 5
Sameh Nassar
1 x2 = 5
2 5 x2 = 1 5
ENGG 407
L20
Spring 2012
Lecture #3
Dr. Sameh Nassar
Textbook Readings
Sections:
4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.10 4.11 4.12 (excluding 4.12.1 14.12.4)
Sameh Nassar

ENGG 407 - P12 - L20 - Lecture - 03

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ENGG 407 - P12 - L20 - Lecture - 03

Uploaded by

Copyright:

Available Formats

ENGG 407 Numerical Methods in Engineering

Lecture #3 Dr. Sameh Nassar

Dr. Sameh Nassar

Recall: ENGG 407 W12 Topics

3. Linear Equations and Systems (Ch. #4)

Dr. Sameh Nassar

Linear Algebraic Equations

as: Constant Coefficients

an1 x 1 + an2 x 2 + an3 x 3 +.... + ann x n = b n

Dr. Sameh Nassar

Linear Algebraic Equations

Dr. Sameh Nassar

Linear Algebraic Equations

a11 a 21 A = a31 M an1

a12 a22 a32 M an2

a13 a23 a33 M an3

...... ...... ...... M ......

a1n a2n a3n M ann

Dr. Sameh Nassar

Small Number of Equations

Something you know or may tried before

Dr. Sameh Nassar

Large Number of Equations

Dr. Sameh Nassar

Graphical Method (1/2)

Dr. Sameh Nassar

Graphical Methods (2/2)

Dr. Sameh Nassar

a12 a22 a32 | A|

a13 a23 a33

a13 a23 a33

a11 a21 x3 = a31

a12 a22 a32 | A|

Where | | is the corresponding Determinant

Dr. Sameh Nassar

Dr. Sameh Nassar

Problems with Some Linear Equations

Infinite No. of Solutions

Dr. Sameh Nassar

Gauss Elimination Method

Dr. Sameh Nassar

Gauss Elimination Method

a12 a22 a32

a13 M b1 a23 M b2 a33 M b3

Dr. Sameh Nassar

Gauss Elimination Method

Phase 1: Forward Elimination

a13 a'23 a"33

Dr. Sameh Nassar

Gauss Elimination Method

b"3 x3 = a" a"33

b'2 a'23 x 3 x2 = a'22

a13 a'23 a"33

Dr. Sameh Nassar

Gauss Elimination Method

Pivot Coefficient or Pivot Element

Division of a row Ri by its pivot element (aii , aii , etc.), is called:

Dr. Sameh Nassar

Gauss Elimination Method

(1) Forward Elimination

(2) Back Substitution

Upper Triangular Matrix

Dr. Sameh Nassar

3 1 ( ) 1 = -1.375 8 3 7 ( ) 2 = 7.75 Sameh Nassar 8