Linear Algebraic Equations

Objectives
Solve system of Linear equations using
Elimination Methods
LU Decomposition

Computer Iterative Methods

Gauss-Seidel
Jacobi Iterative Method

Array operations in Excel

Define matrix names in name manager

Matrix Addition
 A_mat+B_mat = C_mat, press CTRL-SHIFT-ENTER

Matrix Multiplication
 MMULT(A_mat,B_mat), press CTRL-SHIFT-ENTER

Matrix Inverse
 MINVERSE(A_mat), press CTRL-SHIFT-ENTER

Matrix Transpose
 TRANSPOSE(B_mat), press CTRL-SHIFT-ENTER

Matrix Determinant
 MDETERM(C_mat), press ENTER

Matrix Inversion
a11 a12 a13 x1 b1

a

a
a23 x2 = b 2
21 22

a31 a32 a33 x3 b3

[A]{x} = {b}
1

{x} = [A ] {b}
Solution

Naive Gauss Elimination

Express as augmented matrix
Multiply Row 1 by (a21/a11) and subtract from Row 2
Express matrix in upper triangular form

Gauss-Jordan
Express as augmented matrix
Reduce coefficient matrix to identity matrix
No back substitution needed.

Figure 9_09.jpg

LU Decomposition
[A]{x}={b} can be decomposed into two matrices [L]
and [U] such that
[L][U]=[A]
[L][U]{x}={b}
Similar to first phase of Gauss elimination, consider
[U]{x}={d}
[L]{d}={b}
[L]{d}={b} is used to generate an intermediate vector
{d} by forward substitution
Then, [U]{x}={d} is used to get {x} by back
substitution

LU Decomposition

LU Decomposition
Ax=b

System of equation

A=LU

LU Decomposition

Ld=b

Solve for d

Ux=d

Solve for x

LU Decomposition

Example:
Solve the ff. system using LU decomposition

3 0.1 0.2 x1 7.85

0.1

7
0.3 x2 = 19.3

0.3 0.2 10 x3 71.4

[A]

{x}

{b}

LU Decomposition
U

0.1
0.2
3
0 7.00333 0.29333

0
10.01200
0

Upper
Triangular
Matrix

LU Decomposition
L

0
0
1
0.03333

1
0

0.1000 0.02713 1

Lower
Triangular
Matrix

The factors used to obtain the U can be assembled to form L

0.1
f 21 =
= 0.03333
3

0.19
0.3
= 0.02713
f 31 =
= 0.1000 f 32 =
7.00333
3

LU Decomposition
[L][U]=[A]

0
0 3
0.1
0.2
1
0.03333
1
0 0 7.00333 0.29333 =

0.02713 1 0
0.1
0
10.01200

3 0.1 0.2
0 .1

7
0 .3

0.3 0.2 10

This is the original matrix!!!!!!!!!!

LU Decomposition
[L]{d}={b}

0
0 d1 7.85
1

0.03333

1
0 d 2 = 19.3

0.1
0.02713 1 d 3 71.4
[L]

{d}

{b}

[U]{x}={d}

0.1
0.2 x1 d1
3
0 7.00333 0.2933 x = d

2 2
0
0
10.0120 x3 d 3
[U]

{x}

{d}

LU Decomposition
[L]{d}={b}

0
0 d1 7.85
1

0.03333

1
0 d 2 = 19.3

0.1
0.02713 1 d 3 71.4
[L]

{d}

{b}

By forward substitution

d1 = 7.85

d 2 = 19.3 0.0333d1 = 19.5617

d 3 = 71.4 0.1d1 (0.02713)d 2 = 70.0843

LU Decomposition
[U]{x}={d}

0.1
0.2 x1 7.85
3
0 7.00333 0.2933 x = 19.5617

2
0
0
10.0120 x3 70.0843
[U]

{x}

{d}

By back substitution

10.012 x3 = 70.0843 x3 = 7.00003

7.00333 x 2 0.2933 x3 = 19.5617 x 2 = 2.5
3 x1 0.1x 2 0.2 x3 = 7.85 x1 = 3

Matrix norm and Condition number

The length of the vector F = a b c can be simply
computed as
2

F e = a +b +c

Length or Euclidean norm of [F]

For an n dimensional vector
X = x1 x 2 L x n
a Euclidean norm is computed as
n

x i2

i =1

For a matrix [A]

n

i =1

Frobenius norm

j =1

a i2, j

16

 Frobenius norm provides a single value to quantify

the size of [A].

Matrix Condition Number

Defined as

Cond [A] = A A

For a matrix [A], this number will be greater

than or equal to 1.
17

Iterative Methods
Recall Techniques for Root finding of Single Equations

Initial Guess
New Estimate
Error Calculation
Repeat until Convergence

Gauss Seidel
a11 a12 a13 x1 b1

a

a22 a23 x2 = b2
21

a31 a32 a33 x3 b3

b1 a12 x2 a13 x3
x1 =
a11
b2 a21 x1 a23 x3
x2 =
a22

b3 a31x1 a32 x2
x3 =
a33

Gauss Seidel
Initial guesses:

x1 = 0, x2 = 0, x3 = 0

b1 a12 0 a13 0 b1
x =
=
a11
a11
1
1

1
21 1

b2 a x a23 0
x =
a22
1
2

1
31 1

1
32 2

b3 a x a x
1
x3 =
a33

Better Estimate

Better Estimate

Better Estimate

Gauss Seidel
1
1

x, x, x

Second Iteration:
1
12 2

1
13 3

b1 a x a x
x =
a11
2
1

2
21 1

1
23 3

2
31 1

2
32 2

b2 a x a x
x =
a22
2
2

1
2

b3 a x a x
2
x3 =
a33

1
3

Better Estimate

Better Estimate

Better Estimate

Jacobi Iteration
a11 a12 a13 x1 b1

a

a22 a23 x2 = b2
21

a31 a32 a33 x3 b3

b1 a12 x2 a13 x3
x1 =
a11
b2 a21 x1 a23 x3
x2 =
a22

b3 a31x1 a32 x2
x3 =
a33

Jacobi Iteration
First Iteration:

x1 = 0, x2 = 0, x3 = 0

b1 a12 0 a13 0 b1
x =
=
a11
a11
1
1

b2 a21 0 a23 0
x =
a22
1
2

b3 a31 0 a32 0
x =
a33
1
3

Better Estimate

Better Estimate

Better Estimate

Jacobi Iteration
1
1

Second Iteration:
1
12 2

x, x, x
1
13 3

b1 a x a x
x =
a11
2
1

Better Estimate

1
23 3

Better Estimate

1
31 1

1
32 2

Better Estimate

b3 a x a x
x =
a33
2
3

1
3

1
21 1

b2 a x a x
x =
a22
2
2

1
2

Example
3 0.1 0.2 x1 7.85

0.1

7
0.3 x2 = 19.3

0.3 0.2 10 x3 71.4

7.85 + 0.1x2 + 0.2 x3
x1 =
3
19.3 0.1x1 + 0.3x3
x2 =
7

71.4 0.3x1 + 0.2 x2

x3 =
10

Gauss Seidel and Jacobi Convergence

Iteration Error:

a ,i =

xi

j 1

xi
< s
100
%
j
xi

Convergence Criterion:

SUMMARY
a11 a12 a13 x1 b1

a

a
a23 x2 = b 2
21 22

a31 a32 a33 x3 b3

[A]{x} = {b}
1

{x} = [A ] {b}
Solution

CASE STUDIES

CASE STUDIES