LU Decomposition

2/4/2014 http://numericalmethods.eng.usf.
edu 1
LU Decomposition

Major: All Engineering Majors

Authors: Autar Kaw

http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates

LU Decomposition


LU Decomposition
LU Decomposition is another method to solve a set of
simultaneous linear equations

Which is better, Gauss Elimination or LU Decomposition?

To answer this, a closer look at LU decomposition is
needed.
Method
For most non-singular matrix [A] that one could conduct Nave Gauss
Elimination forward elimination steps, one can always write it as
[A] = [L][U]
where
[L] = lower triangular matrix
[U] = upper triangular matrix

LU Decomposition

How does LU Decomposition work?
If solving a set of linear equations
If [A] = [L][U] then
Multiply by
Which gives
Remember [L]
-1
[L] = [I] which leads to
Now, if [I][U] = [U] then
Now, let
Which ends with
and
[A][X] = [C]
[L][U][X] = [C]
[L]
-1

[L]
-1
[L][U][X] = [L]
-1
[C]
[I][U][X] = [L]
-1
[C]
[U][X] = [L]
-1
[C]
[L]
-1
[C]=[Z]
[L][Z] = [C] (1)
[U][X] = [Z] (2)

LU Decomposition
How can this be used?
Given [A][X] = [C]
1. Decompose [A] into [L] and [U]
2. Solve [L][Z] = [C] for [Z]
3. Solve [U][X] = [Z] for [X]

When is LU Decomposition better
than Gaussian Elimination?
To solve [A][X] = [B]
Table. Time taken by methods

where T = clock cycle time and n = size of the matrix

So both methods are equally efficient.
Gaussian Elimination LU Decomposition
|
|
.
|
\
|
+ +
3
4
12
3
8
2
3
n
n
n
T
|
|
.
|
\
|
+ +
3
4
12
3
8
2
3
n
n
n
T

To find inverse of [A]
Time taken by Gaussian Elimination Time taken by LU Decomposition
( )
|
|
.
|
\
|
+ + =
+ =
3
4
12
3
8
| |
2
3
4
n
n
n
T
CT CT n
BS FE
|
|
.
|
\
|
+ + =
+ + =
3
20
12
3
32
| | |
2
3
n
n
n
T
CT n CT n CT
BS FS LU
n 10 100 1000 10000
CT|
inverse GE
/ CT|
inverse LU
3.28 25.83 250.8 2501
Table 1 Comparing computational times of finding inverse of a matrix using
LU decomposition and Gaussian elimination.

Method: [A] Decompose to [L] and [U]
| | | || |
(
(
(
(
(
(
= =
33
23 22
13 12 11
32 31
21
0 0
0
1
0 1
0 0 1
u
u u
u u u
U L A

[U] is the same as the coefficient matrix at the end of the forward
elimination step.
[L] is obtained using the multipliers that were used in the forward
elimination process

Finding the [U] matrix
Using the Forward Elimination Procedure of Gauss Elimination

1 12 144
1 8 64
1 5 25
(
(
(
( )
1 12 144
56 . 1 8 . 4 0
1 5 25
56 . 2 1 2 ; 56 . 2
25
64
(
(
(
= = Row Row
( )
76 . 4 8 . 16 0
56 . 1 8 . 4 0
1 5 25
76 . 5 1 3 ; 76 . 5
25
144
(
(
(

= = Row Row
Step 1:

Finding the [U] Matrix
Step 2:

(
(
(

76 . 4 8 . 16 0
56 . 1 8 . 4 0
1 5 25
( )
7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
5 . 3 2 3 ; 5 . 3
8 . 4
8 . 16
(
(
(
= =
Row Row
| |
(
(
(
=
7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
U
Matrix after Step 1:

Finding the [L] matrix
Using the multipliers used during the Forward Elimination Procedure
(
(
(
1
0 1
0 0 1
32 31
21

56 . 2
25
64
11
21
21
= = =
a
a
76 . 5
25
144
11
31
31
= = =
a
a
From the first step

of forward
elimination

1 12 144
1 8 64
1 5 25
(
(
(

Finding the [L] Matrix
| |
(
(
(
=
1 5 . 3 76 . 5
0 1 56 . 2
0 0 1
L
From the second
step of forward
elimination
(
(
(

76 . 4 8 . 16 0
56 . 1 8 . 4 0
1 5 25
5 . 3
8 . 4
8 . 16
22
32
32
=
= =
a
a

Does [L][U] = [A]?
| || | =
(
(
(

(
(
(
=
7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
1 5 . 3 76 . 5
0 1 56 . 2
0 0 1
U L
?

Using LU Decomposition to solve SLEs
Solve the following set of
linear equations using LU
Decomposition
(
(
(
=
(
(
(
(
(
(
2 279
2 177
8 106
1 12 144
1 8 64
1 5 25
3
2
1
.
.
.
x
x
x

Using the procedure for finding the [L] and [U] matrices
| | | || |
(
(
(

(
(
(
= =
7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
1 5 . 3 76 . 5
0 1 56 . 2
0 0 1
U L A

Example
Set [L][Z] = [C]

Solve for [Z]
(
(
(
=
(
(
(
(
(
(
2 . 279
2 . 177
8 . 106
1 5 . 3 76 . 5
0 1 56 . 2
0 0 1
3
2
1
z
z
z
2 . 279 5 . 3 76 . 5
2 . 177 56 . 2
10
3 2 1
2 1
1
= + +
= +
=
z z z
z z
z

Example
Complete the forward substitution to solve for [Z]
( )
( ) ( )
735 . 0
21 . 96 5 . 3 8 . 106 76 . 5 2 . 279
5 . 3 76 . 5 2 . 279
2 . 96
8 . 106 56 . 2 2 . 177
56 . 2 2 . 177
8 . 106
2 1 3
1 2
1
=
=
=
=
=
=
=
z z z
z z
z
| |
(
(
(
=
(
(
(
=
735 . 0
21 . 96
8 . 106
3
2
1
z
z
z
Z

Example
Set [U][X] = [Z]

Solve for [X] The 3 equations become

(
(
(
=
(
(
(
(
(
(

735 0
21 96
8 106

7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
3
2
1
.
.
.
x
x
x
735 . 0 7 . 0
21 . 96 56 . 1 8 . 4
8 . 106 5 25
3
3 2
3 2 1
=
=
= + +
a
a a
a a a

Example
From the 3
rd
equation
050 1
7 0
735 0
735 0 7 0
3
3
3
. a
.
.
a
. a .
=
=
=
Substituting in a
3
and using the
second equation
21 96 56 1 8 4
3 2
. a . a . =
( )
70 19
8 4
050 1 56 1 21 96
8 4
56 1 21 96
2
2
3
2
. a
.
. . .
a
.
a . .
a
=

+
=

+
=

Example
Substituting in a
3
and a
2
using
the first equation
8 106 5 25
3 2 1
. a a a = + +
Hence the Solution Vector is:
(
(
(
=
(
(
(
050 . 1
70 . 19
2900 . 0
3
2
1
a
a
a
( )
2900 0
25
050 1 70 19 5 8 106
25
5 8 106
3 2
1
.
. . .
a a .
a
=

=

=

Finding the inverse of a square matrix
The inverse [B] of a square matrix [A] is defined as

[A][B] = [I] = [B][A]

Finding the inverse of a square matrix
How can LU Decomposition be used to find the inverse?
Assume the first column of [B] to be [b
11
b
12
b
n1
]
T
Using this and the definition of matrix multiplication

First column of [B] Second column of [B]
| |
(
(
(
(
=
(
(
(
(
0
0
1
1
21
11

n
b
b
b
A | |
(
(
(
(
=
(
(
(
(
0
1
0
2
22
12

n
b
b
b
A
The remaining columns in [B] can be found in the same manner

Example: Inverse of a Matrix
Find the inverse of a square matrix [A]
| |
(
(
(
=
1 12 144
1 8 64
1 5 25
A
| | | || |
(
(
(

(
(
(
= =
7 0 0 0
56 1 8 4 0
1 5 25
1 5 3 76 5
0 1 56 2
0 0 1
.
. .
. .
. U L A
Using the decomposition procedure, the [L] and [U] matrices are found to be

Solving for the each column of [B] requires two steps
1) Solve [L] [Z] = [C] for [Z]
2) Solve [U] [X] = [Z] for [X]
Step 1:
| || | | |
(
(
(
=
(
(
(
(
(
(
=
0
0
1
1 5 . 3 76 . 5
0 1 56 . 2
0 0 1
3
2
1
z
z
z
C Z L
This generates the equations:
0 5 . 3 76 . 5
0 56 . 2
1
3 2 1
2 1
1
= + +
= +
=
z z z
z z
z

Solving for [Z]

( )
( ) ( )
2 3
56 2 5 3 1 76 5 0
5 3 76 5 0
56 2
1 56 2 0
56 2 0
1
2 1 3
1 2
1
.
. . .
z . z . z
.
.
z . z
z
=
=
=
=
=
=
=
| |
(
(
(
=
(
(
(
=
2 3
56 2
1
3
2
1
.
.
z
z
z
Z

Solving [U][X] = [Z] for [X]

(
(
(
=
(
(
(
(
(
(

3.2
2.56
1

7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
31
21
11
b
b
b
2 . 3 7 . 0
56 . 2 56 . 1 8 . 4
1 5 25
31
31 21
31 21 11
=
=
= + +
b
b b
b b b

Using Backward Substitution

( )
( )
04762 . 0
25
571 . 4 9524 . 0 5 1
25
5 1
9524 . 0
8 . 4
571 . 4 560 . 1 56 . 2
8 . 4
560 . 1 56 . 2
571 . 4
7 . 0
2 . 3
31 21
11
31
21
31
=

=

=
=
+
=

+
=
= =
b b
b
b
b
b
So the first column of
the inverse of [A] is:
(
(
(
=
(
(
(
571 . 4
9524 . 0
04762 . 0
31
21
11
b
b
b

Repeating for the second and third columns of the inverse

Second Column Third Column

(
(
(
=
(
(
(
(
(
(
0
1
0
1 12 144
1 8 64
1 5 25
32
22
12
b
b
b
(
(
(
=
(
(
(
000 . 5
417 . 1
08333 . 0
32
22
12
b
b
b
(
(
(
=
(
(
(
(
(
(
1
0
0

1 12 144
1 8 64
1 5 25
33
23
13
b
b
b
(
(
(
=
(
(
(
429 . 1
4643 . 0
03571 . 0
33
23
13
b
b
b

The inverse of [A] is

| |
(
(
(
429 . 1 000 . 5 571 . 4

4643 . 0 417 . 1 9524 . 0
03571 . 0 08333 . 0 04762 . 0
1
A
To check your work do the following operation
[A][A]
-1
= [I] = [A]
-1
[A]
Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit

http://numericalmethods.eng.usf.edu/topics/lu_decomp
osition.html

THE END


LU Decomposition

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

LU Decomposition

Uploaded by

Copyright:

Available Formats

2/4/2014 http://numericalmethods.eng.usf.

From the first step

429 . 1 000 . 5 571 . 4

You might also like