The Islamic University of Gaza

Faculty of Engineering
Civil Engineering Department

Numerical Analysis
ECIV 3306

Chapter 11
Special Matrices and Gauss-Siedel
Symmetric Matrices
Certain matrices have particular structures
that can be exploited to develop efficient
solution schemes.

(
(
(
(

=
88 6 39 16
6 9 7 2
39 7 3 1
16 2 1 5
] [ A
Symmetric matrix:

If a
ij
= a
ji
[A]
nxn
is a
symmetric matrix

Cholesky Decomposition
3
This method is suitable for only symmetric systems
where:

T
ij ji
a a and A A = =
11 12 13 11 11 21 31
21 22 23 21 22 22 32
31 32 33 31 32 33 33
0 0
0 * 0
0 0
a a a l l l l
a a a l l l l
a a a l l l l
( ( (
( ( (
=
( ( (
( ( (

11
21 22
31 32 33
0 0
[ ] 0
l
L l l
l l l
(
(
=
(
(

*
T
A L L =
Cholesky Decomposition
1
1
1
2
1
1, 2, , 1
i
ki ij kj
j
ki
ii
k
kk kk kj
j
a l l
l for i k
l
l a l

=

= =
=

1
1
1
2
1
1, 2, , 1
i
ki ij kj
j
ki
ii
k
kk kk kj
j
a l l
l for i k
l
l a l

=

= =
=

11
31
31
11
21
21
11 11
L
a
L
L
a
L
a L
=
=
=
2
31
2
32 33 33
22
31 21 32
32
2
21 22 22
L L a L
L
L L a
L
L a L
=

=
=
For 3x3
matrix,
Once the matrix L has been calculated, L
T
can easily be found and the
forward , backward substitution steps can be followed to find the
solution {X} for any right hand side {B}.
Pseudocode for Choleskys LU
Decomposition algorithm (contd)
Gauss-Siedel


Iterative or approximate methods provide an
alternative to the elimination methods. The Gauss-
Seidel method is the most commonly used iterative
method.
The system [A]{X}={B} is reshaped by solving the
first equation for x
1
, the second equation for x
2
,
and the third for x
3
, and n
th
equation for x
n
. We
will limit ourselves to a 3x3 set of equations.


Gauss-Siedel
33
2 32 1 31 3
1
22
3 23 1 21 2
2
11
3 13 2 12 1
1
a
x a x a b
x
a
x a x a b
x
a
x a x a b
x

=

=

=
3 3 33 2 32 1 31
2 3 23 2 22 1 21
1 3 13 2 12 1 11
b x a x a x a
b x a x a x a
b x a x a x a
= + +
= + +
= + +

Now we can start the solution process by choosing

guesses for the xs. A simple way to obtain initial
guesses is to assume that they are zero. These
zeros can be substituted into x
1
equation to
calculate a new x
1
=b
1
/a
11
.
Gauss-Siedel

New x
1
is substituted to calculate x
2
and x
3
. The
procedure is repeated until the convergence
criterion is satisfied:

s
new
i
old
i
new
i
i a
x
x x
c c <

= % 100
,
Gauss Seidal-
Graphical Interpretation
Jacobi iteration Method
An alternative approach, called Jacobi iteration,
utilizes a somewhat different technique. This
technique includes computing a set of new xs on the
basis of a set of old xs. Thus, as the new values are
generated, they are not immediately used but are
retained for the next iteration.

Gauss-Siedel
The Gauss-Seidel method The Jacobi iteration method
Convergence Criterion for Gauss-Seidel Method
The gauss-siedel method is similar to the technique of
fixed-point iteration.
The Gauss-Seidel method has two fundamental
problems as any iterative method:
1. It is sometimes non-convergent, and
2. If it converges, converges very slowly.
Sufficient conditions for convergence of two linear
equations, u(x,y) and v(x,y) are:

1
1
<
c
c
+
c
c
<
c
c
+
c
c
y
v
x
v
y
u
x
u
Convergence Criterion for Gauss-Seidel
Method (contd)
Similarly, in case of two simultaneous equations, the
Gauss-Seidel algorithm can be expressed as:

1 12
1 2 2
11 11
2 21
1 2 1
22 22
12
1 2 11
21
1 22 2
( , )
( , )
0
0
b a
u x x x
a a
b a
v x x x
a a
a u u
x x a
a v v
x a x
=
=
c c
= =
c c
c c
= =
c c
Convergence Criterion for Gauss-Seidel
Method (contd)
Substitution into convergence criterion of two linear
equations yield:

In other words, the absolute values of the slopes must be
less than unity for convergence:



That is, the diagonal element must be greater than the sum
of off-diagonal element for each row.

1 , 1
22
21
11
12
< <
a
a
a
a
21 22
12 11
a a
a a
>
>

=
=
>
n
i j
j
j i ii
a a
1
,
equations n For
Gauss-Siedel Method- Example 1
10
x 2 0 x 3 0 4 71
7
x 3 0 x 1 0 3 19
3
x 2 0 x 1 0 85 7
x
x
x
4 71
3 19
85 7
x
x
x
10 2 0 3 0
3 0 7 1 0
2 0 1 0 3
2 1
3 1
3 2
3
2
1
3
2
1
. . .
. . .
. . .
.
.
.
. .
. .
. .
+
+
+ +
=
=
=

(
(
(

Guess x
1
,

x
2
,

x
3
= zero for the first guess
Iter. x
1
x
2
x
3
|c
a,1
|(%)

|c
a,2
|

(%) |c
a,3
| (%)
0 0 0 0 - - -
1 2.6167 -2.7945 7.005610 100 100 100
2 2.990557 -2.499625 7.000291 12.5 11.8 0.076
Improvement of Convergence Using
Relaxation
Where is a weighting factor that is assigned a
value between [0, 2]
If = 1 the method is unmodified.
If is between 0 and 1 (under relaxation) this is
employed to make a non convergent system to
converge.
If is between 1 and 2 (over relaxation) this is
employed to accelerate the convergence.

( )
1
new new old
i i i
x x x = +
Gauss-Siedel Method- Example 2
The system can be re-arranged to the following

6x
1
-x
2
-x
3
= 3
6x
1
+9x
2
+x
3
=40
-3x
1
+x
2
+12x
3
=50

Gauss-Siedel Method- Example 2
12
3 50
9
6 40
6
3
2 1
3
3 1
2
3 2
1
x x
x
x x
x
x x
x
+
=

=
+ +
=
Without Relaxation
Gauss-Siedel Method- Example 2