NUMERICAL METHODS

Linear Algebraic Equations

A system of linear algebraic equations has the form:
A11 x1 A12 x2 ....... A1n xn b1

A21 x1 A22 x2 ....... A2 n xn b2

A31 x1 A32 x2 ....... A3n xn b3

. . +........+ . =.
. . +........+ . =.
. . +........+ . =.
An1 x1 An 2 x2 ....... Ann xn bn

where the coefficients Aij and the constants bj are known, and xi represent the unknowns.
In matrix notation the equations are written as
Ax b
Linear, algebraic equations occur in almost all branches of engineering. But their most visible
application in engineering is in the analysis of linear systems (any system whose response is
proportional to the input is deemed to be linear). Linear systems include structures, elastic
solids, heat flow, seepage of fluids, electromagnetic fields and electric circuits; i.e., most topics
taught in an engineering curriculum.
If the system is discrete, such as a truss or an electric circuit, then its analysis leads directly to
linear algebraic equations. In the case of a statically determinate truss, for example, the
equations arise when the equilibrium conditions of the joints are written down. The unknowns
x1, x2, . . . , xn represent the forces in the members and the support reactions, and the constants
b1, b2, . . . , bn are the prescribed external loads.
The behaviour of continuous systems is described by differential equations, rather than
algebraic equations. However, because numerical analysis can deal only with discrete
variables, it is first necessary to approximate a differential equation with a system of algebraic
equations. The well-known finite difference, finite element and boundary element methods of
analysis work in this manner. They use different approximations to achieve the
discretization, but in each case the final task is the same: solve a system (often a very large
system) of linear, algebraic equations.
In summary, the modelling of linear systems invariably gives rise to equations of the form
Ax = b,
where b is the input and x represents the response of the system. The coefficient matrix A,
which reflects the characteristics of the system, is independent of the input. In other words, if
the input is changed, the equations have to be solved again with a different b, but the same A.
Therefore, it is desirable to have an equation-solving algorithm that can handle any number of
constant vectors with minimal computational effort.
SOLVING SYSTEMS OF LINEAR ALGEBRAIC EQUATIONS
There are two classes of methods for solving systems of linear, algebraic equations: direct and
iterative or indirect methods. The common characteristic of direct methods is that they
transform the original equations into equivalent equations (equations that have the same
solution) that can be solved more easily. The transformation is carried out by applying the three
operations listed below. These so-called elementary operations do not change the solution, but
they may affect the determinant of the coefficient matrix as indicated in parentheses.

Elementary Operations
Exchanging two equations (changes sign of |A|).
Multiplying an equation by a nonzero constant (multiplies |A| by the same constant).
Multiplying an equation by a nonzero constant and then subtracting it from another
equation (leaves |A| unchanged).

Iterative, or indirect methods, start with a guess of the solution x, and then repeatedly refine
the solution until a certain convergence criterion is reached. Iterative methods are generally
less efficient than their direct counterparts due to the large number of iterations required. But
they do have significant computational advantages if the coefficient matrix is very large and
sparsely populated (most coefficients are zero).
Direct Methods
Well consider 3 direct methods, viz: Gauss elimination, GaussJordan elimination and LU
decomposition.
Iterative Methods
Well consider 2 iterative methods, viz: GaussSeidel Method and Conjugate Gradient Method.

GAUSS ELIMINATION
This method has an Initial form of Ax = b, and a Final form of Ux = c; where U represents an
upper triangular matrix as shown below

u11 u12 u13

U 0 u 22 u 23
0 0 u 33
Gauss elimination is a very familiar method for solving simultaneous equations. It consists of
two parts: the elimination phase and the solution/back substitution phase. The function of the
elimination phase is to transform the equations into the form Ux = c. The equations are then
solved by back substitution.
EXAMPLE 1: Using Gauss elimination method, solve equations (1, 2 and 3) for x1 , x 2 and x3 .
4 x1 2 x2 x3 11 ............................................................. 1
2 x1 4 x2 2 x3 16 ..................................................... 2
x1 2 x 2 4 x3 17
............................................................. 3
The elimination phase
Recall the elementary operations above; the elimination phase entails multiplying one equation
(known as the pivot equation) by a constant, and subtracting it from another equation.

Let Eq. (1) be our pivot equation and lets choose a multiplier so as to eliminate x1 from Eqs.
(2) and (3):

Hence Eq. (2) Eq. (2) [( 1/2) Eq. (1)]; Eq. (3) Eq. (3) [1/4 Eq. (1)].
The equations changes to:
4 x1 2 x2 x3 11 .............................................................. 1a
3x2 1.5 x3 10.5 ............................................................. 2a
1.5 x2 3.75x3 14.25 ......................................................3a
We now let Eq. (2a) be our pivot equation and choose a multiplier so as to eliminate x2 from
Eqs. (3a):
Hence Eq. (3a) Eq. (3a) [( 1/2) Eq. (2a)].
The equations then becomes:
4 x1 2 x2 x3 11 .............................................................. 1b
3x2 1.5 x3 10.5 ............................................................. 2b
3x3 9 ................................................................................ 3b
The solution or back substitution phase
The unknowns can now be computed by back substitution. Solving Eqs. (3b), (2b) and (1b) in
that order, we get
9
x3 3;
3
from (2b), we substitute to get x 2 thus:
(10.5 1.5x3 ) (10.5 1.5 3)
3x2 1.5x3 10.5; hence x2 2 ;
3 3
similarly, we substitute the values of x3 and x 2 into (1b) to get x1
11 3 (2 2)
4 x1 2 x 2 x3 11, therefore x1 1
4
EXERCISES
Using Gauss elimination method, solve the equations for x1 , x 2 and x3 .
x1 6 x 2 4 x3 8
2 x1 8 x2 2 x3 12 Ans: x1 = -0.8571; x 2 = -1.8571; x3 = -0.5714
14x1 10x 2 8 x3 2

Using Gauss elimination method, solve the equations for x1 , x 2 and x3 .

x1 8 x 2 2 x3 4
4 x1 6 x2 12x3 4 Ans: x1 = 1.3529; x 2 = -0.4118; x3 = -0.3235
8 x1 6 x 2 4 x3 12

Using Gauss elimination method, solve the equations for x1 , x 2 and x3 .

4 x1 2 x2 x3 11
2 x1 4 x2 2 x3 16 Ans: x1 = -0.2; x 2 = -5; x3 = 1.8
x1 2 x 2 4 x3 17

Remember to express these equations in matrix form for better understanding, then relate it to
the multiple variable equation as shown next.
EXAMPLE 2: Using Gauss elimination method, solve the equations represented by the matrix

6 4 1 x11 x12 14 22
4 6 4 x x 22 = 36 18
21
1 4 6 x31 x32 6 7

The above matrix can be broken down into 2 sets of 3simultaneous equations with 6 variables
as shown:
6 x11 4 x21 x31 14 6 x12 4 x22 x32 22
4 x11 6 x21 4 x31 36 and 4 x12 6 x22 4 x32 18
x11 4 x 21 6 x31 6 x12 4 x22 6 x32 7
When solved correctly using the Gauss elimination method described earlier, well get
x11 =10.0000; x 21 =22.0000; x 31 =14.0000. row 2 row 2 + (2/3) row 1
row 3 row 3 (1/6) row 1
x12 =3.0000 ; x 22 = -1.0000 ; x 32 =0.0000 . finally
row 3 row 3 + row 2

ALGORITHM FOR GAUSS ELIMINATION METHOD

The following algorithm is a matlab program for solving Example 1 using
Gauss elimination method.
function [x,det] = gauss(A,b)
A=[4,-2,1;-2,4,-2;1,-2,4]
b=[11;-16;17]
if size(b,2) > 1; b = b'; end % b must be column vector
n = length(b);
for k = 1:n-1
for i= k+1:n
if A(i,k)= 0
lambda = A(i,k)/A(k,k);
A(i,k+1:n) = A(i,k+1:n)- lambda*A(k,k+1:n);
b(i)= b(i)-lambda*b(k);
end
end
end
if nargout == 2; det = prod(diag(A)); end
for k = n-1:1
b(k) = (b(k) - A(k,k+1:n)*b(k+1:n))/A(k,k);
end
x=b;

LU DECOMPOSITION.
This means lower and upper triangular matrix decomposition method. We all know that any
square matrix A can be expressed as a product of a lower L and an upper U triangular matrix:
A = LU .......................................................... 4
The method entails decomposing the matrix A into a lower and upper matrix, such that the
general system of linear algebraic equations Ax = b, becomes:
LUx = b ....................................................... 5
The norm is that we assign a value for Ux, hence we assume that
Ux = y .......................................................... 6
therefore equation (5) becomes:
Ly = b ......................................................... 7
When the value of y is obtained from (7), we can easily get the unknown x from equation (6).

LU decomposition, also known as LU factorization is the process of computing L and U

for a given matrix A. There are three commonly used decomposition methods, viz:
i. Doolittles decomposition
ii. Choleskis decomposition
iii. Crouts decomposition
but well only consider the first two methods.

Doolittles Decomposition Method

Doolittles decomposition is closely related to Gauss elimination method, and it has two phases
the Decomposition and Solution phase.
The general principle is that a lower L and an upper U triangular matrix is formed out of A
such that A = LU as in equation (4).
1 0 0 u11 u12 u13

L L21 1 0 U 0 u 22 u 23
L31 L32 1 0 0 u 33

How is this decomposition done?

From the foregoing, equation (4) becomes:

u11 u12 u13

A u11 L21 u12 L21 u 22 u13 L21 u 23 .............................................................. 8

u11 L31 u12 L31 u 22 L32 u13 L31 u 23 L32 u 33

In other words, equation 8 is assumed to be the actual matrix A.

When Gauss elimination is applied to (8),
i.e. (row 2 = row 2 L21 row 1 and row 3 = row 3 L31 row 1), we have:

u11 u12
u13

A 0 u 22 .............................................................................................. 9
u 23

0 u 22 L32 u 23 L32 u 33
Considering the second row as the pivot row, and still applying the operation:
row 3 = row 3 L32 row 2 changes (9) to:

u11 u12 u13

A 0 u 22 u 23 .............................................................................................................. 10
0 0 u 33
The explanations given above reveals two important features of Doolittles decomposition:
The matrix U is identical to the upper triangular matrix that results from Gauss
elimination.
 The off-diagonal elements of L are the pivot equation multipliers used during Gauss
elimination.
Normally, the multipliers are stored in the lower triangular portion of the coefficient matrix,
replacing the coefficients as they are eliminated. The diagonal elements of L do not have to be
stored, since it is understood that each of them is unity. The final form of the coefficient matrix
would thus be:

u11 u12 u13

L / U L21 u 22 u 23 ........................................................................................................ 11
L31 L32 u 33

EXAMPLE 3: Use Doolittles LU decomposition method to solve the following equations for
x1 , x 2 and x3 .

x1 4 x2 x3 7 ................................................................. E1
x1 6 x 2 x3 13 ............................................................... E2
2 x1 x 2 2 x3 5 .............................................................. E3
To decompose the equations, we proceed thus:
First express the equations in the form of Ax = b so that the matrices will be clearly shown.
Hence we have:

1 4 1 x1 7
1 6 1 x = 13 ................................................... E4
2
2 1 2 x3 5

Considering row 1 as our pivot equation, we carry out the operation:

row 2 = row 2 1 row 1 and row 3 = row 3 2 row 1; noting well the multipliers, 1 and
2, which are to be stored.
After the first operation, the matrix A in E4 becomes:

1 4 1
0 2 2 .................................................................... E5

0 9 0

Now considering row 2 as our pivot equation, and carrying out the operation:
row 3 = row 3 (4.5) row 2; E5 becomes:

1 4 1
0 2 2 ........................................................................ E6

0 0 9
As can be seen, the elements of the lower triangular portion of E6 has all been reduced to zero
(0) using the following multipliers: 1, 2 and -4.5; hence E6 is the upper triangular matrix that
results from Gauss elimination.
At this point, we now have to complete our L and U triangular matrices thus:
 1 0 0 1 4 1

L 1 1 0 while U 0 2 2

2 4.5 1 0 0 9

1 0 0 y1 7

From equation 7, we have: Ly = b; 1 1 0 y 2 = 13 ....................................... E7
2 4.5 1 y 3 5

The following can be deduced from E7:

y1 7 .................................................................................. E8

y1 y 2 13 ........................................................................ E9

2 y1 4.5 y 2 y3 5 ......................................................... E10

Using forward substitution from E8 into E9 and E10 reveals that: y1 7 , y 2 6 and y3 18 .

The values of the unknown, x, can easily be calculated from equation 6 thus:

1 4 1 x1 7
Ux = y 0 2 2 x 2 = 6 ............................................. E11
0 0 9 x3 18

From E11, we have:

9 x3 18 ......................................................................... E12

2 x2 2 x3 6 .................................................................... E13

x1 4 x2 x3 7 ............................................................... E14

Using forward substitution, we arrive at the solution of equations E1, E2 and E3; this shows
us that x1 5 , x2 1 and x3 2 .

ALGORITHM FOR DOOLITTLES LU DECOMPOSITION METHOD

The following algorithm is a matlab program for solving Example 3 using
Doolittles LU decomposition method.
function A = LUdec(A)
% LU decomposition of matrix A; returns A = [L\U].
% USAGE: A = LUdec(A)
A = [3 -1 4; -2 0 5; 7 2 -2]
b = [-4;2;-5]
n = size(A,1);
for k = 1:n-1
for i = k+1:n
if A(i,k) = 0.0
lambda = A(i,k)/A(k,k);
A(i,k+1:n) = A(i,k+1:n) - lambda*A(k,k+1:n);
A(i,k) = lambda;
end
end
end
if size(b,2) > 1; b = b'; end
n = length(b);
for k = 2:n
b(k) = b(k) - A(k,1:k-1)*b(1:k-1)
end
for k = n-1:1
b(k) = (b(k) - A(k,k+1:n)*b(k+1:n))/A(k,k);
end
x = b

NEWTON-RAPHSON AND SECANT METHODS

These two methods are related and only differ in the last stage. The NewtonRaphson method
is the best-known method of finding roots for a good reason: it is simple and fast. The only
drawback of the method is that it uses the derivative f '(x) of the function as well as the function
f (x) itself. Therefore, the NewtonRaphson method is usable only in problems where f '(x) can
be readily computed.
The NewtonRaphson formula can be derived from the Taylor series expansion of f (x). We
assume the current guess is x and this is incorrect by an amount h, so that x + h is the required
value. It now remains for us to determine h or at least find an approximation to it. The Taylor
expansion for the function f(x) at the point x + h is given by:
f(x + h) = f(x) + h f ' (x) + 0(h2) ........................... 12
This can be interpreted as: the value of the function at x + h is equal to the value of the function
at x plus the product of gradient and the distance between the points.
If x+h is a root of the function f (x), then f(x + h) = 0, Eq. (12) becomes
0 = f(x) + h f ' (x) + 0(h2) ........................................ 13
hence h will approximately be:
()
= (x) ............................................................ 14
( )
This allows us to construct the iterative scheme +1 = ( ) ..................................... 15

where n = 0,1,2,3...... .
The algorithm for the NewtonRaphson method is simple: it repeatedly applies Eq. (15),
starting with an initial value x0, until the convergence criterion |+1 | < is reached,
where is the error tolerance. Only the latest value of x has will be stored.
Here is the algorithm:
1. Let x be a guess for the root of f (x) = 0.
()
2. Compute = (x).
3. Let x + and repeat steps 2-3 until <.
Although the NewtonRaphson method converges fast near the root, its global convergence
characteristics are poor.
EXAMPLE 4: A root of f (x) = x3 10x2 + 5 = 0 lies close to x = 0.7. Compute this root with
the NewtonRaphson method, up to the second iteration.
Solution
The derivative of the function is f ' (x) = 3x2 20 x, so that the NewtonRaphson formula in
Eq. (15) is
 () 3 10 2 +5 2 3 10 2 5
(x)
= ; expanding this gives: .
3 2 20 (320)
Now we are set for the iterations.
First iteration
2(0.7)3 10(0.7)2 5
= 0.73536
0.7(3(0.7)20)

Second iteration:
Here you now use 0.73536 as your new value of ; hence
2(0.73536)3 10(0.73536)2 5
= 0.73460
0.73536(3(0.73536)20)

PRACTICE: The function f (x) = 4x4 2x3 18x2 +8x+60 = 0 has 4 real roots. One of the roots
lies close to x = 3. Follow the method shown in example 4 to compute for this root up to the
fourth iteration.
The actual 4 real roots are: 2, -3, -2, 5.

EXERCISE
1. Using the 3 steps outlined in the algorithm above, determine the zero (root) of the
function f(x) = ex e2x + 1; with an initial guess of 2, and state the iteration number
at which the root was got. Answer = -0.281199; 5th iteration
2. This quintic equation f(x) = x5 x4 + x3 +2x2 1 has five roots. Use NewtonRaphson
method to calculate for the root that has no imaginary component.
3. Determine the two roots of sin(x) + 3 cos(x) 2 = 0 that lie in the interval (2, 2), using
the NewtonRaphson method.
ALGORITHM FOR NEWTON-RAPHSONS METHOD
The following algorithm is a matlab program for solving problems using
Newton-Raphson method.
Here you need to create two function files; the actual function file and the derivative file.
function [f] = func1(x)
f = x^4-2*x^3-18*x^2+8*x+60;

%
function [value] = func1_prime(x)
value = 4*x^3-6*x^2-36*x+8;

%
% Newton_Raphson.m
%
x = input('Starting guess :');
tolerance = 1e-8;
iterations = 0;
while (iterations<30) & (abs(func(x))>tolerance)
x = x-func(x)/func_prime(x);
iterations = iterations + 1;
end
if iterations==30
disp('No root found')
else
disp([' Root = ' num2str(x,10) ' found in ' ...
int2str(iterations) ' iterations.'])
end
SOLVE AND KEEP:
1. Using the NewtonRaphson method calculate the roots of the functions f(x) = x cos(x)
sin x and g(x) = (x3 x) sin(x); take 2 as your initial starting guess.
TIP: These should be done as two separate calculations. You will need to write two
MATLAB functions for each case to evaluate the function and its derivative.
SECANT
The secant method requires two initial approximations 0 and 1 , with both being reasonably
close to the solution . From 0 and 1 we can show that the points (0 , 0 = (0 )) and (1 ,
1 = (0 )) both lie on the graph of . Connecting these points gives the (secant) line ratio
1 1 0
= ............................................... 16
(1 ) 1 0
this can be re-written as shown in equation (17) thus:
1 0
1 = ( 1 ) ................................ 17
1 0

Since the function (0 ) must be = 0, we set y = 0, solve for x, and use that as our next
approximation; (i.e. set y to zero, convert all 1 to , and all zero to 1). Repeating this process
gives us the iteration:
1
+1 = ( )
1
where = ( )

EXAMPLE 5: Given (x) = 4 5, which has a solution 1.5; compute for the root after
the 4th iteration, using the secant method.
SOLUTION: First of all, Choose 0 = 1 and 1 = 2 as initial approximations. When 0 is put
in the function (x), we have: (x0) = 0 = (1)4 - 5 = -4; and when 1 = 2 is put in (x), we
have: (x1) = 1 = (2)4 5 = 11.
First iteration:
1 0
2 = 1 ( )
1 0 1

21 19
2 = 2 (11) = 1.26667.
11(4) 15
Second iteration:
Our equation now changes to:
2 1
3 = 2 ( )
2 1 2
When we put 2 = 1.26667 into the function (x), we have: 2 = 1.266674 5 = -2.4257; and
we still have to use the values of 1 and 1 .
therefore
1.266672
3 = 1.26667 (2.4257) = 1.3992
2.425711
Third iteration:
Our equation now changes to:
 3 2
4 = 3 ( )
3 2 3
When we put 3 = 1.3992 into the function (x) = 4 5, we have: 3 = 1.39924 5 = -1.1672;
and we still have to use the values of 2 and 2 .
therefore
1.39921.26667
4 = 1.3992 (1.1672) = 1.5221
1.1672(2.4257)
Finish up the fourth iteration and keep the result.
Answer = 1.4927
EXAMPLE 6: Using the secant method, determine the root of the function f(x) = ex e2x + 1,
after the 5th iteration; with initial guesses of 1 and 2.
SOLUTION:
When you follow the steps detailed in example 5, youll get the following:
First iteration: = 0.2516
Second iteration: = -0.1879
Third iteration: = -0.3130
Fourth iteration: =-0.2790
Fifth iteration: = -0.2811

ALGORITHM FOR SECANT METHOD

The following algorithm is a matlab program for solving problems using
Secant method.
function x = mysecant (f,x0 ,x1 ,n)
% Solves f(x) = 0 by doing n steps of the secant method
% starting with x0 and x1.
% Inputs: f -- the function
% x0 -- starting guess, a number
% x1 -- second starting guess
% n -- the number of steps to do
% Output: x -- the approximate solution
y0 = f(x0);
y1 = f(x1);
for i = 1:n % Do n times
x = x1 - (x1 -x0)* y1 / (y1 -y0) % secant formula.
y=f(x) % y value at the new approximate solution.
% Move numbers to get ready for the next step
x0=x1;
y0=y1;
x1=x;
y1=y;
end

Clue: You must key-in the function (f) terms on your own.
 UNIVERSITY OF ABUJA
FACULTY OF ENGINEERING
DEPARTMENT OF ELECTRICAL/ELECTRONICS
ENGINEERING
FIRST SEMESTER EXAMINATION
EEE712 (Numerical Methods and Computer Programming) 3 Units Time: 3hrs
INSTRUCTIONS: ATTEMPT ALL QUESTIONS.

1. Write an algorithm on how a postgraduate student will pass his

or her exams; (please remember to include his or her
responsibilities as a father or mother) and draw a flowchart to
represent your algorithm.
b Write an algorithm on how you drive to school from your house
or office; (include all reasonable decisions that influence your
movement). Present your algorithm in the form of a flowchart.
2. The circuit below was analysed in the class.

i 2 r2 r1 i1
+

r3 r5 200 V
i3 i5
-
i4 r4 r6 i6

Write an algorithm on how to determine the six branch currents,

with directions as given, using GAUSS elimination method.
Please ensure complete and strict adherence to the two laws
considered in the class.
3. Solve for x in the tri-diagonal equations Ax = b by Doolittles
decomposition method, where
6 2 0 0 0 2
1 7 2 0 0 3

A= 0 2 8 2 0 , and b= 4

0 0 3 7 2 3
0 0 0 3 5 1

4. If the stability index of the Nigerian electric power system is

2 x
determined by the function f ( x) 4e 4 x 2e x 1
x
, while the upper and
3e e 2
lower values of the stability index for the year 2017 are 1 and 2
respectively, give a forecast of what the stability value will be in
the year2029, if an iteration represents the stability index for
8760 hours.
5. In a warfare between Nigeria and the United States, the Nigerian
military fired a missile at an elevation angle of 450 to the vertical,
with a vertical inclination speed of 19.2km/s. If the missile trails
t

a path given by the function, f ( x) 4800(1 e ) 320t , calculate10

the time it will take the missile to launch an attack in US; assume
a take-off time of 8secs.

Youll submit your exam and any solve and keep questions in 2 weeks time, i.e. on june 27th.

Good luck