Numerical Methods Overview of The Course

ST4291 Numerical Analysis
(2cp, MSc Program International)

1.
OVERVIEW OF THE COURSE
PART I - ALGEBRAIC EQUATIONS

(Week 1: 16-20 October 2000)
1.
2.
Solution of Linear Algebraic Equations (LAE)
Direct Methods: Gaussian Elimination and Its Variants
Iterative Methods: Jacobi Method; Gauss-Seidel Method; Successive Over Relaxation

(SOR) Method.
Numerical Linear Algebra: (Convergence & Existence of Solution)

1. LU & QR Factorisation;
2. Eigenvalues and Eigenvectors.
Solution of Nonlinear Algebraic Equations (NAE)
Variables: Single and Multivariable;
Methods Different in approximating a function

1. Subdividing interval: Bisection Method
2. A straight-line approximation: Regula Falsi; Secant; Newton Methods
3. A quadratic approximation: Muller Method
Other Methods (for several variables)

1. Fixed point Iteration
2. Minimisation
PART II - INTEGRATION AND DIFFERENTIATION

(Week 2 : 23-25 October)
1.
Basic Techniques
Differentiation: Use difference formula
Integration: Newton-Cotes Formulas, like Trapezoid, Simpson, Mid-point Rules;

Gaussian Quadrature.
Linear Analysis and Matrices
LI - 1

2.
3.
Solution of (Systems) Ordinary Differential Equations (ODE)
Type 1 (Initial Value Problems): Taylor Methods; Runge-Kutta Methods; Multi-step

Methods
Type 2 (Boundary Value Problems: Shooting Methods; Finite Difference Methods

(Application of numerical methods for LAEs and NAEs).
Solution of Partial Differential Equations (PDE)
Type of PDE:
1. Parabolic (Heat Equation);
2. Hyperbolic (Wave Equation) and
3. Elliptic (Poisson Equation).
Methods:
1. Finite Difference Methods (Parabolic and Hyperbolic)
2. Finite Element Methods (Elliptic)
2.
NUMERICAL TECHNIQUES
Solving of Engineering Problems

1.
Analytically solved Exact Solution!
2.
Numerically Solved Approximate Solution!
Issues:
1.
2.
3.
Techniques Direct or Iterative

Computational effort
Accuracy of the resulting solution
Techniques
Direct
A direct method, such as Gaussian Elimination, produces an answer to a problem in a fixed
number of computational steps.
Iterative
LI - 2

An iterative method produces a sequence of approximate answer (designed to converge
ever closer to the true solution under the proper condition). Convergence issues!
Typical Problems
Finding Roots
y = x2 3 = 0
Solving a set of equations
3x + 2 y = 4
2x + y = 1
Approximating an integration
3
I =
1
1
dx
x3
Computational effort
Size of the problem

Algorithm
Accuracy of the resulting solution
Numerical Round-off errors

Errors come from inexact arithmetic due to the shortening a number that has more digits
than can be represented by the available floating points ( N
= .d1 d 2 d 3 K d p B e ).
Example: We have three-digit arithmetic

(0.99+0.0044) + 0.0042 = 0.994+0.0042 = 0.998
0.99 + (0.0044 + 0.0042) = 0.99 + 0.0086 = 0.999
Truncation Errors
Errors come from mathematical approximation (approximating a function by a simpler
function)
Example:
a+h
f ( x)dx 2 [ f (a) + f (a + h)]

a
Taylor approximation:
LI - 3

f (a + h) = f (a ) + hf (a ) +
h2
h3
f (a) +
f (c1 ) + L
2!
3!
h
h2
h3
[ f (a + h) + f (a )] = hf (a ) +
f (a) +
f (c 2 ) + L
2
2
4
LI - 4

3.
EQUATIONS
An expression of equality between constants, variables, functions of variables (sums, products,

derivatives or integral of variables)
Type of Equations
Linear
An equation is linear in
Example:
x if it contains x to only the first power.
y = ax + b
y = a 2 x + b (Which is linear in x )
Algebraic
An equation involves a finite number of algebraic-operation on
Example:
x.
f ( x) = 3x 3 + x 2 2 x
f ( x) = 3 x 3 + x 2 2 x = 0
Homogeneous
3x 2 y 8z = 0
x + 5 y 2 z = 0 A coupled linear homogeneous

7 x + 7 y + 7 z = 0
Transcendental
An equation involves not a finite number of algebraic-operation on x.
Example:
sin x; cos x; tan x

sin x cos y xy = 0
x
x
tan( ) = 0 A coupled transcendental set
y
y
2
sin y x = 0
Differential and Integration
Example:
dy
= f ( x, y );
dt
b
I ( x) = e x dx
2
Simultaneous Equations
Consistent, independent equations
Not contradictory and not simply multiplies of one
Example:
3x + 2 y = 4
2x + y = 1
Unique solution!
LI - 5

Inconsistent equations
Contradictory
2x + y = 2
Example:
No solution!
2x + y = 1
Dependent equations
Multiplies of one
4x + 2 y = 4
Example:
Many solutions!
2x + y = 2
Ill-conditioned equations
A small change in any of the coefficients will change the solution set from unique
to either infinite or empty.
(4 + ) x + 2 y = 4
Example:
2x + y = 1
y = 4 +
Exercises
Show that one of the equations is unnecessary (and which one?)
2 x1 + x 2 + x3 = 3
x1
+ x3 = 1
x1 + 2 x 2 + 2 x3 = 2
x 2 x3 = 1
Then, solve for
x1 , x 2 , x3 .
4.
MATRIX PROPERTIES AND ALGEBRA
Why we need to review matrix?
Simplification
3x + 2 y = 4
2x + y = 1
3 2 x 4
=
2 1 y 1
r r
Ax = b
Matlab is a matrix-based computational tool.
Special Matrix Characteristics
LI - 6

Square matrix
The number of row equals to the number of columns
Column vector
It has only one column
Row vector
It has only one row
Diagonal matrix
The elements
a ij of the square matrix A are zero if i j .
Block diagonal matrix

The elements
a ij of the square matrix A are zero except for the principal diagonal
aii , and a few nearby parallel diagonal.

Triangular matrix
A matrix is upper (or lower) triangular if
Symmetric matrix
A square matrix is symmetric if
a ij = 0 for all i > j (or i < j ).
a ij = a ji for all i, j .
Identity matrix
A square matrix with 1s on the principal diagonal and 0s elsewhere.
Commuting matrix
Two matrices commute if AB = BA .
Matrix transpose
A (or AT ) corresponds to interchanging the rows and columns of

= [ A] ji .
The transpose of
[ ]
A (i.e. AT
ij
Matrix Algebra
Equality
A = B is to be valid if and only if a ij = bij for all i, j .

Multiplication
C= AB means that
cij = k =1 aik bkj .

n
Trace of a matrix
Tr ( A) = aii
Dot product
rr r r n
x. y = x T y = x i y i
i =1
r
rr
r
x. y = 0 , then x and y are orthogonal vectors
LI - 7

Norm of a vector
The norm (or length) of a vector x is defined as

Unit vectors
r r
x T .x .
A vector of length one

Orthogonal unit vectors are called orthonormal vectors
Exercises
( A + B ) T = AT + B T .
Show that Tr ( ABC ) = Tr (CBA) .
r r r r
Show that x + y x + y .
Show that
LI - 8

LINEAR ALGEBRAIC EQUATIONS LAE
What is LAE Problem?

Given:
x + 2 y + 3z = 1
2 x + 6 y + 10 z = 0
3 x + 14 y + 28 z = 8
Solve for
x, y and z
LAE:
Linear in variables
Algebraic relations
Involving system equations
How to solve it?
Direct methods Gaussian Elimination

Iterative methods (for large, sparse systems) Jacobi; Gauss-Seidel and Successive Over
Relaxation (SOR)
Applications
Steady state material balances

Equilibrium heat distribution in plate
Balancing the reactants in a chemical reactions
Etc.
LAE Direct Methods

Concepts
1.
2.
Forward elimination If two equations have a point in common, then that point also satisfies
any linear combination of the two equations.
Back substitution
LI - 9

Procedure
1.
Add a multiple
m of row Ri onto row R j to form a new row R j
R j mRi + R j
2.
3.
Repeat (1) until we get an upper (or lower) triangular matrix of LAE coefficients
Apply back substitution to solve for each variable.
Example:
x + 2 y + 3z = 1
2 x + 6 y + 10 z = 0
3 x + 14 y + 28 z = 8
or in a compact form
Ax = b
1 2 3
1
x

where A = 2 6 10 ; b = 0 ; x = y

3 14 28
8
z
Solve for
x.
1 2 3 1
Define an augmented matrix [ A b] = 2 6 10

0 , then apply Gaussian Elimination to
3 14 28 8
the augmented matrix.
Gaussian Elimination
Step 1:
a11 = 1 as pivot element and row 1 as pivot row.

Multiply the 1st equation by 2 and add it to the 2nd equation.
Multiply the 1st equation by 3 and add it to the 3rd equation.
Lets take
We get
1
1 2 3
0 2 4 2
0 8 19 11
Step2:
a 22 = 2 as pivot element and row 2 as pivot row.

Multiply the 2nd equation by 4 and add it to the 3rd equation
Lets take
LI - 10

We get
1 2 3 1
0 2 4 2
0 0 3 3
So, we have an 3 3 upper triangular matrix.
Step 3:
Apply back substitution
3 z = 3
z = 1
2 y + 4(1) = 2 y = 1
x + 2(1) + 3(1) = 1 x = 2
Hence, the solutions are x = 2
1 1] .
T
Matlabs code:
function x = Gauss(A,b),
%
% Inputs:
% A is the n by n coefficient matrix
% b is the n by k right hand side matrix
%
% Outputs:
% x is the n by k solution matrix
%
[n,k1]=size(A); [n1,k] = size(b); x = zeros(n,k);
for i=1:n-1,
m=-A(i+1:n,i)/A(i,i);
A(i+1:n,:) = A(i+1:n,:) + m*A(i,:);
b(i+1:n,:) = b(i+1:n,:) + m*b(i,:);
end;
x(n,;) =b(n,:) ./A(n,n);
for i=n-1:-1:1,
x(i,:) =(b(i,:)-A(i,i+1:n)*x(i+1:n,:))./A(i,i);
end;
LI - 11

Analysis of the Method
Why a linear combination of two equations passes through the point of intersection of the two
equations
Suppose
r = [r1 , r2 ,K, rn ] satisfies
S : ao + a1 x1 + a 2 x 2 + L + a n x n = 0
T : bo + b1 x1 + b2 x 2 + L + bn x n = 0
then r also satisfies
m1 S + m2T , or
m1 a o + m1a1 x1 + a 2 x 2 + L + m1 a n x n + m2 bo + m2 b1 x1 + m2 b2 x 2 + L + m2 bn x n = 0
When the Gauss elimination can produce systems to be solved by back substitution
Ax = b
x = A 1b
This implies that:
When inverse of A exists

When equations are consistent and independent
rank ( A) = dim( A)
Problems with this method

When pivot element equals zero or significantly smaller than the coefficient being used to
eliminate
Example:
0.001 x1 + x 2 = 3
x1 + 2 x 2 = 5
LI - 12

Approach 1:
Choose row 1 as pivoting row and apply Gauss elimination procedure
x 2 = 3; x1 = 0 , which is inaccurate result.
Approach 2:
Choose row 2 as pivoting row and apply Gauss elimination procedure
x 2 = 3; x1 = 1 , which is accurate result.
Tri-diagonal systems
Example a tri-diagonal system:
2 x1 x 2
=1
x1 + 2 x 2 x3
=0
x 2 + 2 x3 x 4 = 0
x3 + 2 x 4 = 1
or in general form
d1 x1 + a1 x 2
= r1
b2 x1 + d 2 x 2 + a 2 x3
= r2
bn 1 x n 2 + d n 1 x n 1 + a n 1 x n = rn1
bn x n1 + d n x n = rn
How to solve this LAE problem?
Thomas Method
Concept: Coefficient of the diagonal elements is scaled to one at each stage
Step 1: For the 1st equation, form the new element
a1 =
a1 and r1 .
a1
r
; r1 = 1
d1
d1
Step2: For each of the equations, from i = 2, K , n 1
ai =
ai
r bi ri 1
; ri = i
d i bi ai 1
d i bi ai 1
Step 3: For the last equation
rn =
rn bn rn 1
d n bn a n 1
Step 4: Solve by back substitution, yielding
x n = rn
xi = ri ai xi +1
for i = n 1, n 2, K ,2,1
LI - 13

Exercises:
Extend the Matlabs code for the standard Gauss elimination to include either the pivoting
strategy or Thomas method.
A 10 stage equilibrium process such as liquid extraction or gas absorption can be modelled by a
tri-diagonal system of linear equations as,
(1 + rr ) x1 + rr x 2 = Fin
xi 1 (1 + rr ) xi + rr xi +1 = 0
for i = 2, K , n 1
x n 1 (1 + rr ) x n = Fout
For rr = 0.9 and
Fin = 0.05; Fout = 0.5 , solve this LAE problem using the standard Gauss
elimination method and Thomas method. What conclusions you can make!
LI - 14

1.
Gauss elimination by pivoting
function x = Gpivot(A,b),
% Inputs:
% A is the n by n coefficient matrix
% b is the n by 1 right hand side matrix
% Outputs:
% x is the n by 1 solution matrix
%
[n,n1]=size(A);
for i=1:n-1,
[pivot,k]=max(abs(A(i:n,i)));
% k is position of pivot, relative to row i
% do not check rows above current row
if k>1
temp1 = A(i,:); temp2 = b(i,:);
A(i,:) = A(i+k-1,:);
b(i,:) = b(i+k-1,:);
A(i+k-1,:) = temp1;
b(i+k-1,:) = temp2;
end
for h=i+1:n
m=A(h,i)/A(i,i);
A(h,:) = A(h,:) - m*A(i,:);
b(h,:) = b(h,:) - m*b(i,:);
end
end;
% back substitution
x(n,;) =b(n,:) ./A(n,n);
for i=n-1:-1:1,
x(i,:) =(b(i,:)-A(i,i+1:n)*x(i+1:n,:))./A(i,i);
end;
LI - 15

2.
Gauss elimination using Thomas method
function x = Thomas(a,d,b,r),
%
% Inputs:
% a is the upper diagonal of matrix A, a(n) = 0
% d is the diagonal of matrix A
% b is the lower diagonal of matrix A, b(1) = 0;
% r is the right-hand side of equation
%
% Outputs:
% x is the n by 1 solution matrix
%
n = length(d);
a(1) = a(1)/d(1);
r(1) = r(1)/d(1);
for i=2:n-1
denom = d(i)-b(i)*a(i-1);
if (denom==0), error('zero in denominator'), end
a(i) = a(i)/denom;
r(i) = (r(i)-b(i)*r(i-1))/denom;
end
r(n) = (r(n) - b(n)*r(n-1))/(d(n)-b(n)*a(n-1));
x(n) = r(n);
for i=n-1:-1:1,
x(i) = r(i) - a(i)*x(i+1)
end
LI - 16

LI - 17

LAE Iterative Methods
Basic Idea
To solve the i-th equation in the system for the i-th variable by converting the given system
to be suitable for finding the solution in the iterative manner
Procedure
Transform Ax = b to x = Cx + d so that we can solve iteratively for
(k )
= Cx
( k 1)
x as
+d.
Example
a11 x1 + a12 x 2 + a13 x3 + a14 x 4 = b1

a 21 x1 + a 22 x 2 + a 23 x3 + a 24 x 4 = b2
a 31 x1 + a 32 x 2 + a 33 x3 + a34 x 4 = b3
a 41 x1 + a 42 x 2 + a 43 x3 + a 44 x 4 = b4
x1k =
a12 k 1 a13 k 1 a14 k 1 b1

x2
x3
x4 +
a11
a11
a11
a11
x 2k =
a 21 k 1 a 23 k 1 a 24 k 1 b2
x1
x3
x4 +
a 22
a 22
a 22
a 22
x3k =
a31 k 1 a32 k 1 a 34 k 1 b3
x1
x2
x4 +
a 33
a33
a33
a33
x 4k =
a 41 k 1 a 42 k 1 a 43 k 1 b4
x1
x2
x3 +
a 44
a 44
a 44
a 44
Stopping Conditions
To stop the iterations when the norm of the change in the solution vector
to the next is sufficiently small.
To stop the iterations when the norm of the residual vector
x from one iteration
A x b is below a specified
tolerance.
Variations
Jacobi method
Gauss-Seidel method
Successive Over Relaxation (SOR) method
LI - 18

Jacobi Method
= b to x = Cx + d , in which the matrix C has zeros on the

diagonal. The vector x is updated using the previous estimate for all components of x to evaluate
Based on the transformation of Ax
the right hand side of the equation (simultaneous updating).
Gauss-Seidel Method
Based on the transformation of Ax = b to x = Cx + d , in which the matrix C has zeros on the
diagonal as in Jacobi method, but each component of the vector x is updated immediately as each
iteration progresses (sequential updating).
SOR Method
Like Gauss-Seidel method, but we introduce an additional parameter, that may accelerate the
convergence of iteration. For 0 < < 1 , the method is known as successive under relaxation, while
for 1 < < 2 is known as successive over relaxation.
Examples:
2 x1 x 2 + x3 = 1
x1 + 2 x 2 x3 = 6
In the form of
Ax = b
In the form of
x = Cx + d
x1 x 2 + 2 x3 = 3
x1 = 0.5 x 2 0.5 x3 0.5

x 2 = 0.5 x1 + 0.5 x3 + 3
x3 = 0.5 x1 + 0.5 x 2 1.5
Iteration scheme:
Jacobi Method
LI - 19

x1k 0
0.5 0.5 x1k 1 0.5
k
0.5 x 2k 1 + 3
x 2 = 0 .5 0
x3k 0.5 0.5
0 x3k 1 1.5

T
o
Initial condition/guess: x = [0 0 0] .
Stopping condition
x k x k 1 < 0.001
Results:
After 13 iterations, we get
x * = [1.0002 2.0001 0.9997 ]
Gauss-Seidel Method
x1new = 0.5 x 2old 0.5 x3old 0.5
x 2new = 0.5 x1new + 0.5 x3old + 3

x3new = 0.5 x1new + 0.5 x 2new 1.5
Initial condition/guess:
Stopping condition
x o = [0 0 0] .
T
x k x k 1 < 0.001
Results:
After 10 iterations, we get
x * = [1.0001 1.9999 1.0001]
Improvement in the number of iteration!

Notes on Jacobi method
Transformation
Ax = b ( L + D + U ) x = b , where D is a diagonal matrix, L is a lower triangular

matrix, and U is an upper triangular matrix.
Dx = ( L U ) x + b
x = D 1 ( L U ) x + {
D 1b
142
4 43
4
d
C
Convergence
Example:
Starting with
x + 2y = 6
x = 2 y + 6
2x + y = 6
y = 2 x + 6
x1 = 0.5 and y 1 = 0.5 , we get:
LI - 20

x 2 = 2 y 1 + 6 = 5
y 2 = 2 x 1 + 6 = 5
Further iteration shows that the solution diverges!
Sufficient condition to converge

The iteration will converge when the matrix A is strictly diagonally dominant (For each row,
the magnitude of the diagonal element is greater than the sum of the magnitude of the other
elements in the row)
Necessary and sufficient conditions to converge

The iteration will converge if
max (C ) < 1 , where max (C ) is the largest eigenvalue of C .
Notes on Gauss-Seidel method
Transformation

U is an upper triangular matrix.
matrix, and
( D + L) x = (U ) x + b
x = ( D + L) 1 (U ) x + ( D + L) 1 b
144244
3
14243
C
Convergence
1.
If A is real and symmetric, with positive diagonal elements, Gauss-Seidel

method will converge if and only if eig ( A) is real and positive.
2.
If
A is positive definite, , Gauss-Seidel method will converge for any initial
vector
3.
If C is non-negative, the Jacobi and Gauss-Seidel methods either both converge

or both diverge, but Gauss-Seidel method converges more rapidly.
LI - 21

Matlabs Codes
Jacobi method
function x = Jacobi(A,b,x0,tol,max)
%
% Inputs
%
A
coefficient matrix (n by n)
%
b
right-hand side (n by 1)
%
x0
initial solution (n by 1)
%
tol
stop if nomr of change in x < tol
%
max
maximum number of iteration
%
% Outputs
%
x
solution vector (n by 1)
[n,m] = size(A);
xold = x0;
C = -A;
for i=1:n,
C(i,i) = 0;
end;
for i=1:n,
C(i,:) = C(i,:)/A(i,i);
end
for i=1:n
d(i,1) = b(i)/A(i,i);
end
i=1
disp('
i
x1
x2
x3
....')
while (i<=max)
xnew = C*xold + d;
if norm(xnew-xold) <= tol,
x = xnew
disp('Jacobi method converged');
return;
else
xold = xnew;
end;
disp(['i
xnew']);
i = i+1;
end
LI - 22

disp('Jacobi method did not converge');
disp('results after maximum number of iterations');
x = xnew;
Gauss-Seidel method
function x = GSeidel(A,b,x0,tol,max)
%
% Inputs
%
A
%
b
%
x0
%
tol
%
max
%
% Outputs
%
x
[n,m] = size(A);
x = x0;
C = -A;
for i=1:n,
C(i,i) = 0;
end;
for i=1:n,
C(i,1:n) = C(i,1:n)/A(i,i);
end
for i=1:n
r(i,1) = b(i)/A(i,i);
end
i=1
disp('
i
x1
x2
x3
....')
while (i<=max)
xold =x;
for j=1:n
x(j) = C(j,:)*x+r(j);
end
if norm(xold-x) <= tol,
disp('Gauss Seidel method converged');
return;
end;
disp(['i
x']);
i = i+1;
end
disp('Gauss-Seidel method did not converge');
LI - 23

Successive Over Relaxation (SOR) Method

Example:
a11 x1 + a12 x 2 + a13 x3 = b1

a 21 x1 + a 22 x 2 + a 23 x3 = b2
a31 x1 + a32 x 2 + a33 x3 = b3
SOR Iterative scheme:
x1new = (1 ) x1old +
x 2new = (1 ) x 2old +
x3new = (1 ) x3old +
It can be seen that if
a11
a 22
a33
(b
(b
a12 x 2old a13 x3old
a 21 x1new a 23 x3old
a 31 x1new a32 x 2new
(b
= 1 , this becomes the iterative scheme for Gauss-Seidel method.
Notes
Transformation

matrix, and U is an upper triangular matrix.
( D + L) x = (U ) x + b
Add
(1 ) Dx
( D L) x = ((1 ) D U ) x + b
x = ( D L) 1 ((1 ) D U ) x + ( D L) 1 b
Convergence
Ostrowskis Theorem
If A > 0 and 0 < < 2 , then SOR method will converge for any initial vector x .
Important condition is that the matrix A is positive definite. How to be determined?
Necessary and sufficient conditions for positive definite matrix
LI - 24

For a real, symmetric matrix
A , A > 0 if and only if:
1.
2.
x Ax > 0 for all x 0

All eig ( A) are positive
3.
4.
All upper left sub-matrices of

All pivots of A are positive
A have positive determinant
Matlabs Code:
function x = Sor(A,b,x0,w,tol,max)
%
% Inputs
%
A
%
b
%
x0
%
w
weighting parameter
%
tol
%
max
%
% Outputs
%
x
[n,m] = size(A);
x = x0;
C = -A;
for i=1:n,
C(i,i) = 0;
end;
for i=1:n,
C(i,1:n) = C(i,1:n)/A(i,i);
end
for i=1:n
r(i,1) = b(i)/A(i,i);
end
i=1
disp('
i
x1
x2
x3
....')
while (i<=max)
xold =x;
for j=1:n
x(j) = (1-w)*xold(j) + w*(C(j,:)*x+r(j));
end
if norm(xold-x) <= tol,
disp('SOR method converged');
return;
end;
disp(['i
x']);
i = i+1;
end
disp('SOR method did not converge');
MATLABs METHODS
Available Matlabs routine to solve LAE ( Ax
= b ):
Preconditioned conjugate gradient method, x = pcg(A,b,tol)
LI - 25

Bi-conjugate gradient method, x = bicg(A,b) (or x = bicgstab(A,b))
Generalised minimum residual method, x = gmres(A,b)
Quasi minimal residual method, x = qmr(A,b)
LI - 26

LU AND QR FACTORISATIONS
Factorisations that play important role in numerical linear algebra, together with eigenvalues and
eigenvectors, will give foundations of many areas in science and engineering.
LU- FACTORISATION
Concept:
A matrix A is factorised into lower and upper matrix factors, for example if
3x3 matrix:
A = L.U
l 11
l 21
l 31
0
l 22
l 32
0 u11
0 . 0
l 33 0
u12
u 22
0
u13 a11
u 23 = a 21
u 33 a31
a12
a 22
a32
A is a
a13
a 23
a33
Methods:
Gauss elimination
Direct computation
Doolittle form (equivalent to Gauss elimination)
Crout form ( u ii = 1 )
Cholesky form ( u ii
= l ii )
Applications:
Finding determinant of a matrix

Finding inverse of a matrix
Solving systems of LAE
LU Factorisation by Gauss Elimination
Suppose a matrix
A,
4 12 8 4
1 7 18 9
; and
A=
2 9 20 20
3 11 15 14
1
l
L = 21
l 31
l 41
0
1
l 32
l 42
0
0
1
l 43
0
0
0
1
LI - 27

Step 1: Assume
U = A . Row 1 is unchanged, and rows 2-4 are modified to give
l 21 =
a21 1
=
a11 4
l 31 =
a31 1
=
a11 2
l 41 =
a41 3
=
a11 4
4 12 8 4
0 4 16 8
U =
0 3 16 18
0 2 9 11
Step 2: Rows 1 and 2 are unchanged. Rows 3 and 4 are transformed to yield,
l 32 =
a32 3
=
a 22 4
l 42 =
a 42 1
=
a 22 2
4 12 8 4
0 4 16 8
U =
0 0 4 12
0 0 1 7
Step 3: The fourth row is modified to complete the forward eliminating stage
l 43 =
a 43 1
=
a33 4
4 12 8 4
0 4 16 8
U =
0 0 4 12
0 0 0 4
Hence the LU factors are
0
0
1
0.25
1
0
L=
0.5 0.75
1
0.75 0.5 0.25
0
0
;
0
4 12 8 4
0 4 16 8
U =
0 0 4 12
0 0 0 4
Matlabs Code:
function [L,U] = LUfactor(A),
%
% Input
%
A n by n matrix
%
% Outputs
%
L lower triagular matrix
%
U upper triangular matrix
%
[n,m] = size(A);
L = eye(n);
U = A;
for j=1:n,
for i=j+1:n,
L(i,j)=U(i,j)/U(j,j);
LI - 28

U(i,:)=U(i,:)-L(i,j)*U(j,:);
end
end
% display L and U
L
U
% Verify results
B = L*U
A
Exercise:
Perform LU factorisation for the following matrix
30 20 10
A = 20 55 10
10 10 50
Result:
10
0
0
30 20
1
L = 2 / 3
1
0; U = 0 125 / 3 50 / 3
0
1 / 3 2 / 5 1
0
40
Notes on the Method
LU factors from simple matrices

Define matrices
M 1 and M 2 as,
1
M 1 = m21
m31
0 0
1 0
1 0; M 2 = 0 1
0 m32
0 1
0
0
1
The associated inverses are
1
1
1
= m21
m31
0 0
0
1
1 0 ; M 2 = 0
1
0 m32
0 1
0
0
1
By observing that:
LI - 29

M 11 .( M 1 . A) = A
( M 11 .M 21 ).{M 2 .( M 1 . A)} = A
We can obtain
L = M 11 .M 21 , or
1
m
21
m31
and
0 0 1
0
1 0. 0
1
0 1 0 m32
0 1
0 = m21
1 m31
0
1
m32
0
0
1
U = M 2 .( M 1 . A) .
LU factors for tri-diagonal matrices
d1
b
1
A=0
0
0
0
a1
d 2 a2
O O
L bn 1
L 0
0
0
O
d n 1
bn
; which can be represented as
a n 1
d n
0
0
0
d = [d1 , d 2 ,K, d n ]; a = [a1 , a 2 , K, a n1 ,0]; b = [0, b1 , b2 ,K, bn ]

Matlabs code:
function [dd,bb] = LUtridiag(a,d,b),
%
% Input
%
a vector of elements above main diagonal,a(n) = 0
%
d vector of matrix T
%
b vector of elements below main diagonal, b(1) = 0
%
% Outputs
%
Lower bidiagonal matrix
%
bb lower diagonal with 1s on main diagonal
%
Upper diagonal matrix
%
main diagonal is dd; upper diagonal is a
%
n = length(d);
bb(1) = 0;
dd(1) = d(1);
for i=2:n,
bb(i) = b(i)/dd(i-1);
dd(i) = d(i) - bb(i)*a(i-1);
end
Exercise:
Perform the LU factorisation for a tri-diagonal matrix as follows:
LI - 30

2 1 0 0
1 2 1 0
A=
0 1 2 1
0 0 1 2
Result:
0
0
1
1 / 2
1
0
L=
0
1
2/3
0
3/ 4
0
0
0
0
2 1
0
0 3/ 2 1
0
;U =
0 0 4 / 3 1
0
1
0 5 / 4
0 0
Direct LU Factorisation
Concept:
Based on equating the elements of the product
elements of A .
LU with the corresponding
Doolittles Method
L are 1s.
The diagonal elements of
Example Consider a 3x3 matrix A
1
l
21
l 31
0
1
l 32
0 u11 u12
0 0 u 22
1 0
0
u13 a11
u 23 = a 21
u 33 a31
a12
a 22
a32
a13
a 23
a33
Procedure:
u11 = a11 , then solve for the remaining elements in the first row of U and
the first column of L .
Find u 22 , then solve for the remainder of the second row of U and the second
column of L
Find
Continue for the rest.
LI - 31

Matlabs Code:
function [L,U] = doolitle(A);
[n,m] = size(A);
U = zeros(n,n);
L = eye(n);
for k=1:n,
U(k,k) = A(k,k)-L(k,1:k-1)*U(1:k-1,k);
for j=k+1:n,
U(k,j)=A(k,j)-L(k,1:k-1)*U(1:k-1,j);
L(j,k)=(A(j,k)-L(j,1:k-1)*U(1:k-1,k))/U(k,k);
end
end
Example
1 4 5
1
A = 4 20 32 l 21
l 31
5 32 64
First row of
0
1
l 32
0 u11
0 . 0
1 0
u12
u 22
0
u13 1 4 5
u 23 = 4 20 32
u 33 5 32 64
U and first column of L
(1).u11 = a11 = 1; (1).u12 = a12 = 4; (1).u13 = a13 = 5

l 21 .u11 = a 21 = 4;
l 31 .u11 = a31 = 5
We get
1 0
4 1
5 l 32
Second row of
0 1 4
0 . 0 u 22
1 0 0
5 1 4 5
u 23 = 4 20 32
u 33 5 32 64
U and second column of L
(4).(4) + u 22 = 20 u 22 = 4;
(4).(5) + u 23 = 32 u 23 = 12;
(5).(4) + l 32 u 22 = 32 l 21 = 3;
We get
1 0 0 1 4 5 1 4 5
4 1 0 . 0 4 12 = 4 20 32
5 3 1 0 0 u 33 5 32 64
LI - 32

The last element
(5).(5) + (3).(12) + u 33 = 64 u 33 = 3;
We get,
1 0 0
1 4 5
L = 4 1 0 ; U = 0 4 12
5 3 1
0 0 3
Choleskys Method
A is symmetric and positive definite
The matrix
L =UT
Example for a 3x3 matrix
x1
l
21
l 31
0
x2
l 32
A = LLT
0 x1
0 0
x3 0
u12
x2
0
u13 a11
u 23 = a 21
x3 a31
a12
a 22
a32
a13
a 23
a33
Procedure:
x1 = (a11 )1 / 2 , then solve for the remaining elements in the first row of U and
the first column of L .
Find x 2 , then solve for the remainder of the second row of U and the second
column of L
Find
Continue for the rest.
Matlabs Code
function [L,U] = Cholesky(A)
%
% A is assumed to be symmetric
%
[n,m] = size(A);
L = zeros(n,n);
for k=1:n,
L(k,k) = sqrt(A(k,k)-L(k,1:k-1)*L(k,1:k-1)');
for i=k+1:n,
L(i,k) = (A(i,k) - L(i,1:k-1)*L(k,1:k-1)')/L(k,k);
end
end
U = L';
LI - 33

Example:
1 4 5
x1
A = 4 20 32 l 21
5 32 64
l 31
First row of
0
x2
l 32
0 x1
0 . 0
x3 0
u12
x2
0
u13 1 4 5
u 23 = 4 20 32
x3 5 32 64
U and first column of L
x1 .x1 = a11 x1 = 1; x1 .u12 = a12 u12 = 4; x1 .u13 = a13 u13 = 5

l 21 .x1 = a 21 l 21 = 4;
Note that since
A is symmetric, a1 j = a j1 l j1 = u1 j , we get
1 0
4 x
2
5 l 32
Second row of
l 31 .x1 = a31 l 31 = 5
0 1 4
0 . 0 x 2
x3 0 0
5 1 4 5
u 23 = 4 20 32
x3 5 32 64
U and second column of L
(4).(4) + x 22 = 20 x 2 = 2;
(4).(5) + x 2 u 23 = 32 u 23 = 6;
(5).(4) + l 32 x 2 = 32 l 21 = 6;
We get
1 0 0 1 4 5 1 4 5
4 2 0 . 0 2 6 = 4 20 32
5 6 x3 0 0 x3 5 32 64
The last element
(5).(5) + (6).(6) + x32 = 64 x33 = 3;

We get,
LI - 34

1 0
L = 4 2
5 6
0
1 4
0 ; U = 0 2
0 0
3
5
6
3
Applications of LU Factorisation
Solving of LAE Systems

1.
2.
3.
4.
Transform Ax = b to LUx = b .
Define a new vector y = Ux
Solve for y by forward substitution
Solve for x by backward substitution
To think about - Develop m-file to solve LAE systems using LU factorisation.
Solving of Determinant of a Matrix

If
i =1
i =1
A = LU , then det( A) = l ii u ii .
To think about - Develop m-file to calculate determinant of a square matrix.
Inverse of a Matrix
A = LU . By definition we have AA 1 = I . This implies that LUX = I . Define

Y = UX . We get LY = I . The algorithm becomes:
Assume
1.
2.
Solve for Y in LY
Solve for X in UX
3.
We have
=I
=Y
A 1 = X .
To think about - Develop m-file to calculate inverse of a square matrix
LI - 35

QR- FACTORISATION
Definition:
A = Q.R
where
R is an upper triangular matrix, and Q is an
orthogonal matrix ( Q
= Q T ).
Householder transformation
A transformation that will reduce certain elements of a given vector to zero.

A householder matrix is defined as,
H = I 2 wwT
where
w is a unit (column) vector, i.e. w
= 1.
Multiplication by a householder matrix
HA = ( I 2 wwT ). A = A 2ww T . A = A wu T
T
where u = 2 A w .
Example: How to find a householder matrix for a given square matrix
A.
21 7 1
A = 5
7
7
4 4 20
A , so k = 1 .
The vector we wish to transform is the first column of
x = A(:,1) = [21,5,4]
We find
g = x12 + x 22 + x32 = norm( x) = 21.954

v = x + g.[1,0,0]T = [42.954 5 4]
s = norm(v) = 43.429
w = v / s = [0.98907 0.11513 0.092104]
u = 2 AT w
and
= [43.429 14.722 3.3178]
42.954 14.561 3.2816

1.6949 0.38198
wu = 5
4
1.356 0.30559
21.954 7.5611 4.2816

0
5.3051
6.618
B = A wu =
0
5.3560 19.694
T
LI - 36

H = B. A 1
Matlabs Code Find a householder matrix
function H = householder(x,k)
[n,nn]= size(x);
w = zeros(n,1);
g = norm(x(k:n));
p = sign(x(k));
s = sqrt(2*g*(g+p*x(k)));
w(k) = (x(k)+p*g)/s;
w(k+1:n) = x(k+1:n)/s;
H = eye(n) - 2*w*w';
Basic QR Factorisation
Algorithm
1.
2.
3.
Given a real nxn matrix

( 0)
Define R =
For k=1,,n-1
Find
H ( k ) (the householder matrix to reduce positions k+1,,n in the kth column
R ( k 1) to zero)
(k )
= H ( k ) R ( k 1)
Define R
of
4.
5.
end
Define Q = I
For k=n-1, ., 1
Q = H (k )Q
end
6.
Define
R = R ( n1) .
Matlabs Code QR Factorisation

function [Q,R] = QRfactor(A)
[n,nn] = size(A);
R = A;
Q = eye(n);
for k=1:n-1,
x = zeros(n,1); x(k:n,1) = R(k:n,k);
g = norm(x)
v = x;
v(k)= x(k)+g;
s = norm(v);
LI - 37

w
u
R
Q
=
=
=
=
v/s;
2*R'*w;
R-w*u';
Q-2*Q*w*w';
end
LI - 38

EIGENVALUES AND EIGENVECTORS
Why the eigenvalues and eigenvectors are important?
Play a key role in the analysis of the convergence characteristics of iterative methods
Also, in solving differential equations
And, others such as stability analysis of dynamic systems
Definition
A (real or complex) number is an eigenvalue of a matrix
corresponding eigenvector of A , if and only if Ax = x .
Property of
A , an a nonzero vector x is a
is an eigenvalue of A if and only if is a zero of the characteristics polynomial

f ( ) = det( A I ) .
Methods to Find
Power Method Finding the dominant eigenvalue

Inverse Power Method Finding the eigenvalue of smallest magnitude
QR Method Finding all of the eigenvalues of a matrix
Power Method
Iterative procedure for determining the dominant eigenvalue of a matrix
Basic Idea
A sequence of approximations to
eigenvalue: Ax = x .
and x provided that and x satisfy the definition of
Algorithm
LI - 39

1.
2.
3.
4.
Initial guess z for the true eigenvector x

Compute w = Az
Normalise w and use it as the next approximation of
Repeat step 2-3 until achieving the desired accuracy
Matlabs Code:
function [z,m] = Powerm(A,max_it,tol),
[n,m] = size(A);
z = ones(n,1);
it=0;
error = norm(A*z-m*z);
disp('
it m
z(1) z(2)
z(3)')
while (it < max_it & error > tol)
w = A*z;
ww = abs(w);
[k,kk] = max(ww);
m = w(kk);
z = w/w(kk);
out=[it+1
m
z'];
disp(out);
it = it+1;
end
error;
Example:
Using the Matlabs Code for the power method, we calculate the estimated eigenvector and the
dominant eigenvalue of the following matrix A
1.
44 9 6
A = 280 57 40
75 15 13
Results:
it
1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
m
-263.0000
2.2471
3.1252
1.3855
3.6667
-1.0699
z(1)
0.1559
0.2115
0.1586
0.2458
0.1387
-0.2635
z(2)
1.0000
1.0000
1.0000
1.0000
1.0000
-0.6804
z(3)
0.2776
-0.1337
0.2799
-0.3873
0.4362
1.0000
LI - 40

7.0000
8.0000
9.0000
10.0000
11.0000
12.0000
13.0000
14.0000
15.0000
-5.0131
-1.8843
-3.2827
-2.7992
-3.0823
-2.9466
-3.0362
-2.9761
-3.0160
0.1060
-0.1129
0.0689
-0.0493
0.0320
-0.0217
0.0143
-0.0096
0.0064
1.0000
0.0894
1.0000
0.4147
0.8303
0.5556
0.7398
0.6175
0.6993
0.6874
1.0000
0.9714
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
After 15 iterations we have the dominant eigenvalue to be 3.0160, while the eigenvector is
[0.001
0.699 1.000] .
2.
21 7 1
A = 5
7
7
4 4 20
Results:
it
m
z(1)
z(2)
z(3)
1.0000 27.0000 1.0000 0.7037 0.7407
2.0000 25.1852 1.0000 0.6000 0.6353
3.0000 24.5647 1.0000 0.5556 0.5824
4.0000 24.3065 1.0000 0.5334 0.5523
5.0000 24.1816 1.0000 0.5211 0.5340
6.0000 24.1135 1.0000 0.5136 0.5224
7.0000 24.0731 1.0000 0.5089 0.5148
After 7 iterations we have the dominant eigenvalue to be 24.07, while the eigenvector is
[1.00
0.51 0.51] .
T
Shifted Power Method

If we already know an eigenvalue of a matrix A , we can find another eigenvalue of
A by applying the power method to the matrix B = A I
Example:
1.
5 2 0
G = 2 3 0 . One eigenvalue is 6; to find another eigenvalue, we apply the power
0
0 6
method to the shifted matrix B = G 6 I , or
1 2 0
B = 2 3 0
0
0 0
LI - 41

We get,
it
1.0000
2.0000
3.0000
4.0000
5.0000
m
-5.0000
-4.2000
-4.2381
-4.2360
-4.2361
z(1)
0.6000
0.6190
0.6180
0.6180
0.6180
z(2)
1.0000
1.0000
1.0000
1.0000
1.0000
The other eigenvalue is 4.2361.
Inverse Power Method

Provide an estimate of the eigenvalue of
A that is of smallest magnitude.
Basic Idea
1
Use the fact that the eigenvalues of B = A are the reciprocals of the eigenvalues of
The reciprocal of m will give the smallest magnitude eigenvalue of A .
A.
Algorithm
1.
2.
Initial guess z for the true eigenvector

Solve for w using Az = w
A 1 ,
4. Calculate the smallest eigenvalue of A as 1 /
3.
Estimate the largest eigenvalue of
5.
Repeat step 2-4 until achieving the desired accuracy
Matlabs Code:
function [z,m] = InvPower(A,max_it,tol),
[n,m] = size(A);
z = ones(n,1);
it=0;
error = 100;
[L,U] = LUfactor(A);
disp('
it m
z(1) z(2)
z(3)')
LI - 42

while (it < max_it & error > tol)
w = LUsolve(L,U,z);
ww = abs(w);
[k,kk] = max(ww);
m = (z'*z)/(z'*w);
z = w/w(kk);
out=[it+1
m
z'];
disp(out);
it = it+1;
end
Exercise: Apply this matlabs code for:
66 21 9
A = 228 73 33
84 28 16
Notes on Power Methods
1.
The proof of the convergence of the basic power method assumes that the dominant eigenvalue
is real and not a repeated eigenvalue, so that the eigenvalue can be ordered as
1 > 2 3 L n
2.
The convergence of the estimate at the jth iteration depends on
2 / 1
, so the method will
converge more rapidly when the dominant eigenvalue is much larger than the next most
dominant eigenvalue.
3.
Convergence may be improved for symmetric matrices by using Raleigh quotient to estimate
eigenvalue given by,
= ( z T w) /( z T z )
In this case, the estimate at the jth iteration depends on
2 / 1
2j
because the eigenvectors
can be taken to be orthonormal.
To think about Extend the basic power method to include the Rayleigh quotient.
LI - 43

QR Method
Basic Idea
A ( m ) that are orthogonally similar to A (and thus
We generate a sequence of matrices

have the same eigenvalues as A ).
A similarity transformation is a transformation of the form

nonsingular matrix.
The sequence
easily.
H 1 AH , where H is any
A ( m ) converges to a matrix from which the eigenvalues can be found

A satisfy 1 > 2 3 L n , the iterations converge
1.
If the eigenvalues of
2.
to an upper triangular matrix with the eigenvalues on the diagonal.

If A is symmetric, the iterations will converge to a block diagonal matrix in which
all blocks have order 1 or 2.
Basic QR Method
Algorithm
1. Given a real n by n matrix
2.
3.
(1)
Define A = A
For k=1:k_max
A ( k ) = Q ( k ) R ( k ) (so that Q ( k )T A ( k ) = R ( k ) )
( k +1)
Define A
= R ( k ) Q ( k ) (so that A ( k +1) = Q ( k )T A ( k ) Q ( k )
Factor
End
Matlabs Code:
function e = QReig(A,max),
for i=1:max,
[Q,R]= QRfactor(A);
A = R*Q;
end
e = diag(A);
Example:
LI - 44

2 / 3 4 / 3 4 / 3
4
2/3
4
0
0
A=
4 / 3 0
6
2
0
2
6
4/3
e = [6 4 8 2]
Improved QR Method
Improvement through a similarity transformation to convert
householder matrix.
A to Hessenberg form using a
Matlabs Code for Hessenberg Transformation

function A = Hesseng(A)
[n,nm]=size(A);
for k=1:n-2
H = householder(A(:,k),k+1);
A = H*A*H;
end
Matlabs Code for Improved QR Method

function e = QReigi(A,max)
[n,nm]=size(A);
for k=1:n-2
H = householder(A(:,k),k+1);
A = H*A*H;
end
for i=1:max,
[Q,R]= QRfactor(A);
A = R*Q;
end
e = diag(A);
Example:
2 / 3 4 / 3 4 / 3
4
2/3
4
0
0
A=
4 / 3 0
6
2
0
2
6
4/3
e = [5.9999 2.0001 4.0000 8.0000]
To think about
1.
2.
Compare the flops between basic and improved QR methods for different maximum iterations
Compare the results with the matlab command eig(A)
LI - 45

Non-Linear Algebraic Equations NAE
Objective
To study several techniques for finding roots or zeros of nonlinear equations of a single
or several variable
Methods
Bisection Method
Regula Falsi Method
Secant Method
Newton Method
Mullers Method
Fixed point Iteration
Bisection Method
A systematic approach to subdividing an interval on which we know the function has a

zero
Commonly used as an initial estimate for more powerful methods
Algorithm
1.
Given an interval
[a, b] in which the location of zero is detected (i.e. { f (a) < 0 and
f (b) > 0 }or { f (a ) > 0 and f (b) < 0 })

2.
Calculate a mid-point as the next estimate of the zero
m=
3.
4.
a+b
2
Test the function whether it changes sign for the interval a, m or

Repeat step 2-3 for the interval that the sign change occurs
[m, b].
Matlabs Code:
function [x,y] = Bisect(fun,a,b,tol,max)
%
a(1) = a;
b(1) = b;
ya(1) = feval(fun,a(1));
yb(1) = feval(fun,b(1));
if ya(1)*yb(1)>0.0
error('Function has same sign at end points')
LI - 46

end
for i=1:max
x(i) = (a(i)+b(i))/2;
y(i) = feval(fun,x(i));
if ((x(i)-a(i))<tol)
disp('Bisection method has converged');
break;
end
if y(i)==0.0
disp('exact zero found');
break;
elseif y(i)*ya(i)<0
a(i+1)=a(i);
ya(i+1)=ya(i)
b(i+1)=x(i);
yb(i+1) = y(i);
else
a(i+1) = x(i);
ya(i+1) = y(i);
b(i+1) = b(i);
yb(i+1) = yb(i);
end;
iter = i;
end
if (iter >= max)
disp('zero not found to desired tolerance');
end
n = length(x); k=1:n;
out = [k'
a(1:n)' b(1:n)' x' y'];
disp('
step
a
b
x
y')
disp(out);
Analysis
Basis
If f ( a )
Characteristics
> 0 and f (b) < 0 , or if f (a ) < 0 and f (b) > 0 , then there is a number
c between a and b such that f (c) = 0
Slow to convergence, but sure. Error at kth stage is at most
ba
.
2k
Convergence
By using the definition of linear convergence
x * x k r k x * x1 for some constant r < 1 ,

Bisection method is linearly convergent with rate .
Example
1. A function
y = x 3 3x 2 + 1
Applying the Bisection method for tolerance 0.001 yields:

it
1.0000
2.0000
3.0000
4.0000
5.0000
a
0
0.5000
0.5000
0.6250
0.6250
b
1.0000
1.0000
0.7500
0.7500
0.6875
x
y
0.5000 0.3750
0.7500 -0.2656
0.6250 0.0723
0.6875 -0.0930
0.6563 -0.0094
LI - 47

6.0000
7.0000
8.0000
9.0000
10.0000
0.6250
0.6406
0.6484
0.6523
0.6523
0.6563
0.6563
0.6563
0.6563
0.6543
0.6406
0.6484
0.6523
0.6543
0.6533
0.0317
0.0112
0.0009
-0.0042
-0.0016
Converge in 10 iterations!
2. A function
y = x2 3
Applying the Bisection method for tolerance 0.001 yields:

it
1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
7.0000
8.0000
9.0000
10.0000
11.0000
12.0000
a
-1.0000
0.5000
1.2500
1.6250
1.6250
1.7188
1.7188
1.7188
1.7305
1.7305
1.7305
1.7319
b
2.0000
2.0000
2.0000
2.0000
1.8125
1.8125
1.7656
1.7422
1.7422
1.7363
1.7334
1.7334
x
0.5000
1.2500
1.6250
1.8125
1.7188
1.7656
1.7422
1.7305
1.7363
1.7334
1.7319
1.7327
y
-2.7500
-1.4375
-0.3594
0.2852
-0.0459
0.1174
0.0352
-0.0055
0.0148
0.0047
-0.0004
0.0021
Regula Falsi and Secant Methods

Concept
Consider a straight-line approximation of the function.
Regula Falsi Method

1.
2.
Select two points ( a, y ( a )) and (b, y (b)) satisfying

Calculate the estimate of zero, x
x =b
y (a ). y (b) < 0
ba
y (b)
y (b) y (a )
LI - 48

3.
4.
5.
Test the function for the intervals [ a, x ] and [ x, b] .

Select the one that changes the sign.
Repeat 2-4 till converging to the desired level
Matlabs Code:
function [x,y] = Falsi(fun,a,b,tol,max)
%
a(1) = a; b(1) = b;
ya(1) = feval(fun,a(1));
yb(1) = feval(fun,b(1));
if ya(1)*yb(1)>0.0
error('Function has same sign at end points')
end
for i=1:max
x(i) = b(i) - yb(i)*(b(i)-a(i)/(yb(i)-ya(i)));
if y(i)==0.0,
disp('exact zero found');break;
elseif y(i)*ya(i)<0
a(i+1)=a(i);
ya(i+1)=ya(i);
b(i+1)=x(i);
yb(i+1) = y(i);
else
a(i+1) = x(i);
ya(i+1) = y(i);
b(i+1) = b(i);
yb(i+1) = yb(i);
end;
if ((i>1) & (abs(x(i)-x(i-1))<tol)
disp('Regula Falsi method has converged');
break;
end
iter = i;
end
if (iter >= max), disp('zero not found to desired tolerance');
end
n = length(x); k=1:n;
out = [k'
a(1:n)' b(1:n)' x' y'];
disp('
it a
b
x
y')
disp(out);
Example
1. A function
y = x 3 3x 2 + 1
Applying the Regula Falsi method for tolerance 0.001:

it
1.0000
2.0000
3.0000
4.0000
5.0000
a
0
0.5000
0.6364
0.6513
0.6526
b
1.0000
1.0000
1.0000
1.0000
1.0000
x
0.5000
0.6364
0.6513
0.6526
0.6527
y
0.3750
0.0428
0.0037
0.0003
0.0000
2. A function
y = x2 3
Applying the Regula Falsi method for tolerance 0.001 yields:
LI - 49

it
a
b
x
y
1.0000 -1.0000 2.0000 1.0000 -2.0000
2.0000 1.0000 2.0000 1.6667 -0.2222
3.0000 1.6667 2.0000 1.7273 -0.0165
4.0000 1.7273 2.0000 1.7317 -0.0012
5.0000 1.7317 2.0000 1.7320 -0.0001
Compare these results with the Bisection method!

To think about - Why is the Regula Falsi method faster convergent?
3. A function
y = x 5 0.5
Applying the Regula Falsi method for tolerance 0.0001 yields:

it
1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
7.0000
8.0000
9.0000
a
0
0.5000
0.7419
0.8335
0.8607
0.8680
0.8699
0.8704
0.8705
b
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
x
0.5000
0.7419
0.8335
0.8607
0.8680
0.8699
0.8704
0.8705
0.8705
y
-0.4688
-0.2752
-0.0976
-0.0276
-0.0073
-0.0019
-0.0005
-0.0001
-0.0000
Converge in 9 iterations! Try with bisection method. Converge in 14 iterations!

Secant Method
( xo , y ( xo )) and ( x1 , y ( x1 )) .
2. Calculate the estimate of zero, x from the two most recently generated points
x1 xo
x k x k 1
x 2 = x1
y ( x1 ) or x k = x k 1
y( xk )
y ( x1 ) y ( xo )
y ( x k ) y ( x k 1 )
1.
Select two points
3.
Repeat 2 till converging to the desired level
Matlabs Code:
function [x,y] = secant(fun,a,b,tol,max)
%
x(1) = a;
x(2) = b;
y(1) = feval(fun,x(1));
for i=2:max
x(i+1) = x(i) - y(i)*(x(i)-x(i-1))/(y(i)-y(i-1));
y(i+1) = feval(fun,x(i+1));
if (abs(x(i+1)-x(i))<tol)
disp('Secant method has converged');
break;
LI - 50

end
if y(i)==0.0,
disp('exact zero found');
break;
end
iter = i;
end
if (iter >= max)
end
n = length(x);
k=1:n;
out = [k'
x' y'];
disp('
it x
y')
disp(out);
Analysis
The secant method does not bracket zero at each iteration, so the secant method is not
guaranteed to converge. However, when it does, the convergence is usually more rapid than
bisection method.
Example
1. A function
y = x 3 3x 2 + 1
Applying the Secant method for tolerance 0.001:

it
1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
7.0000
y
0
1.0000
1.0000 -1.0000
0.5000 0.3750
0.6364 0.0428
0.6539 -0.0033
0.6527 0.0000
0.6527 0.0000
2. A function
y = x2 3
Applying the Secant method for tolerance 0.001 yields:

it
x
y
1.0000 -1.0000 -2.0000
2.0000 2.0000 1.0000
3.0000 1.0000 -2.0000
4.0000 1.6667 -0.2222
5.0000 1.7500 0.0625
6.0000 1.7317 -0.0012
7.0000 1.7320 -0.0000
LI - 51

To think about:
Compare these results with the Bisection method!
Try the secant method for a function:
y = x 5 0.5 with initial estimate [0,1] and [1,0] .
Newton Method
Concept
Use a straight-line approximation to the function whose zero we wish to find, but in
this case the line is the tangent to the curve. The derivative of the function is required
in the algorithm.
The next approximation to the zero is the value where the tangent crosses the x -axis.
Algorithm
1.
Given initial estimate of the zero
2.
Calculate
3.
Up-date the estimate using
y o = f ( xo ) and y o' = f ' ( xo )
x k = x k 1
4.
y k 1
y k' 1
Repeat 2-3 till converging to the desired level
Matlabs Code:
function [x,y] = newton(fun,funpr,x1,tol,max)
%
x(1) = x1;
ypr(1) = feval(funpr,x(1));
for i=2:max,
x(i) = x(i-1)-y(i-1)/ypr(i-1);
if abs(x(i)-x(i-1))<tol
disp('Newton method has converged');break;
end
ypr(i) =feval(funpr,x(i));
iter=i;
LI - 52

end
if (iter >= max)
end
n = length(x);
k=1:n;
out = [k'
x' y'];
disp('
it x y')
disp(out);
Example
1. A function
y = x 3 3x 2 + 1
Applying the Newton method for tolerance 0.001:

it
1.0000
2.0000
3.0000
4.0000
x
1.0000
0.6667
0.6528
0.6527
y
-1.0000
-0.0370
-0.0002
-0.0000
Converge in 4 iteration!
2. A function
y = x2 3
Applying the Newton method for tolerance 0.001 yields:

it
1.0000
2.0000
3.0000
4.0000
5.0000
x
y
1.0000 -2.0000
2.0000 1.0000
1.7500 0.0625
1.7321 0.0003
1.7321 0.0000
Converge in 5 iteration!
Compare these results with the bisection, regula falsi and secant methods!
To think about:
Use example 1, but now the initial estimate is zero. What is going to happen? What is your
conclusion?
LI - 53

Analysis
Convergence theorem
If
f ( x), f ' ( x) and f '' ( x) are continuous for all x in a neighbourhood of x *

'
*
2. f ( x ) 0 and
*
3. x o is chosen sufficiently close to x
f ( xk )
*
will converge to x .
then the iterations x k +1 = x k '
f ( xk )
1.
Error :
x * x k +1 = 0.5( x * x k ) 2
f " ( )
where is some (unknown) point between x and x k .
f ' ( xk )
To think about
How to get this error (Hint! Use Taylor series expansion to derive it).
Apply the Newton method for the function,
y = x 3 3 x 2 + x + 3 with initial estimate
xo = 1 . Do you get an oscillating behaviour? Why?
Mullers Method
LI - 54

Concept
To approximate the function by a quadratic function from three points
Algorithm
1.
Given three points
2.
Compute
c1 =
( x1 , y1 ), ( x 2 , y 2 ) and ( x3 , y3 ) .
y y2
y 2 y1
c c
; c2 = 3
; d1 = 2 1
x3 x1
x 2 x1
x3 x 2
and
s = c 2 + d 1 ( x3 x 2 )
3.
The next approximation is given by
x = x3
2 y3
s + sign( s ) s 2 4 y 3 d1
Matlabs Code:
function [x,y] = muller(fun,x1,x2,tol,max)
%
x(1) = x1; x(2) = x2; x(3) = (x(2)+x(1))/2;
y(1) = feval(fun,x(1)); y(2) = feval(fun,x(2)); y(3) = feval(fun,x(3));
c(1) = (y(2)-y(1))/(x(2)-x(1));
for i=3:max,
c(i-1) = (y(i)-y(i-1))/(x(i)-x(i-1));
d(i-2) = (c(i-1)-c(i-2))/(x(i)-x(i-2));
s = c(i-1)+(x(i)-x(i-1))*d(i-2);
x(i+1) = x(i)- 2*y(i)/(s+sign(s)*sqrt(s^2-4*y(i)*d(i-2)));
y(i+1) = feval(fun,x(i+1));
if abs(x(i+1)-x(i))<tol
disp('Muller method has converged');break;
end
iter=i;
end
if (iter >= max)
end
n = length(x);
k=1:n;
out = [k'
x' y'];
disp('
it x y')
disp(out);
Example
1.
Function y = x 2 . Applying the Muller method to find zero of this function. Initial
estimates are 0.5 and 1.5
it
LI - 55

1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
7.0000
2.
0.5000
1.0000
0.7500
1.1800
1.1171
1.1222
1.1225
-1.9844
-1.0000
-1.8220
0.6989
-0.0568
-0.0025
-0.0000
10
Function y = x 0.5 . Applying the Muller method to find zero of this function. Initial
estimates are 0 and 1
it
1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
7.0000
8.0000
x
0
1.0000
0.5000
0.8087
0.9081
0.9433
0.9327
0.9330
y
-0.5000
0.5000
-0.4990
-0.3803
-0.1186
0.0575
-0.0018
-0.0000
To think about
What is a parabolic equation that is passing through three points?

How to get equations in step 2 and 3 of the Mullers algorithm?
LI - 56

Fixed-point Iteration
A fixed point is a solution of the equation
3
f ( x) = 5 x 10 x + 3 = 0
practically if
x = g (x) . For example, a function
x = g ( x) = 0.5 x 3 + 0.3 . This technique is used
g ' ( x) 1 / 2 .
Matlabs Methods
Matlab has two built-in functions to calculate zeros, roots(p) for p is vector of coefficient of a
polynomial, and fzero(function name,x0).
NAE with Several Variables

General non-linear systems
LI - 57

f 1 ( x1 , x 2 ,K, x n ) = 0
f 2 ( x1 , x 2 ,K, x n ) = 0
M
f n ( x1 , x 2 ,K, x n ) = 0
Problem: how to find
in the vector form:
F ( x) = 0
x .
Newtons Method
Use the Jacobian (matrix of partial derivatives) of the system;

At each stage of the iteration process, an updated approximate solution vector is found
from the current approximate solution according to the equation:
x new = x old J 1 ( x old ).F ( x old )

where
J ( x old ) is the Jacobian matrix at points x old defined as
f1
x1 x1,old
f
2
J ( x old ) = x1
x1, old
M
f n
x1
x1, old
f 1
x 2
x n , old
f 2
L
x n x
n , old
O
M
f n
L
x n x
n , old
L
x1, old
O
M
L
f 1
x n
However, in practice evaluating the inverse of the Jacobian matrix is an expensive

computation. Instead, we follow the equivalent system of linear equations:
J ( x old ). y = F ( x old ) , where y = x new x old , and solve for y .

The updated
x is obtained from: x new = x old + y .
Matlabs Code:
function x = newtonsys(F,JF,x0,tol,max),
%
xold = x0;
iter =1;
while (iter <= max),
y = -feval(JF,xold)\feval(F,xold);
xnew = xold + y';
dif = norm(xnew-xold);
disp([iter xnew dif]);
if (dif <= tol)
x = xnew;
LI - 58

disp('Newton method has converged');
return;
else
xold=xnew;
end
iter = iter+1;
end
disp('Newton method did not converge');
x = xnew;
Example:
Systems of NAE:
f 1 ( x1 , x 2 , x3 ) = x12 + x 22 + x32 1 = 0
f 2 ( x1 , x 2 , x3 ) = x12 + x32 1 / 4 = 0
f 3 ( x1 , x 2 , x3 ) = x12 + x 22 4 x3 = 0
The Jacobian is
2 x1 2 x 2
J ( x) = 2 x1
0
2 x1 2 x 2
Using the initial estimate x o = [1 1
2 x3
2 x3
4
1] , and apply the Newtons method with accuracy 0.00001,
we get:
1.0000
2.0000
3.0000
4.0000
5.0000
6.0000
0.7917
0.5237
0.4473
0.4408
0.4408
0.4408
0.8750
0.8661
0.8660
0.8660
0.8660
0.8660
0.3333
0.2381
0.2361
0.2361
0.2361
0.2361
0.7096
0.2846
0.0764
0.0065
0.0000
0.0000
Minimisation by Gradient Descent
Transform the original problem of finding zeros into a minimisation problem, and then
apply a gradient descent method to find a solution to the minimisation problem
Example:
For the functions
f ( x, y ) = x 2 + y 2 1;
g ( x, y ) = x 2 y
we define
h( x, y ) = ( x 2 + y 2 1) 2 + ( x 2 y ) 2
LI - 59

The minimum value of
h( x, y ) is 0 , which occurs when f ( x, y ) = 0 and g ( x, y ) = 0 .
Matlabs Code:
function xmin = ffmin(F,Fg,x0,tol,max),
%
iter = 1;
xold = x0;
while (iter<=max),
dx = feval(Fg,xold);
xnew = xold + dx;
z0 = feval(F,xold);
z1 = feval(F,xnew);
ch = z1-z0;
while (ch >= 0)
dx = dx/2;
if (norm(dx) < 0.00001)
break;
else
xnew = xold +dx/2;
z1 = feval(F,xnew);
ch = z1-z0;
end
end
if (abs(ch) < tol)
disp('Iterations converged');
break;
end
disp([iter xnew ch])
xold = xnew;
iter = iter+1;
end
xmin = xnew;
Applying this method to solve the example problem, with accuracy 0.00001, and initial estimate
0.5 0.5 , yields:
1.0000
2.0000
3.0000
4.0000
5.0000
0.8750
0.7451
0.7925
0.7845
0.7866
0.6250
0.6113
0.6190
0.6178
0.6181
-0.2683
-0.0360
-0.0080
-0.0002
-0.0000
Matlabs Methods
Matlabs built-in function to find the minimum of a function of several variables uses a NelderMead type of simplex search.
LI - 60

Simplex Search
A simplex n dimensions is the simplest geometrical shape that can enclose a volume in
n dimensions. In two dimension, a simplex is a triangle.
A Nedler-Mead type:
1.
f ( x, y ) . The initial guess for the point of the

r
minimum is x o , and a two dimensional simplex is then a triangle with vertices located
r r
r
r
r
by the vectors x o , x1 = x o + i and x 2 = x o + j , or
Consider a function of two variables,
r a r a + r a
; x 2 =
xo = ; x1 =
b
b
b +
2.
Evaluate the function at these points, and the points are ordered so that
3.
value.
Replace
Function:
r
r
x max , x min and
r
x next to the points where the function is largest, smallest and next to the largest in
r
r
x max by a carefully chosen x in the direction of decreasing f .
x=fmins(F,x0),
where F is a string containing the name of the function to
be minimised, and x0 is the initial estimate of the local
minimum.
LI - 61

NUMERICAL DIFFERENTIATION AND INTEGRATION
Numerical Differentiation
Approximating a first or second (or higher) derivative by a difference quotient
First Derivatives
Taylor Polynomial Use a straight-line approximation to interpolate the given data (or use
two data points to estimate the derivative)
Forward difference formula
f ' ( xi )
f ( xi +1 ) f ( xi ) y i +1 y i
=
xi +1 xi
xi +1 xi
Backward difference formula
f ' ( xi )
f ( xi ) f ( xi 1 ) y i y i 1
=
xi xi 1
xi xi 1
Central difference formula
f ' ( xi )
f ( xi +1 ) f ( xi 1 ) y i +1 y i 1
=
xi +1 xi 1
xi +1 xi 1
Derivation of formulas:
h2
f " ( ) .
2
We get the forward, backward or central by selecting h = xi +1 xi ; h = xi 1 xi or
h = xi +1 xi = xi xi 1 , respectively.
Taylors polynomial:
f ( x + h) = f ( x) + h. f ' ( x) +
Lagrange interpolation Polynomial Use a polynomial approximation that makes use of more
than two data points.
1.
Lagrange form of the polynomial passing through the
n data points
( x1 , y1 ), ( x 2 , y 2 ),K, ( x n , y n ) has n terms on the right hand side:

L( x) = L1 y1 + L2 y 2 + K + Ln y n
where
Lk ( x ) =
( x x1 ) L ( x x k 1 )( x x k +1 ) L ( x x n )
( x k x1 ) L ( x k x k 1 )( x k x k +1 ) L ( x k x n )
LI - 62

( x1 , y1 ), ( x 2 , y 2 ), ( x3 , y3 ) . The Lagrange polynomial can be

written as: L( x ) = L1 ( x ) y1 + L2 ( x ) y 2 + L3 ( x ) y 3 , where
Example for three data points
2.
L1 ( x) =
( x x 2 )( x x3 )
( x x1 )( x x3 )
; L2 ( x ) =
;
( x 2 x1 )( x 2 x3 )
( x1 x 2 )( x1 x3 )
L3 ( x) =
( x x1 )( x x 2 )
( x3 x1 )( x3 x 2 )
Three point forward difference formula
f ' ( xi )
3.
f ( xi + 2 ) + 4 f ( xi +1 ) 3 f ( xi ) y i + 2 + 4 y i +1 3 y i
=
x i + 2 xi
xi + 2 x i
Three point backward difference formula
f ' ( xi )
3 f ( xi ) 4 f ( xi 1 ) + f ( xi 2 ) 3 y i 4 yi 1 + y i 2
=
xi x i 2
xi xi 2
To think about how to get the formula in (2) or (3) from (1)?
Higher Derivatives
Formulas for higher derivatives can be found by differentiating the interpolating polynomial
repeatedly, or by using Taylor expansion:
Example for second derivative
h2
h3
h 4 (4)
f ( x) +
f ( x) +
f (1 )
2!
3!
4!
h2
h3
h 4 ( 4)
f ( x h) = f ( x) h. f ( x) +
f ( x)
f ( x) +
f ( 2 )
2!
3!
4!
f ( x + h) = f ( x) + h. f ( x) +
where x 1
Adding gives
x + h and x h 2 x .
LI - 63

f ( x + h) + f ( x h) = 2 f ( x) + h f ( x) +
2
f ( x)
h 4 (4)
[ f (1 ) + f ( 4) ( 2 )] , or
4!
1
[ f ( x + h) 2 f ( x ) + f ( x h)]
h2
To think about how to get the third, fourth (or so on) derivative?
Richardson Extrapolation
A=
4 A(h / 2) A(h)
3
A(h) and A(h / 2) are low-order polynomials that approximate the true system with
different step sizes, h and h / 2 .
where
Application to Central difference formula:

2
1
[ f ( x + h) f ( x h)] h f ( x) + O(h 4 )
2h
6
1
h2
A(h / 2) = f ( x) = [ f ( x + h / 2) f ( x h / 2)]
f ( x) + O(h 4 )
h
24
A(h) = f ( x) =
Combine two estimates yielding,
A=
4 A(h / 2) A(h)
.
3
Numerical Integration
General formula
b
f ( x).dx ci f ( xi )
i =0
where the coefficients
ci depend on the particular method.
Newton-Cotes Formulas
Trapezoidal Rule
b
f ( x).dx 2 [ f (a) + f (b)]
h =ba
Improved (Composite) Trapezoidal Rule
LI - 64

b
x1
x1
f ( x).dx = f ( x).dx + f ( x).dx

h
[ f (a) + f ( x1 )] + h [ f ( x1 ) + f (b)]
2
2
ba
[ f (a) + 2 f ( x1 ) + f (b)]
4
For
n interval:
f ( x).dx
a
ba
[ f (a) + 2 f ( x1 ) + L + 2 f ( xn1 ) + f (b)]
2.n
Matlabs Code:
function I = trap(f,a,b,n)
%
h = (b-a)/n;
S = feval(f,a);
for i=1:n-1,
x(i)=a+h*i;
S = S+2*feval(f,x(i));
end
S = S+feval(f,b);
I = h*S/2;
Simpson Rule
b
f ( x).dx 3 f (a) + 4 f (
a
b+a
) + f (b)
2
h=
ba
2
Improved (Composite) Simpson Rule

x1
f ( x).dx =
f ( x).dx + f ( x).dx
x1
x1 + a
b + x1
h
h
f
(
a
)
+
4
f
(
)
+
f
(
x
)
+
f
(
x
)
+
4
f
(
) + f (b)
1
1
3
2
2
3
or
b
f ( x).dx 3 [ f (a) + 4 f ( x ) + 2 f ( x ) + 4 f ( x
1
) + f (b)]
In general form, for

b
n even,
f ( x).dx 3 [ f (a) + 4 f ( x ) + 2 f ( x
1
) + 4 f ( x3 ) + 2 f ( x 4 ) + K + 2 f ( x n 2 ) + 4 f ( x n 1 ) + f (b)]
Matlabs Code:
LI - 65

function I = simp(f,a,b,n);
%
% n must be even
h = (b-a)/n;
S = feval(f,a);
for i=1:2:n-1,
x(i)=a+h*i;
end
for i=2:2:n-2,
x(i)=a+h*i;
S=S+2*feval(f,x(i));
end
S=S+feval(f,b);
I=h*S/3;
Midpoint Rule
b
f ( x).dx (b a). f (
a
a+b
)
2
Matlabs Method
1.
Differentiation
Matlabs function p= polyfit(x,y,n) will find the coefficients of the polynomial of degree n that
best fits the data in the least square sense.
The resulting polynomial can be differentiated with polyder(p).
Matlab can also provide the forward or backward difference approximation by using the
function diff.
2.
Integration
Matlab has three built in functions for numerically computing a definite integral.
traps (x,y)
quad(f,xmin,xmax)
quad8(f,xmin,xmax)
use the composite trapezoid rule

use Simpsons rule
use Newton-Cotes eight panel rule
LI - 66

NUMERICAL DIFFERENTIATION AND INTEGRATION
Numerical Differentiation
Approximating a first or second (or higher) derivative by a difference quotient
First Derivatives
Taylor Polynomial Use a straight-line approximation to interpolate the given data (or use
two data points to estimate the derivative)
Forward difference formula
f ' ( xi )
f ( xi +1 ) f ( xi ) y i +1 y i
=
xi +1 xi
xi +1 xi
Backward difference formula
f ' ( xi )
f ( xi ) f ( xi 1 ) y i y i 1
=
xi xi 1
xi xi 1
Central difference formula
f ' ( xi )
f ( xi +1 ) f ( xi 1 ) y i +1 y i 1
=
xi +1 xi 1
xi +1 xi 1
Derivation of formulas:
h2
f " ( ) .
2
We get the forward, backward or central by selecting h = xi +1 xi ; h = xi 1 xi or
h = xi +1 xi = xi xi 1 , respectively.
Taylors polynomial:
f ( x + h) = f ( x) + h. f ' ( x) +
Lagrange interpolation Polynomial Use a polynomial approximation that makes use of more
than two data points.
4.
Lagrange form of the polynomial passing through the
n data points
( x1 , y1 ), ( x 2 , y 2 ),K, ( x n , y n ) has n terms on the right hand side:

L( x) = L1 y1 + L2 y 2 + K + Ln y n
where
Lk ( x ) =
( x x1 ) L ( x x k 1 )( x x k +1 ) L ( x x n )
( x k x1 ) L ( x k x k 1 )( x k x k +1 ) L ( x k x n )
LI - 67

( x1 , y1 ), ( x 2 , y 2 ), ( x3 , y3 ) . The Lagrange polynomial can be

written as: L( x ) = L1 ( x ) y1 + L2 ( x ) y 2 + L3 ( x ) y 3 , where
Example for three data points
5.
L1 ( x) =
( x x 2 )( x x3 )
( x x1 )( x x3 )
; L2 ( x ) =
;
( x 2 x1 )( x 2 x3 )
( x1 x 2 )( x1 x3 )
L3 ( x) =
( x x1 )( x x 2 )
( x3 x1 )( x3 x 2 )
Three point forward difference formula
f ' ( xi )
6.
f ( xi + 2 ) + 4 f ( xi +1 ) 3 f ( xi ) y i + 2 + 4 y i +1 3 y i
=
x i + 2 xi
xi + 2 x i
Three point backward difference formula
f ' ( xi )
3 f ( xi ) 4 f ( xi 1 ) + f ( xi 2 ) 3 y i 4 yi 1 + y i 2
=
xi x i 2
xi xi 2
To think about how to get the formula in (2) or (3) from (1)?
Higher Derivatives
Formulas for higher derivatives can be found by differentiating the interpolating polynomial
repeatedly, or by using Taylor expansion:
Example for second derivative
h2
h3
h 4 (4)
f ( x) +
f ( x) +
f (1 )
2!
3!
4!
h2
h3
h 4 ( 4)
f ( x h) = f ( x) h. f ( x) +
f ( x)
f ( x) +
f ( 2 )
2!
3!
4!
f ( x + h) = f ( x) + h. f ( x) +
where x 1
Adding gives
x + h and x h 2 x .
LI - 68

f ( x + h) + f ( x h) = 2 f ( x) + h f ( x) +
2
f ( x)
h 4 (4)
[ f (1 ) + f ( 4) ( 2 )] , or
4!
1
[ f ( x + h) 2 f ( x ) + f ( x h)]
h2
To think about how to get the third, fourth (or so on) derivative?
Richardson Extrapolation
A=
4 A(h / 2) A(h)
3
A(h) and A(h / 2) are low-order polynomials that approximate the true system with
different step sizes, h and h / 2 .
where
Application to Central difference formula:

2
1
[ f ( x + h) f ( x h)] h f ( x) + O(h 4 )
2h
6
1
h2
A(h / 2) = f ( x) = [ f ( x + h / 2) f ( x h / 2)]
f ( x) + O(h 4 )
h
24
A(h) = f ( x) =
Combine two estimates yielding,
A=
4 A(h / 2) A(h)
.
3
Numerical Integration
General formula
b
f ( x).dx ci f ( xi )
i =0
where the coefficients
ci depend on the particular method.
Newton-Cotes Formulas
Trapezoidal Rule
b
f ( x).dx 2 [ f (a) + f (b)]
h =ba
Improved (Composite) Trapezoidal Rule
LI - 69

b
x1
x1
f ( x).dx = f ( x).dx + f ( x).dx

h
[ f (a) + f ( x1 )] + h [ f ( x1 ) + f (b)]
2
2
ba
[ f (a) + 2 f ( x1 ) + f (b)]
4
For
n interval:
f ( x).dx
a
ba
[ f (a) + 2 f ( x1 ) + L + 2 f ( xn1 ) + f (b)]
2.n
Matlabs Code:
function I = trap(f,a,b,n)
%
h = (b-a)/n;
S = feval(f,a);
for i=1:n-1,
x(i)=a+h*i;
end
S = S+feval(f,b);
I = h*S/2;
Simpson Rule
b
f ( x).dx 3 f (a) + 4 f (
a
b+a
) + f (b)
2
h=
ba
2
Improved (Composite) Simpson Rule

x1
f ( x).dx =
f ( x).dx + f ( x).dx
x1
x1 + a
b + x1
h
h
f
(
a
)
+
4
f
(
)
+
f
(
x
)
+
f
(
x
)
+
4
f
(
) + f (b)
1
1
3
2
2
3
or
b
f ( x).dx 3 [ f (a) + 4 f ( x ) + 2 f ( x ) + 4 f ( x
1
) + f (b)]
In general form, for

b
n even,
f ( x).dx 3 [ f (a) + 4 f ( x ) + 2 f ( x
1
) + 4 f ( x3 ) + 2 f ( x 4 ) + K + 2 f ( x n 2 ) + 4 f ( x n 1 ) + f (b)]
Matlabs Code:
LI - 70

function I = simp(f,a,b,n);
%
% n must be even
h = (b-a)/n;
S = feval(f,a);
for i=1:2:n-1,
x(i)=a+h*i;
end
for i=2:2:n-2,
x(i)=a+h*i;
S=S+2*feval(f,x(i));
end
S=S+feval(f,b);
I=h*S/3;
Midpoint Rule
b
f ( x).dx (b a). f (
a
a+b
)
2
Matlabs Method
3.
Differentiation
Matlabs function p= polyfit(x,y,n) will find the coefficients of the polynomial of degree n that
best fits the data in the least square sense.
The resulting polynomial can be differentiated with polyder(p).
Matlab can also provide the forward or backward difference approximation by using the
function diff.
4.
Integration
Matlab has three built in functions for numerically computing a definite integral.
traps (x,y)
quad(f,xmin,xmax)
quad8(f,xmin,xmax)
use the composite trapezoid rule

use Simpsons rule
use Newton-Cotes eight panel rule
LI - 71

ORDINARY DIFFERENTIAL EQUATIONS Type I
Type I:
ODE is written in the form
y = f ( x, y ) with the value of the function y ( x) given at
xo , i.e. y ( xo ) = y o . So, This type of ODE is ODE with initial value problems.
Basic Idea:
To divide the interval of interest into discrete steps (of fixed width
approximation to the function y at those values of x .
h ), and find
Methods:
One Step Methods:

1. Eulers Method
2. Taylor Method
3. Runge-Kutta
Multi step Methods

1. Explicit Multi-step Methods: Mid-point Method and Adams-Bashforths Method
2. Implicit Multi-step Methods: Adams-Moulton Method
3. Combined Multi step Method: Adams-Brashforth Predictor Corrector Methods
One Step vs Multi Step

The term one step or multi step refers to the way of the methods utilize the previous
solution values. One step only makes use of the computed value of the solution at one
previous point, while multi step ate several previous points.
One Step Methods

Based on Taylor series representation of the unknown function
y ( x) .
Eulers Method
Use only the first term in the Taylor series.
y = f ( x, y ) . The solution of this ODE is approximated by treating the function as

constant, f ( x o , y o ) , and replacing the derivative y by the forward difference quotient:
Consider ODE:
LI - 72

y1 y o = f ( x o , y o )( x1 x o )
or, in general
yi = yi 1 + hf ( xi 1 , y i 1 )
where
h is a sampling step in the given interval.
Matlabs Code:
function [x,y] = eulerm(f,tspan,y0,n),
%
a = tspan(1);
b = tspan(2);
h = (b-a)/n;
x = (a+h:h:b);
y(1) = y0 + h*feval(f,a,y0);
for i=2:n
y(i) = y(i-1) + h*feval(f,x(i-1),y(i-1));
end
x = [a x];
y = [y0 y];
Example:
y = x + y ;
on interval
a x b and y (0) = 2
Results:
0
2.0000
0.1000 2.2000
0.2000 2.4300
0.3000 2.6930
0.4000 2.9923
0.5000 3.3315
0.6000 3.7147
0.7000 4.1462
0.8000 4.6308
0.9000 5.1738
1.0000 5.7812
(Higher Order) Taylor Methods
Use more terms in the Taylor series of

error.
y ( x + h) = y ( x) + hy ( x) +
y in order to obtain a higher order truncation
h2
y ( x) + (h 3 )
2
y ( x) . We need to differentiate f ( x, y ) with respect to

x (using the chain rule, since y is a function of x ) to find a representation for y ( x)
We dont have the formula for
Matlabs Code:
function [x,y] = taylorm(f,fp,tspan,y0,n),
%
LI - 73

a = tspan(1);
b = tspan(2);
h = (b-a)/n;
hh = h*h/2;
x = (a+h:h:b);
y(1) = y0 + h*feval(f,a,y0)+ hh*feval(fp,a,y0);
for i=2:n
y(i) = y(i-1) + h*feval(f,x(i-1),y(i-1)) + hh*feval(fp,x(i-1),y(i-1));
end
x = [a x];
y = [y0 y];
Example:
Reconsider again
y = x + y on interval a x b and y (0) = 2 . We have y = 1 + x + y
Results:
0
2.0000
0.1000 2.2150
0.2000 2.4631
0.3000 2.7477
0.4000 3.0727
0.5000 3.4423
0.6000 3.8613
0.7000 4.3347
0.8000 4.8684
0.9000 5.4685
1.0000 6.1422
Runge-Kutta
Obtaining estimates for the slope of unknown function

differentiating
y that does not require
f ( x, y ) .
Mid-point method use the slope of
y at the midpoint of the interval
k1 = hf ( xi , yi )
k 2 = hf ( xi + 0.5h, y i + 0.5k1 )
yi +1 = y i + k 2
Matlabs Code:
function [x,y] = rk2m(f,tspan,y0,n),
%
a = tspan(1);
b = tspan(2);
h = (b-a)/n;
x = (a+h:h:b);
k1 = h*feval(f,a,y0);
k2 = h*feval(f,a+h/2,y0+k1/2);
y(1) = y0 + k2;
LI - 74

for i=1:n-1
k1 = h*feval(f,x(i),y(i));
k2 = h*feval(f,x(i)+h/2,y(i)+k1/2);
y(i+1) = y(i) + k2;
end
x = [a x];
y = [y0 y];
Example:
y = x + y on interval a x b and y (0) = 2 .
Results:
0
2.0000
0.1000 2.2150
0.2000 2.4631
0.3000 2.7477
0.4000 3.0727
0.5000 3.4423
0.6000 3.8613
0.7000 4.3347
0.8000 4.8684
0.9000 5.4685
1.0000 6.1422
General Form of a second order Runge-Kutta:
k1 = hf ( xi , y i )
k 2 = hf ( xi + c 2 h, yi + a 21 k1 )
yi +1 = y i + w1 k1 + w2 k 2
Parameters:
parameters.
c 2 , a 21 , w1 and w2 . Different variants will be produced for different choice of
Modified Eulers (Euler-Cauchy) Method ( c 2
= 1; a 21 = 1; w1 = w2 = 0.5 )
k1 = hf ( xi , y i )
k 2 = hf ( xi + h, y i + k1 )
1
1
k1 + k 2
2
2
Heuns Method ( c 2 = 2 / 3; a 21 = 2 / 3; w1 = 1 / 4; w2 = 3 / 4 )
yi +1 = y i +
LI - 75

k1 = hf ( xi , y i )
2
2
h, y i + k1 )
3
3
1
3
= y i + k1 + k 2
4
4
k 2 = hf ( xi +
yi +1
To think about Make Matlabs Codes for these variants
Third and Fourth Order Runge-Kutta Methods
Third order Runge-Kutta Method
k1 = hf ( xi , y i )
k 2 = hf ( xi + c 2 h, yi + a 21 k1 )
k 3 = hf ( xi + c3 h, y i + a31 k 2 )
yi +1 = y i + w1 k1 + w2 k 2 + w3 k 3
Parameters:
p = [c 2 , c3 , a 21 , a31 , w1 , w2 , w3 ] .
Variants:
p = [2 / 3,2 / 3,2 / 3,0,2 / 3,2 / 8,3 / 8,3 / 8] .

2. Nearly optimal p = [1 / 2,3 / 4,1 / 2,0,3 / 4,2 / 9,3 / 9,4 / 9]
3. Classical p = [1 / 2,1,1 / 2,1,2,1 / 6,4 / 6,1 / 6]
4. Heun p = [1 / 3,2 / 3,1 / 3,0,2 / 3,1 / 4,0,3 / 4]
1.
Nystrom
Classical Fourth order Runge-Kutta
LI - 76

k1 = hf ( xi , yi )
1
1
h, y i + k 1 )
2
2
1
1
k 3 = hf ( xi + h, y i + k 2 )
2
2
k 4 = hf ( xi + h, y i + k 3 )
k 2 = hf ( xi +
1
1
1
1
yi +1 = y i + k1 + k 2 + k 3 + k 4
6
3
3
6
Matlabs Code:
function [x,y] = rk4m(f,tspan,y0,n),
%
a = tspan(1);
b = tspan(2);
h = (b-a)/n;
x = (a+h:h:b);
k1 = h*feval(f,a,y0);
k4 = h*feval(f,a+h,y0+k3);
y(1) = y0 + (k1)/6+ (k2)/3 + (k3)/3 + (k4)/6;
for i=1:n-1
k1 = h*feval(f,x(i),y(i));
k4 = h*feval(f,x(i)+h,y(i)+k3);
y(i+1) = y(i) + (k1)/6+ (k2)/3 + (k3)/3 + (k4)/6;
end
x = [a x];
y = [y0 y];
Again applied to example before, we get:

0
2.0000
0.1000 2.2155
LI - 77

0.2000
0.3000
0.4000
0.5000
0.6000
0.7000
0.8000
0.9000
1.0000
2.4642
2.7496
3.0755
3.4462
3.8664
4.3413
4.8766
5.4788
6.1548
Multi Step Methods
Involving approximate solution values at several previous points

Example - Two step method
yi +1 = a1 y i + a 2 yi 1 + h[bo f ( xi +1 , y i +1 ) + b1 f ( xi , y i ) + b2 f ( xi 1 , y i 1 )]
p = [a1 , a 2 , bo , b1 , b2 ] depend on particular method

Implicit when bo 0 and Explicit when bo = 0 .
Requirements : initial conditions and starting values
Parameters:
Adams-Bashforths (Explicit) Methods

Two Step
y o is given by the initial condition for differential equation
y1 is found by a one step method, like Runge-Kutta technique,
The next :
yi +1 = y i +
h
[3 f i f i 1 ]
2
Comparing with the general forms, the parameters are chosen as
p = [1,0,0,3 / 2,1 / 2] .
Three Step
LI - 78


y1 and y 2 are found by a one step method, like Runge-Kutta technique,
The next :
yi +1 = y i +
h
[23 f i 16 f i1 + 5 f i 2 ]
12
Four Step
y1 , y 2 and y 3 are found by a one step method, like Runge-Kutta technique,
The next :
yi +1 = y i +
h
[55 f i 59 f i 1 + 37 f i2 9 f i3 ]
24
Adams-Multons (Implicit) Methods

Two Step
y1 is found by a one step method, like Runge-Kutta technique,
The next :
yi +1 = y i +
h
[5 f i +1 + 8 f i f i 1 ]
12
Three Step
The next :
yi +1 = y i +
h
[9 f i +1 + 19 f i 5 f i 1 + f i 2 ]
24
Four Step
y1 , y 2 and y 3 are found by a one step method, like Runge-Kutta technique,
The next :
yi +1 = y i +
h
[251 f i +1 + 646 f i 264 f i 1 + 106 f i 2 19 f i 3 ]
720
LI - 79

Adams (Predictor-Corrector) Methods
Take advantages of the beneficial properties of the implicit method

Avoid the difficulties inherent in solving the implicit equation
Strategy: Explicit method is used to predict a value of
y i +1 , denoted as y i*+1 . Then, this value is
used in the implicit method

Three Step
The next :
h
[23 f i 16 f i1 + 5 f i 2 ]
12
h
= yi +
5 f i*+1 + 8 f i f i 1
12
yi*+1 = y i +
yi +1
MATLABs Methods
Matlab includes three functions ode23, ode45 and ode113 for solving non-stiff ODEs.
The function ode23 and ode45 implement a pair of explicit Runge-Kutta methods, second and
third orders, and fouth and fifth orders, respectively.
The function ode113 is a fully variable step-size ODE solver based on Adams-BashforthMoulton family formulas of orders 1-12.
LI - 80

Systems of Ordinary Differential Equations
Higher Order ODEs
Can be transformed to a set of 1st order ODEs
Example
1.
Consider a 2nd order ODE:

Define:
y = g ( x, y, y )
u = y and v = y , then we get

u = v = f ( x, u , v )
v = g ( x, u , v )
which is a system of two 1st order ODEs.
y (0) = o and y (0) = 1 become the initial condition for the system
u (0) = o and v(0) = 1 .
Initial conditions
2.
y ( n ) = f ( x, y, y , y ,K, y ( n1) ) with initial conditions

y (0) = 1 , y (0) = 2 , L y ( n 1) (0) = n1 becomes a system of first
The n-th order ODE:
y ( 0) = o ,
order ODEs by the following change of variables:
u1 = y,
u 2 = y ,
M
u n = y ( n1)
A system of 1st order ODEs is:
u1 = u 2 ,
u 2 = u 3 ,
M
u n = f ( x, u1 , u 2 ,K, u n )
with the initial conditions
u1 (0) = o , u 2 (0) = 1 , u 3 (0) = 2 , L u n (0) = n1 .
Solution Techniques
1.
Eulers Method
Basic Euler method:
yi +1 = y i + hf ( xi , y i )
LI - 81

For a system of ODEs:
u = f ( x, u , v )
v = g ( x, u , v )
Euler method:
2.
u i +1 = u i + hf ( xi , u i , vi )
vi +1 = vi + hg ( xi , u i , vi )
Midpoint Method
Basic for 2nd order Runge-Kutta:
k1 = hf ( xi , y i )
k 2 = hf ( xi +
1
1
h, y i + k1 )
2
2
yi +1 = y i + k 2
For a system of ODEs:
u = f ( x, u , v )
v = g ( x, u , v )
2nd order Runge-Kutta:
k1 = hf ( xi , u i , vi )
m1 = hg ( xi , u i , vi )
1
1
1
h, u i + k1 , vi + m1 )
2
2
2
1
1
1
m2 = hg ( xi + h, u i + k1 , vi + m1 )
2
2
2
u i +1 = u i + k 2
k 2 = hf ( xi +
vi +1 = vi + m2
Stiff and Ill-Conditioned Problems

Stiff ODE
ODE with a rapidly decaying transient solution
Example:
LI - 82

y = ( y g (t )) + g (t )
with
<< 0 and g (t ) a smooth, slowly time varying
function.
The solution for this ODE is:
y = ( y o g (0))e t + g (t )
The first term in the solution will soon be insignificant compared with
will continue to be governed by
g (t ) , but stability
h , necessitating a very small step size.
Ill-conditioned ODE
ODE for which any error that occurs will increase, regardless the numerical method
employed.
Example:
y = 3 y t 2
The general solution is:
1
2
2
y = Ce 3t + t 2 + t +
3
9
27
Any error in the numerical solution process will introduce the exponential component that
will eventually dominate the true solution
LI - 83

ORDINARY DIFFERENTIAL EQUATIONS TYPE II
ODE type II (Boundary Value Problem)
Second order, two point boundary value problem:
y = f ( x, y, y ),
a xb
with Boundary conditions:

Dirichlets type:
y (a) = ,
Neumans type:
y (a ) = ,
Mixed boundary conditions:
y (b) =
y (b) =
y (a) + c1 y (a) = ,
y (b) + c 2 y (b) =
Solution Techniques
Shooting Method
1. Shooting Method for Linear BVPs
2. Shooting Method for Non-linear BVPs
Finite Difference Method

1. Shooting Method for Linear BVPs
2. Shooting Method for Non-linear BVPs
Shooting Method
y = f ( x, y, y ) on the interval [a, b] . Assume the boundary condition y (a ) is

given and the desired end-condition ( x = b ) on the solution, y b is also given.
Suppose the BVPs:
Linear Shooting
Given:
y = p ( x) y + q ( x) y + r ( x) , with initial conditions y (a ) = and y (b) = .
LI - 84

Algorithm
1.
Solve the IVPs
u = p ( x)u + q ( x)u + r ( x)
v = p ( x)v + q ( x)v
2.
If
u (a ) =
v(a) = 0
u (a ) = 0
v (a ) = 1
v(b) 0 , the solution of the original two-point BVPs is given by
y ( x) = u ( x ) +
u (b)
v(b)
v( x)
LI - 85

Non-linear Shooting
Given:
y = f ( x, y, y ) , with initial conditions y (a) = y a and h( y (b), y (b)) = y b .
Algorithm
1.
2.
Solve the differential equation
u = f ( x, u, u )
Calculate
u (a) = y a
u (a) = 0
h1 = h(u (b), u (b)) and m1 = y b h1 (the error at x = b ). If m1 < tol , accept
u (x) as the solution; otherwise continue.

3.
4.
v = f ( x, v, v )
Calculate
v(a) = y a
v (a) = 1
h2 = h(v(b), v (b)) and m2 = y b h2 (the error at x = b ). If m2 < tol , accept
v(x) as the solution; otherwise continue.

5. Form a new initial guess for y (a ) :
[v (a) u (a)]m1
t = u (a) +
h2 h1
6.
7.
w = f ( x, w, w)
w(a) = y a
w(a) = t
h3 = h( w(b), w(b)) and m3 = y b h3 (the error at x = b ). If m3 < tol , accept

v(x) as the solution; otherwise continue with the linear interpolation to refine the value of t .
Calculate
LI - 86

PARTIAL DIFFERENTIAL EQUATION PDE
A linear second order PDE
Three basic categorises:

1.
2.
3.
Parabolic Equation
Hyperbolic Equation
Elliptic Equation
Solution Techniques
1.
2.
Finite Difference Method

Finite Element Method
General Form of a Linear Second-order PDE:
a u xx + b u xt + c u tt + d u x + e u t + f u + g = 0
where
u xx =
u u
u
u
2u
2u
u
=
=
,
,
, ut =
and u x =
.
u
xt
tt
2
2
x t
t
x
x
t
The PDE is
b 2 4ac = 0
2
2. Hyperbolic if b 4ac > 0
2
3. Elliptic
if b 4ac < 0
1.
Parabolic
if
Heat Equation: Parabolic PDE

A finite difference solution of the one dimensional heat equation:
u t = cu xx
for
0 < x < a and
0<t T
with initial conditions
u ( x ,0 ) = f ( x )
0<x<a
LI - 87

and boundary conditions
u (0, t ) = g1 (t ),
u (a, t ) = g 2 (t )
0 < t T at x = 0 and x = a
begins with the definition of a mesh points at which the solution is sought.
Space interval:
h = x = a / n xi for i = 0,1, K , n
Time interval:
k = t = T / m t j for j = 0,1, K , m
We have:
ut
1
u i , j +1 u i , j
k
and
cu xx
c
u i 1, j 2u i , j + u i +1, j
h2
Explicit method to solve the Heat Equation
1
c
u i , j +1 u i , j = 2 u i 1, j 2u i , j + u i +1, j
k
h
By introducing
r=
ck
, and we try to solve for xi , j +1 , we have
h2
u i , j +1 = ru i 1, j + (1 2r )u i , j + ru i +1, j
The stability condition is:
r=
for
i = 1, K , n 1
ck
0 .5
h2
Implicit method to solve the Heat Equation
Consider:
u t = cu xx
for
0 < x < a and
0<t T
Replacing the space derivative by a centred difference at the forward time step
derivative by a forward difference gives
1
c
u i , j +1 u i , j = 2 u i 1, j +1 2u i , j +1 + u i +1, j +1
k
h
j + 1 and the time
or
u i , j = ru i 1, j +1 + (1 + 2r )u i , j +1 ru i +1, j +1
LI - 88

where
r=
ck
.
h2
This method is unconditionally stable.
LI - 89

Method to solve the Heat Equation with Insulated Boundary
The form of boundary conditions depends on the physical situation being described.
Sometimes the boundary conditions are given in terms of partial derivative
x = a , as follows:
u x at x = 0 or at
u x (0, t ) = 0 or u x (a, t ) = 0
Strategy is to extend the grid by including the point
x n +1 at each time step.
Using the central difference formula, we find the boundary condition
u x (a, t ) becomes
1
u n +1, j u n 1, j = 0
2k
or
un +1, j = un 1, j
Applying the general update equation at
i = n gives
u n , j +1 = r u n 1, j + (1 2r )u n, j + r u n+1, j
which includes the point
u n +1 .
Substituting the information from the boundary condition, we get:
u n , j +1 = r u n 1, j + (1 2r )u n, j + r u n+1, j
The system of equation is:
u1, j +1 = r u 0, j + (1 2r )u1, j + ru 2, j
i =1
u i , j +1 = r u i 1, j + (1 2r )u i , j + ru i +1, j
i = 2, K , n 1
u n , j +1 = 2r u n 1, j + (1 2r )u n , j
i=n
Wave Equation: Parabolic PDE

A finite difference solution of PDE:
u tt c 2 u xx = 0
with initial conditions:
for
0 x a and 0 t
u ( x,0) = f 1 ( x), u t ( x,0) = f 2 ( x)
for
0< x<a
LI - 90

u (0, t ) = g1 (t ), u (a, t ) = g 2 (t )
and boundary conditions:
h = x = a / n, k = t , with p =
Utilise the mesh:
for
t0
ck
t
.
=c
2
x
h
Explicit Method
The general form of the difference equation is:
u i , j +1 = p 2 u i 1, j + 2(1 p 2 )u i , j + p 2 u i +1, j u i , j 1
The equation for
u at the first time step:
u i ,1 = 0.5 p 2 u i 1,0 + (1 p 2 )u i , 0 + 0.5 p 2 u i +1, 0 + kf 2 ( xi )

Stability condition:
p 1
Finite Element Method to Solve Elliptic PDE

Elliptic PDE:
u xx + u yy + r ( x, y )u = f ( x, y )
u ( x, y ) = g ( x, y )
on the region
on boundary of
The corresponding functional to be minimised is:
I [u ] = u x2 + u y2 r ( x, y )u 2 + 2 f ( x, y )u dxdy
R
A finite element solution of the PDE is a function
LI - 91

m
U = c j j
j =1
where the
j are basis function corresponding to each node.
The solution process consists of the following steps:
R by specifying the locations of the nodes
1.
Define the subdivision of
2.
Define each of the basis functions
3.
Determine coefficient
4.
Determine coefficient
j , j = 1, K , m .
c j for the basis function that corresponds to boundary nodes

( j = n + 1, K , m ), so that the solution U satisfies the boundary conditions at those nodes
so that the solution
c j for the basis function that corresponds to interior nodes ( j = 1, K n ),
U minimises the integral I [u ] .
Finding The coefficients

To find the coefficients corresponds to the interior nodes, we must minimises:
[U
2
x
+ U y2 r ( x, y )U 2 + 2 f ( x, y )U dxdy
with
U = c j j
j =1
U
= 0 for 1 i n . This gives a linear system of equation:
ci
Ac = d
The minimum occurs where
where
[ ]
A = a ij ,
1 i; j n
[ ] +[ ] [ ]
a ij = [ i ]x j
i y
j y
r ( x, y ) i j dxdy
and
LI - 92

d i = f ( x, y ) i dxdy
R
c b
j
j = n +1
in which
[ ] +[ ] [ ]
bij = [ i ]x j
i y
j y
r ( x, y ) i j dxdy
for
1 i n and n + 1 j m .
Finding the Basis Function
R is divided into p triangular sub-regions, T1 , T2 , K , T p , there is a piece wise linear basis

function corresponding to each node. The basis function corresponding to node j has the value one
at node j and zero at at all other nodes.
If
LI - 93

Numerical Methods Overview of The Course

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Numerical Methods Overview of The Course

Uploaded by

Copyright:

Available Formats

ST4291 Numerical Analysis

(2cp, MSc Program International)

OVERVIEW OF THE COURSE

PART I - ALGEBRAIC EQUATIONS

Solution of Linear Algebraic Equations (LAE)

Direct Methods: Gaussian Elimination and Its Variants

Iterative Methods: Jacobi Method; Gauss-Seidel Method; Successive Over Relaxation

Numerical Linear Algebra: (Convergence & Existence of Solution)

Solution of Nonlinear Algebraic Equations (NAE)

Variables: Single and Multivariable;

Methods Different in approximating a function

Other Methods (for several variables)

PART II - INTEGRATION AND DIFFERENTIATION

Differentiation: Use difference formula

Integration: Newton-Cotes Formulas, like Trapezoid, Simpson, Mid-point Rules;

Linear Analysis and Matrices

ST4291 Numerical Analysis

Solution of (Systems) Ordinary Differential Equations (ODE)

Type 1 (Initial Value Problems): Taylor Methods; Runge-Kutta Methods; Multi-step

Type 2 (Boundary Value Problems: Shooting Methods; Finite Difference Methods

Solution of Partial Differential Equations (PDE)

Solving of Engineering Problems

Analytically solved Exact Solution!

Numerically Solved Approximate Solution!

Techniques Direct or Iterative

Linear Analysis and Matrices

ST4291 Numerical Analysis

Solving a set of equations

Size of the problem

Accuracy of the resulting solution

Numerical Round-off errors

Example: We have three-digit arithmetic

f ( x)dx 2 [ f (a) + f (a + h)]

Linear Analysis and Matrices

ST4291 Numerical Analysis

Linear Analysis and Matrices

ST4291 Numerical Analysis

An expression of equality between constants, variables, functions of variables (sums, products,

x if it contains x to only the first power.

x + 5 y 2 z = 0 A coupled linear homogeneous

sin x; cos x; tan x

Differential and Integration

Linear Analysis and Matrices

ST4291 Numerical Analysis

Show that one of the equations is unnecessary (and which one?)

Then, solve for

MATRIX PROPERTIES AND ALGEBRA

Why we need to review matrix?

Matlab is a matrix-based computational tool.

Special Matrix Characteristics

Linear Analysis and Matrices

ST4291 Numerical Analysis

a ij of the square matrix A are zero if i j .

Block diagonal matrix

aii , and a few nearby parallel diagonal.

a ij = 0 for all i > j (or i < j ).

A (or AT ) corresponds to interchanging the rows and columns of

A = B is to be valid if and only if a ij = bij for all i, j .

cij = k =1 aik bkj .

Linear Analysis and Matrices

ST4291 Numerical Analysis

The norm (or length) of a vector x is defined as

A vector of length one