You are on page 1of 36

Department of Mathematics

Amer Rasheed
Lecture 1
Numerical Analysis
Math344, Math541

Amer Rasheed (MATH344) Department of Mathematics 1 / 36


Course

Course Evaluations

Theory Assignments : 15 Marks

Programming Assignments : 15 marks

Mid Term : 30 marks

Final Exam : Theory = 30 Marks + Project = 10 marks

Amer Rasheed (MATH344) Department of Mathematics 2 / 36


Course Outlines

Course Outlines

Introduction to course, Brief History, Error Analysis


Solution of Non-linear Equations
Interpolation and polynomial Approximation
Numerical Differentiation
Numerical Integration
Iterative methods for the solution of Linear Systems
Eigenvalue problems
Solutions of Ordinary Differential Equations using numerical tech-
niques
Finite difference methods for Ordinary/Partial Differential Equations
Finite Element Methods for Boundary Value Problems

Amer Rasheed (MATH344) Department of Mathematics 3 / 36


Recommended Books

Recommended Books

Elementary Numerical Analysis, Atkinson and Han


Numerical Analysis, Burden and Faires
A First Course in the Numerical Analysis of Differential Equa-
tions, Arieh Iserles
Numerical solutions of partial differential equations by the fi-
nite element method, Claes Johnson
Finite Element Methods and Their Applications Zhangxin Chen

The Finite Element Methods for Elliptic Problems Philippe G.


Ciarlet

Amer Rasheed (MATH344) Department of Mathematics 4 / 36


Motivations

Motivations

Can one finds the roots of the equations of type

ex 2x2 + 3 = 0.
How to find integral

Z Z
2
sin xdx, ex dx

Amer Rasheed (MATH344) Department of Mathematics 5 / 36


Motivations

Motivations
How to fit a polynomial (function) to given data
How to find rate of change, derivative, in the following data

x y

1 10

2 12

3 15

4 17

5 21

Amer Rasheed (MATH344) Department of Mathematics 6 / 36


Motivations

Motivations

How to solve nonlinear differential equations

d2 y dy
+ y = sin y.
dx2 dx
u 2u
+ k 2 = u3 .
t x

Amer Rasheed (MATH344) Department of Mathematics 7 / 36


Brief History

Brief History

Isaac Newton (16421726)


Root finding techniques, Interpolation for function approximations,
Numerical differentiation and Integration

Leonhard Euler (17071783)


Numerical Differentiation and Solving ODEs

Joseph Loius Lagrange (17361813)


Interpolation of functions

Karl Friedrich Gauss (17771855)


Numerical Integration

Amer Rasheed (MATH344) Department of Mathematics 8 / 36


Modern Techniques

Modern Techniques

Finite Difference Methods


Richard Courant, Kurt Friedrich and Hans Lewy (CFL) (1928)
Finite Element Methods
Lord Rayleigh (1894), Ritz (1908), Galerkin (1908), Courant (1943),
Faedo (1949), Argyris and Clough (1960), Philippe G Ciarlet (1970).

Amer Rasheed (MATH344) Department of Mathematics 9 / 36


Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Suppose that p is an approximation to p.

Absolute Error : denoted by Ep , is defined by

Ep = p p

Relative Error : denoted by Rp , is defined by

p p Ep
Rp = = , p 6= 0.
p p

Amer Rasheed (MATH344) Department of Mathematics 10 / 36


Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Absolute Error is the amount of physical error in a measurement.

Lets, say, a meter stick is used to measure a given distance. The


error is rather hastily made, but it is good to 1mm. This is the
absolute error of the measurement.

Amer Rasheed (MATH344) Department of Mathematics 11 / 36


Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Relative Error gives an indication of how good a measurement is relative


to the size of the thing being measured.

Lets say that two students measure two objects with a meter stick. One
student measures the height of a room and gets a value of 3.215 me-
ters 1mm (0.001m). Another student measures the height of a small
cylinder and measures 0.075 meters 1mm (0.001m). Clearly, the ove-
rall accuracy of the ceiling height is much better than that of the 7.5cm
cylinder. The comparative accuracy of these measurements can be de-
termined by looking at their relative errors

0.001
Rceiling = 100 = 0.0003%
3.125

0.001
Rcylinder = 100 = 0.01%
0.075

Amer Rasheed (MATH344) Department of Mathematics 12 / 36


Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Definition : The number p is said to approximate p to d significant


digits, if d is the largest positive integer for which

|p p| 10d

|p| 2
or
10d
|Rp |
2

Amer Rasheed (MATH344) Department of Mathematics 13 / 36


Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Example : Number of significant digits


103
1). |Rp | = 0.0004896 = 0.4896 103 < , d=3
2
100
2). |Rp | = 0.2561 = 0.2561 100 < , d=0
2

Amer Rasheed (MATH344) Department of Mathematics 14 / 36


Error Analysis Truncation Errors

Truncation Errors

The notion of the truncation errors usually refers to errors introduced


when a more complicated mathematical expression is replaced with a
more elementary formula (truncated Taylors series).
e.g. the Taylors series about x = 0

2 x4 x6 x2n
e x = 1 + x2 + + + + + ...
2! 3! n!
might be replaced with the finite number of terms while approximating
2
integral of ex numerically.

Amer Rasheed (MATH344) Department of Mathematics 15 / 36


Error Analysis Truncation Errors

Truncation Errors

Given that
Z 1/2
2
ex dx = 0.544987104184 = p, (true value)
0

Determine the accuracy of the approximations obtained by replacing


the integrand with a truncated Taylors series

x4 x6 x8
P8 (x) = 1 + x2 + + +
2! 3! 4!
After calculations, we get
Z 1/2
P8 (x)dx = 0.544986720817 = p , (approx. value)
0

Amer Rasheed (MATH344) Department of Mathematics 16 / 36


Error Analysis Truncation Errors

Truncation Errors

since
106
|Rp | = 7.03442 107
2

The approximation p agrees with the true value p to six significant
digits.

Amer Rasheed (MATH344) Department of Mathematics 17 / 36


Error Analysis Chopping Off & Rounding Off Errors

Chopping Off & Rounding Off Errors

Normalized Decimal Form Any real number p can be expressed in a


form
p = 0.d1 d2 d3 . . . dk dk+1 10n

where 1 d1 9 & 0 dk 9, for k > 1, & nZ

Amer Rasheed (MATH344) Department of Mathematics 18 / 36


Error Analysis Chopping Off & Rounding Off Errors

Chopping Off & Rounding Off Errors

The chopped floating point representation of p, denoted by flchop (p), is


defined by

flchop (p) = 0.d1 d2 d3 . . . dk 10n

where 1 d1 9 & 0 dj 9, for 1 < j k, & nZ


Note : kth digit of flchop (p) agrees with kth digit of p.

Amer Rasheed (MATH344) Department of Mathematics 19 / 36


Error Analysis Chopping Off & Rounding Off Errors

Chopping Off & Rounding Off Errors

The rounded floating point representation of p, denoted by flround (p), is


defined by

flround (p) = 0.d1 d2 d3 . . . rk 10n

where 1 d1 9 & 0 dj 9, for 1 < j k, & nZ


and the last digit rk is obtained by rounding off the number

dk .dk+1 dk+2 . . .

to the nearest integer.

Amer Rasheed (MATH344) Department of Mathematics 20 / 36


Error Analysis Chopping Off & Rounding Off Errors

Chopping Off & Rounding Off Errors

Example
22
= = 3.142857142857142857
7

has the following six-digit representations

flchop () = 0.314285 101


flround () = 0.314286 101

Amer Rasheed (MATH344) Department of Mathematics 21 / 36


Error Analysis Loss of Significance

Loss of Significance

LoS is an undesirable error in calculations using floating point arithe-


matic. It occurs when an operation on two numbers increases relative
error substantially more than the absolute error, e.g., in subtracting two
nearly equal numbers.

The number of significant digits in this case reduce unacceptably.

How to avoid LoS ... ?

Amer Rasheed (MATH344) Department of Mathematics 22 / 36


Error Analysis Loss of Significance

Loss of Significance

Example 1
Compare the results of calculating f (500) and g(500) using 6-digits
rounding off calculations with the true value 11.174755300747, where


f (x) = x x+1 x
x
g(x) =
x+1+ x
After computations

f (500) = 11.1500, & g(500) = 11.1748

g(x) gives more accurate estimate.

Note : f and g are algebraically equivalent functions.

Amer Rasheed (MATH344) Department of Mathematics 23 / 36


Error Analysis Loss of Significance

Loss of Significance
Example 2
Compare the results of calculating f (0.01) and g(0.01) using 6-digits
rounding off calculations with the true value 0.50167084168057542,
where

ex 1 x
f (x) =
x2
1 x x2
g(x) = + +
2 6 24
After computations

f (0.01) = 0.500000, & g(0.01) = 0.501671

g(x) gives more accurate estimate.

Note : g is the Taylors expansion of f .


Amer Rasheed (MATH344) Department of Mathematics 24 / 36
Error Analysis Loss of Significance

Loss of Significance

Example 3
For polynomial evaluations, the rearrangement of terms would
sometimes produce better results

P (x) = x3 3x2 + 3x 1
Q(x) = ((x 3)x + 3) x 1
Use 3-digits rounding arithematic to compute P (2.19) and Q(2.19).
Compare with true value 1.685159.
After computations

P (2.19) = 1.67, & Q(2.19) = 1.69

Q(x) gives more accurate estimate.

Amer Rasheed (MATH344) Department of Mathematics 25 / 36


Propagation of Errors Error Propagation in SUM

Propagation of Errors

How errors propagate in the successive computations ?

Let p is an approximation to p and q is an approximation to q. Then


we can write
p = p + p & q = q + q
where p is absolute error in p and q is absolute error in q.

Error Propagation in SUM

p + q = (p + p ) + (q + q )

p + q = (p + q ) + (p + q )
Thus for addition, the error in the sum is the sum of errors of the
addends.
Amer Rasheed (MATH344) Department of Mathematics 26 / 36
Propagation of Errors Error Propagation in Product

Propagation of Errors
Error Propagation in Product

pq = (p + p )(q + q )
pq = p q + p q + q p + p q

If p and q are greater than 1 in absolute values, then there might be


magnification of errors due to terms p q and q p .
We should investigate the relative error of the product pq

pq p q = p q + q p + p q
pq p q p q q p p q
= + +
pq pq pq pq
p q
Suppose that 1, 1 and Rp Rq 0
p q

Amer Rasheed (MATH344) Department of Mathematics 27 / 36


Propagation of Errors Error Propagation in Product

Propagation of Errors

Error Propagation in Product

pq p q p q q p p q
= + +
pq pq pq pq
pq p q q p
= +
pq q p
pq p q
= Rp + Rq (1)
pq

Thus the relative error in the product is the sum of relative errors in p
and q .

Amer Rasheed (MATH344) Department of Mathematics 28 / 36


Propagation of Errors Error Propagation in Product

Propagation of Errors

Exercise : How error propagate in


p
1).
q
2). pqr

Amer Rasheed (MATH344) Department of Mathematics 29 / 36


Order of Approximation

Order of Approximation

Definition : Assume that f (h) is approximated by the function p(h) and


that there exists a real constant M > 0 and a positive integer n, so that

|f (h) p(h)|
< M, for sufficiently small h
|hn |
or
|f (h) p(h)| < M |hn |
we say that p(h) approximates f (h) with order of approximation O(hn )
and write
f (h) = p(h) + O(hn )
The term O(hn ) is pronounced as oh of hn

Amer Rasheed (MATH344) Department of Mathematics 30 / 36


Order of Approximation

Order of Approximation

Properties of O(hn )

1). O(hn ) + O(hn ) = O(hn )

2). O(hp ) + O(hq ) = O(hr ), r = min{p, q}

3). O(hp )O(hq ) = O(hs ), s=p+q

Amer Rasheed (MATH344) Department of Mathematics 31 / 36


Order of Approximation

Order of Approximation

Theorem
Assume that

f (h) = p(h) + O(hn ) & g(h) = q(h) + O(hm )

and p = min{n, m}. Then

1). f (h) + g(h) = p(h) + q(h) + O(hp )

2). f (h)g(h) = p(h)q(h) + O(hp )


f (h) p(h)
3). = + O(hp ), g(h) 6= 0 & q(h) 6= 0.
g(h) q(h)

Amer Rasheed (MATH344) Department of Mathematics 32 / 36


Taylors Theorem

Taylors Theorem

Theorem
Assume that f C n+1 [a, b]. If both x0 and x0 + h lie in [a, b]. Then
n
X f k (x0 )
f (x0 + h) = hk + O(hn+1 ).
k!
k=0

Example
Taylors expansion of O(h4 ) about x0 = 0 of ex is

h2 h3
eh = 1 + h + + + O(h4 )
2! 3!

Amer Rasheed (MATH344) Department of Mathematics 33 / 36


Order of Approximation

Order of Approximation

Example

Consider the Taylors expansions

h2 h3
eh = 1 + h + + + O(h4 )
2! 3!

h2 h4
cos h = 1
+ + O(h6 )
2! 4!
Determine the order of approximations of their SUM and PRODUCT.

Amer Rasheed (MATH344) Department of Mathematics 34 / 36


Order of Convergence

Order of Convergence
The order of convergence of a sequence is analogous to the order of
approximation.
Definition : Assume that lim xn = x and {rn }
n=1 is a sequence with
n
lim rn = 0. We say that {xn }
n=1 converges to x with the order of
n
convergence O(rn ), if there exists a constant K > 0 such that
|xn x|
< K, for sufficiently large n
|rn |
or
|xn x| < K|rn |
This is indicated by writing

xn = x + O(rn )

or xn x with order of convergence O(rn ).


Amer Rasheed (MATH344) Department of Mathematics 35 / 36
Order of Convergence

Order of Convergence
Example

Determine the order of convergence of the sequence


cos n
n2

Let
cos n 1
xn = & rn =
n2 n2
As
cos n 1
0 & 0
n2 n2
and
|xn x| cos n/n2
= = | cos n| 1, n
|rn | 1/n2
cos n 1
Thus the sequence 2
converges with O( 2 ).
n n
Amer Rasheed (MATH344) Department of Mathematics 36 / 36