You are on page 1of 36

Department of Mathematics

Amer Rasheed
Lecture 1
Numerical Analysis
Math344, Math541

Amer Rasheed (MATH344) Department of Mathematics 1 / 36

Course

Course Evaluations

Course Outlines

Course Outlines

Introduction to course, Brief History, Error Analysis

Solution of Non-linear Equations
Interpolation and polynomial Approximation
Numerical Differentiation
Numerical Integration
Iterative methods for the solution of Linear Systems
Eigenvalue problems
Solutions of Ordinary Differential Equations using numerical tech-
niques
Finite difference methods for Ordinary/Partial Differential Equations
Finite Element Methods for Boundary Value Problems

Amer Rasheed (MATH344) Department of Mathematics 3 / 36

Recommended Books

Recommended Books

Elementary Numerical Analysis, Atkinson and Han

Numerical Analysis, Burden and Faires
A First Course in the Numerical Analysis of Differential Equa-
tions, Arieh Iserles
Numerical solutions of partial differential equations by the fi-
nite element method, Claes Johnson
Finite Element Methods and Their Applications Zhangxin Chen

Ciarlet

Motivations

Motivations

Can one finds the roots of the equations of type

ex 2x2 + 3 = 0.
How to find integral

Z Z
2
sin xdx, ex dx

Amer Rasheed (MATH344) Department of Mathematics 5 / 36

Motivations

Motivations
How to fit a polynomial (function) to given data
How to find rate of change, derivative, in the following data

x y

1 10

2 12

3 15

4 17

5 21

Motivations

Motivations

d2 y dy
+ y = sin y.
dx2 dx
u 2u
+ k 2 = u3 .
t x

Brief History

Brief History

Isaac Newton (16421726)

Root finding techniques, Interpolation for function approximations,
Numerical differentiation and Integration

Leonhard Euler (17071783)

Numerical Differentiation and Solving ODEs

Joseph Loius Lagrange (17361813)

Interpolation of functions

Karl Friedrich Gauss (17771855)

Numerical Integration

Amer Rasheed (MATH344) Department of Mathematics 8 / 36

Modern Techniques

Modern Techniques

Finite Difference Methods

Richard Courant, Kurt Friedrich and Hans Lewy (CFL) (1928)
Finite Element Methods
Lord Rayleigh (1894), Ritz (1908), Galerkin (1908), Courant (1943),
Faedo (1949), Argyris and Clough (1960), Philippe G Ciarlet (1970).

Amer Rasheed (MATH344) Department of Mathematics 9 / 36

Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Ep = p p

p p Ep
Rp = = , p 6= 0.
p p

Amer Rasheed (MATH344) Department of Mathematics 10 / 36

Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Lets, say, a meter stick is used to measure a given distance. The

error is rather hastily made, but it is good to 1mm. This is the
absolute error of the measurement.

Amer Rasheed (MATH344) Department of Mathematics 11 / 36

Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Relative Error gives an indication of how good a measurement is relative

to the size of the thing being measured.

Lets say that two students measure two objects with a meter stick. One
student measures the height of a room and gets a value of 3.215 me-
ters 1mm (0.001m). Another student measures the height of a small
cylinder and measures 0.075 meters 1mm (0.001m). Clearly, the ove-
rall accuracy of the ceiling height is much better than that of the 7.5cm
cylinder. The comparative accuracy of these measurements can be de-
termined by looking at their relative errors

0.001
Rceiling = 100 = 0.0003%
3.125

0.001
Rcylinder = 100 = 0.01%
0.075

Amer Rasheed (MATH344) Department of Mathematics 12 / 36

Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Definition : The number p is said to approximate p to d significant

digits, if d is the largest positive integer for which

|p p| 10d

|p| 2
or
10d
|Rp |
2

Amer Rasheed (MATH344) Department of Mathematics 13 / 36

Error Analysis Absolute and Relative Errors

Error Analysis
Absolute Error and Relative Error

Example : Number of significant digits

103
1). |Rp | = 0.0004896 = 0.4896 103 < , d=3
2
100
2). |Rp | = 0.2561 = 0.2561 100 < , d=0
2

Amer Rasheed (MATH344) Department of Mathematics 14 / 36

Error Analysis Truncation Errors

Truncation Errors

The notion of the truncation errors usually refers to errors introduced

when a more complicated mathematical expression is replaced with a
more elementary formula (truncated Taylors series).
e.g. the Taylors series about x = 0

2 x4 x6 x2n
e x = 1 + x2 + + + + + ...
2! 3! n!
might be replaced with the finite number of terms while approximating
2
integral of ex numerically.

Amer Rasheed (MATH344) Department of Mathematics 15 / 36

Error Analysis Truncation Errors

Truncation Errors

Given that
Z 1/2
2
ex dx = 0.544987104184 = p, (true value)
0

Determine the accuracy of the approximations obtained by replacing

the integrand with a truncated Taylors series

x4 x6 x8
P8 (x) = 1 + x2 + + +
2! 3! 4!
After calculations, we get
Z 1/2
P8 (x)dx = 0.544986720817 = p , (approx. value)
0

Amer Rasheed (MATH344) Department of Mathematics 16 / 36

Error Analysis Truncation Errors

Truncation Errors

since
106
|Rp | = 7.03442 107
2

The approximation p agrees with the true value p to six significant
digits.

Amer Rasheed (MATH344) Department of Mathematics 17 / 36

Error Analysis Chopping Off & Rounding Off Errors

Normalized Decimal Form Any real number p can be expressed in a

form
p = 0.d1 d2 d3 . . . dk dk+1 10n

Amer Rasheed (MATH344) Department of Mathematics 18 / 36

Error Analysis Chopping Off & Rounding Off Errors

defined by

where 1 d1 9 & 0 dj 9, for 1 < j k, & nZ

Note : kth digit of flchop (p) agrees with kth digit of p.

Amer Rasheed (MATH344) Department of Mathematics 19 / 36

Error Analysis Chopping Off & Rounding Off Errors

defined by

where 1 d1 9 & 0 dj 9, for 1 < j k, & nZ

and the last digit rk is obtained by rounding off the number

dk .dk+1 dk+2 . . .

Amer Rasheed (MATH344) Department of Mathematics 20 / 36

Error Analysis Chopping Off & Rounding Off Errors

Chopping Off & Rounding Off Errors

Example
22
= = 3.142857142857142857
7

flchop () = 0.314285 101

flround () = 0.314286 101

Amer Rasheed (MATH344) Department of Mathematics 21 / 36

Error Analysis Loss of Significance

Loss of Significance

LoS is an undesirable error in calculations using floating point arithe-

matic. It occurs when an operation on two numbers increases relative
error substantially more than the absolute error, e.g., in subtracting two
nearly equal numbers.

Amer Rasheed (MATH344) Department of Mathematics 22 / 36

Error Analysis Loss of Significance

Loss of Significance

Example 1
Compare the results of calculating f (500) and g(500) using 6-digits
rounding off calculations with the true value 11.174755300747, where


f (x) = x x+1 x
x
g(x) =
x+1+ x
After computations

Amer Rasheed (MATH344) Department of Mathematics 23 / 36

Error Analysis Loss of Significance

Loss of Significance
Example 2
Compare the results of calculating f (0.01) and g(0.01) using 6-digits
rounding off calculations with the true value 0.50167084168057542,
where

ex 1 x
f (x) =
x2
1 x x2
g(x) = + +
2 6 24
After computations

Note : g is the Taylors expansion of f .

Amer Rasheed (MATH344) Department of Mathematics 24 / 36
Error Analysis Loss of Significance

Loss of Significance

Example 3
For polynomial evaluations, the rearrangement of terms would
sometimes produce better results

P (x) = x3 3x2 + 3x 1
Q(x) = ((x 3)x + 3) x 1
Use 3-digits rounding arithematic to compute P (2.19) and Q(2.19).
Compare with true value 1.685159.
After computations

Amer Rasheed (MATH344) Department of Mathematics 25 / 36

Propagation of Errors Error Propagation in SUM

Propagation of Errors

Let p is an approximation to p and q is an approximation to q. Then

we can write
p = p + p & q = q + q
where p is absolute error in p and q is absolute error in q.

Error Propagation in SUM

p + q = (p + p ) + (q + q )

p + q = (p + q ) + (p + q )
Thus for addition, the error in the sum is the sum of errors of the
Amer Rasheed (MATH344) Department of Mathematics 26 / 36
Propagation of Errors Error Propagation in Product

Propagation of Errors
Error Propagation in Product

pq = (p + p )(q + q )
pq = p q + p q + q p + p q

If p and q are greater than 1 in absolute values, then there might be

magnification of errors due to terms p q and q p .
We should investigate the relative error of the product pq

pq p q = p q + q p + p q
pq p q p q q p p q
= + +
pq pq pq pq
p q
Suppose that 1, 1 and Rp Rq 0
p q

Amer Rasheed (MATH344) Department of Mathematics 27 / 36

Propagation of Errors Error Propagation in Product

Propagation of Errors

Error Propagation in Product

pq p q p q q p p q
= + +
pq pq pq pq
pq p q q p
= +
pq q p
pq p q
= Rp + Rq (1)
pq

Thus the relative error in the product is the sum of relative errors in p
and q .

Amer Rasheed (MATH344) Department of Mathematics 28 / 36

Propagation of Errors Error Propagation in Product

Propagation of Errors

p
1).
q
2). pqr

Amer Rasheed (MATH344) Department of Mathematics 29 / 36

Order of Approximation

Order of Approximation

Definition : Assume that f (h) is approximated by the function p(h) and

that there exists a real constant M > 0 and a positive integer n, so that

|f (h) p(h)|
< M, for sufficiently small h
|hn |
or
|f (h) p(h)| < M |hn |
we say that p(h) approximates f (h) with order of approximation O(hn )
and write
f (h) = p(h) + O(hn )
The term O(hn ) is pronounced as oh of hn

Amer Rasheed (MATH344) Department of Mathematics 30 / 36

Order of Approximation

Order of Approximation

Properties of O(hn )

Amer Rasheed (MATH344) Department of Mathematics 31 / 36

Order of Approximation

Order of Approximation

Theorem
Assume that

2). f (h)g(h) = p(h)q(h) + O(hp )

f (h) p(h)
3). = + O(hp ), g(h) 6= 0 & q(h) 6= 0.
g(h) q(h)

Amer Rasheed (MATH344) Department of Mathematics 32 / 36

Taylors Theorem

Taylors Theorem

Theorem
Assume that f C n+1 [a, b]. If both x0 and x0 + h lie in [a, b]. Then
n
X f k (x0 )
f (x0 + h) = hk + O(hn+1 ).
k!
k=0

Example
Taylors expansion of O(h4 ) about x0 = 0 of ex is

h2 h3
eh = 1 + h + + + O(h4 )
2! 3!

Amer Rasheed (MATH344) Department of Mathematics 33 / 36

Order of Approximation

Order of Approximation

Example

Consider the Taylors expansions

h2 h3
eh = 1 + h + + + O(h4 )
2! 3!

h2 h4
cos h = 1
+ + O(h6 )
2! 4!
Determine the order of approximations of their SUM and PRODUCT.

Amer Rasheed (MATH344) Department of Mathematics 34 / 36

Order of Convergence

Order of Convergence
The order of convergence of a sequence is analogous to the order of
approximation.
Definition : Assume that lim xn = x and {rn }
n=1 is a sequence with
n
lim rn = 0. We say that {xn }
n=1 converges to x with the order of
n
convergence O(rn ), if there exists a constant K > 0 such that
|xn x|
< K, for sufficiently large n
|rn |
or
|xn x| < K|rn |
This is indicated by writing

xn = x + O(rn )

or xn x with order of convergence O(rn ).

Amer Rasheed (MATH344) Department of Mathematics 35 / 36
Order of Convergence

Order of Convergence
Example

Determine the order of convergence of the sequence

cos n
n2

Let
cos n 1
xn = & rn =
n2 n2
As
cos n 1
0 & 0
n2 n2
and
|xn x| cos n/n2
= = | cos n| 1, n
|rn | 1/n2
cos n 1
Thus the sequence 2
converges with O( 2 ).
n n
Amer Rasheed (MATH344) Department of Mathematics 36 / 36