Chapter 1

INTRODUCTION TO NUMERICAL METHODS

AND ERROR ANALISIS

“Engineer is the one who converts problems into solutions.”- zar98

Outcomes: After completing Part One, students should be able to

1. describe numerical techniques as compared to analytical methods

2. use Taylor series expansion to approximate a function

3. perform error analysis associated with numerical methods

Scope and Preview: In this part, we will investigate how numerical methods are used to
approximate the true solution of a problem. An analytical method provides an exact solution.
The reliability of the numerical method will be analyzed by performing an error analysis. Taylor
series expansion (TSE) of a function in one variable will be discussed in details. This is
particularly important since TSE is used as a basis of approximation in the numerical analysis.

1.1 Introduction to numerical methods and mathematical modeling. Numerical

methods provide approximate solutions to many of the engineering problems. The results
obtained may used to assist decision-making process in engineering design. There are many
applications of numerical methods: finding root(s) of a function, solving linear algebraic
equations, fitting curves to data, interpolating the function, performing numerical differentiation
and integration, and solving ordinary and partial differential equations. Various algorithms are
available with each has its own advantages and limitations.

1.2 Error Analysis. Error analysis is very important especially when numerical methods are
implemented on computers. We will study sources of error in numerical methods and how the
error is quantified.

1.3 Approximation of a function. TSE will be used to approximate the value of the
function at any point of interest. The error associated with this approximation will be defined.
Application in numerical differentiation of a function will be introduced.

1
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

1.0 INTRODUCTION

In engineering and sciences, we rarely come across simple and straight forward situations which
can be solved analytically. Often the problems are complex that analytical solutions are either
extremely difficult or impossible to obtain. Furthermore when the constraints are imposed on the
problem, the complexity also increases. Therefore, numerical methods provide an alternative
means of approximating the true solution via computer implementation. They are sometimes
referred to as numerical analysis. Note, however, the numerical solution by no means should
replace the analytical solution if it can be easily and practically obtained. The analytical method
provides continuous solution of the problem while the numerical method only gives solution at
discrete points.

Numerical computations combine two important tools in engineering problem solving: 1)

Mathematics, and 2) computers. They influence the practices of engineering design and modeling
process in the present world. Therefore, in order to be competence in solving engineering
problems, a person must enhance his or her knowledge and skills of these tools.

1.1 MATHEMATICAL MODELING AND ENGINEERING SOLUTION

1.1.1 Mathematical Modeling:

Modeling Process: Model is referred to as a mathematical expression relating independent,

dependent variables, parameters, and forcing functions. The variables may be physical,
engineering, and economic variables and etc. In general, we may express the mathematical model
in the form

qo  f (qi , c, u) (1-1)

where q0 = dependent variable

qi = independent variable(s)
c = parameter(s)
u = forcing function(s)

Let’s consider a body subjected to a force as shown in Figure 1.1,

x

F(t)
m

Figure 1-1: A body subjected to force.

Applying Newton’s second law and neglecting the friction, we may write

2
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

F  ma (1-2)

and for one-dimensional problem (as in this case), we can write eq(1-2) as

dv
m F (1-3)
dt
d 2x
or m F (1-4)
dt 2

We see that eq.(1-4) relates a linear position x (dependent variable) to the applied force F
(forcing function) and the time t (independent variable). This relation forms a mathematical
model of a body being acted upon by a force in one dimension. The mass m is the only parameter
in the above model. The complexity of the model increases if we decide to include the effect of
friction and other considerations such as wind resistance.

Once the model is obtained, numerical method is employed in predicting and obtaining the
information about the performance or behaviors of the model. Remember that the method only
provides approximate information of the model.

1.1.2 Analytical vs. Numerical Methods

Consider a free falling object when it is released from rest at time t = 0 s. Neglecting the air
resistance, the equation of motion is given by

y  y0  v0t  21 gt 2 (1-5)

where g = gravitational acceleration = 9.81 m/s2

v0 = initial velocity (= 0 in this case i.e. starts from rest)

Analytical solution of the velocity is given by

dy
 v  v0  gt (1-6)
dt
v = 0 + 9.81t
= 9.81t

We can solve for the velocity at any time t, for example, at t = 2.0 s,

v= 9.81(2.0) = 19.61 m/s (True solution)

Numerical solution may be expressed using one of the standard formulas, such as forward finite
difference as follows.

3
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

yi 1  yi
vi  (1-7)
ti 1  ti

In order to use this formula, we need to have the values of the position y at the discrete time t (at
least at two different points with one being at ti and the other at ti+1). For example, the position y
may be tabulated at time t as shown below

t (s) 0 2 3
y (m) 0 19.62 44.145

Then, from eq.(1-7)

44.145  19.62
at ti = 2.0 s, vi  = 12.2625 m/s
3 2

Repeat the calculation but with smaller time steps or intervals,

t (s) 2 2.5
y (m) 19.62 30.65625

v (t = 2) = 22.0725 m/s

t (s) 2 2.1
y (m) 19.62 21.63105

v (t = 2) = 20.1105 m/s

t (s) 2 2.05
y (m) 19.62 20.61326

v (t = 2) = 19.86525 m/s

We observe that the numerical solution approaches the true value when the step size is reduced.
However, there is a limit to how small could the step size be. This is especially important when a
computer is used in the solution procedures.

(iii) Advantages of Numerical Methods

1. Provide extremely powerful problem-solving tools with the ability to handle large system of
equations, nonlinearities in the models, and complicated geometry.

2. Reinforce understanding of mathematics in describing the physical phenomena with the use of
computers.

3. Provide an insight and opportunity to analyze solution(s) of the problem from many different
4
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

perspectives.

4. Reduce the amount of time used in arithmetic operations.

However, we need to understand the limitations and pitfalls of each numerical methods before we
can effectively and efficiently employ them to solve problems with acceptable accuracy. There
are several factors that influence a decision-making process in choosing one particular method
over the others.

1. complexity of the problem

2. societal importance of the problem
3. solutions are unbiased and precise
4. cost effective
5. minimal environmental impacts.

In general, engineering problem solving and analysis can be represented as shown in the diagram
below.

Problem
definition

Theory Mathematical Experimentation

model

Performance verification:
analytical/numerical methods,
statistics, computers, graphics, etc.

Economic and societal interfaces:

optimization, ,design , scheduling,
communication, environmental
impacts, etc.

Implementation

Figure 1-2: Engineering problem-solving process

5
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

1.2 Error Analysis

(i) Significant Digits (s.ds) - those digits (or numbers) that can be used with confident plus one
estimated digit. For example:

100 110 120

metal sheet

Figure 1-3: Length of a metal sheet

The length of the metal sheet is read as 116.5 units (4 s.ds) which contains one estimated digits.
If we were to read the length as 116.57 units, then the last digit which is 7 is not significant.

Suppose the electric meter indicates the electric power consumption as shown,

3
0 5 2 9 7 0 watts
4

Figure 1-4: Electrical power used.

The reading to the nearest watt is recorded as 52970.35 watts (7 s.ds). It would be incorrect to
read the meter as 52970.348 watts which now has two doubtful digits. The following table
illustrate more about the number of significant digits contain in the numbers.

Number Significant Digits

2.347 4
1.652 0 5
189.708 6
0.001 2 2
0.001 200 4

The reasonable number of significant digits in calculations should be determined from the
relative error required in the final answer. *Rule does not apply to conversion factors and exact
numbers.

6
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

(ii) Accuracy and Precision. Precision in numerical method is referred to the consistency of the
estimates to be closed to each other. On the other hand, accuracy is the measure of closeness of
the estimate to the target.

Increasing Accuracy

(a) (b)

(c) (d)

Increasing Precision
Figure 1.5: Accuracy and precision

The accuracy of the numerical computation depends on a particular method used and on its ease
of implementation on the computer (i.e. accuracy and limitations of the computer). A computer
is used for two functions in regard to numerical computation: 1) to store numbers and 2) to
perform arithmetic operations. Computer can only store a finite number of digits. The remaining
digits are either chopped off or rounded off. This finite number representation in the computer
leads to an unavoidable error called a round-off error (REr).

Example of a computer program that supposedly reads and writes the same number:

C Program Show Error

C
C This program compares input and output number.
C
READ(11, 1) A
WRITE(10, 1) A
1 FORMAT(2X, F15.13)
STOP
END

Programming result,
input : 0.1234567890123

output : 0.1234567910433

7
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

Figure 1-6: Round-off error in a computer.

In performing arithmetic operations, only certain digits are usually retained in order to get an
approximate solution. Several digits are truncated or omitted. These truncated digits lead to what
is called a truncation error (TEr).

Using the above programming result, the output A may be retained only up to 5 decimal places
(d.ps),

input : 0.1234567890123

output : 0.12346 (5 d.ps)

Remark: 1. If any solution is sought with too much precision, REr will be dominant.
Otherwise, lack of mathematical accuracy (TEr) prevails.

2. REr and TEr constitute a limit to how accurately a solution can be obtained by
implementing numerical methods on computers.

Recall the approximation of the velocity v (which is essentially the first derivative of the
function y ), we may express the general form of the first derivative for any function f(x) at the
base point x0 using first finite forward difference formula as

f ( x0  h )  f ( x0 )
mapprox  f ' ( x0 )  (1-8)
h

and the true value (analytical solution) is represented by a slope of the tangent line at x0 which
may be shown in the figure below,

mtrue mapprox
f
f(x)

h
x0 x
x0 + h

Figure 1-7: First derivative of a function.

8
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

We observe that when h : mapprox less accurate 

 shows TEr.
h : mapprox more accurate 

when h is too small, REr dominates. This observation suggests that there must be an optimum
value of h .

Summary: 1. REr is an error introduced in representation of a given number by a computer.

It usually becomes a limiting factor when small changes in certain input
parameters result in relatively large deviations of output. It is likely to occur
when the relative magnitude of the numbers involved in the calculations
differs significantly and when two nearly identical numbers are subtracted
from each other.

2. TEr is an error introduced by approximating some ideal mathematical

functions in a simpler form. It usually stems from the computational method
used and is dictated by the desired accuracy in the computations.

(iii) Error Quantification. When the true value of the quantity is known, the error is quantified
in two different but related ways.

Let x denote an exact (or true) value and x* denote an approximate (or estimated) value or
simply an approximation. We may then express the relation,

True value = Approximation + Error (1-9a)

or Error = True value - Approximation (1-9b)

which defines the error as the difference between the true value and the approximation.

 (True) Absolute error:

Et  x  x * (1-10)

Note that the absolute error does not reflect the degree of its influence.

For example, Et of 0.01m in measurement of the distance to the moon and,

Et of 0.01m in diameter of a piston in a cylinder

do not suggest the significant or scale of the error, though, both have same the value of the
absolute error. In view of this limitation, the second way of quantifying the error is introduced.

9
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

 (True) Relative error:

x  x*
Et , rel  (1-11a)
x

Note that the absolute error is normalized or scaled with respect to (or relative to) the true value
so that the influence of the error can be significantly interpreted. Usually the relative error is
expressed in terms of percentage, that is

x  x*
(True) Percent relative error : t  x100% (1-11b)
x

(iv) Error bound and Its Propagation. Any number (x*) satisfying the inequality

( x*)  x  x * (1-12)

is called an error bound or an absolute error bound for the error of x* as an approximation of
x.

Remark: all numbers larger than (x*) also serve as error bounds.

Similarly, Any number (x*) satisfying the inequality

x  x*
( x*)  (1-13)
x

is called a relative error bound .

Suppose an absolute error bound is known, we may select

 ( x*)
( x*)  (1-14)
x

and if (x*) <<<  x*, then

 ( x*)
( x*)  (1-15)
x*

which is a good approximation of a relative error bound (x*).

10
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

Error Propagation in a Function

Next we would like to estimate the effect of the discrepancy between the true value and the
approximate value on the value of the function. This is called error propagation.

If a function f is dependent on
(a) a single independent variable x : f(x)
(b) two independent variables x and y : f(x, y)
(c) several independent variables x1 , x2 , x3 , ... ,xn : f(x1 , x2 ,..., xn ).

Let’s consider a function of a single variable f(x).

Let x be the true value and

x* be an approximate value of x

Then, TSE for f(x) computed near f(x*) is given by

f ' ' ( x*)

f ( x)  f ( x*)  f ' ( x*)( x  x*)  ( x  x*) 2  ... (1-16)
2

Truncating after the first derivative term and rearranging the remaining terms to give

f ( x)  f ( x*)  f ' ( x*)( x  x*) (1-17a)

f ( x*)  f ' ( x*) x * (1-17b)

where f ( x*)  f ( x)  f ( x*) is an estimate of the error of the function.

x*  x  x * is an estimate of the error of independent variable x.

Eq.(1-17) provides 2 capabilities:

1. to approximate the error in f(x) knowing its derivative.

2. to approximate the error in the independent variable x.

Similarly, for a function of n independent variables f(x1 , x2 ,..., xn ), we may write

f f f
f ( x1 *, x2 *,..., xn *)  x1 *  x2 * ... xn * (1-18)
x1 x2 xn

f f f
where , ,..., are partial derivatives of the function evaluated at points x1*, x2*,...,
x1 x 2 x n
and xn*.

11
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

Condition number (Nc ): It provides a measure of the extent to which a change in x affects a
change in f(x). A function is said to be ill-conditioned if a small change (i.e. perturbation) in x
causes a large change in f(x). Nc is the ratio of the relative error of f(x) and x

1 relative error for f ( x ) and x is identical.

x * f ' ( x*) 
Nc   1 relative error for f ( x) is amplified . (1-19)
f ( x*)  1 relative error for f ( x) is attenuated .

where estimate of the relative error of f(x) and x is given,repectively, by

f ( x)  f ( x*) f ' ( x*)( x  x*)

  f ( x)    (1-20)
f ( x*) f ( x*)
x  x*
and  ( x)  (1-21)
x*

Error estimate of the function.

Example 1.1: Knowing a value of x* = 2.0 with an error of x* = 0.01, estimate the resulting
error in the function

f ( x )  0.5 x 3  0.1x 2  0.8 x  0.7

Solution: At x* = 2.0 , f (2.0)  0.5(2.0) 3  0.1(2.0) 2  0.8(2.0)  0.7 = 4.5

Using eq.(1-17) with the first derivative, f ' ( x )  1.5 x 2  0.2 x  0.8

f(x*) = f(2.0)  [1.5(2.0)2 - 0.2(2.0) + 0.8](0.01) = 0.064

Therefore, we predict that the function value at x* = 2.0 would be in the range

f(2.0) = 4.5  0.064 or 4.436 < f(2.0) < 4.564 (error bounds)

12
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

Error estimate of the function.

Example 1.2: For a beam with one end fixed and the other end free (as shown below), the
deflection of the beam y is given by

FL4
F y
8 EI

where E is modulus of elasticity and I is mass moment of

inertia. Knowing the approximate values of the variables and
L their associated errors as
y
F* = 120 lbf/ft F* = 5 lbf/ft
L* = 27 ft L* = 0.1 ft
E* = 1.65(108) lbf/ft2 E* = 8.0(105) lbf/ft2
I* = 0.072 ft4 I* = 0.0005 ft4

estimate the resulting error in the deflection y.

Solution: The deflection y is

(120)(27) 4
y*  = 0.6710 ft (4 d.ps)
8(1.65x108 )(0.072)

Using eq.(1-18) with the first partial derivative,

y y y y
y(F*, L*, E*, I* )  F *  L *  E *  I *
F L E I

L4 FL3 FL4 FL4

 F *  L *  
2
E *   I *
8 EI 2 EI 8E I 8 EI 2

 0.027959 + 0.009941 + 0.003253 + 0.004660

 0.045813

Therefore, we predict that the deflection y to be in the range

y = 0.6710  0.045813 ft

13
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

(v) Error Quantification in Numerical Methods: In the absent of the true value (which is
usually the case in numerical methods), the following error calculation is employed.

Let xnew denote a present (or new) approximation.

xold denote a previous (or old) approximation.

We may express the relation,

Present approximation = Previous approximation + Approximate Error (1-22a)

or Approximate Error = Present approximation - Previous approximation (1-22b)

(Approximate) Relative error:

x new  x old
Ea  (1-23a)
x new

or in the percentage form, the approximate percent relative error is

x new  x old
a  x100% (1-23b)
x new

In employing the numerical methods, particularly the iterative approaches, we are basically
interested in whether the absolute value of the approximate percent relative error a is less than
the prespecified tolerance s (or desired accuracy) that is

a  s (1-24)

The desired accuracy or prespecified tolerance s is defined as

 s  (0.5x10 2  p )% (1-25a)

or the acceptable error:

E s  (0.5x10  p ) (1-25b)

If eq.(1-24) is satisfied, the approximation is said to be accurate at least to p decimal places

(d.ps)!!! For example, for

p=1 1 d.p accuracy a  s = 0.5x101 = 5%

p=2 2 d.ps accuracy a  s = 0.5x100 = 0.5%

14
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

p=3 3 d.ps accuracy a  s = 0.5x10-1 = 0.05%

Note: It is the user who determines the desired accuracy or the acceptable error. This usually
depends on the importance of the problem and its intended ends.

Error estimates of series approximation.

Example 1-3: Exponential function using the Maclaurin1 Series expansion is given by
x 2 x3 xn
e x  1 x    ...
2! 3! n!
Approximate the function at x = 0.5 with an accuracy to 3 d.ps and also calculate the true
percent relative error.

Solution:
True value, e0.5  1.648721271 = 1.64...

# Equivalent expression and value t % a %

Terms
1 ex  1 1.64..1 100%  39.3 -
1.64..
2 e x  1  x = 1 + 0.5 = 1.5 1.64. .15 . 100%  9.02 . 1 100%
15  33.3
1.64. . 15.

x2 1.64. .1.625 100%  1.44 1.6251.5 100%  7.69

3 ex  1  x  = 1.5 + 0.52/2 1.64. . 1.625
2!
x2 x3
4 ex  1  x   0.175 1.27
2! 3!
x2 x 3 x4
5 ex  1  x    0.0172 0.158
2! 3! 4!
x2 x 3 x 4 x5
6 ex  1  x     0.00142 0.0158
2! 3! 4! 5!

y Note: p = 3 => s = 0.05%

ex
3 terms We stop calculation since a < s .
The approximate value is then
2 terms
1.0 e0.5  1.6487 (accurate to 4 d.ps)
1 terms

x
1
Colin Maclaurin (1698-1746, 48), Scottish mathematician.

15
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

Error estimate of the function.

Worksheet 1.3: Maclaurin series for sin(x) is given by

x 3 x5 x 2 n 1
sin( x)  x    . . .  ( 1) n n = 0,1,2,3,...
3! 5! (2n  1)!
Approximate the function at x = /2 radians with an accuracy to 4 d.ps.

Solution: True value, sin(/2) = 1.0

For p = 4,  s  (0.5x102  4 )%  0.005%

sin(/2)  1.570796 a = 57.1%

sin(/2)  1.570796 – 0.645964 = 0.924832 a = 7.52%

sin(/2)  0.924832 + 0.079693 = 1.004525 a = 0.453%

sin(/2)  1.004525 – 0.004682 = 0.999843 a = 0.0157%

sin(/2)  0.999843 + 0.000160 = 1.000003 a = 0.00003%

Since a < s , we stop the calculation.

16
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

1.3 Function Approximation

(i) Taylor2 series expansion (TSE)

If a function f and its first n+1 derivatives are continuous on the interval [x0, x], the value of
the function at x is then given by

f ' ( x0 ) f ' ' ( x0 ) f ( n ) ( x0 )

f ( x )  f ( x0 )  ( x  x0 )  ( x  x0 ) 2  ... ( x  x0 ) n  Rn (1-26)
1! 2! n!

which is known as the Taylor series for f about x = x0.

Defining x  x0  h or h  x  x0  step size,

we can express eq.(1-26) as

f ' ( x0 ) f ' ' ( x0 ) 2 f ( n ) ( x0 ) n

f ( x )  f ( x0 )  h h  ... h  Rn (1-27)
1! 2! n!

where x0  the base point (or starting value)

h  the step size (or step increment)
Rn  the remainder (or error term)

The remainder is defined as

x
(x   ) ( n  1)
Rn 
 x0 n!
f (  ) d (1-28)

where  is a dummy variable. Note that Maclaurin series is a special form of Taylor series for
which x0 = 0.

2
Brook Taylor (1685-1731,46), English mathematician.

17
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

In a form convenient for computer implementation, eq.(1-27) can be written as

n
f ( k ) ( x0 ) k
f ( x)  
k 0
k!
h  Rn (1-29)

Note that 0! = 1.

From the second theorem of mean for integrals, eq.(1-28) yields

f ( n 1) ( ) n 1
Rn  h where x0    x (1-30)
(n  1)!

which is called the derivative or the Lagrange3 form of the remainder with  being an unknown
value. Other formulas for Rn are also available. They are available in most advanced calculus
text.

(ii) Order of TSE:

zero-order : f ( x )  f ( x0 ) (1-31)

f '( x0 )
first-order : f ( x )  f ( x0 )  h (1-32)
1!

f '( x0 ) f ' ' ( x0 ) 2

second-order : f ( x )  f ( x0 )  h h (1-33)
1! 2!

f '( x0 ) f '' (x0 ) 2 f ( n ) ( x0 ) n

nth-order : f ( x )  f ( x0 )  h h  ... h (1-34)
1! 2! n!

3
Joseph Louis Lagrange(1736-1813, 77), French-Italian mathematician and astronomer. A
successor of Leonhard Euler(1707-1783, 76)-born in Switzerland as director of Berlin Academy
in(1776).

18
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

Taylor series approximation of a polynomial.

Example 1-4: Use zero- through third-order TSE to approximate the function

f ( x)  0.5x3  01
. x2  0.8x  0.7

at x = 1.0 and the base point (a) x0 = 0 and (b) x0 = 0.8. Also calculate both the true and
approximate percent relative errors for each order.

Solution: True value, . (1)3  01

f (1)  05 . (1) 2  08
. (1)  0.7 = 0.5

(a) with h = x - x0 = 1.0, f(x) = 0.5x3 -0.1x2 +0.8x - 0.7 f(0) = -0.7
f’(x) = 1.5x2 -0.2x +0.8 f’(0) = 0.8
f’’(x) = 3.0x -0.2 f’’(0) = -0.2
f’’’(x) = 3.0 f’’’(0) = 3.0
(4)
f (x) = 0.0

order Expansion and Value t % a

0.5  ( 0.7 ) -
0th f (1)  f (0)  0.7 100%  240
0.5
(0.8) 0.5  ( 01
.) .  (0.7 )
01
100%  800
f (1)  0.7  (1.0)  01. 100%  80
1st 1! 0.5 01
.
(0.2 )
2nd f (1)  01
.  (1.0)2  0.0 100 -
2!
(3.0)
3rd f (1)  0.0  (1.0)3  05. 0 100
3!

We observe that though, the true percent relative error is zero, the approximate percent relative
error is 100%. In practice, when using numerical method (in the absent of the true value), we
will continue with higher order approximation(s) that is

( 0)
4th-order approximation: f (1)  0.5  . ) 3  0.5
(10 and a = 0%.
4!

19
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

(b) with h = x - x0 = 0.2,

Using the derivatives from part (a), we obtain

f(0.8) = 0.132
f’(0.8) = 1.6
f’’(0.8) = 2.2
f’’’(0.8) = 3.0
f(4)(x) = 0.0

order Expansion and Value t % a %

0th f (1)  f (0.8)  0132

. 73.6 -

(16. )
1st f (1)  0132
.  ( 0.2 )  0.452 9.6 70.8
1!
( 2.2 )
2nd f (1)  0.452  ( 0.2 ) 2  0.496 0.8 8.9
2!
(3.0)
3rd f (1)  0.496  ( 0.2 ) 3  05
. 0 0.8
3!

In practice, we usually require the result to be within a specified tolerance. For example, we may
want the result to be accurate at least to 3 d.ps. If this is the case, then we will have to continue
with higher order approximation(s).

p = 3, s = 0.05%

( 0)
4th-order approximation: f (1)  0.5  . ) 3  0.5
(10 and a = 0%.
4!

Since a < s , we stop the calculation.

20
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

Taylor series approximation of a polynomial.

Worksheet 1-4: Use TSE to approximate the function

f ( x )  x 4  0.7 x 3  2 x 2  x  62.5

at x = 5.0 and the base point x0 = 4.5 with an accuracy to 3 d.ps. Also calculate the true percent
relative error for each of the orders.

Solution: True value, f (5)  (5) 4  0.7(5)3  2(5)2  (5)  62.5 = 420

h = x - x0 = 0.5, f(x) = x4 - 0.7x3 -2x2 - x - 62.5 f(4.5) = 238.775

f’(x) = 4x3 - 2.1x2 -4x - 1 f’(4.5) = 302.975
f’’(x) = 12x2 - 4.2x - 4 f’’(4.5) = 220.1
f’’’(x) = 24x - 4.2 f’’’(4.5) = 103.8
(4)
f (x) = 24 f(4)(4.5) = 24

order Expansion and Value t % a

0th f (5)  f ( 4.5)  238.775 43.1 -

( 302.975)
1st f (5)  238.775  ( 0.5) 7.08 38.8
1!
= 390.2625
(2201
.)
2nd f (5)  390.2625  ( 0.5) 2 0.530 6.59
2!
= 417.775
(1038
. )
3rd f (5)  417.775  (0.5)3 0.0149 0.515
3!
= 419.9375
( 24)
4th f (5)  419.9375  ( 0.5) 4 0.0 0.0149
4!
= 420

21
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

Remark: 1. The function and its n+1 derivatives are assumed to be continuous in the
interval of interest [x0, x].

2. Order of approximation is the highest order of the derivatives.

3. Accuracy of approximation improves as order of TSE increases or when the

step size h decreases.

4. Higher order terms become apparent when the nonlinearities of the function
increases or when the step size increases.

5. When the function is linear (i.e. a polynomial), the first-order TSE with
reasonably small step size is usually adequate.

6. TSE is used when the function is not known or in order to obtain approximate
solutions for trigonometric and logarithmic functions.

In practice, we may use the known function to illustrate the practicality of the TSE.

(iii) Remainder, Rn

Recall the derivative form of the remainder, eq.(1-30),

f ( n  1) (  ) n 1
Rn  h
( n  1)!

There are two major drawbacks associated with this formula:

1.  is not known exactly. It lies only in between x0 and x.

2. In order to evaluate Rn, we need to know the (n+1)th derivative of f(x) i.e. function f(x) must
be known. But if we knew f(x), it is no need to use TSE. We can use the function directly to
get the exact value.

Despite these two drawbacks, we do however have control over two things:

1. the term hn+1 i.e. we can choose how far away from x0 we want to evaluate f(x).
2. the number of terms to be included in TSE.

22
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

Consequently, Rn can be expressed as

Rn  ( h n 1 ) (1-35)

which implies that the truncation error is of the order of hn+1. In other words, it is proportional
to the step size h raised to the power of (n+1). It enables us to judge the comparative error of
numerical methods based on TSE.

For example,

when Rn  (h) : halving the step size h, will halve the error.
Rn  ( h 2 ) : halving the step size h, will quarter the error.

Remark: In general, we assume that the truncation error is decreased with addition of more
terms in the TSE and with smaller step size h.

We may also gain some other insights about the remainder. Let’s define an upper bound on the
magnitude of the reminder Rn such that

Rn  Rn ,u (1-36)
with
M
Rn ,u  h n 1 and f ( n 1) ()  M (1-37,38)
( n  1)!

where Rn,u is an upper bound on the magnitude of the reminder Rn

M is an upper bound on the magnitude of the (n+1)th derivative at .

Upper bound of the remainder.

Example 1-5: Recall the exponential function using the Maclaurin series example 1-3,

x2 x3 xn
ex  1    ...
2! 3! n!

Define an upper bound of the remainder and determine the number of terms n so that the
approximation about x = 1.0 is accurate to 1 d.p.

23
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

Solution: Note x0 = 0.

f ( n 1) () n 1 e
The remainder Rn  h  x n 1
( n  1)! ( n  1)!
For x = 1.0, we have

1 1 1 e
f (1)  e1  1  1    ...  where 0 <  < 1
2! 3! n! ( n  1)!
Defining,
e e
Rn  
( n  1)! ( n  1)!

For an upper bound,  < 1 and consequently e < e1 = e = M.

Therefore, the upper bound on the magnitude of the remainder is

e e
Rn   Rn,u  (E1.5-1)
(n  1)! (n  1)!

To gain meaningful interpretation of eq.(E1.5-1), let e < 3, and for p = 1,

3
Rn   (0.5x10 p )
( n  1)!

By trial and error, we have n = 4.

24
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

TSE for a function with an infinite number of derivatives.

Example 1.6: Use TSE from zero- through fifth-order to approximate the function,
1
f ( x) 
x
at x = 2.0 with x0 = 1.0. Evaluate the true percent relative error for all cases.

Solution: True value, f(2.0) = 0.707107 (accurate up to 6 d.ps).

1 3
Derivatives, f '( x )   x 2 => f’(1.0) = - 0.5
2
3 5 2
f ''( x )  x => f’’(1.0) = 0.75
4
15 7
f '''( x )   x 2 => f’’’(1.0) = - 1.875
8
(4 ) 105 9 2
f ( x)  x => f (4)(1.0) = 6.5625
16
945 112
f ( 5) ( x )   x => f(5)(1.0) = - 29.53125
32

with h = x - x0 = 2.0 - 1.0 = 1.0,

Order of approximations t %

0th: f(2.0)  1.0 41.4

1st: f(2.0)  1.0 + (-0.5)h

 1.0 + (-0.5)(1.0)  0.5 29.3

2nd: f(2.0)  1.0 + (-0.5) h + (0.75/2) h 2

 0.5 + (0.375)(1.0)2  0.875 23.7

3rd: f(2.0)  1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3

 0.875 + (-0.3125)(1.0)3  0.5625 20.5

25
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

4th: f(2.0)  1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3 + (6.5625/24) h 4

 0.5625+ (0.2734375)(1.0)4  0.8359375 18.22

5th: f(2.0)  1.0 + (-0.5) h + (0.75/2) h 2 + (-1.875/6) h 3 + (6.5625/24) h 4 +

(-29.53125/120) h 5

 0.8359375 + (-0.24609375)(1.0)5  0.58984375 16.58

6th, 7th, 8th: 0.815427, 0.605954, 0.802335 15.32, 14.30,

13.47

Note: Although (in this example), the addition of more terms will further reduce the error, but
there is no significant improvement. It suggests that an infinite number of terms is required in
order to get the approximation ‘closed enough’ to the true value.

This is true for most of the function with

(1) an infinite number of the derivatives and
(2) relatively large step size.

In addition, for some functions, the solution may indeed diverge. This depends on the interval of
validity of x for that function.

26
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

TSE for a function with an infinite number of derivatives.

Worksheet 1.5: Repeat example 1.6 but with x0 = 1.5.

Solution: True value, f(2.0) = 0.707107 (6 d.ps)

Now h = x - x0 = 2.0 - 1.5 = 0.5 (which is half of step size in the previous example)

=> f(1.5) = 0.8165, f’(1.5) = - 0.2722, f’’(1.5) = 0.2722

f’’’(1.5) = - 0.4536, f (4)(1.5) = 1.0584, f(5)(1.5) = - 3.1753

Order of approximations t %

0th: f(2.0)  0.8165 15.47

1st: f(2.0)  0.816 + (-0.2722)h

 0.8165 + (-0.2722)(0.5)  0.6804 3.77

2nd: f(2.0)  0.8165 + (-0.5)h + (0.2722/2)h2

 0.6804 + (0.1361)(0.5)2  0.7144 1.032

3rd: f(2.0)  0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3

 0.7144 + (-0.0756)(0.5)3  0.7050 0.2.97

4th: f(2.0)  0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3 +

(1.0584/24)h4
0.0849
 0.7050 + (0.0441)(0.5)4  0.7077

5th: f(2.0)  0.8165 + (-0.2722)h + (0.2722/2)h2 + (-0.4536/6)h3 +

(1.0584/24)h4 + (-3.1753/120)h5
0.0283
 0.7077 + (-0.02646)(0.5)5  0.7069
0.00745
0.70715, 0.70707 0.00399

Note: the approximate value approaches the true value faster when the step size is reduced. After
the fifth-order approximation, the result is accurate at least up to 3 d.ps.

27
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

(iv) Truncation Error Associated With TSE:

Denoting xi = x0
xi+1 = x

and using the zero-order TSE, we write

f(xi+1)  f(xi) (1-39)

f '( xi ) f ''( xi ) 2
with the remainder, Ro  h h  . . .  Rn (1-40)
1! 2!

We may approximate the zero-order remainder as

Ro  f '( xi )h (1-41)

Figure 1.8(a) illustrates the concept of error approximation in TSE.

f(xi+1) f(x) f(x) f(x)

f(x)
f()
Exact R0
slope = R0/h
f(xi)
0th-order

h
x  xi+1 x
xi xi+1 xi

(a) (b)

Figure 1.8: Graphical depiction of error in TSE

Using derivative mean-value theorem: if a function f(x) and its first derivative are continuous
over an interval from xi and xi+1, then there exists at least one point on the function that has a
slope, designated by f’(), that is parallel to the line joining f(xi ) and f(xi+1 ).

28
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

Thus, the slope f’() is equal to the rise Ro divided by the run h, or

Ro
f '( )  => Ro  f '( ) h (1-42)
h

Similarly, the first-order and second-order remainders are, respectively

f ''( ) 2
R1  h (1-43)
2!

f '''( ) 3
R2  h (1-44)
3!

(v) Application: Truncation Error of the Derivatives

Using TSE and truncating after the first derivative term, we can write

f ( xi 1 )  f ( xi )  f '( xi )( xi 1  xi )  R1 (1-45)

and solving for the first derivative term yield

f ( xi 1 )  f ( xi ) R1
f '( xi )   (1-46)
xi 1  xi xi 1  xi
    
st
1  order Truncation
approximation error

From eq.(1-43) and eq.(1-46), we can then estimate the truncation error associated with the
approximation of the first derivative using TSE as

R1 f ''( )
 h where h = xi+1 - xi
h 2!

R1
or  ( h) (1-47)
h

We conclude that the estimate of the first derivative using TSE has a truncation error of order of
h i.e. the error of the first derivative approximation is proportional to the step size. It follows
that the error will be halved if the step size is halved.

29
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

(vi) Finite Difference Approximations of First Derivative

There are three different methods for the approximation of the first derivative.

a) Forward difference method (FDM):

From eq.(1-45) and eq.(1-47) with h = xi+1 - xi , we may write eq.(1-45) in the form

f (xi  1 )  f (xi )  f ' (xi ) h  R1 (1-48)

Rearranging terms to give the finite difference formula for the first derivative as

f ( xi 1 )  f ( xi ) f i
f ' ( xi )    (h) or f '( xi )    (h) (1-49)
h h

where f i  the first forward difference and h  step size

f i
 the first finite divided difference.
h

b) Backward difference method (BDM):

TS can also be expanded backwards to approximate the previous value f(xi-1 ) based on the
present value f(xi ) as

f '' (xi ) 2 f ' '' (xi ) 3

f (xi 1 )  f ( xi )  f ' ( xi )h  h  h  . . . (1-50)
2! 3!

Truncating all terms after the first derivative term and rearranging the equation yields

f ( xi )  f ( xi 1 ) f i
f ' ( xi )   ( h ) or f ' ( xi )   ( h ) (1-51)
h h

where fi  the first backward difference.

30
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

c) Centered (or central) difference method (CDM):

This can be derived by subtracting the backward TSE from the forward TSE which yields

f ' ' ' ( xi ) 3

f ( xi  1 )  f ( xi 1 )  2 f ' ( xi ) h  h ... (1-52)
3

Again solving for the first derivative and rearranging the terms to give

f ( xi 1 )  f ( xi 1 ) f '' ' (xi ) 2

f '( xi )   h
2h 6

f ( xi 1 )  f ( xi 1 )
or f ' ( xi )    ( h2 ) (1-53)
2h

Notice that the error for the central different method is of the order of h2 as compared to h for
the forward and backward difference methods. Therefore, central difference method gives better
approximation of the derivative than the other two methods.

The three methods can be depicted graphically as shown in figure 1.9 below.

f(xi+1) f(xi+1)
f(xi ) f(xi) f(xi)
exact exact exact

f(xi-1) f(xi-1 )

x x xi-1 x
xi xi+1 xi-1 xi xi xi+1

(a) FDM (b) BDM (c) CDM

Figure 1.9: Approximation of first derivative.

We will study numerical differentiation in more details in part Five. This includes the higher
accuracy formulas and also higher order derivatives.

31
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

Numerical Differentiation for First Derivative.

Example 1.7: Use forward, backward and centered difference to estimate the first
derivative of the function

f (x )  0.5x 3  0.1x2  0.8x  0.7

at x = 0.5 using a step size h = 0.5. Also calculate the true percent relative
error for each approximation.

Solution: True value, . x 2  0.2 x  0.8

f ' (x )  15
. (0.5)2  0.2 (0.5)  0.8 = 1.075
f '(0.5)  15

With h = 0.5 and at x = 0.5, they imply that

xi = 0.5 => f(xi) = -0.2625

xi-1 = 0 => f(xi-1) = -0.7
xi+1 = 1.0 => f(xi+1) = 0.5

Using eqs.(1-43), (1-45), and (1-47), we obtain

t %

FDM: 0.5  ( 0.2625)

f ' (0.5)   1525
. 41.9
0.5

BDM: ( 0.2625)  ( 0.7 )

f ' (0.5)   0.875 18.60
0.5

CDM: 0.5  ( 0.7 )

f ' (0.5)   1200
. 11.63
10
.

32
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

Numerical Differentiation for First Derivative.

Worksheet 1.6: Use forward, backward and centered difference to estimate the first
derivative of the function

f (x )  0.5x 3  0.1x2  0.8x  0.7

at x = 0.5 using a step size h = 0.25. Also calculate the true percent relative error for each
approximation.

Solution: True value, . x 2  0.2 x  0.8

f ' (x )  15
. (0.5)2  0.2 (0.5)  0.8 = 1.075
f '(0.5)  15

With h = 0.25 and at x = 0.5 imply that

xi = 0.5 => f(xi) = -0.2625

xi-1 = 0.25 => f(xi-1) = -0.4984375
xi+1 = 0.75 => f(xi+1) = 0.0546875

t %

FDM ( 0.0546875)  ( 0.2625)

f '(0.5)   126875
. 18.02
0.25

BDM ( 0.2625)  ( 0.4984375)

f '( 0.5)   0.94375 12.21
0.25

CDM 0.0546875  ( 0.4984375) 2.91

f '(0.5)   110625
.
0.5

We observe that the CDM is more accurate than the FDM and BDM. Also, halving the step size
approximately halves the error of FDM and BDM, and quarters the error of CDM.

End of Part One

33
Introduction to Numerical Methods Numerical Methods
and Error Analysis with Applications

Checklist:

In part One, I have acquired the following knowledge and understanding:

Important formula/thing Need
Topics/Concepts Y N
to remember Guide
1. Objectives

2. Introduction to Numerical Methods:

a. Mathematical modeling

b. Differences between analytical &

numerical methods

3. Error analysis:
a. significant digits

b. accuracy and precision

c. Error quantification

d. Error bound and propagation

e. Error in numerical analysis

4. Approximation of function:
a. TSE

b. Remainder

c. Truncation error in TSE

d. Finite difference of the first

derivative

34
Numerical Methods Introduction to Numerical Methods
with Applications and Error Analysis

35