Numerical Methods I

Qiao Zhonghua

Room FSC 1213

Department of Mathematics,
Hong Kong Baptist University
Phone: 852 3411 2531
Email: zqiao@hkbu.edu.hk
Web: www.math.hkbu.edu.hk/˜ zqiao

HONG KONG BAPTIST UNIVERSITY 1

 Contents

• 1. Introduction (Burden & Faires Ch. 1)

• 2. Solution of a Single Nonlinear Equation (Burden & Faires Ch.

2)

• 3. Interpolation (Burden & Faires Ch. 3)

• 4. Numerical Integration (Burden & Faires Ch. 4)

• 5. Linear Systems of Equations (Burden & Faires Ch. 6-7)

• 6. Least Squares (Burden & Faires Ch. 8)

HONG KONG BAPTIST UNIVERSITY 2

 Chapter 4. Numerical Integration

• Overview

• Trapezoidal rule, Simpsons rule

• Newton-Cotes Formulas

• Composite Numerical Integration

• Romberg Method

HONG KONG BAPTIST UNIVERSITY 3

 Overview
In most cases, the antiderivative of a give function f (x) is not
known, and then the explicit expression of the integral
Z b
f (x)dx
a

cannot be obtained. Thus, approximation to this integral is

necessary, which is called numerical quadrature,
Z b n
X
f (x)dx ≈ aif (xi)
a i=0

.
HONG KONG BAPTIST UNIVERSITY 4
The methods of quadrature in this section are based on the
interpolation polynomials given in Chapter 3. We first select a set of
distinct nodes x0, . . . , xn from the interval [a, b]. Then we integrate
the Lagrange interpolating polynomial

n
X
Pn(x) = f (xi)Li(x)
i=0

and its truncation error over [a, b] to obtain

b n
Z bX n
Z bY
f (n+1)(ξ(x))
Z
f (x)dx = f (xi)Li(x)dx + (x − xi) dx
a a i=0 a i=0
(n + 1)!

HONG KONG BAPTIST UNIVERSITY 5

 n n
Z bY
X 1
= aif (xi) + (x − xi)f (n+1)(ξ(x))dx
i=0
(n + 1)! a i=0

where ξ(x) is in [a, b],for each x and

Z b
ai = Li(x)dx. for each i = 0, 1, . . . , n.
a

The quadrature formula is, therefore,

Z b n
X
f (x)dx ≈ aif (xi),
a i=0

HONG KONG BAPTIST UNIVERSITY 6

with error given by
n
Z bY
1
E(f ) = (x − xi)f (n+1)(ξ(x))dx.
(n + 1)! a i=0

HONG KONG BAPTIST UNIVERSITY 7

 Trapezoidal rule, Simpsons rule
1. Trapezoidal rule.
If f (x) is approximated by the linear interpolation polynomial

(x − x1) (x − x0)
P (x) = f (x0) + f (x1)
(x0 − x1) (x1 − x0)

where x0 = a, and x1 = b, then we have

b x1  
(x − x1) (x − x0)
Z Z
f (x)dx = f (x0) + f (x1) dx
a x0 (x0 − x1) (x1 − x0)
Z x1
1
+ f 00(ξ)(x − x0)(x − x1)dx
2 x0

HONG KONG BAPTIST UNIVERSITY 8

 h h3 00
= [f (a) + f (b)] − f (ξ)
2 12

where h = b − a. This is called the Trapezoidal rule.

Example 1
Consider:
Z 2
ln x
dx
1 x

• (a) Find the exact value of the definite integral.

Z Z Z
ln x dx 1
dx = ln x( ) = ln xd(ln x) = (ln x)2 + c
x π 2
HONG KONG BAPTIST UNIVERSITY 9
 Z 2
ln x 1
dx = (ln x)2 ≈ 0.240227
1 x 2

• (b) Use the Trapezoidal rule to calculate the definite integral.

Z 2
ln x 1 ln(1) ln(2)
dx ≈ (2 − 1)[ + ] ≈ 0.173287
1 x 2 1 2

• (c)Find a bound for the error contained in the solution to Part (b)

b
h3 00
Z
1
| f (x)dx − [f (a) + f (b)]| = |f (ξ)| a < ξ < b.
a 2 12
ln x 0 1 − ln x
f (x) = , f (x) = 2
.
x x
HONG KONG BAPTIST UNIVERSITY 10
 00 2 ln x − 3 1
f (x) = (ln x ↑ but ↓ with xincreasing).
x3 x3

000 11 − 6 ln x
f (x) = letf 000(x) = 0 ⇒ x ≈ 0.25470∈
¯ (1, 2).
x4
f 000(1) = 11 > 0, f 000(2) ≈ 0.427570 > 0

⇒ f 00(x)increasing for x ∈ (0, 2).

f 00(1) = −3, f 00(2) ≈ −0.201713.

(1)3
Eabs < = 0.25.
12

HONG KONG BAPTIST UNIVERSITY 11

• (d) Determine the actual error contained in the solution to Part
(b)
Eabs = |0.240227 − 0.173287| = 0.06694

HONG KONG BAPTIST UNIVERSITY 12

2. Simpson’s rule.
Let x0 = a, x2 = b, and x1 = a + h, where h = (b − a)/2. Let
x ∈ [a, b], using Taylor’s theorem we have
00
0 f (x1)
f (x) = f (x1) + f (x1)(x − x1) + (x − x1)2 +
2
f 000(x1) 3 f (4)(ξ)
(x − x1) + (x − x1)4.
6 24
Then we have
Z b
f (x)dx =
a
Z x2  00 000

0 f (x1) 2 f (x1)
f (x1) + f (x1)(x − x1) + (x − x1) + (x − x1)3 dx
x0 2 6

HONG KONG BAPTIST UNIVERSITY 13

 Z x2
1
+ f (4)(ξ)(x − x1)4dx
24 x0
3 (4) Z x2
h f (ξ1)
= 2hf (x1) + f 00(x1) + (x − x1)4dx
3 24 x0

h3 00 f (4)(ξ1) 5
= 2hf (x1) + f (x1) + h dx.
3 60

Using Taylor’s theorem we have

00 1 h2 (4)
f (x1) = 2 [f (x0) − 2f (x1) + f (x2)] − f (ξ2).
h 12

Thus, we have the Simpson’s rule,

b
h5 (4)
Z
h
f (x)dx = [f (x0) + 4f (x1) + f (x2)] − f (ξ).
a 3 90

HONG KONG BAPTIST UNIVERSITY 14

Example 2 The Trapezoidal rule for a function f on the interval [0, 2] is
Z 2
f (x)dx ≈ f (0) + f (2),
0

and Simpson’s rule for f on [0, 2] is

Z 2
1
f (x)dx ≈ [f (0) + 4f (1) + f (2)].
0 3

The results to three places for some elementary functions are summarized in Table
4.7. Notice that in each instance Simpson’s Rule is significantly
√ better.
1
f (x) x2 x4 (x+1) 1 + x2 sin x ex
Exact value 2.667 6.400 1.099 2.958 1.416 6.389
Table 4.7
Trapezoidal 4.000 16.000 1.333 3.326 0.909 8.389
Simpson’s 2.667 6.667 1.111 2.964 1.425 6.421

HONG KONG BAPTIST UNIVERSITY 15

 Newton-Cotes Formulas
In general, if we use the Lagrange interpolation polynomial Pn(x) to approximate
f (x), we obtain the Newton-Cotes Formula,

Z b n
X
f (x)dx ≈ aif (xi)
a i=0

where x0 = a, xn = b, xi = x0 + ih with h = (b − a)/n, and

xn xn n
(x − xj )
Z Z Y
ai = Li(x)dx = dx
x0 x0 (xi − xj )
j=0,j6=i

Theorem 4.2 (Closed Newton-Cotes Formulas) Let x0 = a, xn = b, and

HONG KONG BAPTIST UNIVERSITY 16

xi = x0 + ih with h = (b − a)/n. If n is even and f ∈ C n+2[a, b], then

Z b n n+3 (n+2) Z n
X h f (ξ)
f (x)dx = aif (xi) + t2(t − 1) · · · (t − n)dt.
a i=0
(n + 2)! 0

If n is odd and f ∈ C n+1[a, b], then

b n n
hn+2f (n+1)(ξ)
Z X Z
f (x)dx = aif (xi) + t(t − 1) · · · (t − n)dt.
a i=0
(n + 1)! 0

This quadrature formula is called closed Newton-Cotes Formula since the

endpoints of [a, b] are included as nodes. The Trapezoidal Rule and Simpson’s
Rule are the special cases of the Newton-Cotes Formula when n = 1 and n = 2.

HONG KONG BAPTIST UNIVERSITY 17

• n=1: Trapezoidal rule
Z x1
h h3 00
f (x)dx = [f (x0) + f (x1)] − f (ξ)
x0 2 12

• n=2: Simpson’s rule

Z x2
h h5 (4)
f (x)dx = [f (x0) + 4f (x1) + f (x2)] − f (ξ)
x0 3 90

• n=3: Simpson’s Three-Eights rule

Z x3
3h 3h5 (4)
f (x)dx = [f (x0) + 3f (x1) + 3f (x2) + f (x3)] − f (ξ)
x0 8 80

• n=4:
Z x4
2h 8h7 (6)
f (x)dx = [7f (x0)+32f (x1)+12f (x2)+32f (x3)+7f (x4)]− f (ξ)
x0 45 945

HONG KONG BAPTIST UNIVERSITY 18

Similarly, open Newton-Cotes Formula can also be obtained, which does not
include the endpoints of [a, b] as nodes.
Theorem 4.3. (Open Newton-cotes Formulas) Let x−1 = a, xn+1 = b, and
xi = x0 + ih with h = (b − a)/(n + 2). If n is even and f ∈ C n+2[a, b], then
b n n+1
hn+3f (n+2)(ξ)
Z X Z
f (x)dx = aif (xi) + t2(t − 1) · · · (t − n)dt.
a i=0
(n + 2)! −1

If n is odd and f ∈ C n+1[a, b], then

b n n+1
hn+2f (n+1)(ξ)
Z X Z
f (x)dx = aif (xi) + t(t − 1) · · · (t − n)dt.
a i=0
(n + 1)! −1

• n=0: Midpoint rule

x1
h3 00
Z
f (x)dx = 2hf (x0) + f (ξ)
x−1 3

HONG KONG BAPTIST UNIVERSITY 19

• n=1: x2
3h3 00
Z
3h
f (x)dx = [f (x0) + f (x1)] + f (ξ)
x−1 2 4

• n=2: x3
14h5 (4)
Z
4h
f (x)dx = [2f (x0) − f (x1) + 2f (x2)] + f (ξ)
x−1 3 45

• n=3:
x4
95h5 (4)
Z
5h
f (x)dx = [11f (x0) + f (x1) + f (x2) + 11f (x3)] + f (ξ)
x−1 24 144

HONG KONG BAPTIST UNIVERSITY 20

 Example 3. Using
√ the closed and open Newton-Cotes formulas to approximate
R π4 2
0
sin xdx = 1 − 2 ≈ 0.29289322 gives the results in Table 4.8.

Table 4.8
n 0 1 2 3 4
Closed formulas 0.27768018 0.29293264 0.29291070 0.29289318
Error 0.01521303 0.00003942 0.00001748 0.00000004
Open formulas 0.30055887 0.29798754 0.29285866 0.29286923
Error 0.00766565 0.00509432 0.00003456 0.00002399

HONG KONG BAPTIST UNIVERSITY 21

 Composite Numerical Integration
As we have seen in the last chapter, high order interpolations may led to large
errors in the resulting approximations. For numerical integrations, we face the
same problem. High order quadrature rules may give inaccurate results due to the
stability problem. Similar to piecewise polynomial interpolations, we can also
introduce piecewise lower order quadrature formulas, which are called composite
quadrature rules.
Suppose the interval [a, b] is divided into n subintervals, apply the Trapezoidal rule
in each of the subinterval we obtain
Theorem 4.5. (Composite Trapezoidal rule) Let f ∈ C 2[a, b], h = (b − a)/n,
and xj = a + jh, j = 0, 1, · · · , n. There exists a µ ∈ (a, b) such that
 
b n−1
b − a 2 00
Z
h X
f (x)dx = f (a) + 2 f (xj ) + f (b) − h f (µ)
a 2 j=1
12

Similarly, if n is even, apply Simpson’s rule on each consecutive pair of subinterval

HONG KONG BAPTIST UNIVERSITY 22

we obtain
Theorem 4.4. (Composite Simpson’s rule) Let f ∈ C 4[a, b], n be even,
h = (b − a)/n, and xj = a + jh, j = 0, 1, · · · , n. There exists a µ ∈ (a, b) such
that

 
b (n/2)−1 n/2
b − a 4 (4)
Z
h X X
f (x)dx = f (a) + 2 f (x2j ) + 4 f (x2j−1) + f (b)− h f (µ).
a 3 j=1 j=1
180

Example 4. R2 1
Determine the value of n and h required to approximate 0 x+4 dx to within 10−5
if using:

• (a) Composite Trapezoidal rule; From Theorem 4.5, the error term:

HONG KONG BAPTIST UNIVERSITY 23

 2 00
− b−a
12 h f (µ) in this case, b = 2, a = 0, f (x) =
1
x+4

−1 2
f 0(x) = 2
, f 00(x) =
(x + 4) (x + 4)3

2 1
max f 00(µ) = =
0≤µ≤2 (0 + 4)3 32
Therefore
h2
 
 b − a 2 00  1 1 2
−
 12 h f (µ) = ( )h =

max 12 32 192
Let
h2
< 10−5 = ∆ h < 0.0438178.
192
b−a
Since h = n = n2 , let 2
n < 0.0438178 ⇒ n > 45.6435. Take n = 46.

HONG KONG BAPTIST UNIVERSITY 24

• (b)Composite Simpson rule: From Theorem 4.4, the error term − b−a 2 (4)
180 h f (µ)

4
000 −6 α
f (x) = , f (4)(α) =
(x + 4)4 (x + 4)5

24 3
max f (4)(µ) = =
0≤µ≤2 (0 + 4)5 128
Therefore   4
 b − a 4 (4)  2 3 4 h
−
 180 h 2f (µ) ax = 180 ( 128 )h = 3840 .

m
Let
h4
< 10−5 = ∆ h < 0.0442673.
3840
Since h = b−a
n = 2
n , let 2
n < 0.0442673 ⇒ n > 4.51801 n should be an even
number. So,take n = 6.

HONG KONG BAPTIST UNIVERSITY 25

 Romberg Method
Romberg integration is a acceleration technique to increase the convergence
speed. Romberg integration uses the Composite Trapezoidal rule to give
preliminary approximations and then applies the Richardson extrapolation process
to improve the approximation. Let mn = 2n−1, n = 1, 2, 3, · · ·, and let
hk = (b − a)/mk . Use the Composite Trapezoidal rule we have
 k−1

b 2X −1
(b − a) 2 00
Z
hk 
f (x)dx = f (a) + f (b) + 2 f (a + ihk ) −
 hk f (µk )
a 2 i=1
12

Introduce the notations

h1 (b − a)
R1,1 = [f (a) + f (b)] = [f (a) + f (b)]
2 2
h2
R2,1 = [f (a) + f (b) + 2f (a + h2)]
2
HONG KONG BAPTIST UNIVERSITY 26
   
(b − a) b−a
= f (a) + f (b) + 2f a +
4 2
1
= [R1,1 + h1f (a + h2)]
2
1
R3,1 = [R2,1 + h2f (a + h3) + f (a + 3h3)]
2
and in general
 k−2

2
1 X
Rk,1 = Rk−1,1 + hk−1 f (a + (2i − 1)hk )
2 i=1

Suppose f ∈ C ∞[a, b], then the Composite Trapezoidal rule has the form
Z b ∞
X ∞
X
f (x)dx − Rk,1 = Kih2i
k = K 1 h 2
k+ Kih2i
k
a i=1 i=2

HONG KONG BAPTIST UNIVERSITY 27

Then we have

b ∞ ∞ ∞
Kih2i K1h2k X Kih2i
Z X X
f (x)dx − Rk+1,1 = Kih2i
k+1 =
k
= + k
a i=1 i=2
22i 4 i=2
4i

From these two equations we obtain

b ∞
Ki h2i
   
Rk+1,1 − Rk,1
Z X
k 2i
f (x)dx − Rk+1,1 + = i−1
− h k
a 3 i=2
3 4
∞  i−1

X Ki 1 − 4 2i
= i−1
h k
i=2
3 4

or
∞  i−1

K2 4 X Ki 1 − 4
intbaf (x)dx − Rk+1,2 = − hk + h2i
k
4 i=3
3 4i−1

HONG KONG BAPTIST UNIVERSITY 28

where
Rk+1,1 − Rk,1
Rk+1,2 = Rk+1,1 +
3
which gives the O(h4k ) approximation. Similarly, using Rk,2 and Rk+1,2 we can
obtain an O(h6k )approximation, and so on. In general,

Rk,j−1 − Rk−1,j−1
Rk,j = Rk,j−1 +
4j−1 − 1
j = 2, 3, · · · , n, k = j, j + 1, · · · , n.

will given an O(h2j

k−1 ) approximation. The process is given in the following table.

HONG KONG BAPTIST UNIVERSITY 29

 Table 4.9

R1,1
R2,1 R2,2
R3,1 R3,2 R3,3
R4,1 R4,2 R4,3 R4,4
.. .. .. .. ...
Rn,1 Rn,2 Rn,3 Rn,4 ··· Rn,n

Example 5.
st
R π2
Run the 1 step of the Romberg integration scheme to approximate − π cos xdx
2
up to k = 4.
Soln:

b−a
k = 1, n = 2k−1 = 1, hk−1 does not exist, h k = h1 = π =
n
HONG KONG BAPTIST UNIVERSITY 30
 b−a π
R1,1 = [f (a) + f (b)] = [cos(−π/2) + cos(π/2)] = 0
2 2
π
k = 2, n = 2, hk−1 = h1 = π, hk = h2 =
2

1
1 X 1
R2,1 = [R1,1 + h1 f (a + (2i − 1)h2)] = [R1,1 + πf (a + h2)]
2 i=1
2
1
= [R1,1 + π cos(0)] = 1.57080
2

π π
k = 3, n = 4, hk−1 = h2 = , h k = h3 =
2 4

1
1 π X
R3,1 = [R2,1 + ( ) f (a + (2i − 1)h3)]
2 2 i=1

HONG KONG BAPTIST UNIVERSITY 31

  
1 π −π π
= [R2,1 + ( ) cos( ) + cos( ) ] = 1.89612
2 2 4 4

π π
k = 4, n = 8, hk−1 = h3 = , h k = h4 =
4 8
 
1 π −3π −π π 3π
R3,1 = [R3,1 + ( ) cos( ) + cos( ) + cos( ) + cos( ) ]
2 2 8 8 8 8
= 1.97423

HONG KONG BAPTIST UNIVERSITY 32