MATH2114 Lecture Notes

Numerical Methods and Statistics for Engineers

Numerical Solution of
Ordinary Dierential Equations
Dr Eddie Ly
School of Mathematical & Geospatial Sciences (SMGS)
College of Sciences, Engineering & Health (SEH)
RMIT University, Melbourne, Australia
July 2014
Contents
1 Introduction 2
2 Inherent and Induced Instability 3
3 Taylor Series Methods 3
3.1 Eulers Method (First-Order Taylor Method) . . . . . . . . . . . . . . . . . . . . 4
3.2 Error in Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 Second-Order Taylor Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4 Runge-Kutta Methods 9
4.1 Second-Order Explicit Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.1.1 Modied Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.1.2 Scheme with c
2
= 2/3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.1.3 Ralstons Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.1.4 Improved Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 Fourth-Order Explicit Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . 12
5 Higher-Order Dierential Equations 16
6 Linear Shooting Methods 18
7 Self Assessment Exercises 23
1
2 Numerical Solution of Ordinary Dierential Equations
1 Introduction
In practice, many dierential equations do not possess a closed-form solution; that is, a
specic expression for the dependent variable cannot be found in terms of known functions.
However, a wide variety of numerical methods have been devised to obtain approximate
solutions to such equations. This document aims to introduce you to some of the numerical
methods used to solve the initial value problem,
y

= f(x, y) subject to y(x

0
) = y
0
,
and discusses some of the diculties associated with them. As usual, y

denotes dy/dx. At
rst sight, this equation may seem too elementary to be of general use, but it is possible to
extend its use to higher-order dierential equations by a simple extension of mathematical
notation. For example, consider the following second-order ordinary dierential equation,
d
2
y
dx
2
+ 2x
dy
dx
+ x
2
y = cos x subject to y(0) = 1 and y

(0) = 2.
If we put u(x) = y(x) and v(x) = y

(x), the above higher-order dierential equation can

be rewritten as a system of rst-order dierential equations,
du
dx
=
dy
dx
= v, u(0) = 1;
dv
dx
=
d
2
y
dx
2
= cos x 2xv x
2
u, v(0) = 2.
When we use a numerical technique to solve the initial value problem, we obtain
approximations to the solution y(x) at a discrete set of points. If the domain of interest is
divided into N equal subintervals of length h, then for a solution in the interval a x b,
where a = x
0
, we have
h =
b a
N
=
b x
0
N
.
We obtain approximations to the solution at the points x
1
= x
0
+ h, x
2
= x
0
+ 2h,
x
3
= x
0
+ 3h, etc.,
x
n
= x
0
+ nh for n = 1, 2, . . . , N.
It is important to realise the distinction between the true (exact) solution at x = x
n
,
given by y(x
n
), and the approximation to the solution at x = x
n
denotes by y
n
. In the
rst case, we have a function dened for all values of x (x
0
x b), and in the second
case we have a nite set of values y
n
for n = 0, 1, 2, . . . , N.
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 3
2 Inherent and Induced Instability
When using a numerical method with a small local truncation error, the accumulated error
can magnify to such an extent that the computed results are completely unacceptable
as solutions to the dierential equation. In such cases, we say that instability occurs.
Instability can occur when small variations in the conditions of the problem cause large
variations in the solution. This instability is inherent in the formulation of the problem,
and hence, is referred to as inherent instability. For instance, consider solving the following
initial value problem,
dy
dx
= 3(y x) subject to y(0) =
1
3
and x 0.
This equation is solved exactly to give
y(x) = x +
1
3
.
Suppose now that the initial condition is altered slightly to y(0) = 1/3+, where || 1.
The solution then becomes
y(x) = e
3x
+ x +
1
3
.
A small variation in the initial condition causes a change of e
3x
in the solution, which
for large x values, is large. For example, if = 10
8
and x = 10, the error is in the order
of 10
5
. This problem is said to exhibit inherent instability.
Instability can arise from the numerical method being used to solve the dierential
equation. In some cases, the methods will exhibit instability for step sizes which are
greater than some critical value. For some dierential equations, some methods exhibit
instability for all step sizes. Instability due to the numerical method is known as induced
instability.
3 Taylor Series Methods
Numerical methods for approximating solutions of
y

= f(x, y) subject to y(x

0
) = y
0
,
can be obtained from the Taylor series expansion of y = y(x) about x = x
n
:
y(x
n
+ h) = y(x
n
) + hy

(x
n
) +
h
2
2!
y

(x
n
) +
h
3
3!
y

(x
n
) + +
h
m
m!
y
(m)
(x
n
) + R
m
(x
n
),
where R
m
is the remainder when the Taylor series is truncated after the (m+1)th term,
R
m
(x
n
) =
h
m+1
(m + 1)!
y
(m+1)
(
n
);
n
(x
n
, x
n+1
).
4 Numerical Solution of Ordinary Dierential Equations
3.1 Eulers Method (First-Order Taylor Method)
If we truncate the Taylor series expansion of y = y(x) about x = x
n
after the second
term, we obtain a rst-order Taylor polynomial,
y(x
n
+ h) = y(x
n
) + hy

(x
n
) +
h
2
2
y

(
n
)
. .
= R
1
(x
n
)
;
n
(x
n
, x
n+1
).
We can then generate approximations in the following manner, with y

= f(x, y),
y
n+1
= y
n
+ hy

n
= y
n
+ hf(x
n
, y
n
) for n = 0, 1, 2, . . . , N 1.
This algorithm is known as the Eulers method or rst-order Taylor method. It is an
explicit formula as it expresses y
n+1
explicitly in terms of y
n
, and a single-step formula
as it relates two successive terms of the sequence, y
n
and y
n+1
.
EXAMPLE
Using Eulers method with h = 0.1, lets nd approximate solutions on the interval
[1, 2] of the following initial value problem,
dy
dx
= 1 +
y
x
subject to y(1) = 2.
We will compare the approximate solutions with the exact analytic solution,
y(x) = x(ln x + 2).
Eulers formula:
y
n+1
= y
n
+ hy

n
with y

n
= 1 +
y
n
x
n
.
With (x
0
, y
0
) = (1, 2), and putting n = 0, gives
y

0
= y

(x
0
, y
0
)
= 1 + 2/1
= 3,
x
1
= x
0
+ h
= 1 + 0.1
= 1.1,
y
1
= y
0
+ hy

0
= 2 + (0.1)(3)
= 2.3 .
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 5
The exact solution is y(1.1) 2.304841, so the error in the approximate solution is

Error

2.304841 2.3

0.004841.
Putting n = 1 gives
y

1
= y

(x
1
, y
1
)
3.0909,
x
2
= x
1
+ h
= 1.2,
y
2
= y
1
+ hy

1
2.6091.
The error in y
2
is about 0.009695, which is almost exactly twice the error in y
1
.
Proceeding in this manner, we can obtain y
3
, y
4
, . . . , y
10
, as shown in the table
below. Note that the error increases as n increases. The accuracy could have been
improved by taking a smaller value of h, but this involves more computation and a
possible accumulation of rounding-o errors.
approx soln exact soln
n x
n
y
n
y

n
x
n+1
y
n+1
y(x
n+1
) |Error|
0 1.0 2.0000 3.0000 1.1 2.3000 2.3048 0.0048
1 1.1 2.3000 3.0909 1.2 2.6091 2.6188 0.0097
2 1.2 2.6091 3.1743 1.3 2.9265 2.9411 0.0146
3 1.3 2.9265 3.2512 1.4 3.2516 3.2711 0.0195
4 1.4 3.2516 3.3226 1.5 3.5839 3.6082 0.0243
5 1.5 3.5839 3.3893 1.6 3.9228 3.9520 0.0292
6 1.6 3.9229 3.4518 1.7 4.2680 4.3021 0.0341
7 1.7 4.2680 3.5106 1.8 4.6191 4.6580 0.0389
8 1.8 4.6191 3.5662 1.9 4.9757 5.0195 0.0438
9 1.9 4.9757 3.6188 2.0 5.3376 5.3863 0.0487
6 Numerical Solution of Ordinary Dierential Equations
3.2 Error in Eulers Method
To analyse the error in Eulers method, consider the dierence between the relations for
the exact solution,
y(x
n+1
) = y(x
n
+ h)
= y(x
n
) + hy

(x
n
) +
h
2
2
y

(
n
)
= y(x
n
) + hf
_
x
n
, y(x
n
)
_
+
h
2
2
y

(
n
);
n
(x
n
, x
n+
1
),
and the approximate solution,
y
n+1
= y
n
+ hy

n
= y
n
+ hf(x
n
, y
n
).
Thus, the error in y
n+1
is
y(x
n+1
) y
n+1
= y(x
n
) y
n
+ h
_
f
_
x
n
, y(x
n
)
_
f(x
n
, y
n
)
_
. .
propagated error
+
h
2
2
y

(
n
)
. .
local truncation error
.
Using the Mean Value Theorem, and letting e
n
= y(x
n
) y
n
represents the error in the
nth step, the above relation becomes
e
n+1

_
1 + h
f
x
_
e
n
+
h
2
2
y

(
n
) ;
n
(x
n
, x
n+1
),
illustrating the relationship between the errors and the step sizes. After performing nu-
merous simplications, it can be shown that the error after n steps is given by
|e
n
| =
Mh
2
(x
n
x
0
), where M = max
x
0
xx
n

(x)

.
The error is O(h) (order of h) and Eulers method is said to converge with order one (due
to the power of h in the error bound).
3.3 Second-Order Taylor Method
We can obtain greater accuracy than in the Eulers method by truncating the Taylor
series after a greater number of terms. For example, the second-order Taylor method is
y
n+1
= y
n
+ hy

n
+
h
2
2
y

n
= y
n
+ hf
n
+
h
2
2
_
f
x
+ ff
y
_
n
for n = 0, 1, 2, . . . , N 1,
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 7
since y

= f(x, y) and applying the chain rule of dierentiation,

d
2
y
dx
2
=
d
dx
f(x, y)
=
f
x
+
f
y
y
x
=
f
x
+ f
f
y
.
Note that f
x
and f
y
denote f/x and f/y, respectively. The second-order Taylor
method is a single-step explicit formula like the Eulers method. In each step, the trun-
cation error is
h
3
6
y

(
n
) for
n
(x
n
, x
n+1
).
The truncation error is O
_
h
3
_
, and the global truncation error is O
_
h
2
_
. Therefore, the
second-order Taylor method converges with order two. The truncation error is sometimes
referred to as local truncation error, and the cumulative error is called the global trun-
cation error. This is certainly more accurate than the Eulers method, but it does have
some drawbacks. For instance, the second derivative may, in some situations, be dicult
to determine, and more computation is required in each step.
EXAMPLE
Using the second-order Taylor method with h = 0.1, lets nd approximate solutions
of the following initial value problem,
dy
dx
= 1 +
y
x
subject to y(1) = 2,
on the interval [1, 2].
Here y

n
= 1 +
y
n
x
n
with x
0
= 1 and y
0
= 2.
We obtain y

n
by dierentiating y

with respect to x:
d
2
y
dx
2
=
d
dx
_
1 +
y
x
_
=
1
x
dy
dx
+ y
d
dx
_
1
x
_
=
y

x
+ y
_
1
x
2
_
=
xy

y
x
2
.
Thus, y

n
=
x
n
y

n
y
n
x
2
n
provided x
n
= 0.
8 Numerical Solution of Ordinary Dierential Equations
Putting n = 0 (with x
0
= 1 and y
0
= 2) generates
y

0
= y

(x
0
, y
0
)
= y

(1, 2)
= 1 + 2/1
= 3,
y

0
= y

(x
0
, y
0
, y

0
)
= y

(1, 2, 3)
=
(1)(3) 2
1
2
= 1,
y
1
= y
0
+ hy

0
+
1
2
h
2
y

0
= 2 + (0.1)(3) + (0.5)(0.1)
2
(1)
= 2.305.
The exact solution is y(1.1) 2.304841, and the error is about

Error

2.304841 2.305

0.000159,
which is certainly much less than the error obtained with the Eulers method where
the error is about 0.004841 (from the last example). The table below presents the
approximate solutions and errors for y
1
to y
10
. As with the Eulers method, the
error is seen to increase as n increases, but is much less than that obtained with the
Eulers method.
approx soln exact soln
n x
n
y
n
y

n
y

n
x
n+1
y
n+1
y(x
n+1
) |Error|
0 1.0 2.0000 3.0000 1.0000 1.1 2.3050 2.3048 0.0002
1 1.1 2.3050 3.0955 0.90909 1.2 2.6191 2.6188 0.0003
2 1.2 2.6191 3.1826 0.83333 1.3 2.9415 2.9411 0.0004
3 1.3 2.9415 3.2627 0.76923 1.4 3.2716 3.2711 0.0005
4 1.4 3.2716 3.3369 0.71429 1.5 3.6089 3.6082 0.0007
5 1.5 3.6089 3.4059 0.66667 1.6 3.9528 3.9520 0.0008
6 1.6 3.9528 3.4705 0.62500 1.7 4.3030 4.3021 0.0009
7 1.7 4.3030 3.5312 0.58824 1.8 4.6591 4.6580 0.0011
8 1.8 4.6591 3.5884 0.55556 1.9 5.0207 5.0195 0.0012
9 1.9 5.0207 3.6425 0.52632 2.0 5.3876 5.3863 0.0013
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 9
4 Runge-Kutta Methods
To avoid the need to calculate high-order derivatives as in the Taylor series methods,
the Runge-Kutta methods attain the same accuracy as the Taylor approximations by
evaluating the slope f(x, y) at more points. The Runge-Kutta methods have the general
form,
y
n+1
= y
n
+ h(x
n
, y
n
; h) for n 0,
where (x
n
, y
n
; h) is called an increment function, which can be though of as some kind
of average slope of the solution on the interval [x
n
, x
n+1
]. For an R-stage Runge-Kutta
formula to have order p , the local truncation error must have order (p +1). It is possible
to obtain explicit R-stage Runge-Kutta formula of order R for R = 1, 2, 3 and 4. A 6-
stage method is required for a fth-order explicit formula. If we do not restrict ourselves
to explicit formula, then it is possible to obtain R-stage implicit Runge-Kutta formula of
order 2R. The only explicit rst-order Runge-Kutta method is Eulers method.
4.1 Second-Order Explicit Methods
For an explicit Runge-Kutta method of order two (R = 2), we have
y
n+1
= y
n
+
2

r=1
b
r
k
r
= y
n
+ b
1
k
1
+ b
2
k
2
for n = 0, 1, 2, . . . ,
where
k
1
= hf(x
n
+ c
1
h, y
n
+ a
11
k
1
+ a
12
k
2
),
k
2
= hf(x
n
+ c
2
h, y
n
+ a
21
k
1
+ a
22
k
2
),
c
1
= a
11
+ a
12
,
c
2
= a
21
+ a
22
.
Since this is an explicit scheme, a
ji
= 0 for i j. That is, a
11
= a
12
= a
22
= 0 and
a
21
= 0. The above expressions reduce to
c
1
= 0,
c
2
= a
21
,
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+ c
2
h, y
n
+ c
2
k
1
).
Clearly, this is a one-step scheme, requiring only the given initial value y
0
. The constants
c
2
, b
1
and b
2
, are to be determined such that the truncation error is O
_
h
3
_
and global
error is O
_
h
2
_
, so that the method is of order two.
10 Numerical Solution of Ordinary Dierential Equations
Because we can choose an innite number of values for c
2
to obtain an accuracy of
order two, there are an innite number of second-order explicit Runge-Kutta methods,
depending on the value of c
2
. Usually, c
2
= 1/2, 2/3, 3/4, 1 values are preferred.
4.1.1 Modied Eulers Method
If we choose c
2
= 1/2, we obtain the modied Eulers method :
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+
1
2
h, y
n
+
1
2
k
1
) for n = 0, 1, 2, . . . ,
y
n+1
= y
n
+ k
2
.
The number y
n
+
1
2
k
1
= y
n
+
1
2
hf(x
n
, y
n
) is the Euler solution at x
n+1/2
= x
n
+
1
2
h. Using
it, we obtain an approximation to the slope at x
n+1/2
, namely f(x
n+1/2
, y
n
+
1
2
k
1
). This
is then used to nd a rst-order straight line approximation to y at x
n+1
. Notice that the
ks are recurrence relationships, i.e. k
1
appears in the equation for k
2
. Because each k is a
functional evaluation, this recurrence makes Runge-Kutta methods ecient for computer
calculations.
4.1.2 Scheme with c
2
= 2/3
Choosing c
2
= 2/3, we obtain the following second-order explicit Runge-Kutta scheme,
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+
2
3
h, y
n
+
2
3
k
1
) for n = 0, 1, 2, . . . ,
y
n+1
= y
n
+
1
4
(k
1
+ 3k
2
).
4.1.3 Ralstons Method
Choosing c
2
= 3/4 provides a minimum bound on the truncation error for the second-order
explicit Runge-Kutta methods. Thus, with c
2
= 3/4, we have
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+
3
4
h, y
n
+
3
4
k
1
) for n = 0, 1, 2, . . . ,
y
n+1
= y
n
+
1
3
(k
1
+ 2k
2
).
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 11
4.1.4 Improved Eulers Method
With c
2
= 1, we obtain the improved Eulers method :
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+ h, y
n
+ k
1
) for n = 0, 1, 2, . . . ,
y
n+1
= y
n
+
1
2
(k
1
+ k
2
).
The number y
n
+ k
1
= y
n
+ hf(x
n
, y
n
) is the Euler solution at x = x
n+1
. Using it, we
obtain an approximation to the derivative at x
n+1
, namely f
_
x
n+1
, y
n
+hf(x
n
, y
n
)
_
. The
two slopes are then averaged (
1
2
(k
1
+ k
2
)) and added to y
n
to produce y
n+1
.
EXAMPLE
We will use the improved Eulers method with h = 0.1 to nd the approximate
solutions to
y

= x y subject to y(0) = 1,
at the points x = 0.1 and x = 0.2.
Exact solution: y(x) = x 1 + 2e
x
.
Improved Eulers method (with f(x, y) = x y and h = 0.1):
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+ h, y
n
+ k
1
) for n = 0, 1, 2, . . . ,
y
n+1
= y
n
+
1
2
(k
1
+ k
2
).
First iteration (n = 0) :
(x
0
, y
0
) = (0, 1),
x
1
= x
0
+ h
= 0.1,
f(x
0
, y
0
) = f(0, 1)
= 1,
k
1
= hf(x
0
, y
0
)
= (0.1)f(0, 1)
= 0.1,
k
2
= hf(x
1
, y
0
+ k
1
)
= (0.1)f(0.1, 0.9)
= 0.08,
12 Numerical Solution of Ordinary Dierential Equations
y
1
= y
0
+
1
2
(k
1
+ k
2
)
= 1 +
1
2
(0.1 +0.08)
= 0.91,

Error

= |y(x
1
) y
1
|
0.00032516.
Second iteration (n = 1) :
(x
1
, y
1
) = (0.1, 0.91),
x
2
= x
1
+ h
= 0.2,
f(x
1
, y
1
) = f(0.1, 0.91)
= 0.81,
k
1
= hf(x
1
, y
1
)
= (0.1)f(0.1, 0.91)
= 0.081,
k
2
= hf(x
2
, y
1
+ k
1
)
= (0.1)f(0.2, 0.829)
= 0.0629,
y
2
= y
1
+
1
2
(k
1
+ k
2
)
= 0.91 +
1
2
(0.081 0.0629)
= 0.83805,

Error

= |y(x
2
) y
2
|
0.00058849.
The approximate solutions are (x
1
, y
1
) = (0.1, 0.910) and (x
2
, y
2
) = (0.2, 0.838).
4.2 Fourth-Order Explicit Runge-Kutta Methods
The general explicit Runge-Kutta method of order four has the form,
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+ c
2
h, y
n
+ c
2
k
1
),
k
3
= hf
_
x
n
+ c
3
h, y
n
+ a
32
k
2
+ (c
3
a
32
)k
1
_
,
k
4
= hf
_
x
n
+ c
4
h, y
n
+ a
43
k
3
+ a
42
k
2
+ (c
4
a
43
a
42
)k
1
_
,
y
n+1
= y
n
+ b
1
k
1
+ b
2
k
2
+ b
3
k
3
+ b
4
k
4
for n = 0, 1, 2, . . . .
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 13
The ten unknows (b
1
, b
2
, b
3
, b
4
, c
2
, c
3
, c
4
, a
32
, a
42
, a
43
) are determined such that the trunca-
tion error is O
_
h
5
_
, and the global error is O
_
h
4
_
. After expanding using Taylor series
expansion, we would obtain eight equations in ten unknowns. Two of the most commonly
used fourth-order Runge-Kutta methods are Kutta-Simpson 1/3 rule and Kutta-Simpson
3/8 rule.
kutta-simpson 1/3 rule
The scheme for the Kutta-Simpson 1/3 rule, using Runges coecients, is given by
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+
1
2
h, y
n
+
1
2
k
1
),
k
3
= hf(x
n
+
1
2
h, y
n
+
1
2
k
2
),
k
4
= hf(x
n
+ h, y
n
+ k
3
),
y
n+1
= y
n
+
1
6
(k
1
+ 2k
2
+ 2k
3
+ k
4
) for n = 0, 1, 2, . . . .
If f(x, y) does not depend on y, then this formula reduces to the usual Simpsons inte-
gration formula for interval [x
n
, x
n+1
] :
y
n+1
= y
n
+
1
6
_
k
1
+ 2k
2
+ 2k
3
+ k
4
_
= y
n
+
h
6
_
f(x
n
, y
n
) + 2f(x
n
+
1
2
h, y
n
+
1
2
k
1
) + 2f(x
n
+
1
2
h, y
n
+
1
2
k
2
) + f(x
n
+ h, y
n
+ k
3
)
_
= y
n
+
h
6
_
f(x
n
) + 2f(x
n
+
1
2
h) + 2f(x
n
+
1
2
h) + f(x
n
+ h)
_
= y
n
+
h
6
_
f(x
n
) + 4f(x
n
+
1
2
h) + f(x
n
+ h)
_
.
kutta-simpson 3/8 rule
The scheme for the Kutta-Simpson 3/8 rule, using Runges coecients, is given by
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+
1
3
h, y
n
+
1
3
k
1
),
k
3
= hf(x
n
+
2
3
h, y
n
+ k
2

1
3
k
1
),
k
4
= hf(x
n
+ h, y
n
+ k
3
k
2
+ k
1
),
y
n+1
= y
n
+
1
8
(k
1
+ 3k
2
+ 3k
3
+ k
4
) for n = 0, 1, 2, . . . .
14 Numerical Solution of Ordinary Dierential Equations
EXAMPLE
We will use the Kutta-Simpson 1/3 rule with h = 0.1 to nd approximate solutions
to y

= x y, with y(0) = 1, at the points x = 0.1 and x = 0.2.

Exact solution: y(x) = x 1 + 2e
x
Kutta-Simpson 1/3 rule (with f(x, y) = x y and h = 0.1) :
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+
1
2
h, y
n
+
1
2
k
1
),
k
3
= hf(x
n
+
1
2
h, y
n
+
1
2
k
2
),
k
4
= hf(x
n
+ h, y
n
+ k
3
),
y
n+1
= y
n
+
1
6
(k
1
+ 2k
2
+ 2k
3
+ k
4
) for n = 0, 1, 2, . . . .
First iteration (n = 0) :
(x
0
, y
0
) = (0, 1),
x
1/2
= x
0
+ h/2
= 0.05,
x
1
= x
0
+ h
= 0.1,
k
1
= hf(x
0
, y
0
)
= (0.1)f(0, 1)
= 0.1,
k
2
= hf(x
1/2
, y
0
+
1
2
k
1
)
= (0.1)f(0.05, 0.95)
= 0.09,
k
3
= hf(x
1/2
, y
0
+
1
2
k
2
)
= (0.1)f(0.05, 0.955)
= 0.0905,
k
4
= hf(x
1
, y
0
+ k
3
)
= (0.1)f(0.1, 0.9095)
= 0.08095,
y
1
= y
0
+ (k
1
+ 2k
2
+ 2k
3
+ k
4
)/6
= 1 + (0.1 + 2(0.09) + 2(0.0905) 0.08095)/6
= 0.909675,
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 15

Error

= |y(x
1
) y
1
|
1.639281
_
10
7
_
.
Second iteration (n = 1) :
(x
1
, y
1
) = (0.1, 0.909675),
x
3/2
= x
1
+ h/2
= 0.15,
x
2
= x
1
+ h
= 0.2,
k
1
= hf(x
1
, y
1
)
= (0.1)f(0.1, 0.909675)
= 0.0809675,
k
2
= hf(x
3/2
, y
1
+
1
2
k
1
)
= (0.1)f(0.15, 0.869191)
= 0.07191913,
k
3
= hf(x
3/2
, y
1
+
1
2
k
2
)
= (0.1)f(0.15, 0.87371544)
= 0.07237154,
k
4
= hf(x
2
, y
1
+ k
3
)
= (0.1)f(0.2, 0.83730346)
= 0.06373035,
y
2
= y
1
+ (k
1
+ 2k
2
+ 2k
3
+ k
4
)/6
= 0.909675 + (0.0809675 + 2(0.07191913) + 2(0.07237154)
0.06373035)/6
= 0.83746180,

Error

= |y(x
2
) y
2
|
2.966565
_
10
7
_
.
Approximate solutions : (x
1
, y
1
) = (0.1, 0.9097), (x
2
, y
2
) = (0.2, 0.8375).
16 Numerical Solution of Ordinary Dierential Equations
5 Higher-Order Dierential Equations
Consider nding approximate solutions of the system of rst-order equations,
du
dx
= f(x, u, v); u(x
0
) = u
0
,
dv
dx
= g(x, u, v); v(x
0
) = v
0
.
Any of the methods presented here can be modied to solve the above system of dierential
equations. For example, if we employ the improved Eulers method to solve the equations,
we modify the scheme as follows :
k
1
= hf(x
n
, u
n
, v
n
),

1
= hg(x
n
, u
n
, v
n
),
k
2
= hf(x
n
+ h, u
n
+ k
1
, v
n
+
1
),

2
= hg(x
n
+ h, u
n
+ k
1
, v
n
+
1
),
x
n+1
= x
n
+ h,
u
n+1
= u
n
+
1
2
(k
1
+ k
2
),
v
n+1
= v
n
+
1
2
(
1
+
2
) for n = 0, 1, 2, . . . .
EXAMPLE
Using the above method for the following system of rst-order dierential equations,
du
dx
= v ; u(0) = 1,
dv
dx
= x + u
2
; v(0) = 0,
compute the approximate solutions at x = 0.1 and x = 0.2 using step size h = 0.1.
Here f(x
n
, u
n
, v
n
) = v
n
and g(x
n
, u
n
, v
n
) = x
n
+ u
2
n
.
Initial data :
x
0
= 0,
u
0
= u(x
0
) = 1,
v
0
= v(x
0
) = 0,
f(x
0
, u
0
, v
0
) = v
0
= 0,
g(x
0
, u
0
, v
0
) = x
0
+ u
2
0
= 1.
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 17
First iteration (n = 0) :
x
1
= x
0
+ h
= 0.1,
k
1
= hf(x
0
, u
0
, v
0
)
= (0.1)f(0, 1, 0)
= 0,

1
= hg(x
0
, u
0
, v
0
)
= (0.1)g(0, 1, 0)
= 0.1,
k
2
= hf(x
1
, u
0
+ k
1
, v
0
+
1
)
= (0.1)f(0.1, 1, 0.1)
= 0.01,

2
= hg(x
1
, u
0
+ k
1
, v
0
+
1
)
= (0.1)g(0.1, 1, 0.1)
= 0.11,
u
1
= u
0
+ (k
1
+ k
2
)/2
= 1 + (0 + 0.01)/2
= 1.005,
v
1
= v
0
+ (
1
+
2
)/2
= 0 + (0.1 + 0.11)/2
= 0.105.
Second iteration (n = 1) :
x
2
= x
1
+ h
= 0.2,
k
1
= hf(x
1
, u
1
, v
1
)
= (0.1)f(0.1, 1.005, 0.105)
= 0.0105,

1
= hg(x
1
, u
1
, v
1
)
= (0.1)g(0.1, 1.005, 0.105)
= 0.1110025,
18 Numerical Solution of Ordinary Dierential Equations
k
2
= hf(x
2
, u
1
+ k
1
, v
1
+
1
)
= (0.1)f(0.2, 1.0155, 0.2160025)
= 0.02160025,

2
= hg(x
2
, u
1
+ k
1
, v
1
+
1
)
= (0.1)g(0.2, 1.0155, 0.2160025)
= 0.12312403,
u
2
= u
1
+ (k
1
+ k
2
)/2
= 1.005 + (0.0105 + 0.02160025)/2
= 1.02105013,
v
2
= v
1
+ (
1
+
2
)/2
= 0.105 + (0.1110025 + 0.12312403)/2
= 0.22206326.
Approximate solutions are
(x
1
, u
1
, v
1
) = (0.1, 1.005, 0.105) and (x
2
, u
2
, v
2
) = (0.2, 1.021, 0.222).
6 Linear Shooting Methods
Considering nding the solution y(x) to the following ordinary dierential equation for
a x b :
d
2
y
dx
2
+ 2
dy
dx
+ y = x; a x b.
When we apply the Runge-Kutta methods to obtain the approximate solution to the
above equation, the methods require all conditions to be specied at some initial point x
0
within the interval, usually at x
0
= a. For example, if both the initial conditions y and
y

are given at the same initial point, x

0
= a, we have
d
2
y
dx
2
+ 2
dy
dx
+ y = x subject to y(a) = y
a
and y

(a) = y

a
.
The above dierential equation, together with the conditions specied at the same initial
point, forms an initial value problem. Note that n number of conditions for y(x) are
required for an nth-order ordinary dierential equation. Suppose now that we want to
solve the same ordinary dierential equation, but subject to the conditions specied at
two dierent points within the interval :
d
2
y
dx
2
+ 2
dy
dx
+ y = x subject to y(a) = y
a
and y(b) = y
b
.
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 19
In this case, we have a boundary value problem, because the conditions are given at the
boundaries of the interval, that is, at x = a and x = b. In the following discussion, we
assume that the boundary value problem under consideration has a unique solution.
We can still use the Runge-Kutta methods to solve this boundary value problem even
though we only have one piece of information at x = a, that is, y(a) = y
a
. Since we have
y(a) but not y

(a), we need to guess a value for y

(a), and then solve the problem by the

Runge-Kutta method for up to x = b. We then compare the value of y(x) we get at x = b,
that is y(b), with what we need at x = b (the specied boundary condition). If y(b) does
not match the specied boundary condition there, then we try and guess another value
for y

(a), and solve the problem again. This method of solution is known as the linear
shooting methods for boundary value problems.
EXAMPLE
Consider solving the following boundary value problem for 0 x 1,
d
2
y
dx
2
+ 2
dy
dx
+ y = x subject to y(0) = 1 and y(1) = 2.
In the linear shooting methods, we need to convert the above boundary value prob-
lem into an initial value problem (which is appropriate for the numerical methods
such as the Runge-Kutte methods) by guessing y

(0) = , so that y(1) = 2. That

is, we consider
d
2
y
dx
2
+ 2
dy
dx
+ y = x subject to y(0) = 1 and y

(0) = .
Rewritting the above second-order ordinary dierential equation as a couple rst-
order dierential equations by letting u(x) = y(x) and v(x) = y

(x) :
du
dx
= f(x, u, v) = v ; u(0) = 1 ;
dv
dx
= g(x, u, v) = x 2v u; v(0) = .
Suppose we employ the improved Eulers method (explicit second-order Runge-
Kutta scheme) to solve the above system of dierential equations, using step size
h = 0.1 :
k
1
= hf(x
n
, u
n
, v
n
);

1
= hg(x
n
, u
n
, v
n
);
k
2
= hf(x
n+1
, u
n
+ k
1
, v
n
+
1
);

2
= hg(x
n+1
, u
n
+ k
1
, v
n
+
1
);
20 Numerical Solution of Ordinary Dierential Equations
x
n+1
= x
n
+ h;
u
n+1
= u
n
+
1
2
(k
1
+ k
2
);
v
n+1
= v
n
+
1
2
(
1
+
2
) for n = 0, 1, 2, . . . , n.
First guess y

(0) = = 1 :
n x
n
u
n
v
n
k
1

1
k
2

2
0 0 1 1 0.100000 0.300000 0.070000 0.240000
1 0.1 1.085000 0.730000 0.073000 0.244500 0.048550 0.192900
2 0.2 1.145775 0.511300 0.051130 0.196838 0.031446 0.152583
3 0.3 1.187063 0.336590 0.033659 0.156024 0.018057 0.118185
4 0.4 1.212921 0.199485 0.019948 0.121189 0.007830 0.088946
5 0.5 1.226810 0.094417 0.009442 0.019564 0.000285 0.064196
6 0.6 1.231673 0.016537 0.001654 0.066475 0.004994 0.043345
7 0.7 1.230003 0.038373 0.003837 0.045326 0.008370 0.025877
8 0.8 1.223900 0.073974 0.007397 0.027595 0.010157 0.011336
9 0.9 1.215123 0.093440 0.009344 0.012824 0.010626 0.000675
10 1 1.205137 0.099515
Second guess y

(0) = = 3 :
n x
n
u
n
v
n
k
1

1
k
2

2
0 0 1 3 0.300000 0.700000 0.230000 0.580000
1 0.1 1.265000 2.360000 0.236000 0.588500 0.177150 0.484400
2 0.2 1.471575 1.823550 0.182355 0.491868 0.133168 0.401730
3 0.3 1.629337 1.376752 0.137675 0.408284 0.096847 0.330395
4 0.4 1.746598 1.007412 0.100741 0.336142 0.067127 0.268988
5 0.5 1.830532 0.704847 0.070485 0.274023 0.043082 0.216267
6 0.6 1.887315 0.459703 0.045970 0.220672 0.023903 0.171135
7 0.7 1.922252 0.263799 0.026380 0.174985 0.008881 0.132626
8 0.8 1.939883 0.109994 0.010999 0.135987 0.002599 0.099890
9 0.9 1.944083 0.007945 0.000794 0.102819 0.011076 0.072176
10 1 1.938147 0.095442
Thus, for = 1, y(1) u
10
= 1.205137 and for = 3, y(1) u
10
= 1.938147. But,
the specic condition is y(1) = 2. We can use these two results to give us a better
estimate of to obtain y(1) = 2 by line of best ts between these results :
= m u
10
+ c,
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 21
where m and c are constants. When u
10
= 1.205137, = 1 and when u
10
= 1.938147,
= 3 :
1 = m(1.205137) + c and 3 = m(1.938147) + c .
Solving them provides
m = 2.728476 and c = 2.288187 .
That is,
= 2.728476u
10
2.288187 .
Using the above relation, for y(1) = u
10
= 2, we require = 3.168765.
Third guess y

(0) = = 3.168765 :
n x
n
u
n
v
n
k
1

1
k
2

2
0 0 1 3.168765 0.316877 0.733753 0.243501 0.608690
1 0.1 1.280189 2.497543 0.249754 0.617528 0.188002 0.508997
2 0.2 1.499067 1.934281 0.193428 0.516763 0.141752 0.422753
3 0.3 1.666657 1.464523 0.146452 0.429570 0.103495 0.348301
4 0.4 1.791631 1.075587 0.107559 0.354280 0.072131 0.284180
5 0.5 1.881475 0.756357 0.075636 0.289419 0.046694 0.229099
6 0.6 1.942640 0.497098 0.049710 0.233684 0.026341 0.181918
7 0.7 1.980666 0.289297 0.028930 0.185926 0.010337 0.141634
8 0.8 2.000299 0.125517 0.012552 0.145133 0.001962 0.107362
9 0.9 2.005594 0.000730 0.000073 0.110413 0.011114 0.078323
10 1 2.000000 0.095099
EXAMPLE
A particular boundary value problem has the following boundary conditions:
y(0) = 1 and y(2) = 1.
If two applications of the shooting method, with y

(0) = 1 and y

(0) = 3, produce
values y(2) = 2 and y(2) = 7/2, respectively, nd an improved estimate for the
value of y

(0).
solution:
Let y

(0) = . Line of best ts :

= m y(2) + c .
22 Numerical Solution of Ordinary Dierential Equations
To nd constants m and c, we use = 1 when y(2) = 2 and = 3 when y(2) = 7/2,
resulting m = 4/3 and c = 5/3 :
=
4
3
y(2)
5
3
.
Thus, for y(2) = 1, we require y

(0) = = 1/3.
Lecture Notes MATH2114 Numerical Methods & Statistics for Engineers 23
7 Self Assessment Exercises
[1] Putting u(x) = y(x) and v(x) = y

(x), reformulate the following initial value

problem as a coupled system of rst-order dierential equations,
x
2
d
2
y
dx
2
+ x
dy
dx
+
_
4 x
2
_
y = 0 subject to y(0) = y

(0) = 1.
[2] Redene the third-order nonhomogeneous ordinary dierential equation,
d
3
x
dt
3
+
d
2
x
dt
2
+ 2x = t
2
subject to x(0) = 1, x(0) = 0 and x(0) = 1,
as a system of rst-order dierential equations.
[3] For the initial value problem y

= x + y
2
with y(0) = 1, nd an approximation at
x = 0.2 using two steps of the Eulers method.
[4] For the initial value problem y

= x+y
2
subject to y(0) = 1, nd an approximation
at x = 0.2 using two steps of the second-order Taylor method.
[5] Consider the initial value problem,
dy
dx
=
y
y + x
subject to y(0) = 2.
(a) Find an approximation at x = 0.2 using two steps of Eulers method.
(b) Find an approximation at x = 0.2 using four steps of Eulers method.
(c) Compare your results in part (a) and (b) with the exact answer given by
y(x) =

x
2
+ 4 x.
[6] With step size h = 0.1, use the improved Eulers method to nd approximate
solutions to
dy
dx
= 1 +
y
x
subject to y(1) = 2
at the points x = 1.1 and x = 1.2.
[7] Use the modied Eulers method with (a) h = 0.1 and (b) h = 0.05 to nd an
approximation to the solution of
dy
dx
= xy subject to y(0) = 1
at the point x = 0.2.
24 Numerical Solution of Ordinary Dierential Equations
[8] Use the third-order Runge-Kutta method,
k
1
= hf(x
n
, y
n
),
k
2
= hf(x
n
+
1
2
h, y
n
+
1
2
k
1
),
k
3
= hf(x
n
+ h, y
n
+ 2k
2
k
1
) for n = 0, 1, 2, . . . ,
y
n+1
= y
n
+
1
6
(k
1
+ 4k
2
+ k
3
),
to nd approximate solutions to
dy
dx
= x + y
2
subject to y(0) = 1,
at the points x = 0.1, 0.2 and 0.3.
[9] Use the Kutta-Simpson 1/3 rule with h = 0.1 to nd approximate solutions to
dy
dx
= 1 +
y
x
subject to y(1) = 2,
at the points x = 1.1 and x = 1.2.
[10] For the initial value problem,
dy
dx
=
y
y x
with y(1) = 1.5,
nd an approximation to the solution at x = 1.4 using two steps of the Kutta-
Simpson 3/8 rule.
[11] (a) Express the second-order initial value problem,
d
2
y
dx
2
+
_
y
2
1
10
_
dy
dx
+ y = 0 subject to y(0) = 1 and y

(0) = 0,
as a coupled pair of rst-order dierential equations.
(b) Use two steps of the Kutta-Simpson 1/3 rule to nd an approximation to the
solution at x = 0.2.
(c) Use two steps of the Kutta-Simpson 3/8 rule to nd an approximation to the
solution at x = 0.2.