Chapter 5 Initial-Value Problems for Ordinary Dierential Equations

5.1 Elementary Theory of Initial-Value Problems Dierential equations (DEs) are used to model problems in science and engineering that involve the change of some variable with respect to another. Most of these problems require the solution to a DE that satises a given initial condition (IVP): dy = f (t, y), a t b, y(a) = . (1) dt In most real-life situations, the DE that models the problem is too complicated to solve exactly. We look for approximate solutions using Numerical Methods. In actuality approximations to the solution will be generated at various values, called mesh points, and the approximate solution at other points in the interval are found by interpolation. Before considering the methods for approximating solutions, we rst need to establish the existence of the solution and also need to know whether small changes in the statement of the problem introduce correspondingly small changes in the solution, which is called the well-posedness of the problem. This is important because of the introduction of roundo error when numerical methods are used. Denition 1 (1) A function f (t, y) is said to satisfy a Lipschitz condition in the variable y on a set D R2 if a constant L > 0 exists with |f (t, y1 ) f (t, y2 )| L|y1 y2 |, whenever (t, y1 ), (t, y2 ) D. The constant L is called a Lipschitz constant for f. Example 1 If D = {(t, y)|1 t 2, 3 < y < 4}, then f (t, y) = t|y| satises a Lipschitz condition on D in the variable y. Theorem 1 Suppose f (t, y) is dened on a convex set D R2 . If a constant L > 0 exists with | f | L, y (t, y) D, (2)

then f satises a Lipschitz condition on D in the variable y with Lipschitz constant L. Theorem 2 Suppose D = {(t, y)|a t b, < y < } and that f (t, y) is continuous on D. If f satises a Lipschitz condition on D in the variable y, then the IVP y (t) = f (t, y), has a unique solution y(t) for a t b. Denition 2 The IVP y (t) = f (t, y), is said to be a well-posed problem if: 1. A unique solution, y(t), to the problem exists; 2. For any > 0, there exists a positive constant k( ), such that whenever | 0 | < [a, b], a unique solution, z(t), to dz = f (t, z) + (t), dt exists with |z(t) y(t)| < k( ) , a t b. a t b, z(a) = +
0,

a t b,

y(a) = ,

a t b,

y(a) = ,

and |(t)| <

on

(3)

Chapter 5 Initial-Value Problems for Ordinary Dierential Equations

Theorem 3 Suppose D = {(t, y)|a t b, < y < } and that f (t, y) is continuous on D. If f satises a Lipschitz condition on D in the variable y, then the IVP y (t) = f (t, y), is well-posed. Example 2 Let D = {(t, y)|0 t 1, < y < }, then the IVP: dy = y t2 + 1, dt is well-posed. 5.2 Eulers Method One of the simplest techniques for approximating solutions of DEs is known as Eulers Method or the method of tangent lines . It is seldom used in practice due to relatively big errors but discussion of the method here will serve as an introduction to the more advanced methods. The object of the method is to obtain an approximation to the well-posed IVP dy = f (t, y), a t b, y(a) = . dt We divide the time interval [a,b] into N equal subintervals and select the mesh points t + i = a + ih, for each i = 0, 1, 2, ..., N (4) 0 t 1, y(0) = 0.5 a t b, y(a) = ,

where h = (b a)/N . We will get approximation solutions, say wi at i = 1, ..., N . Suppose that y(t), the unique solution to (4), is in C 2 (a, b). By Taylors theorem for each i = 0, 1, 2, ..., N 1, y(ti+1 ) = y(ti ) + (ti+1 ti )y (ti ) + (ti+1 ti )2 y (i ), 2

for some i in (ti , ti+1 ). Since h = ti+1 ti and y(t) satises the IVP, we have y(ti+1 ) = y(ti ) + hf (ti , y(ti )) + h2 y (i ). 2
h2 2 y

(5) (i ).

Eulers method constructs wi y(ti ), for each i = 0, 1, 2, ..., N , by ignoring the remainder term Thus, Eulers method is w0 = , wi+1 = wi + hf (ti , wi ), for each i = 0, 1, 2, ..., N 1.

(6)

Note that we are actually doing the tangent line approximation at each step of the iteration. Equation (6) is called the dierence equation associated with Eulers method. Refer Algorithm 5.1 on pp.257 for the implementation of Eulers method. From Example 1 on pp.258 (see Table 5.1) we see that the error grows in a linear manner. We actually have the following theorem that shows it is true in general. Theorem 4 Suppose f (t, y) is continuous and satises a Lipschitz condition with constant L on D = {(t, y)|a t b, < y < } and that a constant M exists with |y (t)| M, for all t [a, b],

where y(t) is the unique solution to the IVP. Let w0 , w1 , ..., wN be the approximations generated by Eulers method for some positive integer N . Then, for each i = 0, 1, 2, ..., N , |y(ti ) wi | hM L(ti a) [e 1]. 2L (7)

Chapter 5 Initial-Value Problems for Ordinary Dierential Equations

The weakness of Theorem 4 lies in the requirement that a bound be known for the second derivative of the solution. Example 3 For the IVP; y = y t2 + 1, 0 t 2, y(0) = 0.5,

nd the approximations, actual errors and error bounds when h=0.2. It is expected from Theorem 4 that diminishing the step size would give correspondingly greater accuracy to the approximations. Neglected in the result of the Theorem 4 is, however, that roundo error plays in the choice of step size. As h becomes smaller, more calculations are necessary and more roundo error is expected. In actuality the dierence-equation Equation (6) is not used. We use instead an equation of the form u0 = + 0 , ui+1 = ui + hf (ti , ui ) + i+1 , for each i = 0, 1, 2, ..., N 1. (8)

where i denotes the roundo error associated with ui . Then we have the following error bound for the nite-digit approximations to yi given by Eulers method. Theorem 5 Let y(t) be the unique solution to the IVP (4) and u0 , u1 , ..., uN be the approximations obtained using (8). If |i | < for each i = 0, 1, 2, ..., N 1 and the hypotheses of Theorem 4 hold for the IVP, then for each i = 0, 1, 2, ..., N , |y(ti ) ui | 1 hM ( + )[eL(ti a) 1] + |0 |eL(ti a) . L 2 h
hM 2

(9)
+ h,

The error bound (9) is no longer linear in h but we can minimize error by minimizing E(h) =:

2 of which the minimum value occurs when h = M . Decreasing h beyond this value tends to increase the total error in the approximation. Normally, however, the value of is suciently small that this lower bound for h does not aect the operation of Eulers method.

5.3 Higher-Order Taylor Methods Denition 3 The dierence method w0 = , wi+1 = wi + h(ti , wi ), has local truncation error i+1 (h) = yi+1 (yi + h(ti , wi )) yi+1 yi = (ti , yi ) h h (11) for each i = 0, 1, 2, ..., N 1 (10)

for each i = 0, 1, 2, ..., N 1. By considering the equation y(ti+1 ) = y(ti ) + hf (ti , y(ti )) + h2 y (i ). 2 (12)

in the previous section, we see that Eulers method has i+1 (h) = h y (i ), 2 for some i (ti , ti+1 ).

Chapter 5 Initial-Value Problems for Ordinary Dierential Equations

If y (t) is known to be bounded by M on [a, b], this implies |i+1 (h)| = so the local truncation error in Eulers method is O(h). Taylor method of order n: y(ti+1 ) h2 hn (n) hn+1 (n+1) y (ti ) + + y (ti ) + y (i ) 2 n! (n + 1)! hn (n1) hn+1 (n) h2 f (ti , y(ti )) + f (i , y(i )) = y(ti ) + hf (ti , y(ti )) + f (ti , y(ti )) + + 2 n! (n + 1)! = y(ti ) + hy (ti ) +
n

h M, 2

Let T (n) (t, wi ) = f (ti , wi ) + h f (ti , wi ) + + h f (n1) (ti , wi ). Then the Algorithm is given by 2 n! w0 = , wi+1 = wi + hT (n) (t, wi ) for each i = 0, 1, 2, ...N 1 Example 4 y = y t2 + 1. 5.4 Runge-Kutta Methods Runge-Kutta Methods have the high-order local truncation error of the Taylor methods without the need of computing the derivatives of f (t, y). Theorem 6 Suppose that f (t, y) C n+1 (D) (,i.e., f (t, y) and all its partial derivatives of order less than or equal to n + 1 are continuous on D = {(t, y)|0 t 1, < y < }), and let (t0 , y0 ) D. For every (t, y) D, there exists between t and t0 and between y and y0 with f (t, y) = Pn (t, y) + Rn (t, y), where Pn (t, y) = f (t0 , y0 ) + [(t t0 ) f (t0 , y0 ) + (y y0 ) f (t0 , y0 )] t y +[ (tt0 ) 2 +
1 +[ n! n j=0 n C2 (t t0 )nj (y y0 )j tnjf j (t0 , y0 )] y
n 2

2f t2 (t0 , y0 )

f + (t t0 )(y y0 ) ty (t0 , y0 ) +

(yy0 )2 2 f 2 y 2 (t0 , y0 )]

and Rn (x)[ 1 (n + 1)!

2

n+1 n+1 C2 (t j=0 (y3)2 ] cos(2t 4

t0 )n+1j (y y0 )j

n+1 f (, )]. tn+1j y j

Example 5 f (t, y) = exp[ (t2) 4 (2, 3)

+ y 7). The second Taylor polynomial of f about

9 3 P2 (t, y) = 1 (t 2)2 2(t 2)(y 3) (y 3)2 4 4 Using Taylor polynomial in two variables we derive a Runge-Kutta method of order two. We will nd a formula that approximates h T (2) (t, y) = f (t, y) + f (t, y), 2 2 with error no greater than O(h ) in the form of a1 f (t + 1 , y + 1 ).

Chapter 5 Initial-Value Problems for Ordinary Dierential Equations

We need to determine values for a1 , 1 , and 1 . Expanding a1 f (t + 1 , y + 1 ) in its Taylor polynomial of degree one about (t, y) gives a1 f (t + 1 , y + 1 ) = a1 f (t, y) + a1 1 where R1 (t + 1 , y + 1 ) = f f (t, y) + a1 1 (t, y) + a1 R1 (t + 1 , y + 1 ), t y

2 1 2 f 2f 2 2f (, ) + 1 1 (, ) + 1 2 (, ) 2 2 t ty 2 y

for some between t and t + 1 and between y and y + 1 . Comparing this Taylor polynomial and T (2) (t, y) = f (t, y) + we get a1 = 1, 1 =
h 2

h h f h f f (t, y) = f (t, y) + (t, y) + (t, y) f (t, y), 2 2 t 2 y

and 1 = h f (t, y). So 2 T (2) (t, y) = f (t + h h , y + f (t, y)) 2 2

with error R1 (t + h h h2 2 f h2 2 f h2 2 f , y + f (t, y)) = (, ) + (, ) + (, ) = O(h2 ). 2 2 8 t2 4 ty 8 y 2

This is called the Midpoint method, and the algorithm is given by w0 = , wi+1 = wi + hf (ti + h , wi + h f (ti , wi )), 2 2 for each i = 0, 1, 2, ..., N 1. (13)

There are many dierent Runge-Kutta methods of order two. We list two important ones among them. Modied Euler Method: w0 = , wi+1 = wi + h [f (ti , wi ) + f (ti+1 , wi + hf (ti , wi ))], 2 Heuns Method: w0 = , 2 wi+1 = wi + h [f (ti , wi ) + 3f (ti + 3 h, wi + 2 hf (ti , wi )), 4 3 Runge-Kutta Method of Order Four: The most common Runge-Kutta method in use is of order four. This method employs a weighted average of the slopes at four points near the current point (ti , xi ). Runge-Kutta method of order four, in dierenceequation form, is given by the following. w0 = , k1 = hf (ti , wi ), k2 = hf (ti + h , wi + 1 k1 ), 2 2 k3 = hf (ti + h , wi + 1 k2 ), 2 2 k4 = hf (ti+1 , wi + k3 ), 1 wi+1 = wi + 6 (k1 + 2k2 + 2k3 + k4 ), for each i = 0, 1, 2, ..., N 1. (15) for each i = 0, 1, 2, ..., N 1. (14)

(16)

for each i = 0, 1, 2, ..., N 1. This method has local truncation error O(h4 ), provided the solution y(t) is in C 5 (a, b), i.e., has ve continuous derivatives.

Chapter 5 Initial-Value Problems for Ordinary Dierential Equations

5.9 Higher-Order Equations and Systems of Dierential Equations An mth-order system of rst-order initial-value problems has the form
du1 dt du2 dt

dum dt

= f1 (t, u1 , u2 , ..., um ), = f2 (t, u1 , u2 , ..., um ), . . . = fm (t, u1 , u2 , ..., um ),

(17)

for a t b, with the initial conditions u1 (a) = 1 , u2 (a) = 2 , ..., um (a) = m . To discuss existence and uniqueness of solutions to systems of equations, we need to extend the denition of the Lipschitz condition to functions of several variables. Denition 4 A function f (t, u1 , u2 , ..., um ), dened on a set D = {(t, u1 , u2 , ..., um )|a t b, < ui < , for each i = 1, 2, ..., m} is said to satisfy a Lipschitz condition D in the variables u1 , u2 , ..., um if a constant L > 0 exists with
m

|f (t, u1 , u2 , ..., um ) f (t, z1 , z2 , ..., zm )| L

j=1

|uj zj |,

whenever (t, u1 , u2 , ..., um ), (t, z1 , z2 , ..., zm ) D. The constant L is called a Lipschitz constant for f. By using the Mean Value theorem, it can be shown that if f is in C 1 (D) and if | f (t, u1 , u2 , ..., um ) | L, ui

for each i = 1, 2, ..., m and all (t, u1 , u2 , ..., um ) D, then f satsies a Lipschitz condition on D with Lipschitz constant L. Theorem 7 Suppose D = {(t, u1 , u2 , ..., um )|a t b, < ui < , for each i = 1, 2, ..., m} and let fi (t, u1 , u2 , ..., um ) is continuous and satises a Lipschitz condition on D. The system of rstorder dierential equations (17), subject to the accompanied initial conditions, has a unique solution u1 (t), u2 (t), ..., um (t) for a t b. Runge-Kutta for System of DEs (17) is given by the following. Set initial conditions wi,0 = i , for each i = 1, 2, ..., m, (18)

Suppose that the values w1,j , w2,j , ..., wm,j have been computed. We obtain w1,j+1 , w2,j+1 , ..., wm,j+1 by rst calculating, for each i = 1, 2, ..., m, k1,i = hfi (tj , w1,j , w2,j , ..., wm,j ), k2,i = hfi (tj + k3,i = hfi (tj + h 1 1 1 , w1,j + k1,1 , w2,j + k1,2 , ..., wm,j + k1,m ), 2 2 2 2 h 1 1 1 , w1,j + k2,1 , w2,j + k2,2 , ..., wm,j + k2,m ), 2 2 2 2

Chapter 5 Initial-Value Problems for Ordinary Dierential Equations

k4,i = hfi (tj + h, w1,j + k3,1 , w2,j + k3,2 , ..., wm,j + k3,m ), and then 1 wi,j+1 = wi,j + (k1,i + 2k2,i + 2k3,i + k4,i ), 6 (19)

for each i = 0, 1, 2, ..., N 1. Many important physical problems for example electrical circuits and vibrating systemsinvolve initial-value problems whose equations have orders higher that one. By relabeling the variables, we can reduce a higher-order DEs into a system of rst-order DEs and then apply Runge-Kutta method for systems. A general mth-order initial-value problem y (m) = f (t, y, y , ..., y (m1) ), with initial conditions y(a) = 1 , y (a) = 2 , ..., y (m1) (a) = m . can be converted into a system of equations. Let u1 (t) = y(t), u2 (t) = y (t),..., and um (t) = y (m1) (t). This produces the rst-order system
du1 dt du2 dt

a t b,

= = =

dy dt dy dt dy

= u2 , = u3 ,
(m1)

...
dum dt dt

(20)

= y (m) = f (t, y, y , ..., y (m1) ) = f (t, u1 , u2 , ..., um ),

for a t b, with the initial conditions u1 (a) = y(a) = 1 , u2 (a) = y (a) = 2 , ..., um (a) = y (m1) (a) = m . Example 6 Consider the second-order initial-value problem y 2y + 2y = e2t sin t, for 0 t 1 with y(0) = 0.4, y (0) = 0.6

Let u1 (t) = y(t) and u2 (t) = y (t). This transforms the equation into the system u1 (t) = u2 (t), u2 (t) = e2t sin t 2u1 (t) + 2u2 (t), with initial conditions u1 (0) = 0.4, u2 (0) = 0.6.

Chapter 5 Initial-Value Problems for Ordinary Dierential Equations

5.10 Stability Denition 5 (1) A one-step dierence-equation method with local truncation error i (h) at the i-th step is said to be consistent with the dierential equation it approximate if
h0 1iN

lim max |i (h)| = 0.

(2) A one-step dierence-equation method is said to be convergent with respect to the dierential equation it approximate if lim max |wi y(ti )| = 0.
h0 1iN

where yi = y(ti ) denotes the exact value of the solution of the DE and wi is the approximation obtained from the dierence-equation method at the i-th step. (3) A dierence method is said to be stable if its results depend continuously on the initial data. It is somewhat analogous to the condition of a dierential equation being well-posed. Example 7 The error-bound formula (7) for Eulers method
1iN

max |wi y(ti )|

hM L(ti a) [e 1] 2L

shows that Eulers method is convergent with respect to the dierential equation and the rate of convergence is O(h). Theorem 8 Suppose the initial-value problem y = f (t, y), atb y(a) = ,

is approximated by a one-step dierence method in the form w0 = , wi+1 = wi + h(ti , wi , h), for each i = 0, 1, 2, ..., N 1.

suppose also that a number h0 > 0 exists and that (t, w, h) is continuous and satises a Lipschitz condition in the variable w with Lipschitz constant L on the set D = {(t, w, h)|a t b, < w < , 0 h h0 } Then (i) The method is stable; (ii) The dierence method is convergent if and only if it is consistent, which is equivalent to (t, y, 0) = f (t, y), for all a t b;

(iii) If a function exists such that, for each i = 0, 1, 2, ..., N, the local truncation error i (h) satises |i (h)| (h) whenever 0 h h0 , then |wi y(ti )| Example 8 The modied Euler method given by w0 = , wi+1 = wi + is stable. h [f (ti , wi ) + f (ti+1 , wi + hf (ti , wi )), 2 for each i = 0, 1, 2, ..., N 1. (h) L(ti a) e L