Numerical methods for differential equations

Solution of First order differential equation Solution of High order differential equation In general, as the accuracy of the approximation is increased, so is the complexity of the programming involved. The solution accuracy of differential depends on the step size. Euler method The Euler method is the simplest algorithm for numerical solution of a differential equation. It usually gives the least accurate results !ut provides a !asis for understanding more sophisticated methods. "onsider the equation
dy = r $t # y dt

%here r $t # is a &no%n function. From the definition of the derivative, dy y $t + t # y $t # = lim t ' dt t y $t + t # y $t # = r $t # y t (ssume that the varia!les on the right hand side of a!ove equation are constant over t . )e have the time interval
y $t + t # = y $t # + r $t # y $t # t

t is called the step size. For convenience, %here increment

y $t k +* # = y $t k # + r $t k # y $t k #t t k +* = t k + t
= f $t , y # )e can summarize the Euler method for the general first+order equation y as y $t k +* # = y $t k # + t f -t k , y $t k #,
= ry , r = *', y $'# = . , plots the y tra/ectory over the range Example * y

' t '.0

Step size t = '.'. , %hich is .'1of the time constant. Exactly solution
dy = *' y dt dy *'dt ln y = *'t + c y y = e =*'t C

y $ '# = . C = .

true solution y $t # = .e *'t , time constant =* *' ='.* 2atla! clear3 r 4 +*'3 delta 4 '.'.3 y$*# 4 .3 & 4 '3 for time 4 -delta delta '.0, & 4 & 5 *3 y$&5*# 4 y$&#5r6y$&#6delta3 end t 4 -' delta '.0,3 y7true 4 .6exp$+*'6t#3 plot$t,y,8o8,t,y7true#3

Simulation results:
2 1.8 1.6 1.4 1.2 y trajectory 1 0.8 0.6 0.4 0.2 0

0.05

0.1

0.15

0.2

0.25 0.3 Time(sec.)

0.35

0.4

0.45

0.5

There is some noticea!le error from simulation results. If %e use a step size equal to 01 of the time constant $ t = '.''0 #, the error %ould not !e noticea!le on the plot. Example . To illustrate the difficulties caused !y an oscillating solution, consider the follo%ing equation
= sin t y

for y$'# 4 ' and ' t 9 . True solution

dy = sin t dt dy = sin dt

y = cos t + c
y $'# = ' c =* y $t # =* cos t

2atla! clear3 delta4.6pi:*;3 y$*#4'3 &4'3 for time4-delta delta 96pi, &4&5*3 y$&5*#4y$&#5sin$time+delta#6delta3 end t7true4-' delta:*' 96pi,3 y7true4*+cos$t7true#3 t4-' delta 96pi,3 plot$t,y,8o8,t7true,y7true#3 end

2 1.8 1.6 1.4 1.2 y trajectory 1 0.8 0.6 0.4 0.2 0

6 8 Time(sec.)

10

12

14

t = . .> <x= t = . *; <o= Ho%ever, very small step sizes require longer run times and can result in a large accumulated error !ecause of round+off effects. So %e see& !etter algorithms to use for more challenging applications. )e no% consider t%o methods that are more po%erful than the Euler method. The first is a predictor+corrector method. The second method is the ?unge+@utta family of algorithms.

Predictor-error method (Modified Euler method)

The Euler method can have a serious deficiency in pro!lems %here the varia!les are rapidly changing !ecause the method assumes the varia!les are constant over the time t . Ane %ay of improving the method is to use a !etter approximation to the interval right side of the equation
dy = f $t , y # dt

The Euler approximation is

y $t k +* # = y $t k # + t f -t k , y $t k #,

Suppose instead %e use the average of the right side of equation on the interval $t k , t k +* # . This gives

y $t k +* # = y $t k # +

t ( f k + f k +* ) .

%here
f k = f $t k , y $t k ## f k +* = f $t k +* , y $t k +* ##

So, use Euler method to o!tain y $t k +* # implemented. In summary, Euler predictor x k +* = y k + h f $t k , y k # Bredictor+error y k +* = y k +
' t '.0

such that predictor+error method can !e

h [ f $t k , y k # + f $t k +* , x k +* #] .

= ry , r = *', y $'# = . , plots the y tra/ectory over the range Example ; y

Step size t = '.'. , %hich is .'1of the time constant. $the same as Example *# 2atla! clear3 r 4 +*'3 delta 4 '.'.3 y$*# 4 .3 & 4 '3 for time 4 -delta delta '.0, & 4 & 5 *3 x$&5*#4y$&#5r6y$&#6delta3 y$&5*#4 y$&#5$delta:.#6$r6y$&#5r6x$&5*##3 end t 4 -' delta '.0,3 y7true 4 .6exp$+*'6t#3 plot$t,y,8o8,t,y7true#3

2 1.8 1.6 1.4 1.2 y trajectory 1 0.8 0.6 0.4 0.2 0

0.05

0.1

0.15

0.2

0.25 0.3 Time(sec.)

0.35

0.4

0.45

0.5

)e can find that there is less error than %ith the Euler method using the same step size. Example 9 the same as Example . clear3 delta4.6pi:*;3 y$*#4'3 &4'3 for time4-delta delta 96pi, &4&5*3 y$&5*#4y$&#5$delta:.#6$sin$time+delta#5sin$time##3 end t7true4-' delta:*' 96pi,3 y7true4*+cos$t7true#3 t4-' delta 96pi,3 plot$t,y,8o8,t7true,y7true#3 end

>

1.5

1 y trajectory

0.5

-0.5

6 8 Time(sec.)

10

12

14

From a!ove simulation results, %e can find that the modified Euler method has less error than %ith the Euler method, !ut there is still some noticea!le error near the pea&s and valleys %here the solution is rapidly changing. This error can !e decreased !y decreasing the step size.

Runge-Kutta methods
The ?unge+@utta algorithm lets us solve a differential equation numerically $ie. approximately# %ithout require the calculation of partial derivatives. "onsider the single varia!le pro!lem xC 4 f $t, x# %ith initial condition x$'# 4 x'. Suppose that xn is the value of the varia!le at time tn. The ?unge+@utta formula ta&es xn and tn and calculates an approximation for xn5* at a !rief time later, tn5h. It uses a %eighted average of approximated values of f $t, x# at several times %ithin the interval $tn, tn5h#. The formula is given !y xn5* 4 xn 5 hD> $k* 5 .k. 5 .k; 5 k9# %here k* 4 f $tn, xn# k. 4 f $tn 5 hD., xn 5 hD. k*# k; 4 f $tn 5 hD., xn 5 hD. k.# k9 4 f $tn 5 h, xn 5 h k;# To run the simulation, %e simply start %ith x' and find x* using the formula a!ove. Then %e plug in x* to find x. and so on. The ?unge+@utta algorithm is &no%n to !e very accurate and %ell+!ehaved for a %ide
E

range of pro!lems. The Runge-Kutta algorithm can be easily extended to a set of first order differential equations. You wind up with essentially the same formulas shown above, but all the variables (except for time) are vectors. To give an idea of how it works, here is an example where we expand the vector notation. That is, instead of using the highly compact vector notation, we write out all the components of the vector. Suppose there are only two variables, x, y and two differential equations x' = f (t, x, y) y' = g (t, x, y) Again we let xn be the value of x at time tn, and similarly for yn. Then the formulas for the Runge-Kutta algorithm are xn+1 = xn + h6 (k1 + 2k2 + 2k3 + k4) yn+1 = yn + h6 (j1 + 2j2 + 2j3 + j4) where k1 = f (tn, xn, yn) j1 = g (tn, xn, yn) k2 = f (tn + h2, xn + h2 k1, yn + h2 j1) j2 = g (tn + h2, xn + h2 k1, yn + h2 j1) k3 = f (tn + h2, xn + h2 k2, yn + h2 j2) j3 = g (tn + h2, xn + h2 k2, yn + h2 j2) k4 = f (tn + h, xn + h k3, yn + h j3) j4 = g (tn + h, xn + h k3, yn + h j3) Given starting values x0, y0 we can plug them into the formula to find x1, y1. Then we can plug in x1, y1 to find x2, y2 and so on. Derivation of the second-order Runge-Kutta: Consider 6 = f $t , x $t ## , t ', x$'# = x ' x x n +* = x n + h { a* f $t n , x n # + a . f $t n + h, x n + f $t n , x n #} (a) Choose the a* , a . , , such that the right-hand side of above equation approximates the right-hand side of the following second-order Taylor series with an error o$ h . # .
x n +* = x n + h f $t n , x n # + h. [ f x $t n , x n # f $t n , x n # + f t $t n , x n #] .

(b)

Review: linearization for a given equilibrium point

* = f * $ x* , x . # x . = f . $ x* , x . # x

Equilibrium Point $ x*e , x . e # Taylor series:

F

 f* f* f*$x*, x.# = f*$x*e, x.e# + $x* x*e# x*= x*e + $x. x.e# x*= x*e + H O.. T x* x.= x.e x. x.= x.e f. f. f.$x*, x.# = f.$x*e, x.e# + $x* x*e# x*= x*e + $x. x.e# x*= x*e + H O.. T x* x.= x.e x. x.= x.e
Expanding f $t n + h, x n + f $t n , x n ## in a Taylor series about $t n , x n # , we get f $t n + h, x n + f $t n , x n ## = f $t n , x n # + h f t $t n , x n # + f x $t n , x n # f $t n , x n # + o $ h . # By (a), obtain

x n +* = x n + h { a* f $t n , x n # + a . f $t n + h, x n + f $t n , x n #}
.

= h { a* f $t n , x n # + a . [ f $t n , x n # + hf t $t n , x n # + f x $t n , x n # f $t n , x n #]} = ( a* + a . ) hf $t n , x n # + a .h f t $t n , x n # + a . hf x $t n , x n # f $t n , x n #

(c)

Comparing (b) and (c), we have a* + a . = *, a . = * : ., a . = h : . That is

a* = * a .

= =

* .a . h .a .

If set a . = * : . , then a* = * : ., = *, = h .
x n +* = x n + h h f $t n , x n # + [ f $t n + h , x n + h f $t n , x n # ] . .

Therefore, we have the following formulas for second-order Runge-Kutta method:

x n +* = x n +

h ( k* + k . ) .

k* = f $t n , x n # k . = f $t n + h, x n + h k* #

Example 0 same as Example * 2atla! clear3 r 4 +*'3 delta 4 '.'.3 y$*# 4 .3 & 4 '3yy$*#4.3 for time 4 -delta delta '.0, & 4 & 5 *3 deltah4delta:.3 &*4+*'6y$&#3 &.4+*'6$y$&#5$deltah6&*##3 &;4+*'6$y$&#5$deltah6&.##3 &94+*'6$y$&#5$delta6&;##3 y $&5*#4 y$&#5$delta:>#6$&*5.6&.5.6&;5&9#3 end t 4 -' delta '.0,3 y7true 4 .6exp$+*'6t#3 plot$t,y,CoC,t,y7true#3

*'

2 1.8 1.6 1.4 1.2 y trajectory 1 0.8 0.6 0.4 0.2 0

0.05

0.1

0.15

0.2

0.25 0.3 Time(sec.)

0.35

0.4

0.45

0.5

Example > same as Example . 2atla! clear3 delta4.6pi:*;3 y$*#4'3 &4'3 for time4-delta delta 96pi, &4&5*3 deltah4delta:.3 time7t4time+delta3 &*4sin$time7t#3 &.4sin$time7t5deltah#3 &;4sin$time7t5deltah#3 &94sin$time7t5delta#3 y $&5*#4 y$&#5$delta:>#6$&*5.6&.5.6&;5&9#3 end t7true4-' delta:*' 96pi,3 y7true4*+cos$t7true#3 t4-' delta 96pi,3 plot$t,y,CoC,t7true,y7true#3
**

end Simulation results

2

1.5

1 y trajectory

0.5

-0.5

6 8 Time(sec.)

10

12

14

Hsing 2atla! AIE solver

The ode.; function uses a com!ination of second+ and third+order ?unge+@utta methods, %hereas ode90 uses a com!ination of fourth+ and fifth+order methods. The algorithms of ode.; and ode90 use a varia!le step size, %hich uses the large step size %hen the solution is changing more slo%ly. In general, ode.; is faster, !ut it uses large step sizes that can produce a solution plot that is not as smooth as the plot produced %ith ode90. "onsider the follo%ing differential equation
= f $t , y # y

Solver Syntax -t,y,4ode.;$Jydot8, tspan, y'# ydot name of the function file %hose input must !e t and y and %hose output must !e a column vector representing dy:dt3 that is, f$t,y#. tspan is -t',tf,, %here t' and tf are the desired starting and ending values of the independent parameter t. y' is the initial value of y$t#. Example E same as Example *.
*.

2atla! file example*.m function ydot4example*$t,y# ydot4+*'y3 2atla! file test.m clear3 -t,y,4ode90$Jexample*8,-','.9,,.#3 y7true4.6exp$+*'6t#3 plot$t,y,8o8,t,y7true# 1 note that %e need not generate the array t to evaluate y7true, !ecause t is generated 1!y the ode90 function.
2 1.8 1.6 1.4 1.2 y trajectory 1 0.8 0.6 0.4 0.2 0

0.05

0.1

0.15

0.2 0.25 Time(sec.)

0.3

0.35

0.4

Hsing ode90 solver

*;

2 1.8 1.6 1.4 1.2 y trajectory 1 0.8 0.6 0.4 0.2 0

0.05

0.1

0.15

0.2 0.25 Time(sec.)

0.3

0.35

0.4

Hsing ode.; solver Example F "onsider the follo%ing second+order equation

+ Ey + 9 y = f $t # 0 y

Solve it for the highest derivative

= y * 9 E f $t # y y 0 0 0

. Iefine t%o ne% varia!les x* and x . to !e y and its derivative y

x* = y x. = y

These definitions imply that

* = x. x . = x * 9 E f $t # x* x . 0 0 0

* x ' * ' x 9 E * + * f $t # = ! + bf = = ! x. . x 0 0 0
Suppose that f $t # = sin t . 2atla! exampleF.m function xdot4exampleF$t,x#
*9

xdot$*#4x$.#3 xdot$.#4$*:0#6$sin$t#+96x$*#+E6x$.##3 xdot4-xdot$*#3xdot$.#,3 or function xdot4exampleF$t,x# xdot4-x$.#3 $*:0#6$sin$t#+96x$*#+E6x$.##,3 or function xdot4exampleF$t,x# (4-' *3+$9:0# K$E:0#,3 !4-'3$*:0#,3 xdot4(6x5!6sin$t#3 2atla! test.m -t,x,4ode90$JexampleF8, -',>,,-;,G,#3 plot$t,x$ ,*#3
7 6 5 4 y trajectory 3 2 1 0 -1

3 Time(sec)

Example G given the plant model Transfer function G $ s # =

Y $ s# * = . U $ s# s + ;s + .

Iifferential equation y + ; y + . y = u Iynamical equation

*0

* ' * x* ' x = x + u . . ; x . * x* y = [* '] x . = ! + bu ! y = c!

fuzzy controller "ontrol rules

If e = P L ce = P, Thenc u = P If e = P L ce = N , Thenc u = Z If e = N L ce = P, Thenc u = Z If e = N L ce = N , Thenc u = N

2em!ership functions M B

+9

+. M +. N '

. B .

e,ce

cu

A!/ective chec& its step response $*# using ode90, $.# using ?unge+@utta, $;# use Euler method

$*# Hsing ode90 solver

2atla! exampleG.m function xdot4exampleG$t,x# glo!al r u (4-' *3+. +;,3 !4-'3*,3 e4r+x$*#3 ce4'+x$.#3 u4u5'.'.96fuzz$e,ce#3 xdot4(6x5!6u3 2atla! main.m clear3
*>

glo!al r u u4'3 r4*3 -t,x,4ode90$CexampleFC, -',F,,-',',#3 plot$t,x$ ,*##3 2atla! fuzz.m function y4fuzz$e,ce# 1 if e4B Lce4B Then cu4B r*4B$e#6B$ce#3 1 if e4B Lce4M Then cu4N r.4B$e#6M$ce#3 1 if e4M Lce4B Then cu4N r;4M$e#6B$ce#3 1 if e4M Lce4M Then cu4M r94M$e#6M$ce#3 y4$r*6.5r.6'5r;6'5r96$+.##:$r*5r.5r;5r9#3 2atla! M.m function y4M$x# if xO4+., y4*3 end3 if xP+. L xO4., y4$+*:9#6$x+.#3end3 if xP., y4'3 end3 2atla! B.m function y4B$x# if xO4+., y4'3 end3 if xP+. L xO4., y4$*:9#6$x5.#3end3 if xP., y4*3 end3

*E

1.4

1.2

y trajectory

0.8

0.6

0.4

0.2

4 Time(sec)

$.# Hsing ?unge+@utta method

2atla! clear3 mm4F'''3 h4'.''*3 x*4'3x.4'3u4'3 r4*3e4r+x*3ce4'+x.3 for i4* mm u4u5'.'*6fuzz$e,ce#3 144444444444444444444444444444444 &*4x.3 &.4x.5$$h:.#6&*#3 &;4x.5$$h:.#6&.#3 &94x.5$h6&;#3 x**4x*3 x*4x*5$$h:>#6$&*5.6&.5.6&;5&9##3 144444444444444444444444444444444 /*4$+.6x**+;6x.5u#3 /.4$+.6x**+;6x.5u#5$h:.#6/*3
*F

/;4$+.6x**+;6x.5u#5$h:.#6/.3 /94$+.6x**+;6x.5u#5h6/;3 x.4x.5$$h:>#6$/*5.6/.5.6/;5/9##3 144444444444444444444444444444444 e4r+x*3 ce4'+x.3 xx*$i#4x*3 end tt4-* mm,6h3 plot$tt,xx*#3
1 0.9 0.8 0.7 0.6 y trajectory 0.5 0.4 0.3 0.2 0.1 0

4 Time(sec)

$;# Hsing Euler method

2atla! clear3 mm4F'''3 dt4'.''*3 x*4'3x.4'3u4'3 r4*3e4r+x*3ce4'+x.3 for i4* mm u4u5'.'*6fuzz$e,ce#3 x*4x*5x.6dt3 x.4x.5$+.6x*+;6x.5u#6dt3 e4r+x*3
*G

ce4'+x.3 xx*$i#4x*3 end tt4-* mm,6dt3 plot$tt,xx*#3

1 0.9 0.8 0.7 0.6 y trajectory 0.5 0.4 0.3 0.2 0.1 0

4 Time(sec)

They have the same results in this example %ith the Euler method and ?unge+@utta method. Ho%ever, there is slightly different in using ode90 solver. Example *' adaptive fuzzy controller $direct adaptive# Fuzzy controller
u = # T "$!#

Suppose that the parameters in THEM+part of fuzzy controller can !e tuned !y

= # eT Pb"$ !#

Fuzzy rules and the plant model are given the same as in Example G.
*0 P = 0 0 0

)e simulate it using $*# Euler method, $.# ?unge+@utta method, $;# ode solver.

$*# Euler method

2atla! function -y,eta,4fuzz$e,ce,pp# 1 if e4B Lce4B Then cu4B
.'

r*4Ba$e#6Ba$ce#3 1 if e4B Lce4M Then cu4N r.4Ba$e#6Ma$ce#3 1 if e4M Lce4B Then cu4N r;4Ma$e#6Ba$ce#3 1 if e4M Lce4M Then cu4M r94Ma$e#6Ma$ce#3 dd4r*5r.5r;5r93 eta4-r*:dd r.:dd r;:dd r9:dd,3 144444444 1 pp$96*# y4eta6pp3 2atla! example *' clear3 mm4*.'''3 dt4'.''*3 x*4'3x.4'3u4'3 r4*3e4r+x*3ce4'+x.3 B4-*0 030 0,3 pp4-' ' ' ',3 !4-' *,C3 eta4-' ' ' ',3 gamad4*3 for i4* mm -u,eta,4fuzz$x*,x.,ppC#3 x*4x*5x.6dt3 x.4x.5$+.6x*+;6x.5u#6dt3 e4r+x*3 ce4'+x.3 ee4-e ce,3 1ee$*6.#3B$.6.#3!$.6*#3eta$*69# pp4pp5$gamad6ee6B6!6eta#6dt3 xx*$i#4x*3 end tt4-* mm,6dt3 plot$tt,xx*#3 2atla! function y4Ma$x#
.*

if xO4+., y4*3 end3 if xP+. L xO4., y4$+*:9#6$x+.#3end3 if xP., y4'3 end3 2atla! function y4Ba$x# if xO4+., y4'3 end3 if xP+. L xO4., y4$*:9#6$x5.#3end3 if xP., y4*3 end3

0.8 y trajectory

0.6

0.4

0.2

6 Time(sec)

10

12

$.# ?unge+@utta method

2atla! clear3 mm4*.'''3 h4'.''*3 x*4'3x.4'3u4'3 r4*3e4r+x*3ce4'+x.3 B4-*0 030 0,3 pp4-' ' ' ',3 !4-' *,C3 eta4-' ' ' ',3 gamad4*3 for i4* mm
..

-u,eta,4fuzz$x*,x.,ppC#3 &*4x.3 &.4x.5$$h:.#6&*#3 &;4x.5$$h:.#6&.#3 &94x.5$h6&;#3 x**4x*3 x*4x*5$$h:>#6$&*5.6&.5.6&;5&9##3 144444444444444444444444444444444 /*4$+.6x**+;6x.5u#3 /.4$+.6x**+;6x.5u#5$h:.#6/*3 /;4$+.6x**+;6x.5u#5$h:.#6/.3 /94$+.6x**+;6x.5u#5h6/;3 x.4x.5$$h:>#6$/*5.6/.5.6/;5/9##3 144444444444444444444444444444444444 e4r+x*3 ce4'+x.3 ee4-e ce,3 1ee$*6.#3B$.6.#3!$.6*#3eta$*69# n*4$gamad6ee6B6!6eta#3 n.4$gamad6ee6B6!6eta#5$h:.#6n*3 n;4$gamad6ee6B6!6eta#5$h:.#6n.3 n94$gamad6ee6B6!6eta#5h6n;3 pp4pp5$$h:>#6$n*5.6n.5.6n;5n9##3 xxx*$i#4x*3 end tt4-* mm,6h3 plot$tt,xxx*#3

.;

0.8 y trajectory

0.6

0.4

0.2

6 Time(sec)

10

12

$;# 2atla! ode.; solver

2atla! function xdot4example*'$t,x# glo!al r u B eta gamad %dot to (4-' *3+. +;,3 !4-'3*,3 e4r+x$*#3 ce4'+x$.#3 ee4-e ce,3 -u,eta,4fuzz$e,ce,%dotC#3 h4t+to3 &*4$gamad6ee6B6!6eta#3 &.4$gamad6ee6B6!6eta#5$h:.#6&*3 &;4$gamad6ee6B6!6eta#5$h:.#6&.3 &94$gamad6ee6B6!6eta#5h6&;3 %dot4%dot5$h:>#6$&*5.6&.5.6&;5&9#3 xdot4(6x5!6u3 to4t3 2atla! clear3 glo!al r u eta B gamad %dot to
.9

B4-*0 030 0,3 eta4-' ' ' ',3 %dot4-' ' ' ',3 gamad4*3 u4*3 to4'3 r4*3 -t,x,4ode.;$CexampleFC, -',*.,,-',',#3 plot$t,x$ ,*##3 2atla! function -y,eta,4fuzz$e,ce,pp# glo!al eta 1 if e4B Lce4B Then cu4B r*4Ba$e#6Ba$ce#3 1 if e4B Lce4M Then cu4N r.4Ba$e#6Ma$ce#3 1 if e4M Lce4B Then cu4N r;4Ma$e#6Ba$ce#3 1 if e4M Lce4M Then cu4M r94Ma$e#6Ma$ce#3 dd4r*5r.5r;5r93 eta4-r*:dd r.:dd r;:dd r9:dd,3 144444444 1 pp$96*# y4eta6pp3

.0

0.8 y trajectory

0.6

0.4

0.2

6 Time(sec)

10

12

Mote using ode90

1.2 1 0.8 0.6 y trajectory 0.4 0.2 0 -0.2 -0.4 -0.6

6 Time(sec)

10

12

Finally, %e use the simulin& to simulate the adaptive fuzzy controller.

.>

2atla! function y4fuzzs$e# 1 if e4B Lce4B Then cu4B r*4Ba$e$*##6Ba$e$.##3 1 if e4B Lce4M Then cu4N r.4Ba$e$*##6Ma$e$.##3 1 if e4M Lce4B Then cu4N r;4Ma$e$*##6Ba$e$.##3
.E

1 if e4M Lce4M Then cu4M r94Ma$e$*##6Ma$e$.##3 1pp4- .* .* .* .*,C3 dd4r*5r.5r;5r93 eta4-r*:dd r.:dd r;:dd r9:dd,3 144444444 y4eta3

.F