System Identification and

Simulation
Prof.Ing.Petr Noskievi, CSc.

Modeling of the functions relationship

between the variables
Static characteristics of the systems,
Input signals (desired value) ,
Initial funtions for the delayed variables.
Set of points [xi, f(xi)], i = 0, 1, 2, ..., n
y = f(x)
(x)

[xn, fn]
x
[xi, fi]
x

(x)

[x1, f1]
[x0, f0]
x

x
[xi+1, fi+1]

y = f(x) (x) = ?

Approximation
Aproximac funkce f(x)
Error of the approximation

E( x) = f ( x) ( x)
n

1
2

2n ( f , ) = f (x i ) ( x i ) i
i =1

Weighted criterion
Linear regression

pro x a, b

( x) = a + b x

1
2 2

2n ( f , ) = f ( xi ) (a + b xi )
i =1

]2

S = f (xi ) (a + b xi )
i =1

Conditions for the minimum: S = 0,

a

S
= 0.
b

Linear regression
n
n
n

S
= 2 yi (a + bxi ) ( 1) = 2 y i n a + b xi
a
i =1
i =1
i =1

n
n
n
n

S
= 2 yi (a + bxi ) ( xi ) = 2 x i y i a xi b x2i
b
i =1
i =1
i =1
i =1

System of
linear
equations

i =1

i =1

n a + b x i = f ( x i )
n

a x i + b
i =1

xi f ( xi )
i =1

b=

1 n
xi f ( xi ) n xi f ( xi )
i =1 i =1

i =1

i =1

x2i =

n
2 1
xi n xi
i =1
i =1

a = y b x

1 n
x = xi ,
n i =1

1 n
y = f ( xi )
n i =1

Linear regression
n

Correlation coefficient:
A measure for the
quality of the
approximation

i =1

i =1

n x i f ( x i ) x i f ( x i )
rxy =

i =1

n
n
n
n
2
n x2i xi n f 2 ( xi ) f ( xi )
i =1
i =1
i =1
i =1

Linear Regression
Multiple regression:
Polynomial regression:

( x) = a0 + a1x1 + a2x2 + ... + anxn

(x) = a 0 + a1x + a 2 x2 + ... + a n xn
n

[ (

S(a 0 , a1, ..., a n ) = yi a 0 + a1x1 + ... + a n x nn

Criterion:

i =1

)]

Conditions for the minimum of the criterion S are expressed as follows:

n
S
= 2 yi a 0 + a1x1 + ... + a n xin xik = 0
a k
i=0

[ (

)]

System of n+1 equations for the coeficients a0, ..., an

pro k = 0, 1, ..., n:

a0

i=0

xik + a1

i=0

xik +1 + ... + a n

i=0

xik + n =

yi xik

i=0

Interpolation of the funtions

The interpolationg curves are designed to run through all given points.
Interpolation condition:

(xi ) = f (xi ),

Interpolationg Lagrange Curve

for i = 0, 1, 2, ..., n
n

Ln (x ) = f (xi ) li (x )
i =0

where li(x) are the polynomials degree n , for which is valid:

li (x k ) = 0

pro k i,

li (x k ) = 1

pro k = i.

li ( x) =

(x x0 )(x x1) ... (x - xi-1)(x - xi+1) ... (x - xn )

(xi x0 )(xi x1) ... ( xi - xi-1)(xi - xi+1) ... (xi - xn )

Interpolation of the funtions

Newton Interpolating Polynomial
N( x) = a 0 + a1(x x0 ) + a 2 (x x0 )(x x1) + ... + a n (x x0 )(x x1) ... (x - x n-1)
N( xi ) = f ( xi ),

i = 0, 1, 2, ..., n

a0

= f ( x0 )

a 0 + a1( x1 x0 )

= f ( x1)

a 0 + a1( x2 x0 ) + a 2 ( x2 x0 )( x2 x1)

= f ( x2 )
.
.
.

a 0 + a1( x n x0 ) + ... + a n ( x n x0 )( x n x1) ... ( x n x n 1) = f ( x n )

a1 =

f (x1) a 0
x1 x0

Spline function
The given set of points is
interpolated using the
series of the of the unique
cubic polynomials.
Between each of two given
points the cubic polynomial
has another set of
coefficients. The whole
interpolation is done using
the set of the interpolating
functions called spline
function.

y = f(x)
(x)

y = f(x) {(x)}
i = 0, 1, ..., n-1
fn-1 fn

i(x) = ai+bi(x-xi)+ci(x-xi)2+di(x-xi)3
fn-2
fi+1
f1

f2

fi

x0 x1

x2

xi xi+1

f0

xn-2

xn-1 xn

i ( x) = a i + bi (x xi ) + ci (x xi ) + d i (x xi )

Cubis spline-function
Each cubic polynomial has degree 3 and is described by the set of four
coefficients: ai, bi, ci, di. For n intervals it is to design 4n coefficients.

Spline function
The set of n+1 inperpolation conditions will be used for the derivation of the coefficients.
( xi ) = f ( xi ),

i = 0, 1, ..., n

The spline function fullfils 3(n-1) conditions of the continuity of the cubic
polynomials in form:
i 1(xi ) = i (xi ),

i = 1, 2, ...n -1

i1 (xi ) = i(xi ),
i1 (xi ) = i(xi ),

i = 1, 2, ...n -1
i = 1, 2, ...n -1

Now we have 4n-2 conditions for 4n coefficients, which values should be derived.
The last two conditions can be set as following condition in the end points :
First derivative:

a)

(x0 ) = f0

(x n ) = fn

Second derivative:

b)

(x0 ) = f 0

(xn ) = f n

c)

(x0 ) = 0

(xn ) = 0

... So called natural spline

Spline function
The derivatives of the polynomials i(x) are continuos functions.
i ( x) = bi + 2ci (x xi ) + 3d i (x xi )

i( x) = 2ci + 6d i (x xi )
(xi ) = M i ,

i = 0, 1, ..., n

hi = xi +1 xi

i = 0, 1, ..., n

i(xi ) = 2ci ,

resp. 2ci = Mi

i(xi +1) = 2ci + 6d i hi ,

resp. 2ci + 6d i hi = Mi +1

The coeficients ci, di can be derived from these equations:

ci =

Mi
2

di =

M i +1 M i
6h i

ai = fi

Spline function
Next for x=xi+1: i (xi +1) = a i + bi hi + ci h2i + d i h3i
fi +1 = a i + bi hi + ci h2i + d i h3i

After substitution for the coefficients ai, ci, di we obtain the expression for the
coefficient bi:
f f 2M + M
bi =

i +1

hi

i +1

hi

The equations for the coefficient of the polynomials expressed using Mi for the
second derivative: a = f (x )
i

bi =

f ( x i +1 ) f ( x i )
hi

Mi
2
M i+1 M i
di =
6h i

ci =

2 M i + M i +1
hi
6

pro i = 0, 1, ..., n-1.

Spline function
The calculation of the second derivatives Mi is based on the use of the
condition for the continuity of the first derivative :
i1 (x i ) =

f i f i1 1
1
+ M i1h i1 + M i h i1
h i1
6
3

1 (x i ) =

f i +1 f i 2M i + M i +1
+
hi
hi
6

The equation must be fullfiled in the inner given points: for i = 1, 2, ..., n-1 :
f i+1 f i f i f i1
1
1
1
M i1h i1 + (h i1 + h i )M i + M i+1h i =

6
3
6
hi
h i1

2( h 0 + h 1 )
h1
.

h1
2( h 1 + h 2 ) h 2

.
.
.

.
.

0
.
.

.
.

h i 1 2( h i 1 + h i ) h i
.
.
.

.
.

.
.

.
h n2

f 2 f1 f1 f 0

6
M 0 h 0
h
h 0

M 1 1
0

f
f

3
2
2 f1

M
.

h 1
h 2

.
.
.

.
.
.

.
M i = f i +1 f i f i f i 1

h i 1
.
. h i

.
.
.

.
.
.

.
2( h n 2 + h n 1 ) M n 1

6 f n f n 1 f n 1 f n 2

h n 1
h n 2

Spline function
The calculation of the spline function can be described in the next steps:
1. Input data: n, x0, x1, ..., xn, f(x0), f(x1), ..., f(xn).
2. calculation hi = xi+1 - xi, pro i = 0, 1, ..., n-1.
3. Creating of the systm of the linear algebraic equations, settinf of M0, Mn.
4. Calculation Mi, i = 1, 2, ..., n-1 solution of the equations.
5. Calculation of the coefficients in the for each interval.
The calculation of the value of the spline function in the given point is realized in
two steps:
1. The interval (index i) in which the value x is located.
2. Calculation of the of the cubic function i (x) for given value x.
If also the values of the first derivative fi(xi) are given, the spline function
is called Hermits spline function and in the given points should be also
fulfilled the condition:
f i = i

Next applications
y = f(x)

y = f(x)

y = f(x)

x
periodick kivka

Periodic function

x
smyka

General curve in 2D

x
uzaven kivka

Closed figure

Periodic spline function

y

yn = y0
y0

x0

0 (x0 ) = n 1(xn )
0 (x0 ) = n 1(x n )
0(x0 ) = n 1(x n )

x1

yn

x2

Periodic spline function

We have now a system of n-linear algebraic equations for n second
derivatives variables M0, M1, ..., Mn-1 (periodic spline-function).
2( h 0 + h n 1) h 0

.
.

h0

.
0

h n 1
.

.
.
2( h 0 + h1) h1
.
.

.
.

hn2

M 0 6 y1 y 0 y n y n 1

h

h n 1

0

.
.

.
.
.

.
.

y
y

y
1
M =
.
6 2
1 0

1

h1
h 0

.
.

.
.

.
. y y

y
n
n 1

n 1 n 2
2( h n 2 + h n 1) M n 1 6 h
hn2
n 1


h n 1

Curve parameterization
Let we have a set of given points [xi, yi] and the variable x is not
monotonously growing up for growing index i, that means, it is not valid:
x n > x n 1 > x n 2 > ... > x2 > x1 > x0

For the derivation of the spline we divide th set of points into the two sets:

[t 0 , x0 ], [t1, x1], [t2 , x2 ], ..., [t n , xn ]

[t 0 , y0 ], [t1, y1], [t 2 , y2 ], ..., [t n , yn ]
t 0 < t1 < t 2 < ... < t n
The curve is described in 2-dimensional space.
The new two sets of points [ti, xi] and [ti, yi] we can now interpolate by two
spline functions.

Curve parameterization
t0 = 0

t i +1 = t i + d i ,

i = 0, 1, 2, ..., n -1

The parameterization can be done using one of the following formulas for the
difference (distance) between two points di:

di =

(xi+1 xi )2 + (yi+1 yi )2
2

d i = ( xi +1 xi ) + ( yi +1 yi )

di = xi +1 xi + yi +1 yi

d i = max xi +1 xi , yi +1 yi , i = 0, 1, 2, ..., n -1

Interpolation using the parametric spline

The use of the parametric spline function can be summarized in the following
steps:
1. Curve parameterization seting t0, t1, ..., ti, ..., tn for each point .
2. Calculation of Mxi, Myi - using two created systems of linear equations.
3. Calculation of the coefficients of the spline function of x and spline
function of y.
4. Calculation of x and y for growing t.

Interpolation using the parametric spline

y

6
x

[x1, y1]
x

[x0, y0]
x

x
x

x
x

3=8
[x3, y3] = [x8, y8]

10

Interpolation of the closed figures (cyclic

splines)
7

8x

x 6

10 x
x

0 = 11

x 4

1
x 3
x

[t 0 , x0 ], [t1, x1], [t2 , x2 ], ..., [t n , xn ]

[t 0 , y0 ], [t1, y1], [t 2 , y2 ], ..., [t n , yn ]

It is valid :

x 0 = x n , y0 = y n

Numerical Integration
Methods
The numerical methods of integration are based on the use of approximation
methods. The given function which should be integrated f(x) is substituted by
their approximation (x). The integration is calculated for this function:
b

f ( x)dx

( x)dx + R

The numerical methods of integration are used if:

it is not possible to calculate the integral analytically ,
analytical approach is to complicated,
the function is given using the table or graph.
If the function (x) is a good approximation of the function f(x) , then
the integral calculated from the function (x) is a good approximation
of the integral of f(x).

Numerical integration methods

Quadrature rule
We suppose that the function f(x) is
given in n+1points with constant step h. In
each interval we replace the function with
their interpolationg function (x) = f(xi):
b

y = f(x)
(x) = f(xi)

n 1

f ( x)dx h f (xi ) + R
i= 0

R=

ba
f ( x n ) f ( x0 )
n

If the function is given analytically,

the more precise formula can be
used:
b

n 1

f ( x)dx
a

h f xi + + R

2
i=0

ba 2
h max f ( x)
24
a, b

y0

y1 yi-1

x0=a x1

xi-1

yi yi+1

xi

xi+1

yn-2 yn-1

xn-2

yn

xn-1 xn=b x

Numerical integration methods

Trapezoidal method
The given function is replaced in the
partial intervals by the polynomial
function first order.

y = f(x)
i(x)

For the Newton's polynomial we

obtain:
f (xi +1) f (xi )
i ( x) = f (xi ) +
( x xi )
xi +1 xi

y0

yi yi-1

x0=a xi

xi+1

After integration:
x i+1

f (x)dx

xi

h
f ( x i ) + f ( x i +1 )
2

For the whole interval a, b, respectively :

b

n 1
1
1

f (x )dx h f (x0 ) + f (xi ) + f (xn ) + R

2
i =1
2

ba 2
h max f ( x)
12
a, b

yn-2 yn-1

xn-2

yn

xn-1 xn=b x

Numerical integration methods

Quadratic function
The Simpson's method is base on the use of quadratic function for the
interpolation of the given function in the partial intervals
xi, xi+2.

Starting from the Newtons

polynomial we obtain:
x i+ 2

xi

x i+1

xi

f ( x)dx
a

y = f(x)

x i+2

f (x)dx

f (x)dx

i(x)

x xi 2 fi ( x xi )( x xi +1)
dx
fi + fi h + 2
2
h

xi

yi

h
[ f + 4 fi + 1 + fi + 2 ]
3 i

xi-1

h
f + 4 f1 + 2 f2 + 4 f3 + ... + 2fn-2 + 4 fn 1 + fn + R
3 0

xi

yi+1 yi+2

xi+1 xi+2

xn

ba 4
( )
h max f 4 ( x)
180
a, b

By this method the number n must be even number, that means, that the
function f(x) must be given in odd number of points, in n+1points .

Numerical methods for

computing of the derivation
The calculated numerical derivation (x) of the good approximation function
(x) of the given function f(x) can be good approximation of the derivation f(x),
but this is not guaranteed in general. The influence of the noise must be taken
into the account. .

Numerical methods for

computing of the derivation
Computation of the derivation using the polynomial functions
The given function xi , f(xi) is replaced by the interpolation polynomial, for ex. By
the Newton's polynomial, by the numerical calculation of the derivation.
The approximately values of the derivation we obtain from the derivation of the
interpolation polynomial. The degree of the interpolation polynomial must be
higher than the order of the calculated derivation.
For the derivation order k of the Newton's polynomial expressed using the
differences we obtain:

[
]
+ f [ x0 , x1, ..., x n ](x x0 )(x x1) ... (x x n-1).

N( x) = f0 + f x0 , x1 (x x0 ) + f x0 , x1, x2 (x x0 )(x x1) + ... +

The difference order k is

expressed by:

N ( k ) ( x) = f [x0 , x1, ..., x k ]

dk
dx k

+ f [x0 , x1, ..., x k ]

dk
dx k

{(x x0 )(x x1) ... (x xk )} + ...

{(x x0 )(x x1) ... (x xn-1)}.

Formulas for numerical

derivation
The next formulas for computing of the numerical derivation from two or three
points of the given function were derived in the described way:
f ( x)

f ( x + h) f ( x) 1
hM 2
h
2

f ( x)

f ( x) f ( x h) 1
+ hM 2
h
2

f ( x)

f ( x + h) f ( x h) 1 2
+ h M3
2h
6

f ( x)

- 3f ( x) + 4 f ( x + h) f ( x + 2 h) 1 2
+ h M3
2h
3

f ( x)

3f ( x) 4 f ( x h) + f ( x 2 h) 1 2
+ h M3
2h
3

Formulas for numerical

derivation
The computation of the derivation second order can be calculated
using the formulas:
f ( x)

f ( x)

f ( x)

f ( x + 2h) 2 f ( x + h) + f ( x)
h2
f ( x + h) 2 f ( x) + f ( x h)

hM3

h2

f ( x) 2 f ( x h) + f ( x 2 h)
h2

h2 M 4
12

+ hM 3

Mi = max f ( i) ( x)
a, b

f ( x)

f ( x)

) (

12f ( x) 30f x + h + 24f x + 2 h 6f x + 3h

6h2

) + 0(h2 )

- f ( x - 2h) + 16f ( x h) 30f ( x) + 16f ( x + h) f ( x + 2 h)

12 h

( )

+ 0 h4

Formulas for numerical

derivation
Numerical calculation of the derivation using the spline-function
The spline function can be used for the computing of the derivation and also
derivation higher order.
The first and second derivative of the cubic spline function cubic polynomial
are the continuous functions, which are also the approximation polynomials of
the derivations.
2

F: i ( x) = a i + bi (x xi ) + ci (x xi ) + d i (x xi )
i = 0, 1, 2, ..., n -1

Derivation:
2

F: i ( x) = bi + 2ci (x xi ) + 3d i (x xi )
i = 0, 1, 2, ..., n -1

Numerical Methods for Solving

Differential Equations
The numerical solution of the differential equations with the initial conditions
(so called Cauchy task) is one of the very important tasks for the realization
of the mathematical model using the digital computer and simulation
programme.
Let we have to solve a differential equation
y! = f (t , y)

with initial condition y0 = y(t) for time t from the interval <t0, tk>.

The numerical solution is a sequence of the values

{y(ti )} = y(t0), y(t1), ...

Where ti is from the mentioned interval. The value hi is a step of the numerical
solution:
hi = t i +1 t i ,

i = 0, 1, 2, ...

The values of the independent variable ti, at which the numerical solution of
the differential equation is calculated, define so called node points of the
solution.

Numerical Solution of the

Differential Equation
y

yi

yi+1

t
hi=ti+1-ti

Errors of the numerical

solution
Local error EL(ti) has two parts:

E L ( t i ) = ET ( t i ) + E R ( t i )

ET(ti) - error of the method - trunsaction error, ER(ti) - rounding error - this
error is caused by the limited length of the word in the computer (given by the
number of the decimals used by the numerical computation).
The value of the transaction error is proportional to the power of the step h, for
example h2 , like 0(h2). That means, that the error ET is order hi. The error in one
step of the solution influences the solution in the next steps. The inaccuracy of
result in the next i steps is characterized by the accumulative error global
error.
i = Y(t i ) y(t i ) ,
where Y(ti) is the exact accurate solution at the point ti.

Classification of the Numerical Methods for

Solution of the Differential Equations
Numerical methods for the solution
of the differential equations

One-step Methods

multi steps methods

(k-steps methods )

Corrector methods
(implicit methods)

Predictor methods
(explicit methods)

Combined methods

One-Step Methods
One step method can be expressed in the form:
y i +1 = y i + h ( t i , y i , h )

The expression h(ti, yi, h) defines the increment of the solution between the
nodes i a i+1
We express the increment of the solution using the Taylor series in
dependence on the step h and its power h = ti+1 - ti (fixed step h) as follows:
h2
h n ( n1)
yi +1 yi = h f ( t i , yi ) + f ( t i , yi ) + ... + f
t i , yi ) + 0 h n +1
(
2!
n!

( )

The computation of the increment using the Taylor series needs the calculation of the derivatives
higher order of the function f(t, y) what is difficult to express or calculate. It is also assumed that the
function f(t, y) is differentiable.
The method which calculated the increment of the solution from the Taylor series is called Taylor series
method. Because of the need of the calculation of the derivatives higher order this approach is
practically not used.
This approach is the basis for the derivation of the One Step Methods like Eulers method or Runge Kutta method.

Eulers method
The Eulers method uses only the fist term of the Taylor series:
y i +1 = y i + h f ( t i , y i )

The numerical solution using the

Eulers method represents the
polynomial approximation first order.
The transaction error of the Eulers
method is given by the neglected
terms of the series.

yi+1
yi

yi

h2
h3
ET =
f (ti , yi ) +
f (ti , yi ) + ...
2!
3!

ti

hi

ti+1

Runge Kutta Methods

The basic idea of the Runge - Kutta methods consists in finding an
expression for the increment of the solution, which doesnt need the
computation of the derivatives of the function f(ti, yi) and which is with the
computational accuracy equal to the value given by the r terms of the Taylor
series. This method is called Runge - Kutta methods order r.
The Runge - Kutta methods can be expressed in the form:

y i +1 = y i + h w jk j
j=1

The increments kj are calculated between the nodes ti a ti+1 using the
expressions:
k1 = f (t i , yi )

k j = f t i + jh, yi + jhk j1

j = 2, 3, ..., r.

The variables wj, j, j are the constant parameters. Their values assure
that the numerical solution of the differential equation in the node ti+1 is
equal to the value which can be calculated from the r-terms of the Taylor
series in the node ti .

Runge Kutta Methods

The transaction error of the Runge Kutta methods can be expressed using
the neglected terms of the Taylor series in the following form:
ET =

h r +1 (r )
f ( , y ),
(r + 1)!

ti , t i + h

This expression is not possible to use for the practical calculation of the
estimated value of the transaction error. For that reason the following approach
based on the half step is very frequently used.
Half-step method for the estimation of the occurring error:
1. The solution y(ti+h) in the next node ti+1 is computed using the chosen
one-step method with the step length h.
2. We repeat the calculation with the half step h = h/2 and calculate the
solutions y(ti+h) and y(ti+2h), (two times use of the method).
3. If the difference between the values of the solutions y(ti+h) and y(ti
+2h) is less than the accepted maximal error we continue in the
solution with the step h. Otherwise with the half step.

Half-step method for the estimation

of the occurring error
y(ti+h)

y
y(ti+h/2)
y(ti)

ti

y(ti+2*h/2)

ti+2*h/2
ti+h/2

Runge-Kutta methods 2nd order

Modified Eulers method

y i +1 = y i + h i k 2

k1 = f ( t i , y i )
h
h

k 2 = f t i + i , yi + i k1

2
2

Heuns method

k +k
yi +1 = yi + hi 1 2
2

k1 = f ( t i , y i )
k 2 = f (t i +1, yi + hi k1)

Runge - Kutta Method 4th order

Runge - Kutta Method 4th order
(clasical method, very frequently
used method):

k + 2k 2 + 2k 3 + k 4
yi +1 = yi + hi 1
6
k1 = f ( t i , y i )
h
h

k 2 = f t i + i , yi + i k1

2
2
h
h

k 3 = f t i + i , yi + i k 2

2
2

k4 = f (ti +1 , yi + hi k3 )

The simple change of the step of the solution is the advantage of the Runge-Kutta
Methods. It is possible to change the step in dependence on the estimated value of
the error of the solution. The disadvantage is the multiple calculation of the function
f(t, y) in dependence on the order of the method. If this calculation is enough fast, it
is suitable to do the whole solution using the Runge-Kutta method. If the function f(t,
y) is more difficult and the computation of their value takes more time, it is
recommendable to start the solution with the Runge-Kutta method and continue
with the multi-step method.

Multi-step Methods
Let we assume that we know the solution of the differential equation
y = f ( t , y),

y0 = y(t 0 )

in k+1 nodes
yi = y( t i ), yi 1 = y( t i 1), ..., yi k = y( t i k )

After integration of the equation at the interval ti, ti+1 we obtain:

y( t i +1) = y( t i ) +

t i +1

f (t , y)dt

ti

It is impossible to calculate the integral at the interval ti, ti+1 , because we dont know the the solution y.
but we know the solution in the previous nodes. The multistep methods are based on the approximation
of the function f(t, y) using the interpolation polynomial p(t), which is defined by the values f(ti, yi), f(ti-1,
yi-1), ..., f(ti-k, yi-k), and on the approximation of the integral from f(t,y) by the integral from the p(t,y). Then
we can write:
y( t i +1) = y( t i ) +

t i +1

p(t, y)dt + ET

ti

Multi-step methods
Because of the use the polynomial p(t, y) for the computation of the integral
at the interval ti, ti+1 outside the given values, we will use the polynomial for
the extrapolation of the function f(t, y). The formulas of the methods for the
numerical solution of the differential equations derived in this way are called
explicit methods and are expressed in explicit forms.
If we use by the derivation of the interpolation polynomial p(t, y) the values
f(ti+1, yi+1), f(ti, yi), f(ti-1, yi-1), f(tik, yi-k) we obtain for the computation of the
solution y the interpolation formula and so called implicit formula. Because
of the computation of the value of the solution yi+1 from the value f(ti+1, yi+1)
the derived formula is implicit.
The order of the multistep methods is given by the order of the polynomial
p(t).

Multi-step methods
The explicit methods Adams-Bashforth methods (A-B), predictor formulas
Derivation of the two steps method A-B
Let we replace the function f(t, y) in the equation for the integral
increment of the solution by the Newton interpolation:
f ( t , y) p(t i , yi ) = f (t i , yi ) +

f (t i-1, yi-1)
(t t i )
h

After substitution into the formula for the numerical solution of the differential
equation we obtain:
t i +1
f

yi +1 = yi + fi + i 1 ( t t i )dt
h

ti

y i +1 = y i + f i ( t i + 1 t i ) +

fi 1
2
t i +1 t i )
(
2h

The final expression of the explicit method 2nd order we obtain after the
substitution h = ti+1 - ti and fi-1 = fi - fi-1 and transformation:
y i +1 = y i +

h
(3f f )
2 i i 1

Multi-step methods
The explicit multistep method A-B order k+1 can be expressed by the formula:
k

y i +1 = y i + h b kj fi j
j= 0

The values bkj for k = 0, 1, 2, 3 are summarized in the table:

j

b0j

b1j
b2j
b3j

3
2

1
2

16
12

5
12

59
24

37
24

23
12

55
24

9
24

Multi-step methods
Implicit methods (corrector methods)
The implicit methods are described using the implicit formulas and are also
called Adams - Moulton methods or A-M methods.
The implicit A-M method order k can be expressed using the formula:
k

yi +1 = y i + h b kj fi +1 j
j= 0

The values of the coefficients bkj of the implicit methods A-M for k = 0, 1, 2, 3
are summarized in the table:
j

b0j

b1j

1
2

1
2

5
12

8
12

b2j
b3j

9
24

19
24

1
12

5
24

1
24

Multi-step methods
Predictor corrector methods
The introduced multistep methods are not mostly used stand-alone but in
combination as the predictor-corrector methods. Using the predictor
method the next value of the solution is calculated. The result is used for
the evaluation of the corrector formula which gives the value of the solution.
It is important to use the predictor and corrector formulas same order. The
local errors of both formulas are same order).
The estimation of the local error of the predictor-corrector methods can be
done using the formula:
P
C

ET ( t i+1) yi+1 yi+1

If the value of the error is less than the maximal error, we use the value as the
solution at the point ti+1 . If not, the value of the corrector formula is used as a
prediction of the solution and is used for the second evaluation of the corrector
formula and computation of the new value of the solution in the point ti+1 . The
error is estimated in the mentioned way and the described procedure can be
repeted again.

Numerical solution of the system of the ordinary

differential equations system of the state
equations
x( t 0 ) = x0

x! = f (x, t),

Eulers method

x i +1 = x i + h f t i , x i

k1 = f t i , x i

Heuns method

h
h

k 2 = f t i + i , xi + i k1

2
2

h
xi +1 = xi + i k1 + k 2
2

k1 = f t i , x i

Runge-Kutta Method 4th order

k1 = f t i , x i

k 2 = f t i +1 , x i + h i k 1

h
xi +1 = xi + i k1 + 2 k 2 + k 3 + k 4
6

h
h

k 2 = f t i + i , xi + i k1

2
2

h
h

k 3 = f t i + i , xi + i k 2

2
2

k 4 = f t i +1, xi + hi k 3

Numerical solution of the system of the

ordinary differential equations
The estimation of the transaction error of the method can be realized
at the similar way like by the solution of only one differential equation
using the half step method. The following condition must be full field:
x j( t i + h) x j( t i + 2h) ,

j = 1, 2, ..., n

The accuracy of each equation must be tested and full field!

Numerical solution of the system of the

ordinary differential equations
The multi step methods can be used in the same way for the
solution of the system of the ordinary differential equations.
General formula of the explicit method:
k

xi +1 = xi + h b kj f t i j , xi j
j= 0

General formula of the implicit method:

k

x i +1 = x i + h k kj f t i +1 j , x i +1 j
j= 0

The accuracy is estimated in the same way like by the solution of one equation.
The solution of each equation must full fill the condition.

Stability of the numerical solution of

the differential equation
Test differential equation

y! = y

Stability area for the Eulers method

y i +1 = y i + h f ( y i , t i )

Im (h)

y i +1 = y i + h y i

yi +1 = (1 + h ) yi
-2

Re (h)

-1

y i +1 y i
-1

1+ h 1

Stability area for the Eulers method

The method is stable for the real number < 0 only for the lenght of the
step of the solution:
2
h<

Stability of the numerical solution of

the differential equation
The stability areas for the Runge-Kutta methods are given by thee xpression
1+ h +

2 h2
p h p
+ ... +
1
2!
p!

The stability areas for the Runge-Kutta

methods

Im( h)

And are shown in the Figure

RK 4..

2
RK 2..

RK 3..

1
RK 1..

-3

-2

-1

Re(h)

Stability of the numerical solution of

the differential equation
The stability areas of the multi step methods
Im(h)
1,5

A-B 2..
A-B 3..

A-B 4..
A-B 5..

Re(h)
-2

-1
A-M 2..

0
A-M 3..

A-M 4..

A-M 5..

Stiff systems of the differential

equations
Ration of the stiff system:
max Re i
=

min Re i
i

T
= min
Tmax

It is suitable to solve the stiff systems using the numerical methods which have
the stability area the whole left half plane of the complex plane : Re h < 0 .
These methods are called A-stabil methods :
1. The explicit multistep method cannot be the A - stable method.
2. The highest order of the implicit multistep method is 2, the smallest local
error has the trapezium method.

Astable methods of the solution of

the ODE
Implicit A-stable method first order is the implicit Eulers method
described by the formula:
y

i +1

= y + hf y
i

i +1

, t i+1

The area of the absolute stability is

given by the expression:

1
1
1 h
Im (h)

area of the absolute stability of the

implicit Eulers method

-1

Re (h)

Astable methods of the

solution of the ODE
Trapezium method as derived by the combination of the direct and implicit
Eulers method and is described by the formula:
y

i +1

=y +
i

h
f ( yi , t i ) + f ( yi +1, t i+1)
2

1
h
2
1
1
1 h
2

1+

The trapezium and implicit Eulers methods need thepredicted values of the
slution. These can be calculated from using the direct Eulers method or other
explicit method.

Example
It is to create a state model of the system shown in Figure. Next do the
analysis of the use of the numerical methods for the solution of the differential
equations for the given values of the model parameters.
m = 1 kg
k = 100 Nm-1
b = 1000 Nsm-1
b

Initial conditions:
x0 = 0
x0 = 1 ms-1

x, x

Solution
The state space model has form:

d x1 0
=
dt x2 k
m

1 x
1

b x2

m

x1(0) 0
x(0) =
=
x2 (0) 1
d x1 0
=
dt x 2 100

After substitution of the numerical values we obtain:

We calculate the eigen values of the matrix A:

x1

1000 x 2

det(A I) = 0

1 1 0
0
det

= 0
100 1000 0 1

After the evaluation of the determinant

we obtain the characteristic equation of
the system:
The ratio of the stiff systm is equal:

2 + 1000 + 100 = 0
1 = 0,1
2 = 999,899

999,899
0,1

=! 104

The ratio of the stiff system is order 4 (104 ), that means the model represents
the stiff system.

Solution
Analysis of the numerical solution:
In the case of the real eigen values the following condition must be full filled:
max h 2

For the step of the numerical

solution we obtain:

2
999,899
h 0,002
h

It is also valid for the Runge-Kutta 4th order method:

2,7
999,899
h 0,0027
h

Because of the numerical stability it is to use very short step. Next the achieving
of the given accuracy will need to use more shorter step. The use of the A- stabil
method is suitable. The whole left half plane is a stable area for these methods.