STOCHASTIC

Stochastic Models of Invasions and Epidemics
Linda J. S. Allen
Texas Tech University
2009 Summer School on

The Mathematics of Invasions in Ecology and Epidemiology
The Banff Centre
May 10-17, 2009
Texas Tech University 1

Outline of Presentation
I. Background on Stochastic Models
II. Stochastic SIS Epidemic Models DTMC, CTMC, SDE
Probability of Invasion of a Pathogen
III. Stochastic SIR Epidemic Models CTMC, SDE
Final Epidemic Size
IV Logistic Growth - CTMC, SDE
V Lotka-Volterra Competition - CTMC, SDE
Probability of Species Invasion

I. What is a Stochastic Model?
A stochastic model is formulated in terms of a stochastic process.
A stochastic process is a collection of random variables
{X(t; s))|t T, s S},
where T is the index set and S is a common sample space. Generally

X(t; s) X(t). The index set often represents time, such as
T = {0, 1, 2, . . .} or T = [0, )
Time can be discrete or continuous.
The study of stochastic processes is based on probability theory.

How do Stochastic Models Differ from Deterministic
Models?
A deterministic model is often formulated in terms of a system of

differential equations or difference equations.
A stochastic model is formulated as a stochastic process with a

collection of random variables.
A solution of a deterministic model is a function of time or space

and is generally uniquely dependent on the initial data.
A solution of a stochastic model is a probability distribution for

each of the random variables. One sample path over time or space is
one realization from this distribution.
Stochastic models capture the variability inherent due to the

demographics or environment variablility and are particularly important
when quantities in the processes are small- small population sizes or
small initial number of infectives.

The Following Graphs Illustrate the Dynamics of a
Deterministic versus a Stochastic SIR Epidemic Model
35
30
Number of Infectives, I(t)
25
20
15
10
0
0 500 1000 1500 2000
Time Steps

Whether the Random Variables Associated with The
Stochastic Process are Discrete or Continuous Distinguishes
Some of the Different Types of Stochastic Models.
A random variable X(t) of a stochastic process assigns a real value
to each outcome A S in the sample space and a probability,
Prob{X(t) A} [0, 1].
The values of the random variable constitute the state space. For
example, the number of cases associated with a disease may have the
following discrete or continuous set of values for its state space:
{0, 1, 2, . . .} or [0, N ].
which makes the random variables discrete or continuous. The
stochastic process is completely defined when the set of random variables
{X(t)} are related by a set of rules.

We will Study Stochastic Processes that have the Markov
Property.
A stochastic process with the Markov property is one where the

future state of the process depends only on the current state, and not
on the past. That is, for a discrete-time stochastic process,
Prob{X(t + t)|X(t), X(t t), . . . , X(0)} = Prob{X(t + t)|X(t)}.
At a fixed time t, each random variable X(t) has an associated

probability distribution.
Discrete: Prob{X(t) = i} = pi(t), i {0, 1, 2 . . .}

Rb
Continuous: Prob{X(t) [a, b]} = a p(x, t)dx

The Choice of Discrete or Continuous Random Variables
with a Discrete or Continuous Index Set Defines the Type
of Stochastic Model.
Discrete Time Markov Chain (DTMC): t {0, t, 2t, . . .} X(t) is

a discrete random variable.
X(t) {0, 1, 2, . . . , N }
The term chain implies that the random variable is discrete.

Continuous Time Markov Chain (CTMC): t [0, ), X(t) is a
discrete random variable.
Xt {0, 1, 2, . . . , N }
Diffusion Process, Stochastic Differential Equation (SDE): t [0, ),

X(t) is a continuous random variable.
X(t) [0, N ]

II. Before we Formulate the Stochastic SIS Epidemic
Models, we Review the Dynamics of the Deterministic SIS
Epidemic Model.
Deterministic SIS:
S I
dS
= SI + (b + )I
dt N
dI
= SI (b + )I
dt N
where > 0, > 0, N > 0 and b 0, S(t) + I(t) = N .

The Dynamics of the Deterministic SIS Epidemic Model
Depend on the Basic Reproduction Number.
The parameter values represent
= transmission rate
b = birth rate = death rate
= recovery rate
N = total population size = constant.
Basic Reproduction Number:

R0 =
b+
If R0 1, then limt I(t) = 0.

1
If R0 > 1, then limt I(t) = N 1 R0
.

We will Formulate the Three Types of Stochastic SIS
Epidemic Models by Defining Relationships Among the
Random Variables Assuming the Markov Property Holds.
S(t) = random variable for the number of susceptible individuals.
I(t) = random variable for the number of infected individuals.
S(t) + I(t) = N = maximum population size.
Discrete Time Markov Chain (DTMC): t {0, t, 2t, . . .}, I(t) is a

discrete random variable,
I(t) {0, 1, 2, . . . , N }
Continuous Time Markov Chain (CTMC): t [0, ), I(t) is a discrete

random variable.
I(t) {0, 1, 2, . . . , N }
Diffusion Process, SDEs: t [0, ), I(t) is a continuous random

variable.
I(t) [0, N ]

First, We Formulate a DTMC SIS Epidemic Model.
Let I(t) denote the discrete random variable for the number of infected
(and infectious) individuals with associated probability function
pi(t) = Prob{I(t) = i}
where i = 0, 1, 2, . . . , N is the total number infected at time t. The

probability distribution is
p(t) = (p0(t), p1(t), . . . , pN (t))T
for t = 0, t, 2t, . . . . Now we relate the random variables {I(t)}

indexed by time t by defining the transition probability from state i to
state j, i j, in time t as
pji(t) = Prob{I(t + t) = j|I(t) = i}.

Assume that t is Sufficiently Small, Such that the
Number of Infectives Changes by at Most One in Time t.
That is,
i i + 1, i i 1 or i i.
Either there is a new infection, birth, death, or a recovery. Therefore,

the transition probabilities are

i(N i)/N t = b(i)t, j =i+1
(b + )it = d(i)t, j =i1

pji(t) = 1 [i(N i)/N + (b + )i]t =
1 [b(i) + d(i)]t, j=i

0, j 6= i + 1, i, i 1,


The Probability Distribution Associated with the Epidemic
Process Over Time is Found by Repeated Multiplication of
the Transition Matrix.
Matrix P (t) = (pji(t)) is known as the transition matrix:
p(t + t) = P (t)p(t),
where p(t) = (p0(t), . . . , pN (t))T is the probability distribution and

P (t) is

1 d(1)t 0 0
0 1 [b(1) + d(1)]t d(2)t 0

0
b(1)t 1 [b(2) + d(2)]t 0

0
. 0 b(2)t 0 .
. .. .. .. ..

0 0 0 d(N )t
0 0 0 1 d(N )t
Matrix P (t) is stochastic, the column sums equal one.

The Stochastic Process for the DTMC SIS Model is known
as a Finite State Markov Chain with the Following
Properties.
The stochastic process {I(t)} for t {0, t, 2t, . . .} is time-

homogeneous (transition probabilities do not depend on time) and has
the Markov property.
The probability of no infections p0 is an absorbing state.
0 1 2 N
For any initial distribution p(0) = (p0(0), . . . , pN (0))T , zero

through a total of N infections
lim p(t) = (1, 0, . . . , 0)T lim p0(t) = 1.

t t

Three Sample Paths of the DTMC SIS Model are
Compared to the Solution of the Deterministic Model.
A sample path or stochastic realization of a stochastic process
{I(t)} for t {0, t, 2t, . . .} is an assignment of a possible value to
I(t) for each value of t.
R0 = 2 .
70
60
Number of Infectives
50
40
30
20
10
0
0 5 10 15 20 25
Time
t = 0.01, N = 100, = 1, b = 0.25, = 0.25, I (0) = 2, and

p2(0) = 1.The MATLAB program is in the Appendix.

The Probability Distribution p(t) for the Number of
Infected.
0.75
Probability 0.5
0.25
0
0
0
1000 50
2000 100
State
Time, n
Probability distribution for the DTMC SIS model, t = 0.01, N = 100, = 1,

b = 0.25, = 0.25, R0 = 2, I (0) = 2 and p2(0) = 1.

Next, We Formulate a CTMC SIS Model.
Time is continuous, but the random variable for number of infected
individuals is discrete. The discrete random variable I(t), t [0, ) has
an associated probability function
pi(t) = Prob{I(t) = i}
The infinitesimal transition probability for small t is

i(N i)/N t + o(t) = b(i)t + o(t), j =i+1
(b + )it + o(t) = d(i)t + o(t), j =i1

pji(t) = 1 [i(N i)/N + (b + )i]t + o(t)
= 1 [b(i) + d(i)]t + o(t), j=i

o(t), otherwise,

where o(t) 0 as t 0.
i i + 1, i i 1, or i i.

A System of Differential Equations for the Probabilities Can
be Derived Based on the Transition Probabilities.
Assume Prob{X(0) = m} = 1. For small t,
pi(t + t) = pi1(t)[b(i 1)t] + pi+1(t)[d(i + 1)t]

pi(t)[1 (b(i) + d(i))t] + o(t)
Subtracting pi(t), dividing by t, and letting t 0,
dpi
= pi1b(i 1) + pi+1d(i + 1) pi[b(i) + d(i)]
dt
dp0
= p1d(1)
dt
for i = 1, 2, . . . , N, where
b(i) = i(N i)/N, d(i) = (b + )i.
These differential equations are known as the forward Kolmogorov

differential equations.

The Epidemic Process can be modeled by a System of
Differential Equations
In matrix notation,
dp
= Qp,
dt
where Q is known as the generator matrix and p(t) =
(p0(t), . . . , pN (t))T
0 1
0 d(1) 0 0
B0
B [b(1) + d(1)] d(2) 0 C
C
B0
B b(1) [b(2) + d(2)] 0 C
C
Q = B0 0 b(2) 0 C.
B C
B .. ... ... ... ... C
B. C
B C
@0 0 0 d (N ) A
0 0 0 d(N )
b(i) = i(N i)/N and d(i) = (b + )i

The DTMC Transition Matrix and CTMC Differential
Equations are Closely Related when t is Small.
In the DTMC Model,
p(t + t) = P (t)p(t),
where P (t) is the transition matrix. Letting t 0, we obtain the

Kolmogorov differential equations for the CTMC model,
p(t + t) p(t) P (t) I

= p(t)
t t
dp
= Qp
dt
where
P (t) I
Q = lim .
t0 t
The Discrete-Time Process can be used to Approximate the

Continuous-Time Process.

Because of the Markov Property, the Inter-Event Time in a
CTMC Model Has an Exponential Distribution.
The exponential distribution has the memoryless property. Let I(t) = n
and Tn denote the inter-event time, a continuous random variable for
the time to the next event. Take the sum of all the probabilities of all
possible events where there is a change in state, i i + 1, i i 1:

X
pjn(t) = a(n)t + o(t)
j =0,j6=n
and
pnn(t) = 1 a(n)t + o(t).
The interevent time has an exponential distribution with parameter
a(n):
Tn E(a(n))
Prob{Tn t} = 1 exp(a(n)t).

For the SIS Epidemic Model, with I(t) = n,

X
pjn(t) = [b(n) + d(n)]t + o(t)
j =0,j6=n
= [n(N n)/N + (b + )n]t + o(t)

a(n) = n(N n) + (b + )n
N

To Numerically Simulate the Inter-Event Time in a CTMC
Model, We Use a Uniform Random Variable.
The inter-event time, waiting time until an event occurs, can be
numerically computed using a uniform random variable and the
cumulative distribution for Tn. Let U be uniform random variable
on [0, 1] and Fn(t) the cumulative distribution for Tn
Fn(t) = Prob{Tn t} = 1 exp(a(n)t).
Then
1 1
Prob{Fn (U ) t} = Prob{Fn(Fn (U )) Fn(t)}
= Prob{U Fn(t)}
= F n (t )
The inter-event time Tn, given I(t) = n satisfies
ln(1 U ) ln(U )
Tn = Fn1(U ) = = .
a(n) a(n)

Three Sample Paths of the CTMC SIS Model are
Compared to the Deterministic Solution.
R0 = 2
80
70
60
Number of Infectives
50
40
30
20
10
0
0 5 10 15 20 25
Time
b = 0.25, = 0.25, = 1, N = 100, I(0) = 2, R0 = 2.

For t small, the dynamics of the DTMC and the CTMC Models are
Similar. The DTMC model can be used as an approximation for the
CTMC model. MATLAB program is in the Appendix.

Next, We Formulate the Third Type of Stochastic Model,
an SDE Model.
Time is continuous and the random variable is also continuous. Let
{I(t) : t [0, )} be a stochastic process. We compute the
infinitesimal mean and variance for this diffusion process.
Change (I) Probability
+1 b(I)t
1 d(I)t
Infinitesimal Mean:
E(I|I) = b(I)t d(I)t
Infinitesimal Variance:
2(I|I) E[(I(t))2|I] = b(I)t + d(I)t.

Let Normal(0, 1). Then for large number of changes of I(t)
in a small time interval, we can apply the Central Limit Theorem:
p
I(t) [b(I) d(I)]t + [b(I) + d(I)]t
We arrive at the stochastic differential equation (SDE):
p
dI(t) = [b(I) d(I)]dt + [b(I) + d(I)] dW (t) = drift + diffusion
where W (t) Normal(0, t) is the Wiener process (white noise,

standard Brownian motion).
W (t) Normal(0, t)

The Probability Density Function is a Solution of the
forward Kolmogorov Differential Equation
Given an infinitesimal mean and variance as [b(I) d(I)]t and
[b(I) + d(I)]t of a diffusion process I(t), the probability density
p(x, t),
Z b
Prob{I(t) [a, b]} = p(x, t)dx
a
is a solution of the following forward Kolmogorov differential equation:
p(x, t) ([b(x) d(x)]p(x, t)) 1 2([b(x) + d(x)]p(x, t))

= +
t x 2 x2
The It
o SDE can be expressed as an It
o stochastic integral
Z t Z t q
I (t ) = I (0) + [b(I (s)) d(I (s)]ds + b(I (s)) + d(I (s))dW (s)
0 0
= drift + diffusion

It
o Stochastic Integral
Z b
E(f 2(t)) dt < ,
a
Assume f (t) is a random function, f (t) f (t, X(t)). Let a = t1 < t2 <
< tk < tk+1 = b be a partition of [a, b], t = ti+1 ti = (b a)/k,
and W (ti) = W (ti+1) W (ti), where W (t) is the standard Wiener
process. Then the It o stochastic integral of f is defined as
Z b k
X
I = f (t) dW (t) = l.i.m.k f (ti) W (ti) = l.i.m.kFk
a i=1
where l.i.m. denotes mean square convergence. l.i.m.kFk = I

means
lim E[(Fk I)2] = 0.
k
Note: f (ti) is evaluated at the left endpont (It
o).

The It
o Stochastic Integral and It
o SDE for the SIS
Epidemic Process.
Stochastic Integral Equation:
t t
r

Z Z
I (t) = I (0)+ I (N I ) (b + )I ds+ I (N I ) + (b + )I dW (s)
0 N 0 N
SDE:
r

dI (t) = I (N I ) (b + )I dt + I (N I ) + (b + )I dW (t),
N N
where W (t) is a Wiener process (white noise). Sample paths for a

Wiener process are continuous but nowhere differentiable.
W (t) Normal(0, t), W (t + t) W (t) Normal(0, t).
0.5
0
W(t)
-0.5
-1
-1.5
0 0.2 0.4 0.6 0.8 1
Time

Three Sample Paths for the SDE SIS Model are Computed
Numerically and Compared to the Deterministic Solution.
R0 = 2
80
70
Number of Infectives 60
50
40
30
20
10
0
0 5 10 15 20 25
Time
b = 0.25, = 0.25, = 1, N = 100, I (0) = 2. MATLAB program is in the

Appendix - Euler-Maruyama method.
Note: The SDE model is often a good approximation to the CTMC
model when I(0) and N not too small.

Even Though R0 > 1, the Stochastic SIS Epidemic Model
Predicts the Epidemic Ends.
When R0 > 1, the Deterministic SIS epidemic model predicts that an

endemic equilibrium is reached. This is not the case for the Stochastic
SIS epidemic models.
lim p0(t) = 1.
t
As mentioned earlier, this absorption at zero may take an exponential

amount of time so that when N is large and I(0) = i is small,
0 < p0(t) P0 = constant < 1
for a long period of time.

An Estimate for P0 Can be Obtained From a Simple Birth
and Death Process.

Birth = b(i) = i(N i) i
N
Deaths = d(i) = (b + )i
Based on an random walk model or a simple birth and death process,
d(i) b+ 1
= =
b(i) R0
an estimate for the probability of no pathogen invasion for I(0) = m is
m m
b+ 1
Probability of no Invasion = P0 =
R0

The Approximation Follows From the Probability of
Extinction for a Birth and Death Process on {0, 1, 2, . . .}
Suppose there are initially X(0) = m individuals. What is the
probability of extinction in a birth and death process on {0, 1, 2, . . .}?
P d(1) d(i)
i=m
b(1) b(i)
P0 =
P d(1) d(i)
1+ i=1
b(1) b(i)
d(i) d(1)
In the case that = , then the formula simplifies to
b(i) b(1)
m
d(1)
P0 =
b(1)

There is a Bimodal Probability Distribution Over Time.
Example 1:
R0 = 2, I(0) = 1, P0 1/2
0.8
Prob{I(t)=i}
0.6
0.4
0.2
0
0
0
1000 50
2000 100
i
Time Steps

Example 2:
R0 = 2, I(0) = 3, P0 1/8
0.8
Prob{I(t)=i}
0.6
0.4
0.2
0
0
0
1000 50
2000 100
i
Time Steps

Some Advantages of the SIS Stochastic Models Over the
Deterministic SIS Model:
The SIS Deterministic Model Does Not capture
(i) The Variability Inherent in the Transmission, Recovery, Birth, and

Death Processes
(ii) The Probability of No Pathogen Invasion when R0 > 1.
The Stochastic Models Do Capture these Features.

III. SIR Epidemic Model.
ODE model:
dS
= SI
dt N
dI
= SI I
dt N
dR
= I
dt
Basic Reproduction Number:

R0 =

S(0)
There is an epidemic (an increase in number of infectives) if R0 > 1,
N
limt I(t) = 0.

CTMC SIR Epidemic Model
S (t) + I (t) + R(t) = N = maximum population size.
Since R = N S I, the process is bivariate. Let S(t) and I(t)
denote discrete random variables for the number of susceptible and
infected individuals, respectively with joint probability function
p(s,i)(t) = Prob{S(t) = s, I(t) = i}
where R(t) = N S(t) I(t). For this stochastic process, we define

transition probabilities as follows:
p(s+k,i+j ),(s,i)(t) = Prob{(S, I ) = (k, j )|(S (t), I (t)) = (s, i)}

8
>
> i(N i)t/N + o(t), (k, j ) = (1, 1)
it + o(t), (k, j ) = (0, 1)
<
=
>
> 1 [i(N i)/N + i]t + o(t), (k, j ) = (0, 0)
:
o(t), otherwise

Next we Formulate It
o SDEs
We find the infinitesimal mean and infinitesimal variance. for the bivariate
process {(S(t), I(t)) : t [0, )}.
Changes (S, I) Probability
(1, 1) (SI/N )t
(0, 1) It
Let X(t) = (S(t), I(t))T . Then

SI/N
E(X(t)|X(t)) = t = f t
SI/N I

SI/N SI/N
E(X(X)T |X(t)) = t = t
SI/N SI/N + I
Consider the specific changes listed in the table, a transmission and a
recovery. Define G in terms of those two changes but use square roots
so that GGT = :
p
p SI/N 0
G=
SI/N I

Then the It
o SDE take the following form:
dX(t) = f (X(t), t)dt + G(X(t), t)dW (t)
p
dS = [SI/N ]dt SI/N dW1
p p
dI = [SI/N I]dt + SI/N dW1 IdW2,
where W1 and W2 are two independent Wiener processes,

p
p SI/N 0
G=
SI/N I

SI/N SI/N
GGT = = .
SI/N SI/N + I
If the terms associated with the Wiener processes are dropped, then we
have the ODE model.

Three Stochastic Sample Paths of the SDE SIR Epidemic
Model Are Compared to the Deterministic Solution.
S(0)
R0 = 2, R0 = 1.96.
N
35
Number of Infectives 30
25
20
15
10
0
0 5 10 15 20
Time
t = 0.01, N = 100, = 1, b = 0, = 0.5, I(0) = 2.

The Final Size of the Epidemic (Total Number Infected)
will be Examined in the SIR Epidemic Model.
ODE Model:
dS
= SI
dt N
dI
= SI I
dt N
The Final Size of the Epidemic can be determined directly from the
differential equations, Integrating dI/dS = 1 + N /S,
N S(t)
I(t) + S(t) = I(0) + S(0) + ln .
S(0)

Letting t ,
N S()
S() = I(0) + S(0) + ln .
S(0)
The final size is

R() = N S().
S(0) = N 1, I(0) = 1, = 1
N
R0 20 100 1000
0.5 1.87 1.97 2.00
1 5.74 13.52 44.07
2 16.26 80.02 797.15
5 19.87 99.31 993.03
10 20.00 100.00 999.95

In the Stochastic SIR Epidemic Model There is a
Distribution of Final Sizes.
Let (s, i) be the ordered pairs of values for susceptible and infected
individuals in the CTMC model. Then
N
X
lim p(s,0)(t) = 1.
t
s=0
0,4
0,3 1,3
0,2 1,2 2,2
0,1 1,1 2,1 3,1
0,0 1,0 2,0 3,0 4,0

The Distribution for the Final Size (SIR) Depends on R0,
I(0), and N .
Example 1: I(0) = 1, N = 20, = 1
0.7
0.6 (a) R = 0.5

0
R0 = 2
Probability
0.5 R0 = 5
0.4
0.3
0.2
0.1
0
0 5 10 15 20
Final size

Example 2:
I(0) = 1, N = 100, = 1
0.6 (b) R0 = 0.5

R =2
0
0.5 R =5
0
Probability
0.4
0.3
0.2
0.1
0
0 20 40 60 80 100
Final size

The Average Final Size of the Stochastic Model and the
Deterministic Model Do Not Agree.
= 1, S(0) = N 1, I(0) = 1
Deterministic
N
R0 20 100
0.5 1.87 1.97
1 5.74 13.52
2 16.26 80.02
5 19.87 99.31
10 20.00 100.00
Stochastic (CTMC)
N
R0 20 100
0.5 1.76 1.93
1 3.34 6.10
2 8.12 38.34
5 15.66 79.28
10 17.98 89.98

IV. Logistic SDE

dN N
Deterministic: = rN 1 .
dt K
Let b(n) d(n) = rn(1 n/K).
CTMC:
8
>
> b(n)t + (t), j = 1
d(n)t + o(t), j = 1
<
Prob{N (t) = j|N (t) = n} =
>
> 1 (b(n) + d(n))t + o(t), j = 0
6= 0, 1, 1
:
o(t), j
SDE:
p
dN (t) = [b(N ) d(N )] dt + b(N ) + d(N ) dW (t)

(a) Suppose b(N ) = rN and d(N ) = rN 2/K.
SDE:
s
N (t) N (t)
dN (t) = rN (t) 1 dt+ rN (t) 1 + dW (t), (1)
K K
N (t) [0, ).
(b) Suppose b(N ) = rN (t)(1 N (t)/2K) and d(N ) = rN 2/2K.

SDE:

N (t) p
dN (t) = rN (t) 1 dt + rN (t) dW (t), (2)
K
N (t) [0, 2K].

Logistic SDE in Cases (a) and (b)
800
800
700
700
600 (a)
600 (b)
4
Prob{ N=n } 104

Prob{ N=n } 10
500 500
400 400
300 300
200 200
100 100
0 0
20 30 40 50 60 70 80 20 30 40 50 60 70 80
n n
Figure 1: Probability histogram for SDE logistic, 10,000 realizations; r = 1,

K = 50 = X (0).
(a) Mean of X (5): 49.0; Standard Deviation of X (5): 7.1
(b) Mean of X (5): 49.5; Standard Deviation of X (5): 5.0

Stochastic Birth and Death Process with Demographic and
Environmental Variability
Let b(t) and d(t) be the environmental variability of the per capita birth
and death rates that are independent of the population size N .
SDE:
p
dN (t) = [b(t) d(t)]N dt + [b(t) + d(t)]N dW1(t)
db(t) = a1(be b(t)) dt + 1dW2(t)
dd(t) = a2(de d(t)) dt + 2dW3(t),
where b(t)N = 0 if b(t) < 0 or N > K and d(t)N = 0 if d < 0. The

parameters ai, be, de and i, i = 1, 2 are positive.

The solutions to b(t) and d(t) can be found using It
os formula. If
db(t) = a1(be b(t)) dt + 1dW2(t),
then Z t
b(t) = be + ea1t (b(0) be) 1ea1 dW2( ) ,
0
with expectation and variance
E(b(t)) = be + (b(0) be)ea1t

12
V ar(b(t)) = (1 e2a1t)
2a1

V. A Lotka-Volterra Competition Model
ODE model:
dx1 r1
= x1(r1 a11x1 a12x2) = x1(K1 x1 12x2)
dt K1
dx2 r2
= x2(r2 a21x1 a22x2) = x2(K2 x2 21x1)
dt K2
Species 2 can invade when x1 = K1 if
K2 > 21K1

Lotka-Volterra Competion SDE
As in the case of logistic growth, to define an SDE it is necessary
to determine the contributions to birth and to death. We use the
simplest choice for birth and death rates. Assume b1 = r1X1, d1 =
X1(a11X1 + a12X2), b2 = r2X2, d2 = X2(a21X1 + a22X2). We can
write these in tabular form as follows:
Changes (X1, X2) Probability
(1, 0) r1X1t
(1, 0) X1(a11X1 + a12X2)t
(0, 1) r2X2t
(0, 1) X2(a21X1 + a22X2)t

The infinitesimal mean and variance are

r1X1 X1(a11X1 + a12X2)
E (X ) = t = f t
r2X2 X2(a21X1 + a22X2)

X1(r1 + a11X1 + a12X2)
T 0
E (X (X ) ) = t = t
0 X2(r2 + a21X1 + a22X2)
Matrix G can be chosen as follows:
p
r1X1 X1(a11X1 + a12X2) 0 0
G1 = p
0 0 r2X2 X2(a21X1 + a22X2)
or as
p
X1(r1 + a11X1 + a12X2) 0
G2 = p
0 X2(r2 + a21X1 + a22X2)
o SDEs for Lotka-Volterra competition with G2 ( G2GT

The system of It 2 = ):
q
dX1 = X1(r1 a11X1 a12X2)dt + X1(r1 + a11X1 + a12X2) dW1
q
dX2 = X2(r2 a21X1 a22X2)dt + X2(r2 + a21X1 + a22X2) dW2,
X1(t), X2(t) [0, ).

80 50
70
40
60
Population sizes
30
50
X2
40 20
30
10
20
10 0
0 1 2 3 4 5 0 20 40 60 80
Time X1
Figure 2: A sample path of the SDE Lotka-Volterra competition model over

time and in the phase plane, X1(0) = 50 and X2(0) = 25; (50,25) is a stable
equilibrium in the ODE model.
q
dX1 = X1(2 0.03X1 0.02X2)dt + X1(2 + 0.03X1 + .02X2)dW1
q
dX2 = X2(1.5 .01X1 .0.04X2)dt + X2(1.5 + 0.01X1 + 0.04X2)dW2

Probability of No Invasion
Using the simple birth and death process, when X1 = K1 and K2 > 21K1,
CTMC Model:
BirthX2 = r2X2
r2X2
DeathX2 = (21K1 + X2)
K2
Probability of No Invasion when X1 = K1 and X2 = 1 is
Death1 21K1 + 1
P0
Birth1 K2
Probability of No Invasion when X1 = K1 and X2 = 2 is
Death1Death2 (21K1 + 1)(21K1 + 2)

P0
Birth1Birth2 K22

Applying the parameters from the previous example, K2 > 21K1:
ODE model:
dX1 = X1(2 0.03X1 0.02X2)dt

dX2 = X2(1.5 .01X1 .0.04X2)dt
K1 = 66.67, K2 = 37.5, r2 = 1.5, 21 = 0.25

(a) Probability of No Invasion when X1 = K1 and X2 = 1 is
21K1 + 1
P0 0.47
K2
(b) Probability of No Invasion when X1 = K1 and X2 = 2 is
(21K1 + 1)(21K1 + 2)
P0 0.23
K22

Probability of No Species Invasion in Cases (a) and (b)
(a) P0 0.47 (b) P0 0.23
5000
2500
4000 (a) P0=0.49 (b) P0=0.25

2000
Prob{ X2=n } 104
Prob{ X =n } 104
3000 1500
2
2000 1000
1000 500
0 0
0 10 20 30 40 50 0 10 20 30 40 50
n n
Figure 3: Probability histogram for the invasive species X2 based on CTMC simulations at
t = 10 for 10,000 sample paths, X1(0) = 66.67 and (a) X2(0) = 1, (b) X2(0) = 2
CTMC, 10,000 realizations: (a) P0 = 0.49 (b) P0 = 0.25
SDE, 10,000 realizations: (a) P0 = 0.50 (b) P0 = 0.27

Summary of Main Points:
Stochastic processes model the variability inherent in the transmission, recovery,

birth and death processes - demographic variability.
In addition, stochastic processes model the environmental variability.
The DTMC model is an approximation to the CTMC model when the time
interval t is small.
The SDE model is an approximation to the CTMC model when the population
size and initial values are large.
Estimates for probability of pathogen invasion or species invasion can be

obtained from birth and death processes.

References and MATLAB programs:
1. Allen, EJ. 2007. Modeling with Ito Stochastic Differential Equations. Springer-Verlag.
2. Allen, EJ, LJS Allen, A Arciniega, and P Greenwood. 2008. Construction of
equivalent stochastic differential equations. Stochastic Analysis and Applications 64:
274-297.
3. Allen, LJS 2003. An Introduction to Stochastic Processes with Applications to Biology.
Prentice Hall, Upper Saddle River, N.J.
4. Allen, LJS and EJ Allen. 2003. A comparison of three different stochastic population
models with regard to persistence time. Theor. Pop. Biol. 439-449.
5. Allen, LJS and A. Burgin. 2000. Comparison of deterministic and stochastic SIS
and SIR models in discrete time. Mathematical Biosciences. 163: 1-33.
6. Gard, TC 1988. Introduction to Stochastic Differential Equations. Marcel Dekker,
Inc., New York and Basel.

CTMC SIS Epidemic
% Three Sample Paths

clear
set(0,DefaultAxesFontSize, 18)
set(gca,fontsize,18);
beta=1;
b=0.25;
g=0.25;
N=100;
init=2;
time=25;
sim=3;
for k=1:sim
clear t s i
t(1)=0;
i(1)=init;
s(1)=N-init;
j=1;
while i(j)>0 & t(j)<time
u1=rand;
u2=rand;
a=(beta/N)*i(j)*s(j)+(b+g)*i(j);
probi=(beta*s(j)/N)/(beta*s(j)/N+b+g);
t(j+1)=t(j)-log(u1)/a;
if u2<=probi
i(j+1)=i(j)+1;
s(j+1)=s(j)-1;

else
i(j+1)=i(j)-1;
s(j+1)=s(j)+1;
end
j=j+1;
end
if k==1
stairs(t,i,r-,LineWidth,2)
elseif k==2
stairs(t,i,b-,LineWidth,2)
else
stairs(t,i,g-,LineWidth,2)
end
hold on
end
% Eulers Method Applied to the Deterministic SIS Epidemic Model
dt=0.01;
y(1)=init;
for k=1:time/dt
y(k+1)=y(k)+dt*(beta*(N-y(k))*y(k)/N-(b+g)*y(k));
end
plot([0:dt:time],y,k--,LineWidth,2);
axis([0,time,0,80]);
xlabel(Time);
ylabel(Number of Infectives);
hold off

Logistic SDE
%Probability Histogram
clear all
set(0,DefaultAxesFontSize, 18)
set(gca,fontsize,18);
nsim=10000;
time=5;
r1=1;
K=50;
dt=.01;
for i=1:nsim
t(1)=0.;
x(1)=K;
for j=1:time/dt
ev1=r1*x(j);
ev2=r1*x(j)*x(j)/K;
ev3=r1*x(j)*(1-x(j)/(2*K));
ev4=r1*x(j)*x(j)/(2*K);;
rn1=randn;
rn2=randn;
a1=(ev1-ev2)*dt+sqrt((ev1+ev2)*dt)*rn1;
a2=(ev3-ev4)*dt+sqrt((ev3+ev4)*dt)*rn2;
x(j+1)=x(j)+a1;
if x(j+1)<=0;
x(j+1)=0.;
end
end
xend(i)=x(j+1);

end
hist(xend,[min(xend):1:max(xend)]);
mmm=mean(xend)
sss=std(xend)
xlabel(n);
ylabel(Prob\{ N=n \} \times 10^4, fontsize,18);

STOCHASTIC

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

STOCHASTIC

Uploaded by

Copyright:

Available Formats

Stochastic Models of Invasions and Epidemics

2009 Summer School on

Texas Tech University 1

I. Background on Stochastic Models

II. Stochastic SIS Epidemic Models DTMC, CTMC, SDE

Probability of Invasion of a Pathogen

III. Stochastic SIR Epidemic Models CTMC, SDE

Final Epidemic Size

IV Logistic Growth - CTMC, SDE

V Lotka-Volterra Competition - CTMC, SDE

Probability of Species Invasion

Texas Tech University 2

A stochastic model is formulated in terms of a stochastic process.

A stochastic process is a collection of random variables

{X(t; s))|t T, s S},

where T is the index set and S is a common sample space. Generally

Time can be discrete or continuous.

The study of stochastic processes is based on probability theory.

Texas Tech University 3

A deterministic model is often formulated in terms of a system of

A stochastic model is formulated as a stochastic process with a

A solution of a deterministic model is a function of time or space

A solution of a stochastic model is a probability distribution for

Stochastic models capture the variability inherent due to the

Texas Tech University 4

Texas Tech University 5

Prob{X(t) A} [0, 1].

Texas Tech University 6

A stochastic process with the Markov property is one where the

Prob{X(t + t)|X(t), X(t t), . . . , X(0)} = Prob{X(t + t)|X(t)}.

At a fixed time t, each random variable X(t) has an associated

Discrete: Prob{X(t) = i} = pi(t), i {0, 1, 2 . . .}

Texas Tech University 7

Discrete Time Markov Chain (DTMC): t {0, t, 2t, . . .} X(t) is

The term chain implies that the random variable is discrete.

Diffusion Process, Stochastic Differential Equation (SDE): t [0, ),

Texas Tech University 8

where > 0, > 0, N > 0 and b 0, S(t) + I(t) = N .

Texas Tech University 9

b = birth rate = death rate

N = total population size = constant.

Basic Reproduction Number:

If R0 1, then limt I(t) = 0.

Texas Tech University 10

S(t) + I(t) = N = maximum population size.

Discrete Time Markov Chain (DTMC): t {0, t, 2t, . . .}, I(t) is a

Continuous Time Markov Chain (CTMC): t [0, ), I(t) is a discrete

Diffusion Process, SDEs: t [0, ), I(t) is a continuous random

Texas Tech University 11

where i = 0, 1, 2, . . . , N is the total number infected at time t. The

p(t) = (p0(t), p1(t), . . . , pN (t))T

for t = 0, t, 2t, . . . . Now we relate the random variables {I(t)}

pji(t) = Prob{I(t + t) = j|I(t) = i}.

Texas Tech University 12

Either there is a new infection, birth, death, or a recovery. Therefore,

Texas Tech University 13

where p(t) = (p0(t), . . . , pN (t))T is the probability distribution and

Matrix P (t) is stochastic, the column sums equal one.

Texas Tech University 14

The stochastic process {I(t)} for t {0, t, 2t, . . .} is time-

The probability of no infections p0 is an absorbing state.

For any initial distribution p(0) = (p0(0), . . . , pN (0))T , zero

lim p(t) = (1, 0, . . . , 0)T lim p0(t) = 1.

Texas Tech University 15