Modeling & Simulation

Lecture 10 Part 2

Generating Random
Variates

Instructor:
Eng. Ghada Al-Mashaqbeh
The Hashemite University
Computer Engineering Department
The Hashemite University 2
Outline
Introduction.
Generating continuous random
variates.
Generating empirical random
variates.
Generating discrete random variates.
Generating arrivals processes.
The Hashemite University 3
Introduction
We have explored the general techniques
to generate random variates.
In this lecture we will explore the common
algorithms used with the theoretical
distributions (both continuous and
discrete) that we have dealt with before.
So, just pick the distribution and its
algorithm and work with it.
In addition, we will deal with empirical
random variates generation and arrival
processes.

The Hashemite University 4
Continuous Distributions
Uniform
Use the inverse transform algorithm.

Exponential
Use the inverse transform algorithm.

Weibull
Use the inverse transform algorithm.


U a b a X ) ( + =
U X ln | =
o
|
/ 1
)] 1 ln( [ U X =
The Hashemite University 5
Continuous Distributions
Gamma I
Pdf and cdf function








Where o is the shape parameter and is the
scale parameter.
( )

>
=

otherwise 0
0
!
/
1
) (
1
0
/
x
j
x
e
x F
j
j
x
o
|
|

>
I
=

otherwise 0
0
) (
) (
/ 1
x
e x
x f
x
o
|
| o o
The Hashemite University 6
Continuous Distributions Gamma II
No closed-form inverse cannot use the
inverse transform method.
if
Gamma(1,1) is exponential with mean 1.
For this purpose we will consider Gamma
distribution with = 1 and
) , ( ~ | o | gamma X ) 1 , ( ~ o gamma X
1 , 1 0 > < < o o
The Hashemite University 7
Gamma(o,1) Density
The Hashemite University 8
Continuous Distributions Gamma
(0<o<1) I
0<o<1: Acceptance-rejection with

<
I
s <
I
s
=

x
e
x
x
x
x t
x
1
) (
1 0
) (
0 0
) (
1
o
o
o
e e b b
dx
e
dx
x
dx x t c
x
/ ) ( ere wh )), ( /(
) ( ) (
) (
1
1
0
1
0
o o o
o o
o
+ = I =
I
+
I
= =
} } }

<
s <
s
=

x
b
e
x
b
x
x
x r
x
1
1 0
0 0
) (
1
o
o
o
The Hashemite University 9
Continuous Distributions Gamma
(0<o<1) II
Where



Which is called the Gamma Function
and it is defined for any real number
>0.
dt e t
t
}

= I
0
1
) (
o
o
The Hashemite University 10
Continuous Distributions Gamma
(0<o<1) III
Now how to generate Y from r(x)???
Use inverse transform, but remember r(x) is a
density so we need to find its cdf first:

<
s s
= =

}
x
b
e
x
b
x
dy y r x R
x
x
1 1
1 0
) ( ) (
0
o
o
( )

s
= =

otherwise
) 1 (
ln
1
) (
/ 1
1
o
o
u b
b
u bu
X u R
The Hashemite University 11
Continuous Distributions Gamma
(0<o<1) IV
To generate X apply the acceptance
rejection algorithm:




For our case:

1. Step back to go Otherwise
. return ), ( ) ( If 3.
of t independen Generate . 2
density ith Generate 1.
Y X Y t Y f U
Y U
r w Y
= s

>
s s
=

1 ,
1 0 ,
) (
) (
1
Y Y
Y e
Y t
Y f
Y
o
The Hashemite University 12
Continuous Distributions Gamma
(o>1) I
Acceptance-rejection with the following
t(x) and use inverse transform to
generate Y from r(x):
( )
( )
) (
4
1 2
2
1
o
o
o
o

o o

I
=
=
=
+
=

e
c
x
x
x t

>
+
=
otherwise
x
x
x
x R
, 0
0 ,
) (

1 0 ,
1
) (
/ 1
1
< <
|
.
|

\
|

= =

u
u
u
X u R

The Hashemite University 13

Continuous Distributions Gamma
(o>1) II
Final algorithm:
Follow the steps of acceptance-rejection
method and use t(x) and R(x) specified
in the previous slide.
The Hashemite University 14
Continuous Distributions Normal
Distribution function does not have closed form (so
neither does the inverse)
Can use numerical methods for inverse-transform
Note that


If we can generate unit normal (or called the
standard normal distribution), then we can generate
any normal.
Many algorithms were proposed to generate random
variates from the normal distribution which differ in
their speed and accuracy.
We will discuss two algorithms:
Box and Muller algorithm.
Polar algorithm.
) , ( ~
) 1 , 0 ( ~
o o N X
N X
+
The Hashemite University 15
Continuous Distributions Normal:
Box-Muller Method I
Algorithm





call. function next for the save and Return . 3
2 sin ln 2
, 2 cos ln 2 Set 2.
) 1 , 0 ( ~ , t independen Generate . 1
2 1
2 1 2
2 1 1
2 1
X X
U U X
U U X
U U U
t
t
=
=
The Hashemite University 16
Continuous Distributions Normal:
Box-Muller Method II
Technically, independent N(0,1), but
serious problem if used with LCGs:
U
1
and U
2
if from the same stream could be
adjacent resulting in having X1 and X2 to be
dependent.
Solution:
Generate U
1
and U
2
from different streams
(i.e. start with different seed values).
The Hashemite University 17
Continuous Distributions Normal:
Polar Method
An improved version of the Box and Muller algorithm
Faster than Box and Muller algorithm.
Also generates random variates in pairs.
Algorithm
Y V X
Y V X
W W Y
V V W U V U V
U U U
2 2
1 1
2
2
2
1 2 2 1 1
2 1


/ ) ln 2 (
let Otherwise, 1. step to go 1 W If 2.
and , 1 2 and 1 2 Let
). 1 , 0 ( ~ , t independen Generate . 1
=
=
=
>
+ = = =
The Hashemite University 18
Continuous Empirical Distributions
You have two cases:
You have an equation for the empirical distribution.
Or you use the actual observation.
For the first case you can use the inverse transform
method.
For the second case you can use the following
algorithm exploits such fact which is an equivalent
to the inverse transform method.
Algorithm (first you must rank the data):





n is the sample size (i.e. total number of empirical
Xs).

( )
) ( ) 1 ( ) (
) 1 ( Return . 2
1 let and
, 1 Let ). 1 , 0 ( ~ Generate . 1
I I I
X X I P X X
P I
)U (n- P U U
+ + =
+ =
=
+
The Hashemite University 19
Empirical Distribution Function
For example, for this distribution n = 6
The Hashemite University 20
Discrete Distributions
Can always use the inverse-transform method
but it may not be the most efficient for all
distributions.
Since it involve searching.
Another general method is the alias method,
which works for every finite range discrete
distribution
Requires some initial steps and extra storage.
Will not be discussed here.
We will explore how to generate random variates
from common discrete theoretical distributions
which are mainly based on inverse transform
method.
The Hashemite University 21
Bernoulli
Mass function



Algorithm: Discrete Inverse Transform
0 return Otherwise . 1 return If . 2
) 1 , 0 ( ~ Generate . 1
= = s X X p U
U U

=
=
=
otherwise 0
1
0 1
) ( x p
x p
x p
The Hashemite University 22
Binomial
Mass function





Use the fact that if X~binomial(t,p) then



So use the convolution method which is based on
the discrete inverse transform of Bernoulli.

e
|
|
.
|

\
|
=

otherwise 0
} ,..., 1 , 0 { ) 1 (
) (
t x p p
x
t
x p
x t x
) ( Bernoulli ~
...,
2 1
p Y
Y Y Y X
i
t
+ + =
The Hashemite University 23
Geometric
Mass function


Use inverse-transform
(

=
) 1 ln(
) 1 ln(
Return . 2
) 1 , 0 ( ~ Generate . 1
p
U
X
U U

e
=
otherwise 0
} ,..., 1 , 0 { ) 1 (
) (
t x p p
x p
x
The Hashemite University 24
Negative Binomial
Mass function





Note that X~negbin(s,p) iff



So use the convolution method which is based on
the discrete inverse transform of Geometric.

e
|
|
.
|

\
|

=
otherwise 0
} ,..., 1 , 0 { ) 1 (
1
1
) (
t x p p
s
x
x p
x s
) ( Geometric ~
...,
2 1
p Y
Y Y Y X
i
t
+ + =
The Hashemite University 25
Poisson
Mass function

Algorithm






This algorithm is based on the relation between
Poisson and exponential distributions.
Rather slow especially for large values of . No
very good algorithm for Poisson distribution.

e
=

otherwise 0
} ,..., 1 , 0 {
!
) (
t x
x
e
x p
x

1 step back to go and 1 Let . 3

3. step to go Otherwise . return , If
. by replace and ) 1 , 0 ( ~ Generate 2.
0 , 1 , Let . 1
1 1
+ =
= <
= = =
+ +

i i
i X a b
bU b U U
i b e a
i i

The Hashemite University 26

Generating Arrival Processes
Remember that we model the arrival
process as a Poisson process and batch
process based on the number of arrivals
per an arrival event.
For the Poisson process we have:
Stationary Poisson Process (SPP): the arrival
rate is constant.
Non-Stationary Poisson Process (NSPP): the
arrival rate varies with time.
We will explore algorithms used to
generate arrival times from each type.
The Hashemite University 27
Generating Arrival Times in
SPP
Stationary with rate >0
Time between events A
i
=t
i
-t
i-1
are IID
exponential
Algorithm



The above algorithm can be used with any
IID distribution (not only exponential) just
find the correct inverse of its cdf.
( ) U t t
U U
i i
ln
1
Return . 2
) 1 , 0 ( ~ Generate . 1
1

+ =

The Hashemite University 28
Generating Arrival Times in NSPP
Remember that a NSPP has an arrival rate that varies with
time.
Also remember the methods that we have used in Chapter 6
to fit a NSPP to a stationary one.
Thinning method.
Inversion method.
Based on these two methods we will generate arrival times
from a NSPP.
(t)
i
t
1 + i
t
The Hashemite University 29
Thinning Algorithm
1. Set t=t
i-1

2. Generate U
1
, U
2
IID U(0,1)
3. Replace t by


4. If return t
i
= t. Otherwise,
go back to step 2.

As you see the above algorithm is similar to the acceptance
rejection method.
{ } ) ( max where , ln
1
*
1
*
t U t
t

=
|
.
|

\
|
+
*
2
/ ) ( t U s
The Hashemite University 30
Inversion Algorithm
1. Generate U~ U(0,1)
2. Set t
i
=t
i-1
- ln U
3. Return

Where (see chapter 6):


As you see the above algorithm is similar to the inverse
transform method.
) (
' 1
i i
t t

A =
}
=
t
(s)ds (t)
0
The Hashemite University 31
Additional Notes
The lecture covers the following
sections from the textbook:
Chapter 8
Sections:
8.3 (8.3.1, 8.3.2, 8.3.4, 8.3.5, 8.3.6, 8.3.16),
8.4 (8.4.1, 8.4.4, 8.4.5, 8.4.6, 8.4.7),
8.6 (8.6.1, 8.6.2)