Modeling & Simulation

Lecture 4

Probability Distributions
Review

Instructor:
Eng. Ghada Al-Mashaqbeh
The Hashemite University
Computer Engineering Department
The Hashemite University 2
Outline
Introduction.
Some useful discrete and continuous
probability distributions.
Empirical distributions.


The Hashemite University 3
Some Useful Probability
Distributions
Why we need to know such
distributions?
To find the suitable distribution that
models the simulation inputs where using
the mean instead of generating the
random variants from these distributions
results in serious errors most of the time.
Also they are needed to model the
simulation outputs to study their
behavior and the relation between them.
The Hashemite University 4
Discrete Distributions
Discrete random variables are used
to describe random phenomena in
which only integer values can
occur.
In this section, we will learn about:
Bernoulli trials and Bernoulli distribution
Binomial distribution
Geometric and negative binomial
distribution
Poisson distribution
The Hashemite University 5
Bernoulli Trials and Bernoulli
Distribution
Bernoulli Trials:
Consider an experiment consisting of n trials, each can
be a success or a failure.
Let X
j
= 1 if the jth experiment is a success
and X
j
= 0 if the jth experiment is a failure
The Bernoulli distribution (one trial):




where E(X
j
) = p and V(X
j
) = p (1-p) = p q
Bernoulli process:
The n Bernoulli trials where trails are independent:
p(x
1
,x
2
,, x
n
) = p
1
(x
1
) p
2
(x
2
) p
n
(x
n
)

= = =
= =
= =
otherwise , 0
2 1 0 , 1
,..., 2 , 1 , 1 ,
) ( ) ( ,...,n , ,j x q p
n j x p
x p x p
j
j
j j j
The Hashemite University 6
Binomial Distribution
The number of successes in n Bernoulli trials, X,
has a binomial distribution.







The mean, E(x) = p + p + + p = n*p
The variance, V(X) = pq + pq + + pq = n*pq
The number of
outcomes having the
required number of
successes and
failures
Probability that
there are
x successes and
(n-x) failures

=
|
|
.
|

\
|
=

otherwise , 0
,..., 2 , 1 , 0 ,
) (
n x q p
x
n
x p
x n x
The Hashemite University 7
Example 1 ( Binomial)
The chance that a bit transmitted through a
digital transmission channel is received in error
is 0.1. Also, assume that the transmission trials
are independent. Let X= the number of bits in
error in the next four bits transmitted:

1) Describe the sample space of this experiment
and indicate the value of X for each outcome.

2) Determine P(X = 2).

The Hashemite University 8
Example Solution I
1) Sample space : E =Error , O =Okay

Outcome X Outcome X
________________________________________
OOOO 0 EOOO 1
OOOE 1 EOOE 2
OOEO 1 EOEO 2
OOEE 2 EOEE 3
OEOO 1 EEOO 2
OEOE 2 EEOE 3
OEEO 2 EEEO 3
OEEE 3 EEEE 4
The Hashemite University 9
Example Solution II
The event that X = 2 consists of six outcomes
{ EEOO, EOEO, EOOE, OEEO, OEOE, OOEE}

P(EEOO) = P(E) P(E) P(O)P(O) =
(0.1)(0.1)(0.9)(0.9) =0.0081






0486 . 0 ) 9 . 0 ( ) 1 . 0 ( 6 ) 9 . 0 ( ) 1 . 0 (
2
4
) 2 (
2 2 2 2
= =
|
|
.
|

\
|
= = X P
The Hashemite University 10
Geometric & Negative Binomial
Distribution
Geometric distribution
The number of Bernoulli trials, X, to achieve the 1
st
success:




E(x) = 1/p, and V(X) = q/p
2

Negative binomial distribution
The number of Bernoulli trials, y, until the k
th
success
If Y is a negative binomial distribution with parameters p and
k, then:




E(Y) = k/p, and V(Y) = kq/p
2

=
=

otherwise , 0
,..., 2 , 1 ,
) (
1
n x p q
x p
x

+ + =
|
|
.
|

\
|

=

otherwise , 0
,... 2 , 1 , ,
1
1
) (
k k k y p q
k
y
y p
k k y
The Hashemite University 11
Example 2 ( Geometric)
The chance that a bit transmitted through a
digital transmission channel is received in error
is 0.1. Also, assume that the transmissions are
independent events. Let the random variable X
denote the number of bits transmitted until the
first error.

1) Determine P(X = 5)

P(X=5) = P(OOOOE) = (0.9)(0.9)(0.9)(0.9)(0.1)
= 0.066
The Hashemite University 12
Note
Note that the binomial, geometric,
and the negative binomial
distribution are based on the
following:
All are generated from Bernoulli trials.
All trials are independent.
The Binomial is the summation of n
Bernoulli(p) random variables.
The negative Binomial is the
summation of n Geometric(p) random
variables.
The Hashemite University 13
Poisson Distribution I
Poisson distribution describes many random processes
quite well and is mathematically quite simple.
Describes the number of events that occur at a
constant rate in a specified period of time.
where o > 0, pdf and cdf are:








E(X) = o = V(X)

=
=

otherwise , 0
,... 1 , 0 ,
!
) (
x
x
e
x p
x
o
o

=
x
i
i
i
e
x F
0
!
) (
o
o
The Hashemite University 14
Poisson Distribution II
The Hashemite University 15
Poisson Distribution III
Example: A computer repair person is alarmed
each time there is a call for service. The number
of alarms per hour ~ Poisson (o = 2 per hour).

The probability of three alarms in the next hour:
p(3) = e
-2
2
3
/3! = 0.18
also, p(3) = F(3) F(2) = 0.857-0.677=0.18

The probability of two or more alarms in a 1-hour
period:
p(2 or more) = 1 p(0) p(1)
= 1 F(1)
= 0.594
The Hashemite University 16
Continuous Distributions
Continuous random variables can be used
to describe random phenomena in which
the variable can take on any value in
some interval.
In this section we will study the following
distributions:
Uniform
Exponential
Normal
Weibull
Lognormal
The Hashemite University 17
Parameterization of
Continuous Distributions
A continuous probability distribution is described (or
defined) in terms of three parameters:

is the location parameter which determines the central
tendency of the distribution on the x-axis (it could be the
mean or the median). Sometimes called shift parameter.

is the scale parameter which determines the scale of the
values in the distribution (the standard deviation can be
considered as a scale parameter).

is the shape parameter which determines the shape of
the distribution (skewness of values). Not all continuous
distributions have a shape parameter.
|
o

The Hashemite University 18

Uniform Distribution
A random variable X is uniformly distributed on
the interval (a,b), U(a,b), if its pdf and cdf are:




Properties
P(x
1
< X < x
2
) is proportional to the length of the
interval [F(x
2
) F(x
1
) = (x
2
-x
1
)/(b-a)]
E(X) = (a+b)/2 V(X) = (b-a)
2
/12
U(0,1) provides the means to generate random
numbers, from which random variates can be
generated.
It is called Completely Random distribution,
why??????

s s

=
otherwise , 0
,
1
) (
b x a
a b
x f

>
s

=
b x
b x a
a b
a x
a x
x F
, 1
,
, 0
) (

The Hashemite University 19

Exponential Distribution I
A random variable X is exponentially
distributed with parameter > 0 if its pdf
and cdf are:

>
=

elsewhere , 0
0 ,
) (
x e
x f
x

> =
=
}

0 , 1
0 0,
) (
0
x e dt e
x
x F
x
x t

E(X) = 1/ V(X) = 1/
2

Used to model interarrival times when arrivals are
completely random, and to model service times that are
highly variable
The Hashemite University 20
Exponential Distribution II
For several different exponential pdfs (see figure), the value
of intercept on the vertical axis is , and all pdfs eventually
intersect.
The Hashemite University 21
Exponential Distribution III
Memoryless property
For all s and t greater than or equal to 0:
P(X > s+t | X > s) = P(X > t)
This means that the future is
independent of the past
Or: the fact that it hasn't happened yet,
tells us nothing about how much longer
it will take before it does happen
The Hashemite University 22
Exponential Distribution IV
Example: A lamp failure ~ exp( = 1/3
per hour), hence, on average, 1 failure per
3 hours.
The probability that the lamp lasts longer than
its mean life is: P(X > 3) = 1-(1-e
-3/3
) = e
-1
=
0.368
The probability that the lamp lasts between 2
to 3 hours is:
P(2 <= X <= 3) = F(3) F(2) = 0.145
The probability that it lasts for another hour
given it is operating for 2.5 hours:
P(X > 3.5 | X > 2.5) = P(X > 1) = e
-1/3
= 0.717
The Hashemite University 23
Normal Distribution I
A random variable X is normally distributed if it
has the following pdf:


Mean:
Variance:
Denoted as X ~ N(,o
2
)
Special properties:
.
f(-x)=f(+x); the pdf is symmetric about .
The maximum value of the pdf occurs at x = ; the mean
and mode are equal.
No closed form for its cdf.
0 ) ( lim and , 0 ) ( lim = =

x f x f
x x

(
(

|
.
|

\
|

= x
x
x f ,
2
1
exp
2
1
) (
2
o

t o

0
2
o
The Hashemite University 24
Normal Distribution II
Evaluating the distribution:
Use numerical methods (no closed form)
Independent of and o, using the standard
normal distribution:
Z ~ N(0,1)
Transformation of variables: let Z = (X - ) / o,
}


= u
z
t
dt e z
2 /
2
2
1
) ( where ,
t
( )
) ( ) (
2
1
) (
/ ) (
/ ) (
2 /
2
o

o
o
|
t
o

u = =
=
|
.
|

\
|

s = s =
}
}
x
x
x
z
dz z
dz e
x
Z P x X P x F
The Hashemite University 25
Normal Distribution III
Example: The time required to load an
oceangoing vessel, X, is distributed as N(12,4)
The probability that the vessel is loaded in less than 10
hours:


Using the symmetry property, u(1) is the complement of
u (-1)
1587 . 0 ) 1 (
2
12 10
) 10 ( = u =
|
.
|

\
|

u = F
The Hashemite University 26
Weibull Distribution I
A random variable X has a Weibull distribution if
its pdf has the form:





3 parameters:
Location parameter: u,
Scale parameter: | , (| > 0)
Shape parameter. o, (> 0)

>
(
(

|
|
.
|

\
|

|
|
.
|

\
|
=

otherwise , 0
, exp
) (
1
v
|
v
|
v
|
o
o o
x
x x
x f
) ( v
The Hashemite University 27
Weibull Distribution II
Example: u = 0 and o = 1:

When | = 1,
X ~ exp( = 1/o)
The Hashemite University 28
Lognormal Distribution
A random variable X has a lognormal distribution
if its pdf has the form:



Mean E(X) = e
+o
2
/2

Variance V(X) = e
2+o
2
/2 (
e
o
2

- 1)

Relationship with normal distribution
When Y ~ N(, o
2
), then X = e
Y
~ lognormal(, o
2
)
Parameters and o
2
are not the mean and variance of
the lognormal
( )

(
(

=
otherwise 0,
0 ,
2
ln
exp
2
1
) (
2
2
x

x
x
x f
=1,
o
2
=0.5,1,2.
The Hashemite University 29
Usually used to model time needed to
perform some task.
Also, it can be used to model some process
in the case where collecting actual data is
not possible.
These are the same applications of Weibull
distribution.
Lognormal Distribution II
The Hashemite University 30
A distribution whose parameters are the observed
values in a sample of data.
May be used when it is impossible or unnecessary to
establish that a random variable has any particular
theoretical parametric distribution.
So, we will use the actual data rather than fitting a
theoretical distribution to them as in the previous
described distributions.
Advantage: no assumption beyond the observed values in
the sample.
Disadvantage: sample might not cover the entire range of
possible values.
How to generate such distributions will be covered
later.
Empirical Distributions
The Hashemite University 31
The world that the simulation analyst sees is
probabilistic, not deterministic.
In this chapter:
Reviewed several important probability distributions.
Showed applications of the probability distributions in a
simulation context.
Important task in simulation modeling is the
collection and analysis of input data, e.g.,
hypothesize a distributional form for the input
data. Reader should know:
Difference between discrete, continuous, and empirical
distributions.
Poisson process and its properties.

Summary
The Hashemite University 32
Additional Notes
The lecture covers the following
sections from the textbook:
Chapter 6
Section 6.2