Probability and Random

Processes: A Refresher
ETSN01 Advanced Telecommunications

Emma Fitzgerald
2015
 Probability Random variables The Poisson Process

Probability and Random Processes: A Refresher

Probability

Random variables
Discrete random variables
Continuous random variables

The Poisson Process

Lund University Slide 2 of 35

 Probability Random variables The Poisson Process

Probability Axioms

Set of possible outcomes

Events A, B defined as any subsets of
0 P r(A) 1
P r() = 1
If A and B are mutually exclusive,
P r(A B) = P r(A) + P r(B)

Lund University Slide 3 of 35

 Probability Random variables The Poisson Process

Basic Probability Laws

Laws that follow directly from axioms

P r(A) = 1 P r(A(
P r(A B) = P r(A) + P r(B) P r(A B)
If A and B are mutually exclusive, P r(A B) = 0

Lund University Slide 4 of 35

 Probability Random variables The Poisson Process

Conditional Probability

Probability of event A when it is known that event B has

occurred:
P r(A B)
P r(A|B) =
P r(B)
Events A,B are independent if

P r(A B) = P r(A)P r(B)

If A,B are independent, then

P r(A|B) = P r(A), P r(B|A) = P r(B)

Lund University Slide 5 of 35

 Probability Random variables The Poisson Process

Random Variables

A random variable is a mapping from an outcome to a real

number
Discrete or continuous
Cumulative Distribution Function (CDF):

F (x) = P r(X x)

Example: Rolling a die: we are mapping from the way the die
lands to the number printed on the face.
What are some more examples of random variables?

Lund University Slide 6 of 35

 Probability Random variables The Poisson Process

Discrete random variables

Events can take only discrete values, e.g. integer values

Probability function pk or p(k) = P r(X = k)
Probabilities sum to 1.0
P
pk = 1.0 P
Mean: = kpk
P 2
Variance: = (k ) pk

Lund University Slide 7 of 35

 Probability Random variables The Poisson Process

Example: Rolling a die

Discrete values 1, 2, 3, 4, 5, 6
Each with probability 1/6

Lund University Slide 8 of 35

 Probability Random variables The Poisson Process

Bernoulli random variable

discrete values
1 with probability p
0 with probability 1 p

Mean = p
Variance = (1 p) p

Lund University Slide 9 of 35

 Probability Random variables The Poisson Process

Poisson random variable

Discrete values 0, 1, 2, . . .
Value k has probability pk , with

k
pk = e
k

Mean =
Variance =

Lund University Slide 10 of 35

 Probability Random variables The Poisson Process

Continuous random variables

Events can take real values on arbitrary range

Example: A persons height
What are some more examples of continuous random variables?

Lund University Slide 11 of 35

 Probability Random variables The Poisson Process

Continuous random variables

Continuous RVs have a probability density function

Probabilities integrate to 1.0
R
p(x) dx = R1.0
Mean: = xp(x) dx
R 2
Variance: = (x ) p(x) dx

Lund University Slide 12 of 35

 Probability Random variables The Poisson Process

Continuous random variables

Cumulative distribution function

P r (X x))

Lund University Slide 13 of 35

 Probability Random variables The Poisson Process

Negative exponential random variable

Parameter
p(x) = ex
P r (X x) = 1 ex
Probability density function ( = 2)

Mean = 1
 2
Variance = 1
Cumulative distribution function ( = 2)

Standard deviation = 1

Lund University Slide 14 of 35

 Probability Random variables The Poisson Process

Gaussian random variable

Parameters ,

1 (x)2
p(x) = e 22
2
Probability density function ( = 0, = 1)

Mean =
Cumulative distribution function ( = 0,
Variance = 2
= 1)
Standard deviation =

Lund University Slide 15 of 35

 Probability Random variables The Poisson Process

Multiple random variables

X, Y are random variables

Joint CDF of X, Y

F (x, y) = P r (X x, Y y)

if X, Y discrete: joint probability function

P (x, y) = P r (X = x, Y = y)
if X, Y continuous: joint density function
X, Y are independent iff F (x, y) = F (x)F (y)

Lund University Slide 16 of 35

 Probability Random variables The Poisson Process

Some handy properties

For any random variable X, constants a, b:

E[aX + b] = aE[X] + b
V ar[aX + b] = a2 V ar[X]
For any random variables X, Y :
E[X + Y ] = E[X] + E[Y ]
For any independent random variables X, Y :
E[XY ] = E[X]E[Y ]
V ar[X + Y ] = V ar[X] + V ar[Y ]

Lund University Slide 17 of 35

 Probability Random variables The Poisson Process

Correlation and covariance

Covariance:
Cov[X, Y ] = E[(XE[X])(Y E[Y ])] = E[XY ]E[X]E[Y ]

Compare with variance:

V ar[X] = E[(X E[X])2] = E[X 2 ] E[X]2
Can be positive, negative, or zero
Correlation coefficient:

Cov[X, Y ]
r(X, Y ) =
x y

1 r(X, Y ) 1
X, Y are called uncorrelated if r(X, Y ) = 0
if X, Y are independent, then they are uncorrelated
if X, Y are uncorrelated, they may still not be independent
Lund University Slide 18 of 35
 Probability Random variables The Poisson Process

Correlation and covariance: example

X is a normally distributed (Gaussian) random variable with = 0

Y = X2
X and Y are:
A Correlated and dependent
B Correlated but independent
C Uncorrelated but dependent
D Uncorrelated and independent
E Confusing

Lund University Slide 19 of 35

 Probability Random variables The Poisson Process

Correlation and covariance: example

Answer: C: uncorrelated but dependent. Y is completely

determined by X so they are not independent, but

Cov(X, Y ) = E[XY ] E[X]E[Y ]

= E[X 3 ] 0
=0

so X and Y are uncorrelated.

This is because correlation is only a measure of linear dependence
it does not say anything about nonlinear dependence!

Lund University Slide 20 of 35

 Probability Random variables The Poisson Process

Stochastic processes

A stochastic process (or random process) x(t) is a dual entity:

for every time instant, x(t) is a random variable

for every point (sample) in outcome space , x(t) is a
real-valued function of time
Random processes can be
discrete-time or continuous-time
discrete-value or continuous-value

Lund University Slide 21 of 35

 Probability Random variables The Poisson Process

Stationary processes

For a random process x(t):

autocorrelation function R(t1 , t2 ) = E[x(t1 )x(t2 )]
autocovariance function C(t1 , t2 ) = Cov[x(t1 ), x(t2 )]
x(t) is called stationary (in the weak sense) if:
E[x(t)] is a constant (independent of t)
the autocorrelation R(t1 , t2 ) (or autocovariance C(t1 , t2 )) is a
function of the difference t2 t1 only

Lund University Slide 22 of 35

 Probability Random variables The Poisson Process

The Poisson process

Continuous-time, discrete-value process

Very commonly used to describe arrivals into a queueing
system or network
Characterized by a rate parameter (e.g. packets/second)
Three equivalent viewpoints (definitions) of the Poisson
process

Lund University Slide 23 of 35

 Probability Random variables The Poisson Process

First viewpoint

Behaviour in small time interval

Approximately Bernoulli distribution
1 event with probability t + o(t) (approx. t for small
t)
0 events with probability 1 t + o(t) (approx. 1 t
for small t)
Probabilities are independent between non-overlapping
intervals

Lund University Slide 24 of 35

 Probability Random variables The Poisson Process

Second viewpoint

Behaviour over a long time interval

Poisson distribution

Lund University Slide 25 of 35

 Probability Random variables The Poisson Process

Third viewpoint

Behaviour between events

Negative exponential distribution

Lund University Slide 26 of 35

 Probability Random variables The Poisson Process

Three equivalent viewpoints

1. P r(an arrival in time(t, t + t) = t + o(t) and arrivals

are memoryless, i.e. independent of what has happened
before.
k
2. P r(k arrivals in time t) = et (t)
k! i.e. a Poisson
distribution, with parameter t.
3. The probability density function of the times between events
(the interarrival times) is negative exponential, with parameter
, i.e pT (t) = et .

Lund University Slide 27 of 35

 Probability Random variables The Poisson Process

Why are the definitions equivalent?

Proof that definitions of Poisson process are equivalent

Will show:
a Poisson process according to definition 1 satisfies definition 2
a Poisson process according to definition 2 satisfies definition 3
a Poisson process according to definition 3 satisfies definition 1

Lund University Slide 28 of 35

 Probability Random variables The Poisson Process

1 = 2

Find the probability of k arrivals in time t:

Divide period t into N small periods of t each
will later take N , t 0
Approximately sum of N independent Bernoulli random
variables with probability t/N each
Probability that k out of N will be 1 and the rest 0:

t N k
 k  
N! t
1
k! (N k) N N

Lund University Slide 29 of 35

 Probability Random variables The Poisson Process

1 = 2 (cont.)

t N k
 k  
N! t
1
k! (N k) N N
When N is large:

N! t N t k
   
Nk 1 et 1 1
(N k) N N

so when N , we obtain

(t)k
P r(k arrivals in time t) = et
k!

Lund University Slide 30 of 35

 Probability Random variables The Poisson Process

2 = 3

k
P r(k arrivals in time t) = et (t)
k!

Probability that time between arrivals is more than t =

Probability of 0 arrivals during time t = et
(using formula in definition 2 for k = 0)

so, exponentially distributed as required

Lund University Slide 31 of 35

 Probability Random variables The Poisson Process

3 = 1

Suppose last arrival was time t ago:

a priori probability that next arrival comes after more than t from
previous one = et
a priori probability it is after more than t + t = e(t+t)
a priori probability that it happens within the next
t = et e(t+t)
Conditioned on knowing that no other arrivals in last t:

et e(t+t) t (t)2 (t)3

= 1 e t + ...
et 2 6
which is independent of t, so satisfies the requirement of
independence from previous arrivals

Lund University Slide 32 of 35

 Probability Random variables The Poisson Process

Sums of Poisson processes

Consider superposition of several Poisson processes

m independent Poisson Processes, rates i , i = 1, 2, ..., m
P
Sum is also a Poisson process, rate = i

Lund University Slide 33 of 35

 Probability Random variables The Poisson Process

Sums of Poisson processes (cont.)

Lund University Slide 34 of 35

 Probability Random variables The Poisson Process

Packets arrive at a router according to a Poisson process, i.e.

number of packets arrived after time t = x(t), with
k
P r(x(t) = k) = et (t)
k!

Should x(t) be discrete or continuous? Should time be discrete

or continuous?

Lund University Slide 35 of 35