PROBABILITY DISTRIBUTIONS

5.1 Introduction
In this chapter we will discuss some probability distributions of random variables.
There are discrete and continuous probability distributions.

5.2 Random Variables

Definition 5.1 A random variable is the function which is defined over the elements
in the sample space, S. Random variables are denoted by capital letters X, Y, and so
on, and their corresponding values are denoted by small letters x , y , etc.
Random variables are classified into two groups, discrete random variables and
continuous random variables.

5.2.1 Discrete Random Variables

A discrete random variable is the one which can assume only integer values of
elements in the sample space.
Examples of discrete random variables include
(a) Number of cars passed at a certain zebra marking
(b) Number of possible outcomes in tossing a fair coin or dice twice etc.
Definition 5.2 If X is a discrete random variable, the function given by
f x P X x for each x within the range of X is called the probability distribution
of X.
Probability distributions of discrete random variables are commonly represented in
tabular form, where values of X with their corresponding probabilities are shown.
Sometimes they are given in the form of formulae
Theorem 5.1 The function f x can serve as a probability distribution of a discrete
random variable X if and only if
1. f x 0 for each x X
2. () = 1

Example 5.1 Find the formula for the probability distribution of a total number of
heads obtained by tossing a fair coin three times.

Solution
The following tree diagram is used to obtain the sample space S

H
T
H

HHH
HHT
HTH

T
H

HTT
THH

T
H

THT
TTH

TTT

Therefore S HHH, HHT, HTH, HTT , THH , THT , TTH , TTT

By letting X = number of heads shown up, we find that X can take values 0, 1, 2 or 3.
Then the probability distribution of X in this case is given as follows:
X
P(X)

0
1/8

1
3/8

2
3/8

3
1/8

Hence the formula for the probability distribution is given by

1 3
f x , x 0,1, 2, 3
8 x

5.2.2 Continuous Random Variables

A continuous random variable is the one which assume real numbers within the
specified interval I . The formula which gives the probabilities for a continuous
random variable is called a probability density function (p.d.f) for short or simply a
density.
Definition 5.3 A function with values f x defined over the set of all real numbers is
called a p.d.f. of the continuous random variable X if and only if

Pa X b f x dx
b

Theorem 5.2 A function f x can serve as a probability density of a continuous

random variable X if the following conditions are satisfied
1. f x 0 , x

2.

f x dx 1

Some special continuous random variables will be discussed in the next chapter.

5.3 Expected Values and the Standard deviations

The concepts are expected values and the standard deviation play an important role in
statistical analyses and decision making. While expected value measures the
expectation, the standard deviation measures the risk associated with that expectation.
Note that variances and/or standard deviations are measures of risks in business and
financial projects.
Definition 5.4 If X is a discrete random variable, then its expected value is given by

EX

x p x
all x

Definition 5.5 If X is a discrete random variable, then its variance is given by

Var X Ex E X E X 2 E X
2

x px

where, E X 2

all x

The standard deviation of X is therefore given by

SD X Var X
Example 5.2 The five days incomes of a certain firm in thousands of dollars with
their associated probabilities were given as follows:
Income
Probability

1.2
0.3

3.3
0.15

1.8
0.2

0.9
0.15

2.8
0.2

Find the expected income and the standard deviation of the firm.
Solution
Let X be a daily income of the firm. We summarize the required sums in the following
table
x
p(x)
x p(x)
x2p(x)
1.2
0.30
0.36
0.432
3.3
0.15
0.495
1.6335
1.8
0.20
0.36
0.648
0.9
0.15
0.135
0.1215

2.8
SUM

0.20

0.56
1.91

1.568
4.403

Then,

E X xpx 1.91

Var X E X 2 E X 4.403 1.912 0.7549

2

SD X Var X 0.7549 0.8688

Therefore the expected daily income is $1.91 (000) and the standard deviation is
$0.8688 (000).

5.4 Joint Probability Distributions

The joint distributions are distributions joining two or more random variables. They
are used to determine how random variables are jointly related using different
categories. There are discrete joint probability distributions and continuous joint
probability distributions.

5.4.1 Discrete Joint Probability Distributions

Definition 5.6 If X and Y are discrete random variables, the function given by
f x, y P X x, Y y for each pair of values x, y within the range of X and Y is
called the joint probability distribution of X and Y
Theorem 5.3 Two discrete random variables (say X and Y) are said to be jointly
distributed if the following conditions are satisfied
1. f x, y 0
2.

f x, y 1
x

Example 5.3 Find the value of k if the following is a joint probability distribution

for x 1, 2,3; y 2, 3
otherwise

kxy
f x, y
0
Solution

Given that the function is a joint probability distribution, we have

f x, y 1 .
x

It implies that,

f 1,2 f 1,3 f 2,2 f 2,3 f 3,2 f 3,3 1

2k 3k 4k 6k 6k 9k 1

So that 30k 1 or k

1
30

5.4.2 Marginal Discrete Probability Distributions

These are distributions obtained by considering only some of the categories
(variable) from the joint distribution. For the case of two variables, a marginal
distribution is a function of one variable obtained by summing up all probabilities of
the other variable.
Definition 5.7 If X and Y are discrete random variables and f x, y is the value of
their joint probability distribution at x, y , then the function given by
g x f x, y
y

for each x within the range of X is called the marginal distribution of X, similarly,
the function

h y f x, y
x

for each y within the range of Y is called marginal distribution of Y.

5.5 Covariance
Covariance is a measure of how two random variables change together. If one
random variable varies directly to the other, we get a positive covariance. However,
if they vary inversely, they give a negative covariance. We can precisely define the
covariance as follows;
Definition 5.8 If X and Y are two random variables, their covariance denoted by
COV X , Y or X , Y is given by

COV X , Y E X E X Y EY E XY E X EY
Where E XY is a joint expectation, and E X and E Y are marginal expectations
of X and Y respectively.
For discrete random variables we define E XY as

E XY xyf x, y
x

5.6 Correlation

In similar fashion as the case of non-random variables, the strength of a linear

relationship if it does exist is measured by the coefficient of correlation.
Definition 5.9 If X and Y are two random variables, their correlation coefficient
denoted by CORR X , Y or or r is given by
(, ) =

(, )
()()

5.7 Independence
Definition 5.10 Two jointly distributed random variables are said to be linearly
independent if
E XY E X EY
It follows from the definition that two random variables are said to be linearly
independent if their covariance is equal to zero.
In general, the term independence implies that two or more random variables do not
have any kind of relationship. i.e. neither linear nor non-linear relationships.
Example 5.4 Two cash flows are jointly distributed as shown below

2
10
20
25

10
0.10
0.10
0.10

1
15
0.10
0.20
0.10

20
0.15
0.10
0.05

(a) Find marginal distribution of 1 and hence expected value and standard
deviation.
(b) Find marginal distribution of 2 and hence expected value and standard
deviation.
(c) Compute the covariance, correlation coefficient between these cash flows
and comment on the results.
(d) Are 1 and 2 linearly independent?
Solution
(a) Marginal distribution of 1 is given below
1
Prob.

10
0.3

15
0.4

20
0.3

Then,
(1 ) = 1 (1 ) = 10(0.3) + 15(0.4) + 20(0.3) = 3 + 6 + 6 = 15
(1 ) = (12 ) ((1 ))

But,
(12 ) = 12 (1 ) = 102 (0.3) + 152 (0.4) + 202 (0.3)
= 30 + 90 + 120 = 240
It follows that
(1 ) = 240 152 = 240 225 = 15
Therefore, (1 ) = (1 ) = 15 = 3.873
(b) Marginal distribution of 2 is given below
2
Prob.

10
0.35

20
0.4

25
0.25

Then,
(2 ) = 2 (2 ) = 10(0.35) + 20(0.4) + 25(0.25)
= 3.5 + 8 + 6.25 = 17.75
2

(2 ) = (22 ) ((2 ))
But,
(22 ) = 22 (2 ) = 102 (0.35) + 202 (0.4) + 252 (0.25)
= 35 + 160 + 156.25 = 351.25
It follows that
(2 ) = 351.25 17.752 = 36.1875
Therefore, (2 ) = (2 ) = 36.1875 = 6.016
(c) Covariance between the two cash flows is given by
(1 , 2 ) = (1 2 ) (1 )(2 )
First we need to find the distribution of the product 1 2 for combination of
these two random variables. This is shown below
1 2

100

150

200

200

300

400

250

375

500

Prob.

0.10

0.10

0.15

0.10

0.20

0.10

0.10

0.10

0.05

Then,

(1 2 ) = 1 2 (1 , 2 )
= 100(0.10) + 150(0.10) + 200(0.15) + 200(0.10) + 300(0.20)
+ 400(0.10) + 250(0.10) + 375(0.10) + 500(0.05)
= 10 + 15 + 30 + 20 + 60 + 40 + 25 + 37.5 + 25
= 262.5
It follows that
(1 , 2 ) = (1 2 ) (1 )(2 ) = 262.5 (15)(17.75) = 3.75
This indicates an inverse relationship between the cash flows
Correlation coefficient is given by
(1 , 2 )
3.75
(1 , 2 ) =
=
= 0.161
(3.873)(6.016)
(1 )(2 )
This indicating weak inverse correlation between the cash flows.
(d) Since the covariance between these cash flows is not zero, the cash flows are
not linearly independent.

5.8 Linear Combinations of Random Variables

It appears frequently in statistics that a given random variable is expressed a linear
combination of two or more random variables. The expected value and the standard
deviation of such random variables are also interested.
Definition 5.11 If X 1 , X 2 ,, X n are random variables and a1 , a2 ,, an are
constants, then the sum
n

Y ai X i
i 1

is called the linear combination of random variables

Theorem 5.4 If X 1 , X 2 ,, X n are random variables with a linear combination
n

Y ai X i
i 1

where a1 , a2 ,, an are constants, then

E Y ai E X i and Var Y ai2Var X i 2 ai a j CovX i , X j
n

i 1

i 1

i j i j

Corollary 5.1 If X 1 , X 2 ,, X n are independent random variables such that

n

Y ai X i
i 1

where a1 , a2 ,, an are constants, then

Var Y ai2Var X i
i 1

By considering only two random variables, we have

Y aX bY

with EY aE X bEY and Var Y a 2Var X b 2Var Y 2abCov X , Y

For three random variables, a linear combination becomes

Y a1 X 1 a2 X 2 a3 X 3
The expected value of Y is given by
EY a1 E X 1 a2 E X 2 a3 E X 3
The variance of Y is given by

Var Y a12Var X 1 a 22Var X 2 a32Var X 3 2a1a 2 Cov X 1 , X 2

2a1a3Cov X 1 , X 3 2a 2 a3Cov X 2 , X 3

If we three or more random variables, it is more convenient to use matrix approach

to compute mean and variance of a linear combination.
If we denote the constants and expected values by column matrices as

a1
EX 1

a and E X
a
E X
n
n

Then the expected value of the linear combination

n

Y ai X i is given by E Y a' E X a1
i 1

EX 1

a n
E X
n

Similarly, by defining a variance covariance matrix by

Cov X 1 , X 2
Var X 1

Var X 2
Cov X 2 , X 1
M

Cov X , X Cov X , X
n
1
n
2

Cov X 1 , X n

Cov X 2 , X n

Var X n

The matrix M is an n n and symmetric, such that

CovX i , X j CovX j , X i , i j
The diagonal elements are variances and off-diagonal elements are covariances.
Under this presentation, the variance of a linear combination is given by

Var Y a' M a a1

Cov X 1 , X 2
Var X 1

Var X 2
Cov X 2 , X 1
a n

Cov X , X Cov X , X
n
1
n
2

Cov X 1 , X n
a1
Cov X 2 , X n

a n

Var X n

Example 5.5 Given the random variables X, Y, and Z with the following expected
2
2
2
values x 2 , y 5 , z 2 , variances x 1, y 4, z 2 and co-variances

xy 3, xz 4 , yz 2 . Find the expected value and variance of a random

variable W 2 X 3Y Z .
Solution
Form the linear combination, we have constants a 2, b 3 , c 1
Putting these information in matrices, we have

4
2
2
1 3

a 3 , E X 5 and M 3 4 2
1
2
4 2 2

The expected value of W is given by

2

E W a' E X 2 3 1 5 4 15 2 21.
2

The variance of W is given by

4 2
1 3

Var W a ' M a 2 3 1 3 4 2 3
4 2 2 1

294

2 3 1 6 12 2
862

7

2 3 1 20 14 60 0 74
0