3.

Random Variables, Cumulative Distribution Functions, and Density Functions

3.1 Random Variables
Outcomes of random experiments need to be represented numerically so that we can have
some ideas on averages and variances of outcomes. To that extent, random variables
become a powerful tool to go from a set of outcomes, numerical or otherwise, to a set of
numbers. A random variable is defined a real function that maps the outcomes of an
experiment on to a set of numbers. This concept is shown in Figure 3.1. We use the
CAPITAL letters to represent the random variable. For example, for the coin toss
experiment, we can create a random variable X such that {X=1} represents heads and
X={0} represents tails. For the case of the roll of a die, {X=1}, {X=2}, will constitute
the events associated with the random variable X.

Figure 3.1 Concept of a random variable: For each outcome s, the function X(s)
assigns a number. X is the random variable and x is the value it takes.
We can now identify {X=a} or {X<a} etc. as events. Since these are events, they have
probabilities associated with them. The probability that the random variable X takes a
value less than or equal to x is referred to as the cumulative distribution function of the
random variable and is represented by FX(x), i.e.,

FX ( x ) = P{X x}.

(3.1)

The cumulative distribution function has the following properties:

FX ( ) = P{X } = 0

(3.2 a)

FX ( ) = P{X } = 1

(3.2b)

0 FX ( x ) 1
P{x1 X x2 } = FX ( x2 ) FX ( x1 )

(3.2c)
(3.2d)

The rate of change of the cumulative distribution function is referred to as the probability
density function (pdf), fX(x),
d [F ( x )] .
f X ( x ) = dx
X

(3.3)

The pdf has the following properties:

f X (x ) 0
FX ( x ) =
1=

(3.4a)

f X ( )d

(3.4b)

f X (x ) dx

(3.4c)

x2

x1

x2

P{x1 X x2 } = f X ( )d f X ( )d = f X ( )d .

(3.4d)

x1

If the outcomes are discrete, we have a discrete random variable and if the outcomes are
continuous, we have a continuous random variable. For example, the random variable
formed by the roll of a die is a discrete random variable while the one formed by
observing the voltage developed across a resistor or the strain induced by an applied
force, is a continuous random variable. The expressions for the CDF and pdf of a discrete
random variable with N outcomes can be expressed as
N

FX ( x ) = P ( X = xi )U ( x xi )

(3.5)

i =1

and
N

f X (x ) = P( X
i =1

= xi ) ( x xi )

where U() is the unit step function given by

(3.6)

1, x 0
U (x ) =
.
0
,
x
0
<

(3.7)

Example 3.1
1. A coin is tossed three times. Demonstrate how you will create a random variable that
can represent the outcomes of this experiment.
Answer: Let the random number X represent the heads. The outcomes (events) in this
experiment are HHH HHT HTH THH TTH THT HTT TTT. Since the random number
represents the heads, the events and the corresponding values of the random numbers can
be written as
s
X(s) = x

HHH HHT HTH THH TTH

3
2
2
2
1

THT
1

HTT
1

TTT
0

X thus is a random variable taking values of {0, 1, 2, and 3}. By observation, we have
P{X=3}=P{X=0}=1/8 and P{X=1}=P{X=2}=3/8.
2. Obtain and plot the cumulative distribution function and probability density function of
the random variable in the previous example.
FX(-) = P(X<- ) = 0
FX(0) =P{X<0}=P{X=0}=1/8
FX(1) = P{X<1}=P{X=0}+P{X=1}=4/8=1/2
FX(2) = P{X<2}= P{X=0}+P{X=1}+ P{X=2}=7/8
FX(3) = P{X<3}= P{X=0}+P{X=1}+ P{X=2}+P P{X=2}=8/8
FX() =1

FX ( x ) = 81 U ( x ) + 83 U ( x 1) + 83 U ( x 2 ) + 81 U ( x 3 )

f (x)

0.5

1
)
x(
X
F

0.4
(3/8)

(3/8)

0.9

(7/8)

0.8
0.7

0.3

0.6
(4/8)

0.5
0.2

0.4
(1/8)

0.1

0.3

(1/8)

0.2

0.1
0

2
x

2
x

Figure 3.2 The density function f(x) and the distribution function F(x) in Example 3.1 are shown.

The probability density function is obtained by differentiating and is shown in Figure b.

f X ( x ) = 81 ( x ) + 83 ( x 1) + 83 ( x 2 ) + 81 ( x 3)
Some observations regarding CDF and pdf:
1. If X is a continuous random variable, P{X= x} = 0.
Consider P{a < x< b} where a = x and b = x+x. Using the properties of the CDF, P{a <
x< b} = F(b)-F(a) = F(x+x)-F(x)=0 when x approaches zero.
F(x+x)-F(x) = P{X= x}=0. In other words, if a random variable is continuous, the
probability that it takes a fixed value is zero. This also means that P{X<a}= P{X<a} and
P{X>b}=P{X>b}.
2. P{x < x < x+x } = f(x) x.
A case of a continuous random variable taking values greater than 1 is shown in Figure.
For a random variable taking only positive values, F(0)=0 and f(x) does not exist for
negative values of x.
3. A discrete random variable is one whose cumulative distribution function (cdf) is right
continuous, staircase function of x, with jumps at a discrete set of points x0, x1, x2, A
continuous random variable is one whose cdf is continuous everywhere.
4. In addition to continuous and discrete random variables, we may also have mixed
random variables. A typical density function of a mixed random variable may look like

f x ( x ) = k1 ( x x1 ) + g ( x )
where g(x) is a continuous function of x. Such a random variable is created normally in a
two step process. Consider a case where we are measuring a voltage across a resistor that
is varying from positive to negative values. If we assume that the negative values are as
good (or as bad as a zero voltage), we can create a new random variable that takes on a
discrete value of 0 with a probability that voltage values are less than or equal to zero and
a continuous one whenever the voltage is positive.
A typical pdf, corresponding CDF and various probabilities are shown in Figure 3.3.

1.2

1.0

0.8

F(2)=0.59
)
x(
0.4
X
F

3
x

0.5
)
x(
fX

0.5

0.4

0.4

P(X 2)=F(2)

0.3

0.3

0.2

0.2

0.1

0.1

0.5

0.5

0.4

0.4

0.3

P(2 X 3)=F(3)-F(2)

F()=1

0.3
P(X 3)=1- F(3)

0.2

0.2

0.1
0

0.1
0

Figure 3.3 CDF, pdf, and various probabilities are shown.

3.2 Statistical Averages and Moments

The statistical average or mean of a random variable, denoted by , is given by

M
= E X = xf x dx = xk P X = xk .
k =1

The mean (also known as expected value) is a weighted average of the possible
outcomes; the weights being the probabilities of the outcomes. The concept of the
average is also analogous to the concept of center of gravity with probabilities acting as
masses and the possible outcome values (x) acting as locations.

[ ]

( )

In addition to the expected value of the random variable, we are also interested in
knowing the spread of the values (around the mean) that we are likely to observe when an
experiment is conducted. Since the spread could be on either side of the mean, we
normally look at the squared value, which is calculated as the weighted average of

[X ]2 . This is known as the variance denoted by x2 and is expressed as

x2

= ( x ) f ( x )dx = ( xk )2 P( X = xk ) .
2

k =1

The deviation from the mean is the standard deviation given by . If the standard
deviation is high, we expect significant fluctuations in the observed outcomes while small
values of signify a very tight set of observed outcome values.

We defined the mean and variance by taking the expected values of the random variable
or its power (second). We can define the expected value of a function of a random
variable, g(X), as the weighted average of that function in a fashion similar to the
definition of the mean and variance.

E [g ( X )] = g ( x ) f ( x )dx = g ( x )P( X = xk )

k =1

We are now in a position to define the nth order moments, namely the weighted average
of the powers of X. The nth moment of a random variable is given by

[ ]=

EX

n
x f ( x )dx = ( xk ) P( X = xk ) .
n

k =1

We can also define the nth central moments as

k =1

E ( X )n = ( x )n f ( x )dx = ( xk )n P( X = xk )
Meaning of the moments:
The first moment (n=1) is the mean. The second moment (n=2) is the mean square value.
2
Note that is known as the square of the mean. The second central moment is the
variance. The standard deviation is the square root of the variance and is a measure of the
spread, width or dispersion of the random variable. The third moment is a measure of
the symmetry of the density function and is considered to be a measure of skewness of
the distribution. The skewness cs is defined as

cs =

E ( X )3

If the density function is symmetric around the mean, its odd ordered central moments
will be zero and thus will its skewness is zero. (Prove?)
The fourth central moment is measure of the peakedness of the density function around
the mean and measures the kurtosis. The coefficient of Kurtosis ck is given by

ck =

E (X )

The value of the random variable for which the density function has the peak value is
called the mode of the random variable. Note that some of the random variables such as
uniform have constant values for the density function and mode is not well-defined in
that case. The median is that value of the random variable at which the cumulative
distribution function is equal to 0.5. In other words, median value divides the density
function into two of equal areas, each equal to 0.5. The median, mean and mode are
shown in the figure 3.4 for the case of a Rayleigh distributed random variable.

fX(x)

1.4
no symmetry
skewness 0

pdf 1

0.3

1.2

line of symmetry
(zero skewness)
1

pdf 2
n
oi
t
c
n
uf
yt
si
n
e
d

0.2

0.8

0.6

0.4

0.1
0.2

mode (2.15)

mean (3.35)

10

5
x

median (2.87)

Figure 3.4

3.3 Examples Distribution and Density functions

1. Uniform Distribution:
This distribution arises when we are interested in studying engineering problems dealing
with cases where the outcomes are equally likely. For example, we are asked to pick a
number in the range of 0 to 5. Note that this is a continuous random variable and hence
we can see the probability that the number is equal to a specific value is zero. However,
we can consider the case where this number lies in the range of 2 and 2.5. Now consider
another set that lies in the range of 3.7 to 4.2. For a uniformly distributed random
variable, these probabilities are equal; they are determined by the range of values and not
by the values themselves. Another example is the case of a pointer turning on a wheel.
The probability that it will be come to rest at a range of angles between 45 and 55 is the
same as the probability that it will come to rest in the range of 205 to 215. Note that the
whole range is 360 degrees. In the first case, we say that the random variable is uniformly
distributed in the range 0 to 5 while for the latter it is uniform in the range 0 to 360 or 0
to 2 rad. In general, if a random variable is uniform in the range a to b, the density
function f(x) is

10

 b 1 a a x b
f (x ) =
0 otherwise
and the corresponding distribution function F(x) is given by

0 x < 0

F ( x ) = bx aa a x b

x>b
1

1.2

uniform cdf (a=2; b=4)

0.9
uniform pdf (a=2; b=4)

0.8

0.7
)
x(
fX

)
x(
X
F

0.6
0.5

0.8

0.6

0.4

0.4

0.3
0.2

0.2

0.1
0

Mean = 1
2

3
x

3
x

(a + b ) ; variance = (b12a )

2. Exponential distribution
Exponential is used to model the lifetime of electronic components, decay of energy,
fluctuations in power observed in radar systems, etc. The density function f(x) is given by

( )

f ( x ) = a1 exp ax U ( x ) .
The distribution function is given by

x<0

0
F (x ) =
x
1 exp a

( )

x0

0.7

1.2
exponential cdf (a=1.5)

0.6

exponential pdf (a=1.5)

0.5
)
x(
X
F

)
x(
f X 0.4

0.8

0.6

0.3
0.4

0.2
0.2

0.1
0

3
x

3
x

mean = a; variance =a2

3. Rayleigh distribution
Rayleigh density function used to represent the lifetime of components. It also is used to
describe the fluctuation in the received signal envelope seen in radar systems.
The density function is given by
2
f x = x2 exp x 2 U x .
a
2a

( )

( )

The distribution function is given by

0
F (x ) =
1 exp

x2
2a 2

x<0
x0
1.2
Rayleigh cdf (a=1.1)
1

0.8

)
x( 0.6
X
F
0.4

0.2

0.7
0.6

Rayleigh pdf (a=1.1)

0.5
0.4
)
x(
f X 0.3
0.2
0.1
0

3
x

4. Gaussian or Normal distribution

Gaussian distribution is used to model a number of naturally occurring phenomena such
as the noise observed in electrical circuits, characteristics of biological population such as
weights and heights of students, the statistical fluctuations observed during the
manufacturing of products (the diameter of a precision nut, for example), etc. It also used
to model a phenomenon that is being generated from the contribution from a number of
other phenomena. The density function is given by

f X (x ) =

exp

( x )2

2
2 2
and the cumulative distribution function is given by

FX (x ) =

1
2 2

exp 21 2 ( y ) dy .
2

0.2

1.2
Gaussian pdf

0.18
0.16
)
x(
fX

Gaussian CDF

mean = -1
standard dev. = 2

0.14

mean = -1
standard dev. = 3

0.12

0.8

mean = -1
standard dev. = 3

)
x(
X
F 0.6

0.1
0.08

0.4

0.06
0.04

mean = -1
standard dev. = 2

0.2

0.02
0
-8

-6

-4
x

-2

mean = -1

0
-8

-6

-4

-2

0
x

Note that the Gaussian CDF does not have a closed form solution. The probability values
can however be calculated from a Table containing the values of the function, (z). The
function (z) is given by

(z ) =

exp

1
2

with z =

2
2

This means that (z) is the probability that the random variable Z (Gaussian with a mean
of 0 and standard deviation 1) is less than z.
Example: If X is Gaussian with a mean of 2 and a standard deviation of 1.5. What is the
probability that X is less than 5?
Answer: Step 1: Get the value of z =

5 mean
std . dev.

5 2
1.5

= 2.

Use the Table to get the value of (z) at z = 2.

The probability is shown by the shaded areas, which are equal. The advantage of (z)
results from what can be referred to as a standardized variable, resulting from creating a
new Gaussian random variable of zero mean and unit variance.
0.35
0.5
Gaussian with a mean of 0
and standard deviation of 1.5

Gaussian with a mean of 2 and

standard deviation of 1.5

0.3

P(X 5)

0.4

0.25

P(X 5) = P(Z 2) = (z=2)

0.2

0.3

)
x(
f

)
z(
f

0.15

0.2

0.1
0.1

0
-5

0.05

-4

-3

-2

-1

2
z

Properties of (z):
(0)=0.5;
(-z)= (z);

0
-8

-6

-4

-2

4
x

10

z
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
3.0
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9

0
0.5
0.5398
0.5793
0.6179
0.6554
0.6915
0.7257
0.758
0.7881
0.8159
0.8413
0.8643
0.8849
0.9032
0.9192
0.9332
0.9452
0.9554
0.9641
0.9713
0.9772
0.9821
0.9861
0.9893
0.9918
0.9938
0.9953
0.9965
0.9974
0.9981
0.9987
0.999
0.9993
0.9995
0.9997
0.9998
0.9998
0.9999
0.9999
1

1
0.504
0.5438
0.5832
0.6217
0.6591
0.695
0.7291
0.7611
0.791
0.8186
0.8438
0.8665
0.8869
0.9049
0.9207
0.9345
0.9463
0.9564
0.9649
0.9719
0.9778
0.9826
0.9864
0.9896
0.992
0.994
0.9955
0.9966
0.9975
0.9982
0.9987
0.9991
0.9993
0.9995
0.9997
0.9998
0.9998
0.9999
0.9999
1

2
0.508
0.5478
0.5871
0.6255
0.6628
0.6985
0.7324
0.7642
0.7939
0.8212
0.8461
0.8686
0.8888
0.9066
0.9222
0.9357
0.9474
0.9573
0.9656
0.9726
0.9783
0.983
0.9868
0.9898
0.9922
0.9941
0.9956
0.9967
0.9976
0.9982
0.9987
0.9991
0.9994
0.9995
0.9997
0.9998
0.9999
0.9999
0.9999
1

3
0.512
0.5517
0.591
0.6293
0.6664
0.7019
0.7357
0.7673
0.7967
0.8238
0.8485
0.8708
0.8907
0.9082
0.9236
0.937
0.9484
0.9582
0.9664
0.9732
0.9788
0.9834
0.9871
0.9901
0.9925
0.9943
0.9957
0.9968
0.9977
0.9983
0.9988
0.9991
0.9994
0.9996
0.9997
0.9998
0.9999
0.9999
0.9999
1

4
0.516
0.5557
0.5948
0.6331
0.67
0.7054
0.7389
0.7704
0.7995
0.8264
0.8508
0.8729
0.8925
0.9099
0.9251
0.9382
0.9495
0.9591
0.9671
0.9738
0.9793
0.9838
0.9875
0.9904
0.9927
0.9945
0.9959
0.9969
0.9977
0.9984
0.9988
0.9992
0.9994
0.9996
0.9997
0.9998
0.9999
0.9999
0.9999
1

5
0.5199
0.5596
0.5987
0.6368
0.6736
0.7088
0.7422
0.7734
0.8023
0.8289
0.8531
0.8749
0.8944
0.9115
0.9265
0.9394
0.9505
0.9599
0.9678
0.9744
0.9798
0.9842
0.9878
0.9906
0.9929
0.9946
0.996
0.997
0.9978
0.9984
0.9989
0.9992
0.9994
0.9996
0.9997
0.9998
0.9999
0.9999
0.9999
1

6
0.5239
0.5636
0.6026
0.6406
0.6772
0.7123
0.7454
0.7764
0.8051
0.8315
0.8554
0.877
0.8962
0.9131
0.9279
0.9406
0.9515
0.9608
0.9686
0.975
0.9803
0.9846
0.9881
0.9909
0.9931
0.9948
0.9961
0.9971
0.9979
0.9985
0.9989
0.9992
0.9994
0.9996
0.9997
0.9998
0.9999
0.9999
0.9999
1

7
0.5279
0.5675
0.6064
0.6443
0.6808
0.7157
0.7486
0.7794
0.8078
0.834
0.8577
0.879
0.898
0.9147
0.9292
0.9418
0.9525
0.9616
0.9693
0.9756
0.9808
0.985
0.9884
0.9911
0.9932
0.9949
0.9962
0.9972
0.9979
0.9985
0.9989
0.9992
0.9995
0.9996
0.9997
0.9998
0.9999
0.9999
0.9999
1

8
0.5319
0.5714
0.6103
0.648
0.6844
0.719
0.7517
0.7823
0.8106
0.8365
0.8599
0.881
0.8997
0.9162
0.9306
0.9429
0.9535
0.9625
0.9699
0.9761
0.9812
0.9854
0.9887
0.9913
0.9934
0.9951
0.9963
0.9973
0.998
0.9986
0.999
0.9993
0.9995
0.9996
0.9997
0.9998
0.9999
0.9999
0.9999
1

9
0.5359
0.5753
0.6141
0.6517
0.6879
0.7224
0.7549
0.7852
0.8133
0.8389
0.8621
0.883
0.9015
0.9177
0.9319
0.9441
0.9545
0.9633
0.9706
0.9767
0.9817
0.9857
0.989
0.9916
0.9936
0.9952
0.9964
0.9974
0.9981
0.9986
0.999
0.9993
0.9995
0.9997
0.9998
0.9998
0.9999
0.9999
0.9999
1

5. Lognormal distribution
Lognormal density function arises in situations where the quantity being observed is a
result of the product of a number of quantities. An example would be the sound that will
be measured at a location assuming that it has been reflected from a number of surfaces
or objects. Another example will be pollutants produced by chemical plants where
pollution may a result several processes which multiply. The probability density function
of the lognormal random variable is given by

f X (x ) =

exp 21 2 (ln x ) U ( x )
2

1
2 2

2 x

and the cumulative distribution function is given by

x 1
2
1
2 2 y 2 exp 2 2 (ln y ) dy x 0
.
FX ( x ) = 0
0 x < 0

)
Var ( X ) = [exp( ) 1]exp(
E ( X ) = exp + 12 2
2

+ 2

0.7

1.2

lognormal CDF = 0.1; = 1

0.6

lognormal pdf = 0.1; = 1

0.5

0.8
0.4
)
x(
fx

0.3

)
x( 0.6
x
F

0.2

0.4

0.1

0.2

5
x

10

5
x

10

6. Weibull distribution
We all know the saying: A chain is only strong as its weakest link. Consider now a chain
consisting of number of links, each of which it self is a random variable. Then the
strength of the random variable is determined by the weakest in this chain. Weibull
distribution is used to model the strength of such a chain. It is thus a measure of the
reliability of the system. The density function is given by

f X ( x ) = abx b 1 exp ax b U ( x )
and the distribution function is given by

0
FX ( x ) =
b
1 exp ax

x<0

x0

Note that if b=1, Weibull density function becomes exponential density function.
0.7

1.2
W eibull cdf (a = .5, b = 2.0)

W eibull pdf (a = .5, b = 2.0)

0.6

0.5

F (x)

0.8

f (x)

0.4

0.6

0.3
0.4

0.2
0.2

0.1

3
x

3
x

7. Bionomial distribution
We already saw that Bernoulli trials resulted in bionomial probability. If we now
formulate a random variable X as the number of successes that occur in N trials, then X is
said to be a bionomial random variable. The density function for the bionomial random
variable is given by
k =

f X ( x ) = CkN p k (1 p )N k ( x k )

k =0
and the CDF is

k =

FX ( x ) = CkN p k (1 p )N k U ( x k )

k =0
where p is the probability of success (p < 1).

1.2

0.25
Bionomial pdf (N=10; p=0.45)

Bionomila CDF (N=10; p=0.45)

0.2

0.8

f (x)

F (x)

0.15

0.6

0.1

0.4

0.05

0.2

10

10

8. Poisson Distribution
We had looked at the bionomial distribution. Consider the case of a bionomial random
variable where N is very large and p is very small, keeping Np to be moderate. In this
case, bionomial distribution becomes a Poisson distribution with a parameter, = Np.
The density function of the Poisson random variable is given by
k =

f X (x ) =

e
k!

k ( x k )

e
k!

kU ( x k ) .

k =0
and the CDF is given by
k =

FX ( x ) =

k =0
Mean = ; variance =

Note that in the diagram, CDF has been shown to be continuous function. This is merely
to illustrate the approximation that when is high, Poisson approaches Gaussian.
0.18

1.2

Poisson pdf ( = 5)

0.16

Poisson CDF ( = 5)
1

0.14
0.12
)
x(
fX

0.8
)
x(
X
F

0.1
0.08
0.06

0.6

0.4

0.04
0.2

0.02
0

6
x

10

6
x

10

Poisson distribution is normally used to model the number of deaths in a given period,
death rates for insurance purposes, the number of earth quakes taking place over a certain
time period, the number of traffic accidents during the peak hours, etc.
Example: In a gas pipeline, leaks are noticed. There appears to be two leaks on an
average per every 10 miles of the pipeline. If you are using a Poisson model, what is the
probability that there will be at least 5 leaks in a 50 mile long pipeline?

Answer: In a 50 mile long pipeline, there will be an average of 10 leaks. We have a

Poisson distribution with =10. The probability that we have at least five leaks =
Prob(no leaks)+Prob(1 leak)+prob(2 leaks)+prob(3leaks)+prob(4leaks)+prob(5leaks)
5
k
10
= 10 e
=0.13.
k!
k =0
3.4 Histograms

Quite often, we may not have a density function available to us for modeling purposes.
But, we may still be interested in finding out the relative frequency of occurrences of
certain events. Consider for example, the case of ball bearings being manufactured. We
would like to see the spread in the diameters of the bearings to make sure that they wall
within the tolerance limits. We also would like to see if the mean diameter is acceptable.
To accomplish this, we plot of relative frequency chart. The form of the data distribution
will tell us about the process that was used for making ball bearings. We consider
samples from the plant and count the number of bearings that fall within a certain range
of diameters. We write down all the diameter values and based on the two extreme
values, divide the whole range into a number of bins (N). For each bin, we count the
number of bearings that fall within the diameter range of that bin and plot a bar chart.
This is what we call as a histogram. A typical histogram set for ball bearings with a mean
of 1.2 inch and standard deviation of 0.1 inch is shown. The continuous line represents
the Gaussian curve fitting the data. We see that the fit appears to be better if we have
more bins. Fit is also better if we have more samples (500 instead of 100). AS we
increase the number of bins and the number of samples, the envelope of the histogram
becomes closer and closer to the density function. A special case of a data set is shown in
Figure . It shows that the particular manufacturing plant produces two sets of bearings
(instead of one), one with a mean diameter of 1.2 inch and another with a mean diameter
of 1.4 inch. The advantage of analyzing is the histogram clearly demonstrates the
problems associated with the production.

40

30

100 samples
5 bins
30

y
c
n
e
u
q
er
f
e
vi
t
al
er

y
c
n
e
u
q
er
f
e
vi
t
al
er

20
10
0
0.8

1
1.2
1.4
diameter inch

1.6

300
y
c
n
e
u
q
er
f
e
vi
t
al
er

200

20
10
0
0.8

y
c
n
e
u
q
er
f
e
vi
t
al
er

y
c
n
e
u
q
er
f
e
vi
t
al
er

1
1.2
1.4
diameter inch

1.6

100
50
0
0.8

500 samples
10 bins

80
60
40
20
0
0.8

1.6

500 samples
5 bins

120
100

1.2
1.4
diameter inch

150
500 samples
5 bins

100
0
0.8

100 samples
10 bins

1.2
diameter

1.4

1.6

1
1.2
1.4
diameter inch

1.6

3.5 Multiple Random variables

For multiple random variables, say, X, Y, Z, we can write the expression for the joint
CDF as

FX ,Y ,Z ( x , y , z ) = P ( X x ,Y y , Z z )
and joint pdf is given by

f X ,Y ,Z ( x , y , z ) = xyz [FX ,Y ,Z ( x , y , z )].

3

In general, we may not always use the subscript for the CDF and pdf with the
understanding of the implied existence of them.
If X and Y are two random variables that are independent, the joint pdf will be the
product of marginal probability density functions, i.e.,

f (x , y ) = f (x ) f ( y ) .
Statistical averages and moments of random variables
If g(X) is a function of the random variable, the expected value of this function is given
by

E [g ( X )] = g ( x ) f ( x )dx .

If the random variable is discrete, the expected value is given by

M
E g X = g xk P X = xk .
k =1

[ ( )]

( ) (

In general, if we have a function of several random variables X1, X2, ... XN, the expected
value of g(X1,X2, XN) is given by

E [g ( X 1 , X 2 ,..X N )] = ... g ( x1 , x2 ,..x N ) f ( x1 , x2 ,..x N )dx1dx2 ..dx N .

The joint moment,

E [ XY ], given by

E [ XY ] = xyf ( x , y )dxdy

is known as the correlation of the two random variables X and Y, represented by RXY.
The covariance, CXY of the two random variables is given by

C XY = E [x E [ X ]]E [ y E [Y ]]

= ( x E [ X ])( y E [Y ]) f ( x , y )dxdy

The covariance can be expressed as

C XY = R XY E [ X ]E [Y ].

3.6 Central Limit Theorem

If we have a number of independent random variables, the density function of the sum of
these random variables will approach a normal distribution under certain conditions. If
X1, X2, X3, XN are a number of independent identically distributed (iid) random
variables with finite variances, then

Z = X 1 + X 2 + X 3 + .... X N
will have a pdf given by

f (z ) =

1
2 z2

( x z )2

exp
.
2 z2

Note that if the random variables are not identical and none of them have a variance that
is overwhelmingly high, the pdf of the sum will approach a normal distribution. If the
random variables are iids, N will be about 6. If the random variables are not identical N
will be higher.

MATLAB exercises

1. Use rand(1) function to generate 10 random numbers. Treating the numbers less than
or equal to 0.5 as zero and the other half as 1, simulate the experiment of a coin toss
(1-Heads) and 0-Tails). Starting from 10, increase the numbers in steps of ten and
establish how many numbers one needs to reach the case of an unbiased coin. Is it
possible to understand the concept of mutually exclusive events from this experiment?
2.
How will you prove using a set of random numbers, the probabilities associated
with the roll of a fair die?
3.
Use two sets of random numbers to establish the probability results relating to the
toss of two fair coins simultaneously. Can you also establish the results on independence
of two events?
4. Generate three sets of random numbers. For the first set, create zeros and ones as
described in Problem (1). For the second set, take zero to be all the numbers less than or
equal 0.3 and the rest to be 1. For the third set, take zeros to come from all numbers less
than or equal to 0.6 and the rest to be 1. Can you explain what the probabilities of zeros
and 1s in the three sets mean? Can you establish Bayes rule of total probability from
this experiment?
5. Generate two groups of 1000 uniform random numbers (xi and yi, i=1, 2, . .1000). Plot
histograms of zI = xi+yi and wi = xi - yi. Also calculate the mean and variance of z and w.
Comment on your results.
6. Generate two groups of 100 uniform random numbers (xi and yi, i=1, 2, . .100). Now
compute
zi = 2 ln( xi ) cos ( 2 yi ) . Plot a histogram of zi.

7. Generate 200 uniform random numbers (xi, i=1,2,..200). Calculate ui = ln ( xi ) and

plot its histogram.
Generate two groups of 200 uniform random numbers (xi and yi, i=1, 2, . .200). Now
compute

z i = 2 ln(xi ) cos (2 yi ) . wi = 2 ln(xi ) sin (2 yi )

Plot a histogram of r = z 2 + w 2 . Compare the histograms in (1) and (2). Comment on
i
i
i
your results.

8. Generate Uniformly distributed random numbers. Plot the histogram and cumulative
distribution function. Compare the results to theoretical plots. Use 10 different sets of

random numbers and using Central Limit Theorem, generate Gaussian Random Numbers.
Generate Gaussian directly using MATLAB and compare the results.
9. Generate a Gaussian random number set. Plot the histogram. Plot the distribution
function. Compare the plots to corresponding theoretical plots. Start with a set of 100 and
increase it 1000 in steps of 200. Comment on your results.
10. Now use two Gaussian random number sets and obtain the Rayleigh and exponential
from the Gaussian. Conversely, generate a Gaussian random number from a Rayleigh
distributed random number set and Uniformly distributed random number set.
11. Generate a set of Poisson distributed random numbers (means ranging from 1-10).
Increase the mean. Observe how the histogram changes. For example, generate a Poisson
number set with a mean of 25. Generate a Gaussian set with a mean of 25 and standard
deviation of 5. Compare this Gaussian random number set to the Poisson with mean
equal to 25. Discuss your observation.
12. Generate two sets of Gaussian random numbers, one with a mean of zero and
standard deviation of 1 and another one with a mean of 10 and standard deviation of 1.
Plot the histograms on the same graph.
Now for the first set, find the probability that the numbers are larger than 5, by counting
the numbers that are larger than 5.
For the second set, find the probability that the numbers are less than 5 similarly. Explain
your results. What do these results mean? Where are they likely to be used?(Concept of
ROC)