AMS 131: Introduction to Probability Theory (Spring 2013)

Discrete and continuous univariate distributions (Sections 3.13.3)

A (real-valued) random variable (r.v.) is a function dened on the sample space S associated with an
experiment and taking values on (a subset of) the real line R.
Discrete random variables: A r.v. X is discrete (equivalently, X has a discrete distribution) if it has
either a nite number or countable number of possible values, say, {x
1
, ..., x
k
} or {x
1
, x
2
, ...}, respectively.
The probability distribution of a discrete r.v. X can be dened through its probability function
(p.f.), f(x
i
) = Pr(X = x
i
), for any possible value x
i
of X (the denition can be extended to f(x) for any
x R, since f(x) = 0 for all x that are not possible values of X).
Any p.f. must satisfy the following two conditions:
(1) f(x
i
) 0, for all x
i
(2)

xi
f(x
i
) = 1
The probability of any subset A of the real line can be obtained from Pr(X A) =

xiA
f(x
i
)
Continuous random variables: A r.v. X is continuous (equivalently, X has a continuous distribution)
if it takes an uncountable number of possible values (say, values in a bounded interval, in R
+
, or in R).
The probability distribution of a continuous r.v. X is determined by its probability density function
(p.d.f.), f(x), such that for any subset A of the real line, Pr(X A) =

A
f(x)dx.
Any p.d.f. must satisfy the following two conditions:
(1) f(x) 0, for all x R (2)

f(x)dx = 1
Note that for a continuous r.v. X, Pr(X = x
0
) = 0 for any x
0
R. Moreover, a value of a p.d.f. is not
a probability, but rather a probability density (f(x
0
) indicates how dense probability is around x
0
).
Distribution functions: The distribution function can be used to characterize the distribution of a
r.v. regardless of its type (discrete or continuous). The distribution function (d.f.), or cumulative
distribution function (c.d.f.), of a r.v. X is a function on R with values in [0, 1] dened by
F(x) = Pr(X x), x R
If F is the d.f. for a real-valued r.v., it has the following properties:
F is non-decreasing, i.e., F(x
1
) F(x
2
) for any x
1
< x
2
.
lim
x
F(x) = 0 and lim
x
F(x) = 1.
F is right-continuous, i.e., lim
yx;y>x
F(y) = F(x).
(And conversely, any function from R to [0, 1] that satises these conditions is a d.f. for a r.v.)
The d.f. F of a r.v. X can be used to compute probabilities for various types of intervals other than
interval (, x] in the denition of F. For instance, Pr(x
1
< X x
2
) = F(x
2
) F(x
1
), for any x
1
< x
2
;
Pr(X < x) = F(x

), for any x R, where F(x

) = lim
yx;y<x
F(y); and Pr(X = x) = F(x) F(x

),
for any x R. The last result provides the connection between the p.f. and d.f. for a discrete r.v. X.
For a continuous r.v. X, we have F(x) =

f(t)dt, and therefore f(x) = dF(x)/dx for all x at

which F is dierentiable.
Bivariate distributions (Sections 3.43.6)
A bivariate distribution refers to the joint probability distribution of two random variables dened on
a common sample space and each taking values on (a subset of) the real line R.
Discrete joint distributions: A bivariate r.v. (X, Y ) is discrete if both r.v.s X and Y have either a
nite number or countable number of possible values. The probability distribution of a discrete bivariate
r.v. (X, Y ) can be dened through the joint probability function (joint p.f.),
f(x, y) = Pr({X = x} {Y = y}),
for any possible value (x, y) of (X, Y ) (the denition can be extended for any (x, y) R
2
, since
f(x, y) = 0 for all (x, y) that are not possible values of (X, Y )). A joint p.f. must satisfy two conditions:
(1) f(x, y) 0, for all (x, y); and (2)

y
f(x, y) = 1. The probability of any subset A of R
2
can be
obtained from Pr((X, Y ) A) =

(x,y)A
f(x, y).
The marginal p.f.s of X and Y are given by
f
1
(x) = Pr(X = x) =

y
f(x, y), x R and f
2
(y) = Pr(Y = y) =

x
f(x, y), y R.
For any y such that f
2
(y) > 0, the conditional p.f. of X given that Y = y arises directly from the
denition of the conditional probability of event {X = x} given event {Y = y}. In particular,
g
1
(x | y) = Pr(X = x | Y = y) =
f(x, y)
f
2
(y)
, x R.
Analogously, for any x such that f
1
(x) > 0, the conditional p.f. of Y given X = x is given by g
2
(y | x) =
Pr(Y = y | X = x) = f(x, y)/f
1
(x), y R.
Continuous joint distributions: A bivariate r.v. (X, Y ) is continuous if both r.v.s X and Y take
an uncountable number of possible values (say, values in a bounded interval, in R
+
, or in R). The
probability distribution of a continuous bivariate r.v. (X, Y ) is determined by its joint probability
density function (joint p.d.f.), f(x, y), such that for any subset A of R
2
,
Pr((X, Y ) A) =

A
f(x, y) dxdy.
(Note that under a continuous bivariate distribution, points in R
2
and one-dimensional subsets of R
2
have probability 0.) Any joint p.d.f. must satisfy two conditions: (1) f(x, y) 0, for all (x, y) R
2
; and
(2)

f(x, y) dxdy = 1.
The marginal p.d.f.s of X and Y are given by
f
1
(x) =

f(x, y) dy, x R and f

2
(y) =

f(x, y) dx, y R.
For any y such that f
2
(y) > 0, the conditional p.d.f. of X given that Y = y is dened by
g
1
(x | y) =
f(x, y)
f
2
(y)
, x R.
Analogously, for any x such that f
1
(x) > 0, the conditional p.d.f. of Y given X = x is dened by
g
2
(y | x) = f(x, y)/f
1
(x), y R.
Mixed bivariate distributions: Certain applications in probability and statistics involve pairs of r.v.s
where one is discrete and the other is continuous.
The probability distribution of a mixed bivariate r.v. (X, Y ), where, for example, X is discrete and
Y is continuous, is dened by a joint p.f./p.d.f., f(x, y), which provides the probability of any subset A of
R
2
by summing the values of f(x, y) over x and integrating f(x, y) over y, for all (x, y) A. Such a joint
p.f./p.d.f. must satisfy two conditions: (1) f(x, y) 0, for all (x, y) R
2
; and (2)

x
f(x, y) dy = 1.
Here, the marginal p.f. for X is given by f
1
(x) = Pr(X = x) =

f(x, y) dy, x R, and the

marginal p.d.f. for Y by f
2
(y) =

x
f(x, y), y R.
Bivariate cumulative distribution functions: Although less convenient to work with than the uni-
variate case, the denition of the cumulative distribution function can be extended to the joint cumu-
lative distribution function (joint c.d.f.), which can again be used to characterize the distribution of
a bivariate r.v. regardless of its type.
The joint c.d.f. of a bivariate r.v. (X, Y ) is a function on R
2
with values in [0, 1] dened by
F(x, y) = Pr({X x} {Y y}), (x, y) R
2
.
The marginal c.d.f. for r.v. X can be obtained from F
1
(x) = lim
y
F(x, y), x R, and the marginal
c.d.f. for r.v. Y can be obtained from F
2
(y) = lim
x
F(x, y), y R.
For a continuous bivariate r.v. (X, Y ), we have F(x, y) =

f(r, s) drds. Therefore, the joint

p.d.f. can be obtained from the joint c.d.f. using
f(x, y) =

2
F(x, y)
xy
for all (x, y) at which the second-order partial derivative above exists.
Independent random variables: Two r.v.s X and Y are independent if, by denition, for any sub-
sets A and B of R, Pr({X A} {Y B}) = Pr(X A)Pr(Y B).
It can be shown that r.v.s X and Y are independent if and only if
F(x, y) = F
1
(x)F
2
(y), for all (x, y) R
2
.
Moreover, a similar factorization applies under independence to the joint p.f., joint p.d.f. or joint
p.f./p.d.f., f(x, y), for discrete, continuous or mixed bivariate r.v.s, respectively. Specically, r.v.s X
and Y are independent if and only if
f(x, y) = f
1
(x)f
2
(y), for all (x, y) R
2
.
Bayes theorem for random variables: Assume that the joint p.f., p.d.f or p.f./p.d.f., f(x, y), for r.v.s
X and Y is built from the marginal p.f. or p.d.f., f
2
(y), for Y and the conditional p.f. or p.d.f., g
1
(x | y),
of X given Y . Then, the conditional distribution of Y given X can be obtained through its conditional
p.f. or p.d.f.,
g
2
(y | x) =
g
1
(x | y)f
2
(y)
f
1
(x)
where f
1
(x) =

y
g
1
(x | y)f
2
(y) or f
1
(x) =

g
1
(x | y)f
2
(y) dy, if Y is a discrete or continuous r.v.,
respectively.