Probability Outline

Michael T. Stobb
June 16, 2014

Probability: the study of Random Variables

Random Variables: A variable that assumes values non-deterministically from a set of
known possibilities
EXAMPLE: The value of a Dice. Call it D, it takes on the values 1,2,3,4,5, and 6
randomly.
Discrete RVs
Probability Mass function (PMF)

Maps the outcomes of the RV to a number in the real interval 0 to 1.

Can think of this as the percentage of times the event will occur
If f (X) is the pmf, then f (x) [0, 1] x .
P
Also, we have
f (x) = 1.
x

Matlab Detour 1
(

1
2
1
2

1 - Heads
0 - Tails
N=100; M=100; X = sum(rand(N,M) < 0.5) - N/2; hist(X)
Should see a binomial/normal graph appear
Coin Flipping Experiment: f (x) =

Cumulative Distribution Function (CMF)

F (x) =

x
P

f (i)

i=1

Imagine a dice roll. F (1) = 1/6, F (0) = 0, F (6) = 1.

What is F (7) =? or F (1.5) =?
Matlab Detour 2
(

1
6

1, 2, 3, 4, 5, 6
0 else
N=100; M=100; X = mean(1+floor(6*rand(N,M)); hist(X)
Should see a binomial/normal graph appear about 3.5
Dice Rolling Experiment: f (x) =

Some important discrete distributions:

Uniform (weve already been using it a bunch)
1

Binomial - number of successes in N independent trials

Negative Binomial - number of success before a specified number of failures occurs
Geometric - number of tests before getting a success
Hypergeometric - number of successes in N independent trials without replacement
Poisson - number of events occuring in a fixed interval of time

Continuous RVs
Probability Density Function
Similar to Discrete case, but

f (x) dx = 1

Note: f (x) can exceed 1. Consider the uniform between 0 and 0.5.
Note: f (x) must be strictly positive
Cumulative Distribution Function (CDF)
Rx

Similar to discrete case, but F (x) =

f (x) dx.

Every continuous RV has a CDF, but not necessarily a PDF. So we can specify F (x)
d
and then define f (x) = dx
F (x).
Some important continuous distributions:
Uniform - Continuous case
Normal - Values are closer to the than they are further away
Gamma - Time waited until event occurs
Exponential - The time between events in a Poisson process
Chi-Squared - Sum of k independent standard normals

Matlab Detour 3
Normally Distributed Random Numbers
N=1000; X = randn(N,1); hist(X)
Should see a standard normal graph

Set Theory
Union: , means both, and, add
Intersection: , means or, either, multiplication
Can be visualized with Venn Diagrams
The Five Axioms of Probability:

P (A) = 1 P (A)c
P () = 0
If A B, the P (A) P (B)
P (A) 1
P (A B) = P (A) + P (B) P (A B)

 Conditional Probability
The probability of an event given that (by assumption) another event has already occured.
Axiom of Probability: P (A B) = P (A|B)P (B)
Read as Probability of A given B: P (A|B).
Visualized with Venn Diagrams by reducing sample space to just B.
If two events are statistically independent then P (A|B) = P (A)
Mutually exclusive events have P (A|B) = 0.
Bayes Theorem
In the Bayesian interpretation: Bayes Rule is the link between our past knowledge and
our new knowledge which is influenced by evidence.
Mathematically written as: P (A|B) =

P (B|A)P (A)
P (B)

Law of Total Probability: If all Bi0 s are pairwise disjoint and their union is the entire
P
sample space, then P (A) = P (A|Bi )P (Bi ).
Central Limit Theorem
The mean of a sum of i.i.d. RVs will be normally distributed.
Take a large sample or a RV, each has mean and variance 2 . Take the sample mean.
This sample mean will be normally distributed about .
Mathematically: P

X1 +X2 +...+X
n n

Mathematically: P

n
X

< =P Z<


< =P Z<

Matlab Detour 4
Chi-Squared Data: X = randn(1000,1000). b 2; hist(X)
Sum them: Z = sum(X); hist(X)
We should now see a normal distribution.