Introduction to Probability

Probability is the likelihood that an outcome occurs

Experiment
Situation with uncertainty that can be repeated
1. Toss two dice
2. Draw a Poker Hand
3. Toss a coin 3 times

Sample Space
set of all possible outcomes for an experiment
we like all points in sample space to have same probability
Probabilities of all possible outcomes must add to 1
Toss two coins
HH HT TH TT
Probabilities add to 1 so each point has 1/4 chance
Toss two dice

(1,1), (1,2), (1,6), (2,1).,(2,6),.(6,1), (6,6)

36 outcomes
Each point has chance 1/36

Event
a subset of the sample space
to which a probability is assigned

Axioms of Probability
Probabilities must satisfy
Probability of any event >=0
Event=All points in sample space
has probability 1

Three Interpretations of Probability

Classical Definition: Using Sample Space and experiment approach
Relative frequency approach: Use empirical data to estimate a
probability
A player makes 80 of 100 free throws: estimate chance of making
FT is 80%
With more data relative frequency estimates of probability change.
Subjective Probability: use judgement to estimate a probability
What is chance Dow is >=20,000 at end of 2020/
Wisdom of crowds often helps here
 Toss 2 dice probability total is 8
(2,6) (3,5) (4,4) (5,3) (6,2)
5/36
Family with 3 children
probability exactly one daughter
BBB BBG BGB BGG
GBB GBG GGB GGG

BBG BGB
GBB
3/8

Probability Draw a diamond from a deck of cards

13/52=1/4

ent approach
e of making

bility change.
probability
 1
Law of Complements 2
Events a and Not A are complements 3
Prob(A) + Prob(Not A) = 1 4
A=YELLOW 1 2
Prob(Not A) = 1- Prob(A) B=Not A BLUE

Probability you do not throw a total of 2 with 2 dice

A=total of 2
Not A= Not 2 1 2 3 4
1
2
3
4
5
6
3 Children
Probability at least 1 child is a girl
BBB BBG BGB BGG
GBB GBG GGB GGG
3 4

5 6
 A and Not A
Law of Complements

Prob(A) + Prob(Not A) = 1

Prob(Not A) = 1- Prob(A) A=YELLOW

B=Not A P(A) = 5/16
Probability you do not throw a total of 2 with 2 dice P(Not A)= 11/16
A=total of 2
Not A= Not 2 1 2 3 4
1
P(2) = 1/36 2
Prob(Not 2) = 1 - (1/36) =35/36 3
4
5
6
3 Children
Probability at least 1 child is a girl
BBB BBG BGB BGG Prob(>=1 child girl)= 1-Prob(0 children are girls)
GBB GBG GGB GGG Prob(>=1 child girl)= 1- (1/8) = 7/8
5 6
 Mutually Exclusive events Two events
Addition Rules are mutually exclusive if
the occurrence of one event
Event C Blue precludes occurrence of the
Event B orange
Event A x
1 2 3
1x
2x 1
3x 1
4 2
A and B not mututally exclusive 3
A and C mutually exclusive 4
Prob(A or B) 5
=Prob(A) + Prob(B)-Prob( A and B) 6
Prob(A or B)=

50% Chance rains on Sat 50

Is it sure to rain on the week
Events A and C are mutually exclusive
P(A or C)=

Throw two dice

what is chance you get at least one 4

P(4 on first die or 2nd die)=

 Mutually Exclusive
ually exclusive if A=2020 Texans win Super Bowl
rrence of one event B=Browns win 2020 Super Bowl
es occurrence of the other event Not Mutually Exclusive
A=DOW JONES GOES UP >=20% IN 2020
B=MICROSOFT STOCK GOES UP AT
LEAST 15% IN 2020

2 3 4 5 6

ance rains on Sat 50% chance rains on Sunday

to rain on the weekend?
 Mutually Exclusive events
Addition Rule
Prob(A or B)?? Probability
Event C Blue 2/12
Event B orange 3/12
A and B not Event A x 3/12
Mutually Exclusive 1 2 3
A and C Mutually Exclusive 1x
B and C Mutually Exclusive 2x
3x
4

In general FIRST
Prob(A or B)
=Prob(A) + Prob(B)-Prob( A and B)
Prob(A or B)=
3/12 + 3/12-(1/12)=5/12
If events X and Y are mutually exclusive
P(X and Y)=0
Then Prob( X or Y) = Prob(X) + Prob (Y)
Events A and C are mutually exclusive
so P(A or C)= 3/12 + 2/12 = 5/12
Two events Mutually Exclusive
are mutually exclusive if A=2020 Texans win Super Bowl
the occurrence of one event B=Browns win 2020 Super Bowl
precludes occurrence of the other event Not Mutually Exclusive
A=DOW JONES GOES UP >=20% IN 2020
B=MICROSOFT STOCK GOES UP AT
LEAST 15% IN 2020
SECOND
1 2 3 4 5 6
1
2
3
4 x
5
6

Throw two dice

what is chance you get at least one 4

P(4 on first die or 2nd die)= (1/6) +(1/6)-(1/36)=11/36

A= Rain on Saturday B = Rain on Sunday

50% Chance rains on Sat 50% chance rains on Sunday
Is it sure to rain on the weekend?
Prob(A or B)=P(A) + P(B)-Prob(rain on Sat and Sun)
.5 +.5-something greater than 0<1
Not sure to rain on the weekend
Independent Events

Events A and B are independent if knowledge that A has happened

does not change your estimate of Prob(B) or knowledge that B
has happened does not change your estimate of Prob(A).
Events Dow up >=10% and Microsoft down >=30% are not independent
Events Dow up >=10% and event Manchester United wins Premier League are independent
1 2
Two events A and B are independent if and only if A and B not independent
Prob(A and B) = Prob(A)*Prob(B) 1A B
2A B
3A B
4A B
5 A ,B B
A and B independent
1A B
13 diamonds 2A B
13 spades 3 A,B B
13 aces 4A
13 clubs 5A
Draw card from deck of cards 1 2 3
A=card is spade
B=card is an ace IF A, B C, D, ETC ARE INDEPENDENT
A and B are independent
PROB(A and B and C and D)=PROB(A

A= card is spade
B= card is ace of spades
A and B are not independent
Machines A B and C
work the following fraction of the time
machine failures independent
A 0.95
B 0.9
C 0.92

What fraction of the time is at least one machine working

 4

, ETC ARE INDEPENDENT

nd B and C and D)=PROB(A)*PROB(B)*PROB(Event C)*PROB(D)

Two independent systems each have 90% chance of working.

What is chance at least one system works?

ast one machine working?

Independent Events

Events A and B are independent if knowledge that A has happened

does not change your estimate of Prob(B) or knowledge that B
has happened does not change your estimate of Prob(A).
Events Dow up >=10% and Microsoft down >=30% are not independent
Events Dow up >=10% and event Manchester United wins Premier League are independent

Two events A and B are independent if and only if A and B not independent
Prob(A and B) = Prob(A)*Prob(B) A B
A B
A B
A B
A ,B B
A and B independent
A B
13 diamonds A B
13 spades A,B B
13 aces A
13 clubs A
Draw card from deck of cards
A=card is spade
B=card is an ace final IF A, B C, D, ETC ARE INDEPENDENT
A and B are independent
P(A)=1/4 P(B)=1/13 P(A and B) =1/52 PROB(A and B and C and D)=PROB(A
1/52=(1/4)*(1/13) True so A and B are independent
A= card is spade Two independent systems each have 90% chan
B= card is ace of spades What is chance at least one system works?
A and B are not independent
Machines A B and C
P(A) =1/4 work the following fraction of the time
P(B) = 1/52 machine failures independent
P(A and B) =1/52 A 0.95
1/52=(1/4)*(1/52) B 0.9
C 0.92
A and B not independent
What fraction of the time is at least one machine working

P(>=1 machine working) = 1- P(0 are working)

1-.0004 = .9996
0.9996 P(0 working) = P(all 3 fail) =(.05)*(.10)*(.08)

0.0004 =0.05*0.1*0.08
 Prob(A)=1/2
P(B)= 6/10
Prob(A and B) = 1/10
1/10= (3/5)*(1/2)
Not true so A and B are independent

P(A)=5/20=1/4
P(B)=4/20=1/5
P(A and B)=1/20

1/20=(1/4)*(1/5)
Yes so A and B are independent

, ETC ARE INDEPENDENT

nd B and C and D)=PROB(A)*PROB(B)*PROB(Event C)*PROB(D)

tems each have 90% chance of working.

st one system works?
1-Prob(0 systems work)

Prob (0 systems work)=P(both fail) =.1*.1=.01

Prob >=1 system works) = 1-.01 =.99
P(System A works or System B works)=P(A works) + P(B works) -P(A and B work)
.9+.9-(.9)*(.9)=.99

ast one machine working?

are working)
Conditional Probability
P(A and B)
P(A|B)= __________
P(B) CONDITIONAL PROBABILITY: P(A|B) =P(A and B)/P(B)

P(A and B) = P(B)*P(A|B) A AND B NOT INDEPENDENT

P(A and B) = P(A)*P(B|A) A A A A
A A AB A
A and B are independent if B B
P(A|B) = P(A) B
P(B|A) = P(B)

Throw two dice A AND B INDEPENDENT

A= >=one 6
B=Total 10 A A A A
P(B|A) A A AB AB
B B
B
Draw two cards without replacement DICE
A=first card is ace 1 2 3 4 5
B=Second card is an ace 1
P(B|A) 2
3
4
5
6
6
Conditional Probability
P(A and B)
P(A|B)= __________
P(B) CONDITIONAL PROBABILITY: P(A|B) =P(A and B)/P(B)
P(A) = 8/20 P(B) = 4/20
P(A|B)= 1/20/(4/20) = 1/4 P(A and B)=1/20
P(A and B) = P(B)*P(A|B) A AND B NOT INDEPENDENT P(B)=4/20
P(A and B) = P(A)*P(B|A) A A A A 1/20
A A AB A -------
A and B are independent if B B 4/20
P(A|B) = P(A) B P(A|B)= 1/4
P(B|A) = P(B) P(A) = 8/20 = 2/5

Throw two dice P(A) = 8/20

A= >=one 6 P(A|B) = 2/5 SO A AND B ARE NOW INDEPENDENT
B=Total 10 A A A A
P(B|A) A A AB AB
B B
B
Draw two cards without replacement DICE
A=first card is ace 1 2 3 4 5
B=Second card is an ace 1
P(B|A) 2
3
P(A and B)=2/36 4
P(A)=11/36 5
2/36 6x x x x x
P(B|A) ----------
11/36 P(B|A) 3/51
2/11 =1/17
P(B)=4/52
P(B)=3/36 A and B not Independent
P(A and B)=1/20

=1/4

P(A|B)= 1/4
P(A) = 8/20 = 2/5

P(A and B)=2/20

P(B)=5/20

P(A|B)= 2/20/(5/20) = 2/5

A and B are independent

6
x
x
x
x
x
x
LAW OF TOTAL PROBABILITY P(FL)=.05 P(FAIL|FL)=.8
FIND CHANCE OF AN EVENT P(N FL)=.95 P(FAIL|N FL) = .10
BY ADDING UP ALL THE MUTUALLY EXCLUSIVE WAYS
EVENT CAN HAPPEN
5% cars flood damage 80% of them have engine fail
95% no flood damage only 10% have engine fail

What fraction of cars have engine fail?

Fail Not Fail

Flood
No Flood
LAW OF TOTAL PROBABILITY P(FL)=.05 P(FAIL|FL)=.8
P(N FL)=.95 P(FAIL|N FL) = .10

5% cars flood damage 80% of them have engine fail

95% no flood damage only 10% have engine fail
P(FAIL)=P(FAIL WITH FL) +P(FAIL N F
What fraction of cars have engine fail? P(FL)*P(FAIL|FL)+P(N FL)
.05*.8+.95*.10
0.135
Fail Not Fail 13.5% CHANCE CAR FAIL
Flood 0.04 0.01
No Flood 0.095 0.855
FAIL WITH FL) +P(FAIL N FL)
P(FL)*P(FAIL|FL)+P(N FL)*P(FAIL|N FL)
.05*.8+.95*.10

13.5% CHANCE CAR FAILS

 Bayes Theorem
HOW WE USE INFORMATION TO UPDATE OUR VIEW OF THE WORLD Gilbert Welch
States of the World Overdiagnosed
Cancer No Cancer Receive a signal
+Mammogram
- Mammogram
Prior Probabilities
of States of the World
before receiving a "signal"

P(Cancer) =.004 Likelihoods

P(No Cancer)= .996 P(+|Cancer)=.8
P(+|No Cancer) = .10

Want posterior Probability

LOOK AT 10,000 PEOPLE P(Cancer|+ test result)=
+ -
CANCER
NO CANCER

P(Cancer|+)=
Gilbert Welch
Overdiagnosed

Prior Prob*Likelihoods

P(Cancer and +)
----------------
P(+ test result)
 Bayes Theorem
Gilbert Welch
States of the World Overdiagnosed
Cancer No Cancer Receive a signal
+Mammogram
- Mammogram
Prior Probabilities
of States of the World Want posterior Probability
before receiving a "signal" P(Cancer|+ test result)=

P(Cancer) =.004 Likelihoods

P(No Cencer)= .996 P(+|Cancer)=.8
P(+|No Cancer) = .10

LOOK AT 10,000 PEOPLE

+ -
CANCER 32 8
NO CANCER 996 8964

P(CANCER|+ TEST RESULT)

P(Cancer|+)= 0.0311284 =32/(996+32) 3.10%
Gilbert Welch
Overdiagnosed

Prior Prob*Likelihoods

P(Cancer and +)
----------------
P(+ test result)

P(Cancer)*P(+|Cancer)
-----------------------------
P(+ with Cancer)+P(+ with no cancer)

P(Cancer)*P(+|Cancer)
----------------------------
P(Cancer)*P(+|Cancer)+P(No cancer)*P(+|No Cancer)

Numerator 0.0032 =0.004*0.8

DenominATOR 0.1028 =0.004*0.8+0.996*0.1

0.0311284047 =J20/J21
Problem 1 Problem 2
40% of students get an A in marketing We toss two dice
30% of students get an A in statistics A= Event first die is 3
10% get an A in Both B = Event Total is 6
What fraction of students
do not get an A in either course? Are these events independent?
P(No A's ) = 1- P(>=1 A)
P( A and B) = 1/36
P(>=1 A)= P(A in Marketing) +P(A in Stats)- P(A in both) = .4 +.3 -1. P(A) = 1/6
P(B) = 5/36
P(No A's) = 1- .6 = .4 P(A)*P(B) = 5/216
No A and B are not independent
A in Stats No A in Stats
A in Marketing 0.1 0.3 Let C = Event Total is 7
No A in Marketing 0.2 0.4
A and C are independent
P(Event C) = 1/6
P(A and C) =1/36=P(A)*P(Event C)

Problem 3
A bowl has 3 Red and 5 Blue balls Problem 4
We draw 2 balls without replacement 10 coins
What is the chance that we get >=1 Blue Ball 5 Two Headed
3 Two tailed
P(>=1 blue ball) = 1- P( 0 blue balls) 2 Fair
P(0 Blue Balls) =(3/8)*(2/7) = 6/56 What is chance of head?
P(>=1 blue ball) = 1 - 6/56 = 50/56 Given coin toss comes up heads wh

or add up Prior Probablities

P(BB) (5/8)*(4/7) = 20/56 P(2H) = 1/2 P(2T) = 3/10 P(1H) = 2/1
P(RB) (3/8)*(5/7) = 15/56
P(BR) (5/8)*(3/7) = 15/56 Likelihoods
Total 50/56 P(H|2H)=1 P(H|2T) = 0 P(H|F) .5

P(H) = P(2H)*P(H|2H) + P(2T)*P(H|2T) + P(F)*

.5*1+.3*0+.2*.5= .6
P(F)*P(H|F)
P(F|H)= ---------------------------------
P(H)

.2*.5
--------------
0.6

Problem 5
P(A)=.1 P(B) =.2 (Event C) = .3

What is P(A or B or C)
If events are Mutually Exclusive?
If events are independent?

Mutually Exclusive
P(A) + P(B) + P(Event C) = 0.6

Independent
P(A or B or C) = 1- P(None of A B and C)

P(None of A B and C) = (1-.1)*(1-.2)*(1-.3) =.9*.8*.7 = .504

P(A or B or C) = 1-.504 = .496

first die is 3

e events independent?

(1,5) (2,4) (3,3) (4,2) (5,1)

B are not independent

ent Total is 7

re independent (1,6) (2,5) (3,4) (4,3) (5,2) (6,1)

C) =1/36=P(A)*P(Event C)

hance of head? .6 denominator of Bayes

n toss comes up heads what is chance coin was fair?

/2 P(2T) = 3/10 P(1H) = 2/10

1 P(H|2T) = 0 P(H|F) .5

H) + P(2T)*P(H|2T) + P(F)*P(H|F)

P(F)*P(H|F)
----------------------------------------

1
-------------- = ---------
6