Probability

ASHISH VISWANATH PRAKASH

MANIPAL UNIVERSITY
ashish.viswanath@manipal.edu

9740310708

Importance of Probabilities in Business

Investment
Risk assessment probabilities of gain and loss

Customer Service
Policies related to customer service Queuing Models

Competitive Strategy
Markovs chains in strategy analysis

Product Design
Reliability theory probabilities of failure or breakdown
Drafting warranties and return policies

Simple Queuing System

Buffer or
Queue
Customer
Population
Input
Source

Arrival

Server(s)

Exit

Cost

Trade off

Cost of Service

Combined Costs

Cost of waiting

Capacity to serve

Analytical Queuing Models

Queuing simulator
http://simjs.com/queuing/index.html

Popular Simulators:
Pro model, Simul8, Arena, Anylogic simulator, Matlab simulink

The gambling industry relies on probability distributions to calculate the odds of winning.
The rewards are then fixed precisely so that, on average, players lose and the house wins

Learning Objectives
In this chapter, you will learn:

Basic probability concepts

Conditional probability
To use Bayes Theorem to revise probabilities

What is probability?
Probability is a numerical measure of the
likelihood that an event will occur.
Probability lets you predict the future by assessing how likely outcomes are, and knowing what
could happen helps you make informed decisions.

Probability as a Numerical Measure

of the Likelihood of Occurrence
Increasing Likelihood of Occurrence
Probability:

0
The event
is very
unlikely
to occur.

.5
The occurrence
of the event is
just as likely as
it is unlikely.

1
The event
is almost
certain
to occur.

Statistical Experiments
In statistics, the notion of an experiment differs
somewhat from that of an experiment in the
physical sciences.
In statistical experiments, probability determines
outcomes.
Even though the experiment is repeated in exactly
the same way, an entirely different outcome may
occur.

For this reason, statistical experiments are sometimes called random experiments.

RANDOM EXPERIMENT

Experiment

Flip a coin

We define an experiment as a process that

generates well-defined outcomes.

Satisfies the following conditions:

1.

All possible distinct outcomes are known in advance

2.

The result of experiment cannot be precisely ascertained

before the experiment.

3.

Experiment can be repeated under identical conditions

Heads, Tails

Numbers: 0,
Exam Marks
1, 2, ..., 100

Course
On any single repetition of an experiment, one and only one of Grades
the possible experimental outcomes will occur.

Outcomes

Play a
football
game
Select a part
for
Inspection

F, D, C, B, A,
A+
Win, Lose,
Tie

Defective,
non defective

RANDOMNESS
Randomness is the lack of pattern or predictability in events
A random sequence of events, symbols or steps has no order and does not follow an
intelligible pattern or combination

Randomness can be seen as conflicting with the deterministic ideas of some

religions, such as those where the universe is created by an omniscient deity who is
aware of all past and future events.
According to several standard interpretations of quantum mechanics, microscopic
phenomena are objectively random
In a deterministic universe, based on Newtonian concepts, there would be no
probability if all conditions were known

Im convinced
that God
Doesnt play
Dice

SAMPLE SPACE
The sample space for an experiment is the set of all experimental
outcomes.
Experiment outcome is also known as a sample point to identify it as
an element of the sample space
ex. All 6 faces of a die:

ex. All 52 cards in a deck of cards

ex. All possible outcomes when having a child: Boy or Girl

Tossing a coin
S = {Head, Tail}

Selecting a part for inspection

S = {Defective, Non defective}
Rolling a die: experimental outcomes, defined as the number of dots
appearing on the upward face of the die.
S = {1, 2, 3, 4, 5, 6}

What is probability?
According to classical theorists,

*Pierre-Simon Laplace

Probability

NUMBER of favorable outcomes

= Total
NUMBER of possible outcomes
of an Event

Please note that this particular formula is only applicable in a well defined
purely theoretical settings

What is the probability the spinner will land on the

number 3?

1 2
3 4

Ask yourself the following questions:

1. How many 3s are on the spinner?
2. How many possible numbers could the spinner land on?

1 2
3 4

1
4

What is the probability the die will land on an

even number?

Remember, a die has six sides. Numbers 1, 2, 3, 4,

5, and 6 are each depicted once on the die.

Ask yourself the following questions:

1. How many even numbers are on the die?
2. How many possible numbers could the die land on?

3
6

What is the probability that I will choose a red

marble?
In this bag of marbles, there are:
3 red marbles
2 white marbles
1 purple marble
4 green marbles

Ask yourself the following questions:

1. How many red marbles are in the bag?
2. How many marbles are in the bag IN ALL?

3
10

Counting Rules

Multi step experiments

Consider the experiment of tossing two coins.

Let the experimental outcomes be defined in terms of the

pattern of heads and tails appearing on the upward faces of
the two coins.
How many experimental outcomes are possible for this
experiment?

TREE DIAGRAM FOR THE EXPERIMENT

Without the tree diagram, one might think only three experimental outcomes are possible for
two tosses of a coin: 0 heads, 1 head, and 2 heads.

The counting rule for multiple-step experiments makes it possible to

determine the number of experimental outcomes without listing
them.
COUNTING RULE FOR MULTIPLE-STEP EXPERIMENTS

If an experiment can be described as a sequence of k steps with n1

possible outcomes on the first step, n2 possible outcomes on the
second step, and so on, then the total number of experimental
outcomes is given by (n1)x(n2)x . . . x(nk).

Suppose we carry out two operations in sequence

Let

n1 = the number of ways the first

operation can be performed
n2 = the number of ways the second
operation can be performed once the
first operation has been completed.
Then N = n1 n2 = the number of ways the two
operations can be performed in sequence.

Kentucky Power & Light Company

KP&L is starting a project designed to increase the generating capacity of one of
its plants in northern Kentucky.
The project is divided into two sequential stages or steps: stage 1 (design) and
stage 2 (construction).
Even though each stage will be scheduled and controlled as closely as possible,
management cannot predict beforehand the exact time required to complete
each stage of the project.
An analysis of similar construction projects revealed possible completion times
for the design stage of 2, 3, or 4 months and possible completion times for the
construction stage of 6, 7, or 8 months.

In addition, because of the critical need for additional electrical power,

management set a goal of 10 months for the completion of the entire project.

EXPERIMENTAL OUTCOMES
(SAMPLE_POINTS) FOR THE KP&L PROJECT

TREE DIAGRAM FOR THE KP&L PROJECT

we see that the project will be completed in 8
to 12 months, with six of the nine experimental
outcomes providing the desired completion
time of 10 months or less.
Even though identifying the experimental
outcomes may be helpful, we need to consider
how probability values can be assigned to the
experimental outcomes before making an
assessment of the probability that the project
will be completed within the desired 10
months.

Combinations
This counting rule allows one to count the number of experimental outcomes
when the experiment involves selecting n objects from a (usually larger) set of
N objects.
CnN

where:

N!
N

n n !(N n )!

N! = N(N 1)(N 2) . . . (2)(1)

n! = n(n 1)(n 2) . . . (2)(1)
0! = 1

Factorials!
This is just a short hand notation that is sometimes used to save
time!

Examples:
5! Means 5*4*3*2*1 = 120
4! Means 4*3*2*1 = 24

By definition, 1! =1 and 0! =1

Consider a quality control procedure in which an inspector randomly selects two

of five parts to test for defects.
In a group of five parts, how many combinations of two parts can be selected?

Shows that with N=5 and n=2

If we label the five parts as A, B, C, D, and E, the 10 combinations or

experimental outcomes can be identified as:

AB, AC, AD, AE, BC, BD, BE, CD, CE, and DE.

Solve
Simple random sampling uses a sample of size n from a population of
size N to obtain data that can be used to make inferences about the
characteristics of a population.
Suppose that, from a population of 50 bank accounts, we want to
take a random sample of four accounts in order to learn about the
population.
How many different random samples of four accounts are possible?

Counting Rule for Permutations

Permutations It allows one to compute the number of experimental outcomes
when n objects are to be selected from a set of N objects where the order of
selection is important. The same n objects selected in a different order are
considered a different experimental outcome.
PnN

where:

N!
N
n !
n (N n )!

N! = N(N 1)(N 2) . . . (2)(1)

n! = n(n 1)(n 2) . . . (2)(1)
0! = 1

The counting rule for permutations closely relates to the one for
combinations; however, an experiment results in more
permutations than combinations for the same number of objects
because every selection of n objects can be ordered in n! different
ways.
Consider again the quality control process in which an inspector
selects two of five parts to inspect for defects. How many
permutations may be selected?

Thus, 20 outcomes are possible for the experiment of randomly

selecting two parts from a group of five when the order of selection
must be taken into account.
If we label the parts A, B, C, D, and E, the 20 permutations are:

AB, BA, AC, CA, AD, DA, AE, EA, BC, CB, BD,
DB, BE, EB, CD, DC, CE, EC, DE, and ED.

Example How many ways can you order/arrange the 4 objects

{A, B, C, D}
Solution:
n! = 4! = 4(3)(2)(1) = 24
Here are the orderings.

ABCD

ABDC

ACBD

ACDB

ADBC

ADCB

BACD

BADC

BCAD

BCDA

BDAC

BDCA

CABD

CADB

CBAD

CBDA

CDAB

CDBA

DABC

DACB

DBAC

DBCA

DCAB

DCBA

Assigning Probabilities
Basic Requirements for Assigning Probabilities:
1. The probability assigned to each experimental
outcome must be between 0 and 1, inclusively.
0 < P(Ei) < 1 for all i

where:
Ei is the ith experimental outcome and P(Ei) is its probability

Assigning Probabilities
Basic Requirements for Assigning Probabilities
2. The sum of the probabilities for all experimental
outcomes must equal 1.

P(E1) + P(E2) + . . . + P(En) = 1

where:
n is the number of experimental outcomes
Ei is the ith experimental outcome and P(Ei) is
its probability

Assigning Probability:
There are three approaches to assessing the probability of an
uncertain event:

1. a priori classical probability

It is based on the assumption that the outcomes of an experiment
are equally likely and mutually exclusive.

2. empirical probability or relative

frequency method
the probability of an event is based on observed data.

Objectivists approach

3. subjective probability
the probability of an event is determined by an individual, based on
that persons past experience, personal opinion, and/or analysis of a
particular situation.

Classical Approach
If an experiment has n possible outcomes, the classical method would assign a
probability of 1/n to each outcome.
Experiment:

Rolling a die

Possible outcomes: {1, 2, 3, 4, 5, 6}

Probabilities: Each sample point has a 1/6 chance of occurring.

Fair

Mutually exclusive Events

Mutually exclusive events are events that cannot happen at the same time (simultaneously).

Example of a priori classical probability

Find the probability of selecting a face card (Jack, Queen, or King) from a well
shuffled standard deck of 52 cards.
Probability of Face Card

favourable.outcomes
number of face cards

total.outcomes
total number of cards

12 face cards
3

52 total cards 13

Example of a priori classical probability

Find the probability of selecting a face card (Jack, Queen, or King) from a well
shuffled standard deck of 52 cards.
Probability of Face Card

favourable.outcomes
number of face cards

total.outcomes
total number of cards

12 face cards
3

52 total cards 13

What are the possibilities that:

A man aged 45 will die within the next year?
The consumer in a certain metro will purchase a particular product
during the next month ?
Production process will produce a defective product?

Frequentists approach (relative frequency)

Its not feasible to establish a set of complete and
mutually exclusive outcomes that are equally likely to
occur
They argue that only valid way is to do the experiments repeatedly and note
the outcomes
appropriate when data are available to estimate the proportion of the time
the experimental outcome will occur if the experiment is repeated a large
number of times.

If we toss a coin 1000 times and it comes up

heads 503 times (the probability of getting a
head is 503/1000)

David is really good at talking penalties, he was

able convert 75 out of 100 previous attempts
into goals. (probability of him hitting a goal,
given a penalty is 75%)

The Law of Large Numbers

In simple terms, it says that as
the number of trials keeps
increasing, the empirical
probability for an event expressed as an average converges to the classical
probability.

The probability of
heads is 0.5 = the
proportion of times
you get heads in
many repeated trials.

Consider a study of waiting times in the X-ray department for a local hospital. A
clerk recorded the number of patients waiting for service at 9:00 a.m. on 20
successive days and obtained the following results

Using the relative frequency method, we would assign a probability of 2/20 =.10
to the experimental outcome of zero patients waiting for service, 5/20 = .25 to
the experimental outcome of one patient waiting,so on
satisfies the two basic requirements of equations.

Relative Frequency Method

Example: Lucas Tool Rental

Lucas Tool Rental would like to assign probabilities to the number of car
polishers it rents each day. Office records show the following frequencies of
daily rentals for the last 40 days.
Number of
Polishers Rented
0
1
2
3
4

Number
of Days
4
6
18
10
2

Each probability assignment is given by dividing the frequency (number of days)

by the total frequency (total number of days).
Number of
Number
Polishers Rented of Days
4
0
6
1
18
2
10
3
2
4
40

Probability
.10
.15
.45
4/40
.25
.05
1.00

Relative Frequency Approach

Bits & Bytes Computer Shop tracks the number of desktop computer systems it
sells over a month (30 days):
Desktops Sold

# of Days

From this we can construct

10

the estimated probabilities of an event

12

For example,
10 days out of 30, 2 desktops were sold.

(i.e. the # of desktop sold on a given day)

Relative Frequency Approach

Desktops Sold [X]

# of Days

Desktops Sold

1/30 = .03 = P(X=0)

2/30 = .07 = P(X=1)

10

10/30 = .33 = P(X=2)

12

12/30 = .40 = P(X=3)

5/30 = .17 = P(X=4)

= 1.00

There is a 40% chance Bits & Bytes will sell 3 desktops on any given day [Based on estimates
obtained from sample of 30 days]

The subjective method

Most appropriate when one cannot realistically assume that the
experimental outcomes are equally likely and when little relevant
data are available.

We may use any information available, such as our experience or

intuition.
After considering all available information, a probability value that
expresses our degree of belief (on a scale from 0 to 1) that the
experimental outcome will occur is specified.

Subjective probability
In the subjective approach we define probability as the degree of belief that
we hold in the occurrence of an event
P(you drop this course)
P(NASA successfully land a man on the moon)
P(girlfriend says yes when you ask her to marry you)
The subjective method requires extra care to ensure that the two basic
requirements of probability are satisfied

Consider the case in which Tom and Judy make an offer to

purchase a house.
Two outcomes are possible:

E1 = their offer is accepted

E2 = their offer is rejected
Judy believes that the probability their offer will be accepted is .8; thus, Judy
would set P(E1) = .8 and P(E2)= .2.
Tom, however, believes that the probability that their offer will be accepted is .6;
hence, Tom would set P(E1) = .6 and P(E2) = .4.
Note that Toms probability estimate for E1 reflects a greater pessimism.

Both Judy and Tom assigned probabilities that satisfy the two basic
requirements.

The fact that their probability estimates are different emphasizes the personal
nature of the subjective method.
Even in business situations where either the classical or the relative frequency
approach can be applied, managers may want to provide subjective probability
estimates.
In such cases, the best probability estimates often are obtained by combining
the estimates from the classical or relative frequency approach with subjective
probability estimates.

Bayes theorem provides a means for combining subjectively determined prior

probabilities with probabilities obtained by other means to obtain revised, or
posterior, probabilities.

Probabilities for the KP&L Project

To perform further analysis on the KP&L project, we must develop probabilities for each of the
nine experimental outcomes listed in Table

On the basis of experience and judgment, management concluded that the

experimental outcomes were not equally likely.

Management then decided to conduct a study of the completion times for

similar projects undertaken by KP&L over the past three years. The results of a
study of 40 similar projects are summarized in Table

Management could have provided subjective probability estimates, but felt that the current
project was quite similar to the 40 previous projects. Thus, the relative frequency method was
judged best.

Question
The Powerball lottery is played twice each week in 28 states, the Virgin Islands,
and the District of Columbia. To play Powerball a participant must purchase a
ticket and then select five numbers from the digits 1 through 55 and a Powerball
number from the digits 1 through 42.
To determine the winning numbers for each game, lottery officials draw five
white balls out of a drum with 55 white balls, and one red ball out of a drum
with 42 red balls.
To win the jackpot, a participants numbers must match the numbers on the five
white balls in any order and the number on the red Powerball. A variety of other
cash prizes are awarded each time the game is played. For instance, a prize of
$200,000 is paid if the participants five numbers match the numbers on the five
white balls (Powerball website, March 19, 2006).

Solve
a. Compute the number of ways the first five numbers can be selected.

b. What is the probability of winning a prize of $200,000 by matching the

numbers on the five white balls?
c. What is the probability of winning the Powerball jackpot?

Questions

Events and Their Probabilities

Sample points and events provide the foundation for the study of probability. As
a result, we must now introduce the formal definition of an event as it relates to
sample points.
EVENT
An event is a collection of sample points.

Sample Space

Let us return to the KP&L project assume that the project manager is interested in the event that
the entire project can be completed in 10 months or less.

Let C denote the event that the project is completed in 10 months or less

C = {(2, 6), (2, 7), (2, 8), (3, 6), (3, 7), (4, 6)}
Event C is said to occur if any one of these six sample points appears
as the experimental outcome.

Other events that might be of interest to KP&L management include the

following
L =The event that the project is completed in less than 10 months
M =The event that the project is completed in more than 10 months
L ={(2, 6), (2, 7), (3, 6)}
M ={(3, 8), (4, 7), (4, 8)}
Now lets calculate probability of event C, L, M

PROBABILITY OF AN EVENT
The probability of any event is equal to the sum of the probabilities of the
sample points in the event.
C = {(2, 6), (2, 7), (2, 8), (3, 6), (3, 7), (4, 6)}, the probability of event C, denoted P(C), is given by

P(C) =P(2, 6) +P(2, 7)+ P(2, 8) + P(3, 6) +P(3, 7)+ P(4, 6)

P(C ) =.15 + .15 + .05 +.10 + .20 +.05 = .70
L = {(2, 6), (2, 7), (3, 6)}

P(L) = P(2, 6) + P(2, 7)+ P(3, 6)

=.15 +.15 +.10 = .40

Find P(M)

Any time that we can identify all the sample points of an experiment and
assign probabilities to each, we can compute the probability of an event using
the definition.
However, in many experiments the large number of sample points makes the
identification of the sample points, as well as the determination of their
associated probabilities, extremely cumbersome, if not impossible.

In the remaining sections of this chapter, we present some basic probability

relationships that can be used to compute the probability of an event without
knowledge of all the sample point probabilities.

Dice
You toss two dice. What is the probability of the outcomes summing to 5?

This is S:
{(1,1), (1,2), (1,3),
etc.}

There are 36 possible outcomes in S, all equally likely (given fair dice).
Thus, the probability of any one of them is 1/36.
P(the roll of two dice sums to 5) = P(1,4) + P(2,3) + P(3,2) + P(4,1) = 4 / 36 = 0.111

Sample spaces
Its the question that determines the sample space.

A. A basketball player
shoots three free throws.
What are the possible
sequences of hits (H) and
misses (M)?

H -

HHH

M -

HHM

H
M
M

H -

HMH

M -

HMM

B. A basketball player shoots

three free throws. What is the
number of baskets made?

S = { 0, 1, 2, 3 }

S = { HHH, HHM,
HMH, HMM, MHH,
MHM, MMH, MMM }
Note: 8 elements, 23

Events example
Events

When we say "Event" we mean one (or more) outcomes.

Example Events:
Getting a Tail when tossing a coin is an event

Rolling a "5" is an event.

An event can include several outcomes:
Choosing a "King" from a deck of cards (any of the 4 Kings) is also an event

Rolling an "even number" (2, 4 or 6) is an event

Complement of an Event
The complement of event A is defined to be the event
consisting of all sample points that are not in A.
The complement of A is denoted by Ac.

Event A

Venn
Diagram

Ac

Sample
Space S

A = Event that outcome of a single trial of rolling a dice is

an odd number

A
2

A
1

5
4

A
6
5

3
4
2

A = Event that outcome of a single trial of rolling a dice is more than 2

Consider the case of a sales manager who, after reviewing sales reports, states
that 80% of new customer contacts result in no sale. By allowing A to denote the
event of a sale and Ac to denote the event of no sale, the manager is stating that
P(Ac) = .80.

We can conclude that a new customer contact has a .20 probability of resulting
in a sale

Mutually Exclusive Events (disjoint)

Two events are said to be mutually exclusive if the
events have no sample points in common.
Two events are mutually exclusive if, when one event
occurs, the other cannot occur.

Event A

Event B

Sample
Space S

If two events A and B are are mutually exclusive

then:
They have no outcomes in common.
They cant occur at the same time. The outcome of the
random experiment cannot belong to both A and B.

More examples
Mutually Exclusive means we can't get both events at the same time.

It is either one or the other, but not both

Partially overlapping events

A
B

Sample Space

Union of Two Events

The union of events A and B is the event containing
all sample points that are in A or B or both.
The union of events A and B is denoted by A B

Event A

Event B

Sample
Space S

Event A

Sample
Space S

Event B

Union
Let A and B be two events, then the union of A and B is the
new event (denoted by AB) defined by:

A B = {e | e belongs to A or e belongs to B}
The event A B occurs if
the event A occurs or the
event B occurs .

AB

Intersection of Two Events

The intersection of events A and B is the set of all
sample points that are in both A and B.
The intersection of events A and B is denoted by A

Event A

Intersection of A and B

Event B

Sample
Space S

The area where the two circles overlap is the intersection; it

contains the sample points that are in both A and B.

Intersection
Let A and B be two events, then the intersection
of A and B is a new event (denoted by AB)
defined by: A B = {e| e belongs to A and e belongs to B}
AB

The event A B occurs if the

event A occurs and the event B
occurs
A

Rules of addition
The probability of a single event is called marginal (or unconditional )
probability
Eg. In the coin tossing example the marginal probability of event Head or tail
can be expressed as P(T) or P(H)
If two or more than two events are likely to occur from a Random Experiment
and we are interested to know the probability of occurrence of at least one of
the events then rules of addition are used to do so.
That is, with events A and B we are interested in knowing the probability that
event A or event B or both occur.

Addition rule
Mutually exclusive events

P( A or B) = P(A) + P(B)

Event A

Sample
Space S

Event B

Question
Consider the pattern of arrival of customers at a service counter during one hour
time slot along with its probability
No of
customers

4 or more

Probability

0.1

0.2

0.3

0.3

0.1

Find the probability that either 2 or 3 customers will arrive at the service
counter during this hour
P (2 or more) =?

Partially overlapping (joint) events

If events A and B are not mutually exclusive, it is possible for both

events to occur simultaneously, as they share common sample
points.

AB

P(A or B) = P(A) + P(B) P(A and B)

Or
P(A B) = P(A) + P(B) P(A B)

If events A and B are mutually exclusive, P(A B = 0.

The addition law for mutually exclusive events is:
P(A B) = P(A) + P(B)

There is no need to
include P(A B

Exhaustive events

Condition : P(A B) = 1

P(A B) 1

P(A B C D) = 1

C
B

D
Sample space

Sample space

Are these events Mutually exclusive ?

Event A = card drawn from a deck of cards is a face card
Event B = card drawn from a deck of cards is a red card
Event C = card drawn from a deck of cards is an ace
Event D = card drawn from a deck of cards is a spade
Event E = card drawn from a deck of cards is a black card

1. Event A and B ?
2. Event B and C ?
3. Event D and B ?
4. Event E and card drawn from a deck of cards is a diamond card ?

Question
Consider a recent study conducted by the personnel manager of a major
computer software company.
The study showed that 30% of the employees who left the firm within two years
did so primarily because they were dissatisfied with their salary, 20% left
because they were dissatisfied with their work assignments, and 12% of the
former employees indicated dissatisfaction with both their salary and their work
assignments.
What is the probability that an employee who leaves within two years does so
because of dissatisfaction with salary, dissatisfaction with the work assignment,
or both?

Question
50 sports enthusiasts at the Head First Health Club are asked
whether they play baseball, football, or basketball. 10 only play
baseball. 12 only play football. 18 only play basketball. 6 play
baseball and basketball but not football. 4 play football and
basketball but not baseball.

Draw a Venn diagram for this probability space. How many

enthusiasts play baseball in total? How many play basketball? How
many play football? Are any sports rosters mutually exclusive?
Which sports are exhaustive (fill up the possibility space)?

 By adding up the values in each circle in the Venn diagram, we can determine that there are 16
total baseball players, 28 total basketball players, and 16 total football players.
The baseball and football events are mutually exclusive. Nobody plays both baseball and
football, so P(Baseball Football) = 0
The events for baseball, football, and basketball are exhaustive. Together, they fill the entire
possibility space, so P(Baseball Football Basketball) = 1

Questions

P(A)

P(B)

P(A B)

P(A B)

1- P(A B)

Question
Are A and A mutually exclusive or exhaustive?

Actually theyre both.

A and A can have no common elements, so they are
mutually exclusive.
Together, they make up the entire possibility space so
theyre exhaustive too.

Events in Sample Space

Simple event
An outcome from a sample space with one
characteristic (can be a single or compound event)
ex. A red card from a deck of cards
Complement of an event A (denoted A/)
All outcomes that are not part of event A
ex. All cards that are not diamonds
Joint event
Involves two or more characteristics simultaneously
ex. An ace that is also red from a deck of cards

Visualizing Events in
Sample Space

Ace

Contingency Tables or joint

probability tables:

Black
Red
Total

2
2
4

Not
Ace
24
24
48
2

Tree Diagrams:
Sample
Space

Full Deck
of 52 Cards

24
2
24

Total
26
26
52

Definitions
Simple vs. Joint Probability
Simple (Marginal) Probability refers to the

probability of a simple event.

ex. P(King)

Joint Probability refers to the probability of

an occurrence of two or more events.

ex. P(King and Spade)

Computing Joint and

Marginal Probabilities
The probability of a joint event, A and B:

P( A and B)

number of outcomes satisfying A and B

total number of elementary outcomes

Computing a marginal (or simple) probability:

P(A) P(A and B1 ) P(A and B 2 ) P(A and Bk )

Where B1, B2, , Bk are k mutually exclusive and

collectively exhaustive events

Example:
Joint Probability
P(Red and Ace)

number of cards that are red and ace 2

total number of cards

52
Ace

Not
Ace

Total

Black

24

26

Red

24

26

Total

48

52

Example:
Marginal (Simple) Probability
P(Ace)
P( Ace and Re d) P( Ace and Black )

Ace

Not Ace

Total

Black

24

26

Red

24

26

Total

48

52

2
2
4

52 52 52

Joint Probability Using a

Contingency Table
Event
B1

Event

B2

Total

A1

P(A1 and B1)

P(A1 and B2)

P(A1)

A2

P(A2 and B1)

P(A2 and B2)

P(A2)

Total

P(B1)

P(B2)

Joint Probabilities

Marginal (Simple) Probabilities

Example
Why are some mutual fund managers more successful than others?
One possible factor is where the manager earned his or her MBA.
The following table compares mutual fund performance against
the ranking of the school where the fund manager earned their
MBA.
Mutual fund outperforms
the market

Mutual fund doesnt

outperform the market

Top 20 MBA program

.11

.29

Not top 20 MBA program

.06

.54

E.g. This is the probability that a mutual fund

outperforms AND the manager was in a top20 MBA program; its a joint probability
[intersection].

 Alternatively, we could introduce shorthand notation to represent

the events:

A1 = Fund manager graduated from a top-20 MBA program

A2 = Fund manager did not graduate from a top-20 MBA program
B1 = Fund outperforms the market
B2 = Fund does not outperform the market

A1
A2

B1

B2

.11

.29

.06

.54

E.g. P(A2 and B1) = 0.06

= the probability a fund outperforms the market
and the manager isnt from a top-20 school.

Marginal Probabilities
Marginal probabilities are computed by adding across rows and
down columns; that is they are calculated in the margins of the
table:
P(A2) = .06 + .54
whats the probability a fund
manager isnt from a top school?

B1

B2

P(Ai)

A1
A2

.11

.29

.40

.06

.54

.60

P(Bj)

.17

.83

1.00

P(B1) = .11 + .06

whats the probability a fund
outperforms the market?

BOTH margins must add to 1

(useful error check)

Why do we subtract P(A and B) ?

P(A1) = .11 + .29 = .40
P(B1) = .11 + .06 = .17

By adding P(A1) plus P(B1) we add P(A 1and B1) twice. To correct we subtract P(A
and B) from P(A) + P(B)

B1

A1

B1

B2

P(Ai)

A1
A2

.11

.29

.40

.06

.54

.60

P(Bj)

.17

.83

1.00

P(A1 or B1) = P(A1) + P(B1) P(A1 and B1) = .40 + .17 - .11 = .46

General Addition Rule

Example
Find the probability of selecting a male or a statistics student from the
population described in the following table:
Taking
Stats

Not Taking Stats

Total

Male

84

145

229

Female

76

134

210

160

279

439

Total

P(Male or Stat) = P(M) + P(S) P(M and S)

= 229/439 + 160/439 84/439 = 305/439

Rules of Multiplication
Statistically Independent Events
When occurrence of an event does not affect and is not affected by
the probability of occurrence of any other event, the event is said to
be Statistically independent event.
Independent Events are not affected by previous events.
Eg. You toss a coin and it comes up "Heads" three times ... what is
the chance that the next toss will also be a "Head"?

The chance is simply (or 0.5) just like ANY toss of the coin.
What it did in the past will not affect the current toss! So it is an
independent event

Saying "a Tail is due", or "just one more go, my luck is due" is
called The Gambler's Fallacy

Independent event

The probability of rain today and the probability of a pop

quiz; Quizzes happen, rain or shine.
Choosing a marble from a jar and landing on heads after
tossing a coin.
Choosing a 3 from a deck of cards, replacing it, and then
choosing an ace as the second card.

Non independent event

Eg. You draw one card from a deck and its black and you
draw a second card and its black.
By removing one black card, you made the probability of
drawing a second one slightly smaller. Technically this is
called 'sampling without replacement'.
Taking colored marbles from a bag: as you take each marble
there are less marbles left in the bag, so the probabilities
change.

Rules of Multiplication
Case 1: Statistically independent Events
Joint Probability

The probability of two or more independent events

occurring together or in succession is called joint
probability.
Joint probability of independent events is equal to the
product of their marginal probability.
P (A and B) = P(A) x P(B)
Eg: P(H1 and H2) = P(H) x P(H) = 0.5x 0.5= 0.25

A dresser drawer contains one pair of socks with each of the

following colors: blue, brown, red, white and black. Each pair is
folded together in a matching set. You reach into the sock drawer
and choose a pair of socks without looking. You replace this pair and
then choose another pair of socks. What is the probability that you
will choose the red pair of socks both times?

Roulette (American)
http://www.casinolistings.com/games/playtech/free-american-roulette-game-playtech
http://rouletterus.com/american-roulette/
https://casino.bovada.lv/table-games/roulette?mode=practice

Conditional probability
New information often changes your expectation about the
probability that a particular event will occur.

For example, you may initially think that your football team has an
80% chance of beating its arch rival.
But if you learn that your star striker is injured and will miss the
game, you would probably reduce your initial estimate.

Given two events, A and B, the conditional probability of A

occurring, if B has already occurred, is expressed as P(A|B).
In the football match example,
let A = the event of beating your arch rival team
B = the event that your striker is injured.
Your assumptions are expressed as P(A) = 0.80 and
P(A|B) < 0.80.

Summary
Suppose we have an event A with probability P(A).

If we obtain new information and learn that a related event, denoted by B,

already occurred,
we will want to take advantage of this information by calculating a new
probability for event A.

This new probability of event A is called a conditional probability and is

written

P(A I B)

the notation P(A I B) reads the probability of A given B.

Rules of Multiplication
Case 2: Statistically
dependent Events
When the probability of an event depends upon or affected
by the occurrence of any other event, the event is said to
be statistically dependent .

Joint Probability
If A and B are dependent events, then their joint probability is no
longer equal to the product of their individual probabilities.
P (A and B) = P(A B) P(A) x P(B)

P(A) P(A IB) ; P(B) P(B IA)

P(A B) = P(A) x P(B IA) ;

P(B A) = P(B) x P(A IB)

A card is chosen at random from a standard deck of 52 playing cards.

Without replacing it, a second card is chosen. What is the
probability that the first card chosen is a queen and the second card
chosen is a jack?

Multiplication Rules
Suppose a city council is composed of 5 democrats, 4
republicans, and 3 independents. Find the probability of
randomly selecting a democrat followed by an independent.

P(I and D) P(D) P(I | D) (5/12)(3/11) 5/44 .114

Note that after the democrat is selected (out of 12 people), there are
only 11 people left in the sample space.

Mr. Parietti needs two students to help him with a science

demonstration for his class of 18 girls and 12 boys. He randomly
chooses one student who comes to the front of the room. He then
chooses a second student from those still seated. What is the
probability that both students chosen are girls?

How Do We Check for Independence?

There are three simple ways to check for independence:

Is P(A) P(B) = P(A and B)?

Is P(B|A) = P(B)?

Is P(A|B) = P(A)?
If you answer yes to any one of these three questions then
events A and B are independent.

Statistical Independence
Events A and B are independent if the occurrence of

one event does not affect the probability of the other.

Two events are independent if and only if:

P(A and B) P(A) x P(B)

ie;

P(A | B) P(A)
P(B | A) P(B)

The Case of the Two Classes

The Rochester Health Club prides itself on its ability to find a class for everyone. As a
result, it is extremely popular with both young and old. The Health Club is wondering
how best to market its new yoga class, and the Head of Marketing wonders if
someone who goes swimming is more likely to go to a yoga class. Maybe we could
offer some sort of discount to the swimmers to get them to try out yoga.
The CEO disagrees. I think youre wrong, he says. I think that people who go
swimming and people who go to yoga are independent. I dont think people who go
swimming are any more likely to do yoga than anyone else.
They ask a group of 96 people whether they go to the swimming or yoga classes. Out
of these 96 people, 32 go to yoga and 72 go swimming. 24 people are exceptionally
eager and go to both.
So whos right? Are the yoga and swimming classes dependent or independent?

Solved: The Case of the Two Classes

The CEOs rightthe classes are independent. Heres how he knows.
32 people out of 96 go to yoga classes, so
P(Yoga) = 1/3
72 people go swimming, so
P(Swimming) = 3/4
24 people go to both classes, so
P(Yoga Swimming) = 1/4

So how do we know the classes are independent? Lets multiply together P(Yoga) and
P(Swimming) and see what we get.
P(Yoga) P(Swimming) = 1/3 = 1/4
As this is the same as P(Yoga Swimming), we know that the classes are independent

Are mutually exclusive

events Statistically
independent?
DOES IT MEAN THE SAME???

Dumb Questions
Whats the difference between being independent and being
mutually exclusive?
Are all games on a roulette wheel independent? Why?
Youve shown how a probability tree can demonstrate
independent events. How do I use a Venn diagram to tell if events
are independent?
A Venn diagram really isnt the best way of showing dependence.
Venn diagrams are great if you need to examine intersections and
show mutually exclusive events. Theyre not great for showing
independence though.

Conditional Probability