BASIC CONCEPTS OF
Submitted by:
Pasaoa, Almarie Graceilla C.
Viernes, Aurea Noreen
Orinday, Patrick
Clacio, Joyce Elaine
5.1 Objective and
Subjective Probability
References
http://www.mathsisfun.com/data/probabil
ity.html
http://www.slideshare.net/gsriharsha/fun
damentalsprobability080720091804620
5.2 SAMPLE SPACE
AND EVENTS
DEFINITION: SAMPLE SPACE &
EVENT
SAMPLE
SPACE
A set of possible outcomes of a random
experiment
The symbol S will be used to denote the
sample space
DEFINITION: SAMPLE SPACE &
EVENTS
EVENT
It is a subset of the sample space
May consists of one or more simple events
 EVENTS are denoted using capital letters such as E
EXAMPLE :
A coin is tossed twice and the outcome of
each is recorded. Then,
S= {(H, H), (H, T) , (T, H) (T,T)}
The event that the second toss was a Head is
the subset
E= {(H ,H), (T, H)}
EXAMPLE:
Tossing a die
The sample space is S= {1,2,3,4,5,6} .
E= {2,4,6} is an event, which can be
described in words as the number is even
EXAMPLE:
Tossing a coin twice.
The sample is S= {HH, HT, TH, TT}.
E= {HH, HT} is an event, which can be
described in words as the first toss results
in a Heads
EXAMPLE:
A spinner has 4 equal sectors colored
yellow, blue, green, and red. What is the
probability of landing on each color after
spinning this spinner?
SAMPLE SPACE: {yellow, blue, green, red}
REFERENCES:
http://cims.nyu.edu/~kiryl/Probability/Chap
ter%202.pdf
http://www.math.uiuc.edu/~kkirkpat/Sample
Space.pdf
http://www.mathgoodies.com/lessons/vol6/s
ample_spaces.html
5.3 AXIOMS
OF
PROBABILITY
REFERENCES
http://www.math.utah.edu/~lzhang/teaching/3070summer2008/
DailyUpdates/jun10/sec2_2.pdf
http://www.slideshare.net/mathscontent/probabilityanditsaxioms3209538
5.4 Some Important Theorems on
Probability
5.4 Some Important Theorems on Probability
5.4 Some Important Theorems on Probability
5.4 Some Important Theorems on Probability
5.4 Some Important Theorems on Probability
Examples
Experiment 1:
A single 6sided die
is rolled. What is the
probability of rolling a
2 or a 5?
Probabilities:
1
P(2)
1
P(5)
Experiment 1:
P(2 or 5)
P(2)
A single 6sided die is rolled. What is
the probability of rolling a 2 or a 5?
+
1
=
2
=
P(5)
1
+
Examples
Experiment 2:
On New Year's Eve, the probability of a
person having a car accident is 0.09. The
probability of a person driving while
Experiment 6:
On New Year's
Eve, the is 0.32 and probability of a
intoxicated
probability of person
a person
having a car accident while
having a car accident
is is 0.15. What is the probability of
intoxicated
0.09. The probability
a
a personofdriving
while intoxicated or having a
person drivingcar
while
accident?
intoxicated is 0.32 and
probability of a person
P(intoxicated
having a car accident
+
P(accident)

and
P(intoxicated or = P(intoxicated)
while intoxicated is 0.15.
accident)
accident)
What is the probability of
a person driving while
intoxicated or having a
car accident?
Probabilities:=
0.32
+
0.09

0.15
=
0.26
Examples
Example: Rolling a "5" or "6"
Event A: {5, 6}
Number of ways it can happen: 2
Total number of outcomes: 6
The Complement of Event A is {1, 2, 3, 4}
Number of ways it can happen: 4
Total number of outcomes: 6
Let us add them:
Yep, that makes 1
It makes sense, right? Event A plus all outcomes that are not Event A make up all possible outcomes.
2
P(A')
= =
P(A)
4/6
=
=
2/3
P(A) + P(A') =
1/3
2/3
3/3
= 1
Examples
Example: Scoring Goals
If the probability of:
scoring no goals (Event "A") is20%
scoring exactly 1 goal (Event "B") is15%
Then:
The probability of scoring no goalsand1 goal
is0(Impossible)
The probability of scoring no goalsor1 goal is 20% +
15% =35%
Which is written:
P(AB) = 0
P(AB) = 20% + 15% = 35%
Examples
Example: A Deck of Cards
In a Deck of 52 Cards:
the probability of a King is 1/13, soP(King)=1/13
the probability of an Ace is also 1/13, soP(Ace)=1/13
When we combine those two Events:
The probability of a card being a Kingandan Ace
is0(Impossible)
The probability of a card being a Kingoran Ace is
(1/13) + (1/13) =2/13
Which is written like this:
P(King and Ace) = 0
P(King or Ace) = (1/13) + (1/13) = 2/13
Examples
Example: 16 people study French,
21 study Spanish and there are 30
altogether. Work out the
probabilities!
This is definitely a case
ofnotMutually Exclusive (you can
study French AND Spanish).
Let's saybis how many study both
languages:
people studying French Only must
be16b
people studying Spanish Only must
be21b
And we get:
And we can put in the correct
numbers:
So we know all this now:
P(French) = 16/30
P(Spanish) = 21/30
P(French Only) = 9/30
P(Spanish Only) = 14/30
P(French or Spanish) = 30/30 = 1
And we know there are30people,
so:
(16b) + b + (21b) = 30
37 b = 30
P(French and Spanish) = 7/30
Lastly, let's check with our formula:
P(A or B) = P(A) + P(B) P(A
and B)
Put the values in:
30/30 = 16/30 + 21/30 7/30
Activities
1. A number is chosen at random from the set of twodigit numbers
from 10 to 99 inclusive.
What is the probability the number contains at least one digit 2?
2. There are 30 children in a class and they all have at least one cat or
dog.
14 children have a cat, 19 children have a dog.
What is the probability that a child chosen at random from the class
has both a cat and a dog?
3. A fair die is thrown. What is the probability that the score is not a
factor of 6?
4. A spinner is made from a piece of card in the shape of a regular
pentagon with a toothpick pushed through the center. When the
spinner is spun and it lands on an edge, each of the numbers from 1
to 5 is equally likely.
If the spinner is spun twice, what is the probability the two scores are
different?
5. A day of the week is chosen at random. What is the probability of
choosing a Monday or Tuesday?
6. In a pet store, there are 6 puppies, 9 kittens, 4 gerbils and 7
parakeets. If a pet is chosen at random, what is the probability of
Interesting Trivia
Each time you inhale, you have a
greater than 99.9% chance of
inhaling at least 1 molecule
exhaled by Caesar's last dying
breath.
References
http://www.mathgoodies.com/lessons/vol6/addition_rules.html
https://www.mathsisfun.com/data/probabilitycomplement.html
http://www.mathsisfun.com/data/probabilityeventsmutuallyexclusive.html
5.5 Conditional
Probability
and Independent
Events
When dealing with two events (usually called A
and B), sometimes the events are so related to
each other, that the probability of one depends
on whether the other event has occurred.
When we talk about probabilities based on the
fact that something else has already happened
we call this conditional probability.
What changes when dealing with conditional
probability is that we know for certain that
something else has already happened. This
means that in our definition of probability that
says
where the total number of ways is based on
the fact that we know something else has
already occurred.
There are two ways to approach conditional
probability and depends on the type of problem
that you are given. In a situation where you are
given percentages and probabilities (usually but
not always in a table format) we make use of
the conditional probability rule. In situations
where you are trying to compute probabilities on
your own (instead of them being given to you)
most of the time it is easier to not to use the
formula.
Conditional Probability Rule:
Consider events A and B.
The line between A and B is read given.
So translated, this reads, "the probability of
A given that B has happened." The event
on the right side of the line is the event
that has already happened.
What The Rule Means:
This rule is applied when you have two events
and you already know the outcome of one of the
events. In doing the computations, you will need
to be able to find the probability of A and B, that
is, P(AB). Problems of this type make use of the
multiplication rule. If you need help with the
multiplication rule or understanding what type of
problems make use of the rule.
References:
http://www.regentsprep.or
g/regents/math/algebra/AP
R3/Lconditional
http://www.cuttheknot.org/Curriculum/Proba
bility/ConditionalProbabilit
y
https://www.mathsisfun.co
m/data/probabilityeventsconditional
http://www.wyzant.com/re
sources/lessons/math/stati
stics_and_probability/prob
ability/further_concepts_in
_probability
EXAMPL
E #1:
1. There is a 2/5 chance of pulling out a
Blue marble, and a 3/5 chance for Red:
We can even go one step further
and see what happens when we
select a second marble
If a blue marble was selected first there is now a 1/4 chance
of getting a blue marble and a 3/4 chance of getting a red
marble.
If a red marble was selected first there is now a 2/4 chance
of getting a blue marble and a 2/4 chance of getting a red
marble.
"What are the chances of drawing 2 blue marbles?"
Answer: it is a 2/5 chance followed by
a 1/4 chance:
The chances of drawing 2 blue
marbles is 1/10.
P(A) means "Probability Of Event A"
In our marbles example Event A is "get a Blue Marble
first" with a probability of 2/5:
P(A) = 2/5
And Event B is "get a Blue Marble second" ... but for
that we have 2 choices:
If we got a Blue Marble first the chance is now 1/4
If we got a Red Marble first the chance is now 2/4
So we have to say which one we want, and use the
symbol "" to mean "given":
P(BA) means "Event B given Event A"
In other words, event A has already happened, now what is
the chance of event B?
P(BA) is also called the "Conditional Probability" of B given A.
And in our case:
P(BA) = 1/4
So the probability of getting 2 blue
marbles is:
And we write it as:
Probability of event A and event B equals
the probability of event A times the
probability of event B given event A"
EXAMPLE
#2
In a school of 1200 students,
250 are seniors, 150 students
take math, and 40 students are
seniors and are also taking
math. What is the probability
that a randomly chosen student
who is a senior, is taking
math?
These questions can be
confusing. It sounds, at first
read, that they are asking for the
probability of choosing a student
who is a senior and who is taking
math.
Answer:
It helps to reword the question into:
Find the probability that the student is
taking math, given that the student is a
senior.
B = the student is taking
math
n(A) = the student is a senior =
250.
n(A and B) = the student is a senior
and is taking math = 40.
SOLUTION:
EXAMPLE
#3
In a card game, suppose a player needs to draw
two cards of the same suit in order to win. Of
the 52 cards, there are 13 cards in each suit.
Suppose first the player draws a heart. Now the
player wishes to draw a second heart. Since one
heart has already been chosen, there are now
12 hearts remaining in a deck of 51 cards. So
the conditional probability P(Draw second
heartFirst card a heart) = 12/51.
Suppose an individual applying to a college
determines that he has an 80% chance of being
accepted, and he knows that dormitory housing
will only be provided for 60% of all of the
accepted students. The chance of the student
being accepted and receiving dormitory housing
is defined by
P(Accepted and Dormitory Housing) =
P(Dormitory HousingAccepted)P(Accepted) =
(0.60)*(0.80) = 0.48.
LET'S
PRACTICE!
1. A survey of 500 adults asked about
college expenses. The survey asked
questions about whether or not the
person had a child in college and about
the cost of attending college. Results
are shown in the table below.
Suppose one person is chosen at
random. Given that the person has a
child in college, what is the probability
that he or she ranks the cost of
attending college as cost too much?
2. Suppose you draw two
cards from a standard deck
without replacement. Given
that the first card is an ace,
what is the probability that
the second card is a queen?
ANSWERS
:
1. P(cost too much child in college) =
P(cost too muchchild in college) can be found from the
table as 0.30
P(child in college) can be found by adding 0.30 + 0.13
+ 0.01 = 0.44
Substituting these values into the equation for
conditional probability we get
2. P(ace queen) =
P(acequeen) is found using the multiplication
rule. Since the problem states that there is no
replacement we can find
P(acequeen) as (4/52)(4/51) = 4/663
We know that
P(queen) = 4/52
Substituting values into the conditional
probability formula we get
5.6 Bayes
Theorem
relates current probability to prior probability
Used to find probabilities of various events.
A big advantage of a Bayesian approach
Allows a principled approach to the exploitation of
all available data
with an emphasis on continually updating ones
models as data accumulate
as seen in the consideration of what is learned
from a positive mammogram
Bayesian Reasoning
ASSUMPTIONS
1% of women aged forty who participate in a routine
screening have breast cancer
80% of women with breast cancer will get positive
tests
9.6% of women without breast cancer will also get
positive tests
EVIDENCE
A woman in this age group had a positive test in a
routine screening
PROBLEM
Whats the probability that she has breast cancer?
Bayesian Reasoning
ASSUMPTIONS
10 out of 1000 women aged forty who participate in a
routine screening have breast cancer
800 out of 1000 of women with breast cancer will get
positive tests
95 out of 1000 women without breast cancer will also
get positive tests
PROBLEM
If 1000 women in this age group undergo a routine
screening, about what fraction of women with positive
tests will actually have breast cancer?
Bayesian Reasoning
ASSUMPTIONS
100 out of 10,000 women aged forty who participate
in a routine screening have breast cancer
80 of every 100 women with breast cancer will get
positive tests
950 out of 9,900 women without breast cancer will
also get positive tests
PROBLEM
If 10,000 women in this age group undergo a routine
screening, about what fraction of women with positive
tests will actually have breast cancer?
Bayesian Reasoning
Before the screening:
100 women with breast cancer
9,900 women without breast cancer
After the screening:
A = 80 women with breast cancer and positive test
B = 20 women with breast cancer and negative test
C = 950 women without breast cancer and positive test
D = 8,950 women without breast cancer and negative test
Proportion of cancer patients with positive results, within
the group of ALL patients with positive results:
A/(A+C) = 80/(80+950) = 80/1030 = 0.078 = 7.8%
Compact Formulation
C = cancer present, T = positive test
p(AB) = probability of A, given B, ~ = not
PRIOR PROBABILITY
p(C) = 1%
PRIORS
CONDITIONAL PROBABILITIES
p(TC) = 80%
p(T~C) = 9.6%
POSTERIOR PROBABILITY (or REVISED PROBABILITY)
p(CT) = ?
Bayesian Reasoning
Before the screening:
100 women with breast cancer
9,900 women without breast cancer
After the screening:
A = 80 women with breast cancer and positive test
B = 20 women with breast cancer and negative test
C = 950 women without breast cancer and positive test
D = 8,950 women without breast cancer and negative test
Proportion of cancer patients with positive results, within
the group of ALL patients with positive results:
A/(A+C) = 80/(80+950) = 80/1030 = 0.078 = 7.8%
Bayesian Reasoning
Prior Probabilities:
100/10,000 = 1/100 = 1% = p(C)
9,900/10,000 = 99/100 = 99% = p(~C)
Conditional Probabilities:
A = 80/10,000 = (80/100)*(1/100) = p(TC)*p(C) = 0.008
B = 20/10,000 = (20/100)*(1/100) = p(~TC)*p(C) = 0.002
C = 950/10,000 = (9.6/100)*(99/100) = p(T~C)*p(~C) = 0.095
D = 8,950/10,000 = (90.4/100)*(99/100) = p(~T~C) *p(~C) = 0.895
Rate of cancer patients with positive results, within the
group of ALL patients with positive results:
A/(A+C) = 0.008/(0.008+0.095) = 0.008/0.103 = 0.078 = 7.8%
> Bayes theorem
A
p(TC)*p(C)
p(CT) = ______________________
P(TC)*p(C) + p(T~C)*p(~C)
A + C
Comments
Common mistake: to ignore the prior probability
The conditional probability slides the revised
probability in its direction but doesnt replace the
prior probability
A NATURAL FREQUENCIES presentation is one in
which the information about the prior probability is
embedded in the conditional probabilities (the
proportion of people using Bayesian reasoning rises to
around half).
Test sensitivity issue (or: if two conditional
probabilities are equal, the revised probability equals
the prior probability)
Where do the priors come from?
> Bayes theorem
p(XA)*p(A)
p(AX) = ______________________
P(XA)*p(A) + p(X~A)*p(~A)
Given some phenomenon A that we want to investigate, and an
observation X that is evidence about A, we can update the
original probability of A, given the new evidence X.
ACTIVITIES
Suppose a drug test is 99%sensitiveand 99%
specific. That is, the test will produce 99% true
positive results for drug users and 99% true
negative results for nondrug users. Suppose that
0.5% of people are users of the drug. If a
randomly selected individual tests positive, what
is theprobabilityhe or she is a user?
P(User +)= P(+ User) P(User)
P(+ User) P(User) + P(+
No=user)
P(Nonuser
Answer
=
0.99 x 0.005
0.99 x 0.005 + 0.01 x 0.995
References
Statistics: A Bayesian Perspective D. Berry, 1996, Duxbury Press.
o excellent introductory textbook, if you really want to understand what
its all about.
http://ftp.isds.duke.edu/WorkingPapers/9721.ps
o Using a Bayesian Approach in Medical Device Development, also by
Berry
http://www.pnl.gov/bayesian/Berry/
o a powerpoint presentation by Berry
http://yudkowsky.net/bayes/bayes.html
o Extremely clear presentation of the mammography example; highly
polemical and fun too!
http://www.stat.ucla.edu/history/essay.pdf
o Bayes original essay
Jaynes, E. T., 1956, `Probability Theory in Science and
Engineering,' (No. 4 in `Colloquium Lectures in Pure and Applied
Science,' SoconyMobil Oil Co. USA.
http://bayes.wustl.edu/etj/articles/mobil.pdf
o A physicists take on Bayesian approaches. Proposes an interesting
metric of probability using decibels (yes, the unit used for sound
levels!).
http://www.sportsci.org/resource/stats/
o a skeptical account of Bayesian approaches. The rest of the site is very
informative and sensible about basic statistical issues.
5.7 CONTINGENCY
TABLES
OVERVIEW
Hypotheses of equal proportions
Hypotheses of independence
Exact distributions and Fishers test
The Chi squared approximation
Median test
Measures of dependence
The Chi squared goodnessoffit test
Cochrans test
CONTINGENCY TABLE EXAMPLES
Countries  religion by government
States dominant political party by
geographic region
Mutual funds  style by family
Companies  industry by location of
headquarters
MORE EXAMPLES Countries  government by GDP categories
States  divorce laws by divorce rate categories
Mutual funds  family by Morning Star rankings
Companies  industry by price earnings ratio
category
STATISTICAL INFERENCE HYPOTHESIS OF
EQUAL PROPORTIONS
H0: all probabilities (estimated by proportions,
relative frequencies) in the same column are
equal,
H1:at least two of the probabilities in the same
column are not equal
Here, for an r x c contingency table, r populations
are sampled with fixed row totals, n1, n2, nr.
HYPOTHESIS OF INDEPENDENCE
H0: no association
i.e. row and column variable are independent,
H1: an association,
i.e. row and column variable are not independent
Here, one populations is sampled with
sample size N. Row totals are random
variables.
EXACT DISTRIBUTION FOR 2 X 2 TABLES:
HYPOTHESIS OF EQUAL PROPORTIONS; N1 =
N2 = 2
FISHERS EXACT TEST
For 2 x 2 tables assuming fixed row and column
totals r, Nr, c, Nc:
Test statistic = x, the frequency of cell11
Probability = hypergeometric probability of x
successes in a sample of size r from a population
of size N with c successes
LARGE SAMPLE APPROXIMATION FOR
EITHER TEST
Chi squared
= Observed  Expected]2 /Expected
Observed frequency for cell ij comes from crosstabulation of data
Expected frequency for cell ij
= Probability Cell ij * N
Degrees of freedom (r1)*(c1)
COMPUTING CELL PROBABILITIES
Assumes independence or equal probabilities (the
null hypothesis)
Probability Cell ij = Probability Row i
* Probability Column j
= (R i/N) * (C j/N)
Expected frequency ij = (R/N)*(C/N)*N
= R*C/N.
DISTRIBUTION OF THE SUM
Chi Square with (r1)*(c1) degrees of freedom
Assumes
Observed  Expected]2 /Expected
is standard normal squared
Implies
Observed  Expected] /Square root[Expected]
is standard normal
Implies
and Observed is a Poisson RV
Poisson is approximately normal if > 5, traditional guideline
Conovers relaxed guideline page 201
MEASURES OF STRENGTH:
CATEGORICAL VARIABLES
Phi 2x2
Cramer's V for rxc
Pearson's Contingency Coefficient
Tschuprow's T
MEASURES OF
STRENGTH: ORDINAL
VARIABLES
Lambda A .. Rows dependent
Lambda B .. Columns dependent
Symmetric Lambda
Kendall's tauB
Kendall's tauC
Gamma
STEPS OF STATISTICAL ANALYSIS
Significance  Strength
1 Test for significance of the observed association
2  If significant, measure the strength of the
association
CONSIDER THE CORRELATION
COEFFICIENT
a measure of association (linear relationship
between two quantitative variables)
significant but not strong
significant and strong
not significant but strong
not significant and not strong
R AND PROB (PVALUE)
r
r
r
r
=
=
=
=
.20
.90
.90
.20
pvalue
pvalue
pvalue
pvalue
<
<
>
>
.05
.05
.05
.05
CONCEPTS
Predictive associations must be both
significant and strong
In a particular application, an association may
be important even if it is not predictive (I.e.
strong)
MORE CONCEPTS
Highly significant , weak associations result from
large samples
Insignificant strong associations result from
small samples  they may prove to be either
predictive or weak with larger samples
EXAMPLES
Heart attack Outcomes by Anticoagulant
Treatment
Admission Decisions by Gender
SUMMARY
Is there an association?
Investigate with Chi square pvalue
If so, how strong is it?
Select the appropriate measure of strength of
association
Where does it occur?
Examine cell contributions