PROBABILITY & STATISTICAL

INFERENCE LECTURE 2
MSc in Computing (Data Analytics)

Lecture Outline
Introduction

Introduction to Probability Theory

Discrete Probability Distributions

Question Time

Introduction
Probability & Statistics
We want to make decisions
based on evidence from a
sample i.e. extrapolate
from sample evidence to a
general population
To make such decisions we
need to be able to
quantify our (un)certainty
about how good or bad
our sample information is.

Population
Representative
Sample
Sample
Statistic
Describe
Make
Inference
Example: How many voters will give F.F. a first preference in the
next general election ?

researcher A takes a sample of size 10 and find 4 people who
say they will

-researcher B takes a sample of size 100 and find 25 people
who say they will

Researcher A => 40%
Researcher B => 25%

Who would you believe?
Probability & Statistics - Example
Probability & Statistics - Example
Example: How many voters will give F.F. a first preference in the next
general election ?

researcher A takes a sample of size 10 and find 4 people who say
they will

- researcher B takes a sample of size 100 and find 25 people who
say they will

Researcher A => 40%
Researcher B => 25%

Who would you believe?

Probability & Statistics - Example
Intuitively the bigger sample would get more credence
but how much better is it, and are either of the
samples any good?

Probability helps

Descriptive Statistics are helpful but still lead to
decision making by 'intuition

Probability helps to quantify (un)certainty which is a
more powerful aid to the decision maker


Probability & Statistics


Using probability theory we can measure
the amount of uncertainty/certainty in
our statistics.
Intuitions and Probability Lotto
example
If you had an Irish lotto ticket which of these sets of
numbers is more likely to win:

1. 1 2 3 4 5 6

Odds of winning are 1 in 8145060

2. 2 11 26 27 35 42

Odds of winning are 1 in 8145060
Intuitions and Probability Disease
example
Suppose we have a diagnostic test for a disease which
is 99% accurate.

A person is picked at random and tested for the
disease

The test gives a positive result. What is the probability
that the person actually has the disease?

99% ?
Disease example
Test Results
Those that dont have/do
have the disease
If you take a population
of 1,000,000
1,000,000
999,900
989,991 9,999
100
99 1
No!! IT depends on how common or rare the disease is.
Suppose the disease affects 1 person in 10,000
Of those who test positive only have the
disease
0098 . 0
9999 99
99
=
+
Introduction to Probability Theory
Some Definitions
An experiment that can result in different outcomes, even though it is
repeated in the same manner every time, is called a random
experiment.
The set of all possible outcomes of a random experiment is called
the sample space of an experiment and is denote by S
Example:
Experiment: Toss two coins and observe the up face on each
Sample Space:
1. Observe HH
2. Observe HT
3. Observe TH
4. Observe TT
S : {HH,HT,TH,TT}

Some Definitions
A sample space is discrete if it consists of a finite or
countable infinite set if outcomes

A sample space is continuous if it contains an
interval or real numbers

An event is a subset of the sample space of a
random experiment & we generally calculate the
probability of a certain event accurring
Counting
A permutation of the elements is an ordered
sequence of the elements.
Example: S : {a,b,c}
All the permutations of the elements of S are abc, acb,
bca, bac, cba & cab.

The number of permutations of n different elements
is n!, where:
n! = n * (n-1) * (n-2) * .......* 2 * 1

Above n=3 => 3! = 3 * 2 * 1 = 6


Counting
The number of permutations of subsets r elements
selected from a set of n different elements is



Where order is not important when selecting r
elements from a set of n different elements is called
a combination:




)! (
!
r n
n
P
n
r

=
)! ( !
!
r n r
n
C
n
r

=
Probability
Whenever a sample space consists of N Possible outcomes that are
equally likely, the probability of the outcome 1/N.

For a discrete sample space, the probability of an event E, denoted
by P(E), equals the sum of the probabilities of the outcome in E.

Some rules for probabilities:
For a given sample space containing n event sE
1
, E
2
, ....,E
n
1. All simple event probabilities must lie between 0 and 1:
0 <= P(E
i
) <= 1 for i=1,2,........,n
2. The sum of the probabilities of all the simple events within a
sample space must be equal to 1:



1 ) (
1
=

=
n
i
i
E P
Probability Example 1
Example:
Experiment: Toss two coins and observe the up
face on each
Sample Space: S : {HH,HT,TH,TT}
Probability of each event:
1. E = HH => P(HH) = 1/4
2. E = HT => P(HT) = 1/4
3. E = TH => P(TH) = 1/4
4. E = TT => P(HH) = 1/4

Probability Example 1
The probability of an event A is equal to the sum of all
the probabilities in event A:
Example:
Experiment: Toss two coins and observe the up face on each
Event A: {Observe exactly one head}

P(A) = P(HT) + P(TH) = + =

Event B : {Observe at least one head}

P(B) = P(HH) + P(HT) + P(TH) = + + =





Probability Example 2
Below is the probability distribution of a random variable S for
the sum of values obtained by rolling two dice
Compound Events
The union of two event A and B is the event that
occurs if either A or B, or both, occur on a single
performance of the experiment denoted by A U B (A
or B)
The intersection of two events A and B is the event
that occurs if both A and B occur on a single
performance of an experiment denoted by A B or
(A and B)

Compound Events
Example: Consider a die tossing experiment with equally
likely simple events {1,2,3,4,5,6}. Define the events A, B and C.

A:{Toss an even number} = {2,4,6}
B:{Toss a less than or equal to 3} = {1,2,3}
C:{Toss a number greater than 1} = {2,3,4,5,6}
Find:

) (
) (
C B A P
and
B A P

Complementary Event
The complementary of an event A is the event that
A does not occur denoted by A

Note that AU A` = S, the sample space

P(A) + P(A`) =1 => P(A) = 1 P(A`)
Questions
1. What is the sample space when a coin
is tossed 3 times?
2. What is the probability of tossing all heads or
all tails.
3. What is the sample space of throwing
a fair die.
4. If a fair die is thrown what is the probability
of throwing a prime number (2,3,5)?



Questions
4. A factory has two assembly lines, each of which is shut down (S), at partial
capacity (P), or at full capacity (F). The following table gives the sample
space





For where (S,P) denotes that the first assembly line is shut down and the
second one is operating at partial capacity. What is the probability that:
a) Both assembly lines are shut down?
b) Neither assembly lines are shut down
c) At least one assembly line is on full capacity
d) Exactly one assembly line is at full capacity

Event A P(A) Event A P(A) Event A P(A)
(S,S) 0.02 (S,P) 0.06 (S,F) 0.05
(P,S) 0.07 (P,P) 0.14 (P,F) 0.2
(F,S) 0.06 (F,P) 0.21 (F,F) 0.19
Conditional Probability
The conditional probability of event A conditional on
event B is


for P(B)>0. It measures the probability that event A
occurs when it is known that event B occur.
Example: A = odd result on die = {1,3,5}
B = result > 3 = {4,5,6}




) (
) (
) | (
B P
B A P
B A P

=
3
1
6
3
6
1
) | ( = = B A P
Conditional Probability Example
Example: A study was carried out to investigate the link
between peoples lifestyles and cancer. One of the
areas looked at was the link between lung cancer and
smoking. 10,000 people over the age of 55 were
studied over a 10 year period. In that time 277
developed lung cancer.




What is the likelihood of somebody developing lung
cancer given that they smoke?

Cancer No Cancer Total
Smoker 241 3,325
3,566
Non-Smoker 36 6,398
6,434
Total 277 9,723 10,000
Conditional Probability Example
Event A: A person develops lung cancer
Event B: A person is a smoker

P(A) = 277/10,000 = 0.027
P(B) = 3,566/10,000 = 0.356




068 . 0
3566 . 0
0241 . 0
) (
) (
) | ( = = =
B P
B A P
B A P

0241 . 0 000 , 10 / 241 ) ( = = B A P
Exercises
1. A ball is chosen at random from a bag containing
150 balls that are either red or blue and either
dull or shinny. There are 36 red, shiny balls and
54 blue balls. There are 72 dull balls.
1. What is the probability of a chosen ball being shiny
conditional on it being red?
2. What is the probability of a chosen ball being dull
conditional on it being blue?
Mutually Exclusive Events
Two events, A and B, are mutually exclusive given
that if A happens then B cant also happen.

Example: Roll of a die
A = less than 2
B = even result
There is no way that A and B can happen at the same
time therefore they are mutually exclusive events

Rules for Unions
Additive Rule:


Additive Rule for Mutually Exclusive Events

) ( ) ( ) ( ) ( B A P B P A P B A P + =
) ( ) ( ) ( B P A P B A P + =
Example
Records at an industrial plant show that 12% of all
injured workers are admitted to hospital for
treatment, 16% are back on the job the next day,
and 2% are both admitted to a hospital for
treatment and back to work the next day. If a
worker is injured what is the probability that the
worker will be either admitted to hospital or back
on the job the next day or both?
Independent Events
Events A and B are independent if it is the case that A
happening does not alter the probability that B happens.

Example : A = even result on die
B = result > 2

Then, let us say we are told the result on the die (which
someone has observed but not us) is even so knowing this,
what is the probability that the event B has happened?
Sample space: {2, 4, 6}

B = 4 or 6 => P(B) = 2/3

Independent Events
But if we didnt know about the even result we would
get:
Sample space: {1, 2, 3, 4, 5, 6}

B = 3 or 4 or 5 or 6 => P(B) = 4/6 = 2/3

so knowledge about event A has in no way changed
out probability assessment concerning event B

Rules for Intersection
Multiplicative Rule of Probability



If events A and B are independent then




) ( ) | ( ) ( ) | ( ) ( A P A B P B P B A P B A P = =
) ( ) ( ) ( B P A P B A P =
Bayes Theorem
One of a number of very useful results: - here is simplest definition:

Suppose: You have two events which are ME and exhaustive i.e. account
for all the sample space
Call these events A and event (read not A).

Further suppose there is another event B, such that
P(B|A) > 0 and P(A|B) > 0.

Then Bayes theorem states:

) ' ( ) ' | ( ) ( ) | (
) ( ) | (
) | (
A P A B P A P A B P
A P A B P
B A P
+
=
Discrete Probability Distributions

Some Definitions Random Variables
A random variable is a function that assigns a real
number to each outcome in the sample space of a
random experiment
For example the random variable X is assigned the
number 1 if it rains tomorrow and 0 if it does not
rain tomorrow
Random Variable Example
In statistics we write this example as:


Another Example: The random variable Y is equal
to the amount of rain in inches that is likely to fall
tomorrow

=
row rain tomor not does it if 0
tomorrow rains it if 1
X
row rain tomor of inches of number Y=
Types Of Random Variables
Random
Variables
Discrete
(finite range)
Will it rain tomorrow?
Range X{0,1}
Continuous
(infinite range)
Amount of rain tomorrow:
Range Y[0,2.5 inches]
Probability Distributions
The function that describes a random variable is
called a probability distribution
For discrete random variables the probability
distribution is described by a probability mass
function
For continuous random variables the probability
distribution is described by a probability density
function
Discrete Random Variable
A Random Variable (RV) is obtained by assigning a
numerical value to each outcome of a particular
experiment.

Probability Distribution: A table or formula that
specifies the probability of each possible value for
the Discrete Random Variable (DRV)

DRV: a RV that takes a whole number value only

Probability Example 2
Below is the probability distribution of a random variable S for
the sum of values obtained by rolling two dice
Probability Example 2
Below is the probability distribution of a random variable S for
the sum of values obtained by rolling two dice
Example: What is the probability distribution for the experiment to assess the
no of tails from tossing 2 coins;

Sample Space
Coin 1 Coin 2
T T
T H
H T
H H

x = no. of tails is the RV
x P(x)
0 = P(HH) = 0.25
1 = P(TH) + P(HT) = 0.50
2 = P(TT) = 0.25
P( any other value ) = 0

N.B. E P(x) = 1
0 s P(x) s 1 for all values of x
Mean of a Discrete Random Variable
Mean of a DRV = = x * p(x)
Example: Throw a fair die


x P(x) x * P(x)
1 0.1667 0.17
2 0.1667 0.33
3 0.1667 0.50
4 0.1667 0.67
5 0.1667 0.83
6 0.1667 1.00
P(any other value) = 0 0


Mean = = x * p(x) = 3.5
Standard Deviation of a DRV

= = =
= =
2 2
2 2
) ) ( ( ) (
) (
x X xP x X P x
x X P x
o
o
x P(x) x
2 *
P(x)
1 0.1667 0.17
2 0.1667 0.67
3 0.1667 1.50
4 0.1667 2.67
5 0.1667 4.17
6 0.1667 6.00
P(any other value) = 0 0
= 15.17

15.17 - (3.5)
2
= 15.17 - 12.25 = 2.92
=> S.D. = 1.71
Example: Rolling one die
Binomial (Probability) Distribution
Many experiments lead to dichotomous responses (i.e.
either success/failure, yes/no etc.)
Often a number of independent trials make up the
experiment
Example: number of people in a survey who agree
with a particular statement?
Survey 100 people => 100 independent trials of
Yes/No
The random variable of interest is the no. of successes
(however defined)
These are Binomial Random Variables

4 people tested for the presence of a particular gene.
success = presence of gene

P(gene present / success) = 0.55 P(gene absent / failure) = 0.45

P(3 randomly tested people from 4 have gene)?

Assume trials are independent - e.g. the people are not related

There is 4 ways of getting 3 successes
Binomial Distribution Example
Binomial Distribution Example

Using Independence rule we can calculate the probability
of each outcome:

Outcome 1: 0.55 0.55 0.55 0.45 = 0.07486875
Outcome 2: 0.55 0.55 0.45 0.55 = 0.07486875
Outcome 3: 0.55 0.45 0.55 0.55 = 0.07486875
Outcome 4: 0.45 0.55 0.55 0.55 = 0.07486875

4 ways of getting result each with P=0.07486875
=> 4 0.07486875 = 0.299475
=> P(3 randomly tested people have gene) = 0.299475

Binomial Distribution Example
A more convenient way of mathematically writing the
same result is as follows:


the number of ways you can get three successes from
4 trials is a combination:




2994 . 0 ) 45 . 0 ( ) 55 . 0 (
1 3
=
|
|
.
|

\
|
3
4
! )! (
!
r r n
n
r
n
C
n
r

=
|
|
.
|

\
|
=
Binomial Distribution General
Formula
This all leads to a very general rule for calculating binomial probabilities:

In General Binomial (n,p)
n = no. of trials
p = probability of a success
x = RV (no. of successes)




Where P(X=x) is read as the probability of seeing x
successes.

x n x
p p
x
n
x X P

|
|
.
|

\
|
= = ) 1 ( ) (
? ) 4 (
? ) 3 (
? ) 2 (
? ) 1 (
0410065 . 0 ) 45 . 0 ( ) 55 . 0 (
0
4
) 0 (
0 4 0
= =
= =
= =
= =
=
|
|
.
|

\
|
= =

X P
X P
X P
X P
X P
Binomial Distribution
For all binomials the mean is given by the simple formula;
= n p

Example: from previous example
= 4 0.55 = 2.2

Standard deviation also has simple formula for all Binomials





Example: from previous example = 0.995
) 1 (
2
p np = o
) 1 ( p np = o
o
Binomial Distribution
What is P(< 3 people have gene) from a group of four people
tested at random?
Use the fact that the possible outcome are mutually exclusive
(ME)

= P(0) + P(1) + P(2)
= 0.041 + 0.2 + 0.368
= 0.609 [ to 3 decimal places ]
We can write this probability like this;


P(X>3)=?


( ) 609 . 0 45 . 0 ) 55 . 0 (
4
) 3 (
4
2
0
=
|
|
.
|

\
|
= <

=

x
x
x
x
X P
Binomial Question
There are two hospitals in a town. In Hospital A, 10
babies are born each day, in Hospital B there are 30
babies born each day. If the hospitals only count
those days on which over 70% of babies born are
girls, and assuming the probability that a girl is born
is , which of the two hospitals will record more such
days?
Hospital A: Binomial (n=10, p=0.5)

Hospital B: Binomial (n=30, p=0.5)


Answer
Hospital 1:
Calculate :



Hospital 2 :
Calculate :




There is a higher probability of getting 70% of babies born being
girl from hospital 1.
0.17188 0.828125 1
) ( - 1 7) P(X
6
0
=
= = >=

= i
i
x X P
0.02139 0.978613 1
) ( - 1 21) P(X
20
0
=
= = >=

= i
i
x X P
Binomial Question
A flu virus hits a company employing 180 people.
Independent of other employees , there is a
probability p=0.35 that each person needs to take
sick leave. What is the expectation and variance of
the proportion of the workforce who needs to take
sick leave. In general what is the value of the sick
rate p that produces the largest variance for this
proportion.
Poisson Probability
Many experiments don't have a simple success/failure
response
Responses can be the number of events occurring over time,
area, volume etc.
We don't know the number of 'failures' just the number of
successes.
Example: The number of calls to a telesales company
- we know how many calls got through (successes)
- but don't know how many failed (lines busy etc.)
Knowledge of the mean number of events over time etc =>
Poisson Random Variable
Events must occur randomly

Poisson Probability Distribution
Probability Distribution for Poisson
Where is the known mean:





x is the value of the RV with possible values 0,1,2,3,.
e = irrational constant (like t) with value 2.71828

The standard deviation , o, is given by the simple relationship;

o =

!
) (
x
e
x X P
x

= =

Example: Bombing of London WW2

1944 German V1 rockets feel on London
Were they aimed at specific targets or falling
randomly?
Important in AA strategy & Civil Defence
Divide London into
a 24 24 grid of
equal sizes (576
equal square areas).

Example: Bombing of London WW2
If rockets are random => should fall according to Poisson
random variable per square

(mean) = No. of Bombs/ No of squares
= 535/576
= 0.9288
So, for a particular square (assuming randomness)


Where x is the number of bombs landing in the square on the
map grid.


!
) 9288 . 0 (
) (
9288 . 0
x
e
x X P
x
= =
( )
( )
( )
( )
( )
003 . 0 997 . 0 1
) 4 ( ) 3 ( ) 2 ( ) 1 ( ) 0 ( 1 ) 4 (
012 . 0
! 4
9288 . 0
) 4 (
053 . 0
! 3
9288 . 0
) 3 (
170 . 0
! 2
9288 . 0
) 2 (
367 . 0
! 1
9288 . 0
) 1 (
395 . 0
! 0
9288 . 0
) 0 (
4
9288 . 0
3
9288 . 0
2
9288 . 0
1
9288 . 0
0
9288 . 0
= =
= = = = = = >
= = =
= = =
= = =
= = =
= = =

X P X P X P X P X P X P
e
X P
e
X P
e
X P
e
X P
e
X P
Example: Bombing of London WW2









Prediction from Poisson so good => British concluded rockets were not being
aimed at specific targets - were falling randomly on London

X = no. of
rockets
P(x) 576 p(x)
0 0.395 228
1 0.367 211
2 0.170 98
3 0.053 31
4 0.012 7
> 4 (i.e. 5+) 0.003 2
Actual no. of
squares Hit
229
211
93
35
7
1
Other Basic Discrete Probability Distributions
Geometric No. of independent trials to first success.
Negative Binomial - No. of independent trials to
first, second, third fourth success.
Hypergeometric lottery type experiments.
many others.

Question
The number of cracks in a ceramic tile has a Poisson
distribution with a mean = 2.4.
What is the probability that a tile has no cracks?
What is the probability that a tile has four or more
cracks?