Review of Basic Probability

Unit-1

What is Probability?
Deterministic phenomena

Random phenomena

Daily sunrises and

sunsets

Results of coin tosses

Tides at sea shores

Results of rolling dice

Phases of the moon

Results of horse races

Seasonal changes in
weather

World refer to dice games

and gambling

Annual flooding of the

Yamuna

Probabilistic notions are common place

in everyday language usage
We use words such as

Probable/improbable; possible/impossible
Certain/uncertain; likely/unlikely
Phrases such as there is a 50-50 chance
The probability of precipitation is 20%

PROBABILITY ?
The theory of probability deals with averages
of mass phenomenon occurring sequentially or
simultaneously.
The purpose of theory is to describe and
predict averages in terms of probabilities of
events.

Definition
Three Aproaches to Probability:
Classical
Relative frequency
Axiomatic

Classical (Priori) Approach to Probability:

The probability of Heads is because there are two
sides to the coin
This is called the classical approach to probability
More generally, if there are n possible outcomes of an
experiment, then each outcome has probability 1/n
Justification: Symmetry principle (Equally likely);

Problems with Classical Approach:

What exactly is an outcome?
If we toss two coins, are there three outcomes or four
outcomes?

{0 Heads, 1Head, 2 Heads}?

{(T,T), ( T,H), ( H,T), ( H,H)}?

Note that 2 Heads has probability 1/3 or depending
on the choice
There are only two outcomes: either I Win the
Lottery, or I dont, so probability is 1/2?

Relative Frequency (Posteriori) Approach:

the probability of Heads is because when tossed, the

coin will turn up Heads half the time
How do we know the coin will turn up Heads half the
time?
Suppose multiple heads tosses have resulted in 50%
Heads.
Setting P (Head)=1/2 is the relative frequency approach
to probability

Limit of relative frequency?

A lot of effort was expended in trying to
define probability as the limit of the relative
frequency

P( A ) = lim N A N
N

Unfortunately, the limit does not exist in a

mathematical sense
Physical we will only observe a finite length
prologue of the sequence of trials

Relative Frequency (Posteriori)

Approach contd
If an outcome x occurs m times on N trails, its
relative frequency is m/N & we define its
probability P(x) to be m/N
Does there exist a probability of Heads for new
unbiased untossed coin?
Or do probabilities come into existence only after
multiple tosses?
How large N should be?
Are probabilities re-defined after each toss?

Probability as beliefs:
Many assertions about probability are
essentially statements of beliefs
A fair coin is one for which P(Heads)=1/2
but how do we know whether a given coin
is fair?
Symmetry of the physical object is a belief
That further tosses of a coin for which
P(Heads) =1/2 will result in 50% Heads is
a belief

Axiomatic Approach to Probability:

In the axiomatic approach, probabilities are numbers
in the range [0, 1]
Certain probabilities are assumed to be given (we
dont ask how!)
allows the calculations of other probabilities in a
mathematically & logically consistent manner
It is probability calculas
allows the computation of probabilities without
requiring philosophical discussions about the meaning
of probability
Consistent with all the approaches described above

So, the Probability can be defined

as the Measure of the possibilities of
occurrence of an event in a
random experiment.
Probability of an event A is
denoted by P(A).

Probability in Engineering:

Thermal noise in electrical circuits

Information Theory
Communication systems design
Noise
Games of Chance
Reliability of systems
Failure probabilities
Failure Rates
Mean time to failure

Networks & Systems Problems

Random arrivals of packets/jobs
Random lengths/service times
Random requests for resources
Probability of buffer or queue overflow
Transmission or service delays
Scheduling problem, priorities, QOS
Flow control and routing

Trial ?
Experiement ?
Outcome ?
Sample space ?
Event ?

Experiments and Trials

Fundamental notion: An experiment is
performed and its outcome observed
This is called a trail of experiment
The experiment may be performed by a
human agent, e.g., tossing a coin
or rolling a dice
The experimental outcome might just be
the measurement of a naturally occurring
random phenomenon, e.g. a noise voltage

The Sample Space

The set of all possible outcomes of an
experiment is called sample space
of the experiment
Examples: The experiment is
Tossing a coin: ={H,T}
rolling a dice: ={1,2,3,4,5,6}
noise voltage: ={x:-1x1}

Exercise
Example: The experiment is
rolling a dice: ={1,2,3,4,5,6}
suppose that each outcome is equally likely:
P(1)= P(2)= P(3)= P(4)= P(5)= P(6)
What is probability of rolling an even
number?
What is probability of rolling an prime
number?

 An even number is said to have been

rolled if the outcome is any of {2,4,6}
P (even number)=1/2; more explicitly
P (even number)=3/6 since 3 of the 6
outcomes are in the subset {2,4,6}
An prime number is said to have been
rolled if the outcome is any of {2,3,5}
P (prime number)=1/2 also

Event
A subset of is called an event
Example: A={2,4,6} & B={2,3,5}
are said to be events defined on
sample space ={1,2,3,4,5,6}
events defined on the sample space
is merely a probabilists way of saying
subsets of the sample space
Ac={1,3,5} & Bc={1,4,6} also are
events define on

When does an event occur?

An event A is said to have occurred on a trial
if the outcome of the trial is a member of the
subset A
Event A occurs if the observed outcome is
some member of A; we dont care which
member of A it is
If the observed outcome is not a member of
A, then we say A did not occur, or
equivalently, we say that Ac occurred

Outcomes vs. Events

Every trial results in only one outcome, i.e.,
only one of the elements in can be observed
outcome
The observed outcome is a member of several
different subsets, i.e., events & all these
events are said to have occurred
Fundamental notion: on each trial of the
experiment, one outcomes occurs, but
many events occur

Example: If outcome of rolling a

dice is 4, then
Events A ={2,4,6} & Bc={1,4,6} both
have occurred
Events Ac={1,3,5} & B={2,3,5} did
not occur
Event A U Bc = {1,2,4,6} has
occurred
Event A and Bc = {4,6} has occurred

Two special events

can be regarded as a subset of
On any trial, the event always occurs
The event is called the certain event or the
sure event
, the empty set, is also a subset of
On any trial, the event never occurs
The event is called null event or the impossible
event
A sample space of n elements has 2n
different subsets including &

Probabilities of the special events

always occurs; c= never occurs

Conclusion: the probabilities assigned
to & should be 1 & 0
respectively, regardless of how we
choose to assign probabilities to the
outcomes
P()=1 will be used as an axiom in
the axiomatic approach to probability

Arbitrary probability assignment

Nonclassical approach: The n outcomes

have probabilities p1, p2 ...pn where pi 0
& pi =1
The probability of an event A is the sum
of the probabilities of all the outcomes
that comprise A
P(A)=sum of the pi for all members of A
Example: A={x2,x4,x22}
P(A)=p2+p4+p22

Disjoint Events
Events A & B are said to be disjoint or
mutually exclusive if A I B =
A & B have no element in common
A ={1,3,5} & B={2,4,6} are disjoint events
A U B = {1,2,3,4,5,6}
P(A U B ) =p1+p2+p3+p4+p5+p6
= P(A)+P(B)

Probability Axioms for finite spaces

Probabilities are numbers assigned to events
that satisfy the following rules:
Axiom I: P(A)0 for all events A
Axiom II: P()=1
Axiom III: If events A & B are disjoint, then
P(A U B ) = P(A)+P(B)
Consequences: P()=0
P(Ac)=1 - P(A); P(A)=1 - P(Ac)
0P(A)1 for all events A

Countably infinite sample spaces

Let = {x1,x2,.xn,.} be the Countably
infinite sample spaces
P{xn}=pn where pn0
For a finite subset A of , P(A) is just the
sum of the probabilities of the outcomes
comprising the event A, as before
It seems reasonable to have this idea work
for an infinite subset of A as well
But we need a new improved axiom

New improved Axiom III

Let A1,A2,..An denote a countable sequence
of disjoint events, i.e., A i I A j =
for all ij. Then, P(A1UA2U.. UAnU. )
=P(A1)+ P(A1)+ P(A1)+..P(An)+.

The new axiom implies that P() =0 &

P(A U B ) = P(A)+P(B) for AB =

The Probability Space:

Formal statement of the axiomatic
theory
A probability space (,F,P) consists of
The sample space consisting of all
possible outcomes of the experiment
The -field of events F which includes
all the interesting subsets of
The probability measure P() that
assigns probabilities to the events