Review of Probability

Theory
 Review of Probability Theory

 Experiments, Sample Spaces and Events

 Axioms of Probability
 Conditional Probability
 Bayes’s Rule
 Independence
 Discrete & Continuous Random Variables

© Tallal Elshabrawy 2
 Random Experiment

 It is an experiment whose outcome cannot be predicted

with certainty
 Examples:

Tossing a Coin Rolling a Die

© Tallal Elshabrawy 3
 Random Experiment in Communications

Transmission of Bits across a Communication Channel

v Waveform x y Waveform r
Channel
Generator Detection

+A V.
ri=1 ri
vi vi=1
0 T
xi yi yi>0
+ 0
0 T
vi=0 yi<0
ri=0
-A V. zi  ]-∞, ∞[

Why is this a random experiment?

 We do not know
 The amount of noise that will affect the transmitted bit
 Whether the bit will be received in error or not
© Tallal Elshabrawy 4
 Random Experiment in Networks

Transferring a Packet across a Communication Network

Packet
Packet

Why is this a random experiment?

 We do not know
 Whether the packet will reach the destination or not
 If the packet reaches the destination, how long would it take to get
there?

© Tallal Elshabrawy 5
 Sample Space
 The set of all possible outcomes
 Tossing a coin Heads Tails
 S = {H,T}

 Rolling a die
 S = {1,2,3,4,5,6}

 The AWGN in a Communication Channel

 S = ] -∞, ∞ [ xi yi
+

zi

© Tallal Elshabrawy 6
 Event
 An event is a subset of the sample space S

 Examples
 Let A be the event of observing one head in a coin
flipped two times
 A = {HT,TH}

 Let B be the event of observing two heads in a coin

flipped twice
 B = {HH}

© Tallal Elshabrawy 7
 Axioms of Probability
Probability of an event is a measure of how often
an event might occur

no. of sample pts in A

P( A) 
no. of sample pts in S

Axioms of Probability
1. 0  P  A  1
2. P     0,P  S   1
3. P  A  B   P  A +P  B  -P  A, B 
© Tallal Elshabrawy 8
 Example
 Let Event A characterize
S
that the outcome of rolling
the die once is smaller A B
than 3 6
 A = {1,2} 2
1 4
 P(A) = 2/6 = 1/3
 Let Event B characterize 5 3
that the outcome of rolling
the die once is an even P  A, B   1/ 6
number
 B = {2,4,6} P  A  B   1/3  1/ 2  1/ 6
 P(B) = 3/6 = 1/2

© Tallal Elshabrawy 9
 Conditional Probability
 Probability of event B given A has occurred

P  A, B 
P  B A 
P  A

 Probability of event A given B has occurred

P  A, B 
P A B 
P B
© Tallal Elshabrawy 10
 Example
 Two cards are drawn in succession without
replacement from an ordinary (52 cards) deck.
Find the probability that both cards are aces
 Let A be the event that the first card is an ace
 Let B be the event that the second card is an
ace

P  A, B  =P  A   P  B A 
4 3 1
P  A, B  =  
52 51 16 17

© Tallal Elshabrawy 11
 Conditional Probability in Communications
+A V.
ri=1 ri
vi vi=1
0 T
xi yi yi>0
+ 0
0 T
vi=0 yi<0
ri=0
-A V. zi  ]-∞, ∞[

0.8

Conditioned on v=1, what

is the probability of making
0.7

an error? 0.6

0.5
r=0
Pr[error v=1]= Pr[r=0 v=1] 0.4 Decision
0.3 Zone
Pr[error v=1]= Pr[y<0 x=1] 0.2

Pr[error v=1]= Pr[x+z<0 x=1]

0.1

0
-5 -4 -3 -2 -1 0 1 2 3 4 5

© Tallal Elshabrawy 12
 Bayes’s Rule

P  A, B  P  A, B 
P  B A  P A B 
P  A P B

P(B A)P( A)
P( A B) =
P( B )

© Tallal Elshabrawy 13
 Theorem of Total Probability
Let B1, B2, …, Bn be a set of mutually exclusive
and exhaustive events.

P( A) = ∑ i =1 P A Bi P(Bi )
( )
n

© Tallal Elshabrawy 14
 Bayes’s Theorem
Let B1, B2, …, Bn be a set of mutually exclusive
and exhaustive events.

( ) P A Bi P(Bi )
P(B A) =
∑ P(A B )P(B )
i n

i =1 i i

© Tallal Elshabrawy 15
 Independent Events

 A and B are independent if

 P(B|A) = P(B)
 P(A|B) = P(A)
 P(A,B) = P(A)P(B)

© Tallal Elshabrawy 16
 Example
 Let A be the event that the grades will be out on
Thursday P(A)

 Let B be the even that I will get A+ in Random

Signals and Noise  P(B)

 So What is the probability that I get A+ if the

grades are out on Thursday  P(B|A) = P(B)

© Tallal Elshabrawy 17
 Random Variable
 Characterizes the experiment in terms of real numbers

 Example
 X is the variable for the number of heads for a coin tossed three
times
 X = 0,1,2,3

 Discrete Random Variables

 The random variable can only take a finite number of values

 Continuous Random Variables

 The random variable can take a continuum of values

© Tallal Elshabrawy 18
 Bernoulli Discrete Random Variable
 Represents experiments that have two possible outcomes.
These experiments are called Bernoulli Trials

 Associates values {0, 1} with the two outcomes such that

 P[X = 0] = 1-p
 P[X = 1] = p

 Examples
 Coin tossing experiment maps a ‘Heads’ to X = 1 and a ‘Tails’ to
X = 0 (or vice versa) such that p=0.5 for a fair coin

 Digital communication system where X = 1 represents a bit

received in error and X = 0 corresponds to a bit received correctly.
In such system p represents the channel bit error probability

© Tallal Elshabrawy 19
 Binomial Discrete Random Variable
 A random variable that represents the number of
occurrences of ‘1’ or ‘0’ in n Bernoulli trials

 The corresponding random variable X may take and values

from {0, 1, 2, …, n}

 The probability mass function PMF for having k ‘1’ in n

Bernoulli trials is
P[X = k] = nCk pk(1-p)n-k

 Examples
 In a digital communication system, the number of bits in error in a
packet depicts a Binomial discrete random variable

© Tallal Elshabrawy 20
 Geometric Discrete Random Variable
 Geometric distribution describes the number of Bernoulli
trials in succession are conducted until some particular
outcome is observed (lets say ‘1’)

 The corresponding random variable X may take and values

from {1, 2, 3, …, ∞}

 The probability mass function PMF for having k Bernoulli

trials in succession until an outcome of ‘1’ is observed
P[X = k] = (1-p)k-1p

 Examples:
 In a communication network, the number of transmissions until a
packet is received correctly follows a Geometric distribution

© Tallal Elshabrawy 21