TN 208: STOCHASTIC SIGNALS AND SYSTEMS

Module 1: Probability and Random Variables

To be Covered
Random Experiments, Sample Space and Sample Point, Events, Mutually Exclusive Events, Independent Events. Probability definition and theorems, Random variable definition. Classification of random variables.

To be Covered
Cumulative Distribution Function (cdf). Probability Density Function (pdf). Statistical Averages. Common Probability Distribution functions. Gaussian random variables.

Probability
Probability implies random experiments. A random experiment can have many possible outcomes; each outcome known as a sample point (a.k.a. elementary event) has some probability assigned. This assignment may be based on measured data or guestmates (equally likely is a convenient and often made assumption). Sample Space S : a set of all possible outcomes (elementary events) of a random experiment.
Finite (e.g., if statement execution; two outcomes) Countable (e.g., number of times a while statement is executed; countable number of outcomes) Continuous (e.g., time to failure of a component or signal)

Probability
Definition A probabilistic experiment, or random experiment, or simply an experiment, is the process by which an observation is made.
In probability theory, any action or process that leads to an observation is referred to as an experiment. Examples include:
Tossing a pair of fair coins. Throwing a balanced die.
5

Probability
Definition The sample space associated with a probabilistic experiment is the set consisting of all possible outcomes of the experiment and is denoted by S.
The elements of the sample space are referred to as sample points. A discrete sample space is one that contains either a finite or a countable number of distinct sample points.
6

Probability
Definition An event in a discrete sample space S is a collection of sample points, i.e., any subset of S. In other words, an event is a set consisting of possible outcomes of the experiment. Definition A simple event is an event that cannot be decomposed. Each simple event corresponds to one and only one sample point. Any event that can be decomposed into more than one simple event is called a compound event.
7

Probability
Definition Let A be an event connected with a probabilistic experiment E and let S be the sample space of E. The event B of nonoccurrence of A is called the complementary event of A. This means that the subset B is the complement A of A in S. In an experiment, two or more events are said to be equally likely if, after taking into consideration all relevant evidences, none can be expected in reference to another.
8

Probability

Probability
Axiomatic Approach
Analyzing the concept of equally likely probability, we see that three conditions must hold. 1. The probability of occurrence of any event must be greater than or equal to 0. 2. The probability of the whole sample space must be 1. 3. If two events are mutually exclusive, the probability of their union is the sum of their respective probabilities. These three fundamental concepts form the basis of the definition of probability.
10

Probability

11

Probability
Let A, B and C be events in the sample space, S, then

12

MUTUALLY EXCLUSIVE EVENTS

MUTUALLY EXCLUSIVE EVENTS

Events are mutually exclusive if they cannot happen at the same time. For example, if we toss a coin, either heads or tails might turn up, but not heads and tails at the same time. Similarly, in a single throw of a die, we can only have one number shown at the top face. The numbers on the face are mutually exclusive events

MUTUALLY EXCLUSIVE EVENTS cont..

If A and B are mutually exclusive events then the probability of A happening OR the probability of B happening is P(A) + P(B). P(A or B) = P(A) + P(B)

MUTUALLY EXCLUSIVE EVENTS cont..

Example 1
What is the probability of a die showing a 2 or a 5?

MUTUALLY EXCLUSIVE EVENTS cont..

Practice The probabilities of three teams A, B and C winning a badminton competition are

Calculate the probability that a) either A or B will win b) either A or B or C will win c) none of these teams will win d) neither A nor B will win

Solution/s

c) P(none will win) = 1 P(A or B or C will win) d) P(neither A nor B will win) = 1 P(either A or B will win)

Independent Events
Events are independent if the outcome of one event does not affect the outcome of another. For example, if you throw a die and a coin, the number on the die does not affect whether the result you get on the coin. If A and B are independent events, then the probability of A happening AND the probability of B happening is P(A) P(B). P(A and B) = P(A) P(B)

Example 1
If a dice is thrown twice, find the probability of getting two 5s.

Two sets of cards with a letter on each card as follows are placed into separate bags.

Sara randomly picked one card from each bag. Find the probability that: a) She picked the letters J and R. b) Both letters are L. c) Both letters are vowels.

Solution for no. 2

a) Probability that she picked J and R = b) Probability that both letters are L = c) Probability that both letters are vowels =

Example 3
Two fair dice, one colored white and one colored red, are thrown. Find the probability that: a) the score on the red die is 2 and white die is 5. b) the score on the white die is 1 and red die is even

Solution for No. 3

a) Probability the red die shows 2 and white die 5 = b) Probability the white die shows 1 and red die shows an even number =

DEPENDENT EVENTS
Events are dependent if the outcome of one event affects the outcome of another. For example, if you draw two colored balls from a bag and the first ball is not replaced before you draw the second ball then the outcome of the second draw will be affected by the outcome of the first draw.

DEPENDENT EVENTS cont..

If A and B are dependent events, then the probability of A happening AND the probability of B happening, given A, is P(A) P(B after A). P(A and B) = P(A) P(B after A) P(B after A) can also be written as P(B | A) then P(A and B) = P(A) P(B | A)

Example 1
A purse contains four P50 bills, five P100 bills and three P20 bills. Two bills are selected without the first selection being replaced. Find P(P50, then P50)

Solution
There are four P50 bills. There are a total of twelve bills. P(P50) = 4/12 The result of the first draw affected the probability of the second draw. There are three P50 bills left. There are a total of eleven bills left. P(P50 after P50) = 3/11

 P(P50, then P50) = P(P50) P(P50 after P50) = (4/12)x(3/11)=12/132 The probability of drawing a P50 bill and then a P50bill is

Dependent: Practice
A bag contains 6 red, 5 blue and 4 yellow marbles. Two marbles are drawn, but the first marble drawn is not replaced. a) Find P(red, then blue). b) Find P(blue, then blue)

Independent Events: Practice

Two fair dice, one colored white and one colored red, are thrown. Find the probability that: a) the score on the red die is 2 and white die is 5. b) the score on the white die is 1 and red die is even

Mutually Exclusive Events: Practice

The probabilities of three teams A, B and C winning a badminton competition are Calculate the probability that a) either A or B will win b) either A or B or C will win c) none of these teams will win d) neither A nor B will win

Summary
For mutually exclusive events Pr(A or B) = Pr(AB) = Pr(A)+Pr(B) For independent events Pr(A and B)=Pr(A B) = Pr(A)Pr(B) In general, Pr(A B) = Pr(A)+Pr(B)-Pr(A B) Pr(A B) = Pr(A)+Pr(B)-Pr(A B) Pr(A B)=Pr(B|A)Pr(A) =Pr(A|B)Pr(B)

Sample Space Worked Examples

Problem 1: Count the number of voice packets containing only silence produced from a group of N speakers in a 10-ms period.

Solution: Denote sample space by S then, S = { 0, 1, 2, , N } Problem 2: A block is transmitted repeatedly over a noisy channel until an error-free block arrives at the receiver. Count the number of transmission required. Solution: Denote sample space by S then, S = { 1, 2, 3, , }

Sample Space Worked Examples

Problem 3: Measure the time between two message arrivals at a message center.

Solution: Denote sample space by S then, S = { t: t 0 } = [ 0, )

where t denotes time.

Problem 4: Measure the lifetime of a given computer memory chip in a specified environment.
Solution: Denote sample space by S then, S = { t: t 0 } = [ 0, )
where t denotes time.

Events Worked examples

Problem 1: Write the values of events for problems in case study of sample space for following events:
1. 2. 3.

4.

No active packets are produced Fewer than 10 transmission are required Less than t0 seconds elapse between message arrivals The chip lasts for more than 1000 hours but fewer than 5000 hour

Solution : 1. No active packets are produced, then

A={0}

Events Worked examples cont..

2. Fewer than 10 transmission are required A = { 1, 2, , 9 } 3. Less than t0 seconds elapse between message arrivals A = { t : 0 t < t0 } = [ 0, t0 )

4. The chip lasts for more than 1000 hours but fewer than 5000 hour

A = { t : 1000 < t < 5000 } = (1000, 5000 )