Chapter 3: Probability

Experiment:-
Any planned process of data collection. It consists of a number of trials (replications) under
the same condition.

Probability:-
If an event can occur in N mutually exclusive and equally likely ways, and if m of these
possess a trait, E, the probability of the occurrence of E is read as, P(E) = m/N

Permutations vs. Combinations


Both are ways to count the possibilities

The difference between them is whether order matters or not

Consider a poker hand: – A♦,5♥, 7♣, 10♠, K♠

Is that the same hand as: – K♠,10♠, 7♣, 5♥, A♦

Does the order the cards are handed out matter?

If yes, then we are dealing with permutations

If no, then we are dealing with combinations

Permutations:-

A permutation is an ordered arrangement of the elements of some set S

Let S = {a, b, c}

 c, b, a is a permutation of S
 b, c, a is a different permutation of S
 An r-permutation is an ordered arrangement of r elements of the set


A♦, 5♥, 7♣, 10♠, K♠ is a 5-permutation of the set of cards

The notation for the number of r-permutations: P(n,r)

The poker hand is one of P(52,5) permutations

Number of poker hands (5 cards):

Combinations

Combination is a collection of the objects where the order doesn't matter

Random Variable

Random experiment is an experiment with random outcome.

Random variable is a variable related to a random event

3-1
Discrete Random Variables:

The number of throws of a coin needed before a head first appears

The number of dots when rolling a dice

The number of defective items in a sample of 20 items

The number of customers arriving at a check-out counter in an hour

The number of people in favor of nuclear power in a survey

Continuous Random Variables:


The yearly income for a family

The amount of oil imported into Finland in a particular month

The time that elapses between the installation of a new component and its failure

The percentage of impurity in a batch of chemicals

Mean and Variance of a Discrete Random Variable


The mean of a discrete random variable X is a weighted average of the possible values
that the random variable can take. Unlike the sample mean of a group of observations,
which gives each observation equal weight, the mean of a random variable weights each
outcome xi according to its probability, pi.

The variance of a discrete random variable X measures the spread, or variability, of the
distribution, and is defined by

Normal Distribution:

Many continuous variables are approximately normally distributed

Measurement errors

Physical and mental properties of people •Properties of manufactured products

Daily revenues of investments

area under density function equals 1,area represents probability

3-2
Standardized Distribution N(0,1)

Cumulative function values have been tabulated (in most statistics textbooks) for
normal distribution with expected value 0 and standard deviation 1

This distribution is called standardized distribution and is denoted N(0,1).

Binomial Distribution:-

The binomial distribution occur in games of chance (like rolling a die), quality inspection
(e.g. Counting of the number of defectives), opinion polls (counting number of employees
favoring certain Schedule changes, etc.) medicine (e.g. Recording the number of patients who
recovered on a new medication) and so on.

The conditions of its occurence are as follows.


We are interested in the number of times an event A occurs in n independent trials.

In each trial the event A has the same probability P(A) = p.

Then in a trial, A will not occur with probability q = 1 – p.

In n trials the random variable that interests us is X = Number of times the event A occurs
in n trials.

X can assume the values 0,1,… ,n and we want to determine the
corresponding probabilities.

Now X = x means that A occurs in x trials and in n – x trials it does not occur.

The mean of the binomial distribution is,

The variance is,

Binomial Distributio

3-3
Poisson Distribution:The discrete distribution wih infinilety many possible values
and probability function is called the Poisson Distribution.

f(x)=( (x=0,1,….,n)

Hypergeometric distribution:

Sampling without replacement means that we return no screw to the box

Then we no longer have independence of trials (why?) and instead of (7) the probability
of drawing x defectives in n trials is

, (x= 0,1,2,…)

The hypergeometic distribution has the mean,

 And the variance is
The distribution with this probability function is called the hypergeometric distribution.

Combinatorics
Example . Toss three fair coins. What is the probability of exactly one Heads (H)?
There are 8 equally likely outcomes: HHH, HHT, HTH, HTT, THH, THT, TTH, TTT. Out of
these, 3 have exactly one H. That is, E = fHTT, THT, TTHg, and P (E) = 3=8.

BBBB BBBG BBGB BBGG

BGBB BGBG BGGB BGGG

GBBB GBBG GBGB GBGG

GGBB GGBG GGGB GGGG

We conclude that 3
P (2:2 split) =6 = 3 ;
16 8
P (1:3 split or 3:1 split) 8 1
= = ;
16 2
P (4:0 split or 0:4 split) 2 =1 :
= 16 8
3-4
Discrete Random Variables

¡20
5

3-5
A random variable is a number whose value depends upon the outcome of a random
experiment. Mathematically, a random variable X is a real-valued function on -, the space of
outcomes:
X : - ! R:
Sometimes, when convenient, we also allow X to have the value 1 or, more rarely, ¡1, but this
will not occur in this chapter. The crucial theoretical property that X should have is that, for
each interval B, the set of outcomes for which X 2 B is an event, so we are able to talk about
its probability, P (X 2 B). Random variables are traditionally denoted by capital letters to
distinguish them from deterministic quantities.

Example . Here are some examples of random variables.

1. Toss a coin 10 times and let X be the number of Heads.
2. Choose a random point in the unit square f(x; y) : 0 · x; y · 1g and let X be its distance
from the origin.
3. Choose a random person in a class and let X be the height of the person, in inches.
4. Let X be value of the NASDAQ stock index at the closing of the next business day.
A discrete random variable X has ¯nitely or countably many values xi, i = 1; 2; : : :, and p(xi)
= P (X = xi) with i = 1; 2; : : : is called the probability mass function of X. Sometimes X is
added as the subscript of its p. m. f., p = pX .
A probability mass function p has the following properties:
1. For all i, p(xi) > 0 (we do not list values of X which occur with probability 0).
2. For any interval B, P (X 2 B) = Pxi2B p(xi).
3. As X must have some value, Pi p(xi) = 1.
Example 5.. Let X be the number of Heads in 2 fair coin tosses. Determine its p. m. f.
Possible values of X are 0, 1, and 2. Their probabilities are: P (X = 0) = 14 , P (X = 1) = 12 ,
and P (X = 2) = 14 .You should note that the random variable Y , which counts the number of
Tails in the 2 tosses, has the same p. m. f., that is, pX = pY , but X and Y are far from being the
same random variable! In general, random variables may have the same p. m. f., but may not
even be de¯ned on the same set of outcomes
The possible values are 5; : : : ; 20. To determine the p. m. f., note that we have
4
outcomes, and, then, ¡i¡1¢ P (X = i) = ¡20 ¢:
Finally,
X 20
P (at least one number 15 or more) = P (X ¸ P (X =
15) = i):
i=15
Binomial random variable

A Binomial(n,p) random variable counts the number of successes in n independent trials, each
of which is a success with probability p.

Properties:

1. Probability mass function: P (X = i) = ¡ni¢pi(1 ¡ p)n¡i, i = 0; : : : ; n.

2. EX = np.

3. Var(X) = np(1 ¡ p).

The expectation and variance formulas will be proved in Chapter 8. For now, take them on
faith.

Example . Let X be the number of Heads in 50 tosses of a fair coin. Determine EX, Var(X) and
P (X · 10)? As X is Binomial(50; 12 ), so EX = 25, Var(X) = 12:5, and

10
50 1
25
P (X · 10) = i=0 µ i ¶ 0 :
X

Poisson random variable

A random variable is Poisson(¸), with parameter ¸ > 0, if it has the probability mass function
given below.

Properties:
1. P (X = i) = ¸i!i e¡¸, for i = 0; 1; 2; : : EX = ¸.
2. Var(X) = ¸.

Here is how we compute the expectation:

1 ¸i 1 ¸i¡1

EX = i ¢ e¡¸ i! = e¡¸¸ (i 1)! = e¡¸¸ e¸ = ¸;

i=1 =1
X Xi ¡

3-6
and the variance computation is similar (and a good exercise!).

The Poisson random variable is useful as an approximation to a Binomial random variable

when the number of trials is large and the probability of success is small. In this context it is
often called the law of rare events, ¯rst formulated by L. J. Bortkiewicz (in 1898), who studied
deaths by horse kicks in the Prussian cavalry.

Joint Distributions and Independence

Discrete Case

Assume that you have a pair (X; Y ) of discrete random variables X and Y . Their
joint probability mass function is given by
p(x; y) = P (X = x; Y = y)

so that
p(x;
P ((X; Y ) 2 A) = y):
(x;y)2
A

X
The marginal probability mass functions are the p. m. f.'s of X and Y , given by
X X
P (X = x) = P (X = x; Y = y) = p(x; y)
y y
P (Y = y) = P (X = x; Y = y) = p(x; y)
x x

X X

Example . An urn has 2 red, 5 white, and 3 green balls. Select 3 balls at random and let X be
the number of red balls and Y the number of white balls. Determine (a) joint p. m. f. of (X; Y ),
(b) marginal p. m. f.'s, (c) P (X ¸ Y ), and (d) P (X = 2jX ¸ Y ).

The joint p. m. f. is given by P (X = x; Y = y) for all possible x and y. In our case, x can be
0; 1, or 2 and y can be 0; 1; 2, or 3. The values are given in the table.

3-7
 ynx 0 1 2 P (Y = y)

0 1/120 2 ¢ 3=120 3/120 10/120

1 5 ¢ 3=120 2 ¢ 5 ¢ 3=120 5/120 50/120

2 10 ¢ 3=120 10 ¢ 2=120 0 50/120

3 10/120 0 0 10/120

P (X = x) 56/120 56/120 8/120 1

The last row and column entries are the respective column and row sums and,
therefore, determine the marginal p. m. f.'s. To answer (c) we merely add the
relevant probabilities,
P (X ¸ Y ) = 1 + 6 + 3 + 30 + 5 = 3;
120 8
and, to answer (d), we compute
P (X = 2; X ¸ Y 8
) 12 8
0
= 3 = :
P (X ¸ Y ) 8 45

Two random variables X and Y are independent if

P (X 2 A; Y 2 B) = P (X 2 A)P (Y 2 B)

for all intervals A and B. In the discrete case, X and Y are independent exactly when

P (X = x; Y = y) = P (X = x)P (Y = y)

for all possible values x and y of X and Y , that is, the joint p. m. f. is the
product of the marginal p. m. f.'
3-8