Professional Documents
Culture Documents
PROBABILITY THEORY
Lecture 1
Lecture 2
Lecture 3
Lecture 4
Lecture 5
Lecture 6
Lecture 7
Lecture 8
Lecture 9
Lecture 10
Lecture 11
Lecture 12
Lecture 13
Basics
Independence and Bernoulli Trials
Random Variables
Binomial Random Variable Applications, Conditional
Probability Density Function and Stirlings Formula.
Function of a Random Variable
Mean, Variance, Moments and Characteristic Functions
Two Random Variables
One Function of Two Random Variables
Two Functions of Two Random Variables
Joint Moments and Joint Characteristic Functions
Conditional Density Functions and Conditional Expected Values
Principles of Parameter Estimation
The Weak Law and the Strong Law of Large numbers
1
STOCHASTIC PROCESSES
Lecture 14
Lecture 15
Lecture 16
Lecture 17
Lecture 18
Lecture 19
Lecture 20
Note: These lecture notes are revised periodically with new materials
and examples added from time to time. Lectures 1 11 are
used at Polytechnic for a first level graduate course on Probability
theory and Random Variables. Parts of lectures 14 19 are used at
Polytechnic for a Stochastic Processes course. These notes are intended
for unlimited worldwide use. However the user must acknowledge the
present website www.mhhe.com/papoulis as the source of information.
Any feedback may be addressed to pillai@hora.poly.edu
S. UNNIKRISHNA PILLAI
PROBABILITY THEORY
1. Basics
Probability theory deals with the study of random
phenomena, which under repeated experiments yield
different outcomes that have certain underlying patterns
about them. The notion of an experiment assumes a set of
repeatable conditions that allow any number of identical
repetitions. When an experiment is performed under these
conditions, certain elementary events i occur in different
but completely uncertain ways. We can assign nonnegative
number P(i ), as the probability of the event i in various
3
ways:
PILLAI
(1-1)
PILLAI
PILLAI
1
n/ p
lim n .
p
n
(1-4)
1
2
p
(1-5)
.
pq
(1-6)
and
P pq divides any given number N
1 , 2 ,, k ,
(1-7)
A B | A or B
A B | A and B
and
| A
(1-8)
PILLAI
A
A
A B
A B
Fig.1.1
A .
(1-9)
i 1
A1
B
Aj
A B
Fig. 1.2
A2
Ai
An
PILLAI
De-Morgans Laws:
A B A B ;
A
A B
A B A B
A B
(1-10)
A B
Fig.1.3
PILLAI
1 ( H , H ), 2 ( H , T ), 3 (T , H ), 4 (T , T )
and
1 , 2 , 3 , 4 .
10
PILLAI
(i)
(ii) If A F ,
then A F
(1-11)
Thus if A F , B F , then
F , A, B, A, B, A B, A B, A B, . (1-12)
PILLAI
Axioms of Probability
For any event A, we assign a number P(A), called the
probability of the event A. This number satisfies the
following three conditions that act the axioms of
probability.
(i)
PILLAI
(1-15)
PILLAI
AB
A B
Fig.1.4
P ( A B ) P ( A AB ) P ( A) P ( AB ). (1-17)
To compute P ( AB ), we can express B as
B B B ( A A)
Thus
( B A) ( B A) BA B A
P ( B ) P ( BA) P ( B A),
(1-18)
(1-19)
15
PILLAI
From (1-19),
P ( AB ) P ( B ) P ( AB )
(1-20)
(1-21)
A Ai ?
(1-22)
(1-23)
i 1
PILLAI
(1-25)
n 1
(1-26)
(1-27)
17
PILLAI
A Ai F .
i 1
(1-28)
18
PILLAI
P
A
n
n 1
P( A
n 1
(1-29)
).
(1-30)
P( A) P
A
n
n 1
P( A
n 1
).
(1-31)
19
PILLAI
and
1
P ( An ) n 1,
n 1
n 1 2
(1-32)
NA
N
N
, P( B) B , P( AB) AB .
N
N
N
(1-33)
NB
NB / N
P( B )
(1-34)
21
PILLAI
(1-35)
PILLAI
We have
(i)
P( A | B )
P ( AB ) 0
0,
P( B ) 0
(ii)
P ( | B )
P(B ) P( B )
1,
P( B )
P( B )
(1-36)
since B = B.
(1-37)
.
P( B )
P( B )
(1-38)
P( A | B ) P(C | B ),
P( B )
P( B )
(1-39)
1
P( B)
P( B)
(1-40)
P ( A).
P( B )
P( B )
(1-41)
24
PILLAI
Thus
i 1
(1-42)
PILLAI
P( BA ) P( B | A ) P( A ).
i 1
i 1
(1-44)
PILLAI
P ( A).
P( B )
P( B )
(1-46)
27
PILLAI
But
and
(1-47)
6
6
3
P(W1 )
,
6 4 10 5
4
4
P( B2 | W1 )
,
54 9
and hence
5 4 20
P(W1B2 )
0.25.
9 9 81
28
PILLAI
,
5 4 5 6 3 10 9 5 3 5
15
5
and
2 3
20
P( B2 ) P(W1 ) P( B2W1 )
.
5 5
81
29
PILLAI
From (1-35),
P( AB) P( A | B) P( B).
(1-48)
P ( BA)
P ( AB )
P ( B | A)
,
P ( A)
P ( A)
P( AB) P( B | A) P( A).
(1-49)
or
P( B | A)
P( A | B )
P( A)
P( B )
(1-50)
PILLAI
PILLAI
P( B )
P ( B | Ai ) P ( Ai )
n
P( B | A ) P( A )
i 1
(1-51)
PILLAI
5
P( D | B2 )
0.025.
200
33
PILLAI
1
.
2
PILLAI
P( D | B1 ) P( B1 ) 0.15 1 / 2
0.8571 . (1-52)
P( D)
0.0875