Stochastic Lecture 5

Copyright Syed Ali Khayam 2009
EE 801 Analysis of Stochastic

Systems
Multiple Random Variables
Muhammad Usman Ilyas
School of Electrical Engineering & Computer Science
National University of Sciences & Technology (NUST)
Pakistan
Inthislecture,wewillcover:
JointDistributionsofMultipleRandomVariables
FunctionsofMultipleRandomVariables
MomentsofMultipleRandomVariables
JointlyNormalRandomVariables
SumsofRandomVariables
Whatwillwecoverinthislecture?
VectorRandomVariables
PairsofRandomVariables
3
DefinitionofaJointlyDistributedRandom
Variable
Avectorrandomvariableisamappingfromanoutcomes ofa
randomexperimenttoavector
:
n
X
Z S S
domain
Range or image of X
4
DefinitionofaJointlyDistributedRandom
Variable
Random
Experiment
S
X
X, Y(x, y)
2
,
:
X Y
Z S S
Image courtesy of www.buzzle.com/
1
2
3
4
5
6
6 5 4 3 2 1
S
Y
5
JointDistributions
Thejointcumulativedistributionfunction(cdf)ofapairofrvsX
andYisdefinedas:
Jointdistributionsarealsocalledcompounddistributions
{ }
{ }
,
( , ) Pr
Pr , , ,
X Y
F a b X a Y b
X a Y b a b
=
= - < < - < <
6
ExamplesofJointDistributions:Throwing
TwoDiceExperiment
x
y
p
X,Y
(x,y)
(1,1)
(6,6)
7
ExamplesofJointDistributions:AJoint
GaussianDistribution
8
JointDistributions
Y
X b
Xb
Yd
d
9
JointDistributions
b
Xb
Yd
d
,
Integrating shaded region gives ( , )
X Y
F b d
Y
X
10
PropertiesofJointCDFs
F1:
F2:
F3:IfXandYarecontinuousrvsthenF
X,Y
(a,b)isalso
continuous
( )
,
0 , 1, ,
X Y
F a b a b - < <
( ) ( )
1 2 1 2 , 1 1 , 2 2
and , ,
X Y X Y
a a b b F a b F a b
11
F4:
( )
( )
and
,
or
,
, 1
, 0
a b
X Y
a b
X Y
F a b
F a b
a
b
,
X Y
F a b
Y
X
12
( )
and
,
, 1
a b
X Y
F a b

a
b
b
Y
X
13
( )
and
,
, 1
a b
X Y
F a b

a
b
b
Y
X
14
( )
and
,
, 1
a b
X Y
F a b

a
b
Y
X
15
F5:
( ) ( )
,
,
a
X Y Y
F a b F b
b
These distributions are
called marginal
distributions
a
( )
( )
,
,
b
X Y X
F a b F a
Y
X
16
( )
{ }
{ }
{ } { }
{ } ( )
,
lim , Pr
Pr
Pr Pr
Pr
a X Y
Y
F a b X Y b
X Y b
Y b X X
Y b F b
=
= - -
= - - -
= - =
17
( )
{ }
{ } ( )
,
lim , Pr Pr
a X Y Y
F a b X Y b Y b F b
= = =
b
a
Y
X
18
( )
{ }
{ } ( )
,
lim , Pr Pr
a X Y Y
F a b X Y b Y b F b
= = =
b
a
Y
X
19
b
a
( )
{ }
{ } ( )
,
lim , Pr Pr
a X Y Y
F a b X Y b Y b F b
= = =
Y
X
20
F6:
{ }
, , , ,
Pr and c ( , ) ( , ) ( , ) ( , )
X Y X Y X Y X Y
a X b Y d F b d F a d F b c F a c < < = - - +
21
b
Xb
Yd
d
Y
X
22
b
Xb
Yd
d
,
X Y
F b d
Y
X
23
b
Xb
Yd
d
c
a
{ }
We want Pr c a X b Y d < <
Y
X
24
b
Xb
Yd
d
c
a
Integrating the
entire region gives
F
X,Y
(b,d)
Y
X
25
b
Xb
Yd
d
c
a
F
i
r
s
t

s
u
b
t
r
a
c
t

F
(
a
,
d
)

f
r
o
m

F
X
,
Y

(
b
,
d
)
Y
X
26
b
Xb
Yd
d
c
a
Then subtract F
X,Y
(b,c) from F
X,Y
(b,d)-F
X,Y
(a,d)
Y
X
27
b
Xb
Yd
d
c
a
Finally add F
X,Y
(a,c) to F
X,Y
(b,d)-
F
X,Y
(a,d)-F
X,Y
(b,c)
{ }
, , , ,
Pr and c ( , ) ( , ) ( , ) ( , )
X Y X Y X Y X Y
a X b Y d F b d F a d F b c F a c < < = - - +
Y
X
28
JointProbabilityDensityFunction
Thejointprobabilitydensityfunctionisdefinedas:
Alternatively:
2
, ,
( , ) ( , )
X Y X Y
f x y F x y
x y
=

, ,
( , ) ( , )
y
x
X Y X Y
F x y f a b dadb
- -
=

29
b
Xb
Yd
d
c
a
{ }
,
Pr c ( , )
b d
X Y
a c
a X b Y d f x y dydx < < =

Y
X
30
JointProbabilityDensityFunction
Thejointprobabilitydensityfunctionmustsatisfythefollowing
property:
Moreover:
,
( , ) 1
X Y
f x y dxdy

- -
=

,
,
( ) ( , )
( ) ( , )
X X Y
Y X Y
f x f x y dy
f y f x y dx
-
=
=
31
Homework
ReadingAssignment:Examples4.1to4.9inthebook
Homework
Problem4.3inthetextbook
Problem4.9inthetextbook
32
IndependentRandomVariables
TworandomvariablesXandYareindependentif
Or:
,
( , ) ( ) ( )
X Y X Y
F x y F x F y =
,
,
( , ) ( ) ( ) for discrete rvs
( , ) ( ) ( ) for continuous rvs
X Y X Y
X Y X Y
p x y p x p y
f x y f x f y
=
=
33
ConditionalProbabilityforRandomVariables
Conditionalprobabilityisanimportanttoolwhenanalyzing
randomvariables
Ingeneral,weareinterestedinfindingtheprobabilitythata
randomvariableYtakesonacertainvalue(orarangeof
values)giventhatanotherrandomvariableXhasalreadytaken
somevalue
Conditionalprobabilitycanbeexpressedintermsofthejoint
andmarginalprobabilities
34
Random
Experiment
Whatistheprobabilitythatthesumofthe
twodiceisoddgiventhatX=2?
X
Y
35
Wecanextendthedefinitionofconditionalprobabilityto
randomvariables
( | ) Pr{( ) | ( )}
Pr{ }
Pr{ }
Y
F y x Y y X x
Y y X x
X x
= =
=
=
=
36
Wecanextendthedefinitionofconditionalprobabilityto
randomvariables
Whatswrongwiththisdefinition?
( | ) Pr{( ) | ( )}
Pr{( ) ( )}
Pr{ }
Y
F y x Y y X x
Y y X x
X x
= =
=
=
=
37
Whatswrongwiththisdefinition?
ThisdefinitionwillonlyworkfordiscretervsbecausePr{X=a}=0
forcontinuousrvs
Therefore,weneedtogeneralizethisdefinitiontoencompass
continuousrvs
Pr{( ) ( )}
( | )
Pr{ }
Y
Y y X x
F y x
X x
=
=
=
38
b
0
Pr{( ) ( )}
( | ) lim
Pr{ }
Y
h
Y y x X x h
F y x
x X x h
< +
=
< +
a
h
Y
X
39
Weneedtogeneralizethedefinitionofconditionalprobability
toencompassbothdiscreteandcontinuousrvs
( )
( )
( )
( )
( )
( )
0
, ,
,
Pr{( ) ( )}
( | ) lim
Pr{ }
, ,
,
Y
h
y y
x h
X Y X Y
x
x h
X
X
x
y
X Y
X
Y y x X x h
F y x
x X x h
f x y dx dy hf x y dy
hf x
f x dx
f x y dy
f x
+
- -
+
-
< +
=
< +

=

=

40
WecandifferentiatetheconditionalCDFtoobtainthe
conditionalpdf
( )
( )
,
,
( | )
y
X Y
Y
X
f x y dy
F y x
f x
-

=
41
( )
( )
,
,
( | ) ( | )
X Y
Y Y
X
f x y
d
f y x F y x
dy f x
= =
Inotherwords,ajointpdfcanbewrittenasaproductof
conditionalandmarginalpdfs:
Foradiscreterv,thesameconditioncanbestatedas:
42
( )
( )
,
, ( | )
X Y Y X
f x y f y x f x =
( ) { }
( )
,
, Pr , ( | )
X Y Y X
p x y X x Y y p y x p x = = = =
ConditionalExpectation
Expectationofaconditionaljointdistributionisdefinedas
43
{ }
( | ) continuous rvs
|
( | ) discrete rvs
Y
Y
y
yf y x dy
E Y x
yp y x

UsefulPropertiesofConditionalExpectation
P1:
P2:Forafunctionh(Y):
ThekthmomentofYisgivenby
44
{ }
{ }
{ } | E E Y x E Y =
{ } { }
{ }
( ) ( ) | E h Y E E h Y x =
{ } { } { }
|
k k
E Y E E Y x =
Homework
ReadingAssignment
Example4.15to4.26inthetextbook
45
VectorRandomVariables
MultipleRandomVariables
46
JointDistributionofMultiplervs
Mostoftheideasthatwehavestudiedsofarcanbedirectly
extendedtomorethantwojointlydistributedrandomvariables
Jointpmfofndiscreterandomvariablesis:
Conditionalpmfsareobtainedas
47
( ) { }
1 2
, , , 1 2 1 1 2 2
, , , Pr , , ,
n
X X X n n n
p x x x X x X x X x = = = =

( )
( )
( )
1 2
1 2 1
, , , 1 2 1
1 2 1
, , , 1 2 1
, , , ,
| , , ,
, , ,
n
n
n
X X X n n
X n n
X X X n
p x x x x
p x x x x
p x x x
-
-
-
-
=


Mostoftheideasthatwehavestudiedsofarcanbedirectly
extendedtomorethantwojointlydistributedrandomvariables
Jointpmfofndiscreterandomvariablesis:
Conditionalpmfsareobtainedas
48
( ) { }
1 2
, , , 1 2 1 1 2 2
, , , Pr , , ,
n
X X X n n n
p x x x X x X x X x = = = =

( )
( )
( )
1 2
1 2 1
, , , 1 2 1
1 2 1
, , , 1 2 1
, , , ,
| , , ,
, , ,
n
n
n
X X X n n
X n n
X X X n
p x x x x
p x x x x
p x x x
-
-
-
-
=


ConditinalDistributionofMultiplervs
Conditionalpdfofajointpdfis:
Repeatedlyapplyingthisexpressiongives:
49
( )
( )
( )
1 2
1, 2 1
, , , 1
1 1
, , 1 1
, ,
| , ,
, ,
n
n
n
X X X n
X n n
X X X n
f x x
f x x x
f x x
-
-
-
=

( )
( ) ( ) ( ) ( )
1 2
1 2 1
, , , 1
1 1 1 1 2 2 1 1
, ,
| , , | , , |
n
n n
X X X n
X n n X n n X X
f x x
f x x x f x x x f x x f x
-
- - -
=

ConditinalDistributionofMultiplervs
Conditionalpdfofajointpdfis:
Repeatedlyapplyingthisexpressiongives:
50
( )
( )
( )
1 2
1, 2 1
, , , 1
1 1
, , 1 1
, ,
| , ,
, ,
n
n
n
X X X n
X n n
X X X n
f x x
f x x x
f x x
-
-
-
=

( )
( ) ( ) ( ) ( )
1 2
1 2 1
, , , 1
1 1 1 1 2 2 1 1
, ,
| , , | , , |
n
n n
X X X n
X n n X n n X X
f x x
f x x x f x x x f x x f x
-
- - -
=

Marginalpmfofonervisobtainedbysummingovertheimages
ofallotherrvs
51
( ) { }
( )
1
1 2
2
1 1 1
, , , 1 2
Pr
, , ,
n
n
X
X X X n
x x
p x X x
p x x x
= =
=

Marginalpmfofonervisobtainedbysummingovertheimages
ofallotherrvs
52
( ) { }
( )
1
1 2
2
1 1 1
, , , 1 2
Pr
, , ,
n
n
X
X X X n
x x
p x X x
p x x x
= =
=

JointCDFofncontinuousrandomvariablesis:
Jointpdfisthenobtainedas
53
( )
( )
1 2
1 2
1 2
, , , 1 2
, , , 1 2 1
, , ,
, , ,
n
n
n
X X X n
x x x
X X X n n
F x x x
f x x x dx dx
- - -

=

( ) ( )
1 2 1 2
, , , 1 2 , , , 1 2
1
, , , , , ,
n n
n
X X X n X X X n
n
f x x x F x x x
x x

54
( )
( )
1 2
1 2
1 2
, , , 1 2
, , , 1 2 1
, , ,
, , ,
n
n
n
X X X n
x x x
X X X n n
F x x x
f x x x dx dx
- - -

=

( ) ( )
1 2 1 2
, , , 1 2 , , , 1 2
1
, , , , , ,
n n
n
X X X n X X X n
n
f x x x F x x x
x x

55
( )
( )
1 2
1 2
1 2
, , , 1 2
, , , 1 2 1
, , ,
, , ,
n
n
n
X X X n
x x x
X X X n n
F x x x
f x x x dx dx
- - -

=

( ) ( )
1 2 1 2
, , , 1 2 , , , 1 2
1
, , , , , ,
n n
n
X X X n X X X n
n
f x x x F x x x
x x

56
( )
( )
1 2
1 2
1 2
, , , 1 2
, , , 1 2 1
, , ,
, , ,
n
n
n
X X X n
x x x
X X X n n
F x x x
f x x x dx dx
- - -

=

( ) ( )
1 2 1 2
, , , 1 2 , , , 1 2
1
, , , , , ,
n n
n
X X X n X X X n
n
f x x x F x x x
x x

Asinglemarginalpdf canbeobtainedas:
Also,amarginalpdf forasubvectorrv canbeobtainedas:
57
( )
( )
1 1 2
1 , , , 1 2 2
, , ,
n
X X X X n n
f x f x x x dx dx

- -

=

( )
( )
1 1 1 2
, , 1 1 , , , 1 2 1
, , , , , ,
n n
X X n X X X n n n
f x x f x x x x dx
-
- -
-

=

58
( )
( )
1 1 2
1 , , , 1 2 2
, , ,
n
X X X X n n
f x f x x x dx dx

- -

=

( )
( )
1 1 1 2
, , 1 1 , , , 1 2 1
, , , , , ,
n n
X X n X X X n n n
f x x f x x x x dx
-
- -
-

=

59
( )
( )
1 1 2
1 , , , 1 2 2
, , ,
n
X X X X n n
f x f x x x dx dx

- -

=

( )
( )
1 1 1 2
, , 1 1 , , , 1 2 1
, , , , , ,
n n
X X n X X X n n n
f x x f x x x x dx
-
- -
-

=

Asinglemarginalpdfcanbeobtainedas:
Also,amarginalpdfforasubvectorrvcanbeobtainedas:
60
( )
( )
1 1 2
1 , , , 1 2 2
, , ,
n
X X X X n n
f x f x x x dx dx

- -

=

( )
( )
1 1 1 2
, , 1 1 , , , 1 2 1
, , , , , ,
n n
X X n X X X n n n
f x x f x x x x dx
-
- -
-

=

Summary
Avectorrvisamappingfromoutcomesofarandom
experimenttoavector
Jointdensityanddistributionfunctionsofavectorrvare:
Marginaldensitiescanbeobtainedas:
61
:
n
X
Z S S
2
, ,
( , ) ( , ),
X Y X Y
f x y F x y
x y
=

, ,
( , ) ( , )
y
x
X Y X Y
F x y f a b dadb
- -
=

( ) ( )
,
,
a
X Y Y
F a b F b
( )
( )
,
,
b
X Y X
F a b F a

Summary
experimenttoavector
62
:
n
X
Z S S
2
, ,
( , ) ( , ),
X Y X Y
f x y F x y
x y
=

, ,
( , ) ( , )
y
x
X Y X Y
F x y f a b dadb
- -
=

( ) ( )
,
,
a
X Y Y
F a b F b
( )
( )
,
,
b
X Y X
F a b F a

Summary
experimenttoavector
63
:
n
X
Z S S
2
, ,
( , ) ( , ),
X Y X Y
f x y F x y
x y
=

, ,
( , ) ( , )
y
x
X Y X Y
F x y f a b dadb
- -
=

( ) ( )
,
,
a
X Y Y
F a b F b
( )
( )
,
,
b
X Y X
F a b F a

Summary
Conditionaldensityisdefinedas:
Forindependentrvs,wehave:
64
( )
( )
,
, ( | )
X Y Y X
f x y f y x f x =
,
( , ) ( ) ( )
X Y X Y
F x y F x F y =
,
,
X Y X Y
X Y X Y
p x y p x p y
f x y f x f y
=
=
Summary
Conditionaldensityisdefinedas:
Forindependentrvs,wehave:
65
( )
( )
,
, ( | )
X Y Y X
f x y f y x f x =
,
( , ) ( ) ( )
X Y X Y
F x y F x F y =
,
,
X Y X Y
X Y X Y
p x y p x p y
f x y f x f y
=
=
Homework
ReadingAssignment
Example4.28,4.29,4.30inthetextbook
66
FunctionsofMultipleRandom
Variables
67
Ifwehaveafunctionoftwodiscreterandomvariables,Z=g(X,
Y),wecanwriteitspmfas:
68
( )
( ) { } { } { }
( )
( )
( )
Pr ( , ) Pr ( , ) | Pr ,
for all
( , ) |
Z
Z Y X
x
p z y z x X x y z x X x X x
x
p z p y z x x p x
= = = = =
=

Ifwehaveafunctionoftworandomvariables,Z=g(X,Y),wecan
writeitintermsofthejointpdfandpmfas:
69
( )
( )
( )
( ) ( ) ( , ) | ( , ) |
Z Y X X Y
f z f y z x x f x dx f x z y y f y dy

- -
= =

Example:Z=X+Y
LetZ=g(X,Y)=X+Y
IfZisdiscrete,wecanfinditspmfas:
Or:
70
( )
{ } { }
{ } { }
( ) ( )
Pr Pr
Pr | Pr
Z
x
Y X
x
p z Z z X Y z
Y z x X x X x
p z x p x
= = = + =
= = - = =
= -
( )
( ) ( )
Z X Y
y
p z p z y p y = -

Example:Z=X+Y
LetZ=g(X,Y)=X+Y
IfZiscontinuous,wehave:
Thesamepdfcanalsobewrittenas:
71
( ) ( ) ( )
Z Y X
f z f z x f x dx
-
= -
( )
( ) ( )
Z X Y
f z f z y f y dy
-
= -

Example:Z=X+Y
LetZ=g(X,Y)=X+Y
PDFofthesumoftwocontinuousrandomvariablesisequalto
theirconvolution
72
( ) ( ) ( )
Z Y X
f z f z x f x dx
-
= -
( )
( ) ( )
Z X Y
f z f z y f y dy
-
= -
Convolution integral
Example2:Z=X+Y
LetZ=g(X,Y)=X+Y
LetXandYbethepacketprocessingtimetakenattworouters
connectedinseries
XandYareidenticallydistributedexponentialrandomvariables
withparameters
Findthepdf ofZ.
73
Router 1
(Time taken: X)
Router 2
(Time taken: Y)
Example2:Z=X+Y
Z=X+Y
ThepdfofXis:
74
( )
0
0 0
x
X
e x
f x
x
l
l
-
<
Router 1
(Time taken: X)
Router 2
(Time taken: Y)
Example2:Z=X+Y
Z=X+Y
ThepdfofYis:
75
( )
0
0 0,
y
Y
e y z x
f y
y y z x
l
l
-
< > -
Router 1
(Time taken: X)
Router 2
(Time taken: Y)
Example2:Z=X+Y
Z=X+Y
Inotherwords,thepdfofYis:
76
( )
( )
0,
0
z x
Y
e z x x z
f y
x z
l
l
- -
- > <
>
Router 1
(Time taken: X)
Router 2
(Time taken: Y)
Example2:Z=X+Y
ThepdfofZis:
77
( ) ( ) ( )
( )
2
0 0
2
Z Y X
z z
z x x z x x
z
f z f z x f x dx
e e dx e e e dx
ze
l l l l l
l
l l l
l
-
- - - - -
-
= -
= =
=

This is a 2-stage Erlang PDF
( )
1
( 1)!
r r t
R
t e
f t
r
l
l
- -
=
-
Homework
ReadingAssignment
Example4.314.34inthetextbook;specialemphasisonExample4.34
HomeworkAssignment
Problem4.31intextbook
78
MomentsofFunctionsofMultiple
RandomVariables
79
ExpectedValueofFunctionsofMultipleRVs
Theexpectedvalueofafunctionofmultiplervs,g(X,Y),isgiven
as:
80
{ }
( ) ( )
( ) ( )
,
,
, , , jointly continuous
, , , discrete
X Y
i n X Y i n
i n
g x y f x y dxdy X Y
E Z
g x y p x y X Y

- -

JointMomentsofMultipleRVs
Thejkth jointmomentoftworvs,XandY,isgivenas:
Bysettingj=0,wecanobtainmomentsofY
Similarly,k=0yieldsmomentsofX
The(j=1,k=1)moment,E{XY},isgenerallycalledthecorrelation
ofXandY
IfE{XY}=0=>XandYareorthogonaloruncorrelated
81
{ }
( )
( )
,
,
, , jointly continuous
, , discrete
j k
X Y
j k
j k
i n X Y i n
i n
x y f x y dxdy X Y
E X Y
x y p x y X Y

- -

JointCentralMomentsofMultipleRVs
Thejkth centralmomentoftworvs,XandY,is:
Bysettingj=0andk=2,givesthevarianceofY
Similarly,settingj=2andk=0givesthevarianceofX
The(j=1,k=1)centralmoment,E{(XE{X})(YE{Y})},isgenerally
calledthecovarianceofXandY
AmoreconvenientrepresentationofCOV(X,Y)is:
Independentrvs havezerocovariance
82
{ } ( ) { } ( )
{ }
j k
E X E X Y E Y - -
{ } { } ( ) { } ( )
{ }
, COV X Y E X E X Y E Y = - -
{ } { } { } { } , COV X Y E XY E X E Y = -
CorrelationCoefficient
ThecorrelationcoefficientofXandYis
Itcanbeshownthat
Thuscorrelationcoefficientisanormalizedmeasurethat
quantifiestheamountofdependencebetweentworandom
variables
83
{ } { } { } { }
,
,
X Y
X Y X Y
COV X Y E XY E X E Y
r
s s s s
-
= =
,
1 1
X Y
r -
Source: Wikipedia Pearsonproductmomentcorrelationcoefficient,
http://en.wikipedia.org/wiki/Pearson_productmoment_correlation_coefficient
UsefulPropertiesofMomentsofMultipleRVs
P1:
IfallXsareindependent:
92
{ } { } { } { }
1 2 1 2 n n
E X X X E X E X E X + + + = + + +
( ) ( ) ( )
{ }
( )
{ }
( )
{ }
( )
{ }
1 2 1 2 n n
E g X g X g X E g X E g X E g X =
ReadingAssignment
Section4.7inthetextbook
Examples4.394.42
93
JointlyNormal(Gaussian)Random
Variables
94
Themultivariate Gaussianpdf hasthefollowingform:
Where
95
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
1
2
N
m
(
(
(
=
(
(

2
1 12 1
2
21 2 2
2
1 2
N
N
N N N
o o o
o o o
o o o
(
(
(
E =
(
(
(


FortheBivariatecase:
96
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
1
2
m

(
=
(

2
1 12
2
21 2
o o
o o
(
E =
(

97
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
1
2
m

(
=
(

2
1 12
2
21 2
o o
o o
(
E =
(

98
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
1
2
m

(
=
(

2
1 12
2
21 2
o o
o o
(
E =
(

1 2
12
1 2
( , ) Cov X X
o o
=
1 2 12 12 1 2
( , ) Cov X X o o o = =
99
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
1
2
m

(
=
(

2
1 12 1 2
2
21 2 1 2
o o o
o o o
(
E =
(

1 2
12
1 2
( , ) Cov X X
o o
=
1 2 12 12 1 2
( , ) Cov X X o o o = =
12 21
= =
100
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
1
2
m

(
=
(

2
1 1 2
2
2 1 2
o o o
o o o
(
E =
(

1 2
12
1 2
( , ) Cov X X
o o
=
1 2 12 12 1 2
( , ) Cov X X o o o = =
12 21
= =
TworandomvariablesXandYarejointlynormaliftheirpdfhas
thefollowingform
101
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
2
1 12 1 2
2
21 2 1 2
o o o
o o o
(
E =
(

1
2
1 2
2 2 2 2 2 1 1 2
2
1 2 1 2
2
2 1 2
2
1 2
1
o o o
o o o o
o o o
o o
E = =
=
thefollowingform
102
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
2
1 12 1 2
2
21 2 1 2
o o o
o o o
(
E =
(

1
2
1 2
2 2 2 2 2 1 1 2
2
1 2 1 2
2
2 1 2
2
1 2
1
o o o
o o o o
o o o
o o
E = =
=
thefollowingform
103
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
2
1 1 2
1
2
2
2 1 2
1
1
1 1
o o o

o o o
(
(
(
E =
(
(

2
1 12 1 2
2
21 2 1 2
o o o
o o o
(
E =
(

1
2
1 2
2 2 2 2 2 1 1 2
2
1 2 1 2
2
2 1 2
2
1 2
1
o o o
o o o o
o o o
o o
E = =
=
thefollowingform
104
( )
( )
2 2
,
2
,
,
2
,
1
exp 2
2 1
,
2 1
X X Y Y
X Y
X X Y Y
X Y
X Y
X Y X Y
x m x m y m y m
f x y
r
s s s s
r
ps s r

- - - -
-

- +

-

=
-
( ) ( )
1
1 2
1
2
1 1
( , ,..., ) exp
2
2
T
X N
f x x x x m x m
t

| |
= E
|
\ .
E
105
PicturefromProf.HayderRadhaslecturenotesofECE863course
( )
( )
2 2
,
2
,
,
2
,
1
exp 2
2 1
,
2 1
X X Y Y
X Y
X X Y Y
X Y
X Y
X Y X Y
x m x m y m y m
f x y
r
s s s s
r
ps s r

- - - -
-

- +

-

=
-
X
Y
Bivariate Gaussian PDF
-4 -2 0 2 4
-4
-2
0
2
4
106
( )
( )
2 2
,
2
,
,
2
,
1
exp 2
2 1
,
2 1
X X Y Y
X Y
X X Y Y
X Y
X Y
X Y X Y
x m x m y m y m
f x y
r
s s s s
r
ps s r

- - - -
-

- +

-

=
-
X
Y
-4 -2 0 2 4
-4
-2
0
2
4
107
( )
( )
2 2
,
2
,
,
2
,
1
exp 2
2 1
,
2 1
X X Y Y
X Y
X X Y Y
X Y
X Y
X Y X Y
x m x m y m y m
f x y
r
s s s s
r
ps s r

- - - -
-

- +

-

=
-
X
Y
-4 -2 0 2 4
-4
-2
0
2
4
108
( )
( )
2 2
,
2
,
,
2
,
1
exp 2
2 1
,
2 1
X X Y Y
X Y
X X Y Y
X Y
X Y
X Y X Y
x m x m y m y m
f x y
r
s s s s
r
ps s r

- - - -
-

- +

-

=
-
Ifwesettheexponentsinvolvingxandyintheabove
expressiontoaconstantK,weobtaintheequationforanellipse
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
109
( )
( )
2 2
,
2
,
,
2
,
1
exp 2
2 1
,
2 1
X X Y Y
X Y
X X Y Y
X Y
X Y
X Y X Y
x m x m y m y m
f x y
r
s s s s
r
ps s r

- - - -
-

- +

-

=
-
Ifwesettheexponentsinvolvingxandyintheabove
expressiontoaconstantK,weobtaintheequationforanellipse
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
This is the equation for
an ellipse
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
110
Theorientationoftheellipsedependsonthevalueof
correlation
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
111
correlation
When0,wehave:
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
112
correlation
When0,wehave:
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
113
correlation
When0,wehave:
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
114
When=0(i.e.,X&Yareuncorrelated),wehave:
TheellipsesareparalleltotheXandYaxis
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
115
( )
( )
2
,
,
2
,
1
exp
2 1
,
2 1
X Y
X Y
X Y X Y
K
f x y
r
ps s r

-

-

=
-
116
117
As=0increasestowards1,XandYbecomemoreandmore
correlatedandtheellipsesbecomenarrower
X
Y
-4 -2 0 2 4
-4
-2
0
2
4
118
As=0increasestowards1,XandYbecomemoreandmore
correlatedandtheellipsesbecomenarrower
X
Y
-4 -2 0 2 4
-4
-2
0
2
4
nJointlyNormalRandomVariables
RandomvariablesX1,X2,Xnarejointlynormaliftheirpdfhas
thefollowingform:
where:
119
( ) ( )
( ) ( )
{ }
( )
1 2
1
, , , 1 2
1/ 2 /2
1
exp
2
, , ,
2
n
T
X X X n
X n
x m K x m
f x f x x x
K p
-
-
- -
=

{ } { } { }
{ } { } { }
{ } { } { }
1 2
1 2
1 1 2 1
2 1 1 2
1 2
var
var
var
n
n
n
n
n n n
x x x x
m m m m
X COV X X COV X X
COV X X X COV X X
K
COV X X COV X X X

=

=

=

Covariance
matrix
ReadingAssignment
Section4.8inthetextbook
Examples4.454.48
120

Stochastic Lecture 5

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Stochastic Lecture 5

Uploaded by

Copyright:

Available Formats

Copyright Syed Ali Khayam 2009

EE 801 Analysis of Stochastic

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

Copyright Syed Ali Khayam 2009

You might also like