Elec2600 Lecture 14 1

Elec2600: Lecture 14
Pairs of continuous random variables
Review of 2D functions, differentiation and integration
Joint cumulative distribution function
Joint density function
Elec2600 Lecture 14 2
Two Random Variables
One random variable can be considered as a mapping from the
sample space to the real line.
Two random variables can be considered as a mapping from the
sample space to the plane.
Note that given the outcome of the experiment, both X and Y
are determined simultaneously.
Elec2600 Lecture 14 3
Elec2600: Lecture 14
Pairs of continuous random variables
Review of 2D functions, differentiation and integration (MATH)
Joint cumulative distribution function
Joint density function
Review: 2D Functions
A real-valued 2D function, z = f(x,y), assigns a real value to each point on the
(x,y) plane.
Think of z as being the height of the ground and regard x and y as your
east/west and north/south coordinates.
Example:
Elec2600 Lecture 14 4
x
y
contour plot
-5 0 5
-5
0
5
2 2
z x y = +
lines of constant height
Elec2600 Lecture 14 5
Differentiating 2D functions
The derivative of a 2D function with respect to x indicates how the function
changes as you move from west to east. It is calculated using the normal
rules of differentiation, assuming that y is constant.
The derivative of a 2D function with respect to y indicates how the function
changes as you move from south to north. It is calculated using the normal
rules of differentiation, assuming that x is constant.
Elec2600 Lecture 14 6
x
y
derivative with respect to y
-5 0 5
-5
0
5
x
y
derivative with respect to x
-5 0 5
-5
0
5
x
y
contour plot
-5 0 5
-5
0
5
2 2
z x y = +
2
d
dx
z x =
2
d
dy
z y =
0
d
dx
z < 0
d
dx
z >
Multiple Differentiations
Multiple differentiations are performed in sequence.
The sequence of the differentiation does not matter.
Example:
Elec2600 Lecture 14 7
2 2 4 2
( ) 2 z x y x x y y
2
= + = + +
( )
( )
4 2
2
0 2 2
4
d d d d
z x x y y
dx dy dx dy
d
x y
dx
x
2
2
= + +
= + +
=
( )
( )
4 2
3
2
4 4 0
4
d d d d
z x x y y
dy dx dy dx
d
x xy
dy
x
2
= + +
= + +
=
Elec2600 Lecture 14 8
And, of course.
2D Integration
Recall that the integral of a 1D function gives us the area under the curve
between two endpoints.
The integral of a 2D function gives us the volume under the surface within a
region.
In order to evaluate a 2D integral numerically, we need two things:
The function to be integrated f(x,y).
A region in the 2D plane over which the function will be integrated. The region
determines the limits of the integration.
A 2D integral is evaluated as a sequence of two 1D integrals.
For each 1D integral (e.g. over x), the other variable (e.g. y) is assumed to be
constant.
The order of integration can usually be switched. However, the limits of the
integral may change.
Elec2600 Lecture 14 9
Example
Integrate the function x
2
y over the region R.
Solution:
x first, then y: y first, then x:
Elec2600 Lecture 14 10
x
y
2
1
R
2 1 2 1
2 2
0 0 0 0
1
2
3
0
0
2
0
2
2
2
0
0
3
1
0
3
1 1 2
3 3 2 3
x ydxdy y x dx dy
x
y dy
y dy
y
ydy
| |
=
|
\ .
| |
|
=
|
\ .
| |
=
|
\ .
| |
|
= = =
|
\ .
} } } }
}
}
}
1 2 1 2
2 2
0 0 0 0
2
1
2
0
0
1
2
0
1
1
3
2
0
0
2
4
0
2
2
2 2
3 3
x ydydx x ydy dx
y
y dx
x dx
x
x dx
| |
=
|
\ .
| |
|
=
|
\ .
| |
=
|
\ .
| |
|
= = =
|
\ .
} } } }
}
}
}
Example
Integrate the function c (a constant)
over the shaded region.
Elec2600 Lecture 14 11
y
h
b
x
2
2
h
b
b
h
y x
x y
=
=
2
2
( )
h
b
b
h
y b x
x b y
=
=
Example (x first, then y)
Integrate the function c (a constant)
over the shaded region.
Solution:
Elec2600 Lecture 14 12
y
h
b
x
2
2
h
b
b
h
y x
x y
=
=
2
2
( )
h
b
b
h
y b x
x b y
=
=
( )
( )
( )
2 2
2 2
2
2
0 0
0 0
2
0
2
1
2
1
2
2
b b
h h
b b
h h
b
h
b
h
h b y h b y
y y
h h
b y
b
h
y
h
cdxdy c dx dy
c x dy c b y dy
b y
c by
h
b h
c bh c bh
h

| |
=
|
\ .
= =
| |
=
|
\ .
| |
= =
|
\ .
} } } }
} }
Note:
The 2D integral
of a constant over
a region is equal to
the constant times the
area of the region.
Example (y first, then x)
Integrate the function c (a constant)
over the shaded region.
Solution:
Elec2600 Lecture 14 13
y
h
b
x
2
2
h
b
b
h
y x
x y
=
=
2
2
( )
h
b
b
h
y b x
x b y
=
=
( ) ( )
( )
2 2 2 2
2 2
2 2
2 2
2
2
2
2
2 2
2
2
2
( ) ( )
0 0 0 0 0 0
( )
0 0
0
2 2
0
2 2
2 2
0
2
8
1 1
( )
b h h b h h
b b b b
b b
b
h h
b b
b
b
b
b
b
x b b x x b b x
b
x b x
b
h h
b b
b
h x h x
b b
h b
b
cdydx cdydx c dy dx c dy dx
c y dx c y dx
c xdx c b x dx
c c bx
c

| | | |
+ = +
| |
\ . \ .
= +
= +
| | | |
= +
| |
\ . \ .
= +
} } } } } } } }
} }
} }
( )
( )
2 2 2
2
2
2 2 8
1
2
h b b b
b
c b
c bh
+
=
Elec2600 Lecture 14 14
Elec2600: Lecture 14
Pairs of continuous random variables
Review of 2D functions, differentiation and integration
Joint cumulative distribution function
Joint density function
Elec2600 Lecture 14 15
J oint Cumulative Distribution Function
The joint cumulative distribution function of two random variables X and Y,
F(x,y) is defined as
The subscript indicates the variables and the order they are referred to in the
function.
{ } { } | | 9 e s s = y x y Y x X P y x F
Y X
, where ) , (
,
X
Y
(x,y)
x
y
Elec2600 Lecture 14 16
Properties of the J oint CDF

The joint cdf is continuous from the north and east.

( ) ( )
, 1 1 , 2 2 1 2 1 2
, , if and
X Y X Y
F x y F x y x x y y s s s
X
Y
(x
1
,y
1
)
(x
2
,y
2
)
,
,
,
,
( , ) 0
( , ) 0
( , ) 0
( , ) 1
X Y
X Y
X Y
X Y
F
F y
F x
F
=
=
=
=
, ,
, ,
lim ( , ) ( , )
lim ( , ) ( , )
X Y X Y
x a
X Y X Y
y b
F x y F a y
F x y F x b
+
+

=
=
Elec2600 Lecture 14 17
Properties of the J oint CDF

| | ( ) ( )
| | ( ) ( )
1 2 , 2 , 1
1 2 , 2 , 1
{ } { } , ,
{ } { } , ,
X Y X Y
X Y X Y
P x X x Y y F x y F x y
P X x y Y y F x y F x y
< s s =
s < s =

| | ( )
( ) ( )
( )
1 2 1 2 , 2 2
, 2 1 , 1 2
, 1 1
{ } { } ,
, ,
,
X Y
X Y X Y
X Y
P x X x y Y y F x y
F x y F x y
F x y
< s < s =

+

Elec2600 Lecture 14 18
Properties of the J oint CDF

Thus, the marginal statistics can be derived from the joint

(but not in general vice versa).
( )
( )
,
,
( ) [ ] [ , ] ,
( ) ,
X X Y
Y X Y
F x P X x P X x Y F x
F y F y
= s = s s =
=
Elec2600 Lecture 10 19
Elec2600 Lecture 14 20
Product Form Events
A product form event is an event that is the intersection of one-dimensional
events (events that involve only one random variable).
Product form events have a rectangular shape, and therefore can be computed
easily using the cumulative distribution function
Examples:
1 2
{ } { } A X A Y A = e e
x
y
(x
1
,y
2
)
(x
2
,y
2
)
x
y
x
1
x
2
y
2
y
1
} { } {
2 2 1
y Y x X x s s < } { } {
2 1 2 1
y Y y x X x s s s s
Example
Compute the probability of the following product form event in terms of the
cumulative distribution function. Assume that the cdf is continuous.
Solution
We can compute the probability by finding
the probability of the event {y
1
< Y y
2
} and
subtracting the probability of the event
{-x
1
X x
1
}{y
1
< Y y
2
}
Elec2600 Lecture 14 21
x
y
-x
1
x
1
y
2
y
1
1 1 2
{| | } { } X x y Y y > < s
| | | |
| |
( )
( )
1 1 2 1 2
1 1 1 2
, 2 , 1
, 1 2 , 1 1 , 1 2 , 1 1
{| | } { } { }
{ } { }
( , ) ( , )
( , ) ( , ) ( , ) ( , )
X Y X Y
X Y X Y X Y X Y
P X x y Y y P y Y y
P x X x y Y y
F y F y
F x y F x y F x y F x y
> < s = < s
< s < s
=
+

Example 5.11
The joint cdf for a pair of
random variables is given by
Since F
X,Y
(1,1)=1,
X and Y assume values
less than or equal to 1.
Elec2600 Lecture 14 22
,
0 0 or 0
0 1, 0 1
( , ) 0 1, 1
1, 0 1
1 0, 0
X Y
x y
xy x y
F x y x x y
y x y
x y
< <

s s s s

= s s >

> s s

> >

1 1
Elec2600 Lecture 14 23
The joint cdf for the vector random variable X=(X, Y) is given by
Find the marginal cdfs.

> >
=

otherwise 0
0 , 0 ) 1 )( 1 (
) , (
,
y x e e
y x F
y x
Y X
| o
0 1 ) , ( ) (
,
> = =

x e x F x F
x
Y X X
o
Solution We let one argument go to infinity
0 1 ) , ( ) (
,
> = =

y e y F y F
y
Y X Y
|
Example 5.12
Thus, X and Y individually have exponential distributions with
parameters o and |.
Elec2600 Lecture 14 24
For the cdf in Example 5.12, find the probability of the events
A = {X < 1, Y < 1},
B = {X > x, Y > y} where x > 0 and y > 0, and
D = {1<X < 2, 2<Y < 5}
) 1 )( 1 ( ) 1 , 1 ( ] [
,
| o
= = e e F A P
Y X
( )
( )
y x y x
y x y x
Y X Y X Y X
e e e e
e e e e
y x F y F x F
y Y x X P y Y P x X P B P
| o | o
| o | o


= =
+ =
+ =
s s s + s =
) 1 ( 1
)) 1 )( 1 ( ) 1 ( ) 1 (( 1
) , ( ) , ( ) , ( 1
] and [ ] [ ] [ 1 ] [
, , ,
Solution:
x
y
x
(x,y)
y
B
x
y
1
2
D
5
( )
) 1 )( 1 ( ) 1 )( 1 (
) 1 )( 1 ( ) 1 )( 1 (
) 2 , 1 ( ) 5 , 1 ( ) 2 , 2 ( ) 5 , 2 ( ] [
2 5
2 2 5 2
, , , ,
| o | o
| o | o


+
=
+ =
e e e e
e e e e
F F F F D P
Y X Y X Y X Y X
Example 5.13
Elec2600 Lecture 14 25
Approximations by Product Form Events
Many events cannot be expressed as the union of a finite number of product
form events .
However, they can be approximated arbitrarily well by rectangles of
infinitesimal width.
If the cdf is sufficiently smooth, probabilities can be expressed in terms of a
density function
x
y
(0,10)
(10,0)
y
x
(10,0)
(0,10)
{ } 10 s + = Y X A
{ }
2 2
10 B X Y = + s
, j k
B
, j k
A
, , ,
[ ] [ ] ( , ) ( , )
j k X Y j k X Y
j k j k
y x
P A P A f x y x y f x y dxdy = = A A

} }
Elec2600 Lecture 14 26
Elec2600: Lecture 14
Pairs of continuous random variables
Review of 2D functions, differentiation and integration
Joint cumulative distribution function
Joint density function
J ointly continuous random variables
We say that two random variables are jointly continuous if the joint
cumulative distribution function is continuous and differentiable.
We define the joint probability density function to be the second
derivative of the joint distribution:
The joint probability density function is also referred to as the joint pdf or
the joint density.
Note that the probability density function is not necessarily continuous.
Just as in the case of a single random variable, the probability that a pair of
continuous random variables achieves any particular combination of values is
zero:
Elec2600 Lecture 14 27
2
,
,
( , )
( , )
X Y
X Y
F x y
f x y
x y
c
=
c c
{ } { }
0 P X x Y y = = = (

Elec2600 Lecture 14 28
Properties of the joint probability density function
The joint pdf is always non-negative:
The joint cdf can be computed by integrating the joint pdf:
The total area under the joint pdf is 1:
The probability of any event A is the integral of the joint pdf over the
corresponding region in the X-Y plane:
For example,
,
[( , ) ] ( , )
X Y
A
P X Y A f x y dxdy e =
}}
, ,
( , ) ( , )
y x
X Y X Y
F x y f d d o | o |

=
} }
,
( , ) 1
X Y
f x y dxdy


=
} }
,
( , ) 0 for all ,
X Y
f x y x y >
2 2
1 1
1 2 1 2 ,
[{ } { }] ( , )
y x
X Y
y x
P x X x y Y y f x y dxdy < s < s =
} }

Elec2600 Lecture 14 29
Note that the value of the density function is not a probability.
However, it is large in regions of the plane with high probability
and small in regions of the plane with low probability.
It is true that f
X,Y
(x,y) 0 for all x and y.
However, it is not necessarily true that f
X,Y
(x,y) 1.
The joint pdf is NOT a probability
{ } { }
( )
,
,
( , )
,
y y x x
X Y
y x
X Y
P x X x x y Y y y f d d
f x y x y
o | o |
+A +A
< s + A < s + A = (

~ A A
} }
X
Y
x
y
Ax
Ay
Elec2600 Lecture 14 30
Marginal densities
The marginal densities f
X
(x) or f
Y
(y) can always be recovered from the joint
density f(x,y):
However, it is not generally possible to determine the joint density from the
marginal densities.
}
} }
} }

=
)
`

=
=

= =
| |
| o | o
| o | o
d x f
d d f
dx
d
d d f
dx
d
dx
x dF
dx
x dF
x f
Y X
x
Y X
x
Y X
Y X
X
X
) , (
) , (
) , (
) , (
) (
) (
,
,
,
,
}

= o o d y f y f
Y X Y
) , ( ) (
,
Y
X
x
y
Integrate
along this
vertical line
for f
X
(x)
Integrate along
this horizontal line
for f
Y
(y)
Example 5.15
Find the joint pdf of the pair of random variables with joint cdf given in
Example 5.11
These RVs are said to have a uniform joint pdf in the unit square.
Elec2600 Lecture 14 31
, ,
0 0 or 0 0 0 or 0
0 1, 0 1 1 0 1, 0 1
( , ) ( , ) 0 1, 1 0 0 1, 1
1, 0 1 0 1, 0 1
1 0, 0 0 0, 0
d d
dx dy
X Y X Y
x y x y
xy x y x y
F x y f x y x x y x y
y x y x y
x y x y
< < < <


s s s s s s s s


= = s s > s s >


> s s > s s

> > > >

1 1 1 1
Elec2600 Lecture 14 32
Find the constant c and the marginal densities of X and Y.

s s s
=

elsewhere. 0
0
) , (
,
x y e ce
y x f
y x
Y X
Solution:
x
y
Example 5.16
Suppose that random variables X and Y are jointly distributed according to
Integrate along y first in yellow region, the along x.
( )
( ) ( )
( )
0 0 0 0
0 0
0
2
0
1
1
1
1
2 2
x x
x y x y
x
x y x x
x x
ce e dydx c e e dy dx
c e e dx c e e dx
c
c e e dx c

= =
= =
| |
= = =
|
\ .
} } } }
} }
}
2 = c
1
y y
e dy e

=
}
0
1
x
e dx

=
}
Elec2600 Lecture 14 33
,
2
( ) ( , )
2
2
2 ( ) 2
Y X Y
x y
y
y x
y
y y y
f y f x y dx
e e dx
e e dx
e e e

=
=
=
= =
}
}
}
( )
,
0
0
( ) ( , )
2
2
2 1
X X Y
x
x y
x
x y
x x
f x f x y dy
e e dy
e e dy
e e

=
=
=
=
}
}
}
Example 5.16 (cont.)
If 0 , y s s
Otherwise, ( ) 0.
Y
f y =
If 0 , x s s
Otherwise, ( ) 0.
X
f x =
x
y
y x =
integrate along
this line
x
y
y x =
integrate along
this line
Elec2600 Lecture 14 34
Find P[X+Y < 1] in Example 5.16
Solution:
The probability is computed by integrating over the triangular region which is
the intersection of the area where the pdf is non-zero (yellow), and the region
of the plane where X + Y 1 (green).
We integrate over x first, then y:
{ } | |
( )
1
5 . 0
0
) 1 (
5 . 0
0
1
5 . 0
0
1
2 1
2
2
2 1

=
=
|
.
|

\
|
=
= s +
}
} }
} }
e
dy e e e
dy dx e e
dxdy e e Y X P
y y y
y
y
x y
y
y
y x
Example 5.17
X
Y
1
1
0.5
Elec2600 Lecture 14 35
Example
Suppose X and Y have the joint density shown.
Find the marginal densities.
Solution:
Integrating out the undesired variables, we obtain
( )

s s s s
s s s s
=
otherwise 0
1 5 . 0 , 1 5 . 0 2
5 . 0 0 , 1 0 1
, y x
y x
y x f
XY
( )

s s
s s
=
otherwise 0
1 5 . 0 5 . 1
5 . 0 0 5 . 0
x
x
x f
X
( )

s s
=
otherwise 0
1 0 1 y
y f
Y
Elec2600 Lecture 14 36
Example
Suppose X and Y have the joint density
shown.
Find P[X Y].
Solution:
Integrate the pdf in the overlap of the regions
where the pdf is non-zero and X Y (red).
Since the pdf is constant in each region, the
integral reduces to the area of the region
times the constant
( )

s s s s
s s s s
=
otherwise 0
1 5 . 0 , 1 5 . 0 2
5 . 0 0 , 1 0 1
, y x
y x
y x f
XY
| | ( ) ( )
2 2 1 1
0.5 1 0.5 2
2 2
P X Y s = +
| |
3
Thus,
8
P X Y s =
Elec2600 Lecture 14 37
Random variables that differ in type
In problems where it is necessary to deal with random variables of different
types (e.g. X is discrete and Y is continuous), it is usually preferable to work
with probabilities that have the form of a mixture of a pmf and a cdf.
rather than the joint CDF.
Events of this form are easier because the discreteness of X is explicit.
However, we can always compute the joint CDF from events of this form if
necessary.
{ } { } | |
{ } { } | |
2 1
or
y Y y k X P
y Y k X P
s < =
s =
Elec2600 Lecture 14 38
Let X be the input to a communication channel and let Y be the output. The
input is either 1 volt or +1 volt with equal probability. The output is the input
plus a noise voltage N that is uniformly distributed in the interval from 2 volts
to +2 volts. Find P[X = +1, Y < 0].
Solution:
] 1 [ ] 1 | 0 [ ] 0 , 1 [ + = + = s = s + = X P X Y P Y X P
When the input X = +1, the output Y is uniformly distributed in the interval
[-1,3]. Thus,
Alternatively, the noise N must be less than or equal to -1.
Therefore,
4
1
] 1 | 0 [ = + = s X Y P
According to the definition of the conditional probability:
Example 4.14
8
1
2
1
4
1
] 0 , 1 [ = = s + = Y X P
39
Major Points from this Lecture:
Joint CDF of pairs of random variables
Joint PDF of pairs of random variables
Marginal densities (and how to compute them)
Product form events