CHAPTER THREE OPERATIONS ON ONE RANDOM VARIABLE EXPECTATION

3.0 INTRODUCTION The random variable was introduced in Chapter 2 as a means of providing a systematic definition oof events defined on a sample space. Specifically, it formed a mathematical model for describing characteristics of some real, physical world random phenomenon. In this chapter we extend our wor to include some important operations that may be performed on a random variable. Most of these operations are based on a single concept-expectation.

3.1 EXPECTATION Expectation is the name given to the process of averaging when a random variable is involved. For a random variable X, the expected value of X, the mean value of X, or the statistical average of X. Occasionally we also use the notation X which is read the same way as E[X]; that is, X = E[X]. Nearly everyone is familiar problem to the new concept of expectation may be the casiest way to proceed.

Expected Value of a Random Variable The everyday averaging procedure used in the aove example carries over directly to random variables. In fact, if X is the discrete random variable fractional dollar vallue of pocket coins, it has 100 discrete values x1 that occur with probabilities P(x1), and its expected value E[X]is found in the same way as in the example:

E [ X ] = x1 P(x1 )
| =|

100

(3.1-1)

The values x1 identify with the fractional dollar values in the example, while P(x1) is identified with the ratio of the number of people for the given dollar value to the total number of people had been used in the sample of the example, all fractional dollar values would have shown up and the ratios would have approached P(x1). Thus, the average in the example would have become more like (3.1-1) for many more than 90 people. In general, the expected value of any random variable X is defined by

E[X ] = x = xfx (x)dx

(3.1-2)

If X happens to be discrete with N possible values x1 having probabilities P(x1) of occurence, then

f x ( x ) = P( x1 ) ( x x1 )
| =|
From (2.3-5). Upon substitution of (3.1-3))into (3.1-2), we have

E [x ] =

x P (x )
|= | 1 1

discrete random variable

(3.1-4)

Hencc, (3.1-1) is a special case of (3.1-4) when N=100. for some discrete random variables, N may be infinite in (3.1-3)and (3.1-4).

Example 3.1-2 We determine the mean value of the continuous , exponentially distributed random variable for wich (2.5-9) applies:

1 f (x ) = e x b 0

x a /b

x>a x<a

From (3.1-2) and an integral from Appendix C:

E [x ] =

x ( x a )/ b ea / b e dx = b b

xe x / b dx = a + b

If a random variables density is symmetrical about a line x=a, then E[X]=a if fx (x+a)= fx(-x+a)

(3.1-5)

Expected Value of a Function of a Random Variable As will be evident in the next section, many useful parameters relating to a random variable X can be derived by finding the expected value of a real function (.) of X. It can be shown (Papoulis, 1965,p. 142) that this expected value is given by

E [g ( x )] = g ( x ) f x (x )dx

If X is a discrte random variable, (3.1-3) applies and (3.1-6) reduces to

E [g ( x )] =

g (x )P (x )
|= | 1 1

discrete random variable

(3.1-7)

Where N may be infinite for some random variables.

Example 3.1-3 It is known that a particular random voltage can be represented as a Rayleigh random variable V having a density function given by (2.5-11) with a=0 and b=5. the voltage is applied to a device that generates a voltage Y= g(V)=V2 that is equal, numerically, to the power in V (in a 1- resistor). We find the average power in in V by means of (3.16):

Pover in By letting

V = E [g (V )] = E V 2 =

[ ]

2v 3 v 2 / 5 e dv 5

= v2 / 5, d = 2vdv/ 5,
Power in

V = 5 e d = 5W
0

after using (C-46)

Conditional Expected Value If, in(3.1-2), fx(x|B), where B is any event defined on the sample space, we have the conditional expected value of X, denoted E[X | B ]:

E [x | B ] = xf x ( x | B )dx

(3.1-8)

One way to define event B, as shown in Chapter 2, is to let it depend on the random variable X by defining

B = {x b}

< b <
x<b xb

(3.1-)

We showed there that

fx ( x ) fx ( x | X b ) = b 0 fx ( x )dx

(3.1-10)

Thus, by substituting (3.1-10)into (3.1-8):

E [x | x b] =
3.2 MOMENTS

xf x ( x )dx f x ( x )dx
(3.1-11)

Which is the mean value of X when X is constrained to the set {X b}.

An immediate application of the expected value of a function g(.) of a random variable X is in calculating moments. Two types of moments are of interest, those abou the origin and those about the mean. Moments About the Origin The function

g (x ) = x n

n = 0,1,2,...

(3.2-1)

When used in (3.1-6) gives the moments about the origin of the random variable X. Denote the nth moment by mn. Then,

mn = E X n = x n fx( x )dx

[ ]

(3.2-2)

Clearly m0 =1, the area of the function fx(x), while m1=x, the expected value of X.

Central Moments Moments about the mean value of X are called central moments and are given the symbol n. They are defined as the expected value of the function

g (x ) = (x x )
Which is

n = 0,1,2,...
n n

(3.2-3)

= E [(X X ) ] =

(x X ) fx(x )dx
(3.2-4)

The moment 0=1, the area of fx(x), while 1=0.(why?)

Variance and Skew The second central moment 2 is so important we shall give it the name variance and the special notation 2x . Thus, variance is given by

x2 = 2 = E (x X ) =
2

(x X ) fx(x )dx
2

(3.2-5)

The positive square rool x of variance is called the stantdard deviation of X; it is a measure of the spread in the function fx(x) about the mean.

x2 = E [X 2 2 XX + X 2 ] = E [X 2 ] 2 XE [X ] + X 2
= E X 2 X 2 = m2 m12

[ ]

(3.2-6)

Example 3.2-1 Let X have the exponential density function given in Example 3.1-2 By substitution into (3.2-5), the variance of X is

x2 = (x X )
a

1 ( x a )/ b e dx b

By making the change of variable = x X we obtain

e ( xa )/ b = b
2 x

a x

2 e / b d = (a + b x )2 + b 2

The subscript indicates that 2x is the variance of a random variable X. For a random variable Y its variance would be a We use the fact that the expected value of a sum of functions of X equals the sum of expected values of individual functions, as the reader can readily verify as an exercise.

After using an integral from Appendix C. However, from Example 3.1-2, 2 2 X= E[X]= (a+b), so

= b

The reader may wish to verify this result by finding the second moment E[x2] and using (3.2-6). The third central moment 3 = E[(X-X)3] is a measure of the asymmetry of fx(x) about x=X=m1. It will be called the skew of the density function. If a density is symmetric about x=X, it has zero skew. In fact, for this case n =0 for all odd values of n. (Why?) The normalized third central moment 3 / 3x is nown as skewness of the density function, or, alternatively, as the coefficient of skewness. Example 3.2-2 We continue Example 3.2-1 and compute the skew and coefficient of skewness for the exponential density. From (3.2-4) with n= 3 we have

3 = E (X X) = E = E[X3 3XX2 +3X2 X X3]

3
2 = X 3 3x X 2 + 2 x 3 = X 3 3 X x + x 2 + 2 x 3 2 = X 3 3 X x x 3

Next, we have

x3 =

p ( x a )/ p qx = 3 + 35 p + pap5 + pp3 x3

2 2 After using (C-48). On substituting X= a+b and x = b from the earlier example, and reducing the algebra we find

3 = 2b 3 3 =2 3 x
This density has a relatively large coefficient of skwness, as can be seen intuitively from Figure 2.5-3.

3.3 FUNCTIONS THAT GIVE MOMENTS Two functions can be deined that allow moments to be calculated for a random variable X. They are the characteristic function and the moment generating function. Characteristic Function The characteristic function of a random varible X is defined by
x x ( ) = E e

(3.3-1)

Where f= 1. It is a function of the real number - < < . If (3.3-1) is written in terms of the density function, x ( ) is seen to be the Fourier transform (with the sign of reversed) of fx(x):

x ( ) = fx( x )e fx dx

(3.3-2)

Because of fact, if x () is known, x(x) can be found from the inverse Fourier transform (with sign of x reversed)

fx( x ) =

1 2

x ( )e fx d
(3.3-3)

By formal differentiation of (3.3-2) n times with respect to and setting =0 in the derivative, we may sho that the nth moment of x is given by

mn = ( j )

d n x ( ) d n

(3.3-4)

A major advantage of using x() to find moments is that x() always exist (Davenport, 1970, p. 426), so the moments can always be found if x() is known, provided, of course, the derivatives of x() exist. It can be shown that the maximum magnitude of a characteristic function is unity and occurs at = 0; that is, (3.3-5) | x() | x(0)=1

Example 3.3-1 Again we consider the random variable with the exponential density of Example 3.1-2 and find its characteristic function and first moment. By substituting the density function into (3.3-2),we get

x ( ) =

ea / b 1 ( x a )/ b fx e e dx = b b

e (1/ b f ) x dx

Evaluation of the integral follows the use of an integral from Appendix C:

ea / b x ( ) = b

e (1/ b f )x (1 / b j(1) )

e j = 1 j b
The derivative of x () is

d x ( ) f ja jb =e + d 1 jb (1 jb)2
so the first moment becomes

d x ( ) m1 = ( j ) d

=0

= a + b,

n agreement with m1 found in Example 3.1-2.

Moment Generating Function Another statiscal average closely related to the characteristic function is the moment generating function, defined by Mx(v)= E[evx] Where v is a real number -< v < . Thus, Mx(v) is given by (3.3-6)

(3.3-7) The main advantage of the moment generating function derives from its ability to give the moments. Moments are related to Mx(v) by the expression:

M x (v ) =

f x ( x )e vx dx

d n M x (v ) mn = dv n

v =0

(3.3-8)

The main disadvantage of the moment generating function, as opposed to the characteristic function, is that it may not exist for all random variables. In fact, Mx (v) exists only if all the moments exist (Davenport and Root, 1958, p. 52). Example 3.3-2 To illustrate the calculation and use of the moment generating function, let us reconsider the exponential density of the carlier examples. On use of (3.3-7) we have

M x (v ) =

1 ( x a )/ b vx e e dx b

e a / b [v (1 / b )]x = e dx b a e av = 1 bv

In evaluaating Mx(v) we have used an integral from Appendix C. Bv differentiation we have the first moment

m1 =

eav[a(1 bv) + b] = v=0 = a + b 2 (1 bv)

Which, of course, is the same as previously found. 3.4 TRANSFORMATIONS OF A RANDOM VARIABLE Quite often one may wish to transform (change) one random variable X into a new random variable Y by means of a transformation Y=T(X) (3.4-1)

dM x (v ) dv

v =0

Continuous. Note that the transfrmation in all three cases is asumed continuous. The cocepts introduced in these three situations are broad enough that the reader should have no difficulty in extending them to other cases (see Problem 3-32). Monotonic Transformations of a Continuous Random Variable A transformation T is called monotonically increasing if T(x1)<T(x2) for any x1< x2. t is monotonically decreasing if T(x1)<T(x2) for any x1< x2. Consider first the increasing transformation. We assume that 7is continuous and differentiable at all values of x for which fx (x)0. Let Y have a particular value y0 corresponding to the particular value x0 of X as shown in Figure 3.4-2a. The two numbers are related by

y0 = T (x0)

or

x0 = T-1 (y0)
(3.4-2)

Where T-1 repressent the inverse of the transformation T. Now the probability of the event {Y y0} must equal the probability of the event {x x0} because of the one-to-one correspondence between X and Y. Thus,

Fy ( y0 ) = P{Y y0 } = P{X x0 } = Fx ( x0 )

Figure 3.4-2 Monotonic transformations: (a) increasing, and (b) decreasing. [Adapted from Peebles (1976) with permission of publishers AddisonWesley, Advanced Book Program.]

Or

y0

fy ( y )dy =

x0 =T 1( y0 )

fx( x )dx
(3.4-4)

Next, we differantiate both sides of (3.4-4) with respect to y0 using Leibnizs rule to get 1 1 0 0 0 y x 0 (3.4-5)

f (y ) = f T

( y )] dT ( y )
dy

Since this result applies for any y0 ,we may now drop the subscript and write 1 1 y x (3.4-6) Or, more compactly,

f (y) = f T

( y )] dT ( y )
dy
dx dy

f y ( y ) = f x (x )

(3.4-7)

In (3.4-7) it is understood that x is a function of y through (3.4-2). A consideration of Figure 3.4-2b for the decreasing transformation verifies that

Fy ( y0 ) = P{Y y0 } = P{X x0 } = 1 Fx ( x0 ).
(3.4-8)

A repetition of the steps leading to (3.4-6) will again produce (3.4-6) except that the right side is negative. However, since the slope of T-1 (y) is also negative, we conclude that for either type of monotonic transformation

f y ( y ) = f x T 1 ( y )
or simply

] dT dy( y )
1

(3.4-9)

(y ) =

f x (x )

dx dy

Leibnizs rule, after the great German mathematician Gottfried Wilhelm von Leibniz (1646-1716), states that, if H(x,u) is cntinuous in x and u and

G (u ) =

d (u )

e (u )

H ( x, u )dx

Then the derivative of the integral with respect to the parameter u is

dG (u ) d (u ) (u ) H ( x, u ) = H [ (u ), u ] + dx (u ) du du u

Example 3.4-1 If we take T to be the linear transformation Y= T(X)= aX + b, where a and b are any real constans, then X= T-1 (Y)= (Y b)/a and dx / dy = 1/ a. From (3.4-9)

y b 1 fx (y) = fx a a

If X is assumed to be gaussian with the density function given by (2.4-1), we get

f y (y) =

1
2 2 x

e [( y b )/ a a x ]

/ 2x

1 a

1
2 2a 2 x

e [ y (aax +b )]

/ 2x 2

which is the density function of another gaussian random variable having

ay = aax + b

and

2y = a2 2x

Thus, a linear transformation of a gaussian random variable produces another gaussian random variable. A linear amplifier having a random voltage X as its input is one example of a linear transformation. Nonmnotonic Transformations of a Continuous Random Variable A transformation may not be monotonic in the more general case. Figure 3.4-3 illustrates one such transformation. There may now be more than one interval of values of X that correspond to the event {Y y0}. For the value of y shown in the figure, the event {Y y0} corresponds to the event {X x1 and x2 X x3 }. Thus, the probability of the event {Y y0} now equals the probability

Figure 3.4-3 nonmonotonic transformation . [Adapted from Peebles (1976) with permission of publishers Addison-Wesley, Advanced Book Program.]

Of the event {x values yielding Y y0 }, which we shall write as {x| Y y0 }. In other words

Fy ( y0 ) = P{Y y0 } = P{x | Y yo } =

( x|Y y0 )

f x ( x )dx
(3.4-11)

Formally, one may differantiate to obtain the density function of Y:

f y (y0 ) =

d dy 0

x |Y y 0 )

f x ( x )dx

(3.4-12)

Although we shall not give a roof, the density function is also given by (Papoulis, 1965, p. 126)

f y (y) =
n

dT ( x ) dx x = xn

f x ( xn )

(3.4-13)

Where the sum is taken so as to include all the roots xn, n=1,2,...,which are the real solutions of the equation y=T(x) (3.4-14 We illustrate the above concepts by an example. Example 3.4-2 We find fy(y) for the square- law transformation Y=T(X)=cX2 Shown in Figure 3.4-4, where c is a real constant c >0. we shall use both the procedure leading to (3.4-12) and that leading to (3.4-13). In the former case, the event {Y y} occurs when y / c x y / c = {x | Y y}, so (3.4-12) becomes

fy ( y ) =

d dy

y /c y /c

fx ( x )dx

y0

Upon use of Leibnizs rule we obtain

fy ( y ) = fx

y ( y / c ) d ( dy / c ) fx( fx ( y / c ) + fx ( y / c ) =
2 cy

y/c

) d ( dyy / c )

y0

If y= T(x) has no real roots for a given value of y, then T (y)=0.

Figure 3.4-4 square-law transformation. [Adapted from Peebles (1976) with permission of publishers Addison-Wesley, Advanced Book Program.]

In the latter case where we use (3.4-13), we have X and x2 = y / c . Furthermore, dT(x)/dx = 2cx so

= Y / c,Y 0, so

x1 = y / c

dT ( x ) y = 2 cx1 = 2 c = 2 cy dx x = x1 c dT ( x ) = 2 cy dx x = x 2
From (3.4-13) we again have

fy ( y ) =

fx

y / c + fx y / c 2 cy

) (

y0

Transformation of a Discrete Random Variable If X is a discrete random variable while Y=T(X) is a continuous transformation, the problem is especially simple. Here

fx(= ) P( xn ) ( x xn )
n n

(3.4-15)

Fx ( x ) = P( xn )u ( x xn )

(3.4-16)

Where the sum is taken to include all the possible values xn , n= 1,2,..., of X. If the transformation is monotonic, there is a one-to-ane correspondence between X and Y so that a set {yn } corresponds to the set {xn} through the equation yn = T(xn). :The probability P(yn) equals P(xn). Thus,

f y ( y ) = P( yn ) ( y yn ) Fy ( y ) = P( yn )u ( y yn )
n n

(3.4-17) (3.4-18)

CHAPTER4 MULTIPLERANDOM VARIABLES

4.0INTRODUCTION
InChapters2and3,variousaspectsofthetheoryofsinglerandomvariablewere studied.Therandomvariablewasfoundtobeapowerfulconcept. Itenabledmany realisticproblemstobedescribedinaprobabilisticwaysuchthatpracticalmeasures couldbeappliedtotheproblemeventhoughitwasrandom.Wehaveseenthatshell impactpositionalongthelineoffirefromacannontoatargetcanbedescribedbythe randomvariable.Fromknowledgeoftheprobabilitydistributionordensityfunctionof impactposition.Wecansolveforsuchpracticalmeasuresasthe meanvalueofimpact position,itsvariance,andskew. Thesemeasuresarenot,however,acompleteenoughdescriptionoftheproblemin mostcases. Naturally,wemayalsobeinterestedinhowmuchtheimpactpositions deviatefromthelineoffirein,say,theperpendicular(crossfire)direction.Inother words,wepefertodescribeimpactpositionasapointinaplaneasopposedtobeinga pointalongaline.Tohandlesuchsituationsitisnecessaryto extendthetheoryto includeseverlrandomvariables.Weaccomplishtheseextentionsinthisandthenext chapter. Fortunately,manysituationsofinterestinengineeringcanbehandledbythe theoryoftworandomvariables.Becauseofthisfact,weemphasizethetwovariable case,althoughthemoregeneraltheoryisalsostatedinmostdiscussionstofollow.

4.1VECTORRANDOMVARIABLES
SupposetworandomvariablesXandYaredefinedonasamplespaceS, wherespecificvaluesofXandYaredenotedbyxandy,respectively.Then anyorderedpairofnumbers(x,y)maybeconvenientlyconsidered tobea randompointinthexyplane.Thepointmaybetakenasaspecificvalueofa vectorrandomvariableorarandomvector.Figure4.11illustratesthe mapinginvolvedingoingfromStothexyplane. Theplaneofallpoints(x,y)intherangesofXandYmaybe consideredanewsamplespace.Itisinrealityavectorspacewherethe componentsofanyvectorarethevaluesofthevaluesoftherandomXandY. Thenewspacehasbeencalledtherangesamplespaceorthetwodimesional productspace.Weshalljustcallitajointsamplespaceandgiveitthesymbol SJ. Asinthecaseofonerandomvariable,letusdefineaneventAby A={X x} (4.11) AsimilareventBcanbedefinedforY. B={Y y} (4.12)

EventsAandBrefertothesamplespaceS,whileevents{X x}and{Y y}refertothe jointsamplespaceS.Figure4.12illustratesthecorrespondences Therearesomespecificconditionsthatmustbesatisfiedinacomplete definitionofarandomvectorTheyaresomewhatadvancedforour scopeandweshall simplyassumethevalidityofourrandomvectors. DonotforgetthatelementssofSfromthelinkbetweenthetwoeventssince bywriting{X x}wereallyrefertothesetofthosessuchthatX(s) xforsomereal numberx.Asimilarstatementholdsfortheevent{Y y}.

Figure4.11MappingfromthesamplespaceStothejointsamplespaceS,(xyplane).

Figure4.12ComparisonsofeventsinSwiththoseinS,

betweeneventsinthetwospaces.EventAcorrespondstoallpointsinS,forwhichtheX coordinatevaluesarenotgreaterthanx.Similarly,eventBcorrespondstotheY coordinatevaluesinS,notexceedingy.OfspecialinterestistoobservethattheeventA B defined on S corresponds to the joint event {X x and Y y} defined on SJ , which we write {X x, Y y}. This joint event is shown crosshatched in Figure 4.12. InthemoregeneralcasewhereNrandomvariablesX1 ,X2 ,...,XN aredefinedon asampleS,weconsiderthemtobecomponentsofanNdimensionalrandomvectoror Ndimensionalrandomvariable.ThejointsamplespaceSisnowNdimensional.

4.2JOINTDISTRIBUTIONANDITSPROPERTIES
TheprobabilitiesofthetwoeventsA={X x}andB={Y y}havealreadybeendefinedas functionsofxandy,respectively,calledprobabilitydistributionfunctions: Fx (x)=P{X x} Fy (y)=P{Y y} (4.21)

Wemustintroduceanewconcepttoincludetheprobabilityofthejointevent{X x,Y y}. JointDistributionFunction Wedefinetheprobabilityofthejointevent{X x,Y y},whichisafunctionofthe numbersxandy,byajointprobabilitydistributionfuncionand denoteitbythesymbolFx, y (x,y).Hence, F,(x,y)=P{X x,Y y} (4.23)

ItshouldbeclearthatP{X x,Y y}=P(A B ), where the joint event A B is defined on S. To illustrate joint distribution, we take an example where both random variables X and Y are discrete. Example 4.2-1 Assume that the joint sample space S has only three possible elements: (1,1), (2,1), and (3,3). The probabilities of these elements are assumed to be P(1,1)=0.2, P(2,1)= 0.3, and P(2,1) = 0.3, and P(3,3) = 0.5. We find Fx.y (x,y). Inconstructingthejointdistributionfunction,weobservethat theevent{X x,Y y}hasnoelementsforanyx<1and/ory<1.Onlyatthepoint(1,1)doesthefunction assumeastepvalue.Solongasx 1andy 1,thisprobabilityismaintainedsothatFx.y (x,y). Has a stair step holding in the region x 1andy 1asshowninFigure4.21a.

Forlargerxandy,thepoint(2,1)producesasecondstairstep ofamplitude0.3whichholds intheregionx 2andy 1.Thesecondstepaddsdothefirst.Finally,athirdstairstepof amplitude0.5isaddedtothefirsttwowhenxandyareintheregionx 3andy 3.The finalfunctionisshowninFigure4.21a. Theprecedingexamplecanbeusedtoidentifytheformofthejointdistribution functionfortwogeneraldiscreterandomvariables.LetXhaveNpossiblevaluesxn andY haveMpossiblevaluesym ,then

Fx. y (x, y ) = P( xn , ym )u (x xn )u ( y ym )
n 1 m 1

(4.24)

WhereP(xn ,ym )istheprobabilityofthejointevent{X=xn,Y=ym }andu(.)istheunit stepfunction.AsseeninExample4.21,somecouples(xn ,ym )mayhavezeroprobability. InsomecasesNorM,orboth,maybeinfinite. IfFx.y (x,y)isplottedforcontinuousrandomvariablesXandY,thesamegeneral behaviorasshowninFigure4.21aisobtainedexceptthesurfacebecomessmoothandhas nostairstepdiscontinuities. ForNrandomvariablesXn ,n=1,2,...,N,thegeneralizationof(4.23)isdirect.The jointdistributionfunction,denotedbyFx1.x2....xn (x1,x2,...,xN ),isdefinedastheprobability ofthejointevent{X1 x1 ,X2 x2 ,...,XN xN }: (4.25)

ForasinglerandomvariableX,wefoundinChapter2thatF(x) couldbeexpressedin generalasthesumofafunctionofstairstepform(duetothediscreteportionofa mixedrandomvariableX)andafunctionthatwascontinuous(duetothecontinuous portionofX).SuchasimpledecompositionofthejointdistributionwhenN>1isnot generallytrue[Cramer,1946,Section8.4].However,

Figure4.21Ajointdistributionfunction (a),anditscorrespondingjointdensity ffunction(b),thatapplytoExamples4.2 2.

itistruethatjointdensityfunctionsinpracticeoftencorrespondtoallrandom variablesbeingeitherdiscreteorcontinuous.Therefore,weshalllimitour considerationinthisbookalmostentirelytothesetwocaseswhenN>1.

PropertiesoftheJointDistribution
AjointdistributionfunctionfortworandomvariablesXandYhasseveralproperties thatfollowreadilyfromitsdefinition.Welistthem: (1) Fx , y ( , ) = 0 (2) Fx , y (, ) = 1 (3) 0 Fx , y ( x, y ) 1

Fx , y ( , y ) = 0

Fx , y (x, ) = 0

(4.26a) (4.26b) (4.26c) (4.26d) (4.26e) (4.26f)

Fx , y ( x, y ) (4)sanondecreasingfunctionofbothxandy
(5)
= P{x1 < X x2 , y1 < Y y2 } 0 FX ,Y ( x2 , y2 ) + FX ,Y ( x1 , y1 ) FX ,Y ( x1 , y2 ) FX ,Y ( x2 , y1 )

(6) Fx , y ( x, ) = Fx ( x ) Fx , y (, y ) = Fy ( y )

Thefirstfiveofthesepropertiesarejustthetwodimensionalextensionsofthe propertiesofonerandomvariablegivenin(2.22).Properties1, 2,and5maybe usedasteststodeterminewhethersomefunctioncanbeavaliddistribution functionfortworandomvariablesXandY(Papoulis,1965,p.169).Property6 deservesafewspecialcomments.

MarginalDistributionFunctions

Property6abovestatesthatthedistributionfunctionofonerandomvariablecan beobtainedbysettingthevalueoftheothervariabletoinfinityinFx.y (x,y).The functionsFx (x)orFy (y)obtainedinthismannerarecalledmarginaldistribution functions. Tojustifyproperty6,itiseasiesttoreturntothebasiceventsAandB, definedbyA={X x}andB={Y y},andobservethatF(x,y)=P{X x,Y y}= P(AB). Now if we set y to , this is equivalent to making B the certain event; that is, B= {Y } =S. Furthermore , since A B = A S =A, hen we have Fx.y (x, )=P(AS) = P(A)= P{X x }= Fx (x).Asimilarproofcanbestated forobtainingFy (y).

Example4.22 WefindexplicitexpressionsforFx.y (x,y),andthemarginaldistributionsFx(x) andFy (y)forthejointsamplespaceofExample4.21. Thejointdistributionderivesfrom(4.24)ifwerecognizethatonlythree probabilitiesarenonzero:

Fx , y ( x, y ) = P(1,1)u ( x 1)u ( y 1)
+ P(2,1)u ( x 2 )u ( y 1) + P(3,3)u (x 3)u ( y 3)

WhereP(1,1)=0.2,P(2,1)=0.3,andP(3,3)=0.5.Ifwesety= : Fx (x)=Fx,y (x, ) =P(1,1)u(x1)+P(2.1)u(x2)+P(3,3)u(x3) =0.2u(x1)+0.3u(x2)+0.5u(x3) Ifwesetx= : Fy (y) = Fx.y (,y) = 0.2u(y-1) + 0.3u(y-1) + 0.5 u (y-3) = 0.5u(y-1) + 0.5u(y-3) Plots of these marginal distributions are shown in Figure 4.2-2.

Figure4.22Marginaldistributions applicabletoFigure4.21and Example4.22:(a)Fx (x)and(b)Fy (y).

FromanNdimensionaljointdistributionfunctionwemayobtainakdimensionalmarginal distributionfunction,foranyselectedgroupofkoftheNrandomvariables,bysettingthe valuesoftheotherNkrandomvariablestoinfinity.Herekcanbeanyinteger1,2,3,..., N1.

4.3JOINTDENSITYANDITSPROPERTIES
Inthissectiontheconceptofaprobabilitydensityfunctionis extentedtoinclude multiplerandomvariables. JointDensityFunction f x. y FortworandomvariablesXandY,thejointprobabilitydensity function,denoted ( x, y ), isdefinedbythesecondderivativeofthejointdistributionfunctionwhereveritexists:

f x . y ( x, y ) =

2 Fx. y ( x, y ) xy

f x . y x, y Weshallreferooftentoasthejointdensityfunction. f x . y x, y IfXandYarediscreterandomvariables, willpossessstep discontinuities(seeExample4.21andFigure4.21).Derivativesatthesediscontinuities

Arenormallyundefined.However,byadmittingimpulsefunctions(seeApendixA), we
are able to define f x. y x, y atthesepoints.Therefore,thejointdensityfunctionmay befoundforanytwodiscreterandomvariablesbysubstitutionof(4.24)into(4.31):

n =1 m =1 (4.32) Anexampleofthejointdensityfunctionoftwodiscreterandomvariablesisshownin Figure4.21b. WhenNrandomvariablesX1 ,X2 ,...,XN areinvolved,thejointdensityfunction becomestheNfoldpartialderivativeoftheNdimensionaldistributionfunction:

f x. y (x, y ) = P(xn , ym ) ( x xn ) ( y ym )

f x1 . x2 ... x N (x1 , x2 ,..., xN ) =

N Fx1 , x2 ,..., xN ( x1 , x2 ,..., x N ) x1x2 ...x N

(4.33)

Bydirectintegrationthisresultisequivalentto

Fx1 . x2 ... xN (x1 , x2 ,..., x N )

xN

...

x2

x1

f x1 . x2 .... x N (1 , 2 ,..., N )d1d 2 ...d N

(4.34)

PropertiesoftheJointDensity Severalpropertiesofajointdensityfunctionmaybelistedthatderivefromitsdefinition (4.31)andtheproperties(4.26)ofthejointdistributionfunction:

(1) f x. y (x, y ) 0 (2) f x. y (1. 2 )d1d 2 (3)Fx. y (x, y ) = f x. y (1. 2 )d1d 2

y x

(4.35a) (4.35b)

(4.35c)

(4)Fx (x ) f x. y (1. 2 )d1d 2

y x

(4.35d)

Fx ( x )

f x. y (1. 2 )d1d 2

(4.35e)

(5)P{x1 < X x2 , y1 < Y y2 } = y x

y2
1

x2
1

f x. y ( x, y )dxdy
(4.35f)

(6) f x (x ) = f x. y (x, y )dy

f y (y) =

(4.35h) Properties1and2maybeusedassufficenttesttodetermineif somefunctioncanbea validdensityfunction.Bothtestsmustbesatisfied(Papoulis,1965,p.169). Thefirstfiveofthesepropertiesarereadilyverifiedfromcarlierworkandthe readershouldgothroughthenecessarylogicasanexercise.Property6introducesanew concept. MarginalDensityFunctions

f x. y ( x, y )dx

(4.35g)

Thefunctionsfx(x)andfy(y)ofproperty6arecaledmarginalprobabilitydensityfunctions orjustmarginaldensityfunctions.Theyarethedensityfunctionsofthesinglevariables XandYandaredefinedasthederivativesofthemarginaldistributionfunctions:

f x (x ) = f y (y) =

dFx (x ) dx dFy ( y ) dy

Bysubstituting(4.35d)and(4.35e)into(4.36)and(4.37),respectively,weareableto verifytheequationsofproperty6. Weshallillustratethecalculationofmarginaldensityfunctionsfromagiven jointdensityfunctionwithanexample. Example4.31 Wefindfx(x)andfy(y)whenthejointdensityfunctionisgivenby(Clarke andDisney,1970,p.108): x ( y +1)

f x. y ( x, y ) = u ( x )u ( y )xe

From(4.35g)andtheaboveequation:

f x ( x ) = u ( x )xe x ( y +1)dy = u ( x )xe x e xy dy

= u ( x )xe

(1 / x ) = u ( x )e x

afterusinganintegralfromAppendixC. From(4.35h):

f y ( y ) = u ( y )xe x ( y +1)dx =
0

u( y ) ( y + 1)2

afterusinganotherintegralfromAppendixC.

ForNrandomvariablesX1 ,X2 ,...,XN ,thekdimensionalmarginaldensityfunctionis definedasthekfoldpartialderivativeofthekdimensionalmarginaldistribution function.Itcanalsobefoundfromthejointdensityfunctionbyintegratingoutall variablesexceptthekvariablesofinterestX1 ,X2 ,...,Xk :

f x1 . x2 ,..., x1 (x1 , x2 ,..., xk )

= ...

f x1 , x2 ,..., xN ( x1 , x2 ,..., x N )dxk +1dx N

(4.38)

4.4CONDITIONALDISTRIBUTIONANDDENSITY InSection2.6,theconditionaldistributionfunctionofarandomvariableX,givensome eventB,wasdefinedas

Fx ( x | B ) = P{X x | B} =

P{X x B} P (B )

(4.41)

foranyeventBwithnonzeroprobability.Thecorrespondingconditionaldensityfunction wasdefinedthroughthederivative (4.42) Inthissectionthesetwofunctionsareextendedtoincludeasecondrandomvariable throughsuitabledefinitionsofeventB.

f x (x | B ) =

dFx (x | B ) dx

ConditionalDistributionandDensity PointConditioning Ofteninpracticalproblemsweareinterestedinthedistributionfunctionofonerandom variableXconditionedbythefactthatasecondrandomvariable Yhassomespecificvalue y.ThisiscalledpointconditioningandwecanhandlesuchproblemsbydefiningeventBby

B = {y y < Y y + y}

(4.43)

Wherey is a small quantity that we evetually let approach o. For this event, (4.4-1) can be written

F ( x | y y < Y y + y ) =
x

y + y x

y y

f x , y (1 , 2 )d1d 2 f y ( )d

y + y

(4.44)

y y

Wherewehaveused(4.35f)and(2.36d). Considertwocasesof(4.44).Inthefirstcase,assumeXandYarebothdiscrete randomvariableswithvaluesx1,i=1,2,...,N,andyJ ,j=1,2,...,M,respectively,whilethe prabilitiesofthesevaluesaredenotedP(x1)andP(yJ),

Respectively.Theprobabilityofthejointoccurenceofx1 andyJ isdenotedP(x1,yJ ).Thus,

f y ( y ) = P( y J ) ( y y J )
J 1

(4.45)

f x. y ( x, y ) = P( x1 , y J ) ( x x1 ) ( y y J )
11 J 1

(4.46)

Nowsupposethatthespecificvalueofyofinterestisy.With substitutionof(4.45)and (4.46)into(4.44)andallowingy 0, we obtain

Fx ( x | Y = yk ) =
11

P( x1 , yk ) u ( x x1 ) P ( yk )

(4.4-7)

After differentiation we have

f x ( x | Y = yk ) =
11

P( x1 , yk ) (x x1 ) P( yk )
(4.4-8)

Example 4.41 To illustrate the use of(4.48)assume ajoint density function asgiven in Figure 4.41a.Here P(x1 ,y1)=2/15,P(x2 ,y1)=3/15,etc.SinceP(y3)=(4/15)+(5/15)=9/15, use of(4.48)will give fx (x|Y=y3)asshown inFigure 4.41b. The second case of(4.44)that isofinterest corresponds to Xand Yboth continuous random variables.Asy 0 the denominator in (4.4-4) becomes 0. However, we can still show that the conditional density fx (x|Y=y)may exist.If y is very small, 84.44) can be written as x

Fx ( x | y y < Y y + y ) =
And,inthe limitasy 0

f x , y (1 , y )d1 2y f y ( y )2y

(4.49)

F (x | Y = y ) =
x

f x , y (1 , y )d f y (y)

(4.410)

For every ysuch that fy (y)0.After differentiation ofboth sides of(4.410)with respect to x: x

f x (x | Y = y ) =

f x , y ( x, y ) f y (y)

(4.411)

Figure 4.41 joint density function (a) and a conditional density function (b) applicable to Example 4.4-1.

When there isnoconfusion asto meaning,we shall often write (4.411)as

f x (x | y ) =
It canalso beshown that

f x . y ( x, y ) f y (y)
(4.412)

f y (y | x) =

f x. y ( x, y ) f x (x )
(4.413)

Example 4.42 We find f(y|x)for the density functions defined inExample 4.31.Since

f x. y ( x, y ) = u ( x )u ( y )xe x ( y +1)
and

f x ( x ) = u ( x )e x

Are nonzero only for 0<x,fy (y|x)isnonzero only for 0<yand 0<x.It is
f y ( y | x ) = u ( x )u ( y )xe xy

From (4.413). Conditional Distribution and Density Interval Conditioning It issometimes convenient to defineevent Bin(4.41)and (4.42)interms ofarandom variable Yby (4.414) B={ya <Y yb } where yand yare real numbers and we assume P(B)=P{ya <Y yb} 0.with this definition itisreadily shown that (4.41)and (4.42)become

Fx ( x | ya < Y yb ) =

Fx. y ( x, yb ) Fx. y ( x, ya ) Fy ( yb ) Fy ( ya )
x yb

=
ya

ya yb

f x , y ( , y )ddy f x. y ( x, y )dxdy

f x ( x | y a < Y yb

)=
yb ya

yb

ya

f x , y ( x, y )dy f x. y ( x, y )dxdy

These last two expressions hold for Xand Yeither continuous or discrete case,the joint density isgiven by (4.32).The resulting distribution and density will bedefined.However, only for ya and yb such that the deominators of(4.415)and (4.416)are nonzero .This requirement issatisfied so long asthe interval ya <y yb spans atleast one possible value ofYhaving anonzero probability ofoccurence. anexample will serve to illustrate the application of(4.416)when Xand Yare continuous random variables. Example 4.43We use (4.416;)to find fx (x|Yy)for the joint density function ofExample 4.31.Sincewe have here defined B={Y y},then ya = and yb =y. Furthermore, since fx.y (x,y) is nonzero only for 0 < x and 0 < y, we need only consider this region of x and y in finding the conditional density function. The denominator of (4.4-16) can be
written as f y ( )d . By using results from Example 4.31:
y

f y ( )d =

y u ( )d d y = = y>0 2 2 ( 0 ( y +1 + 1) + 1) y

Figure 4.42Conditional probability density functions applicable to Example 4.43.

and zero for y<0,after using anintegral from Appendix C.The numerator of(4.416) becomes y y x ( +1)

f x. y ( x, )d = u (x )xe
0

= u ( x )xe x e x d
y

= u ( x )e x 1 e xy

y>0

And zero for y<0,after using another integral from Appendix C.Thus

y +1 x xy f x ( x | Y y ) = u ( x)u ( y ) y e 1 e

This function isplotted inFigure 4.42for several values ofy. 4.5STATISTICALINDENPENDENCE It will berecalled from (1.53)that two events Aand Bare statistically indenpendent if (and only if) (4.5-1) P(A B) = P (A) P(B)

This condition canbeused to apply to two random variables Xand Yby defining the events A={X x}and B={Y y}for two real numbers xand y.Thus,Xand Y are said to bestatistically independent random variables if (and only if) P{X x,Y y}=P{X x}P{Y y} (4.52) From this expression and the definitions ofdistribution functions,it follws that Fx.y (x,y)=Fx (x)Fy (y) (4.53)

if Xand Yare independent.From the definitions ofdensity functions,(4.53)gives fx.y (x,y)=fx (x)fy (y) (4.54)

by differentiation,if Xand Yare dependent.Either (4.53)or (4.54)may serve as asufficent definition of,or testfor,independence oftwo random variables.

The formofthe conditional distribution function for independent events is found by use of(4.41)with B={Y y}:

Fx (x | Y y ) =

P{X x, Y y} Fx. y (x, y ) = P{y y} Fy ( y )

(4.55)

By substituting (4.53)into (4.55),we have Fx (x|Y y)=Fx (x) (4.56)

In other words,the conditional distribution ceases to beconditional and simply equals the arginal distribution for independent random variables.It canalso beshown that Fy (y|X x)=Fy (y) (4.57)

Conditional density function forms,for independent Xand Y,are found by differentiation of(4.56)and (4.57):

f x (x | Y y ) = f x (x )

(4.58) (4.59)

f y ( y | X x) = f y ( y)

Example 4.51For the densities ofExample 4.31:

f x. y (x, y ) = u (x )u ( y )xe x ( y +1) ex f x (x ) f y ( y ) = u ( x )u ( y ) f ( x, y ) ( y + 1)2 x. y

Therefore the random variables Xand Yare notindependent.

In the more generalstudy ofthe statistical independence ofNrandom variables X1 are said to bestatistically independent if (1.56)isstatisfied. It canbeshown that if X1 ,X2 ,...,XN are statistically independent then any group ofthese random variables isindependent ofany other group.Furthermore,afunction of any group isindependent ofany function ofany other group ofthe random variables .For example,with N=4random variables:X4 isindependent ofX3 +X2 +X1 ;X3 isindependent of X2 +X1 ,etc.(see Papoulis,1965,p.238). 4.6DISTRIBUTIONANDDENSITYOF ASUMOFRANDOMVARIABLES The problemoffinding the distribution and density functions for asum ofstatistically independent random variables isconsidered inthis section.

Sum ofTwo Random Variables Let Wbearandom variable equal to the sum oftwo independent random variables Xand Y: W=X+Y (4.61)

This isavery practical problembecause Xmight represent arandom signal voltage and Y could represent random noise atsome instant n time.The sum Wwould represent a signalplusnoise voltage available to some receiver. The probability distribution function we seek isdefined by F(w)=P{W w}=P{X+Y w} (4.62)

Figure 4.61illustrates the region inthe xy plane where x+y w.Now from (4.35f),the probability corresponding to anelemental area dx dy inthe xy plane located atthe point (x ,y)isfx.y (x,y)dx dy.If we sum all such probabilities over the region where x+y wwe will w y obtain Fw (w).Thus F w = f x, y dxdy
w

( ) x

x. y

(4.63) And,after using (4.54):

Fw (w) =

f y ( y ) w f x ( x )dxdy
x

w y

(4.64)

Figure 4.61Region inxy plane where x+y w.

By differentiating (4.64),using Leibnizs rule,we get the desired density function

f w (w) = f y ( y ) f x (w y )dy

(4.65)

This expression isrecognized asaconvolution integral.Consequentlyy,we have shown that the density function ofthe sum oftwo statistically independent random variables is the convolution oftheir individual density functions. Example 4.61We use (4.65)to find the density ofW=X+Ywhere the densities ofXand Yare assumed to be

1 [u (x ) u (x a )] a 1 f y ( y ) = [u ( y ) u ( y h )] a f x (x ) =

with 0<a<b,asshown inFigure 4.62aand b.Now because 0<Xand 0<Y,we only need examine the case W=X+Y>0.From (4.65)we write

f w (w ) =

1 [u ( y b )][u (w y ) u (w y a )]dy ab 1 = [1 u ( y b )][u (w y ) u (w y a )]dy ab 0

0 0

[u ( y b )]u (w y )dy + u u ( y b )u (w y a )dy

Figure 4.6Two density functions (a)and (b)and their convolution (c).

All these integrands are unity;the values ofthe integrals are determined by the unitstep functions through their control over limits ofintegration.After straightforward evaluation we get

w / ab 1 / b f w (w) = (a + b w)ab 0

0 w<a aw<b b w< a+b

Which issketched inFigure 4.62c. Sum ofSeveral Random Variables When the sum YofNindenpendent random variables X1 ,X2 ,...,XN isto beconsidered,we may extend the above analysis for two random variables.Let Y1 =X1 +X2 .Then we know from the preceding work that fy1 (y1 )=

fx2 (x2)*fx1 (x1)Next,we know that X3 will beindenpendent ofY1 =X1 +X2 because X3 is indenpendent ofboth X3 and Y1 to find the density function ofY2 =X3 +Y1 ,we get

f x2 = x1 + x2 + x3 ( y2 ) = f x 3 ( x3 )* f y1 = x1 + x2 ( y1 ) = f x 3 (x3 )* f x2 (x2 )* f x1 ( x1 )

(4.66)

By continuing the process we find that the density function ofY1 =X1 +X2 +...+XN isthe (N1)fold convolution ofthe Nindividual density function ofthe Nindividual density functions:

f y ( y ) = f xN ( x N ) * f x N 1 ( x N 1 ) * ... * f x1 ( x1 )

(4.67)

The distribution function ofYisfound from the integral offy (y)using (2.36c).

4.7CENTRALLIMITTHEOREM
Broadly defined,the central limittheorem says that the probability distribution function ofthe sum ofalarge number ofrandom variables approaches agaussian distribution.Although the theorem isknown to apply to some cases ofstatistically dependent random variables ,most applications,and the largest bodyofknowledge,are directed toward statistically independent random variables.Thus,inall succeeding discussions we assume statistically independent random variables.

Unequal Distributions
2 Let X 1 and x bethe means and variances,respectively,ofrandom variables X1 ,i= 1 1,2,...,N,which may have arbitary probability densities.The central limittheorem states that the sum YN =X1 +X2 +...+XN ,which hasmean Y N = X 1 + X 2 + ... + X N and variance 2 yN = x2 + x2 + ... + x2 ,hasaprobability distribution that asymptotically approaches
1 2 N

gaussian asN . Necessary conditions for the theorems validity are difficult to state, but sufficent conditions are known to be (Cramer,1946; Thomas, 1969)
2 x > B1 > 0

E | X 1 X 1 | < B2
3

i = 1,2,...,N i = 1,2,...,N

(4.7-1a) (4.7-1b)

Where B1 and B2 are positive numbers.These conditions guarantee that noone random variable inthe sum dominates. The reader should observe that the central limittheorem guaratees only that the distribution ofthe sum ofrandom variables becomes gaussian.It does notfollow that the probability density isalways gaussian.For continuous The asterisk denotes convolution

Random variables there isusually noproblem,butcertain conditions imposed onthe individual raandom variables (Cramer,1946:Papoulis,1965and 1984)will guarantee that the density isgaussian. For discrete random variables X1 the sum YN will contain impulses and is,therefore notgaussian even though the distribution apporoaches gaussian.When the possible discrete values ofeach random variable are kb,k=0,1,2,...,with baconstant,the envelope ofthe impulses inthe density ofthe sum will begaussian (with mean YN and variance 2YN ).This case isdiscussed insome detail by Papoulis (1965). The practical usefulness ofthe central limittheorem does notreside so much in the exactness ofthe gaussian distribution for N because the variance of YN becomes infinite from (4.7-1a). Usefulness derives more from the fact that YN for finite N may have a distribution that is closely approximated as gaussian. The approximation can be quite accurate , even for relatively small values of N, in the central region of the gaussian curve near the mean. However, the approximation can be very inaccurate in the tail regions away from the mean, even for large values of N (Davenport, 1970;Melsa and Sage, 1973). Of course, the approximation is made more accurate by increasing N.

Equal Distributions If all ofthe statistically independent random variables being summed are continuous and have the same distribution function,and therefore the same density,the proof ofthe central limittheorem isrelatively straighhtforward and isnext developed. Because the sum YN =X+X+...+Xhasaninfinite variance asN . we shall work with the zero-mean, unit-variance random variable
N N 2 WN = YN Y N / YN = X 1 X 1 / x1 i =1 i =1 N 1 = X1 X N x i =1

1/ 2

(4.7-2)

nstead. Here we define Xand 2x by

X1 = X

x2 = x2
1

(4.7-3)

Since all the X1 have the same distribution. (4.7-4) The theorems proof consists of showing that the characteristic function of W is that of a zero-mean, unit-variance gaussian random variable, which is

wN ( ) = exp 2 / 2

(4.7-5)

These are called lattice-type discrete random variables (Papoulis,1965).

From Problem328.If this roved the density ofWN must begaussian from (3.33)and the fact that Fourier transforms are unique.The characteristic function ofWN is

j N WN ( ) = E e JWN = E exp X 1 X N x 11 j = E exp X 1 X N x

(4.76)

The last stepin(4.76)follows from the independence and equal distributin ofthe X1 . Next,the exponential in(4.76)isexpanded inaTaylorpolynomial with aremainder term RN /N: j (X 1 X ) E N x
j = Ex 1 + N X1 X + x = 1 2 / 2 N + E [RN ] / N

j N

(X

X 2

RN N

(4.77)

Where E[RN ]approaches zero asN (Davenport, 1970, p. 442). On substitution of (4.7-7) into (4.7-6) and forming the natural logarithm, we have

In wN ( ) = N
Since

In 1 2 / 2 N + E [RN ] / N

{ (

(4.78)
(4.7-9)

z2 z3 In(1 z ) = z + + + ... 2 3

|z| < 1

We identify z with (2 /2N) E[RN ] /N and write (4.7-8) as

In wN ( ) = 2 / 2 N + E [RN ]
So
wN

] (
N

N 2

2 E [RN ] + ... N 2N
2
2 wN

(4.7-10)

lim {In[ ( )]} = Inlim ( ) =

N

/2

(4.7-11)

Finally, we have

lim ( ) = e
N wN

/ 2
2

(4.7-12) Which was to be shown. We illustrate the use of the central limit theorem through an example.

Example 4.71Consider the sum ofjust two independent uniformly distributed random variables X1 and X2 having the same density
f x (x ) = 1 [u ( x ) u ( x a )] a

Where a>0isaconstanat.The means and variances ofX1 and X2 are X = a / 2 and x2 = a 2 / 12 respectively.The density ofthe sum W=X1 +X2 isavailable from Example 4.61(with b=a ): w 1

f w (w) =

tri a a

Where the function tri (.)isdefined in(E4).The gaussian approximation to Whasvariance W = 2(a / 2 ) = a :

2w =2 2x =a2 /6and mean Approximation to

f w (w) =

2 2 e ( w a ) / (a / 3)

(a 2 / 3)

Figure 4.71illustrates fw (w)and its gaussian approximation.Even for the case ofonly two random variables being summed the gaussian approximation isafairly good one.For other denssities the approximation may bevery poor (see Problem463).

Figure 4.71The triangular density function ofExample 4.71and its gaussian approximation.