London School of Economics and Political Science

Marcia Schafgans

Department of Economics

Michaelmas 2002

EC221: Principles of Econometrics

Solutions to Problem Set 2
1. Distinguish between (a) linear dependence, (b) functional dependence, and (c) stochastic dependence between two random variables X and Y . What do (a), (b) and (c)
imply about the population correlation between X and Y ?
Answer: X and Y are
(a) lineary dependent if there exists constants and 6= 0 such that
Y = + X

(1)

The correlation coecient,

Cov (X, Y )
V ar (X) V ar (Y )

XY = p

satises XY = 1. To see this, we rst calculate

Cov (X, Y ) = E (X E (X)) (Y E (Y ))
= E (X E (X)) ( + X ( + E (X)))
= E (X E (X)) (X E (X))
= E (X E (X)) (X E (X))
= V ar (X)
and
V ar (Y ) = E (Y E (Y ))2
= E ( + X ( + E (X)))2
= E (X E (X))2
= 2E (X E (X))2
= 2V ar (X)

(2)

where we have used that E (Y ) = + E (X). Combining these two,

V ar (X)
xy = p
V ar (X) 2 V ar (X)

=
jj
(
1 , > 0
=
.
1 , < 0
(b) functionally dependent if there exists a function g such that either
Y = g (X)

or X = g (Y )

(3)

The correlation coeecient XY can be anything (note: jXY j
1 always).

(c) stochastically/statistically independent if for any possible sets A and B,
P (X 2 A, Y 2 B) = P (X 2 A) P (Y 2 B) .

(4)

Otherwise, we say that X and Y are stochastically dependent. If (X, Y ) has a

joint density function, fX,Y (x, y), then (4) is equivalent to
fX,Y (x, y) = fX (x) fY (y)

(5)

where fX (x), fY (y) are the marginal density functions of X and Y respectively.
Stochastic independence implies that the correlation coecient equals 0. We
only give a proof for the case where (X, Y ) has a density function. Since
Z Z
E (XY ) =
xyfX,Y (x, y) dxdy
Z Z
=
xyfX (x) fY (y) dxdy
Z
Z
= xfX (x) dx yfY (y) dy
= E (X) E (Y ) ,
we have that
Cov (X, Y ) = E (XY ) E (X) E (Y )
= E (X) E (Y ) E (X) E (Y )
= 0.
Hence, XY = 0.
2

Note: The denition in (a) is not quite satisfactory. Suppose jXY j < 1; we would not
call this lineary independence which is more appropriate if XY = 0. An analogous
shortcoming can be said for the denition (b).
2. If the random variables X and Y have a bivariate normal distribution, show that
Cov(X, Y ) = 0 implies that X and Y are statistically independent.
Answer: We have that (X, Y ) N (, ) where = (X , Y ) is the mean and
!

2
0
X
=
0
Y2

is the covariance matrix. We here have used that Cov(X, Y ) = 0. We now prove
that (5) holds in our case. First, we note that the determinant of , jj, satises
2 2 since is diagonal. Thereby,
jj = X
Y

1
1
1
exp (x X , y Y ) (x X , y Y )
fX,Y (x, y) = p
2
2 jj
!
!#
"

x X
X2 0
1
1
=
exp (x X , y Y )
2X Y
2
0
Y 2
y Y

1
1
1
=
exp 2 (x X )2 2 (y Y )2
2X Y
2X
2Y

1
1
1
1
2
2
exp 2 (x X ) p
exp
(y Y )
=p
2X
2Y2
2X
2Y

= fX (x) fY (y) .
This proves that Cov(X, Y ) = 0 ) X and Y are statistically independent. This result
together with the result from Question 1, (c) gives us the following useful result:
If X and Y have a bivariate normal distribution,
Cov(X, Y ) = 0 , X and Y are statistically independent.
3. The mean square error (M.S.E.) of an estimator b of a scalar parameter , is dened
b = E(b )2. Show that:
as: MSE()

2
b
b
b
MSE() = V ar() + bias() .

Answer:
b = E(b )2
MSE()

2
= E b E b E b

2
2

= E b E b + E b 2E b E b E b

2
b + E b 2 E b E b E b
= V ar()

2
b + E b
= V ar()

2
b
b
= V ar() + bias()
4. Consider drawing independently two random samples from a population distributed
as N (, 2). The rst sample has n1 observations, and its sample mean is given by
P 1
X1 = n11 ni=1
X1i ; the second sample has n2 observations, and its sample mean is
P 2
given by X2 = n12 ni=1
X2i . Two estimators of are proposed:

b1 = 12 X1 + X2

b2 = n1 X1 + n2 X 2 / (n1 + n2) .
Compare the properties of these estimators (unbiasedness, eciency, consistency).
Answer: First observe that
n1
n1
1 X
1 X
n1
EX1 =
EX1i =
= = ,
n1 i=1
n1 i=1
n1

and

n
!
n1
1
X

1
1 X
X1i = 2
V ar (X1i )
V ar X1 = 2 V ar
n1
n
1
i=1
i=1
n1
2
X
1
n1

= 2
2 = 2 2 = ,
n1 i=1
n1
n1

where, in the second equality of the variance we have used the fact that we deal with
a random sample. Similarly, we get that
2
V ar X2 = .
n2

EX2 = ,
Thus,
Eb
1 =

1
2

EX1 + EX2 = 12 ( + ) =
4

E
b2 = n1EX1 + n2 EX2 / (n1 + n2) = (n1 + n2) / (n1 + n2) = ,
therefore we conclude that both estimators are unbiased.
Regarding eciency,

1
1
V ar X1 + V ar X2 =
V ar (b
1 ) =
4
4

V ar (b
2 ) =

n21 V arX1

n22EX2

2 2
+
n1 n2


/ (n1 + n2 ) =

n21

2
=
4

n1

n22

n2

1
1
+
n1 n2

/ (n1 + n2 )2

= 2 (n1 + n2 ) / (n1 + n2) = 2/ (n1 + n2 ) .

where we use the fact that both samples are independent. To compare the eciencies
of the two estimators, we look at V ar (b
1) V ar (b
2),



1
1 1
1
2
2) =
+

V ar (b
1 ) V ar (b
4 n1 n2
n1 + n2
n2 (n1 + n2 ) + n1 (n1 + n2 ) 4n1 n2
= 2
4n1n2 (n1 + n2 )
2
2
n + n1 2n1 n2
= 2 2
4n1n2 (n1 + n2 )
(n1 n2 )2
=
0
4n1n2 (n1 + n2)
2

b1 is less ecient than

b2 since
with equality to zero if and only if n1 = n2. Hence,
its variance is larger. This is due to the fact that
b1 does not take into consideration
that the two samples may be of dierent size. If n1 > n2, X1 is a better estimator of
than X2 since X1 is based on more observations (c.f. the variances of X1 vs. X2);
we should therefore give more weight to X1 in our aggregate estimate of . This is
exactly what
b2 does.
As both estimators are unbiased, it is clear that both are consistent for provided
their variances tend to zero. This happens for
b1 provided n1 ! 1 and n2 ! 1,
whereas for
b2 it is just required that n1 + n2 ! 1.
5. If u is an n-dimensional vector of random variables distributed as N (0, 2 In ), and
A is a symmetric, idempotent matrix of order n and rank p, show that u Au/ 2 is
distributed as 2 with p degrees of freedom. (Hint: diagonalize the matrix A.)

Answer: Since A is a symmetric, idempotent matrix, we may rewrite A as

A = SS

where = diag (1, ..., n ) contains the eigenvalues of A which are either 0 or 1, and
S is a matrix whose columns are the corresponding eigenvectors (of length 1) of A

and S S = I, c.f. solutions to Problem set 1, Question 7. Since A has rank p, we

may choose such that the rst p eigenvalues equal 1 while the last n p eigenvalues
equal 0. We now observe that
u Au u
u
= v v
=
SS
2

where v S u/ is a linear transformation of the normal distributed u. Hence, v is

normal distributed as well. We nd its mean and variance,

Ev =


S Eu
S0
=
=0

1
1 2
2
S
V
ar
(u)
S
=
S
I
S
=
S S = I.

2
2
2
P
P
In total, v N (0, In ). Therefore, v v = ni=1 i vi2 = pi=1 vi2 2p , as v v equals
the sum of the squares of p independent N (0, 1) random variables.
V ar (v) = V ar

Su