You are on page 1of 3

The Rules of Summation

å
n
i=1
x
i
= x
1
÷x
2
÷ ÷x
n
å
n
i=1
a = na
å
n
i=1
ax
i
= a å
n
i=1
x
i
å
n
i=1
(x
i
÷y
i
) = å
n
i=1
x
i
÷ å
n
i=1
y
i
å
n
i=1
(ax
i
÷by
i
) = a å
n
i=1
x
i
÷b å
n
i=1
y
i
å
n
i=1
(a ÷bx
i
) = na ÷b å
n
i=1
x
i
x =
å
n
i=1
x
i
n
=
x
1
÷x
2
÷ ÷x
n
n
å
n
i=1
(x
i
÷x) = 0
å
2
i=1
å
3
j=1
f (x
i
; y
j
) = å
2
i=1
f (x
i
; y
1
) ÷f (x
i
; y
2
) ÷f (x
i
; y
3
) [ [
= f (x
1
; y
1
) ÷f (x
1
; y
2
) ÷f (x
1
; y
3
)
÷f (x
2
; y
1
) ÷f (x
2
; y
2
) ÷f (x
2
; y
3
)
Expected Values & Variances
E(X) = x
1
f (x
1
) ÷x
2
f (x
2
) ÷ ÷x
n
f (x
n
)
= å
n
i=1
x
i
f (x
i
) = å
x
x f (x)
E g(X) [ [ = å
x
g(x) f (x)
E g
1
(X) ÷g
2
(X) [ [ = å
x
g
1
(x) ÷g
2
(x) [ [ f (x)
= å
x
g
1
(x) f (x) ֌
x
g
2
(x) f (x)
= E g
1
(X) [ [ ÷E g
2
(X) [ [
E(c) = c
E(cX) = cE(X)
E(a ÷ cX) = a ÷ cE(X)
var(X) = s
2
= E[X ÷ E(X)]
2
= E(X
2
) ÷ [E(X)]
2
var(a ÷ cX) = E[(a ÷ cX) ÷E(a ÷ cX)]
2
= c
2
var(X)
Marginal and Conditional Distributions
f (x) = å
y
f (x; y) for each value X can take
f (y) = å
x
f (x; y) for each value Y can take
f (x[y) = P X = x[Y = y [ [ =
f (x; y)
f (y)
If X and Y are independent random variables, then
f (x,y) = f (x)f ( y) for each and every pair of values
x and y. The converse is also true.
If X and Y are independent random variables, then the
conditional probability density function of X given that
Y = y is f (x[y) =
f (x; y)
f (y)
=
f (x) f (y)
f (y)
= f (x)
for each and everypair of values x and y. The converse is
also true.
Expectations, Variances & Covariances
cov(X; Y) = E[(X÷E[X[)(Y÷E[Y[)[

x
å
y
x ÷E(X) [ [ y ÷E(Y) [ [ f (x; y)
r =
cov(X;Y)
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
var(X)var(Y)
_
E(c
1
X ÷ c
2
Y) = c
1
E(X) ÷ c
2
E(Y)
E(X ÷ Y) = E(X) ÷ E(Y)
var(aX÷bY÷cZ) =a
2
var(X) ÷b
2
var(Y) ÷c
2
var(Z)
÷2abcov(X,Y) ÷ 2accov(X,Z) ÷ 2bccov(Y,Z)
If X, Y, and Z are independent, or uncorrelated, random
variables, then the covariance terms are zero and:
var(aX ÷bY ÷cZ) = a
2
var(X)
÷b
2
var(Y) ÷c
2
var(Z)
Normal Probabilities
If X - N(m, s
2
), then Z =
X ÷m
s
-N(0; 1)
If X - N(m, s
2
) and a is a constant, then
P(X _ a) = P Z _
a ÷m
s
_ _
If X -N(m; s
2
) and a and b are constants; then
P(a _ X _ b) = P
a÷m
s
_ Z _
b ÷m
s
_ _
Assumptions of the Simple Linear Regression
Model
SR1 The value of y, for each value of x, is y = b
1
÷
b
2
x ÷ e
SR2 The average value of the random error e is
E(e) = 0 since we assume that E(y) = b
1
÷ b
2
x
SR3 The variance of the random error e is var(e) =
s
2
= var(y)
SR4 The covariance between any pair of random
errors, e
i
and e
j
is cov(e
i
, e
j
) = cov(y
i
, y
j
) = 0
SR5 The variable x is not random and must take at
least two different values.
SR6 (optional ) The values of e are normally dis-
tributed about their mean e - N(0, s
2
)
Least Squares Estimation
If b
1
and b
2
are the least squares estimates, then
^y
i
= b
1
÷b
2
x
i
^e
i
= y
i
÷^y
i
= y
i
÷b
1
÷b
2
x
i
The Normal Equations
Nb
1
÷Sx
i
b
2
=Sy
i
Sx
i
b
1
÷Sx
2
i
b
2
= Sx
i
y
i
Least Squares Estimators
b
2
=
S(x
i
÷x)(y
i
÷y)
S(x
i
÷x)
2
b
1
= y ÷b
2
x
Elasticity
h =
percentage change in y
percentage change in x
=
Dy=y
Dx=x
=
Dy
Dx

x
y
h =
DE(y)=E(y)
Dx=x
=
DE(y)
Dx

x
E(y)
= b
2

x
E(y)
Least Squares Expressions Useful for Theory
b
2
= b
2
÷Sw
i
e
i
w
i
=
x
i
÷x
S(x
i
÷x)
2
Sw
i
= 0; Sw
i
x
i
= 1; Sw
2
i
= 1=S(x
i
÷x)
2
Properties of the Least Squares Estimators
var(b
1
) = s
2
Sx
2
i
NS(x
i
÷x)
2
_ _
var(b
2
) =
s
2
S(x
i
÷x)
2
cov(b
1
; b
2
) = s
2
÷x
S(x
i
÷x)
2
_ _
Gauss-Markov Theorem: Under the assumptions
SR1–SR5 of the linear regression model the estimators
b
1
and b
2
have the smallest variance of all linear and
unbiased estimators of b
1
and b
2
. They are the Best
Linear Unbiased Estimators (BLUE) of b
1
and b
2
.
If we make the normality assumption, assumption
SR6, about the error term, then the least squares esti-
mators are normally distributed.
b
1
- N b
1
;
s
2
åx
2
i
NS(x
i
÷x)
2
_ _
; b
2
- N b
2
;
s
2
S(x
i
÷x)
2
_ _
Estimated Error Variance
^ s
2
=
S^e
2
i
N ÷2
Estimator Standard Errors
se(b
1
) =
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
´
var(b
1
)
_
; se(b
2
) =
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
´
var(b
2
)
_
t-distribution
If assumptions SR1–SR6 of the simple linear regression
model hold, then
t =
b
k
÷b
k
se(b
k
)
- t
(N÷2)
; k = 1; 2
Interval Estimates
P[b
2
÷ t
c
se(b
2
) _ b
2
_ b
2
÷ t
c
se(b
2
)] = 1 ÷ a
Hypothesis Testing
Components of Hypothesis Tests
1. A null hypothesis, H
0
2. An alternative hypothesis, H
1
3. A test statistic
4. A rejection region
5. A conclusion
If the null hypothesis H
0
: b
2
= c is true, then
t =
b
2
÷c
se(b
2
)
- t
(N÷2)
Rejection rule for a two-tail test: If the value of the
test statistic falls in the rejection region, either tail of
the t-distribution, then we reject the null hypothesis
and accept the alternative.
Type I error: The null hypothesis is true and we decide
to reject it.
Type II error: The null hypothesis is false and we decide
not to reject it.
p-value rejection rule: When the p-value of a hypoth-
esis test is smaller than the chosen value of a, then the
test procedure leads to rejection of the null hypothesis.
Prediction
y
0
= b
1
÷b
2
x
0
÷e
0
; ^y
0
= b
1
÷b
2
x
0
; f = ^y
0
÷y
0
´
var( f ) = ^ s
2
1 ÷
1
N
÷
(x
0
÷x)
2
S(x
i
÷x)
2
_ _
; se( f ) =
ffiffiffiffiffiffiffiffiffiffiffiffiffi
´
var( f )
_
A (1 ÷ a) × 100% confidence interval, or prediction
interval, for y
0
^y
0
±t
c
se( f )
Goodness of Fit
S(y
i
÷y)
2
= S(^y
i
÷y)
2
÷S^e
2
i
SST = SSR ÷SSE
R
2
=
SSR
SST
= 1 ÷
SSE
SST
= (corr(y; ^y))
2
Log-Linear Model
ln(y) = b
1
÷b
2
x ÷e;
´
ln( y) = b
1
÷b
2
x
100 ×b
2
~ % change in y given a one-unit change in x:
^y
n
= exp(b
1
÷b
2
x)
^y
c
= exp(b
1
÷b
2
x)exp(^ s
2
=2)
Prediction interval:
exp
´
ln(y) ÷t
c
se( f )
_ _
; exp
´
ln( y) ÷t
c
se( f )
_ _
Generalized goodness-of-fit measure R
2
g
=(corr(y;^y
n
))
2
Assumptions of the Multiple Regression Model
MR1 y
i
= b
1
÷ b
2
x
i2
÷ ÷ b
K
x
iK
÷ e
i
MR2 E(y
i
) =b
1
÷b
2
x
i2
÷ ÷b
K
x
iK
=E(e
i
) = 0.
MR3 var(y
i
) = var(e
i
) = s
2
MR4 cov(y
i
, y
j
) = cov(e
i
, e
j
) = 0
MR5 The values of x
ik
are not random and are not
exact linear functions of the other explanatory
variables.
MR6 y
i
- N[(b
1
÷b
2
x
i2
÷ ÷b
K
x
iK
); s
2
[
=e
i
- N(0; s
2
)
Least Squares Estimates in MR Model
Least squares estimates b
1
, b
2
, . . . , b
K
minimize
S(b
1
, b
2
, . . . , b
K
) = å(y
i
÷b
1
÷b
2
x
i2
÷ ÷b
K
x
iK
)
2
Estimated Error Variance and Estimator
Standard Errors
^ s
2
=
å^e
2
i
N ÷K
se(b
k
) =
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
´
var(b
k
)
_
Hypothesis Tests and Interval Estimates for Single Parameters
Use t-distribution t =
b
k
÷b
k
se(b
k
)
- t
(N÷K)
t-test for More than One Parameter
H
0
: b
2
÷cb
3
= a
When H
0
is true t =
b
2
÷cb
3
÷a
se(b
2
÷cb
3
)
- t
(N÷K)
se(b
2
÷cb
3
) =
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
´
var(b
2
) ÷c
2 ´
var(b
3
) ÷2c ×
´
cov(b
2
; b
3
)
_
Joint F-tests
To test J joint hypotheses,
F =
(SSE
R
÷SSE
U
)=J
SSE
U
=(N ÷K)
To test the overall significance of the model the null and alternative
hypotheses and F statistic are
H
0
: b
2
= 0; b
3
= 0; : : : ; b
K
= 0
H
1
: at least one of the b
k
is nonzero
F =
(SST ÷SSE)=(K ÷1)
SSE=(N ÷K)
RESET: A Specification Test
y
i
=b
1
÷b
2
x
i2
÷b
3
x
i3
÷e
i
^y
i
=b
1
÷b
2
x
i2
÷b
3
x
i3
y
i
=b
1
÷b
2
x
i2
÷b
3
x
i3
÷g
1
^y
2
i
÷e
i
; H
0
: g
1
=0
y
i
=b
1
÷b
2
x
i2
÷b
3
x
i3
÷g
1
^y
2
i
÷g
2
^y
3
i
÷e
i
; H
0
: g
1
=g
2
=0
Model Selection
AIC = ln(SSE=N) ÷ 2K=N
SC = ln(SSE=N) ÷ Kln(N)=N
Collinearity and Omitted Variables
y
i
= b
1
÷b
2
x
i2
÷b
3
x
i3
÷e
i
var(b
2
) =
s
2
(1 ÷r
2
23
) å(x
i2
÷x
2
)
2
When x
3
is omitted; bias(b
+
2
) = E(b
+
2
) ÷b
2
= b
3
´
cov(x
2
; x
3
)
´
var(x
2
)
Heteroskedasticity
var(y
i
) = var(e
i
) = s
i
2
General variance function
s
2
i
= exp(a
1
÷a
2
z
i2
÷ ÷a
S
z
iS
)
Breusch-Pagan and White Tests for H
0
: a
2
= a
3
= = a
S
= 0
When H
0
is true x
2
= N ×R
2
- x
2
(S÷1)
Goldfeld-Quandt test for H
0
: s
2
M
= s
2
R
versus H
1
: s
2
M
,= s
2
R
When H
0
is true F = ^ s
2
M
=^ s
2
R
- F
(NM÷KM;NR÷KR)
Transformed model for var(e
i
) = s
2
i
= s
2
x
i
y
i
=
ffiffiffiffi
x
i
_
= b
1
1=
ffiffiffiffi
x
i
_
( ) ÷b
2
x
i
=
ffiffiffiffi
x
i
_
( ) ÷e
i
=
ffiffiffiffi
x
i
_
Estimating the variance function
ln(^e
2
i
) = ln(s
2
i
) ÷v
i
= a
1
÷a
2
z
i2
÷ ÷a
S
z
iS
÷v
i
Grouped data
var(e
i
) = s
2
i
=
s
2
M
i = 1; 2; . . . ; N
M
s
2
R
i = 1; 2; . . . ; N
R
_
Transformed model for feasible generalized least squares
y
i
_
ffiffiffiffiffi
^ s
i
_
= b
1
1
_
ffiffiffiffiffi
^ s
i
_
_ _
÷b
2
x
i
_
ffiffiffiffiffi
^ s
i
_
_ _
÷e
i
_
ffiffiffiffiffi
^ s
i
_
Regression with Stationary Time Series Variables
Finite distributed lag model
y
t
=a ÷b
0
x
t
÷b
1
x
t÷1
÷b
2
x
t÷2
÷ ÷b
q
x
t÷q
÷v
t
Correlogram
r
k
= å(y
t
÷y)(y
t÷k
÷y)= å(y
t
÷y)
2
For H
0
: r
k
= 0; z =
ffiffiffiffi
T
_
r
k
- N(0; 1)
LM test
y
t
=b
1
÷b
2
x
t
÷r^e
t÷1
÷^v
t
Test H
0
: r =0 with t-test
^e
t
=g
1
÷g
2
x
t
÷r^e
t÷1
÷^v
t
Test using LM=T ×R
2
AR(1) error y
t
=b
1
÷b
2
x
t
÷e
t
e
t
= re
t÷1
÷v
t
Nonlinear least squares estimation
y
t
= b
1
(1 ÷r) ÷b
2
x
t
÷ry
t÷1
÷b
2
rx
t÷1
÷v
t
ARDL(p, q) model
y
t
= d ÷d
0
x
t
÷d
l
x
t÷1
÷ ÷ d
q
x
t÷q
÷u
l
y
t÷1
÷ ÷u
p
y
t÷p
÷v
t
AR(p) forecasting model
y
t
= d ÷u
l
y
t÷1
÷u
2
y
t÷2
÷ ÷u
p
y
t÷p
÷v
t
Exponential smoothing ^y
t
= ay
t÷1
÷(1 ÷a)^y
t÷1
Multiplier analysis
d
0
÷d
1
L ÷d
2
L
2
÷ ÷d
q
L
q
= (1 ÷u
1
L ÷u
2
L
2
÷ ÷u
p
L
p
)
×(b
0
÷b
1
L ÷b
2
L
2
÷ )
Unit Roots and Cointegration
Unit Root Test for Stationarity: Null hypothesis:
H
0
: g = 0
Dickey-Fuller Test 1 (no constant and no trend):
Dy
t
= gy
t÷1
÷v
t
Dickey-Fuller Test 2 (with constant but no trend):
Dy
t
= a ÷gy
t÷1
÷v
t
Dickey-Fuller Test 3 (with constant and with trend):
Dy
t
= a ÷gy
t÷1
÷lt ÷v
t
Augmented Dickey-Fuller Tests:
Dy
t
= a ÷gy
t÷1
÷ å
m
s=1
a
s
Dy
t÷s
÷v
t
Test for cointegration
D^e
t
= g^e
t÷1
÷v
t
Random walk: y
t
= y
t÷1
÷v
t
Random walk with drift: y
t
= a ÷y
t÷1
÷v
t
Random walk model with drift and time trend:
y
t
= a ÷dt ÷y
t÷1
÷v
t
Panel Data
Pooled least squares regression
y
it
= b
1
÷b
2
x
2it
÷b
3
x
3it
÷e
it
Cluster robust standard errors cov(e
it
, e
is
) = c
ts
Fixed effects model
y
it
= b
1i
÷b
2
x
2it
÷b
3
x
3it
÷e
it
b
1i
not random
y
it
÷y
i
= b
2
(x
2it
÷x
2i
) ÷b
3
(x
3it
÷x
3i
) ÷(e
it
÷e
i
)
Random effects model
y
it
=b
1i
÷b
2
x
2it
÷b
3
x
3it
÷e
it
b
it
=b
1
÷u
i
random
y
it
÷ay
i
=b
1
(1÷a) ÷b
2
(x
2it
÷ax
2i
) ÷b
3
(x
3it
÷ax
3i
) ÷v
+
it
a=1÷s
e
_
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
Ts
2
u
÷s
2
e
_
Hausman test
t = (b
FE;k
÷b
RE;k
)
_
´
var(b
FE;k
) ÷
´
var(b
RE;k
)
_ _
1=2