Maximum Likelihood Estimation

Maximum Likelihood estimation produces the same estimates as OLS, in a large

sample, so in general the results can be considered in the same way as those produced
by OLS. For example suppose there is a random variable with an unknown mean and
variance, which follows the normal distribution, where the probability density
function of the normal distribution is
!
" # ! $ %
!
%
" #

=
x
e x f
&here ' is the unknown mean of the random variable x and ( is its standard
deviation. )his can then be used to produce the usual normal distribution with a peak
at ', this can then be used to determine the probability density for different values of
x, given its mean value. *iven a particular value for the mean, by calculating the
probability density for specific values such as x + , and x + -, it is then possible to
calculate the likelihood function, which is simply the .oint probability density for the
two observations. /y varying the value for ', it is possible to obtain a number of
likelihood functions corresponding to these values of '. )hese can then be plotted, we
will chose ' to maximise this function. )his can also be generalised to more than !
observations, although for simplicity we tend to use the log likelihood or log of the
likelihood function.
Simple Regression Model
*iven the following simple model, assuming the error term is normally distributed
with 0ero mean and standard deviation of (, we can produce
!
" # ! $ %
!
%
" #




u
t t t
t t t
e u f
x y u
u x y

=
=
+ + =
)he probability density can then be determined by substituting in the u
t
expression.
)he log1likelihood function will then be given by
2 " # ... " 3#
!
%
"
!
%
log# log
! !
% %
!
n n
x y x y n L

+ + =
)hen the values of 4 and 5 that maximise this function are the same as those obtained
by OLS. )he estimate of ( is slightly different.
You do not need to be able to derive the above, this is simply background
information for the estimation of the AR! models"
Autoregressive onditional !eteroskedasticity #AR!$
)he 6789 effect is concerned with a relationship within the heteroskedasticity, often
termed serial correlation of the heteroskedasticity. :t often becomes apparent when
there is bunching in the variance or volatility of a particular variable, producing a
pattern which is determined by some factor. *iven that the volatility of financial
assets is used to represent their risk, it can be argued that the 6789 effect is
measuring the risk of an asset. *iven the following model
" , ; # <
!
% % ;
% ;

+
+ + =
t t
t t t
u N u
u X Y


)his suggests the error term is normally distributed with 0ero mean and conditional
variance depending on the s=uared error term lagged one time period. )he conditional
variance is the variance given the values of the error term lagged once, twice etc
" , > # ...." , > var#
! %
!
! %
!

= =
t t t t t t t
u u u E u u u
&here
!
t
is the conditional variance of the error term. )he 6789 effect is then
modelled by
!
% % ;
!

+ =
t t
u
)his is an 6789#%" model as it contains only a single lag on the s=uared error term,
however it is possible to extend this to any number of lags, if there are = lags it is
termed an 6789#=" model.
%esting for AR! Effects
)he test for an 6789 effect was devised originally by ?ngle #%@A!" and is similar to
the Lagrange Multiplier #LM" test for autocorrelation.
%" 7un the regression of the model using Ordinary Least S=uares #OLS" and
collect the residuals. S=uare the residuals.
!" 7un the following secondary regression
t p t t t t
v u u u u + + + + =

!
B
!
! !
!
% % ;
!
.....
&here u is the residual from the initial regression and p lags are included in
this secondary regression. )he appropriate number of lags can either be
determined by the span of the data #i.e. , for =uarterly data" or by an
information criteria. 8ollect the
!
R
statistic from this regression.
B" 8ompute the statistic )C
!
R
, where ) is the number of observations. :t follows
a chi1s=uared distribution with p degrees of freedom. )he null hypothesis is
that there is no 6789 effect present.