Simple Regression

Y Dependent variable

X Independent Variable (explicativas)

Estimate Y given X E (Y|X) = 1 + 2X

1 Intercept or constant

2 Slope, Coeficiente angular, efeito marginal de X1

Linear regression model: = 1 + 2 +

Y = ^Y + the size of the residuals defines how fit your model is!

Why the error term exists Outliers, complex reality (hard to measure things, such as a sunny
day affecting financial markets), using linear model when the phenomena are not linear,
missing/exceeding variables (bias).

O erro capta a variabilidade que no medida pelas variveis independentes.

Ui = Yi - ^Yi

Ui= Yi (^B1 ^B2Xi)

Determining regression coefficients:

OLS (MQO): Ordinary least squares

Minimize the sum of the square of residuals square the residuals so the
negative/positive dont neutralize each other (best solution for Carvalhal).
O critrio de ajustamento do MQO minimizar a soma dos quadrados dos resduos:
, para todo i

Propriedades do MQO:

- Linear nos parmetros (BETAS). Variveis no precisam ser lineares!

- consistente: Valor esperado do termo de erro = 0
- no viesado,
- eficiente (tem a menor varincia dentre os estimadores lineares no tendenciosos Gauss
Markow)

Gauss Markow: OLS estimators are BLUE: best linear unbiased estimator
Other model: Minimize the sum of the absolute value of residuals (less used than MQO).

Assumptions of the Classic Linear Regression Model (CLRM)

1. Linear parameters
2. E(ui) = 0
3. Homoskedacity of the error term para todo u, mesma variance
4. No autocorrelation between error terms Cov (ui, uj) = 0
5. Zero cov between X and u no relationship between independent variables and error
terms

All 5 assumptions OLS is BLUE! = minimum variance among all unbiased estimators of betas
that are linear functions of Ys

6. Ui is normally distributed, media zero e varincia sigma2 Needed only for inference
With assumption 6: CNLRM; BUE No model better than yours, linear or not.

Best B1 and B2 present minimum st errors

Linear Linear
Unbiased The expected value of sample Beta is the population Beta
Estimator Estimator

Hypothesis testing

Confidence interval:
2 st devs: explain more or less 95% of observations. Out of interval reject

The null hypothesis is rejected if the p-value is less than the significance level ().

The level is the probability of rejecting the null hypothesis given that it is true (type I
error) and is most often set at 0.05 (5%)

p-value:
compare it to the level of confidence
o if p-value =< confidence level reject H0
o p-valores altos so compatveis com H0 p-valor > , aceita H0
o obs: diferente, ento se = 5%, 2,5% pra cada lado. ;
o se < ou >,unicaudal

Compare the coefficient to the standard error! = stat t

Stat t:
t = 1/Stdev. Shows how many times youre above/below the standard error.
 Goodness of Fit

Total sum of squares = explained SS + residual SS

R2: Coeficiente de explicao

measures how close the data are to the fitted regression line.

= ESS/TSS what my model can explain/total to be explained

R2 = 1 RSS/TSS maximizing R2 means minimizing residuals (RSS) OLS!

If R2 = 0, it means your model is equal to the average. Models add value when they can
beat the average

R2 Adjusted:
Considers the increase in the number of regressors. Widely used in multiple regression.

o Adding variables that are irrelevant to the model (p-value compatible with H0)
tends to increase R2

Critrio para o melhor modelo em RegMult o R2 ajustado usar o mais alto!!

S pq o p-valor alto, isso no quer dizer que a varivel irrelevante para o modelo.

Eviews: Forecast Use max r2adjusted

Estimate determinants (of returns for example) Use significant variables!

*** In finance, R2 >= 50% is a lot! In statistics, 70-80%.

*** Parametric test: Considers a specific kind of distribution (ex, normal).

Non-parametric: ignores the type of distribution.

Teste para normalidade: Jarque-Bera

Homocedasticidade: quando o valor esperado do termo de erro = 0

E (ui) = 0 (homoskedacity)

- Skewness of the normal distribution = zero: related to position, Like a mirror

Skewed to the right or left not normal, not centered around zero and not equally
distributed to the left and right.
Teste: 0.28 =/= 0
 H0: Normality
H1: non-normality

Usando p-value: p-valores altos so compatveis com H0. Como p = 0.009, compatvel
com H1, no normalidade!

- Kurtosis: related to the size of the tail.

Normal distribution: Kurtosis = 3
If the tail is fat, too many outliers! /thin, not normal!
Teste: 3.8 =/= 3

O essencial olhar o p-value do teste de Jarque-Bera e comparar com o alfa escolhido.

Regra geral: P-valor baixo, Rejeita H0. (ou H1 V) distribuio no normal!

No caso de agora, alfa = 0,05 e pvalue = 0,03 not normal. Com um alfa de 0,025, aceitaria a
hiptese de normalidade.

No caso da vale, com uma amostra grande (contendo a crise) non-normality. Quando
diminumos a amostra, contendo apenas o perodo de 2003-2007, chegamos a um p-valor de
83%, compatvel com H0 (normalidade!)

- Normality (ui)
- Colinearity of the independent variables
- Var (ui) = CTE

Multiple Regression
Assumption 7 of the Classical Linear Regression Model: No perfect colinearity between the
independent variables. There cant be any exact linear relationship between 2 Xs. Nonlinear,
no problem!

Quando uma varivel independente uma combinao linear de outra, isso um problema !!
No d pra rodar o modelo pois no possvel separar os efeitos dos betas.

Leave the nonsignificant variables: even if a variable is statistically =0, eliminating it can
generate other problems (specially the intercept!!). It could be significant in another sample,
for ex.

Joint Hypothesis Testing

Sometimes we test more than one parameter at the same time! Each separate test does not
take into account the effect that a Beta might present on the other. The results are different!
Ex:

- H0: B2 = B3 = 0
- H1: not H0. If any of the parameters is different than zero, we reject the null hypothesis.
There is only one outcome for H1, but it is split into 3 different combinations (both
betas can be different than zero, or just one of them)!

Restricted x Unrestricted models

The restricted model is the one that considers that the null hypothesis is valid, with B2 and B3
equal to zero. The unrestricted model takes into account all the betas, regardless of their
statistical value.

Compare the R2 of both models and stay with the highest, which will present the greater
predictive power!

The problem lies in the comparison between the Rs. Ex: 0,691 x 0,689 Do a statistical test,
dont trust your eyes. Dont use t-statistics (good for means comparison): use F tests!!!

Use F to test joint hypothesis !!!

Most powerful way to decide for the best model: compare residual sum of squares (RSS)
instead of R2.

Most papers use RSS!!! If you maximize R2, you are automatically minimizing the residuals.
However, RSS is the most common notation.

- RSSu < RSSr

- R2u > R2r

RSS always increases when you drop variables from the model (restrict). If you drop an
important variable, it will increase a lot Reject H0 (b2=b3=0)

Coefficient restriction test: (Wald?)

1) Estimate unrestricted model and its RSS

2) Same for restricted model
3) Compute the F test

(excel)

Obs: t tests are individual. They test 1 hypothesis at a time. The F test is a family formed by all
t tests, but you can also use F for individual hypothesis t2 = F !!! The p-value is the same.
Ramsays RESET test: Fitted

Test to see if you need exponential variables in the model

H0: fitted terms = 0
H1: not H0, meaning you need an exponential term.

Test again to know the power

Test for Structural Stability

Test to check if the relationship between Y and Xs is the same for all the periods.

- Restricted model: a single model for the whole period

- Unrestricted: quebra em diferentes perodos

Test with a JointHyp F test. Low p-value (reject H0) means the parameters are different in
each period, meaning there is structural change

H0: = = = 0 Structural stability (o omega era pra ser um lambda)

H1: Any of them is different No structural stability, better to use unrestricted model and
break the regression in different periods

Heteroskedacity

One of the assumptions of the CLRM is that the variance of the error term is constant
(homoskedacity). Heteroskedacity = when the variance of the error term is NOT constant,
creating more space between the residuals of the regression.

In heteroskedacity, OLS is not the best anymore (still LUE) since errors in OLS are larger than in
WLS. The change in the StDev creates a bias as it changes the computed t-statistic and
consequently the p-value, meaning it can lead to wrong decisions in hypothesis testing.

How to fix use Weighted Least Squares (WLS)

There are many ways to impose the weight. A possible rule is to give more weight to
information that is more precise (lower variance).
1
Ex: W= var(ui)
 Autocorrelation

OLS is not efficient anymore use WLS

Detect with Durbin Watson Test d = 2*(1-p-value).

p=0, d=2, no autocorrelation