Regression Modelling with Actuarial and Financial Applications: Key Notes

Chapter 2 – Basic Linear Regression

 Use logarithmic scale to “normalize” values with large range

2.6 Building a Better Model: Residual Analysis

 Quantile-Quantile Plots (qq plots): Normality Check

Similar concept to probability-probability plots (pp plots) – (𝐹̂𝑛 (𝑋𝑖 ), 𝐹𝑛 (𝑋𝑖 ))
Concept focuses on quantiles, anchoring on linearity as a check to normal conformity
Need to consider number of data points for appropriateness, difficult to assess with only
few data points

 Outliers and High Leverage Points

Outlier is unusual observation in the vertical direction (y’s)
High leverage point is unusual observation in the horizontal direction (x’s)
One can be both outlier and high leverage point
Regression coefficients can be significantly affected by outlier and high leverage point
Chapter 3 – Multiple Linear Regression

3.1 Method of Least Squares

 In summarizing multivariate data, can consider the use of scatterplot matrix and
corresponding correlation matrix. To note correlation matrix merely measures the extent of
linear relationships and is not able to capture non-linear relationships.
 The use of matrix to derive least squares coefficients

3.2 Linear Regression Model and Properties of Estimators

 Regression Coefficient Interpretation

Sign of the regression coefficient
Amount of the regression coefficient
 Model Assumptions
Observables representation and error representation are both equivalent set of
assumptions, each with comparative advantage in understanding regression analysis

Three important properties are derived from the observables representation.

Property 1: 𝐛 = (𝐗 ′ 𝐗)−1 𝐗 ′ 𝐲 is unbiased estimator of the parameter vector 𝛃

Property 2: Var 𝐛 = σ2 (𝐗 ′ 𝐗)−1

Property 3: least squares estimator 𝐛 is normally distributed

3.3 Estimation and Goodness of Fit

Residual standard deviation. An estimator of 𝜎 2 , the mean squared error (MSE), is defined as
𝑛
2
1
𝑠 = ∑(𝑦𝑖 − 𝑦̂𝑖 )2
𝑛 − (𝑘 + 1)
𝑖=1

K represents the number of exogenous variable.

 The Coefficient of Determination: R2.
𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑆𝑆
𝑅2 =
𝑇𝑜𝑡𝑎𝑙 𝑆𝑆
Whenever explanatory variable is added to the model, R2 never decreases. To account for this
issue, use instead the coefficient of determination adjusted for degrees of freedom
𝐸𝑟𝑟𝑜𝑟 𝑆𝑆/[𝑛 − (𝑘 + 1)]
𝑅𝛼2 = 1 −
𝑇𝑜𝑡𝑎𝑙 𝑆𝑆/(𝑛 − 1)
The addition of explanatory variable to k will reduce the denominator for MSE and penalize
useless addition of explanatory variable

3.4 Statistical Inference for a Single Coefficient

Following property 3, the test statistics 𝑏𝑗 /𝑠𝑒(𝑏𝑗 ) can be shown to be the t-distribution with df =
n-(k+1). This is useful in testing the significance and confidence interval of 𝑏𝑗 . The confidence
interval can be used to infer if any hypothesized value (i.e. the null hypothesis) is to be rejected.

Rule of thumb is, if sample size is large enough, we can interpret a variable to be important if its
t-ratio exceeds two in absolute value

A graphical device to overcome the shortcomings of pair correlation is added variable plot, also
called partial regression plot.

It is possible that in a multiple linear regression context, one coefficient indicates negative
relationship with response but shows positive correlation between that variable and response. It
must be noted that however regression coefficients consider other variables in effect and
correlation is based on the pair values only. Added variable plot can provide insight to this
phenomenon.

The correlation for the added variable plot is called as partial correlation coefficient. It is the
correlation between the response and variable of interest in the presence of the other
explanatory variables. Partial correlation coefficient can be calculated either by drawing the plot
or using the formula:
𝑡(𝑏𝑗 )
𝑟(𝑦, 𝑥𝑗 |𝑥1 , … , 𝑥𝑗−1 , 𝑥𝑗+1 , … , 𝑥𝑘 ) =
2
√𝑡(𝑏𝑗 ) + 𝑛 − (𝑘 + 1)

3.5 Some Special Explanatory Variables

Linear in “Linear Regression” implies linearity in the parameters but may be a highly non-linear
function of the explanatory variables