Principle Components Regression(PCR):

The usage of the Principle Components method (PC) dates back to the work of Beltrami 1873 and Jordan
1874, when they, separately, established the Singular Value Decomposition(SVD)

As mentioned in the beginning of this chapter, the PC method is on of the widely-used-biased-linear

models in order to overcome the Multicollinearity problem, which the Multicollinearity Regression
model in the practical environment suffers from. The PC method inverts the original interpretive
variables, without erasing them, into a new variables called the Principle Components, and each
principle component is a linear structure in the original independent variables.

The PC method provides an invaluable number of the original information and observations, such as its
clustering pattern and its relationships with the original variables, it also provides information about the
relationship between the new and old variables and the groups or classes the data or variables contain.
The PCs are sorted according to their contrast, where the first component is the one with the sharpest
contrast and therefore a few number of the components -which are expected to show the sharpest
contrast- are considered, where the components of the least impact are neglected. Finding the principle
components is a crucial a step in removing the impact of the Multicollinearity in order to utilize it in
utilizing the regular least squares methods in estimating the outlines of the original linear regression
model for the interpretive variables[50][25][20].
If (X1,X2,.,Xp) are interpretive variables, a parallel modification can be defined according to
eqauion(57.3).

Z= XA 57.3

Where:

X : is a principle-component matrix of order (n*).

A : is a matrix parallel to the distinguished-standard vector which is parallel to the roots distinguishing
the matrix of the system informations(XX) and its order is (P*P),its elements aij , its columns Aj. It makes

the matrix (XX) diagonal matrix, taking into consideration 1 2 1, 2, are distinguishing
values for the matrix (XX), the variable Zj is distributed with a mean equal zero and a contrst
[68][61][8] .

To express Y as a function in the principle components instead of the independent-correlated variables

(X1,X2,XP). Hence A is a parallel matrix, where I=AA, X= ZA in the regression model equation (2.2),
then the regression model would be according to the following equation (58.3).

= + 58.3

Supposing =A, the regression model equation would be according to the following equation (59.3):
 = + 59.3

Where :

represents the assigned vector (1, 2 , ) equivalent to the principle component

(1, 2 , ) which could be estimated using the least square according to equation (60.3).

= ( )1 = 1 60.3

Matrix is a diagonal matrix of order (P*P),its elements are the roots distinguishing the matrix (XX).
Expectation for this information is
the variance to the regression model usinig the principle component Zj equal to jj2 , therefore the
interpreted variance values in the response variable values Y by the principle component ZJ is (jj2
/YY)*100, whixh is equal to the square of the correlation factor of the response variable and the
principle component Zj multiplied by 100. Therefor the mean square error of the regression trend line
is :
1
= 1 =1( )2 =1 62.3

To obtain the original interpretive variables of the regression model, the relationship between the
original and the regression model.restricted to the variable regression Y on the principle
component Z is utilized as follows:

If =
When using roots distinguishing the correlation factors matrix instead of variance and covariance
matrix, as inputs in analyzing the principle components regression, nj should be used instead of j.

The relation (3.73) shows that the variance of the .for the estimated regression model is also
dependent on the roots -distinguishing the matrix(XX)- and hence it is impacted with the distinguishing-
small roots existence which leads to variances inflation. According to the definition of the principle
components , the small-distinguishing roots which contribute in the inflation of the variance of the
regression model is always correspondent to the last principle components of the matrix (XX),
therefore the total variance of the .should be reduced , excluding the principle components
correspondent to the smallest roots distinguishing the matrix(XX).

Some researcher such as Chatterjee, Price, Jolliffee and Jeffers suggested that the principle components
correspondent to the distinguishing roots less than 70% to be excluded. Morrison suggested selecting
the principle components which interpret at least75% of the variance in the response-variable values.
This pearcent could be obtained dividing the sum of the distinguishing roots - correspondent to k-of the
principle components over the sum of the distinguishing roots when using the variance and covariance
matrix of the variables as input for analyzing the principle component according to [91][68][61][8]:
 =1
( ) 100
=1

When using the correlation matrix as input to perform the principle components analysis, the number of
the interpretive variables are used instead of the sum of the distinguishing roots .

The model of the regression is built to the responses variable Y on the remaining principle components
after excluding those do not satisfy the pre-mentioned criterion. Supposing S is a distinguishing root
with a high value amongst P roots, distinguishing for the matrix(XX), there would be (p-s) of
distinguishing roots with low values -in other words, a number of (p-s) from the principle components Z
is excluded, then the regression model Y is adjusted on the principle components and hence the
prediction equation would be as follows[68][61][8][50]:

= =1 69.3

the sum of the error squired of the regression model is calculated with number S of the principle
components as follows:
 1
2 = = 1 =1( )2 _ =1 2 70.3

Fortunately , the property of parallelization of the estimated least squares of would not differ in the
case of using all the principle components or part of it, and accordingly the regression models ..
s are estimated as follows:
= 1
Z Y 71.3

where :

=diag(1, 2,, s)
The vector . could be obtained by dividing the matrix as follows:

A=[As:Ap-s] 72.3
Wher A1=[a11,a12,a1s] and As=[A1,A2,..As]
the principle component matrix is :

S=[Z1,Z2,,Zs]
And the vector is:
s =[s:p-s-1]
when obtaining . s an estimate for the it can be used in estimating the
original regression model as follows:

= 73.3

The regression ..of the interpretive variables could be obtained using the
principle components regression as follows:

= =1 , I= 1,2,.,p 74.3