PRINCIPAL COMPONENT ANALYSIS

Its aim is to reduce a larger set of variables into a smaller set of 'artificial' variables
(called principal components) that account for most of the variance in the original
variables.

Objective: To study the factors involved for usage of mobile

Variables: All Ordinal variables

Test: Factor Analysis

Assumptions:

Assumption #1: You have multiple variables that are measured at the continuous level
(although ordinal data is very frequently used).

Assumption #2: There should be a linear relationship between all variables.

Assumption #3: There should be no outliers.

Assumption #4: There should be large sample sizes for a principal components analysis to
produce a reliable result. Many different rules-of-thumb have been proposed that differ
mostly by either using absolute sample size numbers or a multiple of the number of
variables in your sample. Generally speaking, a minimum of 150 cases or 5 to 10 cases per
variable have been recommended as minimum sample sizes.

Order of testing to run a principal components analysis

There are six steps, other than the testing of assumptions, when running a principal
components analysis. These are:

1. Initial extraction of the components.

2. Determining the number of 'meaningful' components to retain.
3. Rotation to a final solution.
4. Interpreting the rotated solution.
5. Computing component scores or component-based scores.
6. Reporting the results.

A principal components analysis can be used to solve three major problems:

(a) removing superfluous/unrelated variables;
(b) reducing redundancy in a set of variables; and
(c) removing multicollinearity.

These three study designs are appropriate when you have met the assumptions of principal
components analysis.
Study Design #1
Removing superfluous/unrelated variables

Principal components analysis allows you to 'cluster' variables together that all load on the
same component. If one component only loads on one variable, this could be an indication
that this variable is not related to the other variables in your data set and might not be
measuring anything of importance to your particular study (i.e., it is measuring some other
construct or measure).

Study Design #2
Reducing redundancy in a set of variables

If you have measured many variables and you believe that some of the variables are
measuring the same underlying construct, you might have variables that are highly
correlated. Principal components analysis will allow the reduction of many correlated
variables into a single artificial variable called a principal component. This principal
component can then be used in later analyses. In addition, it can be used to create a scale
of questions.

Study Design #3
Removing multicollinearity

If you have two or more variables that are highly correlated, principal components analysis
might help. It can reduce the highlighted correlated variables into principal components
that can be used to generate a component score which can be used in lieu of the original
variables.

Two of the assumptions that can be tested using SPSS Statistics include:

(a) Linearity between all variables, which can be evaluated using a correlation matrix; and

(b) Sampling adequacy, which can be detected using the Kaiser-Meyer-Olkin (KMO)
measure of sampling adequacy for the overall data set, the KMO measure for each
individual variable and Bartlett's test of sphericity.

Procedure:
Interpretation of Outputs:

You want to examine the correlations in this table to see if there are any variables
that are not strongly correlated with any other variable. The level of correlation
considered worthy of a variable's inclusion is usually r ≥ 0.3. Thus, scan the correlation
matrix for any variable that does not have at least one correlation with another
variable where r ≥ 0.3.

In this data set, all variables have at least one correlation with another variable
greater than the 0.3 cut-off.

Thus the assumption of Linearity is met.

Sampling adequacy
There are a few methods to detect sampling adequacy:
(1) the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy for the overall data
set; (2) the KMO measure for each individual variable; and
(3) Bartlett's test of sphericity.

(1) the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy for the overall
data set
The KMO measure is used as an index of whether there are linear relationships
between the variables and thus whether it is appropriate to run a principal
components analysis on your current data set. Its value can range from 0 to 1, with
values above 0.6 suggested as a minimum requirement for sampling adequacy, but
values above 0.8 considered good and indicative of principal components analysis being
useful. A KMO measure can be calculated for all variables combined and for each
variable individually. The overall KMO measure is provided in the KMO and Bartlett's
Test table, as highlighted below:

In this example, the KMO measure is 0.730, which is good; or "Middling" on Kaiser's
(1974) classification of measure values, as shown in the table below:

(2) the KMO measure for each individual variable

KMO measures (Measures of Sampling Adequacy) for individual variables are found on
the diagonals of the anti-image correlation matrix, as found in the Anti-image Matrices
table under the "Anti-image Correlation" (2nd) section, as highlighted below (not to be
confused with the "Anti-image Covariance" (1st) section):
You are again looking for the KMO measures to be as close to 1 as possible, with values
above 0.5 an absolute minimum and greater than 0.8 considered good. If any particular
variable has a low KMO measure (KMO < .5), you should consider removing it from the
analysis.

3) Bartlett's test of sphericity:

Bartlett's test of sphericity tests the null hypothesis that the correlation matrix (see
screenshot above) is an identity matrix. An identity matrix is one that has 1's on the
diagonal and 0's on all the off-diagonal elements. Effectively, it is saying that there are
no correlations between any of the variables. This is important because if there are no
correlations between variables, you will not be able to reduce the variables to a
smaller number of components (i.e., there is no point in running a principal
components analysis). As such, you want to reject the null hypothesis. The result of
this test is contained within the second part of the KMO and Bartlett's Test table, as
highlighted below:

In this example, you can see that Bartlett's test of sphericity is statistically significant
(i.e. p < .05). This significance value (i.e., p-value) is located in the "Sig." row. In this
example, it is stated as ".000", which actually means p < .0005. This test is particularly
sensitive to deviations from multivariate normality and, as such, you might consider
not using this test.

The output generated by SPSS Statistics is quite extensive and can provide a lot of
information about your analysis. However, you will often find that the analysis is not
yet complete and more re-runs of the analysis will have to take place before you get
to your final solution. This will be discussed over the next three pages, which will
focus on: (a) communalities; (b) extracting and retaining components; and (c) forced
factor extraction.

Communalities: The communality is the proportion of each variable's variance that is

accounted for by the principal components analysis and can also be expressed as a
percentage

Extracting and retaining components: A principal components analysis will produce as

many components as there are variables. However, the purpose of principal
components analysis is to explain as much of the variance in your variables as possible
using as few components as possible. After you have extracted your components, there
are four major criteria that can help you decide on the number of components to
retain: (a) the eigenvalue-one criterion, (b) the proportion of total variance accounted
for, (c) the scree plot test, and (d) the interpretability criterion. All except for the
first criterion will require some degree of subjective analysis.

Forced factor extraction: When extracting components as part of your principal

components analysis, SPSS Statistics does this based on the eigenvalue-one criterion.
However, it is possible to instruct SPSS Statistics how many components you want to
retain.

Communalities

The communality is the proportion of each variable's variance that is accounted for by
the principal components analysis and can also be expressed as a percentage.
Communalities are reported in the Communalities table, as shown below:

Extracting components:

A principal components analysis will produce as many components as there are

variables. There are 25 variables in this example analysis, so there are 25 components.
If you were to retain all components in your analysis you will be able to account for all
the variance in your variables. However, this is not the purpose of principal
components analysis; it is to explain as much of the variance as possible using as few
components as possible. The first component will explain the greatest amount of total
variance, with each subsequent component accounting for relatively less of the total
variance. Generally speaking, only the first few components will need to be retained
for interpretation and these components will account for the majority of the total
variance. The amount of variance each component accounts for plus its contribution
towards total variance is presented in the Total Variance Explained table under the
"Initial Eigenvalues" columns, as highlighted below:

An eigenvalue is a measure of the variance that is accounted for by a component. An

eigenvalue of one represents the variance of one variable, so with 10 variables there
is a total of 10 eigenvalues of variance. Therefore, if you examine the first
component, you will find that it explains 3.938 eigenvalues of variance (the "Total"
column), which is 3.938/10 x 100 = 39.385% of the total variance, as reported in the
"% of Variance" column.

Each successive percentage of variance explained is calculated in the same way with
the cumulative percentage recorded in the "Cumulative %" column. You can see from
the above table that all 10 components explain all of the total variance. Unless you
remove cases (e.g., participants) or variables from the principal components analysis,
the values contained within these three columns will never change regardless of how
many components you decide to retain or how you decide to rotate the solution. The
next sections show you how to decide how many components to retain in your analysis.

Criteria for choosing components to retain

Now that you have extracted all the principal components, you need to determine how
many components to retain for rotation and interpretation. Although there are some
statistical considerations in deciding this number you will also have to make subjective
judgements. You must remember that with principal components analysis there is no
one objectively correct answer; the correct number of components to retain is
something that you have to decide. There are four major criteria that can help you
decide on the number of components to retain. These are the eigenvalue-one
criterion, the proportion of total variance accounted for, the scree plot test, and the
interpretability criterion. All except for the first criterion will require some degree of
subjective analysis. These four criteria are presented in the next four sections:

The eigenvalue-one criterion

The eigenvalue-one criterion (also referred to as the Kaiser criterion (Kaiser, 1960)) is
one of the most popular methods for establishing how many components to retain in a
principal components analysis and it is the default option in SPSS Statistics. Indeed,
this default was kept for this first iteration of principal components analysis in this
guide. An eigenvalue less than one indicates that the component explains less variance
that a variable would and hence shouldn't be retained. The major advantage of this
criterion is that it is very simple. Unfortunately, there are also some major pitfalls.
One pitfall is when the eigenvalue of a component(s) hovers close to one. In this
example, the fifth component has an eigenvalue of 1.049 and the sixth component an
eigenvalue of 0.951. Therefore, the interpretation is fairly clear: components one to
five are retained and the sixth component onwards are not. This is highlighted below:

However, if, for example, the fifth component had an eigenvalue of 1.002 and the
sixth component an eigenvalue of 0.998, you would be declaring the fifth component
to be meaningful (retained), but the sixth component as trivial (not retained), even
though there is virtually no difference in the variance accounted for by the two
components. Using the criterion in this example results in five components being
retained.

Percentage of variance explained

There are two criteria that centre around examining the proportion/percentage of
total variance explained by each component: (1) the proportion of variance explained
by each component individually; and (2) the cumulative percentage of variance
explained by a set number of components. You will find this information in the "% of
Variance" and "Cumulative %" columns under the "Initial Eigenvalues" column in the
Total Variance Explained table, as highlighted below:

As the component number increases, each subsequent component explains less of the
total variance. Thus, the first component explains 26.9% of the total variance, whereas
the second component explains only 13.4%. And this trend continues with increasing
component number. It has been suggested that a component should only be retained if
it explains at least 5% to 10% of the total variance. In this example, this would lead to
the retention of the first four components. Another criterion is to retain all
components that can explain at least 60% or 70% of the total variance. Using the lower
criterion of 60%, would lead to the retention of the first five components (cumulative
percentage of 64.1% versus 59.9% for first four components) in this example. As you
can see, these suggested criteria result in a different number of suggested components
to retain. The percentage figures to determine retention of components are arbitrary
and thus the weakness of this criterion is that it somewhat subjective.

Scree plot

The scree plot (Cattell, 1966) is presented in the output as a line graph entitled Scree
Plot, as shown below:
 INFLECTION POINT

A scree plot is a plot of the total variance explained by each component (its
"eigenvalue") against its respective component. As there are as many components as
there are variables, there are 10 components in the scree plot. The components to
retain are those before the (last) inflection point of the graph (see above). The
inflection point is meant to represent the point where the graph begins to level out
and subsequent components add little to the total variance.

In this example, visual inspection of the scree plot would lead to the retention of four
components. You should already be noticing that the number of components suggested
to retain using the different criteria are not always the same. This is one of the
reasons why you have to make a subjective decision on how many components to
retain.

Interpretability criterion

The interpretability criterion is arguably the most important criterion and it largely
revolves around the concept of "simple structure" and whether the final solution makes
sense. For the number of components to influence the interpretability of the final
solution, you first need to inspect the rotated component matrix, which is presented in
the Rotated Component Matrix table, as shown below:
The Rotated Component Matrix table shows how the retained, rotated components
load on each variable. Be aware that each retained component will load on each
variable and that the only reason there are what appears to be missing values is due to
the settings you made earlier when you instructed SPSS Statistics to suppress all
coefficients less than .03 (see the Factor Analysis: Options dialogue box instructions in
the Procedure section, if needed). You can see from this table that there is still some
"complex structure"; namely, many components loading on the same individual
variables. For example, components 1 and 5 both load on variable Qu18. Ultimately, it
is still difficult to interpret this rotated solution when you have "complex structure" in
evidence. What you are trying to achieve is called "simple structure". Simple structure
is when each variable has only one component that loads strongly on it and each
component loads strongly on at least three variables. The overriding concept of the
interpretability criterion is whether the final rotated solution makes sense (i.e., can
you explain it). In this example, so far, the answer is No.
Findings:

A principal components analysis (PCA) was run on a 10-question questionnaire that

measured desired mobile characteristics on 500 candidates.

The suitability of PCA was assessed prior to analysis.

Inspection of the correlation matrix showed that all variables had at least one
correlation coefficient greater than 0.3.

The overall Kaiser-Meyer-Olkin (KMO) measure was 0.73 with individual KMO measures
all greater than 0.7, classifications of 'middling' to 'meritorious' according to Kaiser
(1974).

Bartlett's test of sphericity was statistically significant (p < .0005), indicating that the
data was likely factorizable.

PCA revealed five components that had eigenvalues greater than one and which
explained 39.385%,19.740% and 11.310% of the total variance, respectively.

Visual inspection of the scree plot indicated that three components should be retained
(Cattell, 1966). In addition, a three-component solution met the interpretability
criterion. As such, three components were retained.

The three-component solution explained 70.435% of the total variance.

A Varimax orthogonal rotation was employed to aid interpretability. The rotated

solution exhibited 'simple structure' (Thurstone, 1947). The interpretation of the data
was consistent with the personality attributes the questionnaire was designed to
measure with strong loadings of PERSONALITY AND SELF efficacy items on Component
1, ASSET VALUE items on Component 2 and ENTERTAINMENT items on Component 3.

Component loadings and communalities of the rotated solution are presented.

Henry Garrett Ranking Method - Factor Analysis data