Factor Analysis

Introduction:
It is a statistical technique used to explore data for patterns, confirm theoretical
hypotheses or to reduce a large number of variables to a smaller manageable
number. It explains the variability amongst observed, correlated variables in
terms of a smaller number of unobserved variables called factors.

Applications (2006, Elizabeth Garrett-Mayer):

1. Helps in identifying underlying factors by clustering variables into
homogeneous sets or by creating new variables (factors)
2. Helps us screen variables by identifying groupings to allow us to select
one variable to represent many
3. Allows us to describe many variables using a few factors
4. Helps us to select a small group of representative variables from a larger
set
5. Allows us to cluster objects based on their factor scores

Flowchart:

Algorithm (1992, Andrew L Comrey, Howard B. Lee):

STEP 1: Select variables and collect data.
Usually done through questionnaires
STEP 2: Compute matrix of correlations between variables.
When the correlation matrix has numerically large correlation coefficients in it, it
indicates that the variables involved are related to each other.
Example: Go to Analyze menu, choose Dimension Reduction and from the
sub-menu choose Factor.

Add all the variables to be used for data reduction, taking care not to include
dummy variables such as Subject Number or Serial Number for example.

Choose Descriptives, a new window as shown below, will appear. Check the
options as shown in the window and click Continue.

Options
Univariate descriptives
Initial solution
Coefficient
Significance levels
Determinant
KMO and Bartletts
Inverse
Reproduced
Anti-image

Descriptions
Mean and standard deviation
Communalities estimate for the factors
R-matrix
Significance value matrix for the R-matrix
Test for multicollinearity or singularity
Kaiser-Meyer-Olkin measure of sampling adequacy
and Bartletts test
Provides inverse of the correlation matrix
Correlation matrix for the model
Anti-image matrix of covariance and correlation

(A Beginners Guide to Factor Analysis: Focusing on Exploratory Factor Analysis,

An Gie Yong and Sean Pearce, 2013)
Now choose Extraction, a new window as shown below, will appear. Check the
options as shown in the window and click Continue.

Option
Description
Correlation
recommended for variables that are not meaningfully
Matrix
comparable
Covariance
Matrix
for variables that are similar
Principal Axis
Factors are successively extracted till there is a large enough
Factoring
variance accounted for in Correlation Matrix
Principal
Component
Extracts maximum variance from data set with each component
Analysis
thus reducing large number of variables into smaller components
(A Beginners Guide to Factor Analysis: Focusing on Exploratory Factor Analysis,
An Gie Yong and Sean Pearce, 2013)
Now choose Rotation, a new window as shown below, will appear. Check the
options as shown in the window and click Continue.

Option

Description
Orthogonal Rotation; Minimization of number of variables that have
high loadings on each factor and makes smaller loadings even
Varimax
smaller
Orthogonal Rotation; Minimization of number of factors needed to
Quartimax
explain each variable
Direct
Oblique Rotation; Simplification of structure and mathematics of the
Oblimin
output
Promax
Oblique Rotation; Raising loadings to a power of 4
(A Beginners Guide to Factor Analysis: Focusing on Exploratory Factor Analysis,
An Gie Yong and Sean Pearce, 2013)
Click OK in the Factor Analysis window.
The observed Correlation Matrix is as follows:
Correlation Matrixa
crime
Correlation

pa_manip

sex_def

sex_subj

sentence

serious

crime

1.000

-.003

.010

.010

.074

.146

pa_manip

-.003

1.000

.003

.003

-.135

-.103

sex_def

.010

.003

1.000

.003

.058

-.066

sex_subj

.010

.003

.003

1.000

-.160

.001

sentence

.074

-.135

.058

-.160

1.000

.551

serious

.146

-.103

-.066

.001

.551

1.000

exciting

-.145

.489

-.085

-.002

.028

.099

calm

-.069

.149

.015

.057

-.028

.085

independ

.058

.194

-.109

.047

.042

.033

sincere

.063

.072

.013

.148

-.007

.007

-.006

.064

.066

.018

-.077

-.162

phy_attr

.000

.901

-.047

-.038

-.147

-.114

kind

.093

.099

.074

.018

-.105

-.151

intellig

-.395

.350

.136

.025

-.035

.009

strong

-.133

.405

-.043

.046

-.047

.076

sophist

-.234

.445

.055

.054

-.066

.059

happy

-.167

.266

.047

-.013

.029

.002

sociable

-.352

.294

-.070

.047

-.011

.058

warm

Correlation Matrixa
exciting
Correlation

crime

calm

independ

sincere

warm

phy_attr

-.145

-.069

.058

.063

-.006

.000

.489

.149

.194

.072

.064

.901

sex_def

-.085

.015

-.109

.013

.066

-.047

sex_subj

-.002

.057

.047

.148

.018

-.038

sentence

.028

-.028

.042

-.007

-.077

-.147

serious

.099

.085

.033

.007

-.162

-.114

exciting

1.000

.166

.240

.014

.035

.507

calm

.166

1.000

.294

.077

.070

.092

independ

.240

.294

1.000

.220

.151

.202

sincere

.014

.077

.220

1.000

.536

.133

warm

.035

.070

.151

.536

1.000

.114

phy_attr

.507

.092

.202

.133

.114

1.000

kind

.051

.078

.118

.389

.538

.182

pa_manip

intellig

.370

.260

.167

.091

.142

.356

strong

.424

.303

.309

.188

.213

.391

sophist

.380

.326

.184

.088

.084

.419

happy

.396

.213

.155

.136

.246

.285

sociable

.443

.247

.118

.015

.013

.279

Correlation Matrixa
kind

intellig

strong

sophist

happy

sociable

crime

.093

-.395

-.133

-.234

-.167

-.352

pa_manip

.099

.350

.405

.445

.266

.294

sex_def

.074

.136

-.043

.055

.047

-.070

sex_subj

.018

.025

.046

.054

-.013

.047

sentence

-.105

-.035

-.047

-.066

.029

-.011

serious

-.151

.009

.076

.059

.002

.058

exciting

.051

.370

.424

.380

.396

.443

calm

.078

.260

.303

.326

.213

.247

independ

.118

.167

.309

.184

.155

.118

sincere

.389

.091

.188

.088

.136

.015

warm

.538

.142

.213

.084

.246

.013

phy_attr

.182

.356

.391

.419

.285

.279

1.000

.148

.267

.134

.251

.020

intellig

.148

1.000

.510

.531

.409

.343

strong

.267

.510

1.000

.568

.353

.349

sophist

.134

.531

.568

1.000

.413

.418

happy

.251

.409

.353

.413

1.000

.377

sociable

.020

.343

.349

.418

.377

1.000

Correlation

kind

a.

Determinant = .002

Variables having a great number of small correlation coefficient (r < +/- .30)
should be removed as they indicate a lack of patterned relationships.
Furthermore, correlations that are above r = +/- .90 show that your data may
have a problem of multicollinearity. As a follow-up, check if the Determinant
score is above the rule of thumb of .00001 as this indicates an absence of
multicollinearity. If your data have an issue of multicollinearity, you will need to
determine the item that is causing the problem and remove it from the analysis.
(A Beginners Guide to Factor Analysis: Focusing on Exploratory Factor Analysis,
An Gie Yong and Sean Pearce, 2013)

STEP 3: Extract un-rotated factors.

This is to determine how many factor constructs are required to account for the
pattern of values found in the correlation matrix. This process involves
operations using the coefficients from the correlation matrix to produce a column
of numbers, one for each variable, which represents the loading of the
variables on that factor and are called factor loadings. Factor loadings can be
positive (indicating a positive correlation with the factor construct), negative
(indicating a negative correlation with the factor construct) or zero (indicating no
correlation with the factor construct). Factors are extracted from the correlational
matrix until all the residual correlations are close to zero that they are
presumed to be of negligible importance.
Example: The observed Factor Matrix is as follows.
Factor Matrixa
Factor
1

phy_attr

.757

-.040

-.434

.428

-.141

-.028

pa_manip

.728

-.111

-.402

.359

-.072

.064

sophist

.698

-.144

.092

-.156

.062

.205

strong

.692

.011

.172

-.040

.172

.101

intellig

.657

-.099

.133

-.289

-.156

.211

exciting

.623

-.242

.009

.090

.015

-.226

happy

.547

.027

.180

-.111

-.109

-.071

sociable

.535

-.255

.092

-.231

.018

-.271

calm

.351

-.041

.189

-.103

.289

.113

independ

.336

.082

.155

.135

.316

-.044

warm

.298

.744

.235

.011

-.160

-.131

kind

.301

.568

.130

.050

-.068

.033

sincere

.239

.521

.236

.128

.056

-.059

sentence

-.116

-.323

.634

.379

-.286

-.016

serious

-.033

-.358

.527

.304

.075

.026

crime

-.242

.179

-.016

.542

.217

.155

sex_subj

.044

.088

-.006

-.075

.200

.029

sex_def

.002

.080

.045

-.072

-.241

.317

STEP 4: Repeat STEP 3 until all factors that are needed to account for the
correlations in the correlation matrix have been extracted. The table which
results after the extraction process is complete, the values are arranged in a
table called matrix of un rotated loadings
STEP 5: Rotate factors.
Factor matrix from STEP 4 is rotated to a mathematically equivalent matrix which
represents factor constructs that are much more useful for scientific purposes
than the original un-rotated factor constructs. The un-rotated matrix from STEP 4
tends to contain highly complex factor constructs that relate to or overlap with
many of the variables than just a few of them, hence rotation is required.

Rotated Factor Matrixa

Factor
1

phy_attr

.962

.125

.124

-.096

.022

-.030

pa_manip

.895

.133

.029

-.089

.113

.025

exciting

.486

.423

.022

.134

.167

-.213

crime

.096

-.638

.094

.149

.077

-.050

sociable

.211

.608

-.023

.047

.171

-.200

intellig

.265

.592

.093

.012

.274

.328

sophist

.362

.473

.039

.027

.437

.196

happy

.230

.460

.253

.080

.160

.041

warm

.001

.088

.850

-.088

.005

.046

kind

.095

.018

.628

-.096

.108

.119

sincere

.035

-.040

.613

.022

.168

-.031

sentence

-.064

.006

-.002

.844

-.160

.059

serious

-.045

-.030

-.117

.679

.156

-.063

strong

.342

.368

.213

.066

.495

.054

calm

.065

.197

.049

.055

.468

.005

independ

.171

.034

.194

.093

.394

-.175

sex_subj

-.049

-.020

.039

-.110

.194

-.045

sex_def

-.022

.013

.056

.006

-.057

.406

Total Variance Explained

Factor

Initial Eigenvalues
Total

% of Variance

Extraction Sums of Squared Loadings

Cumulative %

Total

% of Variance

Cumulative %

4.418

24.542

24.542

4.023

22.348

22.348

2.051

11.396

35.938

1.604

8.910

31.258

1.673

9.294

45.231

1.316

7.310

38.568

1.456

8.091

53.322

1.083

6.016

44.584

1.198

6.656

59.978

.544

3.023

47.607

1.041

5.781

65.759

.394

2.190

49.798

.955

5.308

71.067

.755

4.195

75.262

.730

4.057

79.319

10

.649

3.603

82.922

11

.545

3.030

85.952

12

.510

2.833

88.785

13

.477

2.647

91.432

14

.400

2.222

93.654

15

.373

2.075

95.728

16

.358

1.991

97.719

17

.323

1.796

99.515

18

.087

.485

100.000
Total Variance Explained

Factor

Rotation Sums of Squared Loadings

Total

% of Variance

Cumulative %

2.441

13.560

13.560

1.961

10.894

24.454

1.697

9.426

33.880

1.285

7.140

41.020

1.118

6.211

47.231

.462

2.567

49.798

7
8
9
10
11
12
13
14
15
16
17
18

STEP 6: Interpret rotated factor matrix

Variables having high factor loadings on the rotated factors are studied in detail
to determine what they share in common.
Example:

Kaiser-Meyer-Olkin Measure of Sampling Adequacy

Introduction:
Kaiser-Meyer-Olkin measure of sampling adequacy tells us whether the recorded
data or responses is suitable for Factor Analysis.

Established Scales:
Value
0.90 KMO 1.00
0.80 KMO 0.89
0.70 KMO 0.79
0.60 KMO 0.69

Remarks
Marvellous degree of Common
Variance
Meritorious degree of Common
Variance
Middling degree of Common
Variance
Mediocre degree of Common
Variance

Citation
Kaiser, 1974

0.50 KMO 0.59

0.00 KMO 0.49

Miserable degree of Common

Variance
Don't Factor

Example:
KMO and Bartlett's Test
Kaiser-Meyer-Olkin Measure of Sampling Adequacy.

.753

Approx. Chi-Square
Bartlett's Test of Sphericity

1915.217

df

153

Sig.

.000

Reliability Measurement Using Cronbachs Alpha

Introduction (2011, Mohsen Tavakol, Reg Dennick):

Reliability is the ability to measure consistently. Data becomes valid only if it is
reliable, hence measuring the reliability of data is very important. Cronbachs
Alpha is used to measure the internal consistency of a scale or test. Internal
consistency explains the extent to which all items in a test examine the same
concept or construct. Also, reliability estimates show the amount of
measurement error in a test. It basically is the correlation of a test with itself.
Measurement error is given by subtracting the square of this correlation from
unity.

Applications (1993, Cortina J; 2011, Mohsen Tavakol, Reg Dennick):

1. Check internal consistency of a scale or test. Where internal consistency
means the interrelatedness of a sample of test items.
2. Measures amount of measurement error in tests.
3. Can be used to confirm if a sample of items is one-dimensional. Where
one-dimensional means that the items in a test measure a single latent
trait or construct.

Established Scales:
Value

Remark

.9
.7 < .9

Excellent internal consistency

Good internal consistency

Citation
Flynn, Schroeder & Sakakibara,
1994
George & Mallery, 2003

.6 < .7
.5 < .6
< .5

Acceptable internal consistency

Poor internal consistency
Unacceptable

Hair, Anderson, Tatham & Black,

1998
Indrayan & Sarmukaddam, 2001
Kline, 1999