Guidelines for Reliability, Confirmatory and Exploratory Factor Analysis Diana Suhr, Ph.D.

, University of Northern Colorado, Greeley CO Mary Shay, Cherry Creek Schools, Denver CO
Abstract
Reliability refers to accuracy and precision of a measurement instrument. Confirmatory factor analysis (CFA) is a statistical technique used to verify the factor structure of a measurement instrument. EFA, traditionally, is used to explore the possible underlying factor structure of a measurement instrument. Guidelines for reliability, confirmatory and exploratory factor analysis will be discussed. Examples using the Race and Schooling Instrument (Revised, Shay 2008) and PROC CORR, PROC CALIS and PROC FACTOR illustrate reliability, CFA and EFA statistical techniques.

Introduction
CFA and EFA are powerful statistical techniques. An example of CFA and EFA could occur with the development of measurement instruments, e.g. a satisfaction scale, attitudes toward health, customer service questionnaire. During the process, a blueprint is developed, questions written, a response scale determined, the instrument pilot tested, data collected, and CFA completed. The blueprint identifies the factor structure or what we think it is. If questions do not measure what we thought they should, the factor structure does not follow our blueprint, the factor structure is not confirmed, and EFA is the next step. EFA helps us determine what the factor structure looks like according to participant responses. Exploratory factor analysis is essential to determine underlying constructs for a set of measured variables.

Measurement Instrument
The example for this presentation analyzes data collected with the Race and Schooling School-Based Instrument (Shay, 2008). The 23-item instrument measures the essential principles of culturally responsive teaching and learning. Each question, aligning to a principle and category, provides insight into participants levels of involvement in culturally responsive teaching practices and attitudes and beliefs toward confronting institutional bias and discrimination in schools (Shay, in press). The Race and Schooling School-Based Instrument was developed by Singleton (2003) to include seven categories. Permission to use and adjust the instrument for research purposes was granted by Singleton to Shay (2008). See Appendix A to review the survey instrument. The original survey instrument had a 5-point Likert scale with (1) Rarely, (3) Sometimes, and (5) Often as response choices to survey questions. In the revised version, responses were changed to (1) Almost Never; (2) Seldom; (3) Sometimes; (4) Frequently; and (5) Almost Always. Additional questions were added to allow respondents to provide information on demographics, e.g., years of experience in the district and in the field of education. Data was collected in a metropolitan School District in Midwestern United States. Of the 282 respondents, 16% were male (n=42) and 84% were female (n=220). The majority of respondents were white (89.1%, n=229) with 3.5% Hispanic (n=9), 3.5% Black (n=9), 2.0% Asian (n=5) and 2.0% identified their race as Other.

Reliability
Reliability refers to the accuracy and precision of a measurement procedure (Thorndike, Cunningham, Thorndike, & Hagen, 1991). Reliability may be viewed as an instruments relative lack of error. In addition, reliability is a function of properties of the underlying construct being measured, the test itself, the groups being assessed, the testing environment, and the purpose of assessment. Reliability answers the question, How well does the instrument measure what it purports to measure? Some degree of inconsistency is present in all measurement procedures. The variability in a set of item scores is due to the actual variation across individuals in the phenomenon that the scale measures, made up of true score and error. Therefore, each observation of a measurement (X) is equal to true score (T) plus measurement error (e), or X = T + e. Another way to think about total variation is that it has two components: signal (i.e., true differences in the latent construct) and noise (i.e., differences caused by everything but true differences).

Sources of measurement inconsistency could be due to 1) a person changing from one testing to the next a. the amount of time between tests may have resulted in growth or change b. motivation to perform may be different at each testing c. the individual may be physically better able to perform, e.g., more rested d. the individual may have received tutoring between testings 2) the task being different from one testing to the next a. different environment b. different administrator c. different test items on parallel forms 3) the sample of behavior resulting in an unstable or undependable score a. the sample of behavior and evaluation of it are subject to chance or random influences b. a small sample of behavior does not provide a stable and dependable characterization of an individual c. for example, the average distance of 100 throws of a football would provide a more stable and accurate index of ability than a single throw. Reliability may be expressed 1) as an individuals position in a group (correlation between first and second measurements; the more nearly individuals are ranked in the same order, the higher the correlation and the more reliable the test) 2) within a set of repeated measures for an individual (internal consistency, how consistency are items answered) Reliability can be assessed by 1) repeating the same test or measure (test-retest) 2) administering an equivalent form (parallel test forms) 3) using single-administration methods
a. b. subdividing the test into two or more equivalent parts internal consistency measured with Cronbachs coefficient alpha.

Internal Consistency
Internal consistency is a procedure to estimate the reliability of a test from a single administration of a single form. Internal consistency depends on the individuals performance from item to item based on the standard deviation of the test and the standard deviations of the items. = ( n ) ( n - 1) ( SDt2 - SDi2 ) ( SDt2 )

(1)

where

is the estimate of reliability, n is the number of items in the test, SDt is the standard deviation of the test scores means take the sum and covers the n items, SDi is the standard deviation of the scores from a group of individuals on an item.

KR20, Kuder-Richardson Formula 20, is a special form of coefficient alpha that applies when items are dichotomous (e.g., yes/no, true/false) or are scored as right or wrong.

Factors Influencing Reliability

Many factors can affect the reliability of a measurement instrument. They are the 1) range of the group a. pooling a wider range of grades or ages produces a reliability coefficient of higher magnitude b. take into account the sample on which the reliability coefficient is based when comparing instruments 2) level of ability in the group a. precision of measurement can be related to ability level of the people being measured b. report the standard error of measurement for different score levels 3) methods used for estimating reliability a. amount of time between administrations b. method of calculating reliability

4)

length of the test a. when reliability is moderately high, it takes a considerable increase in test length to increase reliability b. relationship of reliability to length of test can be expressed with rkk = krtt 1 + ( k 1) rtt

(2)

where rkk is the reliability of the test k times as long as the original test, rtt is the reliability of the original test, and k is the factor by which the length of the test is changed. For example, If reliability is .60 for a 10-item instrument, what is reliability for a 20-item instrument? rkk = 2(.60) / (1 + (2 1)(.60)) = 1.20 / 1.60 = 0.75

Levels of Reliability
Acceptable levels of reliability depend on the purpose of the instrument. Acceptable reliability of instruments developed for research purposes can be as low as 0.60. An acceptable reliability level of a diagnostic instrument used for making decisions about individuals (e.g., a psychological measure) should be much higher, e.g., 0.95.

Comparisions
The reliability coefficient provides a basis for assessment instrument comparison when measurement is expressed in different scales. The assessment with the higher reliability coefficient could provide a more consistent measurement of individuals.

Statistical Power
An often overlooked benefit of more reliable scales is that they increase statistical power for a given sample size (or allow smaller sample size to yield equivalent power), relative to less reliable measures. A reliable measure, like a larger sample, contributes relatively less error to the statistical analysis. The power gained from improving reliability depends on a number of factors including 1) the initial sample size 2) the probability level set for detecting a Type I error 3) the effect size (e.g., mean difference) that is considered significant 4) the proportion of error variance that is attributable to measure unreliability rather than sample heterogeneity or other sources. To raise the power, substitute a highly reliable scale for a substantially poorer one. For example, when n = 50, two scales with reliabilities of 0.38 have a correlation of 0.24, p < 0.10, and would be significant at p < 0.01 if their reliabilities were increased to 0.90 or if the sample was more than twice as large (n > 100).

PROC CORR and options for Reliability

DATA= OUTP= ALPHA NOMISS NOCORR NOSIMPLE input data set output data set with Pearson correlation statistics compute Cronbachs coefficient alpha exclude observations with missing analysis values suppresses printing Pearson correlations suppresses printing descriptive statistics

Confirmatory Factor Analysis

CFA allows the researcher to test the hypothesis that a relationship between the observed variables and their underlying latent construct(s) exists. The researcher uses knowledge of the theory, empirical research, or both, postulates the relationship pattern a priori and then tests the hypothesis statistically.
3

PROC CALIS and options for CFA

DATA = OUTSTAT= COV CORR METHOD= MAXITER= KURTOSIS MODIFICATION specifies dataset to be analyzed output statistic analyzes covariance matrix analyzes correlation matrix estimation method max iterations compute and display kurtosis modification indices

Exploratory Factor Analysis

Psychologists searching for a neat and tidy description of human intellectual abilities lead to the development of th th factor analytic methods. Galton, a scientist during the 19 and 20 centuries, laid the foundations for factor analytic methods by developing quantitative methods to determine the interdependence between 2 variables. Karl Pearson was the first to explicitly define factor analysis. In 1902, Macdonnell was the first to publish an application of factor analysis. His study compared physical characteristics of 3000 criminals and 1000 Cambridge undergraduates. Factor analysis could be described as orderly simplification of interrelated measures. Traditionally factor analysis has been used to explore the possible underlying structure of a set of interrelated variables without imposing any preconceived structure on the outcome (Child, 1990). By performing exploratory factor analysis (EFA), the number of constructs and the underlying factor structure are identified. Goals of factor analysis are 1) to help an investigator determine the number of latent constructs underlying a set of items (variables) 2) to provide a means of explaining variation among variables (items) using few newly created variables (factors), e.g., condensing information 3) to define the content or meaning of factors, e.g., latent constructs Assumptions underlying EFA are Interval or ratio level of measurement Random sampling Relationship between observed variables is linear A normal distribution (each observed variable) A bivariate normal distribution (each pair of observed variables) Multivariate normality Limitations of EFA are the correlations, the basis of factor analysis, describe relationships. No causal inferences can be made from correlations alone. the reliabilty of measurement instrument (avoid instrument with low reliability) sample size ( larger sample larger correlation) o minimal number for reliable results is greater than 100 and 5 times the number of items o since some subjects may not answer every item, a larger sample is desirable, e.g., for 30 items, at least 150 subjects (5*30), a sample of 200 subjects would allow for missing data. sample selection o Representative of population o Do not pool populations variables could be sample specific, e.g., a unique quality possessed by a group does not generalize to the population nonnormal distribution of data

Factor Extraction
Factor analysis seeks to discover common factors. The technique for extracting factors attempts to take out as much common variance as possible in the first factor. Subsequent factors are, in turn, intended to account for the maximum amount of the remaining common variance until, hopefully, no common variance remains. Direct extraction methods obtain the factor matrix directly from the correlation matrix by application of specified mathematical models. Most factor analysts agree that direct solutions are not sufficient. Adjustment to the frames of reference by rotation methods improves the interpretation of factor loadings by reducing some of the ambiguities which accompany the preliminary analysis (Child, 1990). The process of manipulating the reference axes is known as rotation. The results of rotation methods are sometimes referred to as derived solution because they are obtained as a second stage from the results of direct solutions. Rotation applied to the reference axes means the axes are turned about the origin until some alternative position has o been reached. The simplest case is when the axes are held at 90 to each other, orthogonal rotation. Rotating the o axes through different angles gives an oblique rotation (not at 90 to each other).

Methods
As an aside, names given to factor extraction methods have some interesting origins.

Procrustes was a highwayman who tied his victims to a bed and shaped them to its structure either by stretching them or by cutting off their limbs. In factor analysis, the Procrustes technique/method involves testing data to see how close they fit a hypothesized factor structure. The plasmode method is taken from well-established areas (e.g., physics, chemistry) so that the factor structure is predictable.

Criteria for Extracting Factors

Determining the number of factors to extract in a factor analytic procedure is dependent on meeting appropriate criteria. They are 1) Kaisers criterion, suggested by Guttman and adapted by Kaiser, considers factors with an eigenvalue greater than one as common factors (Nunnally, 1978) 2) Cattells (1966) scree test. The name is based on an analogy between the debris, called scree, that collects at the bottom of a hill after a landslide, and the relatively meaningless factors that result from overextraction. On a scree plot, because each factor explains less variance than the preceding factors, an imaginary line connecting the markers for successive factors generally runs from top left of the graph to the bottom right. If there is a point below which factors explain relatively little variance and above which they explain substantially more, this usually appears as an elbow in the plot. This plot bears some physical resemblance to the profile of a hillside. The portion beyond the elbow corresponds to the rubble, or scree, that gathers. Cattells guidelines call for retaining factors above the elbow and rejecting those below it. This amounts to keeping the factors that contribute most to the variance 3) Proportion of variance accounted for keeps a factor if it accounts for a predetermined amount of the variance (e.g., 5%, 10%). 4) Interpretability criteria a. Are there at least 3 items with significant loadings (>0.30)? b. Do the variables that load on a factor share some conceptual meaning? c. Do the variables that load on different factors seem to measure different constructs? d. Does the rotated factor pattern demonstrate simple structure? Are there relatively i. high loadings on one factor? ii. low loadings on other factors?

Statistics
EFA decomposes an adjusted correlation matrix. Variables are standardized in EFA, e.g., mean=0, standard deviation=1, diagonals are adjusted for unique factors, 1-u. The amount of variance explained is equal to the trace of the matrix, the sum of the adjusted diagonals or communalities. Squared multiple correlations (SMC) are used as communality estimates on the diagonals. Observed variables are a linear combination of the underlying and unique factors. Factors are estimated, (X1 = b1F1 + b2F2 + . . . e1 where e1 is a unique factor). Factors account for common variance in a data set. The amount of variance explained is the trace (sum of the diagonals) of the decomposed adjusted correlation matrix. Eigenvalues indicate the amount of variance explained by each factor. Eigenvectors are the weights that could be used to calculate factor scores. In common practice, factor scores are calculated with a mean or sum of measured variables that load on a factor.
5

The EFA Model is Y = X+ E where Y is a matrix of measured variables X is a matrix of common factors is a matrix of weights (factor loadings) E is a matrix of unique factors, error variation Communality is the variance of observed variables accounted for by a common factor. A large communality value indicates a strong influence by an underlying construct. Community is computed by summing squares of factor loadings 2 d1 = 1 communality = % variance accounted for by the unique factor d1 = square root (1-community) = unique factor weight (parameter estimate)

EFA Steps
1) initial extraction each factor accounts for a maximum amount of variance that has not previously been accounted for by the other factors factors are uncorrelated eigenvalues represent amount of variance accounted for by each factor determine number of factors to retain scree test, look for elbow proportion of variance prior communality estimates are not perfectly accurate, cumulative proportion must equal 100% so some eigenvalues will be negative after factors are extracted, e.g., if 2 factors are extracted, cumulative proportion equals 100% with 6 items, then 4 items have negative eigenvalues interpretability at least 3 observed variables per factor with significant factors common conceptual meaning measure different constructs rotated factor pattern has simple structure (no cross loadings) rotation a transformation interpret solution calculate factor scores results in a table prepare results, paper

2)

3) 4) 5) 6) 7)

PROC FACTOR and options for EFA

DATA = PRIORS =SMC METHOD =ML,ULS ROTATE = SCREE N = MINEIGEN=1 OUT = FLAG = REORDER = specifies dataset to be analyzed squared multiple correlations used as adjusted diagonals of the correlation matrix specifies maximum likelihood and unweighted least squares methods PROMAX (ORTHOGONAL), VARIMAX(OBLIQUE) requests a scree plot of the eigenvalues specifies number of factors specifies select factors with eigenvalues greater than 1 data and estimated factor scores, use raw data and N= include a flag (*) for factor loadings above a specified value arranges factor loadings from largest to smallest for each factor

An example of SAS code to run EFA. priors specify the prior communality estimate proc factor method=ml priors=smc maximum likelihood factor analysis method=uls priors=smc unweighted least squares factor analysis method=prin priors=smc principal factor analysis.

Similarities between CFA and EFA

Both techniques are based on linear statistical models. Statistical tests associated with both methods are valid if certain assumptions are met. Both techniques assume a normal distribution. Both incorporate measured variables and latent constructs.

Differences between CFA and EFA

CFA requires specification of a model a priori the number of factors which items load on each factor a model supported by theory or previous research error explicitly EFA determines the factor structure (model) explains a maximum amount of variance

Statistical Anaysis
With background knowledge of reliability, exploratory and confirmatory factor analysis, were ready to proceed to the statistical analysis!

Reliablity Analysis with PROC CORR (7 factors)

Data was analyzed with PROC CORR to determine the degree of internal consistency (reliability). A Cronbach alpha statistic indicates the level of reliability. Values could range from 0.0 to 1.0 with a value closer to 1.0 indicating a higher level of reliability. Reliability ranged from 0.62 to 0.88 on the total and category subscales for the sample data on the Race and Schooling School-Based Instrument. For research purposes, reliability was acceptable (see Table 1).

Table 1. The Race and Schooling School-Based Instrument - Reliability

Items Factor (category) Professional Development Relationships Engagement Learning and Teaching Achievement Results Community School Culture Total 1, 2, 3, 4, 5 6, 7, 8 9, 10, 11 12, 13, 14 15, 16, 17 18, 19, 20 21, 22, 23 1-23 Alpha (Standardized) 0.80 0.63 0.70 0.73 0.62 0.63 0.79 0.88

SAS Code
PROC CORR procedure calculates a Cronbach Alpha statistics for the reliability analysis. The following code calculates Cronbach Alpha for the total scale (questions 1-23) and for each category subscale. proc corr data=rawsub2 var q1-q23; proc corr data=rawsub2 var q1-q5; proc corr data=rawsub2 var q6-q8; proc corr data=rawsub2 var q9-q11; proc corr data=rawsub2 var q12-q14; proc corr data=rawsub2 var q15-q17; proc corr data=rawsub2 var q18-q20; proc corr data=rawsub2 var q21-q23; nocorr alpha nomiss; nocorr alpha nomiss; nocorr alpha nomiss; nocorr alpha nomiss; nocorr alpha nomiss; nocorr alpha nomiss; nocorr alpha nomiss; nocorr alpha nomiss;

Confirmatory Factor Analysis (7 factor)

Category subscale scores (factors) were created with the MEAN function. The factors were subjected to a confirmatory factor analysis (CFA) with PROC CALIS. Figure 1 illustrates the CFA model. The SAS code below tests the underlying factor structure. proc calis data=rawsub2 kurtosis; lineqs Professional_Development=s1 F1 + e1, Relationships =s2 F1 + e2, Engagement =s3 F1 + e3, Learning_Teaching =s4 F1 + e4, AchResults =s5 F1 + e5, Community =s6 F1 + e6, SchoolCulture =s7 F1 + e7; std e1-e7=vare1-vare7, F1=1; var Professional_Development Relationships Engagement Learning_Teaching AchResults Community SchoolCulture; Another step that could be taken in the confirmatory approach is to test the structure for each of the 7 factors. The following SAS code would test a model for the Professional Development factor. proc calis data=rawsub2 kurtosis; **modification; lineqs q1= p1 F1 + e1, q2= p2 F1 + e2, q3= p3 F1 + e3, q4= p4 F1 + e4, q5= p5 F1 + e5; std e1-e5=vare1-vare5, F1=1; var q1-q5; If each factor can be confirmed, then a combined model is evaluated. If this model is confirmed, then a model including one latent construct (race and schooling) with a direct relationship to each factor is evaluated. Each factor influences responses to items. For example, the latent construct professional development influences responses to items 1, 2, 3, 4 and 5.

Professional Development

Relationships

Engagement

Learningand Teaching

Race and Schooling

Achievement Results

Community

SchoolCulture

Figure 1. Confirmatory Factor Analysis

Results
PROC CALIS procedure provides the number of observations, variables, estimated parameters, and informations (related to model specification), descriptive statistics, and multivariate kurtosis. PROC CALIS procedure also indicates the observations with the largest contribution to kurtosis (see Figure 2).

Fit Statistics
Determine criteria a priori to access model fit and confirm the factor structure. Some of the criteria indicate acceptable model fit while other are close to meeting values for acceptable fit (see Figure 3). Chi-square describes similarity of the observed and expected matrices. Acceptable model fit. Is indicated by a chi-square probability greater than or equal to 0.05. For this CFA model, the chi-square value is not close to zero (Chi-square=35.7367) and p = 0.0011, not greater than 0.05. RMSEA indicates the amount of unexplained variance or residual. A value of 0.743 RMSEA value is larger than the 0.06 or less criteria. CFI (0.9472), NNI (0.9208), and NFI (0.9174) values meet the criteria (0.90 or larger) for acceptable model fit.

The CALIS Procedure Covariance Structure Analysis: Maximum Likelihood Estimation Observations Variables Informations Variable Professional_Development Relationships Engagement Learning_Teaching AchResults Community SchoolCulture Mean 6.50000 6.44681 6.49291 5.83688 7.68440 7.01418 9.32979 282 7 28 Model Terms Model Matrices Parameters Skewness -0.44553 -0.22743 -0.23384 -0.07231 -0.34269 -0.02535 -1.47618 3.9089 1.0620 2.9239 0.0620 0.0555 0.0555 1 4 14 Kurtosis -0.08904 -0.18439 0.12440 -0.00706 -0.01039 -0.06631 1.39821

Std Dev 1.39712 1.48504 1.54008 1.51681 1.43037 1.40404 1.04068

Mardia's Multivariate Kurtosis Relative Multivariate Kurtosis Normalized Multivariate Kurtosis Mardia Based Kappa (Browne, 1982) Mean Scaled Univariate Kurtosis Adjusted Mean Scaled Univariate Kurtosis

Observation Numbers with Largest Contribution to Kurtosis 145 238 87 45 455.0658 290.2460 282.2438 270.1272 Figure 2. Descriptives and Multivariate Kurtosis

47 203.2169

The CALIS Procedure Covariance Structure Analysis: Maximum Likelihood Estimation Fit Function Goodness of Fit Index (GFI) . . . Chi-Square Chi-Square DF Pr > Chi-Square . . . RMSEA Estimate . . . Bentler's Comparative Fit Index . . . Bentler & Bonett's (1980) Non-normed Index Bentler & Bonett's (1980) NFI . . . Figure 3. Fit Statistics 0.1272 0.9621 35.7367 14 0.0011 0.0743 0.9472 0.9208 0.9174

For purposes of this example, 3 fit statistics indicate acceptable fit and 2 fit statistics indicate unacceptable fit. The CFA analysis has not confirmed the factor structure. If the analysis indicates unacceptable model fit, the factor structure cannot be confirmed, an exploratory factor analysis is the next step. The factor structure is not confirmed. No further investigation of the confirmatory model is necessary, parameter estimates, variances, covariances. Proceed with exploratory factor analysis to determine the factor structure.

10

Exploratory factor analysis with PROC FACTOR

method is maximum likelihood diagonals of the correlation matrix are equal to squared multiple correlations criteria set a priori is each factor retained will explain at least 10% of the variance

SAS Code
proc factor data=rawsub2 method=ml priors=smc rotate=varimax reorder; var q1-q23;
Preliminary Eigenvalues: Total = 19.0588542 Eigenvalue 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 11.3729772 3.6419565 2.1325504 1.6888314 1.0239633 0.7563391 0.6567901 0.3993095 0.1894706 0.1391390 0.1049819 0.0119620 -0.0246090 -0.0625760 -0.1113500 -0.2146331 -0.2450015 -0.2575795 -0.3326418 -0.3994169 -0.4345544 -0.4570144 -0.5200403 Difference 7.7310207 1.5094061 0.4437191 0.6648680 0.2676242 0.0995490 0.2574806 0.2098389 0.0503316 0.0341571 0.0930199 0.0365710 0.0379670 0.0487740 0.1032831 0.0303684 0.0125781 0.0750623 0.0667751 0.0351375 0.0224600 0.0630259 Average = 0.82864583 Cumulative 0.5967 0.7878 0.8997 0.9883 1.0420 1.0817 1.1162 1.1371 1.1471 1.1544 1.1599 1.1605 1.1592 1.1560 1.1501 1.1388 1.1260 1.1125 1.0950 1.0741 1.0513 1.0273 1.0000

Proportion 0.5967 0.1911 0.1119 0.0886 0.0537 0.0397 0.0345 0.0210 0.0099 0.0073 0.0055 0.0006 -0.0013 -0.0033 -0.0058 -0.0113 -0.0129 -0.0135 -0.0175 -0.0210 -0.0228 -0.0240 -0.0273

5 factors will be retained by the PROPORTION criterion.

Figure 4. Eigenvalues

Results show 5 factors. Three factors each explain at least 10% of the variance (59.67%, 19.11%, 11.19%) while 2 factors each explain less than 10% of the variance (8.86%, 5,37%). According to criteria set a priori, each factor retained will explain at least 10% of the variance, three factors will be retained. The analysis will be rerun specifying three factors. proc factor data=rawsub2 method=ml priors=smc rotate=varimax reorder n=3; var q1-q23; Figure 5 indicates that both hypothesis tests are rejected, no common factors and 3 factors are sufficient. In practice, we want to reject the first hypotheses and accept the second hypothesis. Tucker and Lewiss Reliability Coefficient indicates good reliability. Reliability is a value between 0 and 1 with a larger value indicating better reliability.

11

Significance Tests Based on 282 Observations Test H0: HA: H0: HA: No common factors At least one common factor 3 Factors are sufficient More factors are needed DF 253 187 Chi-Square 2392.4292 522.6155 Pr > ChiSq <.0001 <.0001

Chi-Square without Bartlett's Correction Akaike's Information Criterion Schwarz's Bayesian Criterion Tucker and Lewis's Reliability Coefficient

542.90188 168.90188 -512.13474 0.78776

Figure 5. Hypothesis Tests, Reliability

Results show items 21, 22, 23 as a factor, items 1, 2, 3, 4 as a factor, and items 5-20 as a factor. Two of the factors are the same or close to the original factor structure. Items 21, 22, 23 are the School Culture factor. Items 1, 2, 3, 4, 5 are the Professional Development factor in the original factor structure. Item 5 is not included in the revised factor structure.

Interpretability
Is there some conceptual meaning for each factor? Could the factors be given a name? Factor1 could be called Professional Development (items 1, 2, 3, 4). Factor2 could be called Attitudes (items 5-20). Factor3 could be called School Culture (items 21, 22, 23).
Rotated Factor Pattern Factor1 q10 q12 q14 q11 q7 q13 q9 q16 q20 q8 q15 q6 q19 q5 q17 q18 q3 q4 q2 q1 q22 q23 q21 0.72350 0.63153 0.57995 0.57515 0.56258 0.55962 0.54358 0.54234 0.49741 0.49444 0.47474 0.45938 0.43811 0.42258 0.37788 0.34111 0.10059 0.16327 0.25783 0.26039 0.07401 0.13707 0.07095 Factor2 0.22819 0.32713 0.18075 0.14190 0.27168 0.11049 0.07180 0.27697 0.17016 0.01762 0.15822 0.34502 0.09902 0.42156 0.10460 0.08108 0.86961 0.75458 0.64261 0.41033 0.01893 -0.07665 -0.03185 Factor3 0.05126 -0.01767 0.03112 -0.06608 0.04450 0.02234 0.08688 0.04569 0.12907 0.02129 0.16368 0.08328 0.12647 -0.02872 0.06759 0.17346 -0.06967 -0.11327 0.05330 0.03917 0.76583 0.75179 0.70232

Figure 6. Factor Loadings

12

Reliability with PROC CORR tested the reliability for the revised factor structure. The SAS code for reliability is proc corr data=rawsub2 nocorr alpha nomiss; var q1-q4; proc corr data=rawsub2 nocorr alpha nomiss; var q5-q20; proc corr data=rawsub2 nocorr alpha nomiss; var q21-q23; Reliability with Cronbach Alpha for items 21, 22, 23 is 0.79, for items 1, 2, 3, 4 is 0.79, and for items 5-20 is 0.88. The range is 0.79 to 0.88 for the revised factor structure. The range for the original factor structure is 0.62 to 0.80

Conclusion
Confirmatory and Exploratory Factor Analysis are powerful statistical techniques. The techniques have similarities and differences. Determine the type of analysis a priori to answer research questions and maximize your knowledge.

WAM (Walk away message)

Select CFA to verify the factor structure and EFA to determine the factor structure. References Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245-276. Child, D. (1990). The essentials of factor analysis, second edition. London: Cassel Educational Limited. DeVellis, R. F. (1991). Scale Development: Theory and Applications. Newbury Park, California: Sage Publications. Hatcher, L. (1994). A step-by-step approach to using the SAS System for factor analysis and structural equation modeling. Cary, NC: SAS Institute Inc. Hoyle, R. H. (1995). The structural equation modeling approach: Basic concepts and fundamental issues. In Structural equation modeling: Concepts, issues, and applications, R. H. Hoyle (editor). Thousand Oaks, CA: Sage Publications, Inc., pp. 1-15. Hu, L. & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. Joreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis, Psychometrika, 34, 183-202. Kline, R. B. (1998). Principles and Practice of Structural Equation Modeling. New York: The Guilford Press. nd Nunnally, J. C. (1978). Psychometric theory, 2 edition. New York: McGraw-Hill. SAS Language and Procedures, Version 6, First Edition. Cary, N.C.: SAS Institute, 1989. SAS Online Doc 9. Cary, N.C.: SAS Institute. SAS Procedures, Version 6, Third Edition. Cary, N.C.: SAS Institute, 1990. SAS/STAT Users Guide, Version 6, Fourth Edition, Volume 1. Cary, N.C.: SAS Institute, 1990. SAS/STAT Users Guide, Version 6, Fourth Edition, Volume 2. Cary, N.C.: SAS Institute, 1990. Schumacker, R. E. & Lomax, R. G. (1996). A Beginners Guide to Structural Equation Modeling. Mahwah, New Jersey: Lawrence Erlbaum Associates, Publishers. Shay, Mary. (in press). An Investigation of the Attitudes, Beliefs, and Values of Elementary School Teachers Toward Race and Schooling. Greeley CO: University of Northern Colorado. Shay, Mary (2008). Race and Schooling Instrument. Thorndike, R. M., Cunningham, G. K., Thorndike, R. L., & Hagen E. P. (1991). Measurement and evaluation in psychology and education. New York: Macmillan Publishing Company. Truxillo, Catherine. (2003). Multivariate Statistical Methods: Practical Research Applications Course Notes. Cary, N.C.: SAS Institute.

Contact Information
Diana Suhr, Ph.D., Statistical Analyst Office of Budget & Institutional Analysis University of Northern Colorado Greeley,CO 80639, diana.suhr@unco.edu Mary Shay, Principal Cottonwood Creek Elementary School Cherry Creek School District Denver CO 80237

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies.
13

Appendix A Race and Schooling Instrument

Revised Shay, (2008)
Professional Staff Development 1. To what extent do teachers and administrators in your school talk openly and constructively about race with each other? 2. To what extent do staff development activities help educators understand the ways in which race influences student behavior? 3. To what extent do staff development activities help educators acquire knowledge about the history and culture of various racial groups? 4. To what extent do staff development activities help educators become knowledgeable about the diverse racial perspectives on historical and current events? 5. To what extent are successful efforts being made in your school to attract, retain, and advance teachers and administrators of color? 1 Almost Never 1 Almost Never 1 Almost Never 1 Almost Never 2 Seldom 2 Seldom 2 Seldom 2 Seldom 3 Sometimes 3 Sometimes 3 Sometimes 3 Sometimes 4 Frequently 4 Frequently 4 Frequently 4 Frequently 5 Almost Always 5 Almost Always 5 Almost Always 5 Almost Always

1 Almost Never

2 Seldom

3 Sometimes

4 Frequently

5 Almost Always

Relationships 6. To what extent do teachers in your school talk openly and constructively about race with students? 7. To what extent do teachers encourage students to have open and constructive conversations about being victims of racial discrimination and about possessing racial power and white privilege? 8. To what extent do teachers structure interracial cooperative groups that enable students of different racial groups to become acquainted with each other? 1 Almost Never 1 Almost Never 2 Seldom 2 Seldom 3 Sometimes 3 Sometimes 4 Frequently 4 Frequently 5 Almost Always 5 Almost Always

1 Almost Never

2 Seldom

3 Sometimes

4 Frequently

5 Almost Always

Engagement 9. To what extent is decision-making in the school widely shared among administrators, teachers, parents and students of a variety of racial groups? 10. To what extent do teachers encourage students of different racial groups to talk with each other openly and constructively about race? 11. To what extent are deliberate actions taken by administrators and teachers to insure that students of color are represented proportionately in extracurricular activities and leadership roles? 1 Almost Never 1 Almost Never 1 Almost Never 2 Seldom 2 Seldom 2 Seldom 3 Sometimes 3 Sometimes 3 Sometimes 4 Frequently 4 Frequently 4 Frequently 5 Almost Always 5 Almost Always 5 Almost Always

14

Learning and Teaching 12. To what extent do teachers help students acquire the knowledge and skills needed to have thoughtful, constructive, heartfelt discussions about race? 13. To what extent are students provided factual information in social studies and other subject areas that contradicts misconceptions about people of color? 14. To what extent does the school curriculum include a focus on racial power and white privilege through examples in history, art, science, and other disciplines? 1 Almost Never 1 Almost Never 1 Almost Never 2 Seldom 2 Seldom 2 Seldom 3 Sometimes 3 Sometimes 3 Sometimes 4 Frequently 4 Frequently 4 Frequently 5 Almost Always 5 Almost Always 5 Almost Always

Achievement Results 15. To what extent do teachers use a variety of assessment devices to ensure that students of all racial groups meet rigorous standards across the curriculum? 16. To what extent do teachers use a variety of assessment devices to measure improved race relations between students of different racial groups? 17. To what extent are students of all racial groups consistently exposed to and supported in the most rigorous curricular opportunities? 1 Almost Never 1 Almost Never 1 Almost Never 2 Seldom 2 Seldom 2 Seldom 3 Sometimes 3 Sometimes 3 Sometimes 4 Frequently 4 Frequently 4 Frequently 5 Almost Always 5 Almost Always 5 Almost Always

Community 18. To what extent do teachers have regular personal contact (e.g. phone, face to face) with Black and Hispanic families to advise them on the achievement of their children? 19. To what extent are Black and Hispanic children specifically invited to become a part of school-wide activities, committees, and councils? 20. To what extent are families of Black and Hispanic families visible in school and leadership positions? 1 Almost Never 2 Seldom 3 Sometimes 4 Frequently 5 Almost Always

1 Almost Never 1 Almost Never

2 Seldom 2 Seldom

3 Sometimes 3 Sometimes

4 Frequently 4 Frequently

5 Almost Always 5 Almost Always

School Culture 21. To what extent do you believe that Black and Hispanic students should perform at least as well as White students? 22. To what extent do you believe it is our responsibility as educators to make this level of achievement occur for our Black and Hispanic students? 23. To what extent do you believe that our Black and Hispanic students can actually reach this level of achievement? 1 Almost Never 1 Almost Never 1 Almost Never 2 Seldom 2 Seldom 2 Seldom 3 Sometimes 3 Sometimes 3 Sometimes 4 Frequently 4 Frequently 4 Frequently 5 Almost Always 5 Almost Always 5 Almost Always

Note: Information questions are not shown in Appendix A.

15