Measuring Model Fit

I. Fit refers to the ability of a model to reproduce the data (i.e., usually the variance-covariance matrix). It should be noted that a good-fitting model is not necessarily a valid model. There are now literally hundreds of measures of fit. oreover, a model all of whose parameters are !ero is of a "good-fitting" model. This page includes some of the ma#or ones, but does not pretend to include all the measures ($ollen and %ong - Testing structural equation models. &ewbury 'ar(, )*+ ,age, -../).

Chi Square: 2
For models with about 01 to 233 cases, this is a reasonable measure of fit. $ut for models with more cases, the chi s4uare is almost always statistically significant. )hi s4uare is also affected by the si!e of the correlations in the model+ the larger the correlations, the poorer the fit. For these reasons alternative measures of fit have been developed.

Chi Square to df Ratio: 2/df

There are no consistent standards for what is considered an acceptable model.

Bentler Bonett Index or Normed Fit Index (NFI)

5efine the null model as a model in which all of the correlations or covariances are !ero. The null model is referred to as the "Independence odel" in 2(Null Model) - 2(Proposed Model) ----------------------------------2(Null Model) * value between ..3 and ..1 is acceptable, and above ..1 is good. * disadvantage of this measure is that it cannot be smaller if more parameters are added to the model. Thus, the more parameters added to the model, the larger the index. It is for this reason that this measure is not used much anymore, but rather one of the next two is used.

Tucker e!i" Index or Non#normed Fit Index (NNFI)

* problem with the $entler-$onett index is that there is no penalty for adding parameters. The Tuc(er%ewis index does have such a penalty. %et 26df be the ratio of chi s4uare to its degrees of freedom 2/df(Null Model) - 2/df(Proposed Model) ----------------------------------------2/df(Null Model) - 1 If the index is greater than one, it is set at one. It is interpreted as the $entler-$onett index. &ote than for a given model, a lower chi s4uare to df ratio (as long as it is not less than one) implies a better fitting model.

Com$arati%e Fit Index (CFI)

This measure is directly based on the non-centrality measure. %et d 7 2 - df where df are the degrees of freedom of the model. The )omparative Fit Index e4uals d(Null Model) - d(Proposed Model) --------------------------------d(Null Model) If the index is greater than one, it is set at one and if less than !ero, it is set to !ero. It is interpreted as the previous indexes. If the )FI is less than one, then the )FI is always greater than the T%I. )FI pays a penalty of one for every parameter estimated.

Root &ean Square 'rror of ($$roximation (R&S'()

This measure is based on the non-centrality parameter. 8ood models have an 9 ,:* of .31 or less. odels whose 9 ,:* is .-3 or more have poor fit.

Standardi)ed Root &ean Square Re"idual (SR&R)

This measure is the standardi!ed difference between the observed covariance and predicted covariance. * value of !ero indicates perfect fit. This measure tends to be smaller as sample si!e increases and as the number of parameters in the model increases. * value less than .3; is considered a good fit.

(kaike Information Criterion ((IC)

This measure indicates a better fit when it is smaller. The measure is not standardi!ed and is not interpreted for a given model. For two models estimated from the same data set, the model with the smaller *I) is to be preferred. 2 < ((( - -) - 2df where k is the number of variables in the model and df is the degrees of freedom of the model. &ote that ((( - -) - 2df e4uals the number of free parammeters in the model. The *I) ma(es the researcher pay a penalty of two for every parameter that is estimated. The absolute value of *I) has relatively little meaning= rather the focus is on the relative si!e, the model with the smaller *I) being preferred.

*oelter Index
The index should only be computed if the chi s4uare is statistically significant. Its formula is+ (N - 1)2(crit) --------------- + 1 2 2 where N is the sample si!e, is the chi s4uare for the model and 2(crit) is the critical value for the chi s4uare. The index states the sample si!e at which chi s4uare would not be significant, i.e., that is how small one>s sample si!e would have to be for the result to be no longer significant. ?oelter recommends values of at least 233. @alues of less than 01 indicate very poor model fit.

II+ ,FI and (,FI ( ISR' mea"ure")

These measures are affected by sample si!e and can be large for models that are poorly specified. The current consensus is not to use these measures.

,oodne""#of#fit index- ,FI (ABres(og-,Brbom 8FI)+ 8FI 7 F %6FC, where FC is the fit function when all model parameters are !ero. 8FI varies from 3 to -, but theoretically can yield meaningless negative values. * large sample si!e pushes 8FI up. Though analogies are made to 9-s4uare, 8FI cannot be interpreted as percent of error explained by the model. 9ather it is the percent of observed covariances explained by the covariances implied by the model. That is, 92 in multiple regression deals with error variance whereas 8FI deals with error in reproducing the variance-covariance matrix. *s 8FI often runs high compared to other fit models, some suggest using ..1 as the cutoff. $y convention, 8FI should by e4ual to or greater than ..3 to accept the model. %I,9:% and * C, both compute 8FI. *lso, when degrees of freedom are large relative to sample si!e, 8FI is biased downward except when the number of parameters (p) is very large. Dnder these circumstances, ,teiger recommends an ad#usted 8FI (8FI-hat). 8FI-hat 7 p 6 (p < 2 E F-hat), where F-hat is the population estimate of the minimum value of the discrepancy function, F, computed as F-hat 7

(chis4uare - df) 6 (n - -), where df is degrees of freedom and n is sample si!e. 8FI-hat ad#usts 8FI upwards. *lso, 8FI tends to be larger as sample si!e increases= correspondingly, *8FI may underestimate fit for small sample si!es, according to $ollen (-..3). (d.u"ted /oodne""#of#fit index- (,FI. *8FI is a variant of 8FI which uses mean s4uares instead of total sums of s4uares in the numerator and denominator of - - 8FI. It, too, varies from 3 to -, but theoretically can yield meaningless negative values. *8FI F -.3 is associated with #ust-identified models and models with almost perfect fit. *8FI G 3 is associated with models with extremely poor fit, or based on small sample si!e. *8FI should also be at least ..3. %i(e 8FI, *8FI is also biased downward when degrees of freedom are large relative to sample si!e, except when the number of parameters is very large. %i(e 8FI, *8FI tends to be larger as sample si!e increases= correspondingly, *8FI may underestimate fit for small sample si!es, according to $ollen (-..3). %I,9:% and * C, both compute *8FI. *8FI>s use has been declining. Root mean "quare re"idual"- or R&S re"idual"- or R&SR- or R&R. The closer the 9 9 to 3 for a model being tested, the better the model fit. 9 , residuals are the coefficients which result from ta(ing the s4uare root of the mean of the s4uared residuals, which are the amounts by which the sample variances and covariances differ from the corresponding estimated variances and covariances, estimated on the assumption that your model is correct. Fitted residuals result from subtracting the sample covariance matrix from the fitted or estimated covariance matrix. %I,9:% computes 9 ,9. * C, does also, but calls it 9 9. Standardi)ed root mean "quare re"idual- "tandardi)ed R&R (SR&R): The smaller the standardi!ed 9 9, the better the model fit. ,9 9 is the average difference between the predicted and observed variances and covariances in the model, based on standardi!ed residuals. Standardized residuals are fitted residuals (see above) divided by the standard error of the residual (this assumes a large enough sample to assume stability of the standard error). ,9 9 is 3 when model fit is perfect. III+ ,oodne""#of#fit te"t" com$arin/ the /i%en model !ith an alternati%e model: This set of goodness of fit measures compare your model to the fit of another model. This is well and good if there is a second model. Hhen none is specified, statistical pac(ages usually default to comparing your model with the independence model, or even allow this as the only option. The independence model is the null model, which is the model in which variables are assumed to be uncorrelated with the dependent(s). ,ince the fit of the independence model is usually terrible, comparing your model to it will generally ma(e your model loo( good but may not serve your research purposes. The com$arati%e fit index- CFI: , is also (nown as the $entler )omparative Fit Index. )FI compares the existing model fit with a null model which assumes the latent variables in the model are uncorrelated (the "independence model"). That is, it compares the covariance matrix predicted by the model to the observed covariance matrix, and compares the null model (covariance matrix of 3>s) with the observed covariance matrix, to gauge the percent of lac( of fit which is accounted for by going from the null model to the researcher>s ,: model. &ote that to

the extent that the observed covariance matrix has entries approaching 3>s, there will be no non!ero correlation to explain and )FI loses its relevance. )FI is similar in meaning to &FI (see below) but penali!es for sample si!e. )FI varies from 3 to -. )FI close to - indicates a very good fit. )FI is also used in testing modifier variables (those which create a heteroscedastic relation between an independent and a dependent, such that the relationship varies by class of the modifier). $y convention, )FI should be e4ual to or greater than ..3 to accept the model, indicating that .3I of the covariation in the data can be reproduced by the given model. It is computed as (--max(chis4-df,3))6(max(chis4-df),(chis4n-dfn),3)), where chis4 and chis4n are model chi-s4uare for the given and null models, and df and dfn are the corresponding degrees of freedom. The Bentler#Bonnett Index- BBI: is the model chi-s4uare for the given model minus model chis4uare for the null model, this difference divided by model chi-s4uare for the null model. $$I should be greater than .. to consider fit good. The incremental fit index- IFI, also (nown as DELTA2+ $y convention, IFI should be e4ual to or greater than ..3 to accept the model. IFI can be greater than -.3 under certain circumstances. The normed fit index- NFI, also (nown as the Bentler-Bonett normed fit index, or simply DELTA . &FI was developed as an alternative to )FI, but one which did not re4uire ma(ing chis4uare assumptions. It varies from 3 to -, with - 7 perfect fit. &FI reflects the proportion by which the researcher>s model improves fit compared to the null model (random variables). For instance, &FI - .13 means the researcher>s model improves fit by 13I compared to the null model. $y convention, &FI values below ..3 indicate a need to respecify the model. ,ome authors have used the more liberal cutoff of .;3. &FI may underestimate fit for small samples, according to Dllman (233-). The non#normed fit index- NNFI, also called the Bentler-Bonett non-normed fit index, the Tucker-Le!is index" TL# (this is the label in * C,), or $%&2, &&FI is similar to &FI, but penali!es for model complexity. It is computed as (chis4n6dfn - chis46df)6(chis4n6dfn - -), where chis4 and chis4n are model chi-s4uare for the given and null models, and df and dfn are the associated degrees of freedom. &&FI is not guaranteed to vary from 3 to -. It is one of the fit indexes less affected by sample si!e. * negative &&FI indicates that the chis4uare6df ratio for the null model is less than the ratio for the given model, which might occur if one>s given model has very few degrees of freedom and correlations are low. &&FI close to - indicates a good fit. $y convention, &&FI values below ..3 indicate a need to respecify the model. ,ome authors have used the more liberal cutoff of .;3 since T%I tends to run lower than 8FI. ?owever, more recently, ?u and $entler (-...) have suggested &&FI F7 ..1 as the cutoff for a good model fit.