A Statistical

Analysis of Subjective and Objective

Evaluating Fabric Handle

Methods

of

Part

2: Relationships between Subjective and Objective Measurements By A. E. Steam*, R. L. D'Arcy*, R. Postle** * and T. J. Mahar* * *Members , TMSJ
Division

*CSIRO

of Textile Physics, 338 Blaxland Road,

Ryde, NSW

2112.

Australia

**University

of New South Wales, School of Textile Technology,

P. 0. Box 1, Kenington, NSW 2033. Australia

Based on thejournal of the Textile Machinery SocietyoJ,Japan, Transactions, Vol.38, No. 8, T76-T85 (1986-5)

Abstract

The data obtained from a surveyof the fabric handle preferences of panelsof judges drawn from the textile and clothing industries of Japan, Australia, India, NewZealand and the United States of America are compared with data provided from objective mechanical measurements. The model used takes into account the fact that judges may differ in their assessmentin non-random ways, for example, some judges may prefer thicker fabrics, and gives a picture of how objective measurements may perform not only to explain, but also to predict how individual judges perform. Although it is demonstrated that a substantial amount of individual sudjective handle assessmentcan be explained by mechanical test parameters, predictions of handle assessment by Australian judges compared with their actual assessmentswere generally poorer when based on objectivedata, rather than subjective data. The implication of this is that the set of mechanical measurements used in the analysis did not contain all the necessary information to predict fabric handle preference.
1. Introduction

In the first part1 of this series, it has been shown that there is structure in the handle preference assessments of a series of fabrics by a panel of judges. This implies that a single judge's handle assessment can be specified more reliably in terms of the weightings he attaches to specific fabric attributes, rather than some averaged assessment from the judging panel. Important ramifications for the commercial exploitation of handle assessments flow from this. The presence of variability in individual judge's weightings of specific fabric attributes reflects the variability of individual consumer's preferences1l1. Consequently, fabrics with a consistently uniform handle, conforming to some averaged requirement, may have less appeal in the market place than a selection of fabrics with a variety of handle attributes. Clearly, there is an advantage to be gained in the identification of the preferences, as well as,the variability in these preferences, so that selections of fabric types can be matched with the span of preferences existing in different markets. The third paper in this series will cover analysesapplied to summer weight men's suiting fabrics. In this case preliminary analysis1 has indicated stronger differences in the subjective handle assessments of the various national judging panels. A substantial amount of information about a fabric can be obtained from the 16 parameters describing fabric mechanical and surface properties derived from the KESVol. 34, No. 2(1988)

F set of instruments141. The problem exists of how best to determine the extent to which this objective data can explain the subjective handle assessments of fabric characteristics made by a judging panel. This paper applies the technique of canonical correlation, in which two banks of data from the same series of samples are examined with a view to isolating those factors which simultaneously account for the data in each bank. The technique thereafter helps to elucidate fabric attributers which are common to both sets of data. This canonical factor analysis has been applied firstly to establish whether any relationships exist between national preferences of the panels of judges and the fabric mechanical and surface parameters. Secondly, the analysis provides a model to predict the fabric handle assessments of both individual judges and panels of judges. These predicted assessments can be based on either fabric mechanical and surface test data or on the assessments of other judges. In the work reported here a small set of 39 fabrics has been used to assess the accuracy of the prediction models based on both the objective test data and the subjective assessments of the judges.
2. Canonical The examine explain canonical how another one Correlation correlation set set of is a statistical may on measurements method be used the to to same
39

of measurements

population's61It is of particular value when not only there are a large number of different measurements (in this case judgements or tests) but also when there is considerable intercorrelation between the scores within each set of measurements. The theory is to produce two sets of new variables from the original two sets of measurements. These new variables, which taken in pairs (one from each set) are the most highly correlated, but otherwise are uncorrelated. The correlations between each pair of the new variables are called the canonical correlations. Depending on the correlation among the original variables in each set, and also the ability of the two sets of data to explain each other, the number of new variables may be much smaller than the number of original variables. The new variables are formed by linear functions of the original variables, so in as much as they can demonstrate that something in one set of data explains something in the other set, are impossible to interpret by themselves. Two other sources of interpretation follow.
2.1 Structure The original new called second correlations variables, and to the variables the matrices between usually the first From rows the tabled as the S1 and new with variables the columns variables and the are to the see at a as the

3. Results

Initially the total hand evaluations (THVs) of 214 men's winter suiting fabrics by a 77 member international judging panel were used to form a data bank of subjectively assessed fabric attributes. The judging panel was comprised of 9Japanese, 18 Australian, 14 Indian, 13 New Zealand and 14 American judges drawn from the texile and clothing industries, together with 9 Australian consumers. The judges' assessments of THV were made on a scale ranging from a low of 0, an unsatisfactory rating, to a high of 5, an excellent rating''1. A second bank of data consisted of the 16 KES-F parameters, and their squares, for each of the 214 fabrics. Inclusion of the squared terms of the KES-F parameters allows for incorporation of local minima and maxima to improve the fit.
3.1 are most analysis. the levels values defined together Canonical of the in summarised influential In addition, for with the an correlation canonical Fig factor 1, which data of the correlation in each of are the analysis factor the extracted coefficient of the two data In Table statistical analysis of 16 by the correlation between banks, 1 an confidence shows bank canonical amount

Results variability

correlation

in the appropriate factors the factor indication

original S2. and

structure

matrices

by Bartlett~''

also shown.

S1 refers

set of data,

52 refers we can

set of data.

this information

glance how each test contributes to each new variable. If one sums the squares of the rows of the structure matrices, we can calculate the proportion of the variance included aspect of the orignal columns terpretation. ed term In order all variable. by the new tests explained chosen. The This total by the new variables gives rise to a second of one sum in the of inextractof each ranked than in new

variance is the

set of data squares are rather

variable

of the

in the columns included. the text the canonical correlations of magnitude, and the the transforms term `factor' is used

are chosen

so they are

positive

2.2 Prediction Canonical correlations establish a pattern by which two sets of data relate to each other. Using a large number of tests in the two banks of data helps to account for anomalies in scores through possible inhomogeneity of the population of fabrics.
Fig. 1

Table

Extract Australian

from the Judges

Structure Matrix for 77 judges, S1, for four based on the analysis summarised in Figure

selected 1.

40

Journal of the Textile Machinery Societyof Japan

extract from the structure matrix, S1, is shown for four selected Australian judges and the 8 highly significant factors. A. Additional Parameters A second canonical correlation was carried out with the bank of mechanical test data modified to contain 38 mechanical test parameters. The additional six parameters were obtained by replacing the warp-weft averaged results for the three surface parameters131 with three pairs of values which were the maximum and minimum individual values regardless of the direction in which they were measured. The results of this second canonical correlation analysis are summarised in Fig. 2.

Australian judges all of whom had been members of the original judging panel for the 241 fabrics used in the previous canonical correlation analysis. Five of the 12 Australian judges (Group A) had a correlattion of less than 0.80 with the mean assessment of the full judges' predictions for the THV's of 39 fabrics against the mean of the judges' actual assessments. The other seven judges (Group B) had a correlation of greater than 0.80 with the group mean. The predictions were given by an eight canonical factor reconstruction of the judges' assessment from mechanical test data (32 parameters). Table 2 shows the correlations between the judges handle assessments of the second set of 39 fabrics and the values predicted from the mechanical test results by using up to eight canonical factors. Figure 3 is a plot of the mean of all 12 Australian. The correlation coefficient for the relationship between predicted and assessed values was found to be 0.88. Parameters for the least squares linear regression line are an intercept of -0.70 and a slope of 1.02. An extract of the structure matrix, S1, of all seven Group B judges is presented in Table 3. In this case the nine highly significant factors shown in Fig. 2 have been

Fig. 2

3.2 Prediction A. Objective Basis This canonical correlation factor analysis provided a basis for calibrating individually each of the judges' assessments in terms of the mechanical test parameters. Consequently given the mechanical test data from another set of fabrics, one might expect to be able to predict how these judges would assess the fabric handle of this new set of fabrics. A new set of 39 fabrics was assessed by a panel of 12
Table 2 Coefficients assessments test results

for the correlation between Australian judges of 39 test fabrics and the prediction from 32 mechanical using a given number of Canonical Factors

' handle

Vol. 34, No. 2(1988)

41

Table

Structure matrix for seven Australian Group B judges with nine canonical factors. Entries are extracted from the results of a canonical correlation analysis between 77 judges, and 38 mechanical testing terms on 214 fabrics. The right hand column represents the percentage explanation of the judges' assessments (100. r2) by the 9 factors.

included. The values quoted in this Table were based on the second canonical correlation analysis in which the increased number [38] of mechanical test parameters was used. Consequently, the tabulated values for judges 3 and 7 may be compared with the values in Table 1. Table 4 shows the error structure for the seven Group B judges where their predicted actual handle assessments, calculated with up to nine canonical factors, are compared with their actual handle assessments. Both the Root Mean Square (RMS)value which estimates the total error and the standard deviation (SD) of the difference between the assessed and predicted values are tabulated in total hand value units. In the lower part of the table, the judges' mean assessments, their SD's and the mean predicted values are shown for each judge.
B. Subjective basis Predictions of the judges' total hand value assessments based on a battery of objective measurements were compared with the predictions based on the handle

assessments of another judging panel. The Group A and Group B judges were combined into a single 12 member panel. A second calibration was performed for these 12 judges whose handle assessments on the 214 fabrics were analysed by the canonical correlation procedures together with the 16 KESF parameters and their squared terms. Predictions of these judges' THV assessments for the second set of 39 fabrics were calculated from the mechanical test data for these 39 fabrics. The judging panel was divided into two banks by placing the odd numbered judges into one bank and the even numbered judges into the other. This division was assumed to produce two banks of reasonably balanced judging ability. The canonical correlation analysis was utilised again to enable the predictions of each panel to be calculated from either the measured mechanical properties of the fabrics or the subjective assessmentsof the alternative bank of judges. The results of these analyses are summarised in Tables 5 and 6.

Table

Differences assessments ons with

in THV Units) between individual Australian judges' (expressed of 39 test fabrics and their performance predicted by reconstructithe indicated number of canonical factors using an objective basis

42

Iour-nal of the Textile Machinery Societyof Japan

Table

Correlations two sets of

between actual assessments of THV values each predicted by a 2 canonical

and factor

model

Table

Correlations THV and canonical

between the 2 highly correlation

judges' actual assessments signifcant factors of a model

of

4.

Discussion

4.1 Canonical correlation factor analysis The canonical correlation factor analysis summarised by Fig. l demonstrates that the percentage of variance in the mechanical test parameters explained by each canonical factor falls off markedly after the first five factors. In the case of the judges' subjective assessments, there is a strong reduction in the amount of variance explained by each factor after the first two and another reduction after the fourth factor. It should be stressed however, that the amount of variance extracted by each factor may not be a reliable guide to its significance because a factor of apparently small influence can arise either from a widespread small effect, or from considerable disagreement between judges over very few of the 214 fabrics.
Vol. 34, No. 2(1988)

Tables of the complete structure matrices are too large to be included in this paper, however an inspection of these produced the following conclusions. The first factor accounts for the largest amount of the observed variability in both the judges' assessments (32% variance explained) and mechanical test parameters (16% variance explained). It appears to be fairly uniformly assessed by all groups of the international judging in a uniform manner and seems to strongly reflect the surface properties of the fabrics. This implies that the fabric surface properties are most important for the fabric handle assessments of most expert judges. The second factor, explaining the next greatest amount of variance, (11 % and 9% variance explained for the the judges' assessments and mechanical test parameters respectively) is strongly correlated with fabric weight (as
43

opposed to thickness). This factor appears to be recognized most clearly by the Australian judges who generally favour the lighter fabrics. The third factor, which correlates most highly with the tensile energy parameter accounts for approximately 3% of the judges' variability. This factor does not appear to be preferentially assessed by any particular nationality. However, the fourth factor, which is responsible for a further 2% of the judges' data variance, appears to be given a strong weighting by one New Zealand judge as well as some of the less consistent Australian expert judges and consumers. The mechanical test parameter most highly related to this factor is fabric thickness. Finally, the fifth factor relates to the parameter shear hysteresis at 5 of strain and appears to be assessed almost exclusively by Japanese judges who favoured the fabrics with the lower levels of this hysteresis. This finding is consistent with the stated~8' Japanese preference for low values of shear hysteresis in men's suiting materials. Factor five accounts for another 2 .5% of variability and brings the cumulative total explanation of the first five factors to approximately 50%. A. Additional parameters Eleven of the standard set of 16 KESF parameters used in the analysis represent the mean of the values of the parameters in the warp and weft directions. The correlations between the judges' fabric handle assessments and the individual warp and weft values of the three parameters which characterise fabric surface properties were examined in this study. There was very little consistency of the correlations between these surface parameters in the warp and weft directions. Consequently, the validity of using the mean of the two directional measurements is questionable. The mean value of each of the three surface parameters was replaced by a pair of values, one being the maximum and the other the minimum surface parameter value regardless of the direction in which it was measured. The effect of this change in fabric parameters can be gauged by comparing Figs. 1 and 2. Overall, there is very little difference between these two sets of results, but the additional parameters are responsible for an increase in the amount of variances in the subjective assessments accounted for by the first factor (32% compared with 37%) This increase has been made largely at the expense of the second factor whose percentage of explained variance has dropped from approximately 11% by some 5%. The increase in the amount of variability explained by the first factor when the individual maximum and minimum results replaced the warp-weft averages for the surfaces parameters would suggest that the judges assessed either the maximum, or minimum, of some surface property without regards to its direction, or that they were concerned with the ratio of the values of some parameter in the warp and weft directions.
4.2 Prediction

A. Objective basis a. Judging panel Despite the fact that it is possible to make predictions of the subjective THV assessments from objective mechanical data, such predictions are not perfect and
44

usually contain an appreciable amount of randorm variation or `noise'. Examination of the means of the 12 Australian judges' assessments and the means of their canonical factor predictions for the 39 test fabrics demonstrates the situation. There is a high degree of correlation (correlation coeff= .88) between the predicted and assessed values of THV, but there is still a large amount of scatter of individual points about the regression line (Fig. 3) The slope of the regression line in Fig. 3 is 1.02. This mere 2% difference from the ideal unity slope is hardly significant and suggests, that the predicted values are, in the absence of `noise', accurately predicting the response of the subjective assessments to changing fabric parameters. However, the ordinate intercept of -0.7 indicates that there is a bias in the predicted values which, on average, will underestimate the subjectively assessed THV by 0.7 THV units. Although there are far too few points to enable the accurate estimation of the kurtosis in the errors of the predicted values, the error distribution was estimated to be intermediate between a square and a triangular form. In an ideal case these errors would be normally distributed. The cause for the deviation from normality in this case is not readily evident. However, it could be due to restrictions imposed on the range of mechanical attributes of the 39 test fabrics. Thess fabrics were selected such that the national judging panel in Japan agreed amongst themselves on the fabric THV assessments whilst simultaneously, the results from the HESC standard methods objective handle assessment conformed with this agreed value~91. b. Individual judges The performance of each of the judges involved in assessing the 39 test fabrics was investigated on an individual basis. The fraction of the total variance of individual judge's assessments that is explained by the nine most significant canonical factors can be calculated by summing the squares of the correlation coefficients for each of the nine factors concerned. As noted in Table 3 the amount of each of the seven Group B judges' assessments to be explained in this way varies from a lowof 59.1%, for judge 6, to a high of 83.7% for judge 1. Examination of the differences between individual judge's assessments of the THV for 39 test fabrics, and the predictions of these assessments, made from calculations using up to nine canonical factors will enale the predictability of individual judges to be assessed. Table 4 presents the Root Mean Squared (RMS) values and the standard deviations for the differences between individual judge's actual assessments and predictions of each judge's performance made from differing numbers of canonical factors. Each judge's performance was predicted from the mechanical properties of the fabric by reconstructing his assessments using up to nine canonical factors. These nine canonical factors had been previously calibrated using fabric mechanical properties in a reconstruction of the data for the 214 calibration fabrics. In general, the use of more than one canonical factor to predict an individual judge's fabric assessments does very little to reduce the error of that prediction. But the higher Journal of the Textile Machinery Society ofJapan

number factors does markedly improve the mean of the predicted values form all the judges. For many of the predictions of individual judges it can be seen that the incorporation of additional factors actually increases the predictive error. The results in Table 4 suggest that judge 5 is the most predictable since the RMS values of the errors between his assessed and predicted THV's are the lowest (approximately 0.8-0.9 THV units). Similarly judge 1 with RMS errors in the vicinity of 1.5 THV units is the least predictable judge using this technique. B. Subjective basis The canonical correlation analysis was carried out with a large number of judges (77) although it was ultimately only required to calculate the predicted assessments of a small subset of those judges. The reason for this is that the more judges' data which are incorporated the better defined will be the basis on which the fabrics were subjectively assessed. Also the dominant factors will be established with greater accuracy and the random, or `noise' , element of the assessment will be more separable from the lesser defined factors. The prediction accuracy for the THV assessments of one panel of judges by a similar judging panel is reasonably good for 10 of the 12 Australian judges examined by this technique. As indicated in Table 5 the correlations between judges' actual THV assessments and the predictions based on the assessments for the other panel of judges ranged from 0.70 to 0.94 for 10 of the judges. The correlations for the remaining two judges, judges 11 and 12, were 0.33 and 0.18. The differences between judges 11 and 12 on the one hand and judges 1 to 10 on the other are further characterised in Table 6 which shows the correlations between judges' THV assessments and the two highly significant factors of the canonical correlation model. Tables 5 and 6 also show a comparison between THV predictions of a panel of judges based on the THV assessments of a similar panel of expert judges (subjectivebasis) on the one hand and based on a series of fabric mechanical test parameters (objective basis)on the other hand. Mostly the predictions based on the THV assessments of the panel of judges are an improvement on the predictions based on the mechanical tests in spite of a higher rendom element and a lower precision. The correlations between judges' actual THV assessments and predictions from the subjective basis are generally higher (10 of the 12)judges than similar correlation from the objective basis (2 of the 12judges). However, in only two cases,judges 1 and 6, are these differences in correlations significant at a 95% or higher level of confidence. Judge 12 is the only judge predicted significantly (90% confidence) better based on the mechanical tests. It would be reasonable to conclude then that this judge bases his opinions of fabric quality substantially on considerations not given by the rest of the predicting panel. However there is some consistency in his opinions which are based on mechanical considerations. Judge 11 appears to have even more dissimiliar characteristics to the main body of judges, but he has probably changed his assessment patterns since assessing the calibration set of 214 fabrics as his correlation for the second set of fabrics is scarcely significantly difVol. 34, No. 2 (1988)

ferent

from

zero.

5. Conclusion

In this paper two rather large statistical analyses have been briefly outlined. These analyses to some extent confirm the idea that people's preferences for fabrics can actually be predicted from a set of objective measures. However, it can be seen that the objective measurements as they stand are not sufficient to perdict individual's handle assessment, or even the mean of the assessments of Australian experts. Although an averaging of the Australian experts' assessments will enable the factors underlying judgement to explain a higher amount of this assessment, the limit of predictability is set by the correlations of these factors (the canonial correlations) with the factors derived from the mechanical tests. The current HESC test parameters do not predict assessments of one panel better than another judging panel. This implies that some other objective data has to be found which will complement the existing test data in order to provide firm accurate predictions. Another possible method of overcoming the incomplete explanation of International handle requirements for individual groups might be use a combination of the HESC tests and a panel of experts, as a basis of prediction. The first two papers in this series entitled, "A Statistical Analysis of Subjective and Objective Methods of Evaluating Fabric Handle" have been based on an extensive amount of data incorporating the fabric handle assessments of 214 men's suiting fabrics by 77judges from five countries and the testing of the same 214 fabricsby the KESF instrumentation. This range of mechanical and physical tests results in a series of 16 parameters which are used to characterise each of the fabrics tested. A second smaller set of both subjective and objective fabric assessments was performed on a range of 39 fabrics, covering 12 judges. These results were used to test the prediction accuracy of any system based on the larger set of 214 fabrics. It was primarily demonstrated (ref. pt. 1) that underlying patterns of fabric handle preferences existed, though these patterns were overlaid by national and individual judging preferences as well as random varaibility. It is possible to isolate groups of the judges whose assessments were best supported by those of their peers. Some national preferences of the judging panels were also noted. This paper shows that the underlying pattern of the internationally obtained fabric handle assessments isolated earlier can be predicted, at least in part, based on a series of mechanical and physical tests on the fabrics. The analysis has highlighted fabric surface properties as the most important single set the 16 KES-F parameters for the determiation of fabric THV, irrespective of which national penel of experts is assessing fabric handle. It has been further shown that incorporation of miximum/ minimum values of these surface parameters instead of the mean of the warp and weft values improves the prediction of THV when using the single major factor. However, the various combinations of fabric test parameters used were not able to predict the fabric handle
45

assessments of a panel of judges better than a similar judging panel drawn from within the same country. Some of the national characteristics of judging panels noticed in the analysis of the subjective assessments have been successfully related to various fabric mechanical and/or physical test parameters (eg. Japanese preference for low values of shear hysteresis at 5 shear strain, 2HG5). What has been established is a method by which a variable product can be varied to maximise a sales advantage. However much more work is required to determine how judges will change their opinions in time with the impact of fashion changes. References [1] A. E. Stern, R. L. D'Arcy, R. Postle, and T.J. Mahar;J. Text.Mach. Soc.Japan. 38, T157 (1985-8). [2] A. E. Steam, R. L. D'Arcy, R. Postle; "Analysis of Measurements of Fabric Handle and Mechanical Properties" In `Objective Evaluation of Apparel Fabrics', R. Postle, S. Kawabata and M. Niwa (eds.),pp. 281-290, The Texile Machinery Society of Japan, Osaka, 1983. [3] T. J.Mahar, and R.Postle; "Fabric Handle-AComparison of Australian and Japanese Assessments of

[4]

[5] [6] [7] [8]

[9]

Suiting Material", Australas. Text. 1982, 2, (1) 23-26 (1982). S. Kawabata, (ed.); `The Standardisation and Analysis of Hand Evaluation'. 2nd ed. The Hand Evaluation Standardisation Committee, The Textile Machinery Society of Japan, Osaka (1982). W. W. Cooley and P. R. Lohnes; "Multivariate Data Analysis" Ch. 6, Wiley, New York. (1971). T. W. Anderson; "An Introduction to Multivariate Statistical Analysis", Ch. 12, Wiley, New York, (1958). M.S. Bartlett;The Statistical Significance of Canonical Correlations, Biometrika, 32, 29-38. (1941). K. Ito; "The Use of Objective Measurement of Fabric Mechanical Properties for Process and Quality Control in an Apparel Manufacturing Factory". In `Objective Specification of Fabric Quality, Mechanical Properties and Performance' S. Kawabata, R. Postle and M. Niwa (eds.) pp. 331-337, The Textile Machinery Society of Japan, Osaka, (1982). S. Kawabata, (Chairman); `H. E. S. C. Standard of Hand Evaluation' Volume 3, 2nd ed. The Hand Evaluation and Standardisation Committee, The Textile Machinery Society of Japan, Osaka (1982).

ERRATA Following Sincere

n M!1

errata

have occured

inadvertently

in the advertisement

page of Vol. 34, No.

1.

apologies

are given to those concerned

and readers for the inconvenience

they have suf-

46

Journal o) f the Textile Machinery Societyof Japan