This article was downloaded by: [National Chengchi University] On: 21 May 2012, At: 19:50 Publisher: Taylor

& Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Technometrics
Publication details, including instructions for authors and subscription information: http://amstat.tandfonline.com/loi/utch20

Model-Robust Factorial Designs

William Li & Christopher J. Nachtsheim
a a b

Operations and Management Science Department, Carlson School of Management University of Minnesota, Minneapolis, MN, 55455 E-mail:
b

Operations and Management Science Department, Carlson School of Management University of Minnesota, Minneapolis, MN, 55455 E-mail: Available online: 12 Mar 2012

To cite this article: William Li & Christopher J. Nachtsheim (2000): Model-Robust Factorial Designs, Technometrics, 42:4, 345-352 To link to this article: http://dx.doi.org/10.1080/00401706.2000.10485707

PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://amstat.tandfonline.com/page/terms-and-conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

Model-Robust Factorial Designs

William LI and Christopher J. NACHTSHEIM Operations and Management Science Department Carlson School of Management University of Minnesota Minneapolis, MN 55455

(wli@csom.umn.ecfu) (cnachtsheim@csom.umn.edu)

Downloaded by [National Chengchi University] at 19:50 21 May 2012

In industrial experimentation, experimental designs are frequently constructed to estimate all main effects and a few prespecified interactions. The robust-product-design literature is replete with such examples. A major limitation of this approach is the requirement that the experimenter know which interactions are likely to be active in advance.In this article, we develop a class of balanced designs that can be used for estimation of main effects and any combination of up to CJ interactions, where 9 is specified by the user. We view this as an issue of model-robust design: We construct designs that are highly efficient for all models involving main effects and 9 (or fewer) interactions. We compare the performances of these designs with the standard alternatives from the class of maximum-resolution fractional factorial designs for several criteria. The comparison reveals that the new designs are surprisingly robust to model misspecification, something that is generally not true for maximum-resolution fractional factorial designs.This robustnesscomes at a price: The new designs are frequently not orthogonal. We demonstrate, however, that the loss of orthogonality is, in general, quite small. KEY WORDS: Estimation canacitv; Exchange algorithm; Information capacity; Model robust design; Optimal design.

A project conducted recently at a major automotive company was motivated by the need to reduce the leakage of a clutch slave cylinder. Four factors were identified to be potentially significant-body inner diameter, body outer diameter, seal inner diameter, and seal outer diameter. The budget allowed for eight runs in the experiment. The engineers believed that some interactions would be active. Although it was felt that the number of possible interactions was likely to be small (5 2), the engineers did not have prior knowledge on which interactions were likely to be significant. Thus, the set of possible models included all main effects plus 1 or 2 two-factor interactions. The task was to find a good eight-run factorial design that would perform well for all possible models. A standard approach to this problem would employ an orthogonal fractional factorial design, say, a 24P1 resolution IV design with a defining relation I = 1234. This design allows estimation of the four main effects as well as the confounded pairs-12 = 34,13 = 24,14 = 23. If it is known in advance that the only interactions likely to be present are, for example, those involving factor 1, this design might well be of interest. In the clutch experiment, the experimenters were seeking a design capable of estimating any pair of two-factor interactions. Because there are (4) = 6 two-factor interactions, the number of possible models that include all main effects and 2 two-factor interactions is (;) = 15. These models are listed in Table 1. If the 2$ design discussed previously is employed, only some of these models are estimable. A quick check reveals that models (51, (8), and (10) are not estimable. If we define the estimation capacity (EC) of a design to be the ratio of the number of estimable models to the number of possible models (following Sun 1993 and Cheng, Steinberg, and

Sun 1999), we find that EC = 12/15 = 80% for the proposed design. Because the estimation capacity is only 80%, a natural question arises: Is there a better design? In fact, many nontraditional factorial designs with higher estimation capacity values exist. For example, consider the design displayed in Figure 1 with n = 8, m = 4, and g = 2. For this design, EC = 100%. In this article, we develop a new class of modelrobust factorial designs (MRFDs). We propose this class of designs as an alternative to the now-standard use of maximum-resolution fractional factorial designs (FFDs). Use of resolution III and IV FFDs is typically justified by the sparsity-of-effects principle: Although many main effects and interactions are possible, it is expected that few are in fact active. An FFD is thus selected that permits estimation of all main effects and various combinations of interactions, as dictated by a particular confounding scheme. MRFDs also rely on the sparsity-of-effects principle, but in a different way. The experimenter is asked for an upper bound on the number of likely interactions. An MRFD is then selected that guarantees (if possible) that any combination of up to g interactions will be estimable. This eliminates the explicit need for choice of a confounding scheme. We will demonstrate that the proposed designs are remarkably robust in the sense that they are capable of estimating a wide range of empirical models. For example, consider a lo-factor experiment in which the experimenter wishes to estimate all main effects and up to 3

@ 2000 American Statistical Association and the American Society for Quality TECHNOMETRICS, NOVEMBER 2000, VOL. 42, NO. 4

345

346

WILLIAM LI AND CHRISTOPHER Table 1. A// Possible Models That Include All Main Effects and 2 Two-Factor Interactions of a 2,:- Design Model 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Main effects 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 interactions 12 12 12 12 12 13 13 13 13 14 14 14 23 23 24 13 14 23 24 34 14 23 24 34 23 24 34 24 34 34

J. NACHTSHEIM

for g = 3 is therefore 100%. In contrast, the estimation capacity of the corresponding FFD is just .5%. In general, the proposed designs are balanced, but not orthogonal. We will demonstrate, however, that the loss of orthogonality is minor. The outline for the remainder of the article is as follows. In Section 1, we cast this problem in the framework of model-robust factorial design and propose a general optimality criterion for such problems. This optimality criterion permits construction of MRFDs using numerical methods. In Section 2, we characterize the performance of MRFDs in terms of robustness, efficiency, orthogonality, and power. In particular, we contrast the performance of these designs to that of the class of maximum-resolution FFDs in some standard settings. In Section 3, we discuss numerical issues, and we give concluding remarks in Section 4. 1. CONSTRUCTING MRFDS In what follows, we consider only two-level experiments. Using fl and -1 to denote the levels of each factor, the
m=6,g=l

Downloaded by [National Chengchi University] at 19:50 21 May 2012

two-factor interactions using 16 runs. We have constructed an MRFD that is nonsingular for any of the 14,190 combinations of 3 two-factor interactions; its estimation capacity
m=4, g=3 1 2 3 4 5 6 7

n=8 m=5, g=2

a
n = 12 m=5, g=5 1 2 3 4 5 6 7 m=7.a=4 m=a,g=3

a
9 IO 11 12

n = 16

Figure 7. Selected Useful Model-Robust of interactions to be estimated. TECHNOMETRICS,

Factorial Designs for 8, 12, and 16 Runs. m is the number of factors, and g is the maximum number

NOVEMBER 2000, VOL. 42, NO. 4

MODEL-ROBUST

FACTORIAL DESIGNS

347

ith experimental run (or design point) can be characterized by an m vector xi, where XL = (zil~. , 2irn) and xij = rtl. The set of all possible design points is denoted by x. An n-point, exact design c(n) is given by any n points in x. We define the design matrix as XD = [xl: . . . ( x,,]. A response y obtained at x, is modeled in the usual way: y(xi) = f(x,)P+Ei, where p is an unknown p x 1 parameter vector, the errors pi are uncorrelated with mean 0 and constant variance g2, and the functional f indicates which main effects and interactions are present. The expanded design matrix or model matrix is given by X = [f (xl), . , f (xn)], where generally the columns of XD are a subset of those of X. Letting y = (y/(x1), . ,y(xn)) and E = (~1,. ,E,), the model can be written in matrix form as y = xp+.E. If the information matrix XX is nonsingular, the variance of the least squares estimator b = (XX) -X/y is given by c?{fi} = (X/X)- a2 . Many standard design criteria are based on this matrix. For example, an n-point exact design t(n) is orthogonal if XX is diagonal. Note that a design may be orthogonal for main effects but nonorthogonal for estimation of interactions. That is, X~XD can be diagonal when XX is not. A D-optimal design maximizes D(E(n)) = IXXI, w h ereas a G-optimal design minimizes the maximum standardized variance of the fitted value, denoted by d(x, t(n)) = f(x)(XX)P1f(x), for x E x. A criticism frequently associated with the use of optimality criteria such as D and G is the specific reliance on a known model f. One standard alternative is to assume instead that the true model is an unknown element of a finite set of known models F = (fi, . , fd), where d is the number of possible models. For example, Lauter (1974) proposed maximizing the average, taken on 3, of the log of the determinant. Cook and Nachtsheim (1982) considered the same problem for linear criteria of optimality, such as the trace or the integrated variance. Relatedly, Srivastava (1975) defined search designs as being those that allow estimation of all effects in one set and up to Ic elements in another set. DuMouchel and Jones (1994) considered primary and potential models in a Bayesian approach. Recent interest has focused on the construction of twolevel factorial experiments. In an important work, Sun (1993) established a framework for assessing the capability of factorial experimental designs for estimating various models with main effects and interactions. Sun identified model-robust designs selected among designs based on Hadamard matrices. We will call these designs Hadumardmatrix-based factorial designs (HMFDs). Note that a design t(n) can also be defined by m columns xci), where x(~) (i = 1, . , m) is an n x 1 vector corresponding to the ith variable (column). When optimal designs are selected from Hadamard matrices, all columns xci) (i = 1, , m>) are orthogonal to each other. Thus, the design space consists of all columns xci) = (x,~;. . , ZC~~~), where x,~ = &l, xiz) 1 = 0. and, for i # i, x;i)x(t) = 0. (2) (1)

Condition (1) enforces the balance (the equal number of runs being assigned to each level of a factor), and Condition (2) enforces the orthogonality of the design. Following Sun, in this article we focus on the construction of designs that allow the estimation of all main effects and g (or fewer) two-factor interactions. We will retain the balance requirement (1) in our discussion of model-robust factorial designs. We denote the set of such models by Fg. Note that the number of models in Fg is a! = (i), where t = m(nz - 1)/2 is the number of two-factor interactions. Let e,(<(n)) denote the efficiency of design E(n) when the true model fT = f,. A design E*(n) is model robust for F if
d

t*(n) = argmax C ulLez([(n))

i=l

(3)

Downloaded by [National Chengchi University] at 19:50 21 May 2012

where ef(<(n)) is the efficiency of the design [(n>) for model f,, ~1~2 0 is the weight assigned to model f,, and Ci uli = 1. Stated simply, a design E*(n) is model robust if it maximizes the average efficiency over the model space. The maximization is taken over the space of n-point exact designs. The efficiency, ei, will depend on the basic criterion under consideration. We consider two, ECg and D optimality. 1 .l EC, Optimality Cheng et al. (1999) defined estimation capacity in the context of minimum aberration designs. Consider a twolevel FFD with 71 runs. Let EC,([(n)) denote the fraction of models in Fg that are estimable for design c(n). A design is considered EC,-optimal if it maximizes this measure. It can be seen easily that this definition is a special case of (3) when w1 = w2 = ... = Wd = l/d and ei = 1.2 1 if f, is estimable 0 otherwise. (4)

D Optimality and Information Capacity

This criterion is motivated by the pioneering work of Lauter (1974) and, in the factorial design context, by the recent work of Sun (1993). The D efficiency of a design E(n) for model f, is given by (5) where <T(n) is the D-optimal design for model f,, 0%(<(n)) = ]XX] is the determinant of the information matrix for design E(n) and model f,, and it is assumed that a nonsingular optimal design <,*(n ) exists for each f, E F. We refer to the criterion defined by (3) and (5) as D optimality. Because all the models used in this article consider main effects plus a certain number of two-factor interactions, we will use D,q to denote D optimality for a model space composed of all models consisting of all main effects plus g two-factor interactions. As with EC, optimality, we will generally employ equal weight. If certain models are viewed as more important or more likely, the weight can be suitably adjusted.
TECHNOMETRICS, NOVEMBER 2000, VOL. 42, NO. 4

348

WILLIAM LI AND CHRISTOPHER J. NACHTSHEIM Table 2. Comparisons of ECS and /C, (in parentheses) Criterion Values for FFDs, and MRFDs for n = 8

m 4

Designs FFD MRFD d= FFD MRFD d= FFD MRFD d=

EC1 WY) 1.000 (1.000) 1.ooo (1.000) 6 .400 (.400) 1.000 (.822) IO ,200 (.200) 1.000 (.758) 15

EC2 W2) ,800 (.800) 1.000 (.758) 15 ,089 (.089) ,733 (.508) 45

EC3

W3)

,400 (.400) ,850 (608) 20

ondary criterion to distinguish among designs that have the same estimation capacity. We turn now to an assessment of the properties of the MRFDs constructed with this algorithm. 2. ASSESSING MRFD PERFORMANCE In this section we compare the performance of MRFDs to that of the standard alternative-maximum-resolution FFDs-and to that of the class of HMFDs developed by Sun (1993). We compare these three classes of designs on the basis of four criteria-estimation capacity, information capacity, orthogonality, and power for model discrimination. Focusing on designs for 8, 12, and 16 runs, we show in general that the use of MRFDs leads to substantial improvements in robustness and efficiency, with little loss in orthogonality and power. Estimation Capacity. Tables 2, 3, and 4 provide EC, and IC, values for n = 812, and 16; g 5 5; and m < 10. Because FFDs do not exist for n = 12, MRFDs are compared only with HMFDs in this case. We note first that the MRFDs are best in terms of EC in all instances. Consider, for instance, a model-robust 16 x 9 design involving all main effects and 2 two-factor interactions (i.e., g = 2). Here, the number of models, d, in & is (z) = 630. The MRFD has 100% EC, which means that all 630 models are estimable. In comparison, the EC of the corresponding FFD is .381, which implies that 61.9% or 390 models are unestimable. As clearly depicted by the tables, the advantages resulting from the use of the MRFDs tend to increase with either rn, or g. For example, consider the case in Table 4 for which n = 16, m = 9, and g = 5. The EC for Suns HMFD is given by ECF, = .225. In this case, the EC for the MRFD is .987. Note that the number of models in the model space & is 376,992; nearly all (98.7%) are estimable. In contrast, for the maximum-resolution FFDin this case the 2&y-just 1.6% of the 376,992 possible models are estimable. In general, the MRFDs are remarkably robust. For the cases reported, the worst-case estimation capacities are 73.3% for n = 888.7% for n = 12, and 98.7% for n = 16.
Criterion Values for HMFDs and MFRDS for n = 12
EC3 EC4 EC5

Downloaded by [National Chengchi University] at 19:50 21 May 2012

D optimality is closely related to the 5 optimality criterion of Lauter (1974) and the infamtation capacity (IC) criterion of Sun (1993). The IC criterion IC, is defined by (3) with equal weights and ei = Oz([(n))/</n. We note that IC, is nearly identical to the D criterion in this application. We found that, for the simple factorial models, two-level designs, and sample sizes considered here, the optimal designs are frequently orthogonal, or nearly so, having Di(<*(n)) 73 npa. Use of this value in (5) leads to the IC, criterion while reducing the computational burden. For this reason, we view IC, and o9 as essentially synonymous in what follows. A related criterion in the context of supersaturated designs was proposed by Wu (1993). Suppose that there are f active factors (i.e., factors with significant effects) among all m factors. Let F(f) be an n x f submatrix of the full expanded design matrix F, and define Df = CFcf) I(l/n)F(f,F~f,Ilfj(~). where the summation is over all possible submatrices Fcf) for fixed f. A Dfoptimal design maximizes the Df value over all possible designs. In this article, designs are selected based on a compound criterion involving both EC, optimality and IC, (Dg) optimality. Details of the exchange algorithm that we have developed for this purpose are presented in Section 3. In essence, the algorithm seeks designs that maximize the EC, criterion; the IC, criterion is employed as a secTab/e 3. Comparisons of ECS and /C, (in parentheses) m 5 Designs HMFD MRFD d= HMFD MRFD d= HMFD MRFD d= HMFD MRFD d= HMFD MRFD d=
=I EC2

1.000 (.944) 1.000 (.944) 10 1.ooo (.929) 1.ooo (.929) 15 1.000 (.914) 1.000 (.914) 21 1.000 (.896) 1.000 (.855) 28 1.OOO(.872) 1.OOO(.809) 36

1.000 (.891) 1.000 (.920) 45 1.000 (.861) 1.000 (.907) 105 ,914 (.765) 1.000 (.785) 210 ,778 (.630) ,995 (.681) 378 ,514 (.400) ,985 (.597) 630

1.000 (.838) 1.000 (.863) 120 1.000 (.791) 1.000 (.798) 455 ,750 (.570) ,992 (.677) 1330 .409 (.298) .922 (.500) 3,276

,952 (.751) ,995 (.785) 210 ,945 (.679) ,983 (.708) 1365 .449 (.306) ,887 (.520) 5,985

.762 (.569) .976 (.716) 252 ,689 (.444) ,933 (.672) 3,003

TECHNOMETRICS,

NOVEMBER 2000, VOL. 42, NO. 4

MODEL-ROBUST Table 4. Comparisons of ECS and IC, (in parentheses) m 7 Designs FFD HMFD MRFD d= FFD HMFD MRFD d= FFD HMFD MRFD d= FFD HMFD MRFD d= EC1 1.000 (1.000) 1.000 (.926) 1.000 (.959) 21 1.000 (1.000) 1.ooo (.955) 1.000 (.945) 28 ,667 (.667) 1.000 (.916) 1.000 (.902) 36 ,467 (.467) ,933 (.853) 1.000 (.865) 45 ECz

FACTORIAL DESIGNS Criterion Values for FFDs, HMFDs, and MFRDs for n = 16
EC3 EC4 EC5

349

,900 (.900) 1.000 (.858) 1.000 (.911) 210 ,889 (.889) ,992 (.901) 1.000 (.881) 378 ,381 (.381) ,871 (.745) 1.000 (.824) 630 ,178 (.178) .788 (.661) 1.OOO(798) 990

,711 (.711) 1.000 (.795) 1.000 (865j 1,330 .684 (.684) .960 (.825) 1.000 (.798) 3,276 ,179 (.179) ,661 (.533) I .ooo (.75oj 7,140 ,005 (.005) ,553 (.430) 1.000 (654) 14,190

,474 (.474) ,982 (.723) 1.000 (762j 5,985 ,438 (.438) .880 (.713) .999 (.751) 20,475 ,065 (.065) ,431 (.328) .999 (667j 58,905

,251 (.251) ,912 c.619) ,998 (716j 20,349 .219 (.219) ,717 (.546) ,999 (667j 98,280 ,016 (.016) ,225 (.161) ,987 (591 j 376,992

IO

Downloaded by [National Chengchi University] at 19:50 21 May 2012

Irzformation Capacity. We first note that for any FFD ,!$;),,F determinant Pi(<(n)) as given following (5) is P% 0, depending on whether or not the model is or estimable. Therefore, IC, = EC, for these designs. This property does not hold for HMFDs or MRFDs. As noted previously, the X matrices for HMFDs are typically not orthogonal. In general, the pattern observed in Tables 24 for estimation capacities holds true for IC, criterion values. First, the information capacity of the MRFD in nearly every case exceeds that of the corresponding HMFD or FFD. Second, the advantages associated with MRFDs again tend to increase with m and g. For example, for n = 8, rn = 4, and g = 2, the IC of the FFD is .80; for the MRFD it is .758. Nevertheless, the EC of the latter is 100% compared with 80% of the former. In contrast, for n = 16,m = 10, and g = 1, the IC of the FFD t.467) is about half that of the MRFD (.865). The difference can again be substantial. For n = 16,~~ = 10, and 9 = 3, the IC of the FFD is .005; for the MRFD, the IC is ..654. Orthogonal&y. As noted in the introduction, the large gains in EC and IC associated with MRFDs come at a cost: MRFDs are frequently not orthogonal. Table 5 was constructed in an effort to characterize the extent of this loss. In the table we have reported the averages of the absolute values of the correlations between effects, for all models,
Table 5. Orthogonality Check on HMFDs and MRFDs: Average of the Absolute Values of Correlations Between Effects Designs 125 12*6 12*7 12*8 129 Criteria HMFD MRFD HMFD MRFD HMFD MRFD HMFD MRFD HMFD MRFD g= ,063 .067 .063 .063 .060 ,060 .056 ,076 ,198 ,094 1 g=2 .I01 .088 ,100 ,102 ,097 ,140 ,093 ,159 ,195 .I35 g=3 .I 19 .lOO .I23 ,120 ,122 ,171 .I19 .I 82 g=4 .130 .I22 ,138 ,122 ,139 .I63 g=5 ,136 ,136 ,147 ,163

focusing for brevity on the intermediate case, n = 12. We compare the average absolute pairwise correlations for the MRFDs with those of HMFDs. For FFDs, the absolute correlation between estimated effects is either 1 or 0, depending on whether or not the effects are aliased. For this reason we omitted discussion of average absolute correlations in this case. Recall that Sun restricted his search to designs whose main-effects settings are based on Hadamard (orthogonal) matrices. No such restriction was imposed in the construction of MRFDs. Yet, as shown in Table 5, the average absolute correlations for the MRFDs are nearly indistinguishable from those of HMFDs. Specifically, for the 19 cases reported in Table 5, the average absolute correlation for the MRFD is greater than that for HMFDs in eight instances, it is less in seven instances, and these values were the same in four cases. As can be seen, the worst-case average absolute correlation for an MRFD is .182, as repcrted for m = 8 and g = 3. In a separate study, we also compared condition numbers (which are defined to be the square root of the ratio of the largest eigenvalue to the smallest eigenvalue of XX) of MRFDs to HMFDs for n = 12 or 16. In 12 out of 14 cases we considered, the MRFDs have substantially smaller values of average condition numbers (over all possible models for each design). Because a smaller condition number implies a higher level of orthogonality, the MRFDs perform surprisingly well with respect to this criterion. Powel: Because the true model (i.e., the set of active main effects and interactions) is unknown a priori, the experimenter will frequently test for the presence of specific model terms at the analysis stage. One measure of the efficacy of any design is, therefore, the power that it provides for testing for the presence of specific effects. Although many testing scenarios might be considered, we focus here on the simplest case and ask the following: For a given design, what is the average power for testing for presence of a specific interaction term? Let ,&j denote the regression coefficient corresponding to the interaction term zixj. Recall that, for any model in Fg, there are g two-factor interaction terms. We have computed
TECHNOMETRICS, NOVEMBER 2000, VOL. 42, NO. 4

NOTE.

All models consist of all main effects and 9 two-factor

interactions.

350

WILLIAM LI AND CHRISTOPHER J. NACHTSHEIM

Downloaded by [National Chengchi University] at 19:50 21 May 2012

the power of the test, Ha : l-l<, = 0 versus HI : ,til,? # 0, for each of the g two-factor interactions in each of the possible models. The average and minimum of these g$ power calculations are reported for each design in Table 6. We calculated the probability of rejecting Ho for Pi, = 20, and pZj = Z-rlj, where cry = u/fi is the standard error of any regression coellicient in an FFD. Details of the power calculation are provided in Appendix A. Table 6 shows that the MRFDs again perform quite favorably when compared to FFDs and HMFDs. Consider first the case in which n = 8. For Di, = 20g, the average powers for the FFD and the MRFD are .102 and .107, respectively; for pij = 5au, the numbers are .244 and ,253. As one might expect, power levels are quite low because there is only 1 df for residual error. Clearly, for n = 8, the FFD and MRFD perform comparably in terms of average power. The performance of the MRFDs is also comparable to the HMFDs for rl = 12, and all three classes of designs perform similarly for n = 16 when m = 7 or m = 8. Differences emerge for the final two designs for which n = 16 and m = 9 or 10. Here, the average power for the MRFDs dominatesFor example, for n = 16, m, = 9, and /3ij = 5~~7, the average power for the MRFD is ,676; it is .364 and ,553 for the FFD and HMFD, respectively. We also considered the minimum power in an effort to characterize the worst-case scenarios for those designs and to provide some indication as to the extent of variation about the mean. With the exception of the case in which n, = 16 and m = 7, the worst-case power is generally maximized by the MRFD. Admittedly, these worst-case powers for the MRFD are generally small; however, those associated with the FFD or the HMFD are frequently 0. For example, when n = 8: m = 4, and pL7 = 500, the minimum power for the MRFD is ,219; for the FFD, the power is 0. The anomaly for the case n = 16, rn = 7, and & = 50~ is of interest. For example, the minimum powers for the
Table 6. Average Power of the Test Ho : fi,, = 0 Versus HI : p,, = ku/Jii, fork = 2 and 5 and CY= .05 Average power Design 8*4 12*5 12*6 127 128 16*7 168 16*9 16*10 k 2 5 2 5 2 5 2 5 2 5 2 5 2 5 2 5 2 5 FFD ,102 ,244 ,226 ,814 ,167 .633 .I 08 .368 ,061 .I33 ,203 ,781 ,245 .803 .I54 ,553 .I 15 ,408 HMFD MRFD ,107 .253 ,293 .904 .247 ,823 ,137 ,453 ,076 .I59 ,294 ,906 ,244 ,836 ,181 ,676 ,139 .510 Minimum power FFD ,000 ,000 .I59 ,631 ,138 ,529 .ooo .ooo ,000 ,000 ,181 .725 ,000 ,000 ,000 ,000 .ooo ,000 HMFD MRFD ,095 .219 .181 .706 .I60 ,614 ,053 ,068 .ooo .ooo ,081 ,252 ,063 ,134 ,061 .I20 ,053 ,070
tnter-

HMFD and MRFD are ,725 and .252, respectively. An explanation for this exception has not been found. 3. ITERATIVE METHODS

,355 ,886 .329 ,867 ,129 ,364 ,052 .I60

.ooo ,000 ,000 ,000 .ooo ,000 ,000 ,000

3.1 A Restricted CP Algorithm We now consider iterative procedures for solving optimization problem (3). Many algorithms have been proposed for constructing optimal exact designs in the literature. We begin with the columnwise-pairwise (CP) algorithm developed by Li and Wu (1997) because of its speed and its ability to guarantee the balance of a resulting design. We refer to this algorithm as the general CP algorithm. The algorithm starts with a randomly generated balanced factorial design XD = (x1) . ~x,). The design is then improved iteratively. Each iteration consists of m steps. At the ith step, the column xi is chosen for exchange. The best swap of a pair of elements within this column (in terms of the optimality value) is performed. The best exchange is that which leads to the largest increase in EC values. In the event that two or more exchanges tie for the EC,, the largest increase in IC, values is used to break ties. The algorithm terminates when no further improvement can be made. This algorithm is effective in constructing EC,-optimal designs. Its application is limited for the optimization problems considered here without modification, however, because of the time required for convergence for useful values of 17, and g. For example, the construction of a 16 x 8 MRFD for 9 = 2 (d = 378) took 15,863 seconds on a SUN Spare 20 workstation when the number of trials is 20. Thus, further modifications are needed. In an exchange algorithm, when a column (or a row in a rowwise algorithm) is chosen for deletion, the vector from a set of candidate vectors that results in the maximum gain in the criterion is selected as a replacement. Consider an 71run balanced vector v that has an equal number of +s and --)s. The total number of candidate vectors is (;,2). This number is usually too large for large n. Li and Wu (1997) proposed to restrict the search to the lst-order adjustment vector of v, which is obtained by switching one pair of + and - in v. This pairwise procedure has also been used by other authors (e.g., see Morris and Mitchell 1995) and has proven efficient for many types of problems. The computational intensity of model-robust factorial designs brings in the need to further reduce the number of vectors in the candidate set. One such way is to restrict the search only to pairs that include a particular element of the vector v. For illustration, consider a vector of six elements: v = (+: -. +, +: -, -). Suppose that this vector is chosen for the pairwise exchange. Then all the possible exchanges include a swap of one pair of + and -. Thus, there are nine 1St-order adjustment vectors: (-. +. +. +, -. -). (-. -. +.+, +: -). (-: -. +.+. -. +). (+. +. -, +. -. -). (+. -. -. f, +. -). (+. -. -. +. -. +)+ (+. +. +, -. -. -), (+. -. +. -. f: -). (+, -, f. -> -: +).

NOTE: The average IS taken wer all models cons~.t~~g of maw effects and 2 two-factor actions.

TECHNOMETRICS,

NOVEMBER 2000, VOL. 42, NO. 4

MODEL-ROBUST

FACTORIAL DESIGNS

351

This set of vectors is called the Ist-order adjustment vector set of v. The first three vectors are obtained by a swap involving the first + in the original vector v. Similarly, the next three vectors and the final three vectors are obtained by swaps involving the second and the third + in v, respectively. We call each group of three vectors a restricted lst-order adjustment vector set with respect to one of the positive coordinates of v. The restricted set with respect to any negative coordinate element can be similarly defined. We now incorporate the idea of restricted pairs to the CP algorithm. Suppose that in the exchange procedure each exchange on a column is restricted to a particular restricted lst-order adjustment vector set. (Note that the number of such exchanges is reduced further from (7) to n/2.) The resulting exchange procedure, which we refer to as the restricted CP algorithm, is described in Appendix B. 3.2 Performance We now more thoroughly explore the efficacy of the restricted CP algorithm. Although the algorithm is applicable to a general class of factorial designs, we consider the construction of 12 x k and 16 x k designs because those designs were discussed by Sun (1993); thus a benchmark is available. For the purpose of comparison, we construct MRFDs for g = 2 using both exchange procedures, the general CP and the restricted CP algorithms. In each test example, 20 starting designs are randomly selected. The best design among the 20 resulting designs is chosen to be the optimal design. In Table 7, for each test problem, we report the IC, criterion values and required execution times for the general CP algorithm (I&,, tgen) and for the restricted CP algorithm (IC reS,t,,,). We also report the relative efficiency (T,) and the relative execution time (rt) of the final design. Specifically, the relative measures are defined by T, = IC,,,/IC,,,, and rt = tres/tgen. The IC, criterion values for the HMFDs (ICubl~p) are reproduced in the table for reference. As shown in Table 7, the performances of the two algorithms are essentially the same based on IC efficiencies. The average efficiency of the restricted CP algorithm ranged from .994 to 1.OO.In six out of nine test cases, the restricted algorithm produced optimal designs with the same IC efficiency as those generated by the general CP algorithm. Execution time required by the former, however, is typically 15-20s of the total time required by the latter. Note that we did not report EC values because both algorithms
Table 7. Relative Performance of the General and Restricted CP Algorithms n 12 12 12 12 12 16 16 16 16 mg 5 6 7 8 9 6 7 8 9 2 2 2 2 2 2 2 2 2 d 45 105 210 378 630 105 210 378 630
ICHMFD l&en &s tgen tres re ri

generated designs with the same EC value for each case. Our conclusion is that the restricted CP algorithm produces designs that are essentially the same as those produced by the general CP algorithm, with substantial reductions in required computing time. 4. DISCUSSIONS AND EXTENSIONS In this article, we have introduced the class of modelrobust, two-level factorial designs. In doing so, we have focused on the construction of small factorial designs that permit the estimation of main effects and, if possible, any y two-factor interactions. We recommend use of these designs as an alternative to now-standard use of resolution III and IV orthogonal FFDs when an experimenter is concerned about the presence of a few unknown interactions. The MRFDs proposed here, like resolution III and IV orthogonal designs, are justified by the sparsity-of-effects principle. If many two-factor and/or higher-way interactions are likely, full factorial or fractional factorial designs of resolution V or higher are appropriate. Although we have focused on the estimation of interactions, the methodology is more generally applicable. For example, it is easily seen that supersaturated designs, as described by Li and Wu (1997), Wu ( 1993), and Lin ( 1993) are in fact MRFDs. Consider the case of a supersaturated design in the presence of rn factors. The experimenter expects only a subset (say q) of the m main effects to be active; the specific subset is unknown. Here there are d = (7) potential models, and our methodology is applicable without modification. We have focused on the performance of MRFDs for 7~ = 8, n = 12, and n = 16. We did this for two reasons. First, designs of size 8, 12, and 16 are frequently employed, and second, use of these sample sizes permitted comparison with standard orthogonal-array-based designs. In practice, there is nothing to prevent the construction of MRFDs for any sample size. A JAVA-based implementation of the MRFD algorithm is available through the Web site www.csom.umn.edu/wwwpages/facultyy/wli. The designs in Figure 1 are also posted at that site. Finally, during the review process, one reviewer inquired as to the robustness of these designs to the specification of g. Specifically, we were asked to characterize the efficiency of the MRFD constructed for g interactions when fewer than g (say g < g) interactions are actually present. For the range of designs considered here, our investigation suggests that the MRFDs are remarkably robust to the specification of g. This conclusion derives from two analyses. We first constructed the MRFD for a specific value of g, say g = go, and compared the resulting design to that obtained for g < go. Note that our methodology handles this latter case directly: The relevant model space is 3 9190 = U srgo3,. We found that the MRFD for &syo is frequently similar or identical to that for Fg. Because computing time is substantially less for the smaller model space Tq, we recommend the use of Fg rather than .?=qlso. We also examined the efficiency of the MRFD constructed for Fg when exactly g < g interactions result. For example, Table 8 provides the relative efficiencies for the I&TECHNOMETRICS, NOVEMBER 2000, VOL. 42, NO. 4

Downloaded by [National Chengchi University] at 19:50 21 May 2012

.89 .86 .77 .67 .44 .94 .90 .90 .75

.92 .91 .81 .74 .64 .95 .93 .90 .87

.92 .91 .81 .74 .64 .95 .92 .90 .85

261 820 2,238 6,236 16,976 2,479 6,640 17,466 38,805

41 155 453 1,241 2,801 357 996 2,550 5,782

1.000 1.000 1.000 1.000 ,997 1.000 ,994 1.000 .975

.I57 .I89 ,202 .I99 ,165 ,144 .I50 .I46 ,149

352 Tab/e 8.

WILLIAM LI AND CHRISTOPHER J. NACHTSHEIM Helative Efficiency of the K-Optimal Design in Comparison With the Kg-Optimal Design for g< 4 Design l2*5 l2*6 I 27 167 16*8 16*9 g= .948 ,954 .896 ,907 ,925 ,909 1 g=2 ,942 .934 ,969 .92 ,952 ,942 g=3 ,963 .992 1.000 .927 1.000 ,967

Downloaded by [National Chengchi University] at 19:50 21 May 2012

optimal design compared to IC,f-optimal designs for known g < 4. As is evident from the table, these relative efficiencies are always greater than about 90%. This result is not new to the design literature. Cook and Nachtsheim (1982) discussed1 the robustness of the optimal design based on the highest-degree polynomial in 3, when 3 is a space of nested polynomials. Such designs are very robust. Highest degree, in the Cook-Nachtsheim application, is directly analogous to g, the largest number of potential interactions. If the actual number of active interactions exceeds g, the MRFDs may or may not be robust. But this limitation is equally associated with FFDs and HMFDs. ACKNOWLEDGMENTS This research was supported by the Research and Teaching Supplements system in Carlson School of Management at the University of Minnesota. We thank the editor, the associate editor, and the referees for insightful comments that led to substantive improvements to our article. We also thank Lexin Li for JAVA programming assistance. This work was partially supported by the IT Statistical Consulting Department, 3M. APPENDIX A: POWER CALCULATIONS Consider the model E(y) = X,6. Partition the matrix X and the vector fl so that X = (Xl, X2), where X1 has dimension rj>x ~1 and X2 has dimension n x 7-z: /3 = (pi, ,&), where p1 and ,& have dimensions rl x 1 and r2 x 1, respectively. For the hypothesis Ho : Pa = 0, the noncentrality parameter is given (Graybill 1961, p. 140) by X = &[X&X, - X;X,(X;XJ1X,X,]p2/(2a2). (A.l)

Step 2. Set j = 0. Step 3. j = j + 1. Randomly choose an element .z+ in xj. Consider all swaps of x:,~ and x,I~. Find i* such that an exchange of + and zie3 results in a maximum increase in EC,. If there are ties, choose the one that maximizes IC, among the ties. Step 4. Repeat Step 3 until j = m + 1. If the new design matrix after m exchanges makes no improvement, stop. Otherwise, repeat Steps 24. Step 5. Record the optimal design. Repeat Steps 14 for several tries. Choose the best among those resulting designs. Note: When EC, or IC, is computed, the following updating formula is used: Suppose X = (Xl. X2), where X1 consists of all main effects and the intercept term, and X2 consists of g two-factor interactions. Then we employ the standard updating formula: IXXI = Ix;x4x;x2 - x;x1(x;xJx;x2I.

Because for every possible model, X1 is unchanged, use of this equation avoids recomputation IXX( from scratch. Use of the standard rank-l or rank-2 updating formula (e.g., see Cook and Nachtsheim 1980) offers little practical advantage here because of the many dispersion matrices (XX)- that must be maintained.
[Received February 1999. Revised March 2000.]

REFERENCES
Cheng, C.-S., Steinberg, D. M., and Sun, D. X. (1999), Minimum Aberration and Model Robustnessfor Two-Level Fractional Factorial Designs, Journal of the Royal Statistical Society, Ser. B, 61, part 1, 85-93. Cook, R. D., and Nachtsheim, C. J. (19801,A Comparison of Algorithms for Constructing Exact D-Optimal Designs, Technometrics, 22, 3 15324.

__

(1982), Model Robust, Linear-Optimal Designs, Technomerrics,

24, 49-54.

DuMouchel, W., and Jones, B. (19941, A Simple Bayesian Modification of D-Optimal Designs to Reduce Dependenceon an Assumed Model,
Technomerrics, 36, 3141.

Graybill, F. A. (1961), An Introduction lo Linear Srarisrical Models (Vol. l), New York: McGraw-Hill. Lauter, E. (1974), Experimental Planning in a Class of Models, Marhemat&he Operationsforschung und Statistik, 5, 673-108.

Let F, denote the lOO(1 - a) percentile of the central F distribution having the corresponding dfN and dfn degrees of freedom. Then the power of the test is given by P(X) = 1 - Gx(Ect), t-4.2)

Li, W., and Wu, C. F. J. (1997), Columnwise-Pairwise Algorithms With Applications to the Construction of SupersaturatedDesigns, Technometrics, 39, 171-179.

Lin, D. K. J. (1993), A New Class of SupersaturatedDesigns, Terhnometrics, 35, 28-31. Morris, M. D., and Mitchell, T. J. (1995), Exploratory Designs for Computational Experiments, Journal qf Statistical Planning and Inference,
43, 38 l-402.

where GA(.) is the cumulative distribution function of the noncentral F distribution having dfN and dfD degrees of freedom, respectively, and noncentrality parameter X. In our computations, we used (A.l) to compute the noncentrality parameter. We then computed the power of the test given by (A.2) using subroutines by Turiel (1989). APPENDIX B: THE RESTRICTED CP ALGORITHM

Srivastava,J. N. (1975), Designs for Searching Non-negligible Effects, in A Survey of Statistical Designs und Linear Models. ed. J. N. Srivastava, Amsterdam: North-Holland, pp. 507-519. Sun, D. X. (19931,Estimation Capacity and Related Topics in Experimental Designs, unpublished Ph.D. dissertation, University of Waterloo, Dept. of Statistics and Actuarial Science. Turiel, T. P. (1989), A FORTRAN Program to Determine Sample Sizes and Generate Power Curves for Completely Randomized and Randomized Complete Block Designs, Jourr~al of@alig Technoloy?;, 2 1, 277286.

Step 1. Start with a balanced design (randomly generated or given). Compute EC, and IC, for a given 9.
TECHNOMETRICS, NOVEMBER 2000, VOL. 42, NO, 4

Wu, C. F. J. (1993), Construction of Supersaturated Designs Through Partially Aliased Interactions, Biomerrika, 80, 661469.