Making Engineering Design Decisions

INTRODUCTION
If a major project is truly innovative, you cannot possibly know its exact cost and its exact schedule at the beginning. And if in fact you do know the exact cost and the exact schedule, chances are that the technology is obsolete. Joseph G. Gavin Jr. lead engineer for Grumman Aerospace on the Lunar Module Project Team [1994]. Decision making is an essential and nontrivial part of all engineering design projects.

THE DECISION MATRIX

A decision matrix is a design tool based on a table of values (see figure 1). The matrix is usually arranged with decision criteria (goals, objectives, constraints, etc.) forming rows in the matrix and decision alternatives (competing designs) forming columns. Moving down each column (alternative) a value is assigned for each criterion (row). Once all the values have been assigned, the individual criterion scores are aggregated into a single decision value for each alternative. The alternatives are then compared against each other to determine which is the dominant alternative (i.e. the best alternative). Various methods for determining dominance exist and will be discussed later in this paper. Often the aggregation function uses criterion weights to indicate the relative importance of each criterion.

Alternative A

Aggregated Score Criterion 1 Criterion 2 Criterion 3 Criterion 4

Figure 1. An Example Weighted Decision Matrix The decision matrix is basically a formal approach to benchmarking the alternatives and helping the engineer decide which alternative is best. The greatest strength of the decision matrix is that all of the design alternatives are shown together and are compared against a common set of decision criteria. This allows the designer to see the strengths and weaknesses of each alternative design. In the early stages of the design process, the designer can modify the weak areas of any alternative to make it stronger before the final decision is made. That way the decision is made between the best possible set of alternatives.

Decision Alternatives
Each alternative in an engineering design decision is a potentially successful design. The alternatives may be as trivial as multiple ways to fasten parts together or as complex as multiple systems for the exploration of Mars. In the conceptual, or Taguchi systems, stage of the design process, multiple design alternatives are usually identified. Given multiple alternatives for a design decision, the job of the engineer is to select the best alternative.

2000, Dale O. Anderson, Ph.D.

Weight

Example Decision Matrix

Alternative C

Alternative B

Louisiana Tech University

Decision Criteria, Values, and Ratings

Decisions are made by evaluating each alternative against a common set of decision criteria and assigning values that represents how well each alternative satisfies each criterion. A decision criterion may be similar to an objective function in goal programming in that we may wish to minimize or maximize the value. It may be similar to an inequality constraint, in that the value must remain within a specified range. It may also be similar to an equality constraint in that the value must be exactly the specified target value. Since a decision matrix is not a goal programming tool that automatically changes selected parameter values to minimize or maximize an objective function, the desire to minimize or maximize a criterion value must be relaxed somewhat. The real trick in aggregating a heterogeneous mix of values is to map them all into the same domain of ratings, namely the interval [0,M], where M is the maximum rating value. 0 rating M where M is the maximum rating value (1)

This mapping may be done subjectively, algebraically, with a utility function, or with a fuzzy set. Otto & Antonsson [1991] use subjective mapping where the designer evaluates each alternative value for each criterion and decides how well it meets the criterion. Thurston [1991] uses utility functions to do the mapping and they may be nonlinear. Bellman and Zadeh [1970] use fuzzy sets to do the mapping. Once all values are mapped into the same rating domain, they then become similar types of numbers (i.e. ratings) that any aggregation function can then combine to produce an alternative aggregate score (AAS), which can be used to determine dominance. Given a value for each alternative and criterion, a rating must be computed for use by an aggregation function. To facilitate the evaluation of criteria, let us define the following key words... ignore increase decrease target -ignore this criterion in the decision dominance model -the alternative should have a value near the maximum value (maximize) -the alternative should have a value near the minimum value (minimize) -the alternative should have the same value as the datum (equality constraint)

Inequality constraints are handled by simply specifying a useable range of values (bounds) for each criterion. If the alternative value is within the specified criterion range, the constraint is satisfied, if not the constraint is violated and something must be done. The expected range of a criterion value is useful design information in its own right. Ward et. al. [1994] contend that useable intervals of design variable values are familiar to design engineers from standards or experience. Antonsson & Otto [1995] state that these useable ranges should be the intervals over which the preference or dominance ratings should be nonzero. IF NOT ( low value high ), then rating = 0 (2)

Therefore, the range of values can be used to compute ratings. It is expected that in the initial phases of the design process, the value ranges will be specified as wide as possible. Then, as the design process proceeds, the ranges may be narrowed to provide better discrimination between design alternatives in the decision. If we wish to maximize the criterion value, we will compare the value of a criterion against the upper bound in the specified range to compute a rating. high value rating increase = M1 high low (3)

Similarly if we wish to minimize the criterion value, we will compare the value of a criterion against the lower bound in the specified range to compute a rating. value low rating decrease = M1 high low The closer the criterion value is to the bound, the higher the criterion rating. (4)

2000, Dale O. Anderson, Ph.D.

Louisiana Tech University

Equality constraints are handled by determining how close the alternative value is to the selected datum value. IF value = target, THEN ratingtarget = 1, OTHERWISE ratingtarget = 0 An exact match produces the highest rating. In engineering design, equality constraints are rare. (5)

Converting Criterion Values to Fuzzy Ratings

A fuzzy set (see figure 2) with three membership functions (see figures 3a-3c) may be used to perform the mapping from criterion value to rating. The offset value () used in the target membership function may be computed as a percentage of the value range ( = 0.05[ high -low ] for a 5% offset on each side). The offset value is subtracted from the datum value to produce the lower bound of the membership function, while it is added to the datum value to produce the upper bound. The rating produced by the fuzzy mapping is computed as follows... If ignore, then rating = blank (null) Else if increase, then rating = M[ DOM( value in increase ) ] = M(value-low)/(high-low) Else if decrease, then rating = M[ DOM( value in decrease ) ] = M(high-value)/(high-low) Else if target, then rating = M[ DOM( value in target ) ] If value < datum, then rating = M max( 0, value -datum + )/ Else rating = M max( 0, datum + -value )/ End if where DOM is the degree of membership in the specified membership function (0 DOM 1) (6)

The computation of rating for the increase case is identical to equation 3 and the computation of rating for the decrease case is identical to equation 4. The computation of rating for the target case is more relaxed than in equation 5.
Degree of Membership

decrease

target +

increase

low

datum Criterion Value target +

high

Figure 2. The Fuzzy Set for Criteria Ratings

Degree of Membership

decrease

increase

low

datum Criterion Value target +

high

Figure 3a. The increase Membership Function

Degree of Membership

decrease -

increase

low

datum Criterion Value

high

Figure 3b. The decrease Membership Function

2000, Dale O. Anderson, Ph.D.

Louisiana Tech University

Degree of Membership

decrease -

target +

increase

low

datum Criterion Value

high

Figure 3c. The target Membership Function Notice that each criterion will have a different fuzzy set depending on the expected range of values and the datum value. However, all fuzzy sets will be similar in shape and nature.

Criterion Weights
Probably the most formal approach to developing criterion weights comes from the Analytical Hierarchical Process (AHP) [Saaty, 1980] [Hyman, 1998]. It begins by setting up a comparison table (see Table 1) that has the criteria listed both down the left side and across the top. The designer then performs pair-wise comparisons of the criteria along each row. For each criteria pair, the designer selects a preference rating for the row criterion, from the following scale, that compares it to the column criterion... 9. Absolutely more important/preferred 7. Very strongly more important/preferred 5. Strongly more important/preferred 3. Moderately more important/preferred 1. Equally important/preferred (NOTE: even numbers are used for half-steps on the scale.) If the column criterion is more important or preferred than the row criterion, then the designer puts the reciprocal of the rating in the cell. (Notice that the designer need only fill in the upper triangular portion of the matrix and that the lower triangular portion is simply the cell-wise reciprocal of the transposed upper triangular portion.) Once the ratings have been assigned column sums are computed. Table 1. The Criterion Comparison Table for AHP CRITERION A B C D E TOTALS A 1 1/5 1 1 1/3 53/15 B 5 1 1/3 1/7 1/3 133/21 C 1 3 1 5 9 19 D 1 7 1/5 1 1/3 143/15 E 3 3 1/9 3 1 91/9

CRITERION A B C D E TOTALS

A 0.283 0.057 0.283 0.283 0.094 1.000

Table 2. The Completed Criterion Weights ROW B C D E TOTAL 0.789 0.053 0.105 0.297 1.527 0.157 0.158 0.734 0.297 1.403 0.053 0.053 0.021 0.011 0.421 0.023 0.263 0.105 0.297 0.971 0.053 0.474 0.035 0.099 0.755 1.000 1.000 1.000 1.000 5.077

NORMALIZED WEIGHTS 0.301 0.276 0.083 0.191 0.149 1.000

RANK 1st. 2nd. 5th. 3rd. 4th. ---

2000, Dale O. Anderson, Ph.D.

Louisiana Tech University

Next, each rating in the column is divided by the column sum (normalization by columns). The sum of each row is then computed. The row totals may be normalized and used as weights or otherwise scaled (see table 2). The row totals in table 2 may also be used to rank order the criteria and the rankings may be converted to weights in other ways [Dieter, 1991]. The designer could rank the criteria on a scale from 1 to 10, with the most important criterion being assigned the rank of 10... Criterion: Scale: Weight: C 1 2 0.037 E 3 4 0.111 D 5 6 0.185 7 B 8 9 0.296 A 10 0.371 Total 27 1.000

Another approach would be to distribute 100 points among the criteria based on their relative ranking... Criterion: Points: Weight: A 37 0.37 B 35 0.35 D 21 0.21 E 4 0.04 C 3 0.03 Total 100 1.00

These methods are all valid approaches, although they will not usually produce exactly the same weighting values. Of course, the weights could also be assigned subjectively by the designer or a team of people involved in the design. This is probably the most common approach. Many authors in the open literature insist that criteria weights be on the interval [0,1] and that the sum of all the weights must be identically equal to one. This is a good practice, and it is essential for some aggregation functions, but not for others. It is also easier to present the mathematics of aggregation functions if the weights add up to unity.

Properties of Engineering Design Aggregation Functions

Otto [Otto & Antonsson, 1991] [Otto, 1992] defined a set of properties (axioms) for engineering design aggregation functions. Antonsson & Otto [1995] and Scott & Antonsson [1995] reported on the amended set of properties. The following is an updated list of essential properties of engineering design aggregation functions. (NOTE that A is the aggregation function and r i is the rating of the alternative for the ith criterion)... 1. 2. Dominance -- The alternative with the highest aggregate score (AAS) is the dominant alternative, assuming all relevant issues have been included in the aggregation. Boundary Conditions -- If the designer prefers absolutely all of the goals (criteria), then the design (alternative) will also be preferred absolutely. Similarly, if the designer has no preference for the value of any of the goals (criteria) then the overall design (alternative) will also not be preferred [Otto & Antonsson, 1991]. A( 0, 0, ..., 0 ) = 0; A( M, M, , M ) = M; where M is the maximum rating value (0 rk M) Continuity -- as an individual goals (criterions) preference is changed slightly, then the overall preference for the design will change at most slightly. It does not mean that the preference for any goal must be continuous [Otto & Antonsson, 1991]. k, A(r1, , r k, , r n) lim A (r1, L , r k , L, rn )
rk rk

3.

4.

Monotonicity -- if an individual goals (criterions) preference is raised or lowered, then the designs (alternatives) overall preference is raised or lowered in the same direction, if it changes at all [Otto & Antonsson, 1991]. This property requires that the aggregation function behave in a reasonable and predictable manor. k, A( r 1, , r k, , r n ) A (r1, L , rk , L, rn ) iff r k rk Commutativity -- The alternative aggregated score (AAS) must not change if the criteria are reordered without changing preference ratings or weights.

5.

2000, Dale O. Anderson, Ph.D.

Louisiana Tech University

6.

Idempotency -- if a designer has the same preference (rating) for all individual concerns (criteria) in a design (alternative), then the overall preference (AAS) must have this degree of preference as well [Otto & Antonsson, 1991]. This axiom relates to rationality. The aggregated preference should not be greater that the largest criterion preference, nor less that the smallest. A( p, p, ..., p ) = p Annihilation -- an alternative is unacceptable if at least one of its criterion preference ratings is zero. This is a conservative design philosophy which eliminates design alternatives that utterly fail to satisfy one or more criteria. Vincent [1983], Biegel & Pecht [1991] and Otto & Antonsson [1991] all proposed the annihilation property as essential for engineering design. A( r1,..., 0,..., r n) = 0

7.

Aggregation Functions used in Engineering Design

Virtually all formal engineering design methods presented in the literature which select the dominant or best design alternative from a set of competing design alternatives use one or more functions from a small set of functions to aggregate information (numerical ratings) from multiple criteria into a single score that represents the preference for or dominance of each design alternative [Antonsson & Otto, 1995]. Wood et. al. [1990] discuss the use of the weighted sum aggregation function in a number of different functional forms including probabilistic. Interestingly enough, the weighted sum aggregation function is mentioned most often [Bellman & Zadeh, 1970] [Dieter, 1991] [Fishburn, 1982] [Hazelrigg, 1996] [Herling, et. al., 1995] [Saaty, 1980] [Voland, 1999]...

AAS j =

w r
i= 1 n

i ij

w
i= 1

(7)
i

where

AASj is the aggregated score for alternative j n is the number of criteria wi is weight associated with the ith criterion rij is the rating of alternative j for the ith criterion

This is a compensating aggregation function because it allows a high rating to make up for a low rating. Notice that the magnitude of the weights is unimportant (as long as they are not all zero) and that the aggregated alternative score (AAS) will have the same range of values as the criteria ratings. The weighted sum does NOT satisfy the annihilation property (#7) stated above. This appears to be its only weakness, which can be eliminated by embedding the aggregation function inside some simple logic... if min( r 1j , r 2j , , r nj ) > 0 then AASj = weighted sum (equation 6), otherwise AASj = 0 Equation 8 satisfies all of the properties stated previously. Another commonly used aggregation function is the min function Scott & Antonsson [Scott & Antonsson, 1995] [Otto & Antonsson, 1991]. AASj = min( r1j , r 2j , , r nj ) (9) (8)

The min function is a non-compensating aggregation function because an increase in any criterion rating will not necessarily increase the aggregated score (AAS). The AAS is simply the lowest criterion preference (i.e. the worst performance value or the weakest attribute). The min function satisfies all of the properties stated previously. Notice that the criteria weights are NOT taken into account by the min function. This is a VERY conservative, worst case, aggregation function for engineering decision making.

2000, Dale O. Anderson, Ph.D.

Louisiana Tech University

Voland [1999] presents the Kepner-Tregoe [1981] variation of the decision matrix. The rank ordered criteria are separated into two groups -- the musts and the wants. The musts are the crucial criteria that must be met in order for the decision to work at all. The wants are the criteria that can make a workable decision better. The alternatives are evaluated first against the musts using a simple, subjective go/no go evaluation. Any alternative that receives a no go is eliminated from further competition. Next the remaining alternatives are evaluated against the wants using an aggregation function.

The Pugh Dominance Model

The Pugh evaluation table [Pugh, 1990] is a decision matrix that employs a relatively simple comparison model. One of the alternatives is selected as the datum against which all other alternatives are compared. If the rating of an alternative for a particular criterion is better than the datum, then the rating is marked with a plus sign. If it is the same as the datum, the rating is marked with an s, and if it is worse than the datum it is marked with a minus sign. If ignore... then mark rating s Else if increase... If value > datum, then mark rating + Else if value = datum, then mark rating s Else if value < datum, then mark rating - End if Else if decrease... If value < datum, then mark rating + Else if value = datum, then mark rating s Else if value > datum, then mark rating - End if Else if target... If value = datum, then mark rating + otherwise mark rating - End if (10)

For each alternative, the number of negative and positive signs are counted up. The most dominant alternative is the one with no minus signs and the most plus signs. The greatest strength of the Pugh approach is that the weaknesses of each alternative are readily apparent since they are all marked with - signs. Early in the design process, attempts may be made to improve the weaknesses (i.e. eliminate the - signs) and thus produce a better alternative for the final decision. The Pugh dominance model may be made fuzzy by defining a fuzzy set that describes the types of criterion evaluations to be made (see figure 4). Notice that the increase membership function (see figure 5a) is defined for all criterion values greater than the datum value. The decrease membership function (see figure 5b) is defined for all criterion values less than the datum value. The target membership function (see figure 5c) is defined only for a tight range around the datum value. That range is defined by an offset () which is defined as a percentage of the criterion range ( = 0.05[ high - low ] for a 5% offset on each side).
Degree of Membership

decrease

target +

increase

low

datum Criterion Value

high

Fig. 4. The Criterion Evaluation Fuzzy Set

2000, Dale O. Anderson, Ph.D.

Louisiana Tech University

Degree of Membership

decrease -

target +

increase

low

datum Criterion Value target +

high

Fig. 5a. The Increase Membership Function

Degree of Membership

decrease

increase

low

datum Criterion Value target +

high

Fig. 5b. The Decrease Membership Function

Degree of Membership

decrease -

increase

low

datum Criterion Value

high

Fig. 5c. The Target Membership Function The fuzzy Pugh dominance model may be defined using the membership functions just presented (see figures 4 and 5)... If ignore... then mark rating s Else if increase... If DOM( value in target) > DOM( value in increase), then mark rating s Else if DOM( value in increase) > 0, then mark rating + Else mark rating - End if Else if decrease... If DOM( value in target) > DOM( value in decrease), then mark rating s Else if DOM( value in decrease) > 0, then mark rating + Else mark rating - End if Else if target... If DOM( value in target) > 0, then mark rating + otherwise mark rating - End if (11)

Dominance is still determined as with the original Pugh model -- the dominant alternative is the one with no - signs and the most + signs.

REFERENCES:
Antonsson, E. K. & K. N. Otto, 1995, Imprecision in Engineering Design, Journal of Mechanical Design, Special 50th. Anniversary Design Issue, Vol. 117(B), pp. 25-32, ASME International, New York.

2000, Dale O. Anderson, Ph.D.

Louisiana Tech University

Bellman, R. & L. Zadeh, 1970, Decision Making in a Fuzzy Environment, Management Science, Vol. 17, No. 4, pp. B 141-163. Dieter, George E., 1991, Engineering Design A Materials and Processing Approach, 2nd. ed., McGraw-Hill, New York. [ISBN 0-07-016906-3, LOC TA174.D495] Fishburn, P., 1982, Foundations of Expected Utility, D. Reidel Gavin, J. G. Jr., 1994, Fly Me To The Moon: An Interview With Joseph G. Gavin Jr., Technology Review, Vol. 97, No. 5, pp. 61-68 Hazelrigg, George A., 1996, Systems Engineering: An Approach to Information-Based Design, Prentice-Hall, Upper Saddle River, NJ. [ISBN 0-13-461334-9, LOC TA168.H39] Herling, Ullman & Ambrosio, 1995, Engineering Decision Support System (EDSS), DE-Vol. 83, pp. 619-626, Proceedings of the 1995 Design Engineering Technical Conference, 17-20 Sept. 1995, Boston, MA, ASME International, New York. Hyman, B., 1998, Fundamentals of Engineering Design, Prentice-Hall, Upper Saddle River, NJ. [ISBN 0-13531385-6 , LOC TA174.H96] Kepner, C. H. & B. B. Tregoe, 1981, The New Rational Manager , Princeton Univ. Press, Princeton, NJ Otto, K. N., 1992, A Formal Representational Theory for Engineering Design, Ph.D. Thesis, Cal Tech Otto, K. N. and E. K. Antonsson, 1991, Trade-Off Strategies in Engineering Design, Research in Engineering Design, Vol. 3, pp. 87-103, Springer-Verlag, New York Pugh, S., 1990, Total Design: Integrated Methods for Successful Product Engineering, Addison-Wesley, New York. [ISBN: 0-201-41639-5, LOC: TA174.P84] Saaty, T. L., 1980, The Analytical Hierarchical Process: Planning, Priority Setting, Resource Allocation, McGraw-Hill, New York Scott, M. J. & E. K. Antonsson, 1995, Aggregation Functions for Engineering Design Trade-Offs, DE-Vol. 83, pp. 389-396, Proceedings of the 1995 Design Engineering Technical Conference, 17-20 Sept. 1995, Boston, MA, ASME International, New York. Thurston, D. L., 1991, A Formal Method for Subjective Design Evaluation with Multiple Constraints, Research in Engineering Design, Vol. 3, pp. 105-122, Springer-Verlag, New York. Voland, G., 1999, Engineering By Design, Addison-Wesley, Reading, MA. [ISBN 0-201-49851-0 , LOC TA174.V65] Ward, A. C., J. K. Liker, D. K. Sobek, J. J. Cristiano, 1994, Set-Based Concurrent Engineering and Toyota, Design Theory and Methodology - DTM94, ASME DE-68, ASME International, New York Wood, K. L., E. K. Antonsson, J. L. Beck, 1990, Representing Imprecision in Engineering Design - Comparing Fuzzy and Probability Calculus, Research in Engineering Design, Vol. 3-4, pp. 187-203

2000, Dale O. Anderson, Ph.D.

Louisiana Tech University