European Journal of Operational Research 170 (2006) 597612 www.elsevier.

com/locate/ejor

Computing, Articial Intelligence and Information Technology

Genetic algorithms in logic tree decision modeling

Brenda Mak
a

a,*

, Robert Blanning b, Susanna Ho

Department of Information Systems, College of Business Administration, San Francisco State University, 1600 Holloway Avenue, San Francisco, CA 94132, USA b Owen Graduate School of Management, Vanderbilt University, Nashville, TN 37203, USA Department of Accounting and Business Information Systems, The University of Melbourne, Victoria, 3010 Australia Received 30 November 2001; accepted 3 September 2004 Available online 21 November 2004

Abstract An important approach to decision modeling is the induction of knowledge structuressuch as rules, trees, and graphsfrom empirical data describing previous conditions and the resulting decisions. We examine here a specic knowledge structure, a logic tree, in which the conditions are leaves, the decision is the root, and the intermediate nodes are logical operators. We then use genetic algorithms (GAs) to construct logic trees that best represent the correspondence between conditions and decisions described by the empirical data. We also investigate an important characteristic of the GA search, the tness distance correlation. Finally, we comment on the usefulness of GAs in knowledge modeling. 2004 Elsevier B.V. All rights reserved.
Keywords: Genetic algorithms; Logic trees; Fitness distance correlation; GA eciency

1. Introduction Logic trees are important and ecient ways to structure decision processes. They are simple to use and ecient for system storage and retrieval. A logic tree has input variables at its leaf nodes, logical operators at its intermediate nodes, and

Corresponding author. Tel.: +1 415 405 0595/338 2140; fax: +1 415 405 0364. E-mail address: bmak@sfsu.edu (B. Mak).

the decision at the root [51]. Unlike a decision tree [4143], a logic tree does not employ greedy algorithms [2], such as entropy reduction [39], to select the next variable for partitioning the tree. Instead, it uses logical relationships among input variables to classify output. By focusing on the logical operators and logical rules among decision variables, the logic tree oers the decision-maker a simple and powerful model to guide decision-making. A substantial amount of computer time may be consumed in order to nd the best logic tree exhaustively. For problems of reasonable size,

0377-2217/$ - see front matter 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2004.09.030

598

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

Genetic Algorithms (GAs) can substantially shorten the search time. GAs are search algorithms, originally developed by Holland [23], based on the mechanism of natural selection. Using the genetic operators of reproduction, crossover and mutation, binary coded strings of chromosomes are transformed for the purpose of maximizing an objective function (herein called a tness function). GAs have been applied to a variety of operations research problems, such as bin packing, facility location, scheduling, and of course, the traveling salesman problem [44,55]. An appealing feature of GAs in some cases is that they work well with large problems. Another is that they are often combined productively with other heuristic search techniques. However, they are most productive when they are properly encoded to match the problem at hand [29], which in this case means adapting them to the variety of tree structures being investigated, and that certain additional theoretical results on the performance of GAs may lead to even greater productivity [31]. In this paper we apply GAs to search for the logic tree that best describes the decision processes of experts. Knowledge extracted from the experts is used to build a logic tree, and GAs are used to search for the logic tree decision model that best approximates the expert knowledge. In applying GAs to model logic trees, we are interested in investigating how GAs, as a growing technique in operations research, can be used to model a logic tree. We are also interested in the mechanisms involved in GA search and how the structure of the logic tree might aect the performance of GAs. The understanding of the mechanisms involved in GA logic tree modeling would bring insights on how GAs can be combined with logic trees in decision modeling. We apply GAs to logic trees in the context of an important business problemthe timing of new product entry. We assume that a rm has decided to introduce a new product into a market but is concerned about the time at which the introduction should occur. A premature entry may bring nancial loss to the company while too late an entry may forgo opportunities and market position advantages. In order to develop rules, in the

form of a logic tree, we elicit responses from people with expertise in this area and use GAs to construct appropriate logic trees. In order to understand the mechanisms involved in the GA tree search, we analyze the tness distance correlations for logic trees with dierent structures. Fitness distance correlation (FDC), introduced by Jones and Forrest [26], has been offered as a summary statistic in predicting the performance of GAs in optimization. We investigate how FDC may be related to characteristics of the logic tree structures, such as the depth of the search tree, branching factors (the number of pairs of leaf nodes), and the number of subtrees for logic trees with dierent structures. In Section 2, we describe logic trees. In Section 3, we review research on GAs and their use in decision modeling. Section 4 explains the details of the GA logic tree analysis, and describes a study conducted to collect data for our analysis. In Section 5 we analyze FDC for dierent tree structures. Section 6 is a discussion of our work and Section 7 concludes with suggestions for future research.

2. Logic trees A logic tree is a tree in which the root is the decision to be made and the leaves are the variables aecting the decision [6]. The variables at the leaves and the decision at the root are all Boolean variables that take on the values of 0 or 1. An example of a logic tree is shown in Fig. 1, in which the variables at the leaves are x1 through x4. Let us assume a scenario in which x1 = x2 = x3 = x4 = 1. At the leaves of the tree, we have x1, NOT x2, x3, and NOT x4. The intermediate nodes use the AND operators, and the values at the nodes are x1 AND
x3 AND (NOT x7) OR x2 AND (NOT x5)

x3 AND (NOT x7) x2 AND (NOT x 5)

x1

x2

x3

x4

Fig. 1. An example of a logic tree.

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

599

(NOT x2) and x3 AND (NOT x4). When x1 = 1, x2 = 1, x1 AND (NOT x2) gives a value of 0. When x3 = 1, x4 = 1, x3 AND (NOT x4) gives a value of 0. Finally, at the root, we use the OR operator and the expression is (x1 AND (NOT x2) OR (x3 AND (NOT x4))), which evaluates to 0. The root is the decision DEC which takes on the value of 0. For a pair of operands xi and xj, a logical relationship is denoted as (xi R xj). Sixteen R-relations can be constructed: (xi FALSE xj), (xi TRUE xj), (xi AND xj), (xi OR xj), (xi XOR xj), (xi LEFT xj), (xi RIGHT xj), (NOT (xi) AND xj), (xi AND NOT (xj)), (NOT (xi) AND NOT (xj)), (NOT (xi) OR xj), (xi OR NOT (xj)), (NOT (xi) OR NOT (xj)), (NOT (xi XOR xj)), (NOT (xi) LEFT xj), (xi RIGHT NOT (xj)). XOR is the exclusive-OR relationship and returns one if exactly one scenario variable is one. (xi FALSE xj) always returns zero while (xi TRUE xj) always returns one. (xi LEFT xj) returns the value of xi regardless of xj while (xi RIGHT xj) returns xj regardless of xi. In our work we use logic trees of the type described above to describe the impact of rm, product, and market factors (such as the nancial strength of the rm, the length of the product life cycle, and the demand growth rate) on the decision to introduce the product. The factors will be at the leaves of the tree, and the decision to introduce (or not to introduce) will be at the root. GAs will be used to specify the Boolean operations at the intermediate nodes based on information about the factors and the entry decision elicited from the experts.

3. Genetic algorithms The GA method is a machine learning technique that locates good solutions in a fast and ecient manner. Goldberg applied GAs to optimization problems and developed a classier system for machine learning [18]. The method is based on the evolution of chromosomes, or bit strings [16,17]. Selection according to tness allows GAs to exploit the strength of existing chromosomes. In the search space, GAs direct the search towards schemata of high tness regions. Sche-

mata are sets of individuals in the search space, each composed of a template dened over the alphabet {0, 1,*} which is a dening hyperplane partitioning over the search space {0, 1}g, the pattern of bit strings of length g. A bit string that matches a schemas pattern is an instance of the schema. For example, 00 and 10 are instances of *0. 0 and 1 are the dened bits whereas * means do not know and can take on the value of either 0 or 1. The order of a schema is the number of dened bits in the string. The dening length is the distance between the leftmost and rightmost dened bits in the string. GA operators consist of reproduction, mutation and crossover [15]. Reproduction is a process to replicate individual chromosomes according to their tness, that is, the values of the objective function that the GA is trying to maximize. In reproduction, chromosomes with higher tness values have higher chances to generate more ospring in the next generation. Goldberg suggested using roulette wheel selection [17], where the chance of selecting a chromosome to be the parent in the next generation is directly proportional to the parents tness. This algorithm sweeps out the less t chromosomes and keeps the tter ones. The selected chromosomes are put in a mating pool for crossover and mutation in later stages. Crossover is more appropriate for extensive exploration while mutation is better for local search. Mutation ips some of the bits of existing chromosomes at random to ensure diversity in the population, but too high a mutation rate may have a disruptive eect on existing chromosomes [37]. Crossover, on the other hand, explores the strengths of new chromosomes based on the combination of existing ones [12]. Consider two chromosomes, Parent 1 (P1): 1 1 0 1 1 0 0 and Parent 2 (P2): 0 0 1 1 1 1 1. One-point crossover for P1 at its sixth position with P2 would yield a child C1: 1 1 0 1 1 1 1, where elements changed are denoted in boldface. Two-point crossover, on the other hand, involves exchange of string portion between two xed points. Thus, two-point crossover of P1 from its third to fth elements with P2 would yield a child CC1: 1 1 1 1 1 0 0. One-point crossover has strong positional bias [13]. That is, interacting bits placed far apart might

600

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

be easily combined with bits in another chromosome. Interacting bits refer to bits denoting the same characteristic of the tree. For example, bits 1 through 4 describe the structure of the tree, while bits 5 through 8 describe the root of the tree. Noninteracting bits close together might be propagated at the same time and cause spurious correlation between some of these bits and the tness function. In addition, a long sequence of operations may be required in one-point crossover to reach certain points in the search space. The crossover point in one-point crossover can vary from 1 to g 1 position of the parents, where g is the length of the string. A crossover point that is biased toward g/2 will be more disruptive but more explorative than one that is biased toward 1 or g 1. Two-point crossover is commonly used in GA search. It is more disruptive than one-point crossover. It regards the chromosome string as a ring and each bit in the ring has a higher probability of falling inside the disrupted segment, than is the case with one point crossover. Schemata with dening length near g/2 are more likely to be disrupted than schemata with shorter dening length. Its positional bias is moderately strong but is less than that of one-point crossover. Other forms of crossover include uniform crossover [52], where each child has an equal probability of inheriting from either of the parents. Uniform crossover is the most exploratory but is also the most disruptive. The distance between chromosomes in a GA is often measured by means of a Hamming distance. If x and y are two binary chromosomes of the same length, the Hamming distance between these two chromosomes is the number of symbols that disagree. Consider two chromosomes, for example, 10100 and 10011. The rst two bits of the two chromosomes are the same while the third, fourth, and fth bits are dierent, and we obtain 3 as the Hamming distance between the two chromosomes. Gray coding can be used to encode bits in GAs so that the Hamming distance between two consecutive numbers is one unit apart. For example, the Gray code for a string of two bits is as follows: 0 is coded as 00, 1 is coded as 01, 2 is coded as 11, and 3 is coded as 10. A change in a single bit will

change the number by only one position (with the end points 0 and 3 considered adjacent). However, the use of Gray coding may result in misleading information [15]. Consecutive numbers dier from each other by one bit in the Gray code, and the Hamming distance between adjacent numbers is one. However, the converse is not necessarily true. Two numbers may not be adjacent to one another even if they are of one unit Hamming distance apart in their Gray code representation, and this may cause overshooting in the GA search. There are several problems associated with GAs. One problem is convergence on local optima and diculty in GA search when the optimal points of low-order schema have a bit pattern different from that of a higher-order subsumed partition [16]. Another is the problem of deception, which arises when misleading or insucient information is contained in low-order schemata (i.e., schemata with a large number of undened bits) [16]. The tness of a low-order schema is approximated by replacing each do not know * with a randomly generated bit (resulting in a chromosome) and calculating the average tness for several such chromosomes. Low-order schemata of apparent low tness may eventually evolve into a higher-order schema that contains the global maximum, but GAs may mistakenly weed out these schemata and fail to nd the global maximum. Another problem in GAs is premature convergence. This occurs when the population loses diversity before the optimal values have been found. To overcome this problem, Eshelman and Schaer [1113] proposed a strategy of delaying commitment and empirically found that their incest prevention mating strategy (i.e., only allowing crossover between parents if their Hamming distance was above a certain threshold) is the most eective. Their assumption is that crossover from diverse parents would produce more diverse children and increase population diversity. GAs have been used with other data mining techniques for rule induction and knowledge representation [25,32]. They were employed in the design of neural networks for forecasting and classication [24,40,48,56,58]. Chai et al. developed a binary tree genetic algorithm to classify pap smear cells [7], while Sorensen and Janssens

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

601

applied GAs to nd binary decision trees for classication [50]. Terano and Ishino developed a simulated breeding and inductive learning environment (SIBILE) system that used GAs to analyze marketing questionnaires in order to generate sets of decision trees [53]. The trees described consumer satisfaction as a function of product attributes of toothpaste brands. Domain experts then evaluated the trees to identify the most realistic ones. These trees became parents for genetic operations, and performance of these operations resulted in a single decision tree containing the least number of decision variables that most accurately modeled consumer satisfaction. Yang and Honavar [57] used GAs to select the features to be used as inputs for a neural network. They developed DistAL, a neural network system that added hidden processing elements (neurons) one at a time, so that the most parsimonious network was constructed. They then used GA to identify the subset of features that gave the best tness when being used as input to the neural network. The tness function depended on the accuracy of the classication function realized by the neural network and cost of performing the classication. The cost of classication included the risk and cost of measuring the value of a particular feature needed for medical diagnosis. They tested their system on several medical data sets and showed that their approach provided higher predictive accuracy than other approaches. GAs have been used to look for fuzzy rules and logic rules. Thomson and Miller [54] used GAs to look for logic rules and developed an algorithm called XORGA that used GAs to nd a set of variable ordering and circuit logic gates (AND or XOR) for reducing the cost of circuit connection. They tested XORGA on benchmark functions and demonstrated that the method produced lower cost circuits. Using historical stock price data, Mullei and Beling [38] employed a GA with supervised batch learning to develop a scoring model to nd optimal investment strategies. Edmonds et al. [10] used GAs to identify fuzzy logic rules for nancial trading and maximized a tness function that depended jointly on prot and equity. They

evaluated a set of trees containing fuzzy operators and converted the trees to production rules of the form IFhConditioniTHENhCrisp Classicationi. Bickel and Bickel [4] developed the GENES program to elicit rules from medical experts. The knowledge of each expert was made up of a linked list of rules, and each rule was parsed syntactically as a tree structure. There was one action node per rule, while the conditions precedent to the action were linked by Boolean operators. Genetic operators were used to generate rules by varying the logical operators and relational operators (such as > , > = , < , < = , =), and the number of conditions in the rules. Experts evaluated the rules, and convergence was reached if the sum of all the expert scores remained within a narrowly bounded interval over several consecutive iterations. Estivill-Castro [14] developed the EVOPROL (Evolutionary Propositional Logic) system to elicit logic rules to classify whether a country is Spanish speaking. Attributes that were relevant for the classication included population, religion and ethnicity. Decision criteria were extracted from samples of cases solved by experts and a GA explored the set of rules for classication. Antecedents were represented as trees where negations were placed at the leaves and the internal nodes were binary nodes determining whether the logical operators were conjunction or disjunction. De Morgans laws allowed the placement of necessary negations at the leaves, thus internal nodes representing NOT were unnecessary. The rank of a node n is the number of nodes in the subtree rooted at n. To adjust for the cost of misclassication in attribute testing, EVOPROL used a weighted tness function that is correlated with the proportion of positive examples in the training set.

4. Decision modeling with GA logic trees In this paper we apply GAs to develop a logic tree that models the rules of experts on new product entry timing. New product entry is a critical strategic issue and an important and dicult decision facing managers. A company may lose its

602

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

rst-mover advantages and weaken its competitive position if a product is launched too late into the market, while premature entry to the market may mean high opportunity costs and may drain the company of resources [33]. Because of its importance, we extracted expert knowledge in new product entry. Our decision problem consists of nding the new product entry strategy (DEC) when decision-makers are given the values of certain scenario variables. When DEC = 1, the strategy is GO, and when DEC = 0, the strategy is NOGO. The scenario variables are denoted below as vi, i = 1, . . . , 7, and they are: v1 = position of the rm (dominant or a small), v2 = nancial strength (strong or weak), v3 = demand growth rate (high or low), v4 = product life cycle (long or short), v5 = diusion rate across competitors (fast or slow), v6 = cannibalization rate (high or low), and v7 = cost of market development (high or low). For example, v1 = 1 means the rm is dominant and v1 = 0 means the rm is small. There is a similar interpretation for v2 through v7. Computer cases were generated to extract judgments from a group of thirty-six experts. Details of the method are described in [34]. Although there were only 128 (=27) possible cases, a total of 670 responses were elicited, of which 139 had inconsistent conclusions; thus, the number of non-contradictory responses was thus 531. We consider a logic tree with seven leaf nodes. The logic tree can be characterized with the following features: (1) the tree structure (2) the assignment of logical relationships at the non-leaf nodes (3) the ordering of scenario variables at the leaf nodes. The decision outcome at the root of a tree is a logical function involving the scenario variables at the leaf nodes. The accuracy of the
Tree 1 = (NOT v1) AND v3 (NOT v1) AND v3
(NOT v1) LEFT v2

logic tree is the number of responses where the logic tree correctly predicted the decision outcome, divided by the maximum number of correct predictions given the inconsistencies (=531). It is possible for dierent trees (i.e., trees of different shapes) to be logically equivalent [47]that is, trees with dierent structures, logical operators, and variables may reduce to the same logical expression, producing the same truth table. For example, the two trees in Fig. 2 have NOT operators at dierent levels but using De Morgans law these two trees can be reduced to the same logical expression: NOT (v1) AND v3. By analyzing equivalent trees we can reduce the search space and shorten the time required for GA search. We analyzed the equivalence of logic trees and reduced the number of tree structures from 704 to eleven (see Appendix A). Fig. 3 shows these eleven tree structures. We applied De Morgans law and placed negations at the leaves and the root, and it was not necessary to have the NOT operators at the intermediate nodes [14]. The intermediate nodes and the root had eight relationships: (vi FALSE vj), (vi AND vj), (vi OR vj), (vi XOR vj), (vi LEFT vj), (vi RIGHT vj), (NOT (vi) AND vj), (vi AND NOT (vj)). The root had eight additional relationships: (vi TRUE vj), (NOT (vi) AND NOT (vj)), (NOT (vi) OR vj), (vi OR NOT (vj)), (NOT (vi) OR NOT (vj)), (NOT (vi XOR vj)), (NOT (vi) LEFT vj), (vi RIGHT NOT (vj)). These eight relationships could be expressed in terms of the rst eight relationships by rearranging the position of the negation operators. We performed exhaustive search of these trees using a Pentium IV computer with a clock speed of 3 GHz and 1 GB of RAM. The time taken was 15.45 hours and the maximum accuracy was 94.7% (=503/531). To speed up the search, we
Tree 2 = (NOT v1) AND v3 (NOT v1) AND v3
v1 LEFT v2

v1

v2

v3

v1

v2

v3

Fig. 2. Two trees that can be reduced to the same tree using De Morgans law.

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

603

N1 N1 N2 N3 N4 N5 N3 N4 N5 N3 N4 N5 N2 N1 N2

Tree T1

Tree T2
N1 N2 N3

Tree T3
N1 N2 N3 N4 N5

N1 N2 N3

N4

N5

N4

N5

Tree T4

Tree T5

Tree T6
N1

N1 N2 N3 N4

N5

N1 N2 N4

N3 N5

N2 N4

N3 N5

Tree T7
N1 N2 N3 N4

Tree T8
N1 N2 N3 N4 N5

Tree T9

N5

Tree T10

Tree T11

Bits Representing Logical Operators

Non-leaf Node N1

Non-leaf Node N2

Non-leaf Non-leaf Node N3 Node N4

Non-leaf Node N5

Fig. 3. The eleven tree structures for a seven-node logic tree.

used GAs, which shortened the search time to several minutes, as reported in the paragraph after the next. Each tree structure had seven leaf nodes of decision variables and six non-leaf nodes of logical operands (one root and ve intermediates). Since we used 4 bits in our GA string to represent a total of eleven trees, there is a danger that some representations may point to non-existing trees. To solve this problem, we started rst by analyzing the typology of the trees using graphical analysis. We have tree structures T1 to T11 which exist. If the trees do not exist, we do not include them in our analysis. In addition, we considered commutative relationships and reduced the possible number

of variable combinations at the leaf nodes of a seven-node tree to 630 (see Appendix B). We found it required less time to nd the GA optimal when a loop was used to search through the 630 combinations of variable ordering than when we used an additional 10 bits in our GA chromosome to search for the variable ordering. The steps of the GA tree search are shown in Fig. 4. First, GAs were used to search for the tree and the assignment of logical relationships at the non-leaf nodes of the tree. Second, exhaustive search was conducted to nd the best combination of variable ordering among the 630 variable combinations. Third, the result of the search was used

604

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

Tree Structure and Operator Assignment

Using GAs

Exhaustive Search
Find variable ordering with best fitness value

Result
Return chromosome of tree with maximum hit

Fig. 4. Steps in the GAs tree search.

to induce the details of the logic tree that best approximated the judgment data. GA parameters consist of population size, number of generations, probability of crossover and probability of mutation [30,36,49]. Generally, a population contains 20200 chromosomes. Probability of crossover ranges from 0.3 to 0.9 and probability of mutation should be a very small value, say 0.0010.01. Miller et al. [36] and Sexton et al. [49] recommended using 50 to 5000 generations, while Kim and Kim [30] suggested stopping the GA program if there is no improvement in 100 consecutive generations. Table 1 shows the values of the four GA parameters (population size, number of generations, probability of crossover, and probability of mutation) we used. We conducted our analysis using a Pentium IV computer with a clock speed of 3 GHz and 1 GB of RAM. We tested 162 (=3 6 3 3) possible combinations and made ten runs for each combinations. We found that there were 25 combinations of parameter values that resulted in the maximum hit of 498 (this is equal to an accuracy of 498/531, or 93. 8%). The average convergence time for those 25 combinations of parameter values was 3.68 minutes with a standard deviation of 2.52 minutes (convergence time refers to the time required to attain less than

0.01% uctuation in accuracy of prediction). We arbitrarily used one of these combinations and these values are given in column 3 of Table 1. We used roulette wheel selection [17] and twopoint crossover. We tried one point crossover but the result of the GA search is signicantly better with two-point crossover. The nal chromosome (see Fig. 3) consisted of twenty-three bits: the rst four bits for the structure of the tree (T1 to T11), the second four bits for the logical operator at the root, the remaining fteen bits for the logical operators at the intermediate nodes N1, N2, N3, N4, and N5. Thus, the GA search involved nding the maximum in a space of 223 (=8,388,608). The tness function was the classication accuracy of the logic tree. The optimal trees found by GAs were trees T1 and T10, with a maximum hit of 498, and an accuracy of 93.8% (=498/531). For tree T10, its global optimum found from exhaustive search has the same accuracy as the one found from GA search (both have a maximum hit of 498%, or 93.8% accurate). For tree T1, its global optimum found from exhaustive search has a hit of 503, whereas its GA optimum has only a hit of 498. One characteristic of logic trees is that several dierent structures will give the same value of DEC [6]. Just as experts can use dierent logical operations to combine decision variables, we may have dierent logic trees combining decision variables using dierent logic operators. These dierent logical logic trees might give the same conclusion on the product entry decision.

5. Algorithm eciency and tness distance correlation The eciency of GA search depends on three principal factors [16]. First, a tness function that has a large number of local optima may present diculties for GA search, as GAs may tend to converge on one of the local optima. Second, if the optimal points of low-order schema have a bit pattern dierent from that of a higher-order subsumed partition, the function may be dicult for GA search. Third, functions having schemata with long dening lengths may be disrupted by

Table 1 GA parameters tested in the study Genetic algorithm parameters Population size Number of generations Probability of crossover Probability of mutation Tested parameters Selected parameter choice 100 300 0.3 0.01

20, 100, 200 60, 80, 100, 300, 500, 1000 0.2, 0.3, 0.6 0.001, 0.005, 0.01

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

605

crossover, and GA search may be inecient for these functions. Other factors that aect GA diculty include epistasis [9], multimodality, and noise [16]. However, it would be dicult to evaluate a function at every point in its argument space in order to determine the eciency of the function for GA search. Researchers have proposed various measures, such as epistasis correlation [46], to assess the diculty level of a function for GA optimization. Kargupta [28] has proposed the signalto-noise measure and suggested that noises may cause diculty in GA analysis. Altenberg [1] suggested measuring the evolvability, the probability of success that the ospring produced is tter than their parents, to predict GA performance. The tness distance correlation (FDC), the correlation between tness and Hamming distance from the global maximum, is proposed as an indicator of GA eciency [8,26,27,35]. Consider a set of (fi, di) pairs, where F = {f1, f2, . . . , fn} is a set of n individual tness measures and D = {d1, d2, . . . , dn} is a corresponding set of n Hamming distances to the global maxima. FDC, is dened as FDC C FD ; SF SD

P   where C FD 1 fi f d i d is the covariance n of F and D; SF and SD are the standard deviations   of F and D, respectively; f and d are the means of F and D, respectively. For simple maximization problems, a GA chromosome near the global optimum should have a bit pattern similar to that of the chromosome at the optimal point. The tness of the chromosome increases as its Hamming distance from the optimum decreases. FDC lies between 1.0 and +1.0, and a more negative FDC suggests the function is less dicult for a GA to search. Nevertheless, there are reservations about the blind use of FDC as an indicator of GA search eciency. In the extreme case when FDC is close to 1 and if it is computed in Hamming space, a GA is not ecient since a hill climber will also do the search. We computed the FDC for each of the eleven trees. Since our focus is on analyzing the performance of GA in its search and not on nding the

GA optimum, we followed the three-step procedures applied by Jones and Forrest in their analysis of FDC [26]. First, the trees were analyzed using exhaustive search method, and the global maximum of exhaustive tree search is obtained. Second, the GA search was conducted with a population size of 100 for 50 generations, generating 5000 (=number of generations population size = 50 100) pairs of tness and distance in each search. Using the following set of parameters: population size = 100, number of generations = 50, probability of crossover = 0.3, probability of mutation = 0.01, we made thirty searches for each tree to obtain average values of FDC. Each GA search took about 15 seconds to run on a Pentium IV computer with a clock speed of 3 GHz and 1 GB of RAM. Third, FDC was computed together with the percentage of chromosomes with tness measures greater than the following percentages 70%, 75%, 80%, 85%, 90% and 95% of the optimum. For example, if the total number of correct predictions for the optimum of tree T1 is 503, the percentage of chromosomes having tness greater than 70% of the optimum of tree T1 is the percentage of chromosomes with correct predictions greater than 352 (=503 0.7). Fig. 5 shows a plot of the percentage accuracy of the global optimum against FDC for the eleven GA logic trees. Our result indicates that the more negative the FDC, the higher is the accuracy achieved in a short time interval of GA training. Notice here the change in FDC causes the highest drop in percentages of tness for chromsome tness greater than 70%. Table 2 shows the FDC associated with the GA search for trees T1 to T11, as well as the maximum accuracy found by exhaustive search for the trees. FDC for tree T10 is the lowest (= 0.28) and T10 has the highest eciency in GA search, with 1.64% of its chromosomes above 95% and 7.92% above 90%. The maximum attained with trees T1, T2, T3, T4 through exhaustive search (maxium hit = 503%, or 94.7% accuracy) is higher than that of T10 (maximum hit = 498%, or 93.8% accuracy), but GAs identied both trees T1 and T10 as the best solution (maximum hit = 498, see Fig. 6 for details of the two GA optimal trees). For tree T10, the maximum found through GA search is the same as

606

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

70% Percentage of chromsomes with fitness measures greater than 75% 80% 45 40 35 30 25 20 15 10 5 0 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 Fitness distance correlation (fdc) 85% 90%

Fig. 5. Fitness distance correlation and percentage of chromosomes with tness measures greater than certain specied values.

Table 2 Fitness distance correlation and accuracy of optimal trees found through exhaustive search for the eleven trees Tree Number of correct predictions (accuracy) Fitness distance correlation 0.0951 0.1243 0.0927 0.0843 0.0808 0.0201 0.0090 0.1999 0.0316 0.2832 0.2324 Percent of chromosomes with hit >80% 14.46 5.12 12.54 14.22 7.02 10.18 8.34 19 7.28 25.76 17.74 >90% 4.28 1.52 5.14 5.06 1.96 3 3.52 6.58 2.48 7.92 5.4 >95% 0.76 0 0.96 0.84 0.16 0.58 0.8 1.44 0.24 1.64 1.18 Number of pairs of leaf nodes 3 3 3 3 2 2 2 2 2 2 1 Depth Number of pairs of leaf nodes in Left subtree 3 4 4 4 5 5 5 4 4 5 6 2 3 2 2 2 2 1 1 2 2 1 Right subtree 1 0 1 1 0 0 1 1 0 0 0

T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11

503 503 503 503 498 499 499 500 500 498 488

(94.7%) (94.7%) (94.7%) (94.7%) (93.8%) (94%) (94%) (94.2%) (94.2%) (93.8%) (91.9%)

the maximum found through exhaustive search. However, for tree T1, the maximum found through GA search is lower than the maximum found through exhaustive search. Table 2 also indicates that T11 has the lowest accuracy in exhaustive search but it has a more negative FDC than T2 and thus higher accuracy than T2 in GA search. One of the factors aecting GA eciency, as indicated by FDC, may be the amount of disruption involved when crossover is conducted. Search eciency is aected when interacting bits are disrupted while non-interacting bits are propogated

together as related components. A chromosome with more interacting bits may be more susceptible to disruption. A tree with more pairs of leaf nodes at the bottom have more interacting bits, and they may be more susceptible to disruption and be less GA ecient. Fig. 3 shows the coding scheme for the GA chromosomes of the eleven trees. The rst four bits represent the tree structure, the second four bits represent the root, while the remaining fteen bits represent the logical operators found at the ve non-leaf nodes. Notice here the three bits representing N2 are between those bits representing N1 and N3. When N1 is very close to N2

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

v1 OR (v2 AND (NOT v5)) OR (v3 AND (NOT v7))
(v2 AND (NOT v5)) OR (v3 AND (NOT v7)) (v2 AND v3)) OR (v4 AND (NOT v5)) v2 AND (NOT v5) v3 AND (NOT v7) v2 AND v3 v4 AND (NOT v5)

607

(v2 AND v3) OR (v4 AND (NOT v5)) OR (NOT v6)

v1

v2

v5

v3

v7

v2

v3

v4

v5

v6

(a) Optimal tree of tree T10

(b) Optimal tree of tree T1

Fig. 6. Details of the optimal logic tree found by GAs.

in the tree, it may be dicult for GAs to decide which of the six bits representing N1 and N2 should be interacting with one another [11]. Table 2 shows the relationship of FDC to the characteristics of the eleven trees with respect to the following: (1) depth of the trees, (2) the number of pairs in the left and right subtrees of the trees, and (3) the number of leaf nodes at the bottom. In Table 2, T1, T2, T3 and T4 have three pairs of leaf nodes at the bottom, and their FDC average is 0.037. T5, T6, T7, T8, T9 and T10 have two pairs of leaf nodes at the bottom, and their FDC average is 0.11. T11 has a single pair of leaf nodes at the bottom, and its FDC is 0.23. This suggests a logic tree with more pairs of leaf nodes at the bottom may have more interacting bits in the chromosome, and may be more susceptible to disruption and less GA ecient. Table 2 also shows T2 has the most unbalanced structure and the worst FDC (=0.1243), suggesting that a more balanced tree structure may benet GA search.

6. Discussion We have investigated how logic trees can be used to model decision processes. The use of GAs helps to speed up the search of a good tree for knowledge representation and building the decision model. This is consistent with earlier suggestions that GAs, although often limited, are sometimes a useful operations research technique [45], and improvement in understanding the factors leading to GA performance may increase the usefulness even further [32]. Our experiment shows that GAs are a good approximation technique. The maximum classication accuracy of the logic tree found by exhaustive search is 94.7% (=503/ 531) while the maximum classication accuracy

of the logic tree found by GAs is 93.8% (=498/ 531). Exhaustive search to nd the optimal of a logic tree took 15.45 hours and gave an accuracy of 94.7%. On the other hand, if we use a arbitrary set of parameter values in Table 1, GAs took an average of 1.5 minute (standard deviation = 2.15 minute) to converge and gave an average accuracy of 93% (standard deviation = 0.84%). Our study also shows that the GA approach oers a useful heuristic for decision-making. The logic tree obtained from GAs provides the user with an intuitive picture of the logical relationships among the variables in the decision process. In addition, the operators in a logic tree capture the sucient and necessary conditions for a decision, and the critical areas in a decision are summarized to assist the managers in making tradeos. For example, the two GA optimal trees we found from our experiment have indicated demand growth rate (v3) and nancial strength as important factors in aecting the new product entry timing decision. As shown in Fig. 6(a), the logical expression of the optimal tree for tree T10 is DEC = (v1 OR (v2 AND (NOT v5)) OR (v3 AND (NOT v7))). This suggests that a new product should be launched if one of the following conditions holds: (1) the position of the rm is dominant (v1), or (2) expected demand growth rate is high (v3) and cost of market development is low (NOT v7), or (3) nancial strength of the rm is strong (v2) and diusion rate across competitors is low (NOT v5). Similarly, based on the optimal tree of tree T1 (see Fig. 6(b)), where the logical expression is DEC = ((v2 AND v3) OR (v4 AND (NOT v5)) OR NOT v6), a new product should be launched if one of the following conditions holds: (1) the cannibalization rate is low (NOT v6), or (2) expected demand

608

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

growth rate is high (v3) and nancial strength is strong (v2), or (3) product life cycle is short (v4) and diusion rate across competitors is low (NOT v5). Both optimal trees suggest that a new product is launched if demand growth rate is high, but the demand growth rate factor may be considered in conjunction with either the cost of development factor or with the nancial strength factor. Both optimal trees also suggest that a new product is launched if nancial strength is strong, but the nancial strength factor may be considered in conjunction with either the diusion rate factor or with the demand growth rate factor. One caveat in the use of GAs is the assignment of chromosome bit string to represent trees may lead to problems in the tree search. The rst four bits of the chromosomes denote which of the eleven trees is used. Fig. 7 illustrates how this takes place. Suppose the upper chromosome is of tree T4 and has high classication accuracy. If one of its rst four bits mutates, it becomes a chromosome of tree T5. This change alters the entire expression, even though its logical operators may remain the same. As suggested in Fig. 7, tree T4

has the relationship x LEFT y at the root. When the tree has T4 as the structure, the variables ** and * are thrown into the null set. When tree T4 changes to T5, only variable * would be thrown into the null set. As a result, instead of throwing two variables into the null set as in tree T4, the change to tree T5 would have only caused only one variable to be thrown into the null set. This pre-order traversal problem is more serious if the bits are used to denote categories or groups that dier signicantly from one another. Our research here has examined the use of GA in modeling logic trees and how logic tree structure may aect the performance of GA. Future research may further look into how GAs can be used to deal with tree search for fuzzy data, where MIN and MAX operators can be used to model the sixteen R-relations. Hata and Yamato [20] have constructed logic functions made up of MIN and MAX operators to model fuzzy data. Hayashi et al. [21,22] have also looked at how fuzzy ID3 can be combined with AND/OR operators to learn data relationships. GAs may be applied to help with the search of the proper logical functions

Bits Representing Logical Operators

Non-leaf Node N1

Non-leaf Node N2

Non-leaf Node N3

Non-leaf Node N4

Non-leaf Node N5

Tree T4

Tree T5

Tree T4
AND

LEFT

Tree T5
AND AND

LEFT AND

AND AND AND AND

AND

AND

**

Note: In tree T4, both variable ** and variable * are thrown into null set.
When tree changes structure to T5, only variable * is thrown into null set

Fig. 7. Problems in the pre-order traversal of GAs chromosomes.

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

609

to represent the sixteen R-relations with the MIN, MAX operators or for the proper fuzzy ID3 to represent data relationships. Finally, we note that the problem we selected was of limited sizeseven variables. This allowed us to perform an exhaustive search on all possible tree structures and to illustrate in detail all facets of the GA application. For much larger problems the computational eort might be prohibitive, and we would have to select a subset of these structures. The length of the chromosome would depend on both the number of variables (leaf nodes) and the number of structures to be tested. The chromosome consists of three components: the bits representing the root, the bits representing the non-leaf nodes, and the bits indexing the tree structures to be tested. If there are N variables (leaf nodes) in the model and X trees to be tested, then the total number of bits in the chromosome would be the sum of three quantities: (i) 4 bits for the root node, as described above; (ii) 3 (N 2) bits for the non-leaf nodes, since there are N 2 non-leaf nodes, each of which requires 3 bits to index the 8 operators; (iii) [log2 X] + bits indexing the tree to be selected. The [ ] + function rounds up to the next integer. An alternative would be to use genetic programming [3]. In this case an initial set of trees would be combined by means of mutation and crossover to produce a set of new trees, and the principle of survival of the ttest would be invoked to select the next generation of trees. Thus, the variables could be more general than the Boolean variables considered here and the non-leaf and root nodes could represent more general operations than Boolean operations.

tree that best represents specic data on new product introduction. An advantage of logic trees is that they may suggest the structure of the decision processes being modeled. For example, they can identify cases in which pairs of variables are joined together and the relationships (e.g., AND, OR, IFTHEN) between the variables in the group. An unexpected advantage is that logic trees provide an eective foundation for GA search, whether measures by search eciency or by tness distance correlation. The strong relationship between these two quantities suggests that logic trees (as a knowledge structure) combined with GAs (as a search strategy) present an eective approach to operations research and knowledge modeling.

Appendix A. Tree equivalence arising from isomorphic tree structure According to Webster dictionary, the term isomorphic means being of identical or similar form, shape, or structure. Suppose Tree1 is ((x1 LEFT x2) AND3), and Tree2 is (x3 AND (x2 RIGHT x1)). Here Tree1 has a subtree (x1 LEFT x2) that is isomorphic to the subtree (x2 RIGHT x1) in Tree2. The extent of equivalence arising from isomorphic trees can be understood by computing the number of distinct trees. Let Yn and Zn be the number of distinct trees for trees with and without equivalence. Let n be the number of leaf nodes. For a tree with r leaves in the left subtree, where r = 1, . . . , (n 1), there are (n r) leaves in the right subtree [19] and we have Yn
n1 X r1

Y r Y nr Y r Y nr Y n Y n 2 2

when n is odd; when n is even:

7. Conclusion and future research Yn We have examined an important problem in knowledge modelingthe induction of knowledge structures from empirical data describing decision processes [5]. We have focused on a specic knowledge structure, logic trees, a specic knowledge management technique, GAs, and a specic problem domain, the introduction of new products. That is, we have used GAs to determine the logic

n1 X r1

Consider a tree with seven leaf nodes. Y1 = 1, Y2 = 2, Y3 = 4, Y4 = 16, Y5 = 48, Y6 = 192, Y7 = 704. The total number of distinct trees is 704. To compute Zn, we eliminate redundancy by restricting the number of leaves on the left to the root to be larger or equal to that on the right. Thus

610

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612

r 1; . . . 2 , and the [ ]+ function rounds up to the next integer. The number of distinct trees is given by
rn1 2

n1

Zn

X
r1

Z r Z nr :

Since trees with one leaf have exactly one branching, so Z1 = 1. Similarly, trees with two leaves have one branching, so Z2 = 1. Therefore we have (i) Z3 = Z1 Z2 = 1; (ii) Z4 = Z1 Z3 + Z2 Z2 = 2; (iii) Z5 = Z1 Z4 + Z2 Z3 = 3; (iv) Z6 = Z1 Z5 + Z2 Z4 + Z3 Z3 = 6; (v) Z7 = Z1 Z6 + Z2 Z5 + Z3 Z4 = 11. Fig. 1 shows the eleven tree structures associated with a sevennode tree.

Appendix B. Combinations of variable ordering for a logic tree A commutative relation (xi R xj) has the same result as that of (xj R xi), so that changing the order of xi and xj does not aect the truth table. The commutative operators under consideration are FALSE, TRUE, AND, OR, and XOR. For example, for a 3-node tree, the possible variable ordering are: 1 2 3, 3 2 1, 3 1 2, 2 1 3, 1 3 2, 2 3 1. However, 1 2 3 is commutative to 3 2 1, while 3 1 2 is commutative to 2 1 3, and 1 3 2 is commutative to 2 3 1. So after we have considered commutative relationships the possible variable ordering are: 1 2 3, 3 1 2, 1 3 2. In general, the number of variable ordering for an n-node tree is = n! 0:5n, where 2 n truncates to the integer when n is divided by 2. 2 For a tree with seven leaf nodes and with a structure that has one non-leaf node at the lowest levels, there are three pairs of variables that would be commutative, and the number of possible combinations is 630 (=7! 0.5 0.5 0.5). References
[1] L. Altenberg, Fitness distance correlation analysis: An instructive counterexample, in: T. Back (Ed.), Proceedings of the 7th International Conference on Genetic Algorithms, Morgan Kaufmann Publishers, 1997, pp. 57 64.

[2] J.B. Atkinson, A greedy randomized search heuristic for time-constrained vehicle scheduling and the incorporation of a learning strategy, Journal of the Operational Research Society 49 (7) (1998) 700708. [3] W. Banzhaf, P. Nordin, R.E. Keller, F.D. Francone, Genetic ProgrammingAn Introduction, Morgan Kaufman Publishers, Inc., San Francisco, CA, 1998. [4] A.S. Bickel, R.W. Bickel, Determination of near-optimum use of hospital diagnostic resources using the GENES genetic algorithm shell, Computers-in-Biology-and-Medicine 20 (1) (1990) 113. [5] M. Binbasioglu, Key features for model building decision support systems, European Journal of Operational Research 82 (1995) 422437. [6] R.W. Blanning, The sensitivity properties of hierarchical logic-based models, Decision Support Systems 6 (2) (1990) 8998. [7] B. Chai, T. Huang, X. Zhuang, Y. Zhao, J. Sklansky, Piecewise linear classiers using binary tree structure and genetic algorithm, Pattern Recognition 29 (11) (1996) 19051917. [8] P. Collard, C. Escazut, Fitness distance correlation in a dual genetic algorithm, ECAI 96, in: 12th European Conference on Articial Intelligence, Wiley, Chichester, UK, 1996, pp. 218222. [9] Y. Davidor, Epistasis variance: A viewpoint on GAhardness, in: G.J.E. Rawlins (Ed.), Foundations of Genetic Algorithms, vol. 1, Morgan Kaufmann, San Mateo, CA, USA, 1991, pp. 2325. [10] A.N. Edmonds, D. Burkhardt, O. Adjei, Genetic programming of fuzzy logic production rules, in: 1995 IEEE International Conference on Evolutionary Computation, New York, USA, vol. 2, 1995, pp. 765770. [11] L.J. Eshelman, R.A. Caruana, J.D. Schaer, Biases in the crossover landscape, in: J.D. Schaer (Ed.), Proceedings of the Third International Conference on Genetic Algorithms, George Mason University, June 47, 1989, Morgan Kaufmann Publishers, San Mateo, CA, USA, 1989, pp. 1019. [12] L.J. Eshelman, J.D. Schaer, Crossovers niche, in: R.K. Belew, L.B. Booker (Eds.), Proceedings of the Fourth International Conference on Genetic Algorithms, University of California, San Diego, July 1316, 1991, Morgan Kaufmann, San Mateo, CA, USA, 1991, pp. 914. [13] L.J. Eshelman, J.D. Schaer, Preventing premature convergence in genetic algorithms by preventing incest, in: S. Forrest (Ed.), Proceedings of the Fifth International Conference on Genetic Algorithms, University of Illinois at Urbana-Champaign, July 1721, 1993, Morgan Kaufmann, San Mateo, CA, USA, 1993, pp. 115122. [14] V. Estivill-Castro, Collaborative knowledge acquisition with a genetic algorithm, in: Ninth IEEE International Conference on Tools with Articial Intelligence, 1997 Proceedings, 1997, pp. 270277. [15] S. Forrest, Genetic algorithms: Principles of natural selection applied to computation, Science 261 (1993) 872 878.

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612 [16] S. Forrest, M. Mitchell, What makes a problem hard for a genetic algorithm? Some anonymous results and their explanation, Machine Learning 13 (1993) 285319. [17] D.E. Goldberg, Genetic and evolutionary algorithms come of age, Communications of the ACM 37 (3) (1994) 113 119. [18] D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, 1989. [19] R.P. Grimaldi, Discrete and Combinational MathematicsAn Applied Introduction, Addison-Wesley, 1989. [20] Y. Hata, K. Yamato, Multiplied-valued logic functions represented by TSUM, TPRODUCT, NOT and variables, in: Proceedings of the Twenty-Third International Symposium on Multiple-valued Logic, 1993, pp. 222227. [21] I. Hayashi, E. Naito, J. Ozawa, N. Wakami, Fuzzy retrieval system using query networks and AND/OR operators with learning mechanism, International Journal of Knowledge-Based Intelligent Engineering Systems 1 (1) (1997) 2235. [22] I. Hayashi, J. Ozawa, A proposal of fuzzy ID3 with ability of learning for AND/OR operators, in: Software Computing in Intelligent Systems and Information Processing, Proceedings of 1996 Asian Fuzzy Systems Symposium, Kenting, Taiwan, in December 1114, 1996, pp. 24 29. [23] J.H. Holland, Adaptation in Natural and Articial Systems, The University of Michigan Press, Ann Arbor, MI, USA, 1975. [24] J.P. Ignizio, J.R. Soltys, Simultaneous design and training of ontogenic neural network classiers, Computers and Operations Research 23 (6) (1996) 535546. [25] Y. Ikkai, M. Inoue, T. Ohkawa, N. Komoda, A learning method of scheduling knowledge by genetic algorithm, in: Proceedings 1995 INRIA/IEEE Symposium on Emerging Technologies and Factory Automation, ETFA95, Los Alamitos, CA, USA, vol. 1, 1995, pp. 641648. [26] T. Jones, S. Forrest, Fitness distance correlation as a measure of problem diculty for genetic algorithms, in: L.J. Eshelman (Ed.), Proceedings of the Sixth International Conference on Genetic Algorithms, University of Pittsburg, July 1519, 1995, Morgan Kaufmann, San Francisco, CA, USA, 1995, pp. 184192. [27] L. Kallel, D. Naudts, M. Schoenauer, On functions with a given tness-distance relationProceedings of the 1999 Congress on Evolutionary Computation, vol. 3, IEEE Press, 1999, pp. 19101916. [28] H. Kargupta, Signal to noise, crosstalk and long range problem diculty in genetic algorithms, in: L.J. Eshelman (Ed.), Proceedings of the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, San Francisco, CA, USA, 1995, pp. 193200. [29] A. Kershenbaum, When genetic algorithms work best, INFORMS Journal on Computing 9 (3) (1997) 254255. [30] J.U. Kim, Y.D. Kim, Simulated annealing and genetic algorithms for scheduling products with multi-level product structure, Computers and Operational Research 23 (9) (1996) 857868.

611

[31] G.J. Koehler, New directions in genetic algorithm theory, in: R.W. Blanning, M.J. Shaw (Eds.), Articial Intelligence and Management Science, Baltzer Publishers, Amsterdam, NL, 1997, pp. 4968. [32] P. Langley, H. Simon, Applications of machine learning and rule induction, Communications of the ACM 38 (11) (1995) 5564. [33] M. Lieberman, D. Montgomery, First-mover (dis)advantages: Retrospective and link with the resource-based view, Strategic Management Journal 19 (12) (1998) 11111125. [34] B. Mak, R.W. Blanning, An empirical measure of element contribution in neural networks, IEEE Transactions on Systems, Man, and Cybernetics 28 (4) (1998) 561564. [35] B. Manderick, M. de Weger, P. Spiessens, The genetic algorithm and the structure of the tness landscape, in: R.K. Belew, L.B. Booker (Eds.), Proceedings of the Fourth International Conference on Genetic Algorithms, University of California, San Diego, July 1316, 1991, Morgan Kaufmann, San Mateo, CA, USA, 1991, pp. 143150. [36] J.A. Miller, W.D. Potter, R.V. Gandham, C.N. Lapena, An evaluation of local improvement operators for genetic algorithms, IEEE Transactions on Systems, Man and Cybernetics 23 (5) (1993) 13401351. [37] M. Mitchell, An Introduction to Genetic Algorithms, MIT Press, Cambridge, MA, USA, 1996. [38] S. Mullei, P. Beling, Induction of rule-based scoring functions, SMC98 Conference Proceedings 1998 IEEE International Conference on Systems, Man, and Cybernetics, New York, vol. 5, 1998, pp. 29682973. [39] Y. Pao, Adaptive Pattern Recognition and Neural Networks, Addison-Wesley, Reading, MA, USA, 1989. [40] G. Premkumar, C.H. Chu, H. Chou, Telecommunications network designcomparison of alternative approaches, Decision Sciences 31 (2) (2000) 483506. [41] J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers, San Francisco, CA, USA, 1993. [42] J.R. Quinlan, Induction of decision trees, Machine Learning 1 (1986) 81106. [43] J.R. Quinlan, Generating production rules from decision trees, in: Proceedings of the International Joint Conference on Articial Intelligence, 1987, pp. 304307. [44] C.R. Reeves, Genetic algorithms for the operations researcher, INFORMS Journal on Computing 9 (3) (1997) 231250. [45] C.R. Reeves, Genetic algorithms: No panacea, but a valuable tool for the operations researcher, INFORMS Journal on Computing 9 (3) (1997) 263265. [46] S. Rochet, G. Venturini, M. Slimane, E.M. El Kharoubi, A critical and empirical study of epistasis measures for predicting GA performances: A summary, in: J.-K. Hao, E. Lutton, E. Ronald, M. Schoenauer, D. Snyers (Eds.), Articial Evolution97, Third European Conference, AE 97, Nimes, France, October 22024, 1997, Selected Papers, Springer-Verlag, Heidelberg, 1998, pp. 287299. [47] H. Seidl, Deciding equivalence of nite tree automata, SIAM Journal on Computing 19 (3) (1990) 424437.

612

B. Mak et al. / European Journal of Operational Research 170 (2006) 597612 and automated-phases, in: 1995 IEEE International Conference on Evolutionary Computation, New York, USA, vol. 2, 1995, pp. 771776. P. Thomson, J.F. Miller, Symbolic method for simplifying AND-EXOR representations of Boolean functions using a binary-decision technique and a genetic algorithm, IEE Proceedings Computers and Digital Techniques 143 (2) (1996) 151155. E. Turban, J.E. Aronson, Decision Support Systems and Intelligent Systems, sixth ed., Prentice Hall, Upper Saddle River, NJ, USA, 2001. H.G. Wittkemper, M. Steiner, Using neural network to forecast the systematic risk of stocks, European Journal of Operational Research 90 (3) (1996) 577588. J. Yang, V. Honavar, Feature subset selection using a genetic algorithm, IEEE Intelligent Systems 13 (2) (1998) 4449. G. Zhou, M. Gen, Genetic algorithm approach on multicriteria minimum spanning tree problem, European Journal of Operational Research 11 (1) (1999) 141 152.

[48] R. Setiono, H. Liu, Symbolic representation of neural networks, IEEE Computer 29 (3) (1996) 7177. [49] R.S. Sexton, R.E. Dorsey, J.D. Johnson, Optimization of neural networks: A comparative analysis of the genetic algorithm and simulated annealing, European Journal of Operational Research 114 (3) (1999) 589601. [50] K. Sorensen, G.K. Janssens, Data mining with genetic algorithms on binary trees, European Journal of Operational Research 151 (2) (2003) 253264. [51] W.E. Spangler, J.H. May, L.G. Vargas, Choosing datamining methods for multiple classication: Representational and performance measurement implications for decision support, Journal of Management Information Systems 16 (1) (1999) 3762. [52] G. Syswerda, Uniform crossover in genetic algorithms, in: J.D. Schaer (Ed.), Proceedings of the Third International Conference on Genetic Algorithms, George Mason University, June 47, 1989, Morgan Kaufmann, San Mateo, CA, USA, 1989, pp. 29. [53] T. Terano, Y. Ishino, Marketing data analysis using inductive learning and genetic algorithms with interactive-

[54]

[55]

[56]

[57]

[58]