Introduction Algorithm Theory Why GA?

Applications

Genetic Algorithm
Saif Hasan Sagar Chordia Rahul Varshneya

February 6, 2012

Guide Pushpak Bhattacharyya

1 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

INTRODUCTION

2 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Introduction

Genetic algorithms are a family of computational models belonging to the class of evolutionary algorithms, part of articial intelligence

3 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Introduction

Genetic algorithms are a family of computational models belonging to the class of evolutionary algorithms, part of articial intelligence These algorithms encode a potential solution to a specic problem on a simple chromosome like data structure

3 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Introduction

Genetic algorithms are a family of computational models belonging to the class of evolutionary algorithms, part of articial intelligence These algorithms encode a potential solution to a specic problem on a simple chromosome like data structure Uses techniques inspired by natural evolution such as inheritance, mutation, selection and crossover

3 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Introduction

Genetic algorithms are a family of computational models belonging to the class of evolutionary algorithms, part of articial intelligence These algorithms encode a potential solution to a specic problem on a simple chromosome like data structure Uses techniques inspired by natural evolution such as inheritance, mutation, selection and crossover They are often viewed as function optimizers

3 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

History
First appeared in 1950s and early 1960s while biologists were explicitly seeking to the model of natural evolution

4 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

History
First appeared in 1950s and early 1960s while biologists were explicitly seeking to the model of natural evolution Idea of inheritance and mutation introduced by Ingo Rechenberg which is termed as evolution strategy (1965)

4 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

History
First appeared in 1950s and early 1960s while biologists were explicitly seeking to the model of natural evolution Idea of inheritance and mutation introduced by Ingo Rechenberg which is termed as evolution strategy (1965) M.J. Walsh introduced evolutionary programming (1966)

4 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

History
First appeared in 1950s and early 1960s while biologists were explicitly seeking to the model of natural evolution Idea of inheritance and mutation introduced by Ingo Rechenberg which is termed as evolution strategy (1965) M.J. Walsh introduced evolutionary programming (1966) Later versions introduced population which leads to the Genetic Algorithms

4 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

History
First appeared in 1950s and early 1960s while biologists were explicitly seeking to the model of natural evolution Idea of inheritance and mutation introduced by Ingo Rechenberg which is termed as evolution strategy (1965) M.J. Walsh introduced evolutionary programming (1966) Later versions introduced population which leads to the Genetic Algorithms In 1975 John Holland published book Adaptation in Natural and Articial System. This was the rst book to represent concept of adaptive digital systems using mutation, selection and crossover.
4 / 42 Saif Hasan, Sagar Chordia, Rahul Varshneya Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Motivation
Evolution is very powerful theory since biological principles like common descent and selective breeding have been used for the benet of humans

5 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Motivation
Evolution is very powerful theory since biological principles like common descent and selective breeding have been used for the benet of humans Living organisms are consummate problem solvers. They exhibit a versatility that puts the best computer programs to shame.

5 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Motivation
Evolution is very powerful theory since biological principles like common descent and selective breeding have been used for the benet of humans Living organisms are consummate problem solvers. They exhibit a versatility that puts the best computer programs to shame. Most organisms evolve by means of two primary processes: natural selection and sexual reproduction. The rst determines which members of population survive and reproduce, and the second ensures mixing and recombination among the genes of their ospring. Similar analogy is used in GA.

5 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Terminology

Search space/ State space : the space of all feasible solutions.

6 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Terminology

Search space/ State space : the space of all feasible solutions. Chromosome : a set of genes; a chromosome contains the solution in form of genes.

6 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Terminology

Search space/ State space : the space of all feasible solutions. Chromosome : a set of genes; a chromosome contains the solution in form of genes. Population : a set of solutions (or individuals/chromosomes).

6 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Terminology

Search space/ State space : the space of all feasible solutions. Chromosome : a set of genes; a chromosome contains the solution in form of genes. Population : a set of solutions (or individuals/chromosomes). Generation : the process of evaluation, selection, recombination and mutation.

6 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Introduction History Motivation Terminology

Terminology

Search space/ State space : the space of all feasible solutions. Chromosome : a set of genes; a chromosome contains the solution in form of genes. Population : a set of solutions (or individuals/chromosomes). Generation : the process of evaluation, selection, recombination and mutation. Fitness : the value assigned to an individual based on how far or close it is from the solution; greater the tness value better the solution it contains.

6 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

ALGORITHM

7 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm Choose the initial population of individuals

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm Choose the initial population of individuals Evaluate the tness of each individual in population

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm Choose the initial population of individuals Evaluate the tness of each individual in population Repeat until termination condition satised:

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm Choose the initial population of individuals Evaluate the tness of each individual in population Repeat until termination condition satised:
Selection: Select the individuals with greater tness for reproduction

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm Choose the initial population of individuals Evaluate the tness of each individual in population Repeat until termination condition satised:
Selection: Select the individuals with greater tness for reproduction Crossover: Breed new individuals through crossover

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm Choose the initial population of individuals Evaluate the tness of each individual in population Repeat until termination condition satised:
Selection: Select the individuals with greater tness for reproduction Crossover: Breed new individuals through crossover Mutation: Apply probabilistic mutation on new individuals

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm Choose the initial population of individuals Evaluate the tness of each individual in population Repeat until termination condition satised:
Selection: Select the individuals with greater tness for reproduction Crossover: Breed new individuals through crossover Mutation: Apply probabilistic mutation on new individuals Form a new population with these osprings.

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Algorithm
Psuedocode of Genetics Algorithm Choose the initial population of individuals Evaluate the tness of each individual in population Repeat until termination condition satised:
Selection: Select the individuals with greater tness for reproduction Crossover: Breed new individuals through crossover Mutation: Apply probabilistic mutation on new individuals Form a new population with these osprings.

Terminate

8 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Flow Chart

9 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding
Before a genetic algorithm can be put to work on any problem, a method is needed to encode potential solutions to that problem in a form so that a computer can process.

10 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding
Before a genetic algorithm can be put to work on any problem, a method is needed to encode potential solutions to that problem in a form so that a computer can process. Common approaches are: Binary Encoding : every chromosome is a string of 0 or 1

10 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding
Before a genetic algorithm can be put to work on any problem, a method is needed to encode potential solutions to that problem in a form so that a computer can process. Common approaches are: Binary Encoding : every chromosome is a string of 0 or 1 Permutation Encoding : every chromosome is a string of numbers that represent position in a sequence

10 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding
Before a genetic algorithm can be put to work on any problem, a method is needed to encode potential solutions to that problem in a form so that a computer can process. Common approaches are: Binary Encoding : every chromosome is a string of 0 or 1 Permutation Encoding : every chromosome is a string of numbers that represent position in a sequence Tree Encoding : a tree structure represents the chromosome

10 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding
Before a genetic algorithm can be put to work on any problem, a method is needed to encode potential solutions to that problem in a form so that a computer can process. Common approaches are: Binary Encoding : every chromosome is a string of 0 or 1 Permutation Encoding : every chromosome is a string of numbers that represent position in a sequence Tree Encoding : a tree structure represents the chromosome Value Encoding : every chromosome is a sequence of some values (real numbers, characters or objects)

10 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Binary Encoding : Suppose we have a knapsack of capacity C and N items, then we can encode this problem as follows

11 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Binary Encoding : Suppose we have a knapsack of capacity C and N items, then we can encode this problem as follows
Chromosome, in this case, is a string of 0s and 1s with N bits

11 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Binary Encoding : Suppose we have a knapsack of capacity C and N items, then we can encode this problem as follows
Chromosome, in this case, is a string of 0s and 1s with N bits Represent item i of problem with i th bit in the chromosome

11 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Binary Encoding : Suppose we have a knapsack of capacity C and N items, then we can encode this problem as follows
Chromosome, in this case, is a string of 0s and 1s with N bits Represent item i of problem with i th bit in the chromosome i th bit is 1 i i th item has been selected, 0 otherwise.

11 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Binary Encoding : Suppose we have a knapsack of capacity C and N items, then we can encode this problem as follows
Chromosome, in this case, is a string of 0s and 1s with N bits Represent item i of problem with i th bit in the chromosome i th bit is 1 i i th item has been selected, 0 otherwise. The set of all such chromosomes (2N ) is the solution space of the problem.

11 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Binary Encoding : Suppose we have a knapsack of capacity C and N items, then we can encode this problem as follows
Chromosome, in this case, is a string of 0s and 1s with N bits Represent item i of problem with i th bit in the chromosome i th bit is 1 i i th item has been selected, 0 otherwise. The set of all such chromosomes (2N ) is the solution space of the problem.
Chromosome 1: 1 0 1 1 0 0 1 0 1 1 0 0 1 0 1 0 1 1 1 0 0 1 0 1 Chromosome 2: 1 1 1 1 1 1 1 0 0 0 0 0 1 1 0 0 0 0 0 1 1 1 1 1

The example shown above has 24 items (and therefore 24 bits) with item1 selected in both chromosome 1 and 2 whereas item2 is selected in chromosome 2 but not in chromosome 1.

11 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples

Permutation Encoding : Travelling Salesman Problem

12 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples

Permutation Encoding : Travelling Salesman Problem

Problem descripition : There are cities and given distances between them. Travelling salesman has to visit all of them, but he doesnt want to travel more than necessary. Find a sequence of cities with a minimal travelled distance.

12 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples

Permutation Encoding : Travelling Salesman Problem

Problem descripition : There are cities and given distances between them. Travelling salesman has to visit all of them, but he doesnt want to travel more than necessary. Find a sequence of cities with a minimal travelled distance.
Chromosome A: Chromosome B: 1 8 5 5 3 6 2 7 6 2 4 3 7 1 9 4 8 9

Encoding : Here, encoded chromosomes describe the order of cities the salesman visits. For example, in chromosome A, the salesman visits city-1 followed by city-5 followed by city-3 and so on.

12 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Tree Encoding : Genetic Programming

13 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Tree Encoding : Genetic Programming
In tree encoding, every chromosome is a tree of some objects, such as functions or commands in programming language.

13 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Tree Encoding : Genetic Programming
In tree encoding, every chromosome is a tree of some objects, such as functions or commands in programming language. Tree encoding is useful for evolving programs or any other structures that can be encoded in trees.

13 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Tree Encoding : Genetic Programming
In tree encoding, every chromosome is a tree of some objects, such as functions or commands in programming language. Tree encoding is useful for evolving programs or any other structures that can be encoded in trees. The crossover and mutation can be done relatively easy way .

13 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Encoding Examples
Tree Encoding : Genetic Programming
In tree encoding, every chromosome is a tree of some objects, such as functions or commands in programming language. Tree encoding is useful for evolving programs or any other structures that can be encoded in trees. The crossover and mutation can be done relatively easy way .

Image courtesy: http://www.myreaders.info/09 Genetic Algorithms.pdf

13 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Operations of Genetic Algorithm

Genetic operators used in GA maintain genetic diversity.

14 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Operations of Genetic Algorithm

Genetic operators used in GA maintain genetic diversity. Genetic diversity or variation is a necessity for evolution.

14 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Operations of Genetic Algorithm

Genetic operators used in GA maintain genetic diversity. Genetic diversity or variation is a necessity for evolution. Genetic operators are analogous to those which occur in the natural world:

14 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Operations of Genetic Algorithm

Genetic operators used in GA maintain genetic diversity. Genetic diversity or variation is a necessity for evolution. Genetic operators are analogous to those which occur in the natural world:
Reproduction (or Selection)

14 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Operations of Genetic Algorithm

Genetic operators used in GA maintain genetic diversity. Genetic diversity or variation is a necessity for evolution. Genetic operators are analogous to those which occur in the natural world:
Reproduction (or Selection) Crossover (or Recombination)

14 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Operations of Genetic Algorithm

Genetic operators used in GA maintain genetic diversity. Genetic diversity or variation is a necessity for evolution. Genetic operators are analogous to those which occur in the natural world:
Reproduction (or Selection) Crossover (or Recombination) Mutation

14 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection
Concept : From the population, the chromosomes are selected to be parents to crossover and produce ospring.

15 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection
Concept : From the population, the chromosomes are selected to be parents to crossover and produce ospring. Problem : How to select these chromosomes ?

15 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection
Concept : From the population, the chromosomes are selected to be parents to crossover and produce ospring. Problem : How to select these chromosomes ? Hint : According to Charles Darwins evolution theory survival of the ttest - the best ones should survive and create new ospring.

15 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection
Concept : From the population, the chromosomes are selected to be parents to crossover and produce ospring. Problem : How to select these chromosomes ? Hint : According to Charles Darwins evolution theory survival of the ttest - the best ones should survive and create new ospring. Solution : Fitness function quanties the optimality of a solution (chromosome) so that a particular solution may be ranked against all the other solutions. The function depicts the closeness of a given solution to the desired result.

15 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection

Popular methods of selection include :

16 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection

Popular methods of selection include : Roulette-wheel selection

16 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection

Popular methods of selection include : Roulette-wheel selection Tournament selection

16 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection

Popular methods of selection include : Roulette-wheel selection Tournament selection Rank selection

16 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection

Popular methods of selection include : Roulette-wheel selection Tournament selection Rank selection Steady-state selection

16 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection

Popular methods of selection include : Roulette-wheel selection Tournament selection Rank selection Steady-state selection Boltzmann selection

16 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Reproduction or Selection

Popular methods of selection include : Roulette-wheel selection Tournament selection Rank selection Steady-state selection Boltzmann selection Scaling selection

16 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Roulette-Wheel Selection
Concept : the chance of an individuals being selected is proportional to its tness, greater or less than its competitors tness.

17 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Roulette-Wheel Selection
Concept : the chance of an individuals being selected is proportional to its tness, greater or less than its competitors tness.

17 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Roulette-Wheel Selection
Concept : the chance of an individuals being selected is proportional to its tness, greater or less than its competitors tness.

Implementation : Probability of selection of i th individual is: pi = Nfi f

j =1 j

where fi :tness of i th individual, N : number of individuals

Image courtesy: http://www.myreaders.info/09 Genetic Algorithms.pdf 17 / 42 Saif Hasan, Sagar Chordia, Rahul Varshneya Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Elitist Selection

Concept : Most t members of each generation are guaranteed to be selected for next generation.

18 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Elitist Selection

Concept : Most t members of each generation are guaranteed to be selected for next generation. Advantages :

18 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Elitist Selection

Concept : Most t members of each generation are guaranteed to be selected for next generation. Advantages :
Ensures that the best solution found so far is not lost due to crossover and mutation.

18 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Elitist Selection

Concept : Most t members of each generation are guaranteed to be selected for next generation. Advantages :
Ensures that the best solution found so far is not lost due to crossover and mutation. Speeds up convergence once a good solution has been discovered.

18 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
Concept : Selects genes from parent chromosomes, combines them and creates a new ospring.

19 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
Concept : Selects genes from parent chromosomes, combines them and creates a new ospring. Idea : New chromosome may be better than both of the parents if it takes the best characteristics from each of them

19 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
Concept : Selects genes from parent chromosomes, combines them and creates a new ospring. Idea : New chromosome may be better than both of the parents if it takes the best characteristics from each of them
Consider the two parents selected for crossover.

19 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
Concept : Selects genes from parent chromosomes, combines them and creates a new ospring. Idea : New chromosome may be better than both of the parents if it takes the best characteristics from each of them
Consider the two parents selected for crossover.

Interchange the parents chromosomes after crossover points. The osprings produced are :

Image courtesy: http://www.myreaders.info/09 Genetic Algorithms.pdf

19 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
The Crossover operators are of many types.

20 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
The Crossover operators are of many types.
Single-Point crossover

20 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
The Crossover operators are of many types.
Single-Point crossover Two Point crossover

20 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
The Crossover operators are of many types.
Single-Point crossover Two Point crossover Uniform crossover

20 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
The Crossover operators are of many types.
Single-Point crossover Two Point crossover Uniform crossover Arithmetic crossover

20 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
The Crossover operators are of many types.
Single-Point crossover Two Point crossover Uniform crossover Arithmetic crossover

Which Crossover operator is to be selected is based on chromosome encoding

20 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
The Crossover operators are of many types.
Single-Point crossover Two Point crossover Uniform crossover Arithmetic crossover

Which Crossover operator is to be selected is based on chromosome encoding Specic crossover made for a specic problem can improve performance of the genetic algorithm

20 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover
The Crossover operators are of many types.
Single-Point crossover Two Point crossover Uniform crossover Arithmetic crossover

Which Crossover operator is to be selected is based on chromosome encoding Specic crossover made for a specic problem can improve performance of the genetic algorithm Some research suggests more than two parents are better to reproduce a good quality chromosome (Eiben, A.E. et al (1994),
Ting, Chuan-Kang (2005))

20 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Two-Point Crossover
Operation : randomly select two crossover points within a chromosome, then interchange the two parent chromosomes between these points to produce two new ospring.
Consider the two parents selected for crossover.

21 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Uniform Crossover
Operation : mixing ratio decides the contribution of each parent to the gene values in the ospring chromosomes. Advantage : allows the parent chromosomes to be mixed at the gene level rather than the segment level
Consider the two parents selected for crossover.

If the mixing ratio is 0.5 approximately, then the possible set of osprings after crossover would be :

Image courtesy: http://www.myreaders.info/09 Genetic Algorithms.pdf 22 / 42 Saif Hasan, Sagar Chordia, Rahul Varshneya Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation
Concept : Mutation alters one or more gene values in a chromosome from its initial state.

23 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation
Concept : Mutation alters one or more gene values in a chromosome from its initial state. Advantages :

23 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation
Concept : Mutation alters one or more gene values in a chromosome from its initial state. Advantages :
Mutation can generate new genes values not already present in sample space which can lead to better solution.

23 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation
Concept : Mutation alters one or more gene values in a chromosome from its initial state. Advantages :
Mutation can generate new genes values not already present in sample space which can lead to better solution. Randomness introduced by mutation helps in searching for global optima solutions and not geting stuck in local optima. (premature convergence).

23 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation
Concept : Mutation alters one or more gene values in a chromosome from its initial state. Advantages :
Mutation can generate new genes values not already present in sample space which can lead to better solution. Randomness introduced by mutation helps in searching for global optima solutions and not geting stuck in local optima. (premature convergence).

Operators : Mutation operators are of many type :

23 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation
Concept : Mutation alters one or more gene values in a chromosome from its initial state. Advantages :
Mutation can generate new genes values not already present in sample space which can lead to better solution. Randomness introduced by mutation helps in searching for global optima solutions and not geting stuck in local optima. (premature convergence).

Operators : Mutation operators are of many type :

one simple way is, Flip Bit.

23 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation
Concept : Mutation alters one or more gene values in a chromosome from its initial state. Advantages :
Mutation can generate new genes values not already present in sample space which can lead to better solution. Randomness introduced by mutation helps in searching for global optima solutions and not geting stuck in local optima. (premature convergence).

Operators : Mutation operators are of many type :

one simple way is, Flip Bit. the others are Boundary, Uniform, and Gaussian.

23 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation
Concept : Mutation alters one or more gene values in a chromosome from its initial state. Advantages :
Mutation can generate new genes values not already present in sample space which can lead to better solution. Randomness introduced by mutation helps in searching for global optima solutions and not geting stuck in local optima. (premature convergence).

Operators : Mutation operators are of many type :

one simple way is, Flip Bit. the others are Boundary, Uniform, and Gaussian.

Operators are selected based on encoding of chromosomes.

23 / 42 Saif Hasan, Sagar Chordia, Rahul Varshneya Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Flip Bit
The mutation operator simply inverts the value of the chosen gene i.e. 0 goes to 1 and 1 goes to 0. Consider the two original osprings selected for mutation.

The Mutated O-spring produced are :

Image courtesy: http://www.myreaders.info/09 Genetic Algorithms.pdf

24 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Parameters of Genetic Algorithm

There are three basic parameters of Genetic Algorithm.

25 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Parameters of Genetic Algorithm

There are three basic parameters of Genetic Algorithm. Crossover Probability

25 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Parameters of Genetic Algorithm

There are three basic parameters of Genetic Algorithm. Crossover Probability Mutation Probability

25 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Parameters of Genetic Algorithm

There are three basic parameters of Genetic Algorithm. Crossover Probability Mutation Probability Population Size

25 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover Probability
Denition : Crossover probability represents how often crossover is performed.

26 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover Probability
Denition : Crossover probability represents how often crossover is performed. Constraint :

26 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover Probability
Denition : Crossover probability represents how often crossover is performed. Constraint :
If the crossover rate is too high, high performance strings are eliminated faster than selection can produce improvements.

26 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover Probability
Denition : Crossover probability represents how often crossover is performed. Constraint :
If the crossover rate is too high, high performance strings are eliminated faster than selection can produce improvements. A low crossover rate may cause stagnation due to the lower exploration rate.

26 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover Probability
Denition : Crossover probability represents how often crossover is performed. Constraint :
If the crossover rate is too high, high performance strings are eliminated faster than selection can produce improvements. A low crossover rate may cause stagnation due to the lower exploration rate.

Solution : Crossover rate generally should be high, about 80%-95%.

26 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Crossover Probability
Denition : Crossover probability represents how often crossover is performed. Constraint :
If the crossover rate is too high, high performance strings are eliminated faster than selection can produce improvements. A low crossover rate may cause stagnation due to the lower exploration rate.

Solution : Crossover rate generally should be high, about 80%-95%. Some results show that for some problems crossover rate about 60% is the best.

26 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation Probability

Dention : Mutation probability represents how often mutation is performed.

27 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation Probability

Dention : Mutation probability represents how often mutation is performed. Constraints :

27 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation Probability

Dention : Mutation probability represents how often mutation is performed. Constraints :

A very small mutation rate may lead to convergence to local optima areas.

27 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation Probability

Dention : Mutation probability represents how often mutation is performed. Constraints :

A very small mutation rate may lead to convergence to local optima areas. A mutation rate that is too high results in almost random search.

27 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Mutation Probability

Dention : Mutation probability represents how often mutation is performed. Constraints :

A very small mutation rate may lead to convergence to local optima areas. A mutation rate that is too high results in almost random search.

Solution :Best rates reported are about 0.5%-1%.

27 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Population Size

Denition : Number of chromosomes in population (in one generation).

28 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Population Size

Denition : Number of chromosomes in population (in one generation). Constraints :

28 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Population Size

Denition : Number of chromosomes in population (in one generation). Constraints :

Too few chromosomes implies GA have a few possibilities to perform crossover and only a small part of search space is explored.

28 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Population Size

Denition : Number of chromosomes in population (in one generation). Constraints :

Too few chromosomes implies GA have a few possibilities to perform crossover and only a small part of search space is explored. Too many chromosomes implies GA slows down.

28 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Algorithm Encoding Operations of GA Parameters of GA

Population Size

Denition : Number of chromosomes in population (in one generation). Constraints :

Too few chromosomes implies GA have a few possibilities to perform crossover and only a small part of search space is explored. Too many chromosomes implies GA slows down.

Solution : Good population size is about 20-30, however sometimes sizes 50-100 are reported as best.

28 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Why Genetic Algorithms Work?

29 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Schema and HyperPlane

Schema - solution string with some blank elds eg: 01***********

30 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Schema and HyperPlane

Schema - solution string with some blank elds eg: 01*********** Solution is combination of these schemas. Schema represents a particular component of solution.

30 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Schema and HyperPlane

Schema - solution string with some blank elds eg: 01*********** Solution is combination of these schemas. Schema represents a particular component of solution.
Solution space : N-dimensional HyperCube

30 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Schema and HyperPlane

Schema - solution string with some blank elds eg: 01*********** Solution is combination of these schemas. Schema represents a particular component of solution.
Solution space : N-dimensional HyperCube Schema : HyperPlane

30 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Schema and HyperPlane

Schema - solution string with some blank elds eg: 01*********** Solution is combination of these schemas. Schema represents a particular component of solution.
Solution space : N-dimensional HyperCube Schema : HyperPlane

In 3D cube, 0** represent front face.

30 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Schema and HyperPlane

Schema - solution string with some blank elds eg: 01*********** Solution is combination of these schemas. Schema represents a particular component of solution.
Solution space : N-dimensional HyperCube Schema : HyperPlane

In 3D cube, 0** represent front face. There is competition between Schema with n bit values in same positions.eg:- 00*, 01*, 10*, 11* are competing Winner is the schema with highest tness.
30 / 42 Saif Hasan, Sagar Chordia, Rahul Varshneya Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Implicit Parallelism

A solution string belongs to many HyperPlanes (2N 1 ). eg: 010 belongs to 0** , *1*, **0

31 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Implicit Parallelism

A solution string belongs to many HyperPlanes (2N 1 ). eg: 010 belongs to 0** , *1*, **0 Single Evaluation of string leads to evaluation of dierent hyperplanes in an implicitly parallel fashion (John Holland 1975);

31 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Implicit Parallelism

A solution string belongs to many HyperPlanes (2N 1 ). eg: 010 belongs to 0** , *1*, **0 Single Evaluation of string leads to evaluation of dierent hyperplanes in an implicitly parallel fashion (John Holland 1975); Evaluation of population of strings, samples far more hyperplanes as compared to number of strings contained in the population.

31 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

Implicit Parallelism

A solution string belongs to many HyperPlanes (2N 1 ). eg: 010 belongs to 0** , *1*, **0 Single Evaluation of string leads to evaluation of dierent hyperplanes in an implicitly parallel fashion (John Holland 1975); Evaluation of population of strings, samples far more hyperplanes as compared to number of strings contained in the population. These cumulative eects provides statistical information to GA about any particular subset of hyperplanes.

31 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

The Schema Theorem

32 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

The Schema Theorem

The Schema Theorem (Holland 1992; Goldberg 1989). It provides a lower bound on the change in the sample rate for a single hyperplane from generation t to generation t + 1.

32 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

The Schema Theorem

The Schema Theorem (Holland 1992; Goldberg 1989). It provides a lower bound on the change in the sample rate for a single hyperplane from generation t to generation t + 1. Equation:

32 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Schema and HyperPlane Implicit Parallelism The Schema Theorem

The Schema Theorem

The Schema Theorem (Holland 1992; Goldberg 1989). It provides a lower bound on the change in the sample rate for a single hyperplane from generation t to generation t + 1. Equation:

Building Blocks Hypothesis (Holland, 1975; Gold-berg, 1989) Low-order, highly-t schemas recombine to form even better schemas. In Goldbergs words, we construct better and better strings from the best partial solutions of past samplings

32 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

WHY GA?

33 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Advantages

Implicit Parallelism - Solution Space is explored in multiple directions (GoldBerg - GA in Search and Optimization)

34 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Advantages

Implicit Parallelism - Solution Space is explored in multiple directions (GoldBerg - GA in Search and Optimization) Nonlinear problems -Large Solution space, but GA are ideal.(Forrest - 1993 Genetic Algorithm)

34 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Advantages

Implicit Parallelism - Solution Space is explored in multiple directions (GoldBerg - GA in Search and Optimization) Nonlinear problems -Large Solution space, but GA are ideal.(Forrest - 1993 Genetic Algorithm) Works on complex landscape (discontinuous, noisy, changing with time) (John Koza - Genetic Programming IV 2004)

34 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Advantages

Dilemma of global optimum vs many local optima. GA strike perfect balance (John Holland - Genetic Algorithm 1992)

35 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Advantages

Dilemma of global optimum vs many local optima. GA strike perfect balance (John Holland - Genetic Algorithm 1992) GA can manipulate many parameters simultaneously (Forrest Genetic algorithms: principles of natural selection applied to computation. 1993)

35 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Advantages

Dilemma of global optimum vs many local optima. GA strike perfect balance (John Holland - Genetic Algorithm 1992) GA can manipulate many parameters simultaneously (Forrest Genetic algorithms: principles of natural selection applied to computation. 1993)

GA dont have specic knowledge of problem. All possible search pathways are considered in GA.(John Koza - Genetic Programming III 1999)

35 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Disadvantages

Computationally expensive and time consuming

36 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Disadvantages

Computationally expensive and time consuming Issues in representation of problem

36 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Disadvantages

Computationally expensive and time consuming Issues in representation of problem Proper writing of tness function

36 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Disadvantages

Computationally expensive and time consuming Issues in representation of problem Proper writing of tness function Proper values of size of population, crossover and mutation rate

36 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Disadvantages

Computationally expensive and time consuming Issues in representation of problem Proper writing of tness function Proper values of size of population, crossover and mutation rate Deceptive Fitness Function (Mitchell, Melanie 1996)

36 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Disadvantages

Computationally expensive and time consuming Issues in representation of problem Proper writing of tness function Proper values of size of population, crossover and mutation rate Deceptive Fitness Function (Mitchell, Melanie 1996) Premature Convergence

36 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Advantages Disadvantages

Disadvantages

Computationally expensive and time consuming Issues in representation of problem Proper writing of tness function Proper values of size of population, crossover and mutation rate Deceptive Fitness Function (Mitchell, Melanie 1996) Premature Convergence No one mathematically perfect solution since problems of biological adaptation dont have this issue.

36 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

APPLICATIONS

37 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Applications

Image courtesy: http://www.google.com 38 / 42 Saif Hasan, Sagar Chordia, Rahul Varshneya Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft

39 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft Four major considerations for wing design

39 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft Four major considerations for wing design
Minimizing aerodynamic drag at supersonic cruising speeds

39 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft Four major considerations for wing design
Minimizing aerodynamic drag at supersonic cruising speeds Minimizing drag at subsonic speeds

39 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft Four major considerations for wing design
Minimizing aerodynamic drag at supersonic cruising speeds Minimizing drag at subsonic speeds Minimizing aerodynamic load (bending force on wing)

39 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft Four major considerations for wing design
Minimizing Minimizing Minimizing Minimizing aerodynamic drag at supersonic cruising speeds drag at subsonic speeds aerodynamic load (bending force on wing) twisting moment of wing

39 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft Four major considerations for wing design
Minimizing Minimizing Minimizing Minimizing aerodynamic drag at supersonic cruising speeds drag at subsonic speeds aerodynamic load (bending force on wing) twisting moment of wing

Objectives are mutually exclusive and optimizing them requires tradeo

39 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft Four major considerations for wing design
Minimizing Minimizing Minimizing Minimizing aerodynamic drag at supersonic cruising speeds drag at subsonic speeds aerodynamic load (bending force on wing) twisting moment of wing

Objectives are mutually exclusive and optimizing them requires tradeo Chromosomes - 66 real valued numbers, with population size 64 and simulated for 70 generations.

39 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Aeronautics
Multiple-objective genetics algorithm to design wing shape for supersonic aircraft Four major considerations for wing design
Minimizing Minimizing Minimizing Minimizing aerodynamic drag at supersonic cruising speeds drag at subsonic speeds aerodynamic load (bending force on wing) twisting moment of wing

Objectives are mutually exclusive and optimizing them requires tradeo Chromosomes - 66 real valued numbers, with population size 64 and simulated for 70 generations. Evolved wing congurations outperformed existing human designed-wings
39 / 42 Saif Hasan, Sagar Chordia, Rahul Varshneya Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

References
Obayashi, Shigeru, Daisuke Sasaki, Yukihiro Takeguchi, and Naoki Hirose. Multiobjective evolutionary computation for supersonic wing-shape optimization. IEEE Transactions on Evolutionary Computation, vol.4, no.2, p.182-187 (July 2000). Genetic Programming : On the Programming of Computers by Means of Natural Selection by John R. Koza http://www.myreaders.info/09 Genetic Algorithms.pdf http://www.obitko.com/tutorials/genetic-algorithms/searchspace.php http://www.talkorigins.org/faqs/genalg/genalg.html http://en.wikipedia.org/wiki/Genetic algorithm http://brainz.org/15-real-world-applications-genetic-algorithms/

40 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Conclusion
Large Appeal of Genetic Algorithms

41 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Conclusion
Large Appeal of Genetic Algorithms Is it because of Performance?

41 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Conclusion
Large Appeal of Genetic Algorithms Is it because of Performance? Or is it Aesthetic pleasing origins in theory of evolution ?

41 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Conclusion
Large Appeal of Genetic Algorithms Is it because of Performance? Or is it Aesthetic pleasing origins in theory of evolution ?

41 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm

Introduction Algorithm Theory Why GA? Applications

Applications Aeronautics

Questions

Questions ?

42 / 42

Saif Hasan, Sagar Chordia, Rahul Varshneya

Genetic Algorithm