Introduction to


Genetic Algorithms (GAs)

 Evolutionary Computation
(EC)
EC is a computing paradigm based on the principles of biological
evolution.
Meta-heuristic optimization techniques
Population based nature
Mimics natural genetic systems

Components of EC:

Genetic Algorithms
Genetic Programming
Particle Swarm Optimization (PSO)
Ant Colony Algorithms

11/25/17 GA 2
 Genetic Algorithms (GAs)
• Definition:
Randomized Search and Optimization
technique guided by the Principles of Natural
Genetics.
• Natural selection : Survival of the fittest
• Natural reproduction

11/25/17 GA 3
 Randomized Search

– Not a simple random search

– Randomized search : random choice in a guided
path !
– Use the payoff function to guide search : fitness

11/25/17 GA 4
 Classes of Search Techniques

Search techniques

Calculus-based techniques Guided random search techniques Enumerative techniques

Direct methods Indirect methods Evolutionary algorithms Simulated annealing Dynamic programming

Finonacci Newton Evolutionary strategies Genetic algorithms

Parallel Sequential

Centralized Distributed Steady-state Generational

11/25/17 GA 5
 Different Search Techniques

Calculus based techniques – seek

local extrema by gradient descent –
hill climbing
–
Lack of robustness

• Single point search

• Restrictive requirements of
continuity and derivative
existence
• Multi peak function causes
dilemma : random restart
• Best in neighborhood of the
current point

11/25/17 GA 6
 Search Techniques (contd.)
Enumerative Techniques:
look at objective function values at every point in
the search space
– dynamic programming – n points

If n very very large ?

Simple but inefficient (time consuming)

11/25/17 GA 7
 GAs: A quick Overview

• Developed by John Holland, his colleagues, and

his students, K. DeJong, D. Goldberg, University of
Michigan (1970’s), USA
– to understand the adaptive processes of natural
systems
– to design artificial systems software that retains the
robustness of natural systems

11/25/17 GA 8
 Why GAs
• Most of the real life problems are very complex and can not be
solved in polynomial time using a deterministic algorithm.

• Sometimes attainment to the best is less important for complex

problem. Rather near optimal solutions that can be generated
quickly are more desirable than optimal solutions which require
huge amount of time.
– Ex: Traveling Salesman Problem (TSP).

GAs strive to improve as close as possible to optimality.

• When the problem can be modeled as an optimization one.

11/25/17 GA 9
 TSP example

Problem:
Given n cities
Find a complete tour with
minimal length/cost

Computation Time ??

Search space is BIG:

for 30 cities there are 30! ≈ 1032
possible tours

NP-Hard Problem

11/25/17 GA 10
 Why GAs (contd.)
• Not limited by restrictive assumptions about the
search space
– Continuity
– Existence of derivative
– Uni-modality and local minima
• Robust search in complex space
– Balance between efficiency and efficacy (flexibility of
biological system necessary for survival in many different
situations)

11/25/17 GA 11
 Why GAs (contd.)
• Efficiently exploit historical information to speculate on new search
points with expected improved performance : nature is full of
precedents

• Adaptive : Features of self-repair, self-guidance and reproduction are

the rules of the biological systems

• Costly redesign can be reduced

GAs work from a rich database of points simultaneously, climbing

many peaks in parallel; thus probability of finding a false peak (local
optima) is reduced.

11/25/17 GA 12
 GAs

Distribution of Individuals in Generation 0

Distribution of Individuals in Generation N

11/25/17 GA 13
 Why GAs (contd.)
• Always an answer/solution; answer gets better with time
empirically proved to be convergent
• Inherently parallel; easily distributed
• Supports multi-objective optimization
• Modular
• Finally, the power of GAs comes from simple concepts;
easy to understand

11/25/17 GA 14
 Features of GAs

• Optimization Technique

• Darwin’s principle of evolution (survival of the fittest) has

made this optimization algorithm effective.

• Work with a coding of the parameter set

• Search from a population of points, not a single point

• Use payoff (objective function) information, not derivatives or

auxiliary knowledge

• Use probabilistic transition rules, not deterministic rules

11/25/17 GA 15
 GAs vs. Nature

• A solution (phenotype) Individual

• Representation of a solution Chromosome
(genotype )
• Components of the representation Genes
• Set of solutions Population
• Selection:survival of Darwin’s Theory
the fittest
• Search operators Crossover and
Mutation

11/25/17 GA 16
 Simple Genetic Algorithm (SGA)

1. Generate random population of chromosomes

2. Decode each chromosome to get an individual
3. Evaluate the fitness of each individual
4. Perform selection, crossover and mutation
5. Repeat steps 2, 3 and 4 until it reaches a
termination condition.

There is no overlapping between generations

11/25/17 GA 17
 I–nitialize Population

GA Flowchart
– Evaluate Fitness

Yes
– Terminate? –
Output
solution
No

Perform selection,
–
crossover and mutation

– Evaluate Fitness
11/25/17 GA 18
 Encoding and Population
• Encoding: A chromosome encodes a solution in the search space
– Usually binary string of 0’s and 1’s
– Each bit in the string can represent some characteristics of the
solution
– Chromosome size depends on parameter set to be encoded

• Population:
– A set of chromosomes in a population
– Population size is usually constant
– Common practice is to choose the initial population randomly.

11/25/17 GA 19
 Chromosome Size
Chromosome: A set of alleles/genes
A chromosome is an encoded form of all parameters describing the
given problem
An encoded parameter – A subset of alleles
Chromosome size = ??

An allele parameter n
A Chromosome

11/25/17 GA 20
 Population

string 1
string 2
A population –
string m

11/25/17 GA 21
 Fitness Evaluation
• Fitness (objective function) associated with each
chromosome
• Indicates the degree of goodness of the encoded solution
(chromosome)
• only problem specific information (also known as the
payoff information) that GAs use
• for minimization problem
fitness ∝ 1/objective

11/25/17 GA 22
 Selection (Reproduction)
• A process in which individual strings are copied according
to their objective function or fitness values : Darwin’s
survival of fittest
• Better individuals get higher chance
– more copies to good strings
– fewer copies to bad strings
• Mimics the natural selection procedure to some extent
• Implementation:
• Proportional selection :
• Roulette wheel selection
• Tournament selection etc.

11/25/17 GA 23
 Roulette Wheel Selection

» Assign to each individual a part of the

roulette wheel
» Spin the wheel n times to select n individuals

1/6 = 17%
fitness(A) = 3

A B fitness(B) = 1
– C fitness(C) = 2
3/6 = 50% 2/6 = 33%

11/25/17 GA 24
 Elitist Model of GAs

The best string up to the current generation is

preserved

11/25/17 GA 25
 Crossover
• Partial exchange of information between two
randomly selected parent chromosome
• Single point crossover : most commonly
used
• Probabilistic operation, µc

11/25/17 GA 26
 Single point Crossover (example)

Crossover point

before Crossover after

11/25/17 GA 27
 n-point crossover

• Choose n random crossover points

• Split along those points
• Glue parts, alternating between parents
• Generalisation of 1 point (still some positional bias)

11/25/17 GA 28
 Mutation
• Random alteration in the genetic structure
• Mutating a binary gene : simple negation of the
bit
• Probabilistic operation : alter each gene
independently with a probability pm
• Exploring the search area : new search points
• Introduce genetic diversity into the population

11/25/17 GA 29
 Mutation (example)

Before mutation

After mutation

11/25/17 GA 30
 Exploitation vs. Exploration
• Exploration: Discovering promising areas in the search space, i.e.
gaining information on the problem – new search points
• Exploitation: Optimizing within a promising area, i.e. using
information - Exploit historical information: survival of the fittest
co-operation AND competition between them
• Crossover is explorative, it makes a big jump to an area somewhere “in
between” two (parent) areas
• Mutation is exploitative as well as explorative, it creates random small
diversions, thereby staying near (in the area of ) the parent
• Selection is totally exploitive

11/25/17 GA 31
 An example of SGA (Goldberg)

• Simple problem: max x2 over {0,1,…,31}

• GA approach:
– Representation: binary code, e.g. 01101 ↔ 13
– Random initialization
– Population size: 4
– Roulette wheel selection
– 1-point crossover, bit wise mutation
We show one generational cycle done by hand

11/25/17 GA 32
 x2 example (contd.) : Selection

11/25/17 GA 33
 x2 example (contd.) :Crossover

11/25/17 GA 34
 x2 example (contd.) :Mutation

11/25/17 GA 35
 Shortcomings of SGA

– Representation is too restrictive

– Mutation & crossovers only applicable for bit-
string & integer representations
– Selection mechanism sensitive for converging
populations with close fitness values

11/25/17 GA 36
 Parameters
• Population size : usually fixed
• String/chromosome length : usually fixed
• Crossover probability : (µc)
• Mutation probability : (µm)
µc >> µm
• Termination criteria :
– desired fitness, if possible (known)
– generally a maximum number of iterations

11/25/17 GA 37
 Termination Criterion
The cycle of GA-operators (selection, crossover and
mutation) is repeated a number of times till :
– A desired objective function value is attained in any string/
chromosome of the population
or
– The average fitness value of a population becomes more or less
constant over a specified number of generations
or
– The number of generations/iterations is greater than some
pre-specified value.

11/25/17 GA 38
 TSP representation

Representation is an ordered list of

city numbers known as an order-
based GA.
• Encoding:
• Label the cities 1, 2, 3, …..,
n
• One complete tour is one
permutation, e.g. for n = 8

CityList1 (3 5 7 2 1 6 4 8)
CityList2 (2 5 7 6 8 1 3 4)

11/25/17 GA 39
 Crossover
Crossover combines inversion and
recombination:
* *
Parent1 (3 5 – 7 2 1 6 4 8)
Parent2 (2 5 7 6 8 1 3 4)

Child (5 8 – 7 2 1 6 3 4)

This operator is called the Order1 crossover.

11/25/17 GA 40
 Mutation

Mutation involves reordering of the list:

* *
Before: (5 8 7 2 1 6 3 4)
– –
After: (5 8 6 2 1 7 3 4)

11/25/17 GA 41
 TSP30 - Overview of Performance

1800

1600

1400

1200
Distance

1000

800

600

400

200

0 Best
11

13
15

17
19

21
23

25
27

29
31
1
3

5
7

Worst
Generations (1000)
Average

11/25/17 GA 42
 ??
• Why not a bad chromosome is totally
eliminated ?
• Mutation is mandatory?
• Population size : Population size is a critical factor
to determine.
– Large : time vs. diversity
– Too small : converge to local optima

11/25/17 GA 43
 ??
• Total execution time T(approx) ?
I + n ??
given : n = no. of iterations (no. of generations)
p = population size (no. of chromosome)
d = decoding time of the parameters (from chromosome
representation to actual problem/function domain)
E = time to execute the function (problem to be optimized)
to calculate its output/result from the decoded parameters
F = time to calculate the fitness of the function/output/result
I = time to generate initial population

11/25/17 GA 44
 Some GA Application Types

Domain Application Types

Control gas pipeline, pole balancing, missile evasion, pursuit

Design semiconductor layout, aircraft design, keyboard

configuration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation

Robotics trajectory planning

Machine Learning designing neural networks, improving classification
algorithms, classifier systems
Signal Processing filter design

Game Playing poker, checkers, prisoner’s dilemma

Combinatorial set covering, travelling salesman, routing, bin packing,

graph colouring and partitioning
Optimization

11/25/17 GA 45
 Conclusion

Question: ‘If GAs are so smart, why ain’t they rich?’

Answer: ‘Genetic algorithms are rich - rich in
application across a large and growing
number of disciplines.’
- David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning

11/25/17 GA 46
 Discussion

??

11/25/17 GA 47
 Thanks !

11/25/17 GA 48