Optimization

CH 3510 / CH 6640: Optimization Techniques I
Autumn 2016-17
Instructor: Dr. Kishalay Mitra

Department of Chemical Engineering
Indian Institute of Technology Hyderabad
(kishalay@iith.ac.in)
https://sites.google.com/site/kishalaymitra/
Overview
Topics
Optimization Formulation (6 lecs)

Single Variable search techniques w/o constraints (8 lecs)
Multi Variable search techniques w/o constraints (8 lecs)
Multi Variable search techniques with constraints (10 lecs)
Nonconventional search techniques (2 lecs)
Real life optimization problem solving
Mid Test / Final Test / Project in MATLAB (2 lecs)
Attendance (85%)
Pre-requisites of the Course
Basic Mathematics Background of Engineering

Programming Skills (MATLAB)
Textbook for the Course
S. S. Rao, Engineering Optimization: Theory and Practice, New Age Intl.

Publishers, New Delhi, 3rd Enlarged Ed., 2011.
T. F. Edger, D. M. Himmelblau, L S Lasdon, Optimization of Chemical
Processes, McGrawHill, 2nd Edition, 2001
2
Evaluation
No.
Type
Marks
Final
Weightage
Tentative
Time
1.
Mid Test
30
30
Sept
Starting
2.
End Test
30
30
Course
End
3.
Real Life Project
20
20
Aug-Sept
3.
Attendance (85%),
Class performance
20
20
Almost
each
class
Total = 100
Motivating Example
I want to minimize the cost of manufacturing a cylindrical
vessel with fixed volume V find me a suitable diameter and
height
Objective Function Min f cos t c s DL c t / 2 D 2
L,D
s.t.
Constraints
V / 4 D L
2
Decision Variables D 0, L 0
Min f cos t 4c s V / D c t / 2 D 2
D
df
4c s V / D 2 c t D 0
dD
1/ 3
4c s V
4V
, L
D

c t
1/ 3
ct

cs
2/3
L ct
D cs
What is Optimization?
Optimization is use of specific methods to determine the most

cost effective and efficient solution to a problem or design for a
process
A process can be represented by some equations or
experimental data - single performance criterion e.g. minimum
cost - goal is to run the process in such a manner that yield
the best value of the performance criterion
Typically optimization problems have many solutions task is
to find the best among these solutions use of computer &
s/w helps in making necessary computation feasible and cost
effective
One of the major quantitative tools in industrial decision making
Design, construction, operation, analysis of any plant can be
resolved
What is Optimization
A well known approach to principle of optimization can be found as back as

ROMAN times
Ramon bathhouse scribbling says: of two evils, always choose the

lesser in context with choosing between two aspirants for emperor of
ROME
Ubiquitous in the fields of science, engineering and business
Physics: optimum principle
Statistics: various principles termed "maximum likelihood," "minimum

loss," and "least squares,"
Business: maximum profit, minimum cost, maximum use of

resources, minimum effort
Why Optimization
Engineers work to improve the initial design of equipment and strive to

enhance the operation of that equipment to realize
largest production, the greatest profit, the minimum cost, the least
energy usage, and
In plant operations, benefits arise from improved plant performance, e.g.

improved yields of valuable products (or reduced yields of contaminants),
reduced energy consumption, higher processing rates, longer times
between shutdowns etc.
Optimization can also lead to reduced maintenance costs, less equipment

wear, better staff utilization.
It is extremely helpful to systematically identify the objective, constraints,

and degrees of freedom in a process or a plant, leading to such benefits as
improved quality of design, faster and more reliable troubleshooting,

and faster decision making
7
Levels of Optimization
Management
Plant
Process
......
Plant
....................
Process
Process
......
Process
Scope of optimization can be the entire company, a plant, a

process, a single unit operation, a single piece of equipment in
that operation, or any intermediate system between these
Essential Features
Optimization problems
One objective function
Multiple constraints
(equality)
Multiple constraints
(inequality)
Decision variables
& their bounds
Equality (nE)
Inequality (nI)
Independent Decision Variables
(nDV)
Optimization is a case of
nDV > (nE + nI)
nDV = (nE + nI) unique solution
nDV < (nE + nI) over-determined
system
Linear Regression
Linear Regression: to fit the best straight
line through the given points
y m a 0 a 1x
2
E y i y m y i a 0 a 1x i
i 0
i 0
E
2 y i a 0 a 1x i 0
a 0
i 0
n+1
n
E
2 x i y i a 0 a 1x i 0
a 1
i 0
x0
y0
x1
y1
xn
yn
a 0 n 1 a 1 x i y i
a 0 x i a 1
x y x
2
10
Linear Regression (contd.)
n 1 yi x i x i yi
a1
2
2
n 1 x x i
a 0 n 1 a 1 x i y i
a 0 x i a 1
n 1
x i
1
1
.
X
.
.
x y x
2
x a
x a
i
2
i
x0
x1
.
.
.
x n
yi
yx
1
i i
0
y0
y1

.
Y
.

.
y
n
1
a0
a 1 x i y i
n 1
Aa X X a X Y b
T
a 0
a
a1
11
Linear Regression (contd.)
n+1
y a 0 a 1x1 a 2 x 2 ... a N x N
Aa X X a X Y b
T
1 x1, 0
1 x
1,1
X
.
.
1 x1,n
12
x1
x2
x1,0
x1,1
..
xN
x2,0
xN,0
y0
x2,1
xN,1
y1
x1,n
. x N,0
. x N ,1
.
.
. x N ,n
x2,n
y0
y1

.
Y
.

.
y
n
xN,n
yn
a0
a 1

a .
.

a N
Linear Regression - example
= + + 1 120 550
1
1
1
X
1
1
1
1
130 600
108 520
110 420
84 400
90 300
80 230
55 120
64 190
50 100
Aa X X a X Y b
T
13
Nonlinear Regression (contd.)

To fit any general nonlinear function to a set of (n + 1) data
points, we need optimization techniques
2
Ea 1 , a 2 , a 3 , a 4 , a 5 , a 6 , a 7 , a 8 , a 9 y i y calc x i
i 0
(, > )
y calc x i
a3
a6
a 1 exp(a 2 x i ) a 4 sin(a 5 x i )
a9
a 7 log10 (a 8 x i )
Minimization of error can be achieved by

Derivative based techniques
Non derivative based techniques
Popular Optimization Problems
Determining the best sites for plant location

Routing tankers for the distribution of crude and refined products
Sizing and layout of a pipeline
Designing equipment and an entire plant
Scheduling maintenance and equipment replacement
Operating equipment, such as tubular reactors, columns, and
absorbers
Evaluating plant data to construct a model of a process
Minimizing inventory charges
Allocating resources or services among several processes
Planning and scheduling construction
And the list continues
15
Optimization Example 1
16
1
2
3
4
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
Classification
Existence of Constraints
Constrained Optimization
Unconstrained Optimization
Nature of Decision Variables

Static Optimization
Dynamic Optimization
40
Classification
Nature of the equations

involved
Nonlinear Optimization
Linear Optimization
Quadratic
Linear
Geometric
41
Classification
Based on deterministic nature of the variable involved

Stochastic Programming
Stochastic
42
Classification
Based on number of objective functions

Multi-objective Optimization
43
Graphical Solution
Optimum at the boundary of

the constraint set
44
Graphical
Optimum inside the feasible

region
45
Graphical
Local Optimum due to
objective function
Graphical methods can be

used to solve a problem up
to 2 variables
More than 2 variables
problem needs analytical /
numerical way to solve them
46
Numerical Solution
How can we solve this numerically?
Obvious answer: with an
optimization solver
What about MS Excel? What
- excel is an Optimization
solver!
OK - Lets try!
Get is set first
Open Excel File Options
Add-Ins Solver addins Go
Find a SOLVER icon in DATA
menu
Ready to use optimization module
47
MATLAB Routines
MATLAB functions used for optimization
48
MATLAB Routines
49
MATLAB Routines
50
Case 1: Linear Programming

Use linprog of MATLAB & MS XL to solve the problem - compare
52
Case 2: Scalar optimization
Use optimtool of MATLAB to solve the problem.

Choose fminbnd as the algorithm of choice.
Solve few cases: Minimize the function (.5 2.5) & Maximize the
function for 2 cases (.5-2.5 & 1.5-2.5)
53
Case 3: Nonlinear Least Square

100 data points given to you in the form of arrays xdata and ydata.
Assuming the model of the following form, find the coefficients
1. Use lsqcurvefit of MATLAB

Use Levenberg-Marquardt as the choice of the algorithm
2. Use fminunc of MATLAB and solve the problem
54
Case 4: Unconstrained Min

First order parallel reactions happening in an isothermal batch
reactor:
k2
k
a ( t ) exp k1 k 2 t
da
k1 k 2 a ( t ), a (0) 1
dt
db
k1a ( t ), b(0) 0
dt
a ( 0) a ( t ) b ( t ) c ( t )
k1
1 exp k1 k 2 t
b( t )
k1 k 2
k2
1 exp k1 k 2 t
c( t )
k1 k 2
56

Concentration data for first order parallel chemical reaction
i
time, ti
(ti)
(ti)
(ti)
0.1
0.913
0.0478
0.0382
0.2
0.835
0.0915
0.0732
0.3
0.763
0.1314
0.1051
0.4
0.697
0.1679
0.1343
0.5
0.637
0.2013
0.1610
0.6
0.582
0.2318
0.1854
0.7
0.532
0.2596
0.2077
0.8
0.486
0.2851
0.2281
0.9
0.444
0.3084
0.2467
10
1.0
0.406
0.3296
0.2637
57
Choose a value of k1, k2

Take a value of ti and
calculate a(ti), b(ti), c(ti) (i =
1, 2, , 10)
Sum them over all ti to
calculate the least square
error
Find that value of k1, k2
which gives you minimum
error or the perfect match
Use fminunc
a ( t ) exp k1 k 2 t
b( t )
k1
1 exp k1 k 2 t
k1 k 2
k2
1 exp k1 k 2 t
c( t )
k1 k 2
2
2
2
Min f x a t i a t i bt i bt i c t i ct i
k1 , k 2
i 1
N
58
Case 5: Unconst Min with ODE45

First order series reactions happening in an isothermal batch
reactor:
k1
A
B
da
k1a ( t ), a (0) 1
dt
db
k1a ( t ) k 2 b( t ), b(0) 0
dt
dc
k 2 b( t ), c(0) 0
dt
k2
C
Solve using ODE 45
& then perform parameter
estimation using fminunc
61

Concentration data for first order series chemical reaction
i
time, ti
(ti)
(ti)
(ti)
0.0000
1.0000
0.0000
0.0000
0.1000
0.7047
0.2759
0.0194
0.2000
0.4966
0.4367
0.0668
0.3000
0.3499
0.5204
0.1296
0.4000
0.2466
0.5535
0.1999
0.5000
0.1738
0.5541
0.2722
0.6000
0.1225
0.5345
0.3431
0.7000
0.0863
0.5031
0.4106
0.8000
0.0608
0.4656
0.4736
10
0.9000
0.0429
0.4256
0.5316
11
1.0000
0.0302
0.3855
0.5843
62
Case 6: Constrained Optimization
Solve this problem using fmincon function of MATLAB.

Plot the contour of the function in the backdrop and show the
movement of points over several iterations on the contour.
63
Case 7: Constrained Optimization
Min x
2
1
x 2 11 x1 x 2 7
2
x1, x 2
s.t.
x12 x 22 25
5 x1 , x 2 5
Solve using fmincon function of
MATLAB and show the effect of
initial guesses.
Plot the function in the backdrop and show
the movement of points over several iterations
on the contour.
66
Case 8: Global Optimization
Min x
2
1
x 2 11 x1 x 2 7
2
x1, x 2
s.t.
x12 x 22 25
5 x1 , x 2 5
Solve using ga function of
MATLAB and see what happens.
Use your binary & real coded GA to see the
effect.
Plot the function in the backdrop and show
the movement of points over several
generations on the contour.
67
Genetic Algorithms
Logic
Initialization: Started with a number of
solutions (Population) instead of a single
solution
Fitness: Computation of function
values for each of them
F(x)
Selection: Select better solutions from
the population (Parents)
Crossover, Mutation: Creating new
solutions from parent solutions (Children)
Continue this process till some
termination criteria it met (say maximum
number of iterations)
Local Basin
Global Basin
X
GA Example
Max sin( x )
String DV
01
10
00
11
001
100
001
010
String
01
10
10
11
001
100
100
010
9
20
1
26
f(x)
0.912
2.027
0.101
2.685
0.791
0.898
0.101
0.485
f(x)/favg Copies Mating Pool

1.39
1.58
0.18
0.58
favg 0.569
String DV
x
01
10
10
11
000
101
010
100
8
21
18
28
01
10
10
11
001
100
100
010
f(x)
0.811
2.128
1.824
2.838
favg
01001
x
1
2
0
1
0 x
0.725
0.849
0.968
0.299
0.710
01001 24 0 23 1 22 0 21 0 20 1
Max sin(x )
0x
x max x min
DecodeValue
2L 1
3.14 0
x 0 5
9 0.912
2 1
x x min
Genetic Algorithms User Inputs

User choices (inputbinary.txt)
No of binary variables (Nb)
Population size (Npop)
No of generations (Ngen)
Crossover probability (pc)
Mutation probability (pm)
Length of individual strings (Lsubs)
Upper and lower bounds for variable 1
Outputs (generationstat.txt)
No of binary variables
Population size
No of generations
Crossover probability
Mutation probability
Length of individual strings
Maximum, Minimum & Average value of
fitness in the population
Count on crossover, mutation
Requirements
Program should be modular (different
functions for initialization, fitness
evaluation, selection, crossover, mutation)
Plot evolution of population with time
Outputs (finalvariable.txt)
Values of variables (binary form & real
value) and objective functions of the last
generation
Outputs (allvariable.txt)
Values of variables (binary form & real
value) and objective functions of all
generations

Optimization

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Optimization

Uploaded by

Copyright:

Available Formats

CH 3510 / CH 6640: Optimization Techniques I

Instructor: Dr. Kishalay Mitra

Indian Institute of Technology Hyderabad

Optimization Formulation (6 lecs)

Pre-requisites of the Course

Basic Mathematics Background of Engineering

Textbook for the Course

S. S. Rao, Engineering Optimization: Theory and Practice, New Age Intl.

Indian Institute of Technology Hyderabad

Real Life Project

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Optimization is use of specific methods to determine the most

Indian Institute of Technology Hyderabad

A well known approach to principle of optimization can be found as back as

Ramon bathhouse scribbling says: of two evils, always choose the

Physics: optimum principle

Statistics: various principles termed "maximum likelihood," "minimum

Business: maximum profit, minimum cost, maximum use of

Indian Institute of Technology Hyderabad

Engineers work to improve the initial design of equipment and strive to

In plant operations, benefits arise from improved plant performance, e.g.

Optimization can also lead to reduced maintenance costs, less equipment

It is extremely helpful to systematically identify the objective, constraints,

improved quality of design, faster and more reliable troubleshooting,

Indian Institute of Technology Hyderabad

Scope of optimization can be the entire company, a plant, a

Indian Institute of Technology Hyderabad

One objective function

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Linear Regression (contd.)

Indian Institute of Technology Hyderabad

Linear Regression (contd.)

Indian Institute of Technology Hyderabad

Linear Regression - example

Indian Institute of Technology Hyderabad

Nonlinear Regression (contd.)

Minimization of error can be achieved by

Popular Optimization Problems

Determining the best sites for plant location

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad

Indian Institute of Technology Hyderabad