IATOM17 OR CO Slideshow PDF

Operations Research
Part 1: combinatorial optimization
Referent : Catherine Mancel

Catherine.mancel@enac.fr
lecturers :
Gilles Baroin
Catherine Mancel
Introduction
 What is Operations Research?
“Operations Research (O.R.), or operational research in the U.K, is a
discipline that deals with the application of advanced analytical methods
to help make better decisions.
The term management science is sometimes used as synonym for
operations research. Employing techniques from other mathematical
sciences, such as mathematical modeling, statistical analysis, and
mathematical optimization, operations research arrives at optimal or near-
optimal solutions to complex decision-making problems.
Operations research overlaps with other disciplines, notably industrial

engineering and operations management. It is often concerned with
determining a maximum (such as profit, performance, or yield) or minimum
(such as loss, risk, or cost.)…” INFORMS (The institute for Operations
Research and the Management Sciences)
2
OR – Combinatorial Optimization – IATOM
How can OR help
the Decision Process in Industry
Managers OR specialists
Modeling
Industrial theoretical
Problematics models
Operational level
Solution design
Decision Aid or/and
Results Analysis Optimization tools
3
Key points in the OR approach
 Understanding/ formulation of the problem

 Modeling
 Searching for methods that fit:
– needs, model structure, problem size …
 Method design (algorithm and programming),
resolution
 Analysis and use of results
4
what does modeling mean?
Modeling = describing in a formal unambiguous way.
Mathematical models are often used to describe systems or

(optimization) problems. Nevertheless, there are other kind of
models like logical models or graph models.
Using a natural language (French, English) could also be a way to

model a problem but it is very difficult not to be ambiguous while
using it.
5
what is an algorithm?
An algorithm is a set of steps for solving a particular

problem.
To be an algorithm, this set of steps has to be unambiguous (each

instruction is well defined, the starting state and initial input are
precisely described) and has to have a clear stopping point.
Pseudocode and flowcharts are often used rather than natural
languages to express algorithms because they avoid many
ambiguities common in natural language statements.
Pseudocode is an informal way to describe algorithms that uses the
structural conventions of a programming language but that is
intended for human reading rather than machine reading.
6
Course contents
 Aim of the course :
– Provide students with Operations Research basis to solve
decision (management) problems especially in Air
Transportation field.
 Focus items :
– Fundamental optimization models and methods
– Classical problems (well-studied problems associated
with efficient models and methods important to know)
7
Outline
1st Part : Combinatorial Optimization

1- Linear Programming
2- Integer (linear) Programming
3- Graph theory
Ref : Nemhauser and Wolsey “Integer and combinatorial optimization”, Wiley
2nd Part : Continuous Optimization

(Marcel Mongeau)
8
Combinatorial optimization vs
continuous optimization
In applied mathematics and computer science, combinatorial optimization is a
research and engineering field that consists of finding an optimal object
from a finite set of objects. (for example : what is the best way to assign
aircraft to a set of flights, among all the assignment possibilities for a given
airline, with a given fleet and on a given time horizon?). In many such
problems, exhaustive search is not feasible.
Continuous optimization is a topic of applied mathematics in which the

variables used in the mathematical model are assumed to be real values (so
the set of possible solutions is not a finite set).
9
1. Linear Programming
Linear programming = solving Linear Programs
Reminder : a linear equation is an algebraic equation in which each term

is either a constant or the product of a constant and a (first power of)
single variable.
10
Optimization problems and
mathematical models
An optimization problem is composed of :

- Some decisions to be made Decision
- some constraints to be satisfied pb Optimization
- One (or several) criterion to be optimized pb
Mathematical (optimization) model :

Describe decisions with mathematical variables
Describe constraints and criteria with equations
11
Linear optimization problems
If all equations of the mathematical optimization model are

linear, then the problem is called
a Linear Program (LP) or a linear optimization problem
Min. a cost/ max. a profit……….......min/max z =c1x1+c2x2+…+cnxn

Satisfy a demand …………………… a1x1+a2x2+…+anxn b1
With limited resources……………… a’1x1+a’2x2+…+a’nxn b2
Quantities to choose………………... x1,x2,…, xn  0
There are very efficient methods to solve these problems
12
1.1 General form of a linear program
a maximization problem with n variables and m constraints :

 Objective function or optimization criterion
s.t.
(constraints)
13
Matrix representation
- parameters-
Right hand-side member

of constraints :
m x n matrix of constraints :
s.t.
Profit (or cost) coefficients :

14
Matrix representation
- variables -
n decision variables :
s.t.
15
Vocabulary
xi is called a decision variable of the problem, for each i=1 to n
a vector of values X= (x1, …, xn) is a feasible solution if and only if it

satisfies all the constraints
The set of all feasible solutions is called feasible set, or feasible domain,
or feasible area
X= (x1, …, xn) is an optimal solution if and only if it is feasible and it

optimizes (maximizes or minimizes) the objective function
16
Canonical form of a linear program
 Maximization
 Non-negative variables
 Constraints : " "
s.t.
17
An example
– An air freight carrier has a freight capacity
of 900 kg and 12 m3 left.
– 2 kinds of products can be loaded in the

aircraft, respectively defined by a unit
profit of 1 and 2 value units (100€ for
example), by a unit weight of 100 and 300
kg and by a unit volume of 2 and 3 m3.
– significant assumption : the products are

supposed to be subdivisible at will
–How shall we load the aircraft so that to maximize the total

profit? 18
Method to tackle this problem
 Understanding …
 Modeling
– Which are the decision variables?
– What is the optimization criterion?
– Which are the constraints to be satisfied?
 Solving
 Analyzing results
19
Modeling
– 2 decision variables :
Let x = the quantity of product 1 we decide to load
Let y = the quantity of product 2 we decide to load
– two constraints
Maximum weight has to be respected (100x+300y  900)
Maximum volume has to be respected (2x+3y  12 )
– one objective function (maximization)
Maximize the profit (z = x+2y)
20
Model
Max z = x+2y
s.t. x+3y  9 The objective fonction and the
constraints are linear :
2x+3y  12
x, y  0 this is a Linear Program (LP)
x and y are the decision variables (or unknowns) of the LP ; the coefficients of x and y in
the objective function and in the constraints are the parameters of the LP. Here
we have a LP in its canonical form, i.e. a maximization problem, with n+m inequalities,
where n is the number of variables (n=2) and m is the number of constraints (m=2).
When constraints are given as equalities instead of inequalities, the LP is in its standard
form.
21
Remark
 Production Problem :
– A factory produces two kinds of products, using two types of
resources (raw materials). The manufacturing of these products
respectively yields a profit of 1 and 2 euros by unit. The manufacturing
of one unit of each product needs respectively 1 and 3 units of
resource 1, and 2 and 3 units of resource 2.
There is a stock of 9 units of resource 1 and 12 units of resource 2.
Which quantity of each product shall the factory produce so that to

maximize the profit?
We obtain exactly the same model!
22
1.2 Solving a LP
 The model is built, now we have to solve the linear

program
 Two cases
– 2 variables : graphical representation is possible,
associated with a graphical solution (very easy)
– More than 2 variables : we need an algebraic method to
solve the linear problem.
23
1.2.1 Graphical resolution
 graphical representation of the example 2-variables

LP
feasible domain (convex polyedra)
(any point in this region is a (feasible) solution of the
LP)
S1
isocriteria cornerpoint solutions
lines S2 (vertices of the feasible domain)
S3
S0 z=4
10 x+3y=9
z=0 2x+3y=12 24
Graphical solution
Solution :
x+ 3y=9 x*=3
2x+3y=12 y*=2 and z*=7
S1
S2 The optimal solution is
always at a cornerpoint !
S3
S0 z=4
10 x+3y=9
z=0 2x+3y=12 25
1.2.2 algebraic solution : the Simplex
Method
 In case n>2 (more than 2 variables), it is impossible to give
a graphical representation of the problem
 Back to the example : consider a third type of product that

can be loaded in the aircraft with :
– unit profit: 3, unit weight: 400 kg, unit volume: 4m 3
 the model has to be updated and we get n=3
 How to solve this new problem ?

 We need an algebraic method : the Simplex Method
26
Geometry of a LP
 for any n, the feasible domain is a convex

polyhedron (intersection of half-spaces)
 Iso criteria domains are affine hyperplans (n=2

line, n=3  plane, …)
 The optimal value of the objective function, if any, is

reached at a cornerpoint of the feasible domain
27
1.2.2.1 Algebraic solution of the 2-
variable example
 Standard form of the LP :
We need a system with linear equations instead of
inequalities :
 add one slack variable per constraint :
a represents the gap between the

Max z = x1+2x2
linear combination " x1+ 3x2" and
s.t. x1+ 3x2+ a = 9 the value "9“
2x1+ 3x2+ b = 12
x1, x2 , a, b  0 b represents the gap between the
linear combination " 2x1+ 3x2" and
the value "12"
28
Finding a first solution
z, a and b can easily be defined in terms of x1 and x2 :
(max) z = x1+ 2x2

a = 9- x1- 3x2
b = 12- 2x1- 3x2
x1 and x2 can be considered as free variables, while z, a and b are

deduced from x1 and x2
This linear system is called a dictionary

of var. z, a, b in terms of variables x1 and x2 29
First solution
(max) z = x1+ 2x2
a = 9- x1- 3x2
b = 12- 2x1- 3x2
let set x1=x2=0 (easy to decide)

 a=9, b =12 , z=0
 feasible solution at a cornerpoint of the
feasible domain
or basic solution (basis (a,b))
What should we do to increase z? 30

Improve the current solution
We have X=(0,0,9,12), z=0, we search for X’=(x1’,x2’,a,’b’) such that :

x1’+ 2x2’> 0
and X’ is a cornerpoint of the feasible domain
notice that increasing x1 or x2 (not both):

S1 Allow to increase z (= x1+ 2x2)
S2 Allow to reach a new cornerpoint (S1 or S3)
S3
S0
31
Let’s choose to increase x2
• x2 is a ‘’better candidate ‘’ than x1 (c2 >c1 in z)

• the more x2 increases, the more z increases
 what is the maximal value for x2?

 x1 has to stay equal to 0
S1
 positivity constraints have to be satisfied
S2
a, b  0
S3
S0
32
New solution
a = 9- x1- 3x2  0  x2  3
b = 12- 2x1- 3x2  0  x2  4
 x1 = 0, x2 = 3 , a= 0, b= 3 et z= 6
 It’s better !
 It’s a new cornerpoint, or new basic solution (S1, basis
(x2,b))
Can we still increase z?
 To answer this, we need the same dictionary structure than previously:

expression of z and non-zero variables in the current solution (x2 and
b) in terms of the currently-zero variables (a and x1).
33
New dictionary
(associated to the new solution)
 Express the new non-zero variable, x2, in terms of
currently-zero variables (a and x1)
– using the constraint that is limiting the value of x2
– a = 9- x1- 3x2  x2= 3 – (a + x1)/3
 Replace x2 by this expression in others equations
 The obtained system is equivalent to the initial one, but it
gives a new dictionary
 The solution process can be iterated !

34
Summary of the algebraic solution
process
•Initialization :
-express m variables in terms of the n-m others
-first solution: S0 (basis (a,b), x1=x2=0), z=0
•if x2 increases then z increases (x2 can increase up to 3; above 3,
a becomes negative)
New solution: x2=3, b=3, x1=a=0 : S1 (basis (x2,b)), z=6
Express x2, b and z in terms of x1 and a (basis modification)
S1 •one can notice that if x1 increases, then z increases
S2  New solution: x1=3, x2=2, a=0, b=0 : S2 (basis (x1,x2)), z=7
Express x1, x2 and z in terms of a and b
•one can see that it is impossible
S3 to increase z anymore :
S0 STOP
35
1.2.2.2 Generalization of the linear
programming method to any LP
 Standard form of a Linear Program :

Max z = c.x
s.t. A.x=b, x  0
with A(m,n), m<n, rank(A)= m
• It is always possible to get rank(A)=m

• Any LP can be expressed in the standard form
36
Fundamental Theorem of Linear
Programming
 Let consider (P) :

Max z= cx
P st Ax=b, x0
with A(m,n), m<n, rank(A)=m
1) If (P) is feasible, then a basic feasible solution exists (i.e. if a
solution exists, then there is a cornerpoint solution)
2) If (P) has a finite optimal value, then it has an optimal basic
feasible solution (i.e. if an optimal solution exists, then there is an
optimal cornerpoint solution)
  Solving (P) can be reduced to the evaluation of its
feasible basic solutions (i.e. its cornerpoints).
37
Simplex method
 General Idea (Dantzig, 1949)

– It consists in moving along the edges of the feasible
convex polyedra from basic feasible solution to basic
feasible solution, each time improving the objective
function until the optimal basic feasible solution is
reached.
 Optimality Condition
The objective function expressed in terms of the currently-
zero variables (variables equal to zero in the current
solution) can not be improved anymore.
38
Simplex Algorithm
– Initialization: Find a first basic solution (i.e. a cornerpoint of the
polyhedron feasible domain)
 It gives a set of variables equal to zero (non-basic variables), a set of
non-zero variables (basic variables), and an associated value for z
– Repeat:
 Look for a variable, among the non-zero variables in the current
solution, whose increasing would increase the value of z
 If such a variable exists: Compute the maximal value for this variable,
as well as the new associated basic solution. This new solution
becomes the current solution
 Else : stop, the current solution is the optimal one
Until the optimal solution is found

39
Details of initialization step:
Find a first basic solution
Consider P expressed in its canonical form :
s.t. 1 to m
1 to n
add m slack variables in order to express P in its standard form:
s.t.
40
First basic solution
We obtain a dictionary that defines values of z and of variables xn+1
to xn+m in terms of variables x1 to xn :
1 to m
Set x1= …= xn =0 and compute the values of the other variables
 compute the basic solution associated with this dictionary
41
Dictionary
Let (P) a LP with m+n variables and m constraints

 a dictionary is a linear system that defines m variables in
terms of the n others.
 A dictionary is feasible if, setting right hand-side member

variables to zero, a feasible solution is obtained for (P)
 Basic (feasible) solution : solution that can be described

by a dictionary.
It also corresponds to the coordinates of a cornerpoint of the
feasible domain of (P) 42
Details of the iterative process of the
Simplex Algorithm
At each iteration :
Look for a cj>0 in the current dictionary :
If such value exists :
 If there are several, choose maxj (cj)
 Decide to increase as much as possible the value of xj
Constraint : xj bi/aij , for each i=1 to m
 Deduce the other values
A new basic solution is obtained, that improves the value of z
 Get the corresponding new dictionary (dictionary updating)
Else (cj0 j) : the current solution is the optimal one
43
Details of the Dictionary Updating
 New dictionary features:
– xj becomes a variable of the left hand-side member of the dictionary
(non-zero variable)
– One specific variable, xk, previously in the left hand-side member,
becomes zero and is moved in the right hand-side member
" xj enters in the basis, xk is removed from the basis"
 To get the new dictionary :
– xj is expressed in terms of xk with help of the constraint that limits the
value of xj in the new current solution
– xj is replaced by its expression in the other equations of the
dictionary
44
Exercise :
Solve this 3-variables example
Apply the Simplex Algorithm to the following linear

program :
Max z = x1+2x2+3x3
S.t. x1+ 3x2+ 4x3 9
2x1+ 3x2+ 4x3 12
x1, x2, x3  0
45
1.3 Duality
 Each linear program (P) is associated to its Dual

problem (D).
 Primal (P) Dual(D)
max z = cx min w = b
st. Axb st.  0
x0 A c
 Lemma: Dual problem of (D) is (P)
"The dual of the dual is the primal"
46
Primal problem with equality
constraints
Primal problem with equality constraints

 Primal (P) Dual(D)
st. Ax=b st.  free
x0 A c
47
1.3.1 Duality rules
Primal Dual
max min
constraint i  variable i  0
constraint i  variable i  0
constraint i = free variable i
variable j  0 constraint j 
variable j  0 constraint j 
free variable j constraint j =
Dual Primal
48
1.3.2 Duality theorems
 let the pair of dual problems :

Primal (P) Dual(D)
x0 A  c
 Weak duality theorem : if x is a feasible solution to

(P) and  is a feasible solution to (D), then cx b
49
Strong duality theorem
 Strong duality theorem :

– A feasible solution x* to the primal is optimal if and only if
there is a feasible solution * to the dual such that
cx* = *b.
In particular, * is an optimal solution to the dual.
Note that if one of the two problems has a non-bounded

criteria, then the other problem has no feasible solution.
50
Complementary slackness theorem
 Let the pair of dual problems :
Primal (P) Dual(D)
x0 A c
 Assume (P) has an optimal solution x* and problem (D) has an optimal
solution *
– If xj*>0 , then the jth constraint in (D) is binding (Aj=cj)
– If the ith constraint in (P) is not binding (Aix*<bi), then i*=0.
– If i*>0 , then the ith constraint in (P) is binding.
– If the jth constraint in (D) is not binding (*Aj>cj), then xj*=0.
51
1.3.3 Two-Phase Simplex method
Used when it's not easy to find a starting basic feasible

solution (pb with “ajXjbi” constraint where bi>)
– Phase 1 :
 Introduce artificial variables
 Criteria : minimize the total value of artificial variables
– Phase 2 :
 The optimal basis obtained at the end of phase 1 gives the starting basic
feasible solution of the original problem, thus we can apply the Simplex
algorithm
 Or solve the dual and use the complementary slackness

theorem!
52
1.4 Conclusion
 Linear programming is an effective widely used modelling
and solving set of methods
 Several algorithms to get the exact optimal solution
– Graphical solution, simplex method
– + interior point methods, dual simplex …
 several software to solve large size linear problems
– Cplex, Xpress-MP, Lp-Solve, …
 But : here we have considered continuous variables !
What about Integer Variables?
53
2. Integer Linear Programming
 Now we consider linear problems with the

additional constraint : variables are integer
Max z = c.x Max z = c.x

st. Axb st. Axb
xN or x{0,1}
54
2.1 Example
 Aircraft Load
– An air freight carrier has a freight capacity of 900 kg and 12 m3 left.
– 2 kinds of products (subdivisible at will) can be loaded in the aircraft
respectively defined by a unit profit of 1 and 2 value units, by a unit
weight of 100 and 300 kg et by a unit volume of 2 and 3 m3.
– How shall we load the aircraft so that to maximize the total profit?
 Model :
Max z = x+2y
s.t. x+3y  9
2x+3y  12
x, y N
x+3y=9
2x+3y=12
12 55
Another example :
Assignment Problem
– At a given time T, an airline has 4 aircraft at an airport A and 3 flights to achieve
starting from A in a small time window [T, T+d](so that one aircraft is not able to
achieve two flights).
– The assignment of each aircraft a to a flight f leads to an operational cost caf for the
airline.
– How to assign aircraft to flights in order to minimize the total operational cost?
 Model ?
56
Assignment Problem
– At a given time T, an airline has 4 aircraft at an airport A and 3 flights to achieve starting from A in
a small time window [T, T+d](so that one aircraft is not able to achieve two flights).
– The assignment of each aircraft a to a flight f leads to an operational cost caf for the airline.
– How to assign aircraft to flights so that to minimize the total operational cost?
 Model :
Variables :
let xaf =1if aircraft a is assigned to flight f and 0 otherwise.
Constraints :
each flight has to be achieved exactly once
each aircraft has to be assigned at most to one flight
Objective function :
minimize the cost of assignment
57
Assignment Problem
– Integer Linear Model :
Min z = a=1to4f=1to3cafxaf
s.t. a=1to4 xaf = 1, for each f =1 to 3
f=1to3 xaf  1, for each a =1 to 4
xaf{0,1} for each a =1 to 4, for each f=1 to 3
58
2.2 How to solve an Integer Problem?
 Linear programming methods don't work because the set of
solutions is not a convex polyedra anymore but only the set
of the integer points in this convex polyedra
In this example S2 corresponds to an

S1 integer solution but usually it is not the
S2 case (consider for example that the
total weight is 80kg instead of 90kg).
S3
S0 10 x+3y=9
2x+3y=12 x+3y=8
59
Methods for Integer linear Problems
 Enumerate each integer solution (in a search tree)

– Computation time increases exponentially with the
number of variables : very bad idea!
 Use linear programming to approximate solutions
– No certainty to get a solution with the rounding values
 Use linear programming to solve a relaxation of the
problem : branch and bound method
– Exact method to solve integer linear problems
60
Is it a good idea to choose rounding
values as solution of an ILP?
Let consider the following integer
linear model :
Max z = x+y
s.t. x- 2y  0
x+4y  16
x, y N
On this example, the rounding values of

the corresponding linear problem
optimal solution are not feasible
solutions !
61
2.3 Linear Relaxation
 Relaxing a constraint means considering the

problem without this constraint
 The Linear Relaxation (L) of a Integer Program (P)
is the corresponding linear program (without
integrity constraint).
(P) max zP= c.x (L) max zL=c.x
st Ax  b st Ax  b
xN x0
62
2.4 Upper bound, lower bound
 In case of maximization : zP*  zL*

– Indeed, (L) is less constrained than (P) so if xP* is the optimal
solution of (P), then xP* is a solution of (L). Let xL* be the optimal
solution of (L), we have :
zL*= z(xL*)  z(xP*)= zP*
 The optimal solution of (L) gives an upper bound of the
optimal solution of (P)
 In case of minimization : zL*  zP*
 The optimal solution of (L) gives a lower bound of the
optimal solution of (P)
63
2.5 Branch and Bound method
 Tree search method

 Branch : separate the set of solutions into subsets
so that on each branch of the search tree, only one
particular subset is explored.
 Bound : use linear programming (simplex method
for example) to evaluate the ability of each branch
to lead to a good solution
64
Branch and Bound method
 Let consider a node of the search tree. (P) is the corresponding integer
problem. (L) is the linear relaxation of (P). Let xL* the optimal solution of
(L). If xLi* is not integer then we can state that (P) is equivalent to
((P1)OR(P2))
with (P1)=((P) plus the additional constraint xi xLi*)
and (P2)=((P) plus the additional constraint xi xLi*+1)
 Then at this node, the search space (set of solutions) can be separated
into two, each one corresponding to a branch of the search tree starting
from this node (one for (P1) and one for (P2))
65
Example (1)
(P) :
Max z = x+2y
st 2x+y
-x+2y
x,y N
zL*= 9.4 zP* 9 (upper bound)

xL*= 2.2
yL*= 3.6
66
Example (1)
P zP* 9 )
x2 x
P1 P2
no integer integer solution:
solution: x=3 ; y=2 ; z= 7
xL*= 2; zP* 7
yL*= 3.5; (lower bound)
zL*= 9
67
Example (1)
P zP*
9)
P1 P2
X
y y
P3 P4
integer solution: no solution
x=2 ; y=3 ; z= 8 X
zP* 8
new lower bound
End of search 
zP*=8 ; xP*= 2 ; yP*= 3
68
Example (2)
aircraft load problem with 80kg left instead of 90kg
Max z = x+2y
s.t. x+3y  8
2x+3y  12
x, y N
69
Example (2)
70
Example (2)
71
2.6 Conclusion
 An exact method to solve combinatorial

optimization problems
 Branch and bound method can be used with other
evaluation algorithm than the simplex
 Branch and bound method can be used to solve
other problems than integer linear problems
 Limited use because of computation time
72
3- Graph theory
 Lots of decision problems can be formulated with

help of graphs
 They can often be modeled as classical problems
of graph theory (shortest path problems in a graph,
max flow problems in a network, spanning tree
problems, …)
 Efficient algorithms exist to solve these classical
problems
73
3.1 Definitions (1)
 Let X : a finite set of nodes (or vertices)
 Let A : a set of directed pairs of nodes, called arcs
 G=(X,A) is a directed (or oriented) graph
 Let aA, a=(x,y) with x,y  X, a is "the arc from x to y"
a
x y
 Let E : a set of (non-directed) pairs of nodes, called edges
 G=(X,E) is a non-directed graph
 Let eE, e=(x,y)= (y,x) with x,y  X, e is "the edge between x and y"
e
x y
74
Example
 In 1736, Euler proved that it is not possible to cross each bridge of
Königsberg exactly once and then come back to the starting point
A
A
D
D B C B
C
75
Examples
76
Definitions (2)
Let G=(X, U), where U= A or E. G is either an oriented or non-oriented
graph
 Weighted graph
– A weight (or cost, or value) can be associated with each edge (or arc) of a
graph G.
– G=(X,U,w) is a weighted graph, where w : U  R is a weight function.
 Sub-graph, partial graph
– Let G'=(X',U') with X'X and U'={u =(x,y)U with x,y X'}. G' is a subgraph
of G=(X,U)
– Let G'=(X,U') with U' U. G' is a partial graph of G =(X,U)
 Adjacency
– Two nodes (x,y) are adjacent if there is an arc (or edge) u=(x,y) in U.
77
Definitions (3)
Let G=(X, A) an oriented graph
 Successors and predecessors
– with each node xX are associated the set (x) of its successors and the
set  -1(x) of its predecessors. Let (x,y) A, x is a predecessor of y and y is
a successor of x
 Degree of a node
– out degree of a node x : d+(x) = number of arcs starting from x
– in degree of a node x : d-(x) = number of arcs ending at x
– degree of a node x : d(x) = d+(x) + d-(x)
– Note that In non-oriented graph, d(x)= number of nodes adjacent to x
78
Definitions (4)
 Path
– A path is a graph P=(X,U) where X={x0, x1,…xn} and
U={(x0,x1)(x1,x2)…(xn-1,xn)}.
– P is a path from x0 to xn if U is a set of arcs (P is oriented) ;P is a path
between x0 and xn if U is a set of edges (P is non-oriented) in that case, P is
also called a chain ;
– The weight (or length) of path P is the sum of the weights associated with the
arcs of A.
 Cycle
– A cycle is a closed path (where x0= xn)
– An Euler cycle (or tour) of a graph is a cycle that covers each edge of the
graph exactly once
– An Hamiltonian cycle of a graph is a cycle that covers each node of the
graph exactly once
79
Definitions (5)
 Connectivity
– In a non-oriented graph G two nodes x and y are connected if a chain exists
between x and y
– A non-oriented graph is connected if each pair of nodes are connected
– In an oriented graph G two nodes x and y are strongly connected if a path
exists from x to y AND from y to x
– An oriented graph is strongly connected if each pair of nodes are strongly
connected
– A connected component of a graph G is a (strongly) connected subgraph
of G.
80
Definitions (6)
 Tree
– An acyclic graph (graph containing no cycle) is called a forest. A connected
forest is called a tree
– Let G be a graph with m nodes. The following statements are equivalent
 G is a tree
 Any two nodes of G are linked by a unique path or chain
 G contains m-1 edges and is connected
 G contains m-1 edges and is acyclic
 G is connected and acyclic
81
Graph problems
 3.2-Minimum-weight spanning tree problem

 3.3-Optimal path problem
 3.4-Maximum Network flow problem
82
3.2- Minimum-weight spanning tree
problem
 Example : minimizing the connexion cost in a network (road
network, data network, water network …)
 Considering a weighted graph G the problem consists in
searching a partial graph of G that connects each pair of
nodes for a minimum weight
 2 algorithms :
– Prim (1957)
– Kruskal (1956)
83
Example
 Let consider a campus to be provided with hot water from a
building (H) to 5 others (A, B, C, D, E). Suppose you have to
design the supply network in order to minimize the construction
cost. Let assume that the construction cost of a pipe link between
each pair of building is known.
 The problem can be modelled by the following graph, where each
node represents a building, each weighted edge (i,j) represents the
possibility to build a pipe link between the buildings i and j, with the
associated cost cij. The problem is then to find a minimum weight
spanning tree in this graph. B
1
A 1 2
3 3
1
2 3
H C
3
2
E 1 2
2 D 84
Prim algorithm
 Idea : starting from a tree reduced to one node. At each step the tree
grows by adding a connection to the "closest" node in terms of weight.
 Let G=(X,E,w) a weighted graph
– Let T={x} with xX (choice of x is free), ET=  ,
WT=0 (current weight of T),T’=X\{x}

– while T’ not empty
 for each y T’ let d(y) = minz T w(z,y)
 Cy = z T such that w(z,y) = d(y)
 choose u T’ such that d(u)=min {d(y) | y T’}
 T’=T’\{u}; T=T U {u} ; ET = ET U {(u,Cu)} ; WT=WT+d(u)
GT=(X,ET) is a minimum weight spanning tree for G, with weight WT

85
Kruskal algorithm
 Idea : search for minimum weighted connected acyclic components until
each nodes are connected
 Let G=(X,E,w) a weighted graph with m nodes

– Let ET = ; WT=0
– while | ET|<m-1
 select eE such that w(e)=min {w(e)/ eE},
 if e doesn’t create a cycle in ET then ET = ET U {e} ; WT=WT+w(e)
 E=E\{e}
GT=(X,ET) is a minimum weight spanning tree for G, with weight WT
86
3-3 Optimal path problems
 Shortest (or minimum-weight) path problems. Or longest path problems
 Lots of applications. Each time one have to carry something from one
point to another in a network, so as to minimize a cost, a distance, a
duration.
 Lots of sub-problems of great combinatorial problems can also be
expressed as optimal path problems. Specially in scheduling
problems.
 3.3.1 Shortest Path Problem (next slide)
3.3.2 Project Scheduling Problem (slide 94)
87
3.3.1 The Shortest Path Problem
 Three kinds of shortest path problems :
– Pb A : Given two nodes x and y, find a shortest path from x to y
(road itinerary for example)
– Pb B: Given one node x, find a shortest path from x to each other
node (choice of a destination for example)
– Pb C : Find a shortest path between each pair of nodes
(computation of a distance table)
 Problems A, B and C are linked.

 An algorithm able to solve pb A can be used several times to solve pb B
or pb C. An algorithm able to solve pb B (C respectively) will give the
results to pb A (or A and B respectively).
88
Algorithms to solve shortest path
problems
 Most of well known algorithms solve problem B

– For each node y, they compute a label which gives the
value of a shortest path from the origin node x to node y
 Two groups of exact algorithms for problem B
– Label setting algorithms (at each iteration of the
process one label is definitely computed)  Dijkstra
– Label correcting algorithms (each label is initialized
and then can be updated several times during the
process)  Bellman
89
Dijkstra's Algorithm
 Applicable in the case where all arc weights are nonnegative. Thus if
p is a path contained in a path p' the weight of p can't be greater than
the weight of p'.
 Dijkstra's algorithm is based on the following fundamental property of
shortest paths : suppose k is an intermediate node on a shortest 1-i path
pi, then the 1-k subpath pk of pi is a shortest 1-k path.
pi
1 2 3 pk k i
90
Dijkstra's Algorithm
 Let G=(X,A,w) ,X = {x0, … ,xN}. Dijkstra's algorithm finds a shortest path from x0
to each node xi  X
 Algorithm :
– Initialization
 vector V =[0, … ,min (w0i, ),…, min (w0N, )] (weight of paths) ;
 set D = {x0} (“labeled” nodes);
 vector P =[0, … ,0] (predecessors in the shortest paths)
– While |D|< N+1
 Select xj such that Vj= Minxi X\D (Vi)} (label of xj is set to Vj, the shortest past from
x0 to xj is found)
 D=D U {xj}
 For each xk (xj)\D if Vj+ wjk< Vk then Vk= Vj+ wjk ; Pk= xj
At each step, Dijkstra's algorithm selects the node xj with the minimum label and
extends the path x0-xj to each successor xk of xj if this new path improve the
current known path from x0 to xk.
91
Bellman's Algorithm
 Applicable in the general case where arc weights can be negative. Thus it can
help to detect negative-weight cycle in a graph.
 Based on the fact that a shortest path between two nodes in a graph of N+1
nodes shouldn't contain more than N arcs. If there is a x0-xi path containing
more than N arcs with a weight less than the shortest x0-xi path containing at
most N arcs, then the graph contains a negative-weight cycle.
 The idea is to start from a solution and iteratively improve the paths from x0 to
other nodes, decreasing at least one length at each iteration.
 If you can still decrease some path length after N iterations, then the graph
contains a negative-weight cycle (at least one shortest path should contain N+1
arcs which is contradictory).
92
Bellman's Algorithm
 Let G=(X,A,w) , x0 X = {x0, … ,xN} Bellman's algorithm finds a shortest path from x0 to
each node xi  X
 Algorithm
– Initialization
 k=0, vector V =[0, … , ] (weight of paths) ; vector P =[0, … ,0] (predecessors)
– Repeat
 No_improving = true
 For each xj X\ x0
– For each xi -1(xj)
 if Vj> Vi + wi j then
Vj = Vi + wi j (label of xj is updated with Vj)

Pj= xi
No_improving = false
 k=k+1
– Until (No_improving = true) or (k>N)
– If k > N then there is a negative-weight cycle
93
3.3.2 Project Scheduling Problem
 Let consider the following PSP. A project made of a set X of N tasks of
known duration di (i =1 to N) has to be achieved with sequencing
constraints between its tasks, so as to minimise the total duration of
the project.
 Given two tasks i and j, a sequencing constraint between i and j can be
stated as follow :
– tj  ti+di where tj is the starting time of task j means that tasks j can't start
before i is ended,
– tj  ti+3 means that tasks j can't start before 3 time units after i is started.
 This can be formulated with a graph, called AON (Activities On Nodes)
graph G=(X,A,w) where (i,j) A if there is a sequencing constraint
between i and j and wij = di (the constraint tj > ti+di states).
94
Example
 Project : build a house

# task 1 2 3 4 5 6 7 8
Task masonry roof Plumbing, facade windows garden ceiling/wall furnish
definition electricity painting
duration 7 4 8 2 1 1 5 1
Previous none 1 1 2,3 2,3 2,3 5 4,6,7
tasks
4
2 6 1
7 4
0 8 2 1
s 1 3 8 4 8 e
7 4
8
5
1
5 7
95
Minimal duration/ longest path
 s and e state as fictive tasks for the start and the end of the project.
 The global problem consists in searching a start time for each task of
the project so as to minimize the total duration.
 A sub-problem consists in searching the minimal duration of the
project.
 Each path p from s to e in the graph states for a sequence of constraints
that have to be satisfied. Its weight wp is a duration (a sum of durations)
which means that : te  ts+wp wp is a lower bound for the minimal
project duration
 This means that the minimal duration of the project is given by the
longest path in the associated AON graph.
96
Level decomposition of an AON graph
 Graph without cycle  level decomposition
– A graph without cycle can be decomposed into k levels Li (i=1 to k) with :
 each Li is a set of no-adjacents nodes (Li is stable)
 for each x Li :
(x) Li+1U …ULk
-1(x) L1U…U Li-1
 A level is a set of tasks that can be processed in the same time
L1 L2 L3 L4 L5 L6
4
2 6 1
7 4
0 8 8 2 1
s 1 7
3 4 8 e
8 4
1 5
5 7
97
Dijkstra's extension for the longest
path problem in a scheduling problem
 Rank nodes according to the level decomposition
– Nodes of a same level are free ranked, te is allways ranked at last
 Let y be the first node ty=0 , Pred (y) = s
 For each successive node x
– tx = max z -1(x)(tz+wzx )
– Pred (x) = z such that tx= tz+wzx
 te gives the minimal total project duration
In the example : t1= 0 Pred(1)=s; t2=7 Pred(2)=1; t3=7 Pred(3)=1; t6=15
Pred(6)=3; t4=15 Pred(4)=3; t5=15 Pred(5)=3; t7=16 Pred(7)=5; t8=21
Pred(8)=7; te=22 Pred(e)=8;
L1 L2 L3 L4 L5 L6
Critical path : 2 4
6 1
7 4
s, 1, 3, 5, 7, 8, e s 0 1 7 3 8 8 4 2 8 1
e
4
critical tasks : 1,3,5,7,8 8
1 5
5 7
98
Earliest start time/ lastest start time
 Search of the longest path gives also for each tasks its earliest start time ti
In the example : t1= 0; t2=7; t3=7; t4=15; t5=15; t6=15; t7=16; t8=21; te=22;
 From the minimal project duration one can easily find the latest start time t'i of
each task i :
– Rank the tasks in descending order of level decomposition
– For each successive node x (from e to s)
 If x=e : t'e= te else tx = min z (x)(tz - wxz )
 Succ (x) = z such that tx= tz-wxz
 For each critical task i, note that t'i = ti L1 L2 L3 L4 L5 L6

 In the example : t'3=7; t'5=15; 2 4
6 1
7 4
t'7=16; t'8=21 (critical tasks) s 0 8 8 2 1
1 7
3 4 8 e
t'6= t'8 -1=20 ; t'4= t'8 -2= 19; 8 4
5
1
t'2= min (t'6 – 4, t'5 – 4, t'4 – 4) = 11 5 7
99
Gantt diagram
 Schedule graph
4
6 Latest schedule
2
4
2 6 Earliest schedule
1 3 5 7 8
0 7 11 15 16 19 20 te=22
100
3.4 Maximum network flow problem
 Consider capacity on arcs (besides weight)

 Lots of applications : transportation, communication
networks, any fluid materials supplying networks
 Problem : search the maximum flow that can go
through the network – without exceeding the
capacity of any arc – from source node to sink
node (possibly for a minimum cost).
101
Examples
102
Definitions
 (G,c,s,t) is a network iff:
– G is an oriented graph
– Each arc (i,j) has a capacity c(i,j)
– s is the source node with in degree (d-(s)) null
– t is the sink node with out degree (d+(s)) null
 A feasible flow (across a network) is a function f : AR+ that
satisfies
– the balance constraint at each node : "all that goes in must go out"
(for each v N, v  s,t, u  -1(v) f(u,v) =  u  (v) f(v,u))
– and the capacity on each arc
(for each u,vN f(u,v)  c(u,v))
103
Example
Max f(t,s) ? Example of application :
The maximum flow +
problem is to find a 12 An oil company needs to ship oil
feasible flow that A C from a refinery to a storage facility
maximizes the flow 16 20 using the pipelines of its underlying
on arc (t,s) with no s 4 10 9 7 t distribution network. In this
exogenous supplies problem context, the refinery
13 4
or demands. 14 corresponds to a particular node s in
B D
the distribution network and the
storage facility corresponds to
another node t. The capacity of each
arc is the maximum amount of oil
Flow in v Flow out per unit time that can flow along it.
The value of a maximum s-t flow
determines the maximum flow rate
from the source node s to the sink
node t.
104
Definitions
 If a fictive arc (t,s) is added to the network, the value of a
flow across the network is the value of the flow on arc (t,s)
 An arc is saturated if the flow value on this arc is equal to
its capacity (A maximum flow contains at least one saturated arc)
 Example : flow of value 19
19
12
A 12 C
11 15
16 20
1 0
s 4 10 9 7 t
4 7
8
13 4
14 4
B D
11
105
Min cut problem
 A closely related problem to the MFP is the Minimum cut problem :
find a set of arcs with the smallest total capacity whose removal
separates node s and node t.
That is, there is no path from s to t after its removal
 Max-Flow Min-cut theorem :
12
The maximum value of the flow from a
A C source node s to a sink node t in a
16 20 capacitated network equals the
s 4 10 9 7 t
minimum capacity among all s-t cuts
13 4
B
14
D  Solving the MCP gives a solution to
MFP ! But solving the MCP is difficult …
106
Ford-Fulkerson's algorithm
 Algorithm to solve the MFP
 Based on iterative search of an Augmenting path
Path that can support a feasible flow that will increase the value of the global
flow.
 Definition : let G=(X,A) a directed graph and (G,c,s,t) an associated network with
a flow f.
– If (i,j) A, with f(i,j)< c(i,j), arc (i,j) is called a forward arc and is associated
with the residual capacity c(i,j)-f(i,j)
– If (i,j) A, with f(i,j)> 0 , arc (j,i) is called a reverse arc and is associated with
the residual capacity f(i,j)
 The residual network is the network containing the forward arcs and the reverse
arcs of ((G,c,s,t),f)
 An augmenting path for the flow f in (G,c,s,t), is a path from s to t in the residual
network.
107
Idea of FF's algorithm
Problem network
19
Residual network
12 12
A 12 C A C
11 15
11 15
16 20 10 1 5
1 0 5 5
s 4 10 9 7 t s 7 t
4 7 3 4
8 5 4
13 4 8
14 4
B D B 3 D
11 11
23
12
12 A C
A 12 C 11 15
11 19 10 1 5
16 20 5 5
0 1 s 7 t
s 10 4 9 7 t 3 4
0 7 5 4
12 8
13 4
14 4 B 3 D
B D 11
11
Increased flow (23) Augmenting path (+4)

(the flow is maximum when there
is no more augmenting path )
(the minimum capacity of the arcs
of the path gives the augmentation
108
value of the flow)
Idea of FF's algorithm (2)
Problem network
23 Residual network
12 12
A 12 C A C
11 19
11 19
16 20 10 1 9
0 1 5 1
s 10 4 9 7 t s 7 t
0 7 3
12 1 4
13 4 12
14 4
B D B 3 D
11 11
12
A C
11 19
10 1 9
5 1
s 7 t
3
1 4
STOP : Max f = 23 12
B 3 D
11
No path from s to t
109
Algorithm
 Each node i will be labeled with (p(i), ) where p(i) is the node from which i receives flow and  is the
amount of flow sent from p(i) to i
 Step 1 (initialization)
– f =0 (or any feasible flow)
– s is labeled (s,). All nodes are "unscanned", and all nodes except s are unlabeled.
– Let i= s
 Step 2 (scan node i)
– For all j such that (i,j)A, f(i,j) < c(i,j) and j is unlabeled, (i,j) is a forward arc
 label j (i, min (, c(i,j)-f(i,j)))
– For all j such that (j,i)A, f(j,i) >0, and j is unlabeled, (i,j) is a reverse arc
 label j (i, min (, f(j,i)))
 Step 3 : if t is labeled, go to step 4. Else, choose a labeled and unscanned node i and go to step 2 .
If none exists, the current flow is maximum
 Step 4 : t is labeled (p(t), ) . An augmenting path has been found.
– Use the first element of each label to trace back the path back to s
– Increase the flow by  on all forward arcs of the path and decrease the flow by  on all reverse
arcs.
– Erase all labels and return to step 1
110
3.5 Conclusion
 Graph theory offers models and exact methods to solve lots
of optimization problems.
 Integer programming and graph theory are basic tools in

operations research
 Airlines use these methods and techniques to make good
decisions in network design, fleet planning and crew
management.
 For example Air France began using operations research
tools to make crew planning in the 60’s.
111

IATOM17 OR CO Slideshow PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

IATOM17 OR CO Slideshow PDF

Uploaded by

Copyright:

Available Formats

Operations Research

Part 1: combinatorial optimization

Referent : Catherine Mancel

Operations research overlaps with other disciplines, notably industrial

 Understanding/ formulation of the problem

Modeling = describing in a formal unambiguous way.

Mathematical models are often used to describe systems or

Using a natural language (French, English) could also be a way to

An algorithm is a set of steps for solving a particular

To be an algorithm, this set of steps has to be unambiguous (each

1st Part : Combinatorial Optimization

Ref : Nemhauser and Wolsey “Integer and combinatorial optimization”, Wiley

2nd Part : Continuous Optimization

Continuous optimization is a topic of applied mathematics in which the

Linear programming = solving Linear Programs

Reminder : a linear equation is an algebraic equation in which each term

An optimization problem is composed of :

Mathematical (optimization) model :

If all equations of the mathematical optimization model are

Min. a cost/ max. a profit……….......min/max z =c1x1+c2x2+…+cnxn

a maximization problem with n variables and m constraints :

Right hand-side member

Profit (or cost) coefficients :

xi is called a decision variable of the problem, for each i=1 to n

a vector of values X= (x1, …, xn) is a feasible solution if and only if it

X= (x1, …, xn) is an optimal solution if and only if it is feasible and it

– 2 kinds of products can be loaded in the

– significant assumption : the products are

–How shall we load the aircraft so that to maximize the total

Which quantity of each product shall the factory produce so that to

We obtain exactly the same model!

 The model is built, now we have to solve the linear

 graphical representation of the example 2-variables

 Back to the example : consider a third type of product that

 How to solve this new problem ?

 for any n, the feasible domain is a convex

 Iso criteria domains are affine hyperplans (n=2

 The optimal value of the objective function, if any, is

a represents the gap between the

(max) z = x1+ 2x2

x1 and x2 can be considered as free variables, while z, a and b are

This linear system is called a dictionary

let set x1=x2=0 (easy to decide)

What should we do to increase z? 30

We have X=(0,0,9,12), z=0, we search for X’=(x1’,x2’,a,’b’) such that :

notice that increasing x1 or x2 (not both):

• x2 is a ‘’better candidate ‘’ than x1 (c2 >c1 in z)

 what is the maximal value for x2?

 To answer this, we need the same dictionary structure than previously:

 The solution process can be iterated !

 Standard form of a Linear Program :

• It is always possible to get rank(A)=m

 Let consider (P) :

 General Idea (Dantzig, 1949)

Until the optimal solution is found

add m slack variables in order to express P in its standard form:

Let (P) a LP with m+n variables and m constraints

 A dictionary is feasible if, setting right hand-side member

 Basic (feasible) solution : solution that can be described

Apply the Simplex Algorithm to the following linear

 Each linear program (P) is associated to its Dual

Primal problem with equality constraints

 let the pair of dual problems :

zL= 9.4 zP 9 (upper bound)