ColGen

620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 1
620
620
-
-
362
362
Applied Operations Research
Applied Operations Research
Column Generation
Column Generation
Department of Mathematics and Statistics
Department of Mathematics and Statistics
The University of Melbourne
The University of Melbourne
This presentation has been made in accordance with the provision This presentation has been made in accordance with the provisions of Part VB of the copyright s of Part VB of the copyright
act for the teaching purposes of the University. act for the teaching purposes of the University.
For use of students of the University of Melbourne enrolled in t For use of students of the University of Melbourne enrolled in the subject 620 he subject 620- -362. 362.
Copyright Copyright

2008 by 2008 by Heng Heng- -Soon Soon Gan Gan
Some contents of this presentation are adapted from year 2005 co Some contents of this presentation are adapted from year 2005 course notes for 620 urse notes for 620- -362 362
Applied Operations Research, Department of Mathematics and Stati Applied Operations Research, Department of Mathematics and Statistics, The University of stics, The University of
Melbourne (compiled by Prof Natashia Boland and Dr Renata Melbourne (compiled by Prof Natashia Boland and Dr Renata Sotirov Sotirov) )
Column Generation
Column Generation
Why is column generation used?
Why is column generation used?
LPs with very large numbers of variables need to be solved. LPs with very large numbers of variables need to be solved.
Why formulate problems with large numbers of (integer)
Why formulate problems with large numbers of (integer)
variables?
variables?
The The bounds bounds provided by their LP provided by their LP- -relaxations are much relaxations are much
better better than those of alternative compact formulations, . than those of alternative compact formulations, .e.g e.g
the cutting stock gap is well the cutting stock gap is well- -known to be known to be 1 generally, so 1 generally, so
very little branching is required to get integer optimum. very little branching is required to get integer optimum.
Symmetry is eliminated or reduced Symmetry is eliminated or reduced as compared to as compared to
compact formulations, leading to much smaller B&B trees. compact formulations, leading to much smaller B&B trees.
Complex constraints Complex constraints, e.g. regulations specified in , e.g. regulations specified in
Enterprise Bargaining Agreement (EBA) can be relegated to Enterprise Bargaining Agreement (EBA) can be relegated to
a a subproblem subproblem, and solved by specialised techniques. , and solved by specialised techniques.
What is column generation?
What is column generation?
A method for solving LPs in which
A method for solving LPs in which
there are very large number of
there are very large number of
variables, without having to enumerate
variables, without having to enumerate
all the variables
all the variables
a priori
a priori
.
.
Where has column generation been applied?
Where has column generation been applied?
Multi
Multi
-
-
commodity flow problems
commodity flow problems
Cutting stock problems

Cutting stock problems
Binary cutting stock problems

Binary cutting stock problems
Air crew scheduling

Air crew scheduling
Crew rostering
Crew rostering
Aircraft fleeting and routing

Aircraft fleeting and routing
Vehicle routing
Vehicle routing
Optical telecommunications design

Optical telecommunications design
Ship scheduling and routing

Ship scheduling and routing
A Quick revision on LP
A Quick revision on LP
1. 1. LP: LP: z z
lp lp
= = min{cx min{cx: : Ax Ax b, x b, x 0, 0, x x R R
n n
} }
2. 2. Very large LPS can be solved very efficiently with today Very large LPS can be solved very efficiently with today s s
software, e.g. software, e.g. Cplex Cplex, , Xpress Xpress
MP MP
. .
Usual method is still simplex method. Usual method is still simplex method.
3. 3. Dual LP: w = Dual LP: w = max{ub max{ub: : uA uA c, c, u u 0, 0, u u R R
m m
} }
4. 4. Simplex method maintains basis B for column space of A via Simplex method maintains basis B for column space of A via
subset of columns of A, indexed by B subset of columns of A, indexed by B {1, {1, ,n}. ,n}.
It maintains feasible x and dual variable u It maintains feasible x and dual variable u 0 with 0 with ub ub = = cx cx and strives and strives
to achieve to achieve uA uA c. c.
Simplex iteration exchanges two columns in basis, i.e. B:=B Simplex iteration exchanges two columns in basis, i.e. B:=B\ \{ {i}U{j i}U{j}, },
i i B B, , j j B B
To choose j: find most infeasible dual constraint, i.e. select To choose j: find most infeasible dual constraint, i.e. select
j j arg arg min(c min(c
k k
- - ua ua
k k
) < 0 ) < 0
where (c where (c
k k
- - ua ua
k k
) is the ) is the reduced cost reduced cost of of x x
k k
. .
Column Generation
Column Generation
What if A has a huge number of columns? How can we
What if A has a huge number of columns? How can we
solve the LP?
solve the LP?
Definition:
Definition:
We call the LP with a huge number of
We call the LP with a huge number of
columns the
columns the
Master LP
Master LP
.
.
Definition:
Definition:
If A
If A
s s
is a matrix formed from a subset of the
is a matrix formed from a subset of the
columns of A, and
columns of A, and
c
c
s s
is the corresponding
is the corresponding
subvector
subvector
of c, the
of c, the
LP LP
s s
: : z z
s s
LP LP
= = min{c min{c
s s
y y: : A A
s s
y y b, y b, y 0} 0}
is called the
is called the
Restricted LP
Restricted LP
.
.
Note: A
Note: A
s s
must be chosen so that LP is feasible and
must be chosen so that LP is feasible and
bounded below.
bounded below.
Algorithm to solve the Master LP
Algorithm to solve the Master LP
Step 0: Step 0:
Choose an initial set of columns A Choose an initial set of columns A
s s
. .
Step 1: Step 1:
Solve the Restricted LP to obtain optimal basic solution Solve the Restricted LP to obtain optimal basic solution y y and and
corresponding duals corresponding duals u u. .
Step 2: Step 2:
Note that Note that x x given by given by
x x
i i
= = y y
i i
, if column , if column i i is in A is in A
s s
; 0, otherwise ; 0, otherwise
is a basic feasible point for the Master LP and is optimal if is a basic feasible point for the Master LP and is optimal if
uA uA c. c.
Solve column generation Solve column generation subproblem subproblem: :
z z
sub sub
= = min{c min{c
i i
ua ua
i i
: : a a
i i
is a column of A} is a column of A}
If If z z
sub sub
0 then STOP: 0 then STOP: x x is optimal for Master LP. is optimal for Master LP.
Otherwise add Otherwise add a a
i i
such that such that c c
i i
ua ua
i i
= = z z
sub sub
to A to A
s s
and go to Step 1. and go to Step 1.
A Cutting Stock Problem
A cutting stock problem with 17m stock lengths
A cutting stock problem with 17m stock lengths
and with orders for 25 pieces of length 3m, 20
and with orders for 25 pieces of length 3m, 20
pieces of length 5m and 15 pieces of 9m.
pieces of length 5m and 15 pieces of 9m.
Let P be the set of all cutting patterns.
Let P be the set of all cutting patterns.
Let
Let
a
a
ip ip
denote the number of pieces of length
denote the number of pieces of length
l
l
i i
cut in pattern p, where l
cut in pattern p, where l
1 1
= 3, l
= 3, l
2 2
= 5 and l
= 5 and l
3 3
= 9.
= 9.
Let the demand for each length
Let the demand for each length
l
l
i i
be b
be b
i i
, where b
, where b
1 1
= 25, b
= 25, b
2 2
= 20, b
= 20, b
3 3
= 15.
= 15.
a
1p
a
2p
a
3p
Pattern p
Variable:
Variable:
x
x
p p
= number of times pattern p is cut
= number of times pattern p is cut
Let A be the constraint matrix, i.e. A = (
Let A be the constraint matrix, i.e. A = (
a
a
ip ip
)
)
0
3 , 2 , 1 ,
. .
min
x
i b x a
t s
x
i
P p
p ip
P p
p
Step 0
Step 0
:
:
Choose subset of columns corresponding to
Choose subset of columns corresponding to
cutting patterns (5 0 0)
cutting patterns (5 0 0)
T T
, (0 3 0)
, (0 3 0)
T T
, (0 0 1)
, (0 0 1)
T T
, i.e.
, i.e.
(
(
(
=
1 0 0
0 3 0
0 0 5
s
A
Step 1:
Step 1:
Solve
Solve
Optimal basic solution is
Optimal basic solution is
y
y
1 1
= 5, y
= 5, y
2 2
= 6 2/3, y
= 6 2/3, y
3 3
= 15
= 15
and optimal dual variables
and optimal dual variables
u = (1/5, 1/3, 1)
u = (1/5, 1/3, 1)
+ +
0
15
20 3
25 5
. .
min
: LP Restricted
3
2
1
3 2 1
y
y
y
y
t s
y y y
Step 2:
Step 2:
Solve column generation
Solve column generation
subproblem
subproblem
:
:
But what does it mean for
But what does it mean for
a
a
p p
to be a column of
to be a column of
the Master LP?
the Master LP?
a
a
p p
= (a
= (a
1p 1p
, a
, a
2p 2p
, a
, a
3p 3p
)
)
T T
is integer,
is integer,
0 and
0 and
satisfies
satisfies
3a
3a
1p 1p
+ 5a
+ 5a
2p 2p
+ 9a
+ 9a
3p 3p

17
17
( )

LP Master the of column a is
. .
1 , 3 / 1 , 5 / 1 1 min
p
p
a
t s
a
Therefore column generation
Therefore column generation
subproblem
subproblem
is
is
+ +
+ +
=
+ +
|
\
|
+ +
=
integer , 0 , ,
17 9 5 3
. .
3
1
5
1
max
1
integer , 0 , ,
17 9 5 3
. .
3
1
5
1
1 min
3 2 1
3 2 1
3 2 1
3 2 1
3 2 1
3 2 1
p p p
p p p
p p p
p p p
p p p
p p p
sub
a a a
a a a
t s
a a a
a a a
a a a
t s
a a a
z
Optimal solution:
a
1p
= a
2p
= a
3p
= 1
z
sub
= 1 (1/5 + 1/3 + 1)
= -8/15 < 0
Minimum reduced cost is Minimum reduced cost is negative negative, so column , so column
(1 1 1) (1 1 1)
T T
should be added to A should be added to A
s s
. .
We return to Step 1 and solve the new restricted LP: We return to Step 1 and solve the new restricted LP:
Optimal basic solution is Optimal basic solution is
y y
1 1
= 2, y = 2, y
2 2
= 1 2/3, y = 1 2/3, y
3 3
= 0, y = 0, y
4 4
= 15 = 15
and optimal dual variables and optimal dual variables
u = (1/5, 1/3, 7/15) u = (1/5, 1/3, 7/15)
+
+
+
+ + +
0
15
20 3
25 5
. .
min
: LP Restricted
4 3
4 2
4 1
4 3 2 1
y
y y
y y
y y
t s
y y y y
Column generation
Column generation
subproblem
subproblem
is
is
Optimal solution to
Optimal solution to
subproblem
subproblem
is
is
a
a
1 1
= a
= a
2 2
= a
= a
3 3
= 1
= 1
Therefore
Therefore
z
z
sub sub
= 1
= 1
(1/5+1/3+7/15) = 0
(1/5+1/3+7/15) = 0
+ +
+ +
=
integer , 0 , ,
17 9 5 3
. .
15
7
3
1
5
1
max
1
3 2 1
3 2 1
3 2 1
p p p
p p p
p p p
sub
a a a
a a a
t s
a a a
z
Thus there can be no negative reduced cost columns
Thus there can be no negative reduced cost columns
and optimal solution to Master LP is to use
and optimal solution to Master LP is to use
2 x (5 0 0)
2 x (5 0 0)
T T
,
,
1 2/3 x (0 3 0)
1 2/3 x (0 3 0)
T T
,
,
15 x (1 1 1)
15 x (1 1 1)
T T
Note:
Note:
This is not integer!! This is not integer!!
But LP value = 2 + 1 2/3 + 15 = 18 2/3 is lower bound on But LP value = 2 + 1 2/3 + 15 = 18 2/3 is lower bound on
problem. problem.
Observation: Observation: Now Now
2 x (5 0 0) 2 x (5 0 0)
T T
, 2 x (0 3 0) , 2 x (0 3 0)
T T
, 15 x (1 1 1) , 15 x (1 1 1)
T T
is an integer feasible solution with value 19. is an integer feasible solution with value 19.
Is this the optimal solution? Is this the optimal solution?
Sample
Sample
Xpress
Xpress
MP MP
implementation
implementation
Human Resource Planning (HRP)
Tasks to be performed: 1, 2, Tasks to be performed: 1, 2, , n , n
h h
i i
= time required for task i (hours) = time required for task i (hours)
H = max number of hours worker can work in a day H = max number of hours worker can work in a day
What is the minimum workforce required? What is the minimum workforce required?
{ } J s x
n i x
t s
x z
s
s i J s
s
J s
s

= =
=

, 1 , 0
, , 1 , 1
. .
min
: IP Master
:
K
{ }
=
=
n i y
H y h
s.t.
y u
z
i
n
i
i i
n
i
i i
sub
, , 1 , 1 , 0
max
1
: Subproblem
1
1
K
Solve Master LP
Solve subproblem
Add columns?
Yes
?
No
Initialise Columns
Sample
Sample
Xpress
Xpress
MP MP
implementation
implementation
1 (70) 1 (70)
33 33
33 33
384 384
50 50
1 (76) 1 (76)
23 23
23 23
390 390
40 40
HRP HRP
(Partition (Partition- -
Heuristic) Heuristic)
33 33 33 33 33 33
(~7.8%, 60s) (~7.8%, 60s)
MIP value MIP value
23 23 23 23 23 23
MIP value MIP value
33 33 30.425 30.425 30.425 30.425
LP value LP value
1 (60) 1 (60)
50 50 colgen colgen iters iters
141 (530) 141 (530) 63,023 (242) 63,023 (242)
# nodes # nodes
(# simplex iterations) (# simplex iterations)
83 83 1,734 1,734 1,734 1,734
# variables # variables
50 50 117 117 84 84
# constraints # constraints
50 tasks 50 tasks
1 (46) 1 (46)
48 48 colgen colgen iters iters
1,079 (443) 1,079 (443) 471 (204) 471 (204)
# nodes # nodes
(# simplex iterations) (# simplex iterations)
23 23 22.25 22.25 22.25 22.25
LP value LP value
71 71 984 984 984 984
# variables # variables
40 40 87 87 64 64
# constraints # constraints
40 tasks 40 tasks
HRP HRP
(Partition (Partition- -
ColGen ColGen) )
HRP HRP
(Assignment (Assignment - -
SymBreak SymBreak) )
HRP HRP
(Assignment) (Assignment)
Scenario
Scenario
A Vehicle Routing Problem (VRP)
An Air Crew Scheduling Problem
Further reading
Further reading
Winston Chapter 10.3
Winston Chapter 10.3
Next lecture
Next lecture
Column Generation Column Generation

& &
Branch Branch- -and and- -Bound Bound
Dantzig Dantzig- -Wolfe Decomposition Wolfe Decomposition

ColGen

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ColGen

Uploaded by

Copyright:

Available Formats

620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 1

Cutting stock problems

Binary cutting stock problems

Air crew scheduling

Aircraft fleeting and routing

Optical telecommunications design

Ship scheduling and routing

Column Generation Column Generation

You might also like