You are on page 1of 95

MULTILEVEL OPTIMIZATION

JRGEN TIND
Department of Operations Research
Institute for Mathematical Sciences
University of Copenhagen
1998
Kluwer Academic Publishers
Boston/Dordrecht/London
Contents
1. INTRODUCTION. 1
2. COLUMN GENERATION 5
2.1 Introduction 5
2.2 Crew Scheduling 8
3. LINEAR DECOMPOSITION 13
3.1 An Example 16
4. GENERAL DUALITY 19
4.1 General Mathematical Programming. 19
4.2 Nonlinear Programming. 23
5. FUNDAMENTAL DECOMPOSITION SCHEME 27
5.1 Saddlepoints. 27
5.2 The Decomposition Scheme. 30
5.3 Decomposition in Linear Programming. 32
5.4 Blockangularity and Economic Interpretation. 35
6. VARIABLE DECOMPOSITION. 39
6.1 General Procedure. 39
6.2 An Economic Interpretation. 41
6.3 The Separable Case. 43
6.4 An Example. 44
7. ROW DECOMPOSITION 49
7.1 General Procedure. 49
7.2 Duality in Mixed Integer Programming. 51
7.3 Decomposition by Cutting Planes 55
7.4 Mixed Integer Cuts. 59
7.4.1 A Numerical Example. 62
7.4.2 Appendix 65
7.5 Decomposition by Branch and Bound 70
v
vi
7.5.1 An Illustrative Example. 76
7.5.2 Unboundedness and Infeasibilities. 80
7.6 An Economic Interpretation. 81
7.7 Alternative Approaches. 82
8. MIXED DECOMPOSITION 85
8.1 Cross Decomposition. 85
8.2 Some Additional Remarks. 87
References 89
1 INTRODUCTION.
Quantitative modelling of decision problems in multilevel planning has a rela-
tively long history. The main model inspiration undoubtedly comes from the
decomposition procedures developed by Benders, 1962 and Dantzig and Wolfe,
1961. These models have inuenced most of the research work.
The book by Dirickx and Jennergren, 1979 contains a description of the
fundamental modelling principles for decomposable structures as well as a dis-
cussion of selected case exercises in economics and management. Obel, 1981
and Burton and Obel, 1984 discuss the modelling and implementation aspects
with particular attention to organizational design. A book on planning in de-
centralized rms has appeared by Meijboom, 1987. Even though this list is
by no means exhaustive, the above books represent and describe the major
research eorts done in mathematical modelling of decentralized planning.
Common to this area of research is the extensive use of mathematical pro-
gramming technique. In particular, linear programming duality plays a funda-
mental role. Moreover, the procedures operate with a series of smaller optimiza-
tion problems in an iterative way instead of considering one large optimization
problem.
The economic aspect of the procedures considers a central unit and a sub-
unit, that typically may consist of several subunits. The procedure makes it
possible to decentralize information for example on constraints, so that detailed
information about constraints in the central unit are unknown to the subunit,
and vice versa. Instead, information is processed between the units by means
1
2
of prices in the space of central constraints, which typically describe limita-
tion of some resources handled by the central unit. Those prices are usually
determined as the shadow prices, originating from linear programming duality.
However, the application of shadow prices restricts the mathematical mod-
elling of the interaction between the central unit and the subunit to be done
in the framework of linear programming or its direct extension, convex pro-
gramming, see Georion, 1970 and Holloway, 1973. This imposes linearity (or
convexity) restrictions that may not be fullled in an actual setting.
The purpose of this manuscript is to present one single quantitative frame-
work and a procedure without any of the above assumptions. Some progress has
already been made. Thus, Wolsey, 1981 discusses a generalization of Benders
decomposition procedure. In the Ph. D. thesis of Flippo, 1991 both procedures
are generalized in parallel.
The approach taken here is completely general. It includes all the classical
instances and opens for treatment of some new instances as well. One of the
most interesting aspects is the possibility to include discrete variables, which for
example can handle xed costs and other discrete cost structures. In addition,
it is here demonstrated that a majority of the economic interpretations done
in connection with the classical cases is carried over in full generality.
Of course, this cannot be done without any change in the specications of
the standard model parameters. Together with the removal of the linearity
assumptions then, by duality, more complicated pricing systems have to be
introduced to replace the shadow prices. In more technical terms, duality in
linear programming is replaced by general duality theory. However, it is pointed
out that the derivation of a pricing system in a particular case is intimately
connected with the actual algorithmic approach. In fact, the pricing system is
usually determined by price functions that are derived as a byproduct of an
algorithmic process.
The introduction of more complicated pricing systems might be unwanted.
It should, however, be observed that discreteness, for example in the size of
a production batch or in the selection of investment alternatives, in practice
gives rise to nonlinearities via the introduction of discounts, etc. It therefore
ought to be possible to impose the necessary information exchange between a
central unit and a subunit, also when nonlinearity occurs, by means of an ap-
propriately selected pricing system. The present manuscript contains material,
which conrms the existence of such pricing systems. This observation may be
useless for the complete general case. However, some special cases are of in-
terest. Here, particular attention shall be devoted to the integer programming
case.
The general model is a very simple max-min model. Due to its simplicity and
generality as well, it appears to be the correct choice of model. In the model to
be discussed improving upper and lower bounds for the optimal objective value
drop out immediately. They are important in case of premature interruption
of the procedure, for example due to slow convergence.
INTRODUCTION. 3
However, talking about convergence. It is truly of importance to know
whether a procedure converges or not. So some discussion is devoted to that
issue for the procedures to be developed. However focus is not put on the
computational eciency or complexity of the various procedures, for example
as a tool for solving large scale problem. The viewpoint is here from a systems
analysis point of you. So, in the descriptions we are going to emphasize the
interrelationships between the various procedures and to outline their economic
interpretations in a multilevel context.
The manuscript is organized as follows.
Chapter 2 introduces the important concept of column generation. This is
done simultaneously with the treatment of the multicommodity ow problem.
This problem was among the rst optimization problems that were treated my
the technique of column generation.
Chapter 3 gives an introduction to the application of column generation
technique in linear programming leading to the classical and important decom-
position procedure by Dantzig and Wolfe.
Chapter 4 presents the fundamentals of the duality theory to be used in the
following chapters and some special cases are discussed. However, for the case
of integer programming the appropriate duality theory is not discussed until
Section 7.2, where it is going to be used in the surrounding material.
In Chapter 5 we present the fundamental decomposition procedure, from
which we may derive all the subsequent procedures as special cases. Addition-
ally, some general conditions for convergence are stated.
According to classical development we present the two dual procedures, vari-
able decomposition and row decomposition separately in the next two chapters.
Chapter 6 presents a generalization of the Benders decomposition procedure.
First a general procedure is introduced followed by a treatment of the separable
case.
Chapter 7 presents a generel row decomposition procedure in addition to a
more detailed discussion of the integer programming case. This is considered in
the framework of the two major solutions technique for integer programming:
cutting plane and branch and bound.
Chapter 8 discusses various mixtures of the variable and row decomposition
procedures.
The text has been supplemented by some illustrative examples, which covers
some of the classical as well as some non-classical instances.
For convenience, the term sup is used almost everywhere, even at places
where the term max may be wanted and could be used instead. In this way
an optimal value (maybe innite) of an objective function always exists, even
though no optimal solution can be found. Also inf is used almost everywhere.
2 COLUMN GENERATION
2.1 INTRODUCTION
The purpose of this chapter is to give an introduction to the basic idea of column
generation, which is the major principle behind all decomposition methods in
mathematical programming. We shall do it by applying the principle on the
max-ow multicommodity ow problem, rst presented by Ford and Fulkerson,
1958. In the next section we shall treat a crew scheduling problem by column
generation.
The max-ow multicommodity ow problem can be stated in various forms.
We shall here use the so-called arc-chain formulation which is particularly suited
for column generation.
Consider an undirected network. Let m denote the number of arcs and let q
be the number of commodity types that may ow through the network. Each
commodity type has among the nodes a selection of sources, from which ow
of the commodity may originate, and a selection of sinks among the nodes
serving as destination points of the commodity. Thus, for each commodity
type ow may pass along dierent chains from a source to a sink. Usually
many such chains will exist. Assume that we know all chains and let n denote
the total number of chains. They may be represented by an mn matrix with
0-1 elements a
ij
, where a
ij
= 1 if the arc i is included in the chain j and 0
otherwise. There is an upper limit or capacity of total ow in each arc. Let
k
i
denote the capacity of arc i. Finally let f
j
be a variable denoting the size
5
6
of the ow along chain j. The objective is to maximize the total ow along all
chains subject to the capacity constraints of the arcs. This problem is stated
by the following linear programming problem.
max
n

j=1
f
j
s.t.
n

j=1
a
ij
f
j
k
i
for i = 1, . . . , m (2.1)
f
j
0 for j = 1, . . . , n.
Typically there may be be many chains corresponding to the many alterna-
tive routes for the ow of a commodity. This implies that n is very large and it
may be impossible to generate all chains as required in formulation (2.1). The
idea of column generation is to avoid the generation of all chains and instead
in an iterative manner generate fewer of those hopefully including the chains
to be used in an optimal solution.
Let us look at an illustrative example consisting of m = 6 numbered arcs
and q = 2 commodity types. Let s
1
, t
1
denote a source and a sink, respectively,
for commodity 1. Similarly let s
2
, t
2
denote a source and sink for commodity 2.
(In general the number of sources may dier from the number of sinks). The
sinks and sources are indicated on Figure 2.1.

d
d
d
d
d
dd
e
e
e
e
e
e
e
e
e
e
e
ee
s
2
= t
1
t
2
s
1
2
1
4
3
5
6
Figure 2.1. Multicommodity Flow Problem
The complete arc-chain matrix is included in Table 2.1, which also states
the capacities of the arcs.
Assume next that only some of the chains are known as given by Table 2.2.
Table 2.2 contains only three chains, two for commodity 1 and one for com-
modity 2. To be more specic let f
1
j
and f
2
j
denote ows in the chains corre-
sponding to commodity 1 and 2, respectively. With only those chains we shall
COLUMN GENERATION 7
Table 2.1. Complete Arc-Chain Matrix.
Arc Commodity 1 Commodity 2 Capacity
1 1 1 1 1
2 1 1 1 1 3
3 1 1 1 1 1
4 1 1 1 1
5 1 1 1 1 1
6 1 1 1 1 1
Table 2.2. Partial Arc-Chain Matrix 1.
Arc Commodity 1 Commodity 2 Capacity Dual Variables y
i
1 1 1 1
2 1 1 3 0
3 1 1 1
4 1 0
5 1 1 1
6 1 0
solve the partial problem (2.2).
max f
1
1
+ f
1
2
+ f
2
1
s.t. f
1
2
1
f
1
1
+ f
2
1
3
f
1
1
1
+ f
2
1
1
f
k
j
0 for j = 1, . . . , 3 and k = 1, 2.
(2.2)
The problem lists only four constraints as the current chains do not include
the arcs 4 and 6. The optimal value of the program is 3. The corresponding
basic dual variables to be denoted by y
i
are listed in Table 2.2. From linear
programming it is known that the solution obtained is also optimal for the
original problem if and only if the corresponding dual variables satisfy the
constraints of the dual problem. The dual constraints of the general problem
are
m

i=1
y
i
a
ij
1 for j = 1, . . . , n. (2.3)
Satisfaction of the inequalities (2.3) can here be checked by application of
a shortest path algorithm as follows. Let a dual variable indicate the length
of its corresponding arc. The length of a chain is then equal to the sum of
8
the lengths of the arcs on the chain. This means that the length of a chain j
is equal to

m
i=1
y
i
a
ij
, which is the left hand side of (2.3). A shortest chain
beween a source-sink pair for a commodity is easily determined by dynamic
programming for example by the shortest path algorithm of Dijkstra. If the
length is strictly below 1 then the corresponding row among the inequalities
2.3 is violated and the corresponding column shall be inserted into the partial
formulation. Otherwise an optimal solution has been found with the current
partial problem.
Presently we may among several possibilities introduce two new chains as
done in table 2.3, one for each commodity.
Table 2.3. Partial Arc-Chain Matrix 2.
Arc Commodity 1 Commodity 2 Capacity Dual Variables y
i
1 1 1 1
2 1 1 3 0
3 1 1 1
4 1 1 1 1
5 1 1 1
6 1 1 0
This implies an expansion of problem (2.2) to the problem
max f
1
1
+ f
1
2
+ f
1
3
+ f
2
1
+ f
2
2
s.t. f
1
2
1
f
1
1
+ f
2
1
3
f
1
1
1
f
1
3
+ f
2
2
1
+ f
2
1
1
+ f
1
3
1
f
k
j
0
(2.4)
for j = 1, . . . , 3 and k = 1, 2.
The optimal dual variables of problem (2.4) are listed in table 2.3. Using
them as the lengths of the arcs we see that no chain exists between a possible
pair of source and sink with length strictly less than 1. Hence all inequalities of
(2.3) are satised. An optimal solution is f
1
1
= f
1
2
= f
1
3
= f
2
1
= 1 and f
2
2
= 0
giving the objective function value 4. By solving (2.4) we have thus obtained
an optimal solution without explicit incorporation of all chains. This illustrates
the essential idea behind decomposition in mathematical programming.
2.2 CREW SCHEDULING
Application of column generation has been quite successful in solving crew
scheduling problems in the transport sector: airlines, bus companies etc. The
COLUMN GENERATION 9
problem is to assign people to consecutive tasks. Talking about airlines a task
may be the duty on a specic ight. Given the ight time schedule for some
period of time the objective is to assign crews to the ights while minimizing
the assignment costs.
There are in practice many restrictions in the formation of a feasible crew
assignment. Several regulations exist on the total duration for the same crew
on consecutive ights, on breaks, on overnights in foreign cities, free time etc.
The regulations are based on working agreements and security rules. However,
even with these regulations the number of possible schedules for a crew is
a tremendeously large. The goal is to generate those that will t well into
an overall schedule. Here the technique of column generation again plays a
role together with the formulation of the problem as the so-called set covering
problem.
Consider a selection of possible assignments, here called pairings, of crews
and ights in the following way. Let n denote the number of pairings considered,
and let m denote the number of ights to be manned by a crew. The pairings
of crews and ights are indicated by an m n matrix where a
ij
= 1 if the j

th
pairing covers the i

th ight. Let c
j
denote the cost of the j

th pairing for the


entire schedule of ights that it covers. Introduce a binary variable x
j
, where
x
j
= 1 if the pairing is used and 0 otherwise. The problem can be stated as
follows:
min
n

j=1
c
j
x
j
s.t.
n

j=1
a
ij
x
j
1 (2.5)
x
j
= 0 or 1 for j = 1, . . . , n.
The objective is to minimize the total cost, while the constraints (2.5) ensure
that all ights get a crew on board. In principle more that one crew may cover
the same ight. This possibility may be excluded by replacing the inequality
sign in (2.5) with an equality sign obtaining the so-called set partitioning model.
We shall not approach (2.5) directly but consider its linear programming
relaxation, i. e.
min
n

j=1
c
j
x
j
s.t.
n

j=1
a
ij
x
j
1 (2.6)
x
j
0 for j = 1, . . . , n.
10
Solve (2.6) and let y
i
denote the optimal dual solution, i = 1, . . . , m. Again
we shall consider the dual constraints taking all possible assignments of crews
into consideration:
m

i=1
y
i
a
ij
c
j
for j = 1, . . . , n. (2.7)
where n denotes the total number of all possible pairings. As discussed n
may be a very large number and an explicit test of all inequalities in (2.7) is
prohibitive. Nevertheless, we shall examine if a pairing j may be generated
together with its corresponding column such that

m
i=1
y
i
a
ij
> c
j
.
This question is to some extent answered by solving a shortest path problem
with side constraints. Construct an oriented network with nodes corresponding
to the ights. A directed arc (i, k) leads from one node i to another node k if
the ights corresponding to the nodes may be manned by the same crew in the
order of the arc direction. Let c
ik
denote the crew cost on the ith ight plus
the connection costs from ight i to ight k. Introduce directed arcs from all
nodes to a common termination point t to be called the homebase and let c
it
include the costs of the ith ight only. Similarly, draw directed arcs from the
homebase to all other nodes and with zero consts. Let A denote the set of all
arcs. This implies that the validity of (2.7) may be checked via the following
network ow program, where f
ik
is the ow in arc (i, k).
min

(i,k)A
(c
ik
y
i
)f
ik
s.t.

{k|(i,k)A}
f
ik

{k|(k,i)A}
f
ki
= 0 for i = 0, . . . , m
m

i=1
f
ti
= 1 (2.8)
m

i=1
f
it
= 1
f
it
0 for all (i, t) A.
Interpret c
ik
y
i
as the length of arc (i, k). Problem (2.8) computes the
length of a shortest chain, and this can easily be determined by a shortest
path algorithm. If a chain exists of total length strictly less than 0 then the
corresponding row of (2.7) is violated and the corresponding pairing should be
taken into consideration. The nodes passed on the chain correspond to the
ights covered by the new pairing. The corresponding column of 01 elements
a
ij
is added to (2.6) increasing the index n. Experience shows that this leads
to the generation of appropriate pairings without an explicit enumeration of all
pairings. However, some side conditions must be added to satisfy the previosly
mentioned regulations for crew assignments. Additionally, a main obstacle
may be that we treat only the LP-relaxation. Hence we might end up with a
COLUMN GENERATION 11
non-integer solution. Some rounding of the solution may be required or some
Branch and Bound technique applied to obtain an integer optimal solution. For
details see Desaulniers et al., 1994.
3 DECOMPOSITION IN LINEAR
PROGRAMMING
This chapter gives an elementary introduction of decomposition in linear pro-
gramming. The history goes back to the seminal paper by Dantzig and Wolfe,
1961. The purpose is to present the fundamental idea making some simplifying
assumptions to be removed later.
The basic model is a linear programming problem with two sets of constraints
to be stated as follows:
max cx
s.t. A
1
x b
1
A
2
b
2
(3.1)
x 0.
where c '
n
, A
1
'
mn
, b
1
'
m
, A
2
'
qn
and b
2
'
q
are given constants
and x '
n
is a vector of variables.
The fundamental idea is to solve (3.1) by interaction between two programs,
one of which only treats the rst set of constraints and the other the second
set of constraints. Denote the second set by
X = x 0 [ A
2
x b
2
.
Assume that X is bounded and nonempty. This assumption shall be removed
later. Let x
i
denote an extreme point of X for i P where P is the nite index
13
14
set of all extreme points. The polytope X can alternatively be represented as
the convex hull of the extreme points, i. e.
X = x [ x =

iP

i
x
i
, where

iP

i
= 1,
i
0 for i P.
See for example Bazaraa et al., 1990. Hence (3.1) is equivalent to
max

iP

i
cx
i
s.t.

iP

i
A
1
x
i
b
1
(3.2)

iP

i
= 1

i
0.
Problem (3.2) operates with fewer rows than the original formulation (3.1). The
variable x
i
has been substituted by the variables
i
. However, since the number
of extreme points is usually very large in comparison with the dimension n of
the problem, the number of -variables may also be very large. This implies
that a lot of eort is required to enumerate and calculate all extreme points.
Fortunately, this is unnecessary. In the spirit of column generation we only
need to know some of them. Additionally, by the theorem of Caratheodory at
most n + 1 extreme points are required. See for example Bazaraa et al., 1990.
The trouble is to nd the correct ones. For this purpose column generation
technique shall be applied. To simplify notation let w
i
= cx
i
and L
i
= A
1
x
i
for i P. This simplies (3.2) to
max

iP

i
w
i
s.t.

iP

i
L
i
b
1
(3.3)

iP

i
= 1

i
0 for i P.
Problem (3.2) or stated in the alternative form (3.3) is called the full master
problem since all extreme points are introduced in the formulation. As already
indicated we shall consider formulations dealing with only a subset of extreme
points. For this purpose let

P denote a subset of the index set P leading to a
tightening of (3.3), called the restricted master problem.
z = max

i
w
i
s.t.

i
L
i
b
1
(3.4)
LINEAR DECOMPOSITION 15

i
= 1

i
0 for i

P.
Assume for the moment that (3.4) is feasible. We shall later treat the more
general case. Hence an optimal solution exist with optimal dual variables de-
noted by y '
m
and v ' corresponding to the m+1 rows of (3.4). Moreover,
consider the dual constraints of the full master problem (3.2).
yL
i
+v w
i
for i P. (3.5)
By linear programming we know that an optimal solution has been found for
the full masterproblem (3.2) if and only if the dual solution (y, v) satises (3.5)
for all i P. Since it is the optimal dual solution of (3.4) we know that the
dual inequalies are satised for i

P, but there may be other extreme points
with i ,

P not satisfying the corresponding inequality of (3.5). Instead of
examining all extreme points of X the idea is to check the existence by solving
the so-called subproblem to be stated as follows.
u = max (c yA
1
)x
s.t. A
2
x b
2
(3.6)
x 0.
By assumption an optimal solution exists among the extreme points of X.
Let i

P denote the index of an optimal extreme point. Observe that the


objective function calculates the maximal value u of cx
i
yA
1
x
i
= w
i
yL
i
among all extreme points x
i
in X. Hence by (3.5) it remains to check if
u v. (3.7)
If so, then all constraints of (3.5) are satised and we may stop. Otherwise
introduce the elements (w
i
, L
i
) as a new column in the restricted master-
problem (3.4) and continue by solving it.
The above discussion can be summarized into the following
Algorithm
Step 1. Calculate the optimal dual solution (y, v) of the restricted
masterproblem (3.4).
Step 2. Determine an extreme point by solving the subproblem. If
(3.7) is violated expand the index set

P by including the extreme point
and go to Step 1.
Step 3. An optimal solution has been obtained by the solution of
the last master problem as x =

P

i
x
i
.
The number of extreme points in X is nite. Hence only a nite number of
mutually dierent columns may be introduced in the restricted master problem.
This implies that the algorithm must terminate in a nite number of steps and
thas has nite convergence.
16
3.1 AN EXAMPLE
We shall here illustrate the decomposition procedure by considering the prob-
lem:
max 3x
1
+ 4x
2
s.t. x
1
+ x
2
7
x
1
+ 2x
2
12
4x
1
+ x
2
20
(x
1
, x
2
) 0.
(3.8)
The problem has three constraints. The rst of the above constraints belongs
to the master problem and the remaining two to the subproblem. So
A
1
= (1, 1), b
1
= 7 and
X = (x
1
, x
2
) 0 [ x
1
+ 2x
2
12 and 4x
1
+x
2
20.
Assume that the extreme point x
1
= (0, 0) X is available at the outset
together with the dual variable values y = (0, 0) and v = 0. This implies that
w
1
= 0 and L
1
= 0. We may then state the
1. subproblem.
u = max 3x
1
+ 4x
2
s.t. x
1
+ 2x
2
12
4x
1
+ x
2
20
(x
1
, x
2
) 0.
An optimal extreme point is x
2
= (x
1
, x
2
) = (4, 4) with u = 28. Since uv > 0
this new extreme point does not satisfy (3.7). Hence we continue by calculating
w
2
= c
1
x
1
+c
2
x
2
= 34+44 = 28 and L
2
= A
1
(x
1
, x
2
) = (1, 1)(4, 4) = 8.
This gives us the
1. masterproblem.
z = max 0
1
+ 28
2
s.t. 0
1
+ 8
2
7

1
+
2
= 1

1
,
2
0.
An optimal solution is (
1
,
2
) = (
1
8
,
7
8
) with optimal dual variables (y, v) =
(
7
2
, 0). The optimal value of the objective function is 24
1
2
. The coecients for
the next subproblem are calculated by cyA
1
= (3
7
2
1, 4
7
2
1) = (
1
2
,
1
2
).
Hence we get the following problem.
2. subproblem.
u = max
1
2
x
1
+
1
2
x
2
s.t. x
1
+ 2x
2
12
4x
1
+ x
2
20
(x
1
, x
2
) 0.
An optimal solution is x
3
= (x
1
, x
2
) = (0, 6) with w = 3. Since uv = 30 > 0
the new extreme point does not satisfy (3.7). The corresponding coecients
for the next master problem are calculated according to w
3
= c
1
x
1
+ c
2
x
2
=
LINEAR DECOMPOSITION 17
3 0 + 4 6 = 24 and L
3
= A
1
(x
1
, x
2
) = (1, 1) (0, 6) = 6. We continue
with the
2. masterproblem.
z = max 28
2
+ 24
3
s.t. 8
2
+ 6
3
7

2
+
3
1

2
,
3
0.
For simplicity the rst column with 0-coecients corresponding to the vari-
able
1
has been removed. This turns the convexity equation of (3.4) into an
inequality, as shown. An optimal solution is (
2
,
3
) = (
1
2
,
1
2
) with optimal
dual variables (y, v) = (2, 12). The optimal value of the objective function
is 26. The coecients for the next subproblem are calculated as c yA
1
=
(3 2 1, 4 2 1) = (1, 2). Hence we get the
3. subproblem.
u = max x
1
+ 2 x
2
s.t. x
1
+ 2x
2
12
4x
1
+ x
2
20
(x
1
, x
2
) 0.
The optimal solution is (x
1
, x
2
) = (0, 6) with u = 12. This extreme point
was included in the last masterproblem. Hence the next master problem is
unchanged and we get u v = 12 12 = 0. At this stage all inequalites
of (3.7) are satised and the procedure stops. To calculate the optimal x-
values we use the last optimal -values and calculate (x
1
, x
2
) =

3
i=2
x
i
=
1
2
(4, 4) +
1
2
(0, 6) = (2, 5).
The polytope X has a total of 4 extreme points. In the last master problem
considered in the example we use 3 of those including the point (0,0). So in
this example the remaining extreme point (x
1
, x
2
) = (5, 0) is not required. As
the calculations also show the extreme point (0, 0) is superuous.
In general, of course, the aim is to include as few extreme points as possible
to get an early termination of the procedure.
Observe that the optimal solution for the original problem (3.8) is obtained
via a master problem. The last solution obtained from the 3. subproblem with
(x
1
, x
2
) = (0, 6) is not feasible in the rst row of constraints. However the 3.
subproblem is degenerate and the nal solution (x
1
, x
2
) = (2, 5) is among the
optimal points of the 3. subproblem. This is a general property since u = v
at termination. We shall later discuss the implications of this property in the
context of decentralized planning.
4 GENERAL DUALITY
4.1 GENERAL MATHEMATICAL PROGRAMMING.
Truly, duality is one of the most frequently used terms within the entire opti-
mization area. However, it is only in convex programming and in particular in
linear programming that this concept has penetrated in a really profound way.
It has there had a major impact on the development of algorithms, on economic
modelling issues and on the interpretation of theoretical and numerical results
etc.
There is an explanation for that: The duality theory for convex program-
ming is intimately connected to a wellknown vector, dened in the constraint
space with many interpretations. It has a geometric interpretation as the nor-
mal vector of a hyperplane dening a (convex) halfspace, containing a set of
feasible solutions. It has also an economic interpretation as the so-called Kuhn-
Tucker multiplier, or an algorithmic/theoretical interpretation as the so-called
Lagrange multiplier. The many interpretations explain the widespread use of
the duality concept. However, use of only a vector cannot extend the theory
beyond convex programming due to the possible existence of a duality gap.
The way to circumvent this obstacle is to extend the notion of a vector in
the constraint space to a function dened on the constraint space. An early
reference in this context is Everett, 1963. In this way most of the economic
and algorithmic results can still be kept. However, an operational treatment of
a function in place of a vector is more complicated.
19
20
Hence a careful selection of appropriate functions must always be done to
cover exactly the requirements needed for the establishment of a duality theory
for an actual problem or class of problems.
After these introductory remarks we shall develop the framework of general
duality. The material is based on the developments in Tind and Wolsey, 1981.
Let f(x) : IR
n
IR, g(x) : IR
n
IR
m
, X IR
n
, b IR
m
. Consider the
following general mathematical programming problem,
v = sup
x
f(x)
s.t. g(x) b (4.1)
x X.
Let IR = IR + be the set of extended real numbers. Let
T
m
(T
m
) be the set of all functions F: IR
m
IR(IR) and let T
m
+
(T
m
+
) denote
those functions in T
m
(T
m
) that are nondecreasing, i.e. F(d
1
) F(d
2
) for all
d
1
= (d
1
1
, . . . , d
1
m
), d
2
= (d
2
1
, . . . , d
2
m
) IR
m
satisfying d
1
i
d
2
i
for i = 1, . . . , m.
Consider a subset T T
m
+
. For (4.1) we dene the following dual program-
ming problem
t = inf
F
F(b)
s.t. F(g(x)) f(x) x X (4.2)
F T.
Consider also the perturbation function (d) : IR
m
IR of (4.1) dened by
(d) = sup
x
f(x)
s.t. g(x) d (4.3)
x X.
Let D = d IR
m
[ x X with g(x) d, i.e. D is the set of right hand
sides in (4.3) such that (4.3) is feasible. By convention we let (d) = for
d / D.
Consider additionally the program
inf
F
F(b)
s.t. F(d) (d) d D (4.4)
F T.
The next two theorems state equivalence properties among the above pro-
grams.
Theorem 4.1 The programs (4.2) and (4.4) are equivalent.
GENERAL DUALITY 21
Proof: We shall show that the programs have the same set of feasible solutions.
So, let F be feasible in (4.4). Then in particular F(g(x)) (g(x)) f(x)
for all x X. Conversely, if F is feasible in (4.2) and d D then, as F is
non-decreasing, F(d) F(g(x)) f(x) for all x x X [ g(x) d. Hence,
F(d) (d) for all d D.2
We say that the function class T supports at b if
(b) = inf
FF
F(b)
s.t. F(d) (d) d D. (4.5)
Theorem 4.2 (Duality theorem)
v = t if and only if T supports at b.
Proof: v = (b). Moreover, as (4.2) and (4.4) are equivalent by theorem 4.1
we obtain that v = t if and only if (4.5) is satised. 2
Two comments are in order here. First, theorem 4.2 is particularly useful
for economic and algorithmic purposes when (4.1) and (4.2) both have optimal
solutions. We shall return to this point. Secondly, it is of major interest to
dene an appropriate set of functions T. The support condition (4.5) can
always be fullled if T is chosen big enough, for example by having T =
T
m
+
. But of course it will be more interesting to narrow down the choice for
selection of T. The next sections shall demonstrate how the current framework
incorporates the duality theory for the classical cases, including convex and
integer programming, via appropriate selections of T.
If (4.5) is satised by an optimal solution F

of (4.2) or equivalently (4.4)


we say that is supported not only by the class F but by the function F

. In
order to give an economic interpretation of the models in this framework, the
next theorem will be useful.
Theorem 4.3 x

is an optimal solution of the primal problem (4.1) and F

supports at b if and only if


i) f(x) F

(g(x)) 0 for all x X,


ii) f(x

) F

(g(x

)) = 0,
iii) b g(x

) 0,
and iv) F

(g(x

)) = F

(b) < .
Proof: Let x

be an optimal solution of the primal problem (4.1) and let F

be a support of at b. Then iii) and i) follow by primal and dual feasibility,


respectively. By the support condition we have f(x

) = F

(b) < . Since


f(x

) F

(g(x

)) F

(b) by i), iii) and F

nondecreasing, we then get


f(x

) = F

(g(x

)) = F

(b) < , i.e. ii) and iv).


22
Conversely, assume i) . . . iv). Then x

is feasible in (4.1) by iii) and F

is
feasible in (4.2) by i). For arbitrary pairs (x, F), feasible in (4.1) and (4.2),
respectively, we have f(x) F(g(x)) F(b). By ii) and iv) f(x

) = F(b).
Hence (x

, F

) is optimal in (4.1) and (4.2), respectively. Moreover F

sup-
ports at b. 2
Next, an attempt shall be made to give an economic interpretation of the
above results, within the usual activity analysis framework.
Suppose that (4.1) is a problem of income maximization where b denotes
the amount of some available resources. So f(x) denotes the income and g(x)
the consumption of resources for a certain activity level indicated by the vector
x. The objective of the dual program (4.2) is to nd a function F, call it a
price function, for an estimation of the resource value. The price function is
nondecreasing (F T
m
+
). It must be selected so that no feasible process in the
technology makes a positive prot. This implies that f(x) F(g(x)) 0 x
X, i.e. i). Moreover, the total price F(b) paid for the available resources
should be minimized, which is the purpose of the program (4.2). Additionally,
if an optimal primal-dual pair (x

, F

) has been obtained, then, according to


theorem 4.2, the chosen process makes a zero prot (by ii)). Finally, the price
for the actual resources consumed is not changed by adding in parts or all of
the vector of unused resources (by iv)).
Since addition and subtraction are going to be involved in the following
material we shall for a moment restrict attention to functions with only nite
values. Hence T T
m
+
. Let L : IR
n
T
m
+
IR, where L(x, F) = f(x) +F(b)
F(g(x)) is the generalized Lagrangean function of (4.1).
Let us consider the following two programs.
z = sup
xX
inf
FF
L(x, F) (4.6)
and
w = inf
FF
sup
xX
L(x, F). (4.7)
Some simple results relating these programs and the previous ones shall be
stated.
Theorem 4.4 v z w t.
Proof: f(x) f(x) + F(b) F(g(x)) for x feasible in (4.1), as F is non-
decreasing. Hence v z. Similarly, w t. Finally z w since the in-
equality L(x
1
, F) sup
xX
L(x, F) for x
1
X implies inf
FF
L(x
1
, F)
inf
FF
sup
xX
L(x, F) for x
1
X. 2
Theorem 4.5 If T supports at b, then all the considered programs (4.1),
(4.2), (4.6) and (4.7) are equivalent, i.e.
z = v = t = w.
GENERAL DUALITY 23
Proof: By application of theorem 4.2 and theorem 4.4. 2
In light of theorem 4.5 one may therefore discuss whether (4.2) or (4.7)
shall be considered as an appropriate dual program of (4.1). They each have
their advantages. For a discussion of this point, see Ponstein, 1983. In our
discussions later we are going to consider more general forms of the function
L(x, F) than the Lagrangean function dened above. We are therefore going
to consider the programs (4.6) and (4.7) as the fundamental primal and dual
program, respectively, throughout the text. However, let us rst study the
classical case of nonlinear programming.
4.2 NONLINEAR PROGRAMMING.
We shall here study the classical development of a dual problem of the math-
ematical programming problem (4.1), in which X = IR
n
+
. Hence we are going
to consider the program
v = sup
x
f(x)
s.t. g(x) b (4.8)
x 0.
This is done by specifying the set of functions T to be the set of all nondecreas-
ing ane functions in IR
m
. So, let u IR
m
+
and u
0
IR. We then let the set T
= F [ F(d) = ud +u
0
, where d IR
m
. Accordingly the form of L(x, F) will
be f(x) ug(x) +ub.
The primal problem (4.6) undertakes with this specication the form:
sup
x0
inf
u0
f(x) ug(x) +ub. (4.9)
Since the vector u can be arbitrarily large the inner part is for x values
that are infeasible in (4.8). However, for feasible values of x the inner part is
equal to f(x). Hence the programs (4.8) and (4.9) are here equivalent.
The dual program (4.7) undertakes the form:
inf
u0
sup
x0
f(x) ug(x) +ub.
or equivalently the form of the dual programming problem (4.2):
inf
u,u
0
ub +u
0
s.t. ug(x) +u
0
f(x) x 0 (4.10)
u 0.
In the general statement of the duality theorem 4.2 we asked for a support of
the perturbation function . Since the support is here obtained by an ane
24
function dened by (u, u
0
), we get that is supported by an ane function if
and only if u, u
0
is an optimal solution of (4.10) and
(b) = inf
u0,u
0
ub +u
0
s.t. ud +u
0
(d) d D. (4.11)
We are in a moment going to utilize theorem 4.3, in which the existence of a
supporting function is considered. In the present context this means that there
exist optimal values (u

, u

0
) of (4.11), i. e.
u

b +u

0
= (b) and u

d +u

0
(d) d D.
or equivalently
(d) (b) +u

(d b) d D.
This coincides with the denition of a supergradient u

of at the point b.
Based on these considerations we can now rewrite theorem 4.3 into
Theorem 4.6 x

is an optimal solution of (4.8) and has a nonnegative


supergradient at b if and only if there exists (x

, u

) satisfying:
(a) g(x

) b,
(b) sup
x
(f(x) u

g(x)) = f(x

) u

g(x

),
(c) u

0,
(d) u

(b g(x

)) = 0.
Proof: Let x

be optimal in (4.8). Let also u

be a nonnegative supergradient
of at b. This means that a u
0
exists such that u

b+u
0
= (b) and u

d+u
0

(d) d D. This means further that the function F

dened by F

(d) =
u

d + u
0
is an optimal solution of (4.2), which currently has the form (4.11),
and is supporting at b as well. We can then use theorem 4.3. The statements
i) and ii) of that theorem imply that u
0
= f(x

) u

g(x

) f(x) u

g(x) for
all x 0, i. e. (b), and iv) implies (d).
Conversely, assume that (x

, u

) satisfy (a) . . . (d). Let F

(d) = u

d + u
0
where u
0
= f(x

) u

g(x

). Now (b) imply i) and ii) in theorem (4.3). More-


over, (d) imply iv), and iii) and (a) are the same. Hence, by theorem 4.3 we
get that x

is optimal in (4.8). Additionally, F

supports at b or equivalently
u

is a supergradient of at b. The supergradient is nonnegative by (c). 2


In the case that the functions f and g are dierentiable then
f(x

) u

g(x

) 0
and (f(x

) u

g(x

))x

= 0
(4.12)
GENERAL DUALITY 25
are necessary conditions for equation (b) in theorem 4.6. If in addition the
function f is concave and the elements of g are convex functions then the
conditions (4.12) are also sucient for (b) in theorem 4.6. The conditions (4.12)
are the classical Karush- Kuhn-Tucker conditions for a nonlinear programming
problem. If the functions are dierentiable as well as convex we can replace
condition (b) by the conditions (4.12). If the perturbation function is concave
(which is the case, if in particular f is concave and all elements of g are convex
functions) then theorem 4.6 becomes a version of the Karush-Kuhn-Tucker
theorem, for example as stated in Rockafellar, 1970, theorem 29.1.
Some comments shall also here be made with respect to the concept of an
augmented Lagrangean. An augmented Lagrangean adds a quadratic term of
the constraints into the Langrangean expression. We shall here briey mention
the augmented Langrangean introduced by Rockafellar, 1974. For a more de-
tailed discussion including a discussion of alternative augmented Lagrangeans,
see Minoux, 1986.
Let d = (d
1
, . . . , d
m
) IR
m
and u = (u
1
, . . . , u
m
) IR
m
. Further, let r IR
with r > 0, and let

d
i
= maxd
i
b
i
,
u
i
2r
. We then let T = T where
T = F [ F(d) =
m

i=1
(u
i

d
i
+r[

d
i
[
2
) +u
0
.
It is observed that the above class T T
m
+
. We can now state the following
Theorem 4.7 (Theorems 2 and 4 in Rockafellar, 1974). If F T is feasible
in (4.2) ((4.2) satises the quadratic growth condition), where T = T, then
(4.8) and (4.2) have equal objective function values, if and only if is upper
semicontinuous at b, i. e. (b) limsup
db
(d).
Rockafellar showed the theorem subject to the assumption that the objective
function of (4.7) = limsup
db
(d). However, since T is translation invariant,
i. e. T + u
0
T for u
0
IR then, as proven later by remark 5.2 we get that
the programs (4.7) and (4.2) are equivalent programs.
5 FUNDAMENTAL DECOMPOSITION
SCHEME
In this chapter we are going to establish the fundamental decomposition scheme.
The scheme is called fundamental because all decomposition procedures with
origin in mathematical programming apparently can be cast into this scheme.
Moreover, it represents a direct simultaneous generalization of the two original
and basic procedures of Benders, 1962 and Dantzig and Wolfe, 1961.
The chapter contains some introductory material in Section 5.1. In Section
5.2 we introduce the scheme itself. Section 5.3 considers the classical scheme
of Dantzig and Wolfe. Finally, in Section 5.4 we discuss the classical case
with a blockangular structure of the constraints. We also present the classical
economic interpretation of the procedure in hierarchical planning. Extensions
of the Dantzig Wolfe decomposition scheme shall later be discussed in Chapter
7. The Benders decomposition scheme and variations of this scheme will be
discussed in Chapter 6.
5.1 SADDLEPOINTS.
Many of the results in the previous Chapter 4 could be generalized to include
functions that are dened over more general spaces that the Euclidean space.
However since most mathematical programming problems are stated in that
way we refrained from doing that. However, in the present chapter we are
going to have a rather general starting point. So in this context we are going
to consider two arbitrary nonempty sets X and Y . The set Y is here going to
27
28
substitute the specic set of functions T studied in chapter 4. Let x denote an
element of X and let y denote an element of Y . Dene the function K(x, y) :
X Y IR + .
We shall next consider the following two fundamental problems. First we
state the primal program
z = sup
xX
inf
yY
K(x, y). (5.1)
Let z(x) = inf
yY
K(x, y). We speak of a feasible solution x
0
of (5.1) with
value z(x
0
) if x
0
X and z(x
0
) > . Furthermore, x
0
is an optimal solution
of (5.1) if it is feasible and if z(x
0
) = sup
xX
z(x).
By transposition of the operators sup and inf we shall also consider the dual
program
w = inf
yY
sup
xX
K(x, y). (5.2)
Let w(y) = sup
xX
K(x, y). Similarly y
0
is feasible in (5.2) if y
0
Y and
w(y
0
) < . Moreover, a feasible solution y
0
is optimal in (5.2) if w(y
0
) =
inf
yY
w(y).
This terminology is in concordance with the conventional terminology ap-
plied in mathematical programming. Consider for example the standard non-
linear mathematical programming problem (4.8) studied in chapter 4. So let
X = IR
n
+
, where IR
n
+
denotes the nonnegative elements in IR
n
. Equivalently
let u Y = IR
m
+
, where IR
m
+
denotes the nonnegative elements in IR
m
. Dene
K(x, y) as the usual Langrangean, i. e. K(x, u) = f(x) ug(x) + ub. In this
setting (5.1) and (4.8) are equivalent programs. In particular, x IR
n
+
is fea-
sible in (5.1) if and only if x is feasible in (4.8) in the conventional sense, i. e.
g(x) b.
We always have weak duality between (5.1) and (5.2), i. e. z(x
0
) w(y
0
)
for all (x
0
, y
0
) X Y . We speak of strong duality when there is no duality
gap between (5.1) and (5.2), i. e. when z = w.
We say that (x

, y

) (X Y ) is a saddlepoint of K(x, y) if
K(x, y

) K(x

, y

) K(x

, y) x X and y Y.
Theorem 5.1 There exists a saddlepoint (x

, y

) X Y if and only if x

is a feasible solution of the primal program (5.1), y

is a feasible solution of
the dual program (5.2) and z = w. In that case x

is also an optimal solution


of the primal program (5.1) and y

is an optimal solution of the dual program


(5.2).
Proof: Assume that x

is feasible in (5.1), y

is feasible in (5.2) and z = w. So


sup
xX
K(x, y

) = inf
yY
K(x

, y)
We always have
inf
yY
K(x

, y) K(x

, y

) sup
xX
K(x, y

).
FUNDAMENTAL DECOMPOSITION SCHEME 29
Hence
sup
xX
K(x, y

) = K(x

, y

) = inf
yY
K(x

, y).
So, (x

, y

) is a saddlepoint.
Conversely, assume that (x

, y

) is a saddlepoint. Then
sup
xX
K(x, y

) K(x

, y

) inf
yY
K(x

, y). (5.3)
Hence x

is feasible in (5.1) and y

is feasible in (5.2). Moreover (5.3) implies


that
inf
yY
sup
xX
K(x, y) K(x

, y

) sup
xX
inf
yY
K(x, y).
By weak duality we always have the opposite relation
inf
yY
sup
xX
K(x, y) sup
xX
inf
yY
K(x, y).
So
inf
yY
sup
xX
K(x, y) = K(x

, y

) = sup
xX
inf
yY
K(x, y).
Hence x

is optimal in the primal program (5.1) and y

is optimal in the dual


program (5.2) and z = w = K(x

, y

). 2
Observe that the above theorem is valid without any specic assumptions
on the sets X and Y and the function K(x, y). With specic selection of
those terms we can deduce from theorem 5.1 similar theorems in relation to
specic mathematical programming problem. Let us do this for the general
mathematical programming problem (4.1) considered in chapter 4. For this
purpose let Y = T with elements F T and let X IR
n
. Further, let K be
the generalized Langrangean L considered in Chapter 4. Hence
K(x, F) = L(x, F) = f(x) F(g(x)) +F(b).
As already noticed for the nonlinear programming problem we can establish
the following
Remark 5.1 If T contains all linear functions then x is feasible in (4.1) if and
only if it is feasible in (5.1). Moreover, in that case we have the same value of
the programs for a feasible solution.
Similarly consider the dual program (5.2) which here obtains the form
inf
FF
sup
xX
f(x) F(g(x)) +F(b). (5.4)
If F T is feasible in (5.4) then F +u
0
is also feasible in the same program for
all u
0
IR and without any change in the corresponding value of the program.
30
Remark 5.2 If T is translation invariant, i. e. F + u
0
T if F T then
any feasible solution may be translated so that sup
xX
f(x) F(g(x)) = 0. In
this case any feasible solution of (5.4) is also feasible in (4.2) and with the same
values of the objective functions. Conversely, if for any feasible solution F in
(4.2) we select u
0
= sup
xX
f(x) F(g(x)) then F +u
0
is also feasible in (5.4)
and we have the same objective function values.
Utilizing the above remarks and theorem 5.1 we thus obtain the following
result. Assume that T is tranlation invariant, i. e. F+u
0
T for all F T and
that T contains the set of linear functions. We then have that (x

, F

) XT
is a saddlepoint of the generalized Lagrange function L(x, F) if and only if i) x

is a feasible solution of the general mathematical programming problem (4.1),


ii) F

is a feasible solution (subject to translation) of the dual programming


problem, and iii) we have equal objective function values. In that case x

is also
an optimal solution of the primal program (4.1) and F

is an optimal solution
of the dual program (4.2) (subject to translation).
5.2 THE DECOMPOSITION SCHEME.
We are now ready to set up the fundamental decomposition procedure, which
has earlier been proposed in Tind, 1991. The basic idea is to consider a relax-
ation of the primal program (5.1). For this purpose consider a subset

Y Y
and the following problem called the upper problem:
z = sup
xX
inf
y

Y
K(x, y). (5.5)
Since z z we obtain that the upper problem (5.5) produces an upper bound
for the optimal value z of the primal problem (5.1).
Similarly, we consider the set

X X and the corresponding problem called
the lower problem:
w = inf
yY
sup
x

X
K(x, y). (5.6)
This problem produces a lower bound for the optimal value of the dual problem
(5.2).
It is of interest to consider instances in which w is also a lower bound of the
primal program (5.1). Similarly it is of interest to study instances where z also
produces an upper bound for the dual program (5.2). The next two remarks
characterize such instances.
Remark 5.3 If

X contains only one element x

X, i. e.,

X = x

then
w = inf
yY
K(x

, y) sup
xX
inf
yY
K(x, y) = z. Hence, w is also a lower
bound of the primal program (5.1).
Remark 5.4 If z = w, i. e. strong duality holds between the primal program
(5.1) and the dual program (5.2) then z = w w. Hence, w is also a lower
bound for the primal program (5.1) in this case, and by symmetry z is also an
upper bound of the dual program (5.2).
FUNDAMENTAL DECOMPOSITION SCHEME 31
In the following procedure we shall set up specications for the generation of
the subsets

X and

Y . The procedure is iterative. Let IN denote the set of nat-
ural numbers. Each iteration is enumerated by an index k IN. Furthermore,
let 2
X
denote the set of subsets in X. For each iteration k we introduce a map

X(k): IN 2
X
. With these specications we let the subset

X be dened by

X =

X(k) in the kth iteration. Similarly, introduce a map

Y (k): IN 2
Y
and
let

Y =

Y (k). In many of the special situations to be discussed later the above
maps do not change during the iterations. They are thus usually independent
of the iteration number. However, there are instances where this is not the
case. For example in cross decomposition to be discussed in Chapter 8.
Let

b, b IR

denote upper and lower bounds. They are


going to be updated during the course of the procedure.
We would like always to have lower and upper bounds. So we introduce
Assumption 5.1 In the following procedure we assume that (5.6) provides a
lower bound for the value z of the primal program (5.1).
This assumption is for example fulllled subject to the conditions stated in
Remarks (5.3) or (5.4).
The basic procedure aimed at solving the primal program (5.1) can now be
stated as follows.
Start: Let

X X and

Y Y be non-empty.
Let

b = +, b = and k = 1.
Step 1: Solve the upper problem (5.5).
Update

b := min

b, z.
Let

X =

X(k).
Step 2: Solve the lower problem (5.6).
Update b := maxb, w.
Let

Y =

Y (k).
Step 3: If

b = b then stop. Otherwise, let k := k + 1
and go to Step 1.
The sequence of Step 1 and step 2 may be interchanged, so that the procedure
treats a lower problem as the rst problem.
Observe that the setup is entirely symmetric in x and y. Hence an equivalent
scheme can be stated for the dual program (5.2) provided that an assumption
similar to assumption 5.1 is satised for the upper problem (5.5).
If, in step 1, z = then the upper problem (5.5) is infeasible. Since

Y Y we have in this case that also the primal program (5.1) is infeasible. In
this situation the procedure will stop in step 3 as

b = b = . This situation
will be discovered during the rst iteration.
Assumption 5.1 is used to ensure that we have obtained an optimal solution
value if the termination criterion is satised in step 3.
At this general stage of the development nothing more can be said about
convergence. However, it should be noted as advantage that the procedure at
32
all stages obtain improving bounds which provides an interval for the optimal
value z of the primal program (5.1). This will be of value in case of premature
interruption.
Of course the selection of the updating process for

X and

Y is crucial. To
simplify matters it is of advantage to keep those sets as small as possible.
However, they should hopefully be selected, so that the termination criterion
in Step 3 is satised at some stage of the process. The next theorem gives a
more precise characterization of the requested situation.
Theorem 5.2 If
(i) sup
x

X
inf
y

Y
K(x, y) = sup
xX
inf
y

Y
K(x, y)
and (ii) inf
yY
sup
x

X
K(x, y) = inf
y

Y
sup
x

X
K(x, y)
then z = w.
Proof: By assumption 5.1 we have that z z w.
By denition we have that z = sup
xX
inf
u

Y
K(x, y) and w = inf
yY
sup
x

X
K(x, y).
Generally sup
x

X
inf
y

Y
K(x, y) inf
y

Y
sup
x

X
K(x, y).
Hence by (i) and (ii) we get that z w. This implies that z = w. 2
Truly (i) and (ii) is fulllled if

X = X and

Y = Y . However, we wish (i)
and (ii) to be satised for smaller sets

X and

Y . In fact this is the essential
idea of decomposition.
Consider
z = sup
x

X
inf
y

Y
K(x, y).
This program is based on only the subsets

X and

Y and thus usually simpler to
treat than the original program (5.1). If the conditions (i) and (ii) are satised
it follows by theorem (5.2) that z = z, and the optimal value can thus be
determined with a smaller amount of work.
Typically,

X or

Y are expanded or otherwise changed during the iterations
of the procedure by the maps

X(k) or

Y (k) until the conditions (i) and (ii) are
satised. The objective is of course to make any expansion and the number of
changes as small as possible and still ensure convergence.
In the following chapters we are going to divide the various procedures into
dierent classes. Afterwards an actual procedure can the be outlined from the
treatment in the class to which the procedure belongs. However, it may be
natural already here to deduce one of the fundamental procedures for illustra-
tive reasons. Hence, the next section shall describe the classical Dantzig-Wolfe
procedure in linear programming.
5.3 DECOMPOSITION IN LINEAR PROGRAMMING.
In the treatment of this fundamental case rst studied Dantzig and Wolfe, 1961
we consider a linear programming problem with the constraints divided in two
sets of rows.
FUNDAMENTAL DECOMPOSITION SCHEME 33
So, we shall consider the following linear programming problem:
sup
x
cx
s.t. A
1
x b
1
A
2
x b
2
(5.7)
x 0.
where c IR
n
, A
1
IR
mn
, b
1
IR
m
, A
2
IR
qn
and b
2
IR
q
are given
constants and the variable x IR
n
.
Let X = x 0 [ A
2
x b
2
and Y = IR
m
+
. Let y IR
m
+
. Based on the
above program we dene the function K as the Lagrangean function K(x, y) =
cxyA
1
x+yb
1
. Assuming that X is nonempty we will see in moment that the
upper problem will be a linear programming problem with an optimal extreme
point solution x
i
X or an unbounded solution along an extreme ray x
l
of
X.

X(k) is then going to be updated recursively by

X(k + 1) =

X(k) x
i
or

X(k + 1) =

X(k) +
l
x
l
,
l
IR
+
, dependent on whether an optimal solution
or an extreme ray has been obtained. At the kth iteration, let K(k) denote an
index set of all extreme points obtained so far. Similarly, let R(k) be the index
set of all extreme rays obtained so far. The set

Y is here going to consist of a
single point, as

Y (k) = y
k
where y
k
is an optimal solution of the lower problem
(5.6).
With this terminology the upper problem (5.5) undertakes the form:
sup
x
(c y
k
A
1
)x (+y
k
b
1
)
s.t. A
2
x b
2
(5.8)
x 0
in the (k + 1)st iteration.
In the kth iteration the lower problem (5.6) obtains the form:
inf
y0
sup
iK(k),
l
0,lR(k)
cx
i
+
l
c x
l
yA
1
x
i

l
yA
1
x
l
+yb
1
.
Let v IR. We then transscribe this problem into the equivalent form
inf
y0, v
v
s.t. v cx
i
yA
1
x
i
+yb
1
for i K(k)
c x
l
yA
1
x
l
0 for l R(k)
or
inf v
s.t. v +y(A
1
x
i
b
1
) cx
i
for i K(k) (5.9)
yA
1
x
l
c x
l
for l R(k)
y 0.
34
Introduce dual variables
i
for i K(k) and
l
for l R(k). The dual of (5.9)
can then be formulated as
sup

i
,
l

iK(k)

i
cx
i
+

lR(k)

l
c x
l
s.t.

iK(k)

i
A
1
x
i
+

lR(k)

l
A
1
x
l
b
1
(5.10)

iK(k)

i
= 1

i
,
l
0 for i K(k) and l R(k)
utilizing that

iK(k)

i
b
1
= b
1
(

iK(k)

i
) = b
1
.
As seen directly the variable v in (5.9) is merely calculated as a biproduct of
y. Hence (5.9) is feasible if and only if the last set of constraints yA
1
x
l
c x
l
is feasible. So (5.9) has a feasible solution if and only if the following program
inf y 0
s.t. yA
1
x
l
c x
l
for l R(k)
y 0 (5.11)
has an optimal solution. By duality (or Farkas Lemma) this is so if and only if
sup

lR(k)

l
c x
l
s.t.

lR(k)

l
A
1
x
l
0 (5.12)

l
0 for all l R(k)
has an optimal solution. Otherwise, as seen, (5.12) is unbounded. Since the
variables are bounded by the constraint

iK(k)

i
= 1 we get that (5.12)
is unbounded if and only if (5.10) is unbounded. To summarize we have thus
obtained that (5.9) is infeasible if and only if (5.10) is unbounded.
This implies that (5.9) and (5.10) cannot be simultaneously infeasible. So by
duality we also have that (5.10) is infeasible if and only if (5.9) is unbounded.
Following standard convention and our own more general denition of feasi-
bility, given in the beginning of this chapter, we let the objective function of
(5.9) be + if (5.9) is infeasible. Similarly the value of (5.10) is if (5.10)
is infeasible. With this convention we get that the programs (5.9) and (5.10)
always have equal value.
It is customary at this point to require that the current set of extreme points
and extreme rays have been selected so that the program (5.9) is bounded, or
equivalently that (5.10) is feasible. If this is not the case, then a phase 1
procedure should naturally be applied to make (5.10) feasible, if possible. So,
we can assume that an optimal solution exists for (5.9).
FUNDAMENTAL DECOMPOSITION SCHEME 35
The programs (5.8) and (5.10) are here the classical version of the subprob-
lem and the master problem, respectively, considered in the Dantzig-Wolfe de-
composition procedure applied on problem (5.7). The Dantzig-Wolfe procedure
is presented by means of those versions in many textbooks. See for example
Bazaraa et al., 1990. These presentations are based on a utilization of the the-
orem of Minkowski, which states that the polyhedron X can be described as
the convex hull of its extreme points plus a nonnegative linear combination of
its extreme rays. However, the presentation above is based on the fact that an
optimal solution of the upper problem can be found as an extreme point of X
and that unbounded solutions can be detected along an extreme ray of X. In
stead of using Minkowskis theorem we are lead by linear programming duality
to the requested convex combination of extreme points and extreme rays by
solving (5.10).
As is wellknown, the procedure has nite convergence. In the above terms
this can be seen as follows.
Assumption 5.1 is satised. In fact, remark 5.4 is satised since there is no
duality gap between the primal problem (5.1) and the dual problem (5.2) by
linear programming duality. Since

Y is a single point we get that remark 5.3
is satised too, however with respect to the dual problem. Each lower problem
updates

Y so that (ii) in theorem 5.2 is always fullled upon exit of step 2 in the
procedure. Also (i) is eventually fullled, since the number of extreme points
and extreme rays is nite. Hence at some point

X is not changed keeping (ii)
satised without change in

Y . At termination the values of the primal problem
(5.1) which here has the form
sup
xX
inf
y0
cx yA
1
x +yb
1
(5.13)
and the dual problem which here has the form
inf
y0
sup
xX
cx yA
1
x +yb
1
(5.14)
coincide. Problem (5.13) is equivalent to the problem (5.7) at our starting point.
All the considered problems have equal value at termination, and an optimal
solution of (5.7) may be deduced from the optimal convex combinations found
in the nal problem (5.10). Similarly, problem (5.14) after dualization of the
inner part is equal to the dual programming problem of our starting problem
(5.7).
5.4 BLOCKANGULARITY AND ECONOMIC INTERPRETATION.
Frequently decomposition is utilized in situations in which either the set Y or
X is naturally divided into smaller sets. This often implies that either the
lower problem or the upper problem separate into several smaller problems to
be solved. Following a classical approach we shall discuss this issue on the
linear programming problem considered in the previous Section 5.3.
So the problem (5.7) shall be imposed a so-called blockangular structure. For
this let the variables be divided into p disjoint sets. So x = (x
1
, . . . , x
p
). A
1
is
36
separated into corresponding groups of columns, so that A
1
= (A
11
, . . . , A
1p
).
Similarly, the vetor b
2
is divided into p subsets of elements, so that b
2
=
(b
21
, . . . , b
2p
). Also c = (c
1
, . . . , c
p
). With respect to the constraints A
2
x b
2
it is assumed that no interrelationship exists between variables belonging to
dierent groups. So this set of constraints can be stated in the form:
A
2j
x
j
b
2j
for j = 1, . . . , p.
Hence the upper problem (5.8) separates into p problems. The jth problem
has the form:
z
j
= sup
j
(c
j
y
k
A
1j
)x
j
s.t. A
2j
x
j
b
j
(5.15)
x
j
0
for j = 1, . . . , p.
The upper bound is then calculated as
z =
p

j=1
z
j
+y
k
b
1
.
With the given separation we shall thus consider extreme points x
i
j
and
extreme rays x
l
j
of the constraints in (5.15) for j = 1, . . . , p. Let K
j
(k) and
R
j
(k), denote indices of a current set of extreme points and extreme rays,
respectively in iteration k for the constraints of the jth problem, where j =
1, . . . , p. We split up the variable v into v
j
IR where v =

p
j=1
v
j
. Then the
lower problem obtains the form:
w = inf
p

j=1
v
j
s.t. v
j
+y(A
1j
x
i
j
b
1
) c
j
x
i
j
for i K
j
(k)
and j = 1, . . . , p
yA
1
x
l
j
c
j
x
l
j
for l R
j
(k)
and j = 1, . . . , p
(5.16)
y 0.
With the upper problem divided into p problems it is natural to modify the
updating of the set

X(k), so that this may receive extreme points and extreme
rays from each problem independently of each other. The standard selection
is the following one. Consider among the p problems problem j as stated in
(5.15). The solution answer will be either an extreme point x
i
j
or an extreme
ray x
l
j
. In the extreme point case take the value v
j
from the previous lower
problem. Since v =

p
j=1
v
j
it is observed from that x
i
j
can increase the lower
bound value if and only if
v
j
< cx
i
j
yA
1
x
i
j
+yb
1
. (5.17)
FUNDAMENTAL DECOMPOSITION SCHEME 37
Similarly, it is observed for an extreme ray, that this can improve v if and only
if
c x
l
j
yA
1
x
l
j
> 0. (5.18)
Let X

denote the total set of extreme point solutions satisfying their respective
inequality (5.17), and let

X denote the halines equal to the set of extreme rays
satisfying their respective inequality (5.18). We then according to standard
approach update

X(k), so that

X(k + 1) =

X(k)

+

X.
The above blockangular formulation is of particular interest for activity anal-
ysis considerations in hierarchical planning. Consider a hierarchical system with
one central unit and p subunits. Information is divided between the central unit
and the subunits in the following way. The central unit alone has information
about the size of the vector b
1
, which naturally may stand for available central
resources to be distributed among the subunits. Each subunit operates by its
own activity levels to be determined by the vector x
j
for the jth subunit. Only
the jth subunit has information about some local restrictions to be given by
the constraints A
2j
x
j
b
2j
.
Let c denote the unit contributions to prot. The aim is to nd activity
levels x that will maximize overall prot, for example as dened by (5.7), by
means of the above iterative process. With the above division of information it
is natural to let each part of the upper problem, as given by (5.15), be treated
by the corresponding subunit.
Correspondingly, it is natural to let the lower problem be treated by the
central unit.
Now the iterative process may be interpreted as follows. By solution of the
lower problem the central unit quotes a price y for the consumption of the
central resources. The subunits have to pay the central unit for use of central
resources. The total cost for this is subtracted from the prot in each subunit
as done in the objective function in (5.15). Based on this reduced prot each
subunit optimizes their levels of activity x
j
. Suggestions are transmitted to the
central unit. Those suggestions are incorporated in the lower problem for cen-
tral evaluation if they may contribute to an improvement through satisfaction
of (5.17) (or (5.18)). Renewed optimization in the central unit leads to a new
price to be transferred to the subunits.
At termination the overall optimal solution x is determined by the central
unit in the following way. If we dualize the lower problem (5.16) in the same way
as done for the lower problem (5.9) we introduce dual varibles
j
i
corresponding
to the rst set of constraints in (5.16). Consider the optimal value of those
variables generated at the nal iteration k. We then get the optimal solution
x = (

iK
1
(k)

1
i
x
i
1
, . . . ,

iK
p
(k)

p
i
x
i
p
).
Above we have here limited ourselves to the discussion of the linear pro-
gramming case. In many instances in the following chapters it is also possible
to make of division of variables or constraints into groups corresponding to
subunits in a hierarchical system. As seen it requires technically only the intro-
duction of an additional index j. However, for simplicity we shall not do that.
We shall only consider systems with one subunit, but as demonstrated here it
38
can in many of the subsequent instances be extended to incorporate systems
with more than one subunit.
6 VARIABLE DECOMPOSITION.
6.1 GENERAL PROCEDURE.
In this chapter we are going to consider the Benders decomposition principle,
but in a more general context than originally introduced in Benders, 1962. A
major step in this direction has been taken by Georion, 1970 and Georion,
1972. Further important steps have been taken by Wolsey, 1981, who rst
presented a procedure using general duality theory. It is our purpose here to
outline and discuss such a procedure as a special case of the decomposition
procedure in Chapter 5.
For x IR
n
, v IR
p
dene f(x, v): IR
n
IR
p
IR and g(x, v): IR
n
IR
p

IR. Let X IR
n
and V IR
p
. We can now consider a general mathematical
programming problem with two sets of variables:
sup f(x, v)
s.t. g(x, v) b (6.1)
x X
v V.
We are going to solve this fairly general program by means of the decomposition
procedure in Chapter 5
39
40
Separate (6.1) into
sup
xX
_
sup
vV
f(x, v)
s.t. g(x, v) b
_
. (6.2)
For xed x we are going to consider the inner optimization problem in v of (6.2)
and on this problem we shall use the duality theory considered in Chapter 4.
Hence, we establish for xed x the generalized Lagrangean for the inner part
of problem (6.2). This generalized Lagrangean has the form
f(x, v) F(g(x, v)) +F(b).
We assume that T has been selected so that Theorem 4.5 can be used for all
considered x values. If we for xed x consider the dual of the inner part of
(6.2) in the form as dened by (4.7) then the above program (6.2) undertakes
the form
sup
xX
inf
FF
[F(b) + sup
vV
(F(g(x, v)) +f(x, v))]. (6.3)
We are going to apply the decomposition procedure from Chapter 5 on (6.3).
So in this case Y = T and K(x, F) = F(b) + sup
vV
(F(g(x, v)) + f(x, v)).
Furthermore, let x

be an optimal solution of (5.5) in step 1 at the kth iteration


and let

X =

X(k) = x

, i.e.

X is equal to the latest optimal solution. Let also
F

denote an optimal solution of (5.6) in step 2 and let



Y =

Y (k) =

T, where

T :=

T F

. That is

T is increased by the latest optimal solution.
If an optimal solution in (5.6) or (5.5) does not exist, the procedure must
be modied along the lines that are usually followed in the case of linear pro-
gramming. For example, this would replace an optimal solution by an extreme
direction in case of unboundedness, etc.
With these specications the program (5.5) has the form
z = sup
xX
inf
F

F
[F(b) + sup
vV
(F(g(x, v)) +f(x, v))]. (6.4)
This is a direct generalization of what in the Benders decomposition procedure
is denoted as the master problem.
The problem (5.6) undertakes the form
w = inf
FF
[F(b) + sup
vV
(F(g(x

, v)) +f(x

, v))], (6.5)
which is a generalization of the standard form of the subproblem in Benders
decomposition. We note that since

X = x

we get by Remark 5.3 that As-


sumption 5.1 is fullled in the present case.
We also note that

X = x

implies satisfaction of (i) in Theorem 5.2. Hence,


by this theorem the procedure stops if (ii) is satised, too, i.e. if
inf
FF
K(x

, F) = inf
F

F
K(x

, F) (6.6)
VARIABLE DECOMPOSITION. 41
where K(x

, F) = F(b) + sup
vV
(F(g(x

, v)) +f(x

, v)).
This equation expresses that the current selection

T T is sucient to
ensure the optimality of x

in (6.3) and hence in (6.1). In other words, all the


optimal solutions of (6.5) obtained so far and which all have been included in

T are enough to prove the optimality of x

. In particular if T is nite (and


no cycling occurs) then (6.6) must be true at some iteration. Hence in this
case the procedure has nite convergence. This is a frequently used argument
for nite convergence and, as shown here, it can be used also in a very general
setting.
For further characterization of criteria for termination we have the following
theorem.
Theorem 6.1 The procedure terminates, if
F(g(x, v)) f(x, v), F

T, x X and v V.
Proof: So, let sup
vV
(F(g(x, v)) + f(x, v)) 0 for all x X and F

T.
Then
F

(b) + sup
vV
(F

(g(x, v)) +f(x, v)) F

(b)
for all x X and F



T.
This implies that
inf
F

F
[F(b) + sup
vV
(F(g(x, v)) +f(x, v))] F

(b)
for all x X and F



T.
Hence
z = sup
xX
inf
F

F
[F(b) + sup
vV
(F(g(x, v)) +f(x, v))]
F

(b) (6.7)
for all F



T.
Now, consider the following program
w = inf
FF
F(b)
s.t. F(g(x

, u)) f(x

, u) v V. (6.8)
From our continuing assumption about the selection of T we obtain by Theo-
rem 4.5 that (6.5) and (6.8) are equivalent programs. The updating procedure
of the lower bound b therefore implies that F

(b) = w b for all optimal so-


lutions of (6.5). Similarly, the updating procedure of the upper bound implies
that z u. Hence, by (6.7) F

(b) u. Since in general b



b we obtain
b = F

(b) = u and the procedure terminates. 2


6.2 AN ECONOMIC INTERPRETATION.
We shall here give an economic interpretation of the procedure in the setting
from the previous section. It will be done within the usual activity analysis
42
framework. The problem here consists of two units in a planning environment.
These units may be of equal rank of importance. However, in agreement with
standard literature one unit shall be called the central unit and the other the
subunit. In most instances it will probably be natural to deal with more sub-
units, but for ease of exposition only one subunit shall be considered here.
The objective is to solve (6.1) by means of the decomposition procedure
where x indicates the activity levels to be determined by the central unit.
Similarly, the subunit determines the level v for its activities. The function
f(x, v) indicates the overall income. The vector b denotes the capacity of some
given production factors and g(x, v) denotes the utilization of those capacities
for given activity levels.
Here we assume that only the central unit has complete information about
the set X which restricts the set of feasible solutions. Also the subunit has
only complete information about the set V for its activities. Additionally, it
is assumed that the subunit has all information required to compute the inner
part of (6.2) for any given activity level x. However, instead of performing this
computation for all potential activity levels at the central unit, the subunit
provides cost information for these levels. This cost information is provided
by the cost function F

generated by the subunit as the solution of the lower


problem in the form (6.5).
Since F

in each iteration is going to be included in



T, the central unit
increases its cost information in this way. Based on this information new sug-
gestions x

for activity levels are proposed via the solution of the upper problem
in the form of (6.4). In particular, if termination has not yet occured then there
exists by Theorem 6.1 an activity level x X and a cost function F

T such
that
sup
vV
[f(x, v) F(g(x, v))] > 0. (6.9)
The function F(d) can therefore be interpreted as an estimation of the alternate
costs for the utilization of a resource vector d IR
m
. In this terminology (6.9)
expresses that by changing x a gain can be obtained in the overall income. So
the procedure continues.
In order to keep information decentralized, the subunit subsumes in principle
its knowledge into a function G(x) dened on the activity levels belonging to
the central unit. G(x):IR
n
IR + is given by
G(x) = F(b) + sup
vV
[f(x, v) F(g(x, v))] x. (6.10)
So we get a set

( of such functions, where

( = G [ F

T where
G(x) = F(b) + sup
vV
(f(x, v) F(g(x, v))).
In this terminology the problem of the central unit subsumes to
sup
xX
inf
G

G
G(x). (6.11)
VARIABLE DECOMPOSITION. 43
Hence, in the present interpretation the subunit generates the functions G(x)
to be used by the central unit. Since the subunit does not know in advance the
set X, the subunit must in principle compute G(x) by means of (6.10) for all
x IR
n
. This might at rst sight look laborious. However, taking the generality
of the model (6.1) into account, this should not be too surprising, since the
relationships between the activity levels x and v, given by the functions f(x, v)
and g(x, v) can be very complicated. In the next section we shall see how the
procedure simplies, when more structure is introduced on these functions.
6.3 THE SEPARABLE CASE.
This instance, which has also been considered in Burkard et al., 1985, assumes
separability of the objective function and the constraint function in the follow-
ing way. Let
f(x, v) = c(x) +e(v) and g(x, v) = h(x) +k(v),
where c(x) : IR
n
IR, e(v) : IR
p
IR,
h(x) : IR
n
IR
m
and k(v) : IR
p
IR
m
.
In this setup (5.1) undertakes the form
sup
xX
inf
FF
[F(b) + sup
vV
(F(h(x) +k(v)) +c(x) +e(v))]. (6.12)
We shall here see that a direct computation of G(x) for all x as done in the
economic interpretation of (6.10) can be avoided.
For each x IR
n
and F T introduce the function F
x
: IR
n
IR
+ dened by
F
x
(d) = F(d +h(x)) for all d IR
m
.
Then F(d) = F
x
(d h(x)) for all x IR
n
.
We assume that F
x
T for all x. This is not a very restrictive assumption
and is satised in most cases. In this way (6.12) is converted into
sup
xX
inf
F
x
F
[F
x
(b h(x)) + sup
vV
(F
x
(k(v)) +e(v) +c(x))]
or by replacing F
x
by F into
sup
xX
inf
FF
[F(b h(x)) + sup
vV
(F(k(v)) +e(v) +c(x))] (6.13)
Again, by Theorem 4.5 (6.13) is equivalent to
sup
xX
inf
FF
[F(b h(x)) +c(x)]
s.t. F(k(v)) e(v) for all v V.
Dene H = F T[F(k(v)) e(v). Hence (6.13), which is our present form
of (5.1), can be stated as
sup
xX
inf
FH
[F(b h(x)) +c(x)]. (6.14)
44
Application of the decomposition procedure then simplies the upper problem
to
sup
xX
inf
F

H
[F(b h(x)) +c(x)]. (6.15)
where

H H, and the lower problem gets the form
inf
FH
[F(b h(x

)) +c(x

)].
By direct insertion for example in (5.1) the actual problem has the form
sup c(x) +e(v)
s.t. h(x) +k(v) b
x X, v V.
In an economic context for this problem along the lines from section 6.2
then the variable x X may typically stand for investment activities to be
handled by the central unit and v V may stand for production activities
to be handled by a subunit. It is observed that this separable case does not
require the computation of the function G(x) for all x, as dened by (6.10),
in order to solve the upper problem as stated in the form of (6.11). To solve
the upper problem which is performed by the central unit, it suces here to
operate directly with the cost function F. This cost function indicates the value
of the resources bh(x

) left over to the subunit. Also the subunit needs not to


know the currently proposed activity level x

. Information about the available


resources bh(x

) suces together with the contribution c(x

) in the objective
function.
It is the purpose for the central unit via the solution of the upper problem
in the form of (6.15) to make an economic balance between the investment and
production activities based on all currently known cost functions

H.
Suppose the functions e(v) and k(v) are linear, i.e. e(v) = ev and k(v) = Av,
where e IR
p
+
and A IR
mp
. Assume also that V = IR
p
+
. In this case the
program (5.1) gets the form
sup c(x) +ev
s.t. h(x) +Av b
x X, v IR
p
+
.
This is the form on which the original Benders decomposition scheme is
applied. The decomposition is based on linear programming duality. So, in
this instance it suces to let H contain linear functions.
6.4 AN EXAMPLE.
Consider the following integer, convex programming problem
sup 3x
1
+ 2x
2
v
2
VARIABLE DECOMPOSITION. 45
s.t. x
1
+x
2
v 0
x
1
, x
2
0 and integer
v 0.
Thus in this setting X = (x
1
, x
2
) 0 and integer and V = IR
+
. Moreover,
since we for xed (x
1
, x
2
) are left with a convex programming problem our
function class T will consist of ane nondecreasing functions, i.e.
T = F : IR IR [ F(d) = ud +u
0
, u IR
+
and u
0
IR.
Select as a starting point

X = (x
1
, x
2
) = (1, 1) and

Y = . We let the proce-
dure start with a lower problem:
1. lower problem.
We shall consider the dual of
sup v
2
s.t. v 2
v 0
which in the form of (4.10) is
inf
u0,u
0
2u +u
0
s.t. uv +u
0
v
2
v 0.
We shall here take the opportunity to consider some alternative formulations,
that are typically considered in convex programming.
First we have by elimination of u
0
the Lagrangean form:
inf
u0
2u + sup
v0
(v
2
+uv).
Using optimality conditions for the dierential function to be maximized we
have alternatively
inf
u,v
2u v
2
+uv
s.t. 2v +u 0
(2v +u)v = 0
u 0
or
inf
u,v
2u +v
2
s.t. 2v +u 0
(2v +u)v = 0
u 0
46
An optimal solution is u = 4, v = 2, implying that u
0
= 4. The lower bound
b = 3x
1
+ 2x
2
2u +u
0
= 3 1 + 2 1 2 4 + 4 = 1.
1. upper problem.
Here we are going to consider
sup
(x
1
,x
2
)X
inf
(u,u
0
)

F
u(x
1
x
2
) + 3x
1
+ 2x
2
+u
0
= sup
(x
1
,x
2
)X
4(x
1
x
2
) + 3x
1
+ 2x
2
+ 4
= sup x
1
2x
2
+ 4
s.t. (x
1
, x
2
) 0 and integer.
An optimal solution is (x
1
, x
2
) = (0, 0) providing an upper bound

b = 4.
2. lower problem.
The problem to be considered is
inf u
0
s.t. uv +u
0
v
2
v 0
An optimal solution is u = u
0
= 0. The lower bound is 0, so we cannot improve
the existing value of b.
2. upper problem.
We are here going to solve
sup z
s.t. z x
1
2x
2
+ 4
z 3x
1
+ 2x
2
x
1
, x
2
0 and integer
z IR.
(x
1
, x
2
) = (1, 0) is an optimal solution and we can update

b = z = 3.
3. lower problem.
We consider
inf
u,u
0
u +u
0
s.t. uv +u
0
v
2
v 0
u 0
An optimal solution is (u, u
0
) = (2, 1). The lower bound b is updated to
3 1 2 + 1 = 2.
VARIABLE DECOMPOSITION. 47
3. upper problem.
The problem to be considered is
sup z
s.t. z x
1
2x
2
+ 4
z 3x
1
+ 2x
2
z x
1
+ 1
x
1
, x
2
0 and integer
z IR.
An optimal solution is (x
1
, x
2
) = (1, 0). The upper bound is updated to

b = z = 2.
Since now

b = b the procedure terminates. The solution (x
1
, x
2
) = (1, 0)
obtained in the last upper problem is part of an optimal solution. The optimal
value of v is determined by solving the inner part of (6.2) which here obtains
the form
sup v
2
s.t. v 1
v 0
We get the optimal solution v = 1.
There exists an alternate optimal solution (x
1
, x
2
, v) = (2, 0, 2) which is also
provided by the last upper problem.
7 ROW DECOMPOSITION
7.1 GENERAL PROCEDURE.
In chapter 6 we considered two separate sets of variables. In an economic
activity analysis context we discussed that one set of activities may indicate
levels of activities for a central unit in an organizational structure. Conversely,
the other set may indicate levels of activities for one (or possibly more) subunits.
In chapter 6 an iterative scheme was proposed in which the central unit issues
proposals for its level of activities. A response is given from the subunit in form
of a cost function. This estimates the value of the central activities, based on
the constraints interrelating the central unit and the subunit.
The approach in this chapter takes a dual starting point. We are here going
to consider a generalization of the Dantzig-Wolfe decomposition scheme. This
is based on a mathematical programming problem in which the constraint set
is divided into two sets of rows. In an economic hierarchical structure one
set of constraints may dene limitations of resources at the central level in the
organizational structure. Typically, those limitations will be of a more strategic
nature, and the size of the limitations is only known by the central level. The
other set may describe resource limitations for activities in the subunit (or
subunits in case of more than one subunit). Those limitations may be of a
more technical nature. The structure and size of those contraints are only
known by the subunit. In the corresponding iterative scheme the central unit
proposes a cost function evaluating the consumption of the resources which are
49
50
going to be allocated from the central unit to the subunit. We have earlier in
Section 5.3 studied this scheme in the classical case of linear programming and
in Section 5.4 we have discussed the economic interpretation of the procedure.
As in Chapter 6 we are going to take a rather general starting point. In
particular, there will be no assumption on linearity from the beginning. Thus,
as in Chapter 6 we are going to use the general duality theory presented in
Chapter 4.
After those introductory remarks we are going to consider a specialization
of (4.1) in which the constraints
g(x) b
are divided into
g
1
(x) b
1
g
2
(x) b
2
,
where g
1
: IR
n
IR
p
, g
2
: IR
n
IR
q
, b
1
IR
p
, b
2
IR
q
. We change notation
so that the set X is replaced by X
1
IR
n
.
Hence we are going to consider the program
sup f(x)
s.t. g
1
(x) b
1
(7.1)
g
2
(x) b
2
x X
1
.
Our set X is then being dened by
X = x [ x X
1
and g
2
(x) b
2
.
Correspondingly, our set of functions T is dened by
T = F : IR
p
IR [ F(g
1
(x)) f(x) x X
1
.
With those specications we are going to consider the following generalized
Lagrangean L(x, F) dened by
L(x, F) = f(x) F(g
1
(x)) +F(b
1
).
We shall apply the fundamental decomposition procedure on the present set-
tings. We require that T is selected in such a way that any duality gap is
closed. So, in this case Theorem 4.5 can be applied. Hence by Remark 5.4 the
Assumption 5.1 is satised, too. So the upper and lower problems respectively
provide valid bounds. As usual let

X X. We also require that Theorem
4.5 can be applied if X is replaced by any subset

X. In this setting the lower
problem obtains the form
w = inf
FF
sup
x

X
f(x) F(g
1
(x)) +F(b
1
)
ROW DECOMPOSITION 51
which by the above stated assumptions is equivalent to
inf
FF
F(b
1
)
s.t. F(g
1
(x)) f(x) x

X. (7.2)
We are also going to consider a subset

Y =

T T. If (7.2) has an optimal
solution F

we are going to use the updating rule



Y =

T := F

. This rule is
adopted from the classical Dantzig-Wolfe decomposition procedure. If (7.2) is
feasible, but has no optimal solution, then take any feasible solution, preferably
not considered before. If no feasible solution exists, then (7.2) with

X = X is
infeasible too, and no nite optimum is going to exist for problem (7.1). This
observation follows the more general comments stated after the presentation of
the basic procedure in Chapter 5.
The upper problem (5.5) undertakes the form
z = sup
xX
1
f(x) F

(g
1
(x)) [+F

(b
1
)] (7.3)
s.t. g
2
(x) b
2
.
Assume that (7.3) has an optimal solution x

. Again adopting the classical


Dantzig-Wolfe decomposition procedure

X is updated by

X :=

X x

. If no
feasible solution exists then the original problem (7.1) is infeasible too. If the
problem is feasible, but has no nite optimal solution then take any feasible
point or set of points, preferably not seen before.
Observe, that if all the considered functions are linear and X polyhedral,
then the procedure becomes the classical Dantzig- Wolfe procedure earlier stud-
ied in Section 5.3.
Variations also exist so that the updating procedure of

Y has no direct con-
nection with the outcome of the lower bound calculations in the lower problem
(7.2) as stated here. See for example the linear programming case, as studied in
Gon et al., 1993. I this study

Y is selected as the so- called analytical center,
which is obtained by a variant of Karmarkars projective algorithm. Numerical
experiments show good performance with this idea.
Talking in the economic terms as done in the beginning of this section then
the constraints g
1
(x) b
1
are going to express the limitation of resources at the
central unit. Similarly, the constraints g
2
(x) b
2
describe resource limitations
in the subunit. The upper problem is here going to be treated by the subunit.
This problem provides suggestions of activities to be considered by the central
unit. Similarly, the central unit is going to treat the lower problem (7.2). The
solution of this problem quotes a price function on the value of the central
resources.
In the next section we shall utilize the above development in the context of
integer programming.
7.2 DUALITY IN MIXED INTEGER PROGRAMMING.
In order to discuss row decomposition in integer programming we are cast into
the framework of mixed-integer programming. We must therefore rst discuss
52
some preliminary results from this area. So let us consider a general mixed
integer programming problem of the form:
(b) = sup

jI
c
j
x
j
+

jJ
e
j
y
j
s.t.

jI
a
j
x
j
+

jJ
h
j
y
j
b (7.4)
x
j
0 and integer for j I
y
j
0 for j J.
where c
j
, and e
j
IR, a
j
, h
j
and b IR
m
are given constants. x
j
and y
j
IR
are variables dened over the nite index sets I and J, respectively.
Let us again consider functions F : IR
m
IR. We say that F is a super-
additive function if F(d
1
+ d
2
) F(d
1
) + F(d
2
) for all d
1
, d
2
IR
m
. In this
context we are going to choose the set of dual functions T as
T = F : IR
m
IR [ F is superadditive
F is nondecreasing and F(0) = 0.
(7.5)
Furthermore, let

F denote the directional derivative of F, i. e.

F(d) = lim
0
+
F(d)

.
We assume that

F exists and is nite for all d IR
m
.
We can now state the following
Lemma 7.1 Assume that values exist for b such that (7.4) has an optimal
solution. Then the perturbation function dened by (7.4) is superadditive,
nondecreasing and (0) = 0, i. e. T as dened by (7.5). Moreover (h
j
)
e
j
for all j J provided that (h
j
) exists with a nite value. Additionally,
(a
j
) c
j
for all j I.
Proof: Let d
1
and d
2
IR
m
. Subject to existence let (x
1
, y
1
) denote a feasible
solution of (7.4) with b = d
1
. Similarly, let (x
2
, y
2
) denote a feasible solution of
(7.4) with b = d
2
. Then (x
1
+x
2
, y
1
+y
2
) is feasible in (7.4) with b = d
1
+d
2
.
Hence (d
1
+ d
2
) (d
1
) + (d
2
). If (7.4) is infeasible for b = d we set by
convention (d) = . We thus always have that (d
1
+d
2
) (d
1
) +(d
2
)
subject to the convention that + = .
As the set of feasible solutions of (7.4) do not decrease by increasing right-
hand side b we have that (b) is nondecreasing in b. (0) can either take that
value 0 or +. In the last case (b) = + for all b-values. This contradicts
the assumption that there exist values of the righthand side b so that (7.4)
obtains an optimal solution.
By direct insertion (h
j
y
j
) e
j
y
j
for all y
j
0, j J. Hence with = y
j
we get that
(h
j
)

e
j
for > 0. Therefore (h
j
) = lim
0
+
(h
j
)

e
j
provided that (h
j
) exists with a nite value.
Finally, select an index j I and let b = a
j
. Let x
j
= 1, x
i
= 0 for i I
i ,= j and y
j
= 0 for all j J. This solution is feasible in (7.4) with the selected
right hand side. Hence (a
j
) c
j
. 2
ROW DECOMPOSITION 53
Hence by Lemma 7.1 we have that (b) T. So, it suces to consider dual
functions F T dened by (7.5) in order to close the duality gap, for example
as done by Theorem 4.2 or Theorem 4.5. It is even more interesting to note
that a subclass of T is enough to do the job. An actual subclass is determined
for example implicitly via the selection of solution method. We shall discuss
this point in more detail later in Section 7.4.
We shall state another lemma, the contents of which may be found in
Nemhauser and Wolsey, 1988.
Lemma 7.2 Given a nondecreasing superadditive function F(d), for which

F(d) is dened and nite for all d, it follows that (i)



F is positively homo-
geneous of degree 1, (ii)

F is dominated by F, (iii)

F is superadditive and (iv)

F is nondecreasing.
Proof: (i). For any given > 0 and any d IR
m
we have

F(d) = lim
0
+
F(d)

= lim
0
+
F(d)
/
= lim
/0
+
F((/)d)
/
=

F(d).
(ii). Select t positive and integer and let =
1
t
. By the superadditivity of F
we have the following relationships.
F(d)

= tF(
1
t
d) F(t
1
t
d) = F(d)
. This implies that

F(d) F(d).
(iii). For any > 0 we can select > 0 so that
[

F(d
1
)
F(d
1
)

[ , [

F(d
2
)
F(d
2
)

[
and [
F((d
1
+d
2
))



F(d
1
+d
2
)[ .
This implies that

F(d
1
) +

F(d
2
)
F(d
1
)

+ +
F(d
2
)

F((d
1
+d
2
))

+ 2


F(d
1
+d
2
) + 3.
Hence

F(d
1
) +

F(d
2
)

F(d
1
+d
2
).
(iv) Let d
2
d
1
. Since F is nondecreasing,
F(d
2
)


F(d
1
)

0 for all > 0.


Letting 0
+
implies

F(d
2
)

F(d
1
) 0. So

F is nondecreasing. 2
54
Observe that by denition

F(0) = 0, when F(0) = 0. Hence by Lemma 7.2

F T if F T, where T is dened by (7.5).


We are next going to consider the dual of the mixed integer programming
problem (7.4) in the form of dened by (4.2). So, in the terminology of Chapter
4 we let
X = (x
1
, . . . , x
|I|
, y
1
, . . . , y
|J|
) [
x
j
0 and integer j I and y
j
0 j J.
Correspondingly we set
f(x) =

jI
c
j
x
j
+

jJ
e
j
y
j
and g(x) =

jI
a
j
x
j
+

jJ
h
j
y
j
.
So, the dual of (7.4) is going to be stated as
inf
FF
F(b)
s.t. F(

jI
a
j
x
j
+

jJ
h
j
y
j
)

jI
c
j
x
j
+

jJ
e
j
y
j
(7.6)
x
j
0 and integer j I
and y
j
0 j J.
Consider additionally the following program
inf
FF
F(b)
s.t. F(a
j
) c
j
j I (7.7)

F(h
j
) e
j
j J.
Observe that (7.7) has an attractive form with a nite number of constraints.
Each constraint corresponds to a variable in the primal problem (7.4), as in
linear programming duality. As seen by the next lemma we may use (7.7) as
an equivalent alternative to (7.6).
Lemma 7.3 The programs (7.6) and (7.7) are equivalent.
Proof: Assume that F T is selected so that the constraints of (7.6) are
satised. Now select an index i I. Let x
i
= 1 and x
j
= 0 for all j ,= i, j I.
Moreover let y
j
= 0 for all j J. Satisfaction of the constraints in (7.6) implies
for the selected index i that F(a
i
) c
i
. Since i is selected arbitrarily in I we
have satisfaction of the rst set of constraints in (7.7). Next let x
j
= 0 for all
j I. Moreover select a particular index i J and set y
j
= 0 for all j ,= i,
j J. Satisfaction of the constraints in (7.6) implies that F(h
i
y
i
) e
i
y
i
for
all y
i
0. By letting = y
i
we thus have that
h
i
)

e
i
for all > 0. Hence
ROW DECOMPOSITION 55

F(h
i
) e
i
. Since again the index i is arbitrarily selected we get that the last
set of constraints in (7.7) is satised too.
Conversely, assume that F T satises the constraints of (7.7). Let x
j
0
and integer for j I and y
j
0 for j J. Then
F(a
j
)x
j
+

F(h
j
)y
j
c
j
x
j
+e
j
y
j
.
By Lemma 7.2 we get that F(h
j
y
j
)

F(h
j
y
j
) =

F(h
j
)y
j
. Hence
F(a
j
)x
j
+F(h
j
y
j
) c
j
x
j
+e
j
y
j
.
Finally, by superadditivity and F(0) = 0 we get that
F(

jI
a
j
x
j
+

jJ
h
j
y
j
)

jI
c
j
x
j
+

jJ
e
j
y
j
. 2
Remark 7.1 In the case that some of the constraints in (7.4) are formulated
as equations, then (b) in (7.4) is no longer necessarily nondecreasing in the
elements corresponding to those constraints. Accordingly, the denition (7.5)
of the set of functions T is changed so that F T is only required to be non-
decreasing in the elements corresponding to the inequality constraints. Dually,
if a variable in (7.4) is not required to be nonnegative then the inequality sign
in the corresponding row of (7.7) is replaced by an equality sign.
In particular, with equality constraints and without sign restrictions on vari-
ables Lemma 7.3 is still valid, as seen by performing the slight modications of
the arguments in the proof according to the above remark.
7.3 DECOMPOSITION BY CUTTING PLANES
In this section we are going to study a pure integer programming problem of
the following form
sup
x
cx
s.t. A
1
x b
1
(7.8)
A
2
x b
2
x 0 and integer,
where A
1
Z
mn
, A
2
Z
qn
, c, b
1
and b
2
are vectors of compatible sizes and
with integer elements. Z denotes the set of integer numbers. Hence all data are
given as integers. In this formulation the constraints have been decomposed
into two sets of rows. It is thus a specialization of the program (7.1). Again, in
an economic context the rst set of constraints is going to describe limitations
in the resources of a central unit in an organizational structure. Similarly, the
last set of constraints is going to refer to restrictions for a subunit. As earlier
discussed this subunit may be divided into more subunits. We let here the
integrality condition belong to the central unit. This is natural in an economic
56
context, as the integer variables typically stand for investment activities and
other xed cost long term allocations. Decisions with relation to such activities
are most often taken by the central unit.
After some reformulations we shall apply our decomposition procedure on
(7.8). As a solution method in the procedure we are using cutting plane tech-
nique, which is one of the two fundamental solution techniques in integer pro-
gramming. We are going to consider the other fundamental solution technique,
branch and bound, in Section 7.5.
For simplicity, we shall further assume that the polyhedron
x 0 [ A
2
x b
2
(7.9)
is bounded. If not, extreme rays could be considered additionally as done in the
linear programming case treated in Section 5.3. This means that the polyhedron
dened by (7.9) is the convex hull of its extreme points. Let s denote the total
number of extreme points. Introduce the variables = (
1
, . . . ,
s
) IR
s
+
.
Let x
k
denote the kth extreme point, k = 1, . . . , s. We may now consider the
following problem.
sup

k
,x
s

k=1

k
cx
k
s.t.
s

k=1

k
A
1
x
k
b
1
s

k=1

k
x
k
x = 0 (7.10)
s

k=1

k
= 1

k
0, for k = 1, . . . , s
x integer.
The two programs (7.8) and (7.10) are equivalent by the above considerations.
Moreover, the introduction of (7.10) represents a direct extension of the sim-
ilar standard Dantzig-Wolfe transcription performed in linear programming.
Compare for example with the formulation (5.10) for the linear programming
case.
The program (7.10) is a mixed integer programming problem. Let
j
denote
the jth n- dimensional unit vector. Using the integer programming duality
theory including remark 7.1 we may now consider the dual of (7.10). In the
format of (7.7) this is going to obtain the form:
inf
FF
F(b
1
, 0, 1)
s.t. F(0,
j
, 0) = 0 for j = 1, . . . , n

F(A
1
x
k
, x
k
, 1) cx
k
for k = 1, . . . , s.
ROW DECOMPOSITION 57
Let Y = F T [ F(0,
j
, 0) = 0 for all j = 1, . . . , n. Introducing =
(
1
, . . . ,
s
) IR
s
+
this dual can alternatively be stated as the program
inf
FY
sup

k
0
s

k=1

k
cx
k

k=1

k

F(A
1
x
k
, x
k
, 1) (7.11)
+F(b
1
, 0, 1).
Next let
K(, F) =
s

k=1

k
cx
k

k=1

k

F(A
1
x
k
, x
k
, 1)
+F(b
1
, 0, 1)
In this way (7.11) can be written in the following abbreviated form:
inf
FY
sup
0
K(, F).
So we have obtained a reformulation of the original integer programming prob-
lem (7.8) into the fundamental form (5.2). By an appropriate denition of the
updating process we can perform our decomposition procedure on the above
formulation. Again we shall here apply the standard updating procedure of
Dantzig and Wolfe. As usual, at a given stage we are considering some pre-
viously selected extreme points of the polyhedron dened by (7.9). Let Q
denote the index set for the extreme points under current consideration. In
the terminology of Chapter 5 we set X = IR
s
+
and at the current stage we
set

X = IR
s
+
[
k
= 0, k / Q. Hence,

X is a nonnegative orthant of
dimension equal to the number of currently known extreme points.
In this setting the lower problem (5.6) undertakes the form:
inf
FY
sup

k
0

kQ

k
cx
k

kQ

k

F(A
1
x
k
, x
k
, 1)
+F(b
1
, 0, 1). (7.12)
Above we showed equivalence between the program (7.10) and the program
(7.11) in which all extreme points have been included of the polyhedron dened
by (7.9).
With the current extreme points indexed by the set Q consider the following
program:
sup

k
,x

kQ

k
cx
k
s.t.

kQ

k
A
1
x
k
b
1
s

kQ

k
x
k
x = 0 (7.13)
58
s

kQ

k
= 1

k
0, x integer.
Similarly we can show equivalence between (7.13) and (7.12).
As done in the linear programming case in Section 5.3 it is natural here to
assume that the extreme points have been selected so that (7.12) is bounded
below or equivalently that (7.13) is feasible. If this is not the case, then some
kind of a phase 1 procedure may be applied to ensure feasibility of (7.13).
Hence, by duality, an optimal solution F

exists for (7.12). According to the


standard updating procedure of Dantzig and Wolfe we set the next

Y = F

.
So the upper problem is going to have the form:
sup

k
0
s

k=1

k
cx
k

k=1

k

F

(A
1
x
k
, x
k
, 1)
+F

(b
1
, 0, 1). (7.14)
We shall make some simplications of the upper problem (7.14).
The value of the upper problem (7.14) is either F

(b
1
, 0, 1) or the problem is
unbounded. In the last case and in the terminology of the dual program (5.2)
then F

is infeasible in (7.11). The purpose of the upper problem is to eliminate


such infeasible solutions via extensions of Q, and hence of

X. However, for this
purpose it is enough to consider the program
sup

k
0
s

k=1

k
cx
k

k=1

k

F

(ax
k
, x
k
, 1) (7.15)
s.t.
s

k=1

k
= 1
since the value of (7.15) is positive if and only if (7.14) is unbounded.
For an arbitrary superadditive and positively homogeneous function h :
IR
m
IR we have for d
1
IR
m
, d
2
IR
m
and 0 1 that
h(d
1
+ (1 )d
2
) h(d
1
) +h((1 )d
2
)
by superadditivity
= h(d
1
) + (1 )h(d
2
)
by homogeneity.
Hence h is a concave function. Since

F

is superadditive and positively homo-


geneous by Lemma 7.2, it is also concave.
Now consider
sup
x0
cx

F

(A
1
x, x, 1)
s.t. A
2
x b
2
. (7.16)
x 0.
ROW DECOMPOSITION 59
By the concavity of F

the objective function of this program is convex. Hence


an optimal solution will exist among the extreme points x
k
for k = 1, . . . , s. So
the program (7.15) may be replaced by (7.16).
Finally some words about nite convergence. We shall in the next section
show that algorithms for mixed integer programming may be selected, so that

is polyhedral. In this case (7.16) can be transformed into a linear program.


This implies that there exists a nite procedure for the solution of the upper
problem.
Also the solution of the lower problem can be performed by a procedure
having nite convergence. More specically, we are going to create the functions
F and

F by means of the classical cutting plane technique for mixed integer
programming constructed by Gomory, 1960. This method is known to have
nite convergence provided that the objective function can be restricted to an
integer number. In the present context, we are here going to apply the cutting
plane procedure on the lower program stated in the form of (7.13). For this
program the optimal objective function value is automatically integer due to
the integrality of the initial coecients c and the integrality condition on the
variable x. In Section 7.4 we are going to study in detail how the generation of
F and

F is performed. However, based on the niteness of the Gomory cutting
plane procedure the generation of those functions are performed by a procedure
with nite convergence.
The upper problem produces extreme points of a polyhedron. Since there is
only a nite number of extreme points the overall procedure has nite conver-
gence.
7.4 MIXED INTEGER CUTS.
In this section we are going to establish the close connection between the clas-
sical generation of a valid cut in mixed integer programming constructed by
Gomory, 1960 and the generation of a corresponding superadditive function.
When the algorithm terminates the generated function closes the duality gap
between the primal program for example written in the form of (7.4) and its
dual for example in the form of (7.7). As mentioned in the previous section we
are going to apply this approach on the lower problem (7.12) and its dual form
(7.13). By the way, the idea of pointing out the interrelationship between an al-
gorithmic approach and the generation of a dual function is of a general nature,
and does not particularly apply to the case of mixed integer programming.
The present treatment follows to some extent the treatment of Nemhauser
and Wolsey, 1988. We shall again state the general mixed integer programming
problem (7.4):
(b) = sup

jI
c
j
x
j
+

jJ
e
j
y
j
s.t.

jI
a
j
x
j
+

jJ
h
j
y
j
b (7.17)
x
j
0 and integer for j I
60
y
j
0 for j J.
where c
j
, and e
j
IR, a
j
, h
j
and b IR
m
are given constants. x
j
and y
j
IR
are variables dened over the nite index sets I and J, respectively. We shall
introduce valid cuts for (7.17), dened by linear inequalities, i. e. all feasible
solutions of (7.17) must satisfy those inequalities. Valid cuts may be derived by
superadditive functions in the following way. Consider again the set of functions
T dened by (7.5). Assume also as before that the directional derivative

F(d)
exists for all the considered functions F T d IR
m
. Let G T have those
properties. We may then consider the following inequality.

jI
G(a
j
)x
j
+

jJ

G(h
j
)y
j
G(b). (7.18)
This inequality is a valid, i. e. it does not cut o any feasible solution of (7.17).
This is seen as follows.
Let the elements x
j
and y
j
, j constitute a feasible solution of (7.17). Then
G(b) G(

jI
(a
j
x
j
) +

jJ
(h
j
y
j
))
(as G is nondecreasing)

jI
G(a
j
)x
j
+

jJ
G(h
j
y
j
)
(as G is superadditive, x 0 and integer)

jI
G(a
j
)x
j
+

jJ

G(h
j
y
j
)
(as

G is dominated by G,
according to Lemma 7.2)
=

jI
G(a
j
)x
j
+

jJ

G(h
j
)y
j
(as

G is positively homogeneous
by Lemma 7.2).
As is wellknown, the algorithm of Gomory utilizes the cutting plane tech-
nique. It solves a series of linear programming problems in which cuts consecu-
tively are appended to the set of constraints. This is done in order to generate
the optimal integral solution as an extreme point of a polyhedron generated by
the updated set of constraints. More specically, if an integer solution has not
yet been obtained at an iteration then a cut of the type (7.18) is going to be
appended with a specic choice of the function G. We are going to dene this
choice.
In the case that a nonintegral optimal solution has been obtained based
on a current set of constraints then the nal simplex tableau contains a row
corresponding to a basic integer variable which however assumes a fractional
ROW DECOMPOSITION 61
value. A cut is generated from such a row, which often for this reason is called
a source row.
For simplicity we shall initially consider the generation of the rst cut to be
appended. In this case the dimension of the basis is equal to the number of
original constraints. So the dimension is equal to m. Now, consider a source
row, and let v = (v
1
, . . . , v
m
) denote the row elements of the inverse basis
matrix corresponding to the source row. Let V = 1, . . . , m and partition this
set into
V
+
= i V [ v
i
0 and V

= i V [ v
i
< 0.
In order to dene the function G let a| denote the integral part of an element
a IR and let f
a
denote the fractional part, i. e. a = a| +f
a
and 0 f
a
< 1.
Take IR, where 0 < 1, and dene the function F

: IR IR by
F

(a) = a| + max(0,
f
a

1
). (7.19)
Specically, let denote the fractional part of the nonintegral value of the basic
variable in the source row. We are now able to dene the requested function G
as
G(d) = F

(vd)
1
1

iV

v
i
d
i
. (7.20)
By Proposition 7.1 on page 65 in Appendix 7.4.2 and by the subsequent remarks
in the same Appendix it is shown that G T as dened by (7.5). Moreover,

G(d) =
1
1
min(

iV

v
i
d
i
,

iV
+
v
i
d
i
). (7.21)
(If V

or V
+
is empty then one of the above summations is empty and its
value is set to zero.)
Note that

G is concave and piecewise linear.
Further by Proposition 7.2 on page 68 in the Appendix 7.4.2 is proven that
the G function dened by (7.20) generates the cuts in the Gomory algorithm.
With more than one cut under consideration the number of rows increases
implying an increase in the dimension of the vectors v and d in (7.20). The
added elements of the vectors correspond to the cuts. Also the columns a
j
and b are extended when used in (7.18) for generation of subsequent cuts.
Thus the columns a
j
and b will in general contain elements that are calculated
by earlier created functions G and

G. In short, an actual function G and its
partial derivative

G will be a composite function on the original m-space dened
recursively by (7.20) and (7.21), respectively. In the Appendix 7.4.2 it is shown
that this recursively generated function G is still in T as dened by (7.5) and
that

G is still concave and piecewise linear.
If an optimal solution exists for (7.4) and if the objective function value can
be restricted to an integer value and thus treated as an integer variable then it
is known that an optimal solution will be obtained by the Gomory algorithm
after a nite number of cut additions. See for example Salkin, 1975. It should
62
be remarked that a cutting plane method has been developed which has nite
convergence without the integrality requirement of the objective function value.
See W. Cook and Schrijver, 1990.
The solution is detected as an optimal solution of the last linear programming
problem resulting in integer values for all x
i
, i I. The nal set of constraints
consists of the original constraints plus some cuts which we can state by the
inequalities

jI
G
r
(a
j
)x
j
+

jJ

G
r
(h
j
)y
j
G(b),
where r = 1, . . . , p and p = number of cuts.
At termination dual variables u
1
, . . . , u
m
, u
m+1
, . . . , u
p
0 are obtained for
all constraints dening the dual function F(d) : IR
m
IR by
F(d) =
m

i=1
u
i
d
i
+
p

i=m+1
u
i
G
i
(d)
and its directional derivative

F(d) =
m

i=1
u
i
d
i
+
p

i=m+1
u
i

G
i
(d).
Observe that

F(d) is also concave and piecewise linear.
By its construction this function F is an element of the considered function
class T. Moreover, it is an optimal solution of the dual program (7.7), and the
duality gap is closed between (7.4) and (7.7)
In complete accordance with the discussion in the end of Section 7.2 we
observe that if some of the constraints in (7.17) are described by equations
then the above development applies as well. However, F(d) is no longer non-
decreasing with respect to arguments corresponding to equations, as their dual
variables have no sign restrictions.
7.4.1 A Numerical Example.
The material here is from Holm and Tind, 1985. Consider the integer program-
ming problem
max 4x
1
+x
2
s.t. x
1
+ 2x
2
4 (7.22)
2x
1
+x
2
3
x
1
, x
2
0 and integer.
The rst constraint is assumed to belong to the central unit and the sec-
ond constraint the subunit. Hence the polyhedron dened by (7.9) is here
(x
1
, x
2
) 0 [ 2x
1
+ x
2
3. Assume that (0, 0) is the rst and only known
extreme point of this polyhedron. Then the 1. lower problem will create the
ROW DECOMPOSITION 63
function

F(d) = 0 for all d
1. upper problem, stated in the form of (7.16).
max 4x
1
+x
2
s.t. 2x
1
+x
2
3
x
1
, x
2
0.
An optimal solution is (x
1
, x
2
) = (
3
2
, 0) with the objective function value =
6 > 0.
2. lower problem.
We consider the lower problem in its primal form dened by (7.13).
max 0
1
+ 6
2
s.t. 0
1
+
3
2

2
4
0
1
+
3
2

2
x
1
= 0
0
1
+ 0
2
x
2
= 0

1
+
2
= 1

k
0 for k = 1, 2
x
j
0 and integer for j = 1, 2.
Disregarding the integrality condition temporarily an optimal solution obtained
by linear programming is (x
1
, x
2
) = (
3
2
, 0). x
1
is fractional. By calculation of
the updated simplex tableau we get from the corresponding row of this tableau
the following values
=
1
2
and v = (0, 1, 0,
3
2
).
Hence by (7.21)

G =
1
1
1
2
mind
2
,
3
2
d
4
= 2 mind
2
,
3
2
d
4
. After addition of the
corresponding cut an integer optimal solution (x
1
, x
2
) = (1, 0) is obtained with
dual variables (u
1
, . . . , u
5
) = (0, 4, 0, 0, 4). Hence

F(d
1
, . . . , d
4
) =
4

i=1
u
i
d
i
+u
5
(2 mind
2
,
3
2
d
4
)
= 4d
2
+ 8 mind
2
,
3
2
d
4
.
2. upper problem.
The objective function gets the form
cx

F(A
1
x, x
1
, x
2
, 1)
= 4x
1
+x
2
+ 4x
1
8 minx
1
,
3
2

64
= maxx
2
, 8x
1
+x
2
12
and the optimization problem becomes
max x
2
, 8x
1
+x
2
12
s.t. 2x
1
+x
2
3
x
1
, x
2
0.
An optimal solution is (x
1
, x
2
) = (0, 3) and the value of the objective function
is 3. Since this value is still positive we continue with the
3. lower problem.
max 6
2
+ 3
3
s.t.
3
2

2
+ 6
3
4

3
2

2
+x
1
= 0
3
3
+x
2
= 0

1
+
2
+
3
= 1

k
0, k = 1, 2, 3
x
j
0 and integer, j = 1, 2.
An optimal LP solution is (x
1
, x
2
) = (
3
2
, 0).
From the optimal LP tableau a cut is derived based on the data
=
1
2
and v = (0, 1, 0,
3
2
).
creating the function G together with

G(d
1
, . . . , d
4
) =
1
1
1
2
min0, d
2
+
3
2
d
4
.
With this cut an integer optimal solution (x
1
, x
2
) = (1, 1) is obtained together
with the dual variables (u
1
, . . . , u
5
) = (0, 2, 0, 3, 2). Hence

F(d
1
, . . . , d
4
) = 2d
2
+ 3d
4
+ 2 min0, d
2
+
3
2
d
4

= min2d
2
+ 3d
4
, 6d
4
.
3. upper problem.
The objective function is
4x
1
+x
2
min2x
1
+ 3, 6
= max2x
1
+x
2
3, 4x
1
+x
2
6
ROW DECOMPOSITION 65
resulting in the program
max 2x
1
+x
2
3, 4x
1
+x
2
6
s.t. 2x
1
+x
2
3
x
j
0, j = 1, 2.
The optimal value is 0. Hence the procedure terminates with the solution
(x
1
, x
2
) = (1, 1) obtained in the last lower problem. This solution is optimal in
the original program (7.22).
7.4.2 Appendix
Consider again F

(a) dened by (7.19). We then state the following proposition


by Nemhauser and Wolsey, 1990
Proposition 7.1 Given two superadditive functions F
1
and F
2
: IR
m
IR
where F
1
(0) = F
2
(0) = 0. Then the composite function G : IR
m
IR given by
G =
1
1
F
1
+F

(F
2
F
1
) is
(a) superadditive,
(b) nondecreasing if F
1
and F
2
are nondecreasing and
(c) if

F
1
and

F
2
exist and are nite then

G =
1
1
min(

F
1
,

F
2
).
To prove Proposition 7.1 we need the following three lemmas.
Lemma 7.4 F

(a) dened by (7.19) is superadditive and nondecreasing.


Proof: F

is piecewise linear with slope either 0 or


1
1
. Additionally it has
no jumps. Hence the function is nondecreasing. With a IR and b IR let
f
a
= a a| and f
b
= b b|. Let x
+
= max(0, x). To prove superadditivity
we shall consider three cases:
(i) f
a
+f
b
< 1.
F

(a) +F

(b) = a| +
(f
a
)
+
1
+b| +
(f
b
)
+
1
a +b| +
(f
a
+f
b
)
+
1
= F

(a +b).
(ii) f
a
+f
b
1, f
b
.
F

(a) +F

(b) = a| +
(f
a
)
+
1
+b|
< a| +b| + 1 = a +b| F

(a +b).
(The same argument can be used if f
a
.)
66
(iii) f
a
+f
b
1 and f
a
, f
b
> .
F

(a) +F

(b) = a| +
(f
a
)
+
1
+b| +
(f
b
)
+
1
a| +b| + 1 +
f
a
+f
b
1
1
F

(a +b). 2
We shall now state the next lemma.
Lemma 7.5 Let L : IR
k
IR be superadditive and nondecreasing and let
F
i
: IR
m
IR be superadditive for i = 1, . . . , k. Then the composite function
L(F
1
, . . . , F
k
) is superadditive. Moreover if the functions F
i
are also nonde-
creasing, then L(F
1
, . . . , F
k
) is nondecreasing.
Proof: Let a
1
, a
2
IR
m
. Directly by the assumptions in the lemma we obtain
the following inequalities.
L(F
1
(a
1
), . . . , F
k
(a
1
)) +L(F
1
(a
2
), . . . , F
k
(a
2
))
(by L superadditive)
L(F
1
(a
1
) +F
1
(a
2
), . . . F
k
(a
1
) +F
k
(a
2
))
(by L nondecreasing and F superadditive).
L(F
1
(a
1
+a
2
), . . . F
k
(a
1
+a
2
)) 2
Let d
1
IR and d
2
IR. Consider the twodimensional function H

(d
1
, d
2
)
dened by
H

(d
1
, d
2
) =
d
1
1
+F

(d
2
d
1
). (7.23)
We can now state the third lemma:
Lemma 7.6 H

given by (7.23) is nondecreasing and superaddive.


Proof: Since F

is nondecreasing, H

is nondecreasing in d
2
. Looking at d
1
we observe that the rst term of (7.23) has slope
1
1
and the second term has
slope
1
1
or 0. So, H

is also nondecreasing in d
1
.
The rst term of (7.23) is superadditive. F

is nondecreasing and superad-


ditive by Lemma 7.4. The term d
2
d
1
is a superadditive (in particular here
linear) function in (d
1
, d
2
). Hence by Lemma 7.5 we have that the second term
of (7.23 is superadditive too. So H

is superadditive. 2
We can now prove the stated Proposition.
Proof of Proposition 7.1: By Lemma 7.6 the function H

dened by (7.23 is
ROW DECOMPOSITION 67
nondecreasing and superadditive. Hence (a) and (b) follows by aplication of
Lemma (7.5).
We shall now prove (c). By denition of the directional derivatives we have
for any function F where F(0) = 0 and > 0 that
F
1
(d) =

F(d) +(),
where () : IR IR and lim
0
+
() = 0. In particular
F
1
(d) =

F
1
(d) +
1
()
and
F
2
(d) F
1
(d) =

F
2
(d)

F
1
(d) +
2
(),
where
i
() : IR IR and lim
0
+

i
() = 0 for i = 1, 2. So
G(d) = (

F
1
(d) +
1
())/(1 )
+F

((

F
2
(d)

F
1
(d)) +
2
()).
For suciently small we have by the denition of F

in (7.19) for x IR that

(x) =

(x) = min0,
x
1
.
Hence for suciently small
G(d) = (

F
1
(d) +
1
())/(1 )
+

1
min0,

F
2
(d)

F
1
(d) +
2
()
=
1
1
min

F
1
(d) +
1
(),

F
2
(d) +
1
() +
2
().
Division by followed by letting 0
+
implies the desired result.

G(d) =
1
1
min

F
1
(d),

F
2
(d). 2
Let again v = (v
1
, . . . , v
m
) R
m
, V = 1, . . . , m, V
+
= i [ v
i
0 and
V

= i [ v
i
< 0. Moreover let
F
1
(d) =

iV

v
i
d
i
and F
2
(d) =

iV
+
v
i
d
i
These functions satisfy the conditions in Proposition 7.1 and in this case
G(d) = F

(vd)
1
1

iV

v
i
d
i
(7.24)
and

G(d) =
1
1
min(

iV

v
i
d
i
,

iV
+
v
i
d
i
).
68
So by Proposition 7.1 we get that G T as dened by (7.5) and we have
derived the function

G already stated in (7.21).
Proposition 7.2 G and

G generate the cuts in Gomorys algorithm for mixed
integer programming.
Proof: Consider the mixed integer programming problem (7.17) in element
notation
sup

jI
c
j
x
j
+

jJ
e
j
y
j
s.t.

jI
a
ij
x
j
+

jJ
h
ij
y
j
+s
i
= b
i
i V (7.25)
x
j
0 and integer j I
y
j
, s
i
0 j J, i V
with s
i
being continuous slackvariables. The linear programming relaxation of
(7.25) is rst solved. If x
j
is integer for all j I an optimal solution has been
found. Otherwise there exists a row in the simplex tableau of the form

b = x
r
+

jI\r
a
j
x
j
+

jJ

h
j
y
j
+

iV
v
i
s
i
(7.26)
where the terms

b, a
j
,

h
j
and v
j
are the updated coecients and x
r
is a basic
integer variable with current fractional value

b.
Being the coecients of the slack variable the elements v
i
are the elements
of the current inverse basis matrix in the selected row. Hence a
j
=

iV
v
i
a
ij
and

b =

iV
v
i
b
i
. Let =

b

b| and
j
= a
j
a
j
|, j I. Partition I as
I
+
= j I [
j
and I

= j I [
j
> and J as J
+
= j J [

h
j

0 and J

= j J [

h
j
< 0.
With the previous splitting of V the cut dened by Gomory can now be
stated in the form

jI
+

j
x
j

jI

(1
j
)
1
x
j

jJ
+

h
j
y
j
+

jJ

h
j
1
y
j
(7.27)

iV
+
v
i
s
i
+

iV

v
i
1
s
i
.
Usually in most presentations, see for example Nemhauser and Wolsey, 1988,
page 250, the index sets in (7.27) are restricted to include only the current
nonbasic variables. However, note that all coecints for basic variables in
(7.27) are zero except for the basic integer variable x
r
corresponding to the
considered row. This variable has coecient 1. So
r
= 0 and r I
+
. With
ROW DECOMPOSITION 69
this observation the coecients for the current basic variables all get the value
zero in (7.27).
Reformulation of coecients transforms (7.27) into

jI
(
j
+ max0,

j

1
)x
j
+

jJ
(
1
1
min0,

h
j

h
j
)y
j

iV
+
v
i
s
i
+

iV

v
i
1
s
i

or

jI
(

iV
v
i
a
ij
+

iV
v
i
a
ij
| + max0,

j

1
)x
j
+

jJ
(
1
1
min

iV

v
i
h
ij
,

iV
+
v
i
h
ij

iV
v
i
h
ij
+

iV

v
i
h
ij
1
)y
j

iV
+
v
i
s
i
+

iV

v
i
1
s
i

iV
v
i
b
i
+

iV
v
i
b
i
|.
By insertion of s
i
= b
i

jI
a
ij
x
j

jJ
h
ij
y
j
the cut is further transformed
into

jI
(

iV
v
i
a
ij
| + max0,

j

1

iV

v
i
a
ij
1
)x
j
+

jJ
1
1
min

iV

v
i
h
ij
,

iV
+
v
i
h
ij
y
j

iV
v
i
b
i
|
1
1

iI

v
i
b
i
or

jI
G(a
j
)x
j
+

jJ

G(h
j
)y
j
G(b).
which is the desired form (7.18). 2
The above discussion takes only the generation of one cut into consideration.
In general more cuts are required. Assume for simplicity of exposition that
no cut is discarded. Hence the dimension of the basis is increased by one in
each step. With r cuts we can thus enumerate the generated elements by an
70
iteration couter k = 1, . . . , r. In this way we get the functions G
k
(d) and

G
k
(d),
the fractional number
k
, the vector v
k
= (v
k
1
, . . . , v
k
m+k1
) IR
m+k1
and the
disjoint index sets V

k
, V
+
k
where V

k
V
+
k
= 1, . . . , m+k 1.
G
r
(d) and

G
r
(d) are recursively generated by (7.24) as
G
r
(d) = F

r
(
m+r1

i=1
v
r
i
d
i
)
1
1
r

iV

r
v
i
d
i
and

G
r
(d) =
1
1
r
min

iV

r
v
r
i
d
i
,

iV
+
r
v
r
i
d
i
,
where d
m+i
= G
i
(d) for i = 1, . . . , r 1.
In this general case the functions F
1
and F
2
dening G
r
originally as in
Proposition 7.1 get the form
F
1
(d) =

iV

r
v
i
d
i
and F
2
(d) =

iV
+
r
v
i
d
i
where d
m+i
= G
i
(d) for i = 1, . . . , r 1.
Hence by a recursive argument F
1
(d) and F
2
(d) are also in this general
context superadditive and nondecreasing and by Proposition 7.1 we get in the
same way as before that G
r
(d) F.

G
r
(d) is the minimum of two concave
functions. Hence

G
r
(d) is concave, and since all functions involved in the
recursion are piecewise linear,

G
r
(d) is also piecewise linear.
7.5 DECOMPOSITION BY BRANCH AND BOUND
The underlying program is again the integer programming problem (7.8), which
was treated by cutting plane technique in section 7.3. Here we shall solve the
problem by means of the other major solution technique in integer program-
ming: branch and bound. Parts of the material is based on Holm and Tind,
1988.
Let us consider an arbitrary branch and bound algorithm, using bounds cal-
culated by linear programming. Every stage in the solution process for this
kind of algorithm is characterized by the shape of the attached enumeration
tree, call it T. Additionally, each terminal, or end point, of T is characterized
by a subpolyhedron of the LP-relaxation of the original constraints in (7.8).
Let I
j
denote the subpolyhedron for the jth terminal. The subpolyhedra of
the terminals are mutually disjoint. Their union constitutes a subset of the
LP-relaxation of the original constraints in (7.8) by exclusion of areas contain-
ing only fractional solutions. (At this point of the discussion we do not pay
attention to the decomposed structure of the constraints. The decomposed
structure shall be utilized later.)
ROW DECOMPOSITION 71
Let E
T
denote the terminals of a current tree T and let I
j
denote the sub-
polyhedron belonging to the jth terminal, where j E
T
.
For each tree T
sup
jE
T
sup
xI
j
cx
is an upper bound for the value of the program (7.8). The objective of the
branch and bound algorithm is to nd the lowest upper bound, i. e. to select
a tree which solves the problem
inf
T
sup
jE
T
sup
xI
j
cx. (7.28)
The value of the program (7.28) is equal to the optimal objective function value
of (7.8). An optimal solution x

of (7.8) can be obtained from a terminal k


giving the supremal value, i. e. x

I
k
and cx

= sup
jE
T
sup
xI
j
cx.
A branch and bound algorithm is good, when the optimal value of (7.28)
can be determined by examination of only a limited number of small trees.
Following a traditional approach we attach to each terminal j E
T
integer
lower and upper bounds for the variables. So, let l
j
IR
n
and u
j
IR
n
denote
such lower and upper bounds, repectively, and with integer elements. For each
terminal j E
T
we thus consider the following subpolyhedron.
I
j
= x 0 [ A
1
x b
1
, A
2
x b
2
and l
j
x u
j
. (7.29)
We shall develop a decomposition procedure through specication of the un-
specied sets and terms in the general procedure from Chapter 5.
First we let X = x IR
n
+
[ A
2
x b
2
. We shall next dualize with respect
to the remaining set of constraints. So for each j E
T
introduce dual variables
y
j
0 corresponding to the constraints A
1
x b
1
and dual variables
j
and

j
corresponding to the constraints x u
j
and x l
j
, respectively. For
simplicity we still assume that the polyhedron X is bounded. As in the cutting
plane case we know at each stage of the procedure some extreme points x
k
X.
Let Q denote the index set of the currently known extreme points. We then
specify the set

X by having

X = x
k
[ k Q. With this specication we
consider restrictions

I
j
of the subpolyhedra I
j
studied in (7.29) by letting

I
j
= x

X [ A
1
x b
1
, l
j
x u
j
.
Those sets are going to be the constraints for the optimization problems consid-
ered at each terminal of a tree to be developped in connection with the solution
of the lower problem. Notice, that a set

I
j
may be empty, even in cases where
I
j
is nonempty.
The set Y to be used in the procedure is here going to be dened as the set
of all trees together with all nonnegative values of y
j
,
j
and
j
. Compared
with earlier specications of Y this is a rather peculiar one. However, it enables
us in a multilevel context to let the central unit determine the way to select
alternatives via the separation into terminals of the tree. Simultaneously the
72
central unit quotes the prices y
j
for use of joint resources and give penalties
(
j
,
j
) for potential violation of the integrality condition. We shall return to
the economic interpretation jointly with the cutting plane approach in Section
7.6.
Let I denote the identity matrix. In order to shorten notation we dene the
matrix:
B =
_
_
A
1
I
I
_
_
and for each terminal j E
T
the vector:

j
=
_
_
b
1
u
j
l
j
_
_
.
Additionally let
j
= (y
j
,
j
,
j
). Collect further those variables into the vec-
tor of dual variables = (
1
, . . . ,
|E
T
|
), where [E
T
[ denotes the number of
terminals of the tree T. We may then specify an element of the collection Y
as y = (T, ) indicating the selection of a tree together with values of the dual
variables at all terminals.
We can now describe the function K(x, y) : X Y IR by the following
denition:
K(x, (T, )) = sup
jE
T
cx
j
Bx +
j

j
. (7.30)
The basic problem to be considered is then
w = inf
(T,)Y
sup
xX
K(x, (T, )). (7.31)
This is specied in the dual form (5.2). We shall see later (in the comments
following problem (7.37)) that this problem is identical with (7.28), and hence
that it solves the original problem (7.8). However, the lower problem must
undergo a similar treatment, which we are going to perform now.
For this purpose we shall state a lemma. So, let J be an arbitrary index set
and dene for each j J a function f
j
(z
j
) : S
j
IR, where S
j
is an arbitrary
set. Let further z = (z
1
, . . . , z
|J|
) and S = (S
1
. . . S
|J|
). We then have
Lemma 7.7
sup
jJ
inf
z
j
S
j
f
j
(z
j
) = inf
zS
sup
jJ
f
j
(z
j
).
Proof: Select z

j
S
j
for all j J. Then
sup
jJ
f
j
(z

j
) inf
zS
sup
jJ
f
j
(z
j
).
Since z

j
is arbitrary, we get
sup
jJ
inf
z
j
S
j
f
j
(z
j
) inf
zS
sup
jJ
f
j
(z
j
).
ROW DECOMPOSITION 73
Conversely,
sup
jJ
f
j
(z

j
) f
k
(z

k
) for z

S and k J.
Hence
sup
jJ
f
j
(z

j
) inf
z
k
S
k
f
k
(z
k
) for z

S and k J.
So
inf
z

S
sup
jJ
f
j
(z

j
) inf
z
k
S
k
f
k
(z
k
) and k J.
This implies that
inf
z

S
sup
jJ
f
j
(z

j
) sup
kJ
inf
z
k
S
k
f
k
(z
k
)
or by redenition of terms
inf
zS
sup
jJ
f
j
(z
j
) sup
jJ
inf
z
j
S
j
f
j
(z
j
). 2
We now return to consider the lower problem, which here gets the form:
w = inf
(T,)Y
sup
x

X
K(x, (T, )) (7.32)
or by insertion
w = inf
T
inf
0
sup
x

X
sup
jE
T
cx
j
Bx +
j

j
,
which again is equivalent to
w = inf
T
inf
0
sup
jE
T
sup
kQ
cx
k

j
Bx
k
+
j

j
,
By application of Lemma (7.7) we may for any tree T interchange the order of
the two operations inf
0
and sup
jE
T
. Hence we get further equivalence to
w = inf
T
sup
jE
T
inf

j
0
sup
kQ
cx
j
Bx +
j

j
. (7.33)
Looking at the inner part of (7.33) we are for a given tree T and a given terminal
j of the tree going to consider the problem:
inf

j
0
sup
kQ
cx
k

j
Bx
k
+
j

j
. (7.34)
By introduction of a variable IR (7.34) may be formulated as the follow-
ing linear programming problem:
inf

j
0,

s.t.
j
Bx
k

j
+ cx
k
for all k Q (7.35)
74
or in dual form, introducing dual variables
k
for k Q:
sup

k
0

kQ
cx
k

k
s.t.

kQ
Bx
k

k

j
(7.36)

kQ

k
= 1.
In the derivation of the rst block of constraints we have utilized that

kQ

j
=

j
.
By insertion of this formulation into the lower problem in the form (7.33)
we obtain:
w = inf
T
sup
jE
T
sup

k
0

kQ
cx
k

k
s.t.

kQ
Bx
k

k

j
(7.37)

kQ

k
= 1.
At any stage x =

kQ

k
x
k
. At termination an optimal integer solution
can be obtained from the terminal which obtains the value w. If all extreme
points x
k
X were included in the above formulation (7.37) then this formu-
lation is equivalent to our original problem (7.28). Furthermore, if all extreme
points have been included then the lower problem in the form (7.32) is equal to
(7.31). Since (7.37) and (7.31) by the above derivation are equivalent, by hav-
ing

X = X, we obtain that (7.31) solves (7.28) and hence (7.8), as announced
in the comments stated after (7.31).
Returning to problem (7.34) then for the selected terminal j two things
may happen. Either we get an optimal solution

j
of (7.34), in which case
we introduce V

j
=

j
, or the problem is unbounded below. In the last case
a vector
j
0 is pointed out so that
j
(Bx
k
+
j
) < 0 for all k Q, and
correspondingly we set V

j
=
j
[
j
=
j
and IR
+
. So in this case V

j
represents a haline.
Let T

denote the chosen tree. We can now specify the set



Y which in additon
to T

will consist of all the solutions V

j
obtained in either way. They may be
collected into the following collection of elements V

= (V

1
. . . V

|E
T
|
), in
which each element might be either a point or a ray. The set

Y is transferred
to the upper problem.
We may then, in accordance with the general procedure, state the upper
problem as follows.
z = sup
xX
inf
V

K(x, (T

, ))
= sup
xX
inf
V

sup
jE
T

(c
j
B)x +
j

j
.
ROW DECOMPOSITION 75
By lemma 7.7 we have equivalence to
z = sup
xX
sup
jE
T

inf

j
V

j
(c
j
B)x +
j

j
.
Thus for xed x and j we are going to consider the problem
H
j
(x) = inf

j
V

j
(c
j
B)x +
j

j
.
If V

j
is a ray we then have
H
j
(x) = inf
0
(c
j
B)x +
j

j
.
If V

j
is a point there is no optimization to perform. To summarize the dierent
instances we thus have
H
j
(x) =
_
_
_
(c

j
B)x +

j
, if V

j
is a point
cx, if V

j
is a ray and
j
(Bx +
j
) 0
, if V

j
is a ray and
j
(Bx +
j
) < 0
(7.38)
With this notation the upper problem can be stated as
z = sup
x 0
s.t. A
2
x b
2
sup
jE
T

H
j
(x). (7.39)
Note that sup
jE
T
H
j
(x) is a piecewise linear and convex function. So, we
always have among the optimal solutions an extreme point x
k
of X, as expected.
If the procedure is not terminated this extreme point is put into the set

X and
the index set Q is updated accordingly.
We shall demonstrate nite convergence of the procedure by direct appli-
cation of Theorem 5.2. This requires that assumption (5.1) is valid. This is
however correct as seen directly by the following argumentation. Our formu-
lation is based on the dual form (5.2). So, to satisfy our assumption we shall
show that z provides an upper bound for w. For any tree T

and any subset


V

of dual variables we have


w inf
0
sup
xX
K(x, (T

, ))
= sup
xX
inf
v0
K(x, (T

, ))
sup
xX
inf
vV

K(x, (T

, ))
= z.
The rst inequality comes directly from the denition of w as given in (7.31).
The second equation is due to linear programming duality applied on the LP-
problem in a terminal j obtaining the maximal value in (7.30). However, this
76
equation is not valid if the considered terminal has the primal problem as well as
the dual problem infeasible. This situation cannot occur here. See, for example
the primal problem in the form (7.35), which is always feasible. In this context
it should be noted that the form (7.35) is dierent if X is not bounded. If so,
extremes rays of X must be taken into consideration as well. There will be
more discussion of this point in section 5.5.2.
We are now in the position to apply Theorem 5.2. In this theorem the
condition (ii) is always fullled by the solution of the lower problem together
with the specic construction of the set

Y . If z > w then Theorem 5.2 implies
that condition (i) the theorem is violated, i.e. that
sup
x

X
inf
y

Y
K(x, y) < sup
xX
inf
y

Y
K(x, y) = z.
Hence in this case the upper problem brings in a new extreme point x
k
. Since
the number of extreme points is nite only nitely many iterations can occur in
the decomposition procedure. Each lower problem is solved by a nite branch
and bound procedure. The upper problem requires the solution of a nite
number of linear programming problems, that may be turned into one linear
programming problem. All problems have thus nite solution procedures. This
proves nite convergence of the entire procedure.
7.5.1 An Illustrative Example.
We shall apply the above procedure on the same example as was discussed in
the section describing the cutting plane approach.
So, we shall reintroduce the problem
sup 4x
1
+ x
2
s.t. x
1
+ 2x
2
4
2x
1
+ x
2
3
(7.40)
(x
1
, x
2
) 0 and integer.
Let X = (x
1
, x
2
) 0 [ 2x
1
+ x
2
3. Assume at the beginning that

X =
(0, 0), (
3
2
, 0) which then contains some rst selected extreme points of the set
X. Here we have
B =
_
_
_
_
_
_
1 2
1 0
0 1
1 0
0 1
_
_
_
_
_
_
.
ROW DECOMPOSITION 77
Let l
j
= (l
j1
, l
j2
) and u
j
= (u
j1
, u
j2
), where the number of elements is equal to
the number of variables. At each terminal to be constructed we thus have

j
=
_
_
_
_
_
_
4
u
j1
u
j2
l
j1
l
j2
_
_
_
_
_
_
and also
j
=
_
_
_
_
_
_
y

j1

j2

j1

j2
_
_
_
_
_
_
.
1. lower problem.
We shall here use the linear programming form outlined in (7.35).
inf
s.t. 0y 4y + 0
(1
3
2
+ 2 0)y 4y + 6
y 0
or simplied
inf
s.t. 4y + 0

5
2
+ 6
y 0
The optimal dual variables solving the dual in the form (7.36) are (
1
,
2
) =
(0, 1). Since (x
1
, x
2
) =
1
(0, 0) +
2
(
3
2
, 0) = (
3
2
, 0) which is nonintegral, we
shall separate on x
1
, so that x
1
1 or x
1
2. So we develop the following two
terminals.
terminal 1. Here x
1
u
11
= 1. So we are led to study
inf
s.t. 4y
1

11
+ 0

5
2
y
1
+
1
2

11
+ 6
y
1
,
11
0.
An optimal solution is (y
1
,
11
) = (0, 4) with = 4. So in full dimension we
get

1
= (y
1
,
11
,
12
,
11
,
21
) = (0, 4, 0, 0, 0). The dual solution is (
1
,
2
) =
(
1
3
,
2
3
), implying that x =
1
3
(0, 0) +
2
3
(
3
2
, 0) = (1, 0) and hence integer.
terminal 2. Here x
1
l
21
= 2. So we shall study
inf
s.t 4y
2
+ 2
21
+ 0

5
2
y
2
+
1
2

21
+ 6
y
2
,
21
0.
78
This problem has an unbounded solution. An extreme ray leading to unbound-
edness is ( y
2
,
21
) = (0, 1). We therefore introduce

2
= ( y
2
,
21
,
22
,
21
,
22
) = (0, 0, 0, 1, 0).
So the lower problem terminates with the bound w = 4 obtained as the optimal
objective function value in terminal 1.
1. upper problem.
For terminal 1 we have by (7.38) that
H
1
(x) = 4x
1
+x
2
4x
1
+ 4 1 = x
2
+ 4.
For the unbounded terminal 2 we restrict our attention to points x so that
(Bx +
j
) 0, according to (7.38). Otherwise x can never enter in an
optimal solution of the upper problem as for example stated in the form (7.39).
Here

2
= (4, +, +, 2, ).
So
2
(Bx+
2
) 0 gets the form x
1
2 0 or x
1
2. So according to (7.39)
H
2
(x) =
_
4x
1
+x
2
, if x
1
2
otherwise.
So the upper problem based on the formulation (7.39) becomes
z = sup (x
2
+ 4, H
2
(x))
s.t. 2x
1
+x
2
3
x
1
, x
2
0
Since no feasible solution of this problem satises the inserted constraint x
1
2
for H
2
(x) we get
z = sup x
2
+ 4
s.t. 2x
1
+x
2
3
x
1
, x
2
0.
An optimal solution is (x
1
, x
2
) = (0, 3) with z = 7.
2. lower problem.
From the previous upper problem we shall introduce (x
1
, x
2
) = (0, 3) by the
addition of an extra row. The lower problem then becomes
inf
s.t. 4y + 0

5
2
y + 6
2y + 3
y 0.
ROW DECOMPOSITION 79
An optimal solution for the dual problem is (
1
,
2
,
3
) = (0, 1, 0). Since
(x
1
, x
2
) =
1
(0, 0) +
2
(
3
2
, 0) +
3
(0, 3) = (
3
2
, 0) is noninteger we separate as
before and get two new terminals:
terminal 1. Here we have x
1
1, and we get the problem
inf
s.t. 4y
1

11
+ 0

5
2
y
1
+
1
2

11
+ 6
2y
1

11
+ 3
y
1
,
11
0.
An optimal solution is (y
1
,
11
) = (0, 2) with = 5. The corresponding dual
solution is (
1
,
2
,
3
) = (0,
2
3
,
1
3
) implying that x =
2
3
(
3
2
, 0) +
1
3
(0, 3) = (1, 1),
which is integer.
terminal 2. Here x
1
2 and we are going to consider the problem
inf
s.t 4y
2
+ 2
21
+ 0

5
2
y
2
+
1
2

21
+ 6
2y
2
+ 2
21
+ 3
y
2
,
21
0.
Again we have unbounded solution. As previously we nd the ray
v
2
= (0, 0, 0, 1, 0).
So the lower problem terminates with the bound z = 5 found in terminal 1.
2. upper problem.
With respect to the unbounded terminal there is no deviation from the
calculations made during the previous upper problem as the tranferred ray is
unchanged. However some change occurs in the calculations with respect to
the other terminal. So the upper problem is here of the form
sup (4x
1
+x
2
2x
1
+ 2 1, H
2
(x))
s.t. 2x
1
+x
2
3
x
1
, x
2
0.
Since again H
2
(x) = for all feasible solutions we get
z = sup 2x
1
+x
2
+ 2
s.t. 2x
1
+x
2
3
x
1
, x
2
0.
80
The optimal value z = 5 = w. Hence the procedure stops. The nal solution
is determined during the last calculations in the lower problem. The optimal
solution for (7.40) is therefore (x
1
, x
2
) = (1, 1).
7.5.2 Unboundedness and Infeasibilities.
As the example shows the tree T is naturally developped simultaneously with
the calculations of the lower problem according to a standard branch and bound
approach. Since, due to separation, the sets

I
j
are likely to be empty it is per-
haps here worthwhile to discuss more carefully the situation of unboundedness
or infeasibility of the various programs. We have earlier studied similar sit-
uations, for example in section 5.3 on linear programming. In such cases we
have assumed existence of an optimal solution. We have also mentioned the
possibility to introduce a phase 1 procedure to obtain feasibility, if possible.
The same diculties may arise in all terminals during the construction of the
tree. So, for simplicity we shall just study the rst tree which only contains the
root. Here we study the linear programming relaxation of the upper problem.
Hence what may be derived here can be directly compared with the develop-
ments in section 5.3 on linear programming, and we shall take the opportunity
to do that. Let j = 1 denote the index of the root and the only terminal at the
moment. The root does not contain additional bounds on the variables. So in
this case the dual variables (
1
,
1
) = 0. The calculations for the lower problem
in the form (7.35) subsumes to
w = inf
s.t. y
1
(A
1
x
k
b
1
) + cx
k
for all k Q (7.41)
y
1
0.
Already here at the root it may happen that (7.41) becomes unbounded.
Whenever feasible solutions exist for the original problem (7.8) this is due to
an unfortunate selection of the beginning set of extreme points

X. The dual
program of (7.41) is
sup

k
0

kQ

k
cx
k
s.t.

kQ

k
A
1
x
k
b
1
(7.42)

kQ

k
= 1.
Since (7.41) is always feasible we have by duality that (7.41) is unbounded if and
only if (7.42) is infeasible. Observe, that the problem (7.42) is very similar to
the lower problem (5.10), studied in section 5.3. The only distinction between
(5.10) and (7.42) is the inclusion of extreme rays of X in (5.10).
ROW DECOMPOSITION 81
The above procedure, treating infeasible terminals as well, demonstrates how
a phase 1 procedure may be derived.
If (7.41) is unbounded or equivalently (7.42) is infeasible we then discover an
extreme ray
1
. This is transferred to the next upper problem which according
to (7.38) and (7.39) may be written as:
z = sup
x0
cx
s.t. A
2
x b
1
(7.43)

1
(A
1
x +b
1
) 0.
If (7.43) has an optimal solution x
k
this solution is transferred to the lower
poblem for inclusion in

X.
It is exactly at this point we may speak of a phase 1 procedure. The purpose
of (7.43) is to transfer extreme points to the lower problem until (7.41) is no
longer unbounded, if possible.
The upper program (7.43) may be infeasible. According to standard con-
vention we have then z = . So we have z = w = . Therefore, according
to our termination criterion we shall stop. In this situation the upper bound
is , implying that (7.41) stays unbounded even if Q included all extreme
points of X, in which case the original problem (7.8), even without the inte-
grality condition, is infeasible.
We have in the present context assumed that X = x 0 [ A
2
x b
2
is
bounded. If not, unboundedness of the upper problem (7.43) must be consid-
ered too. In that case an extreme ray x
l
X is selected and tranferred to
the lower problem as done in (5.10). The developments will be exactly as the
development of (5.9) and its dual (5.10) considered in section 5.3. Let here R
be the index set of the current extreme rays of X. With rays included the lower
problem (7.41) is then expanded to
inf
s.t. y
1
(A
1
x
k
b
1
) + cx
k
for all k Q (7.44)
y
1
A
1
x
l
c x
l
for l R
y
1
0.
If (7.44) is unbounded we proceed as before. However, it may now addition-
ally happen that (7.44) is infeasible. In that case we may argue as in section 5.3.
We thereby demonstrate that the linear programming relaxation of (7.8) is un-
bounded. So, if (7.8) has an integral feasible solution then (7.8) is unbounded,
too.
7.6 AN ECONOMIC INTERPRETATION.
We shall here give an economic interpretation of the decomposition procedure
applied on integer programming by means of cutting plane as well as branch
and bound technique.
82
So let us reconsider the underlying integer programming problem (7.8). We
shall operate with a central unit and a subunit. In the current row decom-
position scheme the two units operate by means of a joint set of activities to
be indicated by their activity levels. They should be selected so that income
is maximized subject to the constraints, as specied in (7.8). However, the
central unit is supposed only to consider the rst set of constraints dened by
x 0[A
1
x b
1
in addition to the integrality requirement on the activity
levels. This last requirement seems natural, since the introduction of integral-
ity conditions in most economic modelling of this sort usually relates to xed
cost decisions that are generally taken by the central unit. The subunit has
knowledge about the local constraints given by x 0[A
2
x b
2
. The purpose
of the procedure is via a pricing mechanism to motivate the subunit by solving
its own problem, which here is the upper problem, in order to suggest good
and locally feasible activity proposals (given by x
k
) for transfer to the central
unit.
The central unit combines those proposals into a current best and also cen-
trally feasible solution and generates simultaneously via solution of its own
(here the lower) problem a current price information, which provides the nec-
essary parameters for the denition of the objective to be used by the subunit.
Thus, the objective functions in the upper problems (6.8) and (6.13) gives the
appropriate current price information incorporating income, the imputed costs
for utilization of the central resources as specied by the set P
1
and the inte-
grality requirement.
The upper problems, for example (7.16) in cutting plane, and (7.39) in
branch and bound, both have the same set of constraints given. Moreover,
it is emphasized that the objective functions have similar structure, both being
convex and polyhedral. This tells us that the structure of the price information
is independent on the choice of the two main types of algorithms in integer
programming.
Finally, it should be pointed out that the existence of a good algorithm in
both cases is equivalent to the construction of a simple price information, as
the complexity of the functions involved increases with the number of cuts or
the size of the tree, respectively.
7.7 ALTERNATIVE APPROACHES.
In the preceeding sections we have developed a decomposition method for inte-
ger programming in which the lower problem is a mixed integer programming
problem and the upper problem is a maximization problem with a convex ob-
jective function. The procedure can be extended to cover the mixed integer
programming case as well without any change in the types of the lower and
upper problem, respectively. In this context we should mention that Rana,
1992 has established a procedure in which the lower problem is a pure inte-
ger programming problem and the upper problem is a (simpler) mixed integer
programs. It is based on classical Lagrangean approach. Hence optimality can-
ROW DECOMPOSITION 83
not be guaranteed. However, good performance is demonstrated on an airline
routing problem. The technique is based on Sweeney and Murphy, 1979.
8 MIXED DECOMPOSITION
In the previous chapters we have studied procedures with quite dierent forms
for the upper problem and the lower problem, respectively. Typically one of
the two is signicantly simpler to solve than the other. This dierence in often
due to accumulation or non-accumulation of previous solutions. For example
the set

X may accumulate all solutions from previous iterations or may consist
of just one point. The idea of a mixed decomposition procedure is to keep
the lower problem as well as the upper problem simple. This might invalidate
proof of converge. However it sometimes signicantly improves progress of the
procedure by obtaining tight bounds quickly, and good feasible solutions.
8.1 CROSS DECOMPOSITION.
Van-Roy, 1983 has used this idea in the development of his cross decomposition
procedure. Again, assume for simplicity that the lower problem and the upper
problem always have optimal solutions. Let x
k
and y
k
denote the optimal
solutions in the kth iteration of (5.5) and (5.6), respectively. Then the idea is
to keep the sets

X and

Y small by setting

X =

X(k) = x
k
and

Y =

Y (k) =
y
k
. Cross decomposition can therefore be described as a coordinatewise search
procedure.
As already indicated one cannot in general expect to obtain an optimal
solution for (5.1), or (5.2), with this scheme. So occasionally, one has to modify
85
86
the map

X(k) (or

Y (k)) for example to the standard updating procedure for
Benders decomposition.
Some special instances are of particular interest because of the attractiveness
of (5.5) and (5.6) in cross decomposition. Consider for example the classical
capacitated plant location problem treated by Van-Roy, 1986 with n plants and
m markets. For this let d
j
denote the demand of the jth market. Let s
i
be
the capacity of plant i and let f
i
be the xed cost for opening plant i, Let x
ij
denote the transportation ow and c
ij
the unit transportation costs form plant
i to market j. Finally, introduce the binary variable y
i
so that y
i
= 1 if plant
i is opened, and 0 otherwise.
We can now specify the terms of the general procedure. We shall as an
exception, following our assumption about existence of optimal solutions, here
use max and min instead of sup and inf, respectively. Let = (
1
, . . . ,
n
)
IR
n
+
and consider
K(, y) = min
x
ij
n

i=1
m

j=1
(c
ij
+d
j

i
)x
ij

n

i=1
s
i
y
i

i
+
n

i=1
f
i
y
i
s.t.
n

i=1
x
ij
= 1 j
0 x
ij
y
i
i, j.
Further, let Y = (y
1
, . . . , y
n
) [ y
i
= 0 or 1 and X = IR
n
+
. We shall consider
the problem in the dual form (5.2). Hence we consider
min
yY
max
0
K(, y). (8.1)
By linear programming duality applied on the inner problem then (8.1) is equiv-
alent to
min
yY
min
x
ij
n

i=1
m

j=1
c
ij
x
ij
+
n

i=1
f
i
y
i
s.t.
n

i=1
x
ij
= 1 j
m

j=1
d
j
x
ij
s
i
y
j
i
y
i
=
_
0
1
x
ij
0
which is the capacitated plant location problem in standard formulation.
MIXED DECOMPOSITION 87
In the cross decomposition scheme for (5.2) then the lower problem (5.6)
gets the following form with xed =

:
min
y
i
min
x
ij
n

i=1
m

j=1
(c
ij
+d
j

i
)x
ij

n

i=1
s
i
y
i

i
+
n

i=1
f
i
y
i
s.t.
n

i=1
x
ij
= 1 j
0 x
ij
y
i
i, j
y
i
=
_
0
1.
This is the simple plant location problem.
The upper problem (5.5) gets the form with xed y

i
:
min
x
ij
n

i=1
m

j=1
c
ij
x
ij
+
n

i=1
f
i
y

i
s.t.
n

i=1
x
ij
= 1 j
m

j=1
d
j
x
ij
s
i
y

j
i
x
ij
0
which is a transportation problem, providing the values of the dual variable

i
belonging to the constraints,

m
i=1
d
j
x
ij
s
i
y

i
.
8.2 SOME ADDITIONAL REMARKS.
One might naturally include several additional decomposition procedures into
the category of mixed procedures discussed in the present chapter. For example,
the decomposition scheme of Kornai and Liptak, 1965. In this case the sets

X and

Y contain the averages of all solutions, obtained during the previous
iterations. Various proposals of Holmberg, 1985 fall into this category as well.
In particular, se should mention the work by Holmberg, 1992 on linear mean
value cross decomposition, which represents a generalisation of the Kornai and
Liptak procedure, see Kornai and Liptak, 1965.
By combination of variable decomposition and row decomposition technique
then mixed procedures have been proposed by Obel, 1978 and Meijboom, 1987.
Finally, we should also mention that approaches for comparison of dierent
decomposition procedures t well into the suggested framework, for example
the comparison performed in Aardal and Ari, 1990 between the Kornai-Liptak
and the cross decomposition procedure.
It all depends on the specic choice of the function K(x, y), the sets X and
Y , and the updating formulas for

X =

X(k) and

Y =

Y (k).
References
Aardal, K. and Ari, A. (1990). On the resemblance between the kornai-liptak
and cross decomposition techniques for block-angular linear programs. Eu-
ropean Journal of Operational Research, 46:393 398.
Bazaraa, M. S., Jarvis, J. J., and Sherali, H. D. (1990). Linear programming
and network ows, second edition. Wiley.
Benders, J. F. (1962). Partitioning procedures for solving mixed-variables pro-
gramming problems. Numerische Mathematik, 4:238 252.
Burkard, R. E., Hamacher, H. W., and Tind, J. (1985). On general decompo-
sition schemes in mathematical programming. Mathematical Programming
Study, 24:238 252.
Burton, R. M. and Obel, B. (1984). Designing Ecient Organizations: Mod-
elling and Experimentation. North-Holland, Amsterdam.
Dantzig, G. B. and Wolfe, P. (1961). The decomposition algorithm for linear
programs. Econometrica, 9:767 778.
Desaulniers, G., Desrosiers, J., Solomon, M. M., and Soumis, F. (1994). Daily
aircraft routing and scheduling. Technical Report Report G-94-21, GERAD,
Montreal, Canada.
Dirickx, Y. M. I. and Jennergren, L. P. (1979). Systems analysis by Multi-
Level Methods: With Applications to Economics and Management. Wiley,
Chichester.
Everett, H. (1963). Generalized lagrange multiplier method for solving problems
of optimum allocation of resources. Operations Research, 11:399 417.
Flippo, O. E. (1991). Stability, Duality and Decomposition in General Math-
ematical Programming, volume 76 of CWI Tract. Stichting Mathematisch
Centrum Centrum voor Wiskunde en Informatica, Amsterdam.
Ford, L. R. and Fulkerson, D. R. (1958). A suggested computation for multi-
commodity ows. Management Science, 5:97 101.
Georion, A. M. (1970). Primal resource directive approaches for optimizing
nonlinear decomposable systems. Operations Research, 18:395 403.
Georion, A. M. (1972). Generalized benders decomposition. Journal of Opti-
mization Theory and Applications, 10:237 260.
89
90
Gon, J. L., Haurie, A., Vial, J.-P., and Zhu, D. L. (1993). Using central prices
in the decomposition of linear programs. European Journal of Operational
Research, 64:393 409.
Gomory, R. E. (1960). An algorithm for the mixed integer problem. RM-2597,
(RAND Corp.).
Holloway, C. A. (1973). A generalized approach to dantzig-wolfe decomposition
for concave programs. Operations Research, 21:210 220.
Holm, S. and Tind, J. (1985). Decomposition in integer programming by su-
peradditive functions. Technical report, Department of Operations Research,
University of Aarhus, Denmark.
Holm, S. and Tind, J. (1988). A unied approach for price directive decom-
position procedures in integer prgramming. Discrete Applied Mathematics,
20:205 219.
Holmberg, K. (1985). Decomposition in Large Scale Mathematical Program-
ming. PhD thesis, Department of Mathematics, University of Linkoping,
Sweden.
Holmberg, K. (1992). Linear mean value cross decomposition: A generaliza-
tion of the kornai-liptak method. European Journal of Operational Research,
62:55 73.
Kornai, J. and Liptak, T. (1965). Two-level planning. Econometrica, 33:141
169.
Meijboom, B. R. (1987). Planning in Decentralized Firms: A Contribution to
the Theory on Multilevel Decisions. Lecture Notes in Economics and Math-
ematical Systems. Springer, Berlin.
Minoux, M. (1986). Mathematical Programming: Theory and Algorithms. Wiley.
Nemhauser, G. L. and Wolsey, L. A. (1988). Integer and Combinatorial Opti-
mization. Wiley-Interscience, New York.
Nemhauser, G. L. and Wolsey, L. A. (1990). A recursive procedure to generate
all cuts for 0-1 mixed integer programs. Mathematical Programming, 46:379
390.
Obel, B. (1978). A note on mixed procedures for decomposing linear program-
ming problems. Mathematische Operationsforschung und Statistik, Series
Optimization, 9:537 544.
Obel, B. (1981). Issues of organizational design: A mathematical programming
view of organizations. Pergamon Press.
Ponstein, J. (1983). Comments on the general duality survey by j. tind and l.
a. wolsery. Mathematical Programming, 25:240244.
Rana, K. (1992). A decomposition technique for mixed integer programming
problems. Computers and Operations Research, 19(6):505519.
Rockafellar, R. T. (1970). Convex Analysis. Princeton University Press.
Rockafellar, R. T. (1974). Augmented lagrange multiplier function and duality
in nonconvex programming. SIAM Journal on Control, 12:268 283.
Salkin, H. M. (1975). Integer Programming. Addison-Wesley Publishing Com-
pany.
REFERENCES 91
Sweeney, D. J. and Murphy, R. A. (1979). A method of decomposition for
integer programs. Operations Research, 27(6):11281141.
Tind, J. (1991). Decomposition in global optimization. Journal of Global Opti-
mization, 1:131 141.
Tind, J. and Wolsey, L. A. (1981). An elementary survey of general duality
theory in mathematical programming. Mathematical Programming, 21:241
261.
Van-Roy, T. J. (1983). Cross decomposition for mixed integer programming.
Mathematical Programming, 25:46 83.
Van-Roy, T. J. (1986). A cross decomposition algorithm for capacitated facility
location. Operations Research, 34:145 163.
W. Cook, R. K. and Schrijver, R. (1990). Chvatal closures for mixed integer
programming. Mathematical Programming, 47:155 174.
Wolsey, L. A. (1981). A resource decomposition algorithm for general mathe-
matical programs. Mathematical Programming Study, 14:244257.

You might also like