Lecture: Optimization of network flows

1. Minimum cost flow problems in networks

2. Modelling and graph theory.
3. The Simplex method.

Lecture Optimization on Networks

Optimization on networks

The minimum cost flow problem in networks

40

x13 /c13

x12 /c12 x23 /c23

35

x24 /c24

30

x35 /c35

x34 /c34

20

x45 /c45

25

Data is sent from servers in nodes 1 and 2 to terminals in nodes 3, 4, 5.

The cost for traffic in the link between node i and j is cij SEK/Kbyte.
We want to minimize the total cost for the data traffic
Lecture Optimization on Networks

Optimization on networks

The optimization problem formulation

X
minimize
cij xij
all arcs
s.t. x12 + x13 = 40
x12 + x23 + x24 = 35
x13 x23 + x34 + x35 = 30
x24 x34 + x45 = 25
x35 x45 = 20
xij 0, all flows in the links

Lecture Optimization on Networks

Optimization on networks

We will address the following questions

Modelling of the network flow problem

Graphs, trees, cycles, spanning trees.

How can the particular structure of the graph be used in the
Simplex method.

Lecture Optimization on Networks

Optimization on networks

Modelling and graph theory

1

3
5

A directed graph G = (N , B) consists of a set of nodes N = {1, . . . , m}

and a set of arcs B
The arc from node i to node j is denoted (i, j).
We differentiate from the arc (i, j) and the arc (j, i).

In the graph above

N = {1, . . . , 5}
B = {(1, 2), (1, 3), (2, 3), (2, 4), (3, 4), (3, 5), (4, 5)}
Lecture Optimization on Networks

Optimization on networks

If we arrange the arcs in B in some order, e.g.

B = {(1, 2), (1, 3), (2, 3), (2, 4), (3, 4), (3, 5), (4, 5)}
1

(1)

Rmn of the graph is defined as

then the incidence matrix A

aij =

1,

arc j starting in node i

1, arc j ending in node i

0,

(2)

otherwise

i
= 0, where eT = 1 1 . . . 1 . The rows
It is easy to see that eT A
are thus linearly dependent and it is possible to eliminate the last row to
obtain the reduced incidence matrix A.
Lecture Optimization on Networks

Optimization on networks

In our example we have

1
1
0
0
0

1 0
1
1
0

A=
1
0 1 1 0

0
0
0 1 1

0
0

1
0

0
1

0
0
0
0
0 1 1

1
1
0
0
0 0 0

1 0

1
1
0
0
0

A=

0 1 1 0

1
1
0

0
0
0 1 1 0 1

Lecture Optimization on Networks

Optimization on networks

Path and cycle

A path from node s to node t in a directed graph is a connected

sequence of arcs 1 , . . . , N where

k = (ik1 , ik ), or k = (ik , ik1 ), and i0 = s, iN = t

A directed path from node s to node t in a directed graph is a
connected sequence of arcs 1 , . . . , N where

k = (ik1 , ik ), and i0 = s, iN = t
A (directed) cycle is a (directed) path from a node to itself.

Lecture Optimization on Networks

Optimization on networks

 A graph is called connected if there exists a path between each pair

of nodes.
A graph is called acyclic if it contains no cycles.
A tree is formed by a connected subset of the graph that contains
no cycles (i.e., it is acyclic).
In a graph with n nodes a tree with n 1 arcs form a spanning tree.

Connected graph
NOT acyclic

Lecture Optimization on Networks

Acyclic graph
NOT connected

Tree

Spanning Tree

Optimization on networks

 If an arc is added to a spanning tree, then a unique cycle is created.

If an arc is removed from a spanning tree, then the tree is
decomposed into two new trees.

If an arc is removed
it is decomposed
into two new trees
If an arc is added
a unique cycle is created

Lecture Optimization on Networks

10

Optimization on networks

Balance of flow
b3
b1

x12
b2

x13
x23
x24

x35

x34

b5

x45

b4

xij denotes the flow (data traffic, oil, e.t.c.) in arc (i, j)

If the flow xij goes in the direction i j, then xij 0.

Otherwise xij 0.
bi denotes external flow in/out to node i.
Flow in if bi 0, and flow out if bi 0.

Lecture Optimization on Networks

11

Optimization on networks

Flow balance in node i (flow out = flow in):

n
X

aij xij = bi , i = 1, . . . , n

=b
Ax

j=1

is the incidence matrix of the graph

In the above balance equation, A
defined in equation (2) and x is a column vector with the flows xij
sorted in the same order as the arcs in equation (1).
h
iT
= b1 . . . bn .
Finally, b
T

Pn

The balance equation has solutions iff e b = i=1 bi = 0. With this

assumption we can reduce the flow balance equation by eliminating the
last row (i.e., the n:th balance equation is redundant). We arrive at the
h
iT
equation system Ax = b, where b = b1 . . . bn1 .
Lecture Optimization on Networks

12

Optimization on networks

Initial BFS

= (1 , . . . , m )
= (1 , . . . , l )

Calculate (y, r , b)
T
A y = c
r = c AT
y
=b
A b

Optimal solution

r 0

Yes

x = 0
x = b,
T

z
= cT
b = y b

No

Take q so that rq < 0

xq enters the basis

Solve A a
k = ak ; k := q

a
k 0

Yes

Unbounded problem
Solution does not exist

No

tmax = min

i
b

|
aik > 0

bp

a
ik
a
pk
old := q , q := p , p := old

Lecture Optimization on Networks

13

xp exits the basis

Optimization on networks

For network flow problems the simplex algorithm is simplified in the

following way:
1. Arcs corresponding to a basic index vector () corresponds to a
spanning tree
2. The values of the basic variables (determined by A x = b) follows
simply by balancing the flows in the spanning tree.
3. The Simplex multiplicators given by AT y = c are easily determined
by the spanning tree.
4. The quotient test is performed by studying the flow in a cycle.
5. The pivot step, (change of basis) is done by exchanging an arc in
the spanning tree.
We illustrate this on the example studied earlier.
Lecture Optimization on Networks

14

Optimization on networks

Basic matrices corresponds to spanning trees

Theorem m 1 columns from the (m 1) n matrix A in a
minimum cost flow problem are linearly independent iff the
corresponding m 1 arcs form a spanning tree.
The figure shows two basic matrices and the corresponding spanning
trees:
1

x12

x23

x24

x12

x45

6
61
A = 6
6
4 0
0

7
07
7
7
05

Lecture Optimization on Networks

x24

= {(1, 2), (2, 3), (2, 4), (3, 5)}

2

x35

x23

= {(1, 2), (2, 3), (2, 4), (4, 5)}

6
61
A = 6
6
4 0

15

7
07
7
7
15
0

Optimization on networks

Determining the basic solution Bx = b

b3
b1

x12
b2

x23
x24

b5

x45

b4

The basic variables can be determined from the flow balance equations in
the spanning tree (recall (b1 , b2 , b3 , b4 ) = (40, 35, 30, 25), b5 = 20.)

b1
1
0
0 0
x12 = b1 = 40

1 1
1
0
x23 = b3 = 30
2

x
=

b3

0
1
0
0
x24 = b2 + x12 x23 = 45

b4
0
0 1 1
x45 = x24 + b4 = 20

Note that x45 + x35 = 20, i.e., flow balance in node 5, follows automatically

Lecture Optimization on Networks

16

Optimization on networks

The basic flow can also be determined through the balance equations
downstreams:
x45 = b5 = 20
x24 = x45 b4 = 45
x23 = b3 = 30
x12 = x23 + x24 b2 = b1 = 40
Do you see why ?

b12
40

b 30
23

Note that x = b = = 0 so we have a feasible basic solution.

b24 45

b45
20
Lecture Optimization on Networks

17

Optimization on networks

Determining the simplex multiplicators yT A = cT

The Simplex multiplicators are determined by the equation yT A = cT .
To node 1, . . . , m 1 corresponds the simplex multiplicator yk ,
k = 1, . . . , m 1. If we let ym = 0 then
yi yj = cij , for all (i, j) (all basic arcs)
Hence, the simplex multiplicators can be determined from the network
3 y3

y1 1
c12
y2 2

Lecture Optimization on Networks

c23
c24

5 y5 = 0
4 y4

18

c45

Optimization on networks

3 y3

y1 1
c12
y2 2

c23
c24

5 y5 = 0
4 y4

c45

With cT = (c12 , c13 , c23 , c24 , c34 , c35 , c45 ) = (2, 5, 2, 2, 1, 1, 2) we obtain
(in this order since y5 = 0)
y4 y5 = c45 y4 = c45 = 2
y2 y4 = c24 y2 = y4 + c24 = 2 + 2 = 4
y2 y3 = c23 y3 = y2 c23 = 4 2 = 2
y1 y2 = c12 y1 = y2 + c12 = 4 + 2 = 6

Lecture Optimization on Networks

19

Optimization on networks

Determining the reduced costs r = cT yT A

The formula for the reduced costs r = cT yT A results in the
equations (ym = 0)
rij = cij yi + yj for all (i, j) (i.e., for all non-basic arcs)
In our example, we have

1
0 0

0
0

A =

1 1 1

0 1 0

= {(1, 3), (3, 4), (4, 5)},

Lecture Optimization on Networks

20

Optimization on networks

We can perform the calculations with help of the network

y1 1

r13 /c13

3 y3

r35 /c35

r34 /c34
y2 2

y5=0

4 y4

r13 = c13 y1 + y3 = 5 6 + 2 = 1
r34 = c34 y3 + y4 = 1 2 + 2 = 1
r35 = c35 y3 + y5 = 1 2 + 0 = 1
Note that r35 < 0 and the current basic feasible solution is not optimal.
Lecture Optimization on Networks

21

Optimization on networks

The Quotient test

Since r35 < 0 we let x35 = t, i.e., the arc (3, 5) enters the basis.
The question is now which arc should be removed in order to obtain a
new spanning tree corresponding to a new basic matrix.
b1

b3
1

x12
b2

x35 = t

x23
x24

b5

x45

b4



bp
bi
max
|
aik > 0 =
The test in the usual simplex method t
= min
aik
apk
is simplified by just looking at the flow balance in the cycle that has
been formed.
Lecture Optimization on Networks

22

Optimization on networks

30
40

x23

x35 = t

= b23 + t

x12 = b12
35

5
x24 = b24 t

20

x45 = b45 t

25

x23 = b23 + t = 30 + t
x24 = b24 t = 45 t
x45 = b45 t = 20 t
Note that t can be increased until tmax = 20 when x45 = 0.
Then x45 exits the basis.
Lecture Optimization on Networks

23

Optimization on networks

The Pivot step (change of basis)

30
40

x35 = 20

x23 = 50
x12 = 40
35

5
x24 = 25

20

25

We have = {(1, 2), (2, 3), (2, 4), (3, 5)} and

40

50

=
x = b
0
25

20

which is a new basic feasible solution.

Lecture Optimization on Networks

24

Optimization on networks

Next iteration

If we repeat the above steps, we get

y1
5

y 3
2
y = = ,
y3 1

y4
1

r13
1

= 1 0
r =
r
34

r45
1

Since all the reduced costs are positive, the current basic feasible
solution is optimal.

Lecture Optimization on Networks

25

Optimization on networks

Resum
e of the network flow simplex method
For network optimization, most steps in the simplex method are
simplified:
(i) A basic index vector corresponds to the arcs in a spanning tree.
(ii) The basic solution corresponding to can be determined from the
flow balance equations in a spanning tree. The basic solution is
feasible if the flow in all basic arcs are non-negative.
(iii) The Simplex multiplicators are determined by (ym = 0)
yi yj = cij , for all (i, j) (all basic arcs)
(iv) The reduced costs are determined from
rij = cij yi + yj for all (i, j) (i.e. for all non-basic arcs)
(v) If all rij 0 an optimal basic solution has been found.
Lecture Optimization on Networks

26

Optimization on networks

(vi) If some reduced cost rij is negative (use the most negative), add the
arc (i, j) to the spanning tree corresponding to the basis . A cycle
is formed and we apply a flow xij = t 0. By studying the balance
of flow in the cycle, it is easy to see how much t can be increased
until one flow in the cycle is zero.
If t can be increased arbitrarily, the optimization problem has no
bounded solution.
(vii) Update the spanning tree by adding the arc (i, j) to and removing
the arc with zero flow. A new basis is obtained (i.e., a new spanning
tree) and a new basic feasible solution.
(viii) Repeat from (iii).

Lecture Optimization on Networks

27

Optimization on networks

The four fundamental subspaces for the incidence matrix

This part is for illustration only
We demonstrate how the four fundamental subspaces for the incidence
can be interpreted (with an electric analogue) with the help of
matrix A
the associated graph G = (N , B). Returning to our example:
y1

x13

y3

x35
y5

x23

x12

x34
x45

2
y2

x24

4
y4

To every node j we associate a potential yj , j = 1, . . . , m.

To every arc (i, j) we associate a flow (current) xij .
Lecture Optimization on Networks

28

Optimization on networks

We note that the network corresponds to the electric circuit below:

y1
x12

x13

u13

y3

u35

x34

u12

u34
x23

y2

x35

x24

y5

u23

u24

y4

x45

u45

yj , j = 1, . . . , 5 denotes the potential in node j.

xij denotes the current in arc (i, j).
uij denotes the voltage over the resistor in arc (i, j).
Lecture Optimization on Networks

29

Optimization on networks

Left nullspace
= u corresponds to the physical laws
Interpretation: The equation y
TA
over the arcs (resistors)
yi yj = uij

voltage = difference in potential

There is no unique solution since we can add an arbitrary constant to all

node potentials without changing the voltages over the arcs.
The equation yT A = u correspond to the potential y5 is fixed at zero.
(earth)

T ) = span e = 1
N (A

Lecture Optimization on Networks

T ), then u = 0.
If y N (A
i.e., if all potentials are equal,
then all voltages are zero.

30

Optimization on networks

The (right) Null space

1
0
0

1
0
0

1
1
0

n
o

= x R7 : Ax
= 0 = span 0 , 1 , 0 = N (A)
N (A)

0
1
1

0
0
1

0
0
1

Interpretation: The Nullspace corresponds to three cycles in the network

3

1
1
2

2
Lecture Optimization on Networks

4
31

Optimization on networks

Other cycles in the graph are given by linear combinations of the three
basis cycles in the previous figure.
1

5
2

4
(basis)cycle 1 cycle 2

5
2

4
Cycle 1 cycle 2 + cycle 3

4
Cycle 2 cycle 3

The flow in/out of the circuit is not affected by the current x

in a cycle
x = 0.
since A

Lecture Optimization on Networks

32

Optimization on networks

The Rowspace

n
o
T) = u = A
Ty
R(A

(= u = A y )

= N (A)
(= N (A) )


7
= u R : u13 = u12 + u23 ; u24 = u23 + u34 ; u35 = u24 + u45

Interpretation: The sum of the voltages over a cycle is equal to zero.

This is also called Kirchoffs voltage law.
Note that there are more possible cycles, e.g., we have the constraint
u12 + u24 = u13 + u34 , but these constraints follows from linear
combinations of the three others (see previous slide).

Lecture Optimization on Networks

33

Optimization on networks

The Rangespace

= N (A
T ) = {b Rm : eT b = 0}
R(A)
= {b Rm :

m
X

bk = 0}

k=1

If bj is interpreted as an external current that is fed in to node j, then

the result says that the sum of these currents must be zero.

Lecture Optimization on Networks

34

Optimization on networks

Kirchoffs laws
x13

b1

u13

b3
x35 u35

x12

x34

u12

u34
x23

u23

b2
x24

b5

x45
u24

u45

b4

Kirchoffs Current Law: The sum of the currents in to a node is

= b.
zero. For the circuit in the figure, this can be written Ax
Pm
A necessary assumption is of course that j=1 bj = 0.

Kirchoffs Voltage Law: The sum of all voltages over a cycle is equal
to zero.
Lecture Optimization on Networks

35

Optimization on networks

Reading instructions

Chapter 7.2 in the book.

Lecture Optimization on Networks

36

Optimization on networks