LINEAR PROGRAMMING I

Patrik Forssén
Office: 2404
Phone: 018/471 29 66
E-mail: patrik@tdb.uu.se
INTRODUCTION
ABOUT THE LECTURE NOTES

l This material is based on:

– Introduction to Linear Optimization by Dimitris Bertsimas and John
N. Tsitsiklis.
– Lecture notes from the course EE236A held at UCLA by Prof.
Lieven Vandenberghe.

l The lecture notes can be downloaded from the course homepage:

http://www.tdb.uu.se/kurs/optim-mn1/
THE LINEAR PROGRAMMING PROBLEM (LP)

An LP-problem can be formulated as:

min c1x1 + K + cn x n
subject to

 a11x1 + K + a1n x n ≤ b1
 min cT x
 M 
 or in matrix notation Ax ≤ b ,
am1x1 + K + amn x n ≤ bm Gx = h
g11x1 + K + g1n x n = h1 

 M
g p1x1 + K + g pn x n = hp


i .e., the goal is to find n decision variables x1,K, xn fulfilling the m linear inequality
constraints ai 1x1 + K + ain ≤ bi and the p linear equality constraints g i 1x1 + K + g in xn = hi ,
i .e., a feasible solution, so that the linear objective function c1x1 + K + cn xn is minimized.
AN ENGINEERING EXAMPLE
Consider the following single input/single output system:
y (t ) = h0u(t ) + h1u(t − 1) + h2u(t − 2) + K
where
• u(t ) is the input and y (t ) the output,
• hi are called the impulse response coefficients,
• impulse response is finite (of order k ) if hi = 0 for i > k .

OUTPUT TRACKING PROBLEM :

Given desired output y des (t ), t = 0,K, N determine the input u(t ) so that the peak deviation
max y(t) − y des (t) is minimized.
t = 0 ,K,N
We also impose the following constraint on the amplitude and the slew rate of the input:
u(t ) ≤ U, u(t + 1) − u(t ) ≤ S.

This problem is easily solved as an LP-problem (more later!)

WHAT WE WILL COVER

l Geometry of the LP-problem.

l The simplex method.

l Duality theory.

l Transportation and network flow problems.

l Applications.
BRIEF HISTORY
1930s. Kantorovich (Soviet Union): economic applications (production planning).

1940s. Dantzig: military logistics problems during WW2. 1947 Simplex algorithm.

1950s-1960s. Recognition of applications in many other fields, e.g., structural optimization,

control theory, filter design, …

1979 Ellipsoid algorithm (Khachiyan), more efficient than simplex in worst case, but slower in
practice.

1984 Projective (interior-point) algorithm (Karmarkar), more efficient than simplex in worst
case and efficient in practice.

1984-today Many variations of improved interior-point methods, software for large-scale

problems.
GEOMETRY OF LINEAR PROGRAMS
WHAT WE WILL DO

• Solve a simple LP geometrica lly.

• Learn how to rewrite the LP - problem to different but equivalent forms.

• Examine the feasible set of an LP from a geometrical viewpoint, i.e.,

examine the set { x Ax ≤ b, Gx = h}.

• Examine if there exist optimal solutions and how they are positioned .
A DIET PROBLEM
A farmer is planning to buy food for his cows. He can choose between two different brands, A
and B. A costs 30 cent/kg and B costs 12 cent/kg.

The brands contains the following units (per kg) of the three nutrients n1, n2 and n3.
n1 n2 n3
A 3 7 3
B 2 2 6

The farmer needs at lest 60 units of n1, 84 units of n2 and 72 units of n3. Determine the
cheapest way to fulfil these requirements, i.e., how much he should buy of brand A and of
brand B to minimize the cost!

What happens if the price of brand A is changed to 27 cent/kg and the price of brand B is
changed to 18 cent/kg?
 B
7 A + 2B ≥ 84

40

3 A + 2B ≥ 60
30

20

3 A + 6B ≥ 72
10

A
10 20 30 40
VARIANTS OF THE LP-PROBLEM
l General form
 min cT x

 Ax ≤ b
 Gx = h

where c,x ∈ R n , A ∈ R m×n , b ∈ R m , G ∈ R p×n and h ∈ R p .

l Inequality form:
min cT x

Ax ≤ b

l Standard form
min cT x

Ax = b
x ≥ 0

REDUCTION TO INEQUALITY / STANDARD FORM

Suppose we have an LP in general form:

min cT x

Ax ≤ b
Gx = h


l Reduction to inequality form:

min cT x

 Ax ≤ b

 Gx ≤ h
− Gx ≤ −h

l Reduction to standard form:

 min cT x + − cT x −
 +
 Ax − Ax − + s = b
 + −
 Gx − Gx = h
 x + , x − , s ≥ 0
variables are x + , x − and s; recover x as x = x + − x −
s ∈ R m is called a slack variable.
 SUBSPACES AND AFFINE SETS
DEFINITION : S ⊆ R n ( S ≠ ∅ ) is called a subspace if
x, y ∈ S, α ,β ∈ R ⇒ α x + β y ∈ S.
α x + β y is called a linear combination of x and y.
a subspace

A subspace can be written as:

v
S = span ( v1,K, vk ) = {α1v1 + K + α k vk α i ∈ R } where vi ∈ R n

DEFINITION : Let S0 be a subspace in R n and let x 0 ∈ R n .

Define the set V as:
V = S0 + x 0 = { x + x 0 x ∈ S0 }, an affine set
then V is called an affine subspace.
x0
The above definition states that if we translate the subspace S0
by a vector x 0 we get an affine subspace.

The dimension of V is defined to be the same as the underlying

subspace S0 , i.e., dim ( V ) = dim ( S0 ) .