Background on Constrained Optimization and LP

Decision Models & Optimization Page 1

Background on Constrained Optimization and LP

Like all constrained optimization (mathematical programming) problems,
linear programs have three basic ingredients: decision variables, which are
items whose quantities that we can choose or control; an objective function,
whose value is a function of the decision variables and is minimized or
maximized; and constraints (, =, ) , which are also defined in terms of the
decision variables and must be satisfied for the solution to the problem to be
acceptable. More specifically, a constraint can always be formulated so that its
left-hand-side (the stuff to the left of the , =, or ) is a function of the decision
variables, and the right-hand-side is a constant, a simple number, rather than a
formula.

In constrained optimization problems, a solution refers to choice of a
specific set of values for the decision variables, while the Objective Function
Value (OFV) is the value the objective function takes on for a solution. For
example. if the objective function is defined to be

2 X
1
+6 X
2
,

then the solution (3 , 2) produces an OFV of 18. A feasible solution is a choice
of values that satisfies all of the constraints, while an infeasible solution is a
choice for which at least one constraint is violated. An optimal solution is a
feasible solution for which the OFV cannot be improved upon -- further minimized
or maximized -- by any other feasible solution. Note that there may be more than
one optimal solution.

Unlike other constrained optimization models, LPs always have a linear
objective function and linear functions on the left-hand-sides of their constraints.
Mathematically, a linear function of the decision variables X
1
, X
2
, . . ., X
n
can
always be written as

f(X
1
, X
2
, . . ., X
n
) =a
1
X
1
+a
2
X
2
+. . . +a
n
X
n
,

a function for which each decision variable is multiplied by a coefficient and
added to its peers. Two examples of nonlinear functions are

f(X
1
, X
2
) =a X
1
X
2
and f(X
1
, X
2
, X
3
) =a
1
X
1
+a
2
X
2
+a
3
log(X
3
).

Practically, linearity is intended to model situations in which the quantities
represented by the decision variables are divisible and for which the function
changes proportionally with changes in each variable. Continuous quantities,
such as tons of steel to produce and gallons of gasoline to consume, are
divisible, but lumpy quantities, such as numbers of aircraft carriers to produce,
Background on Constrained Optimization and LP
Decision Models & Optimization Page 2
are not. That is, if we constructed and solved an LP whose optimal solution
included the consumption of 5.57 gallons of gasoline, the answer would make
sense to us; we are able to consume fractions of gallons of gasoline. If the
optimal solution called for the production of 5.57 aircraft carriers, however, the
solution probably wouldnt make sense, since 0.57 aircraft carriers arent of much
use to anyone. Even if we assume the value is 0, its difficult to describe how to
measure fractional aircraft carriers, or what the costs of producing them would
be. The proportionality idea is similar: while the percentage of fat in dog food
changes in proportion to the mix and fat-content of its ingredients, the area of a
circle ( r
2
) is not proportional to its radius or circumference.

Later in the course, we will discuss how to use techniques similar to linear
programming to allow us to relax the assumptions of divisibility and
proportionality.

One final assumption implicit in the formulation of LPs (and of constrained
optimization problems, in general) is that the collection of numbers that make up
the objective function coefficients, constraint coefficients, and right-hand-sides
are fixed quantities that are known with certainty. In real life, however, there is
often uncertainty about the values these data take on, and we are forced to use
estimates of what the numbers are. For example, in the perfume production
problem, we might not know with certainty the contribution from selling each
ounce of perfume, because we anticipate industry-wide price increases to take
effect in the next selling period. It would still be useful, however, to run the LP
model developed for the problem, using our best estimate of what the
contribution will bewith the understanding that we may need to take the
models recommendation with a grain of salt when making our management
decision.