Readings

Economics
E
i 172
Introduction to Operation Research
(Part 2)

Hillier & Lieberman (8th edition)

Ch. 12: Intro.
12.1:
12 1: Sample Applications.
Applications
12.2: Graphical Illustration on Nonlinear
Programming Problems
Problems.
12.3: Types of Nonlinear Programming
Problems.
Problems
Appendix 2: Convexity

1. Introduction
Spring 2009
Herb Newhouse
1

Outline

Non-Linear
Non
Linear Programming

Introduction to non-linear
non linear programming.

In economics applications we often run

into non-linear optimization problems.

Unconstrained optimization.
Equality constrained optimization.
Inequality constrained optimization.

The basic objective is the same as in

linear programming.

Concavity & Convexity

Concave/convex functions of one variable.
Concave/convex functions of multiple
p
variables.
Convex sets.

The techniques are different than those

used in linear programming because we
may get interior solutions.
3

Unconstrained Optimization

Unconstrained Optimization

We want to find the global (or local)

maximum or minimum for a function.

max
f x

For instance, a monopolist may wish to

choose
h
a quantity
tit tto produce
d
to
t maximize
i i
profit.

x x1 , x2 , xn

Unconstrained Optimization

Equality Constrained
We want to find the maximum or minimum
for a function subject to an equality
constraint.
constraint

f x

F
For instance,
i t
maximize
i i utility
tilit subject
bj t tto a
budget constraint.
Or minimize the cost of producing a given
amount of output.

x
7

Equality Constrained

Equality Constrained
x2

max
f x subject to gi x bi for i 1,
1 2,
2 , m

x x1 , x2 , xn

g x1 , x2 100
x1
9

Equality Constrained

10

Inequality Constrained
We want to find the maximum or minimum
for a function subject to an inequality
constraint.

x2

For instance, maximize utility subject to a

budget constraint (that doesnt have to be
p
y spent).
p )
completely

g x1 , x2 b

Or minimize the cost of producing at least a

given amount of output.

x1
11

12

Inequality Constrained

Inequality Constrained
x2

max
f x subject to gi x bi for i 1,
1 2,
2 , m

x x1 , x2 , xn

g1 x1 , x2 100

g 2 x1 , x2 150
x1

13

Inequality Constrained

14

Inequality Constrained

x2

x2
g1 x1 , x2 b1

g1 x1 , x2 b1

g 2 x1 , x2 b2

g 2 x1 , x2 b2

x1

x1
15

16

Concavity and Convexity

We use the terms concave and convex to
describe the curvature of a function.
We say f(x) is concave if the entire line
segment connecting any two points on the
function lies below the function
function.

Concavity and Convexity

We can test this with the second derivative.

A function
f
ti is
i concave if f ''( x) 0 ffor allll x in
i
the domain of f(x)
17

18

Convex Functions

A Concave Function

We say f(x) is convex if the entire line

segment connecting any two points on the
function lies above the function
function.

y x2 4 x

We can test this with the second derivative.

A function is convex if f ''(( x) 0 for all x in the
domain of f(x)

19

20

A Concave Function

Concavity and Convexity

Formally:

We can also show this function is negative

g
by
y
determining the sign of the second derivative.

A function g x is concave if

y x 4x
2

g x 1 y g x 1 g y

y ' 2 x 4
y '' 2 0

for all x y and for any 0 1.

A function g x is convex if
g x 1 y g x 1 g y
for all x y and for anyy 0 1.
21

22

A function g x is concave if
g a 1 b g a 1 g b

A Concave Function

g x x 2 4 x, a 1, b 4,

g x x 4x
2

a 1 b

2
3

2
2
1 1 4 2
3
3

g a 1 b g 2 2 4 2 4
2

g a g 1 1 4 1 3, g b g 4 4 4 4 0
2

g a 1 g b
23

2
2
3 1 0 2
3
3

24

Strict/Weak Concavity

Strict/Weak Convexity

If the inequalities are strict we say the function is strictly concave.

If the inequalities are strict we say the function is strictly convex.

If the inequalities are weak we say the function is weakly concave.

If the inequalities are weak we say the function is weakly convex.

A ffunction
i iis strictly
i l concave if f '' x 0 for
f all
ll x in
i the
h doma
d
i
in.

A ffunction
i iis strictly
i l convex if f '' x 0 for
f all
ll x in
i the
h domain.
d
i

A function is weakly concave if f '' x 0 for all x in the domain.

A function is weakly convex if f '' x 0 for all x in the domain.

A function g x is strictly concave if

A function g x is strictly convex if

g x 1 y g x 1 g y

g x 1 y g x 1 g y
for all x y and for any 0 1.

for all x y and for any 0 1.

A function is weakly concave if we have in the above expression.

A function is weakly convex if we have in the above expression.

25

26

Example 1
f x

Example 2
f x

f x x 4

f x x 8 x 16

27

28

Example 3
f x

Example 4
f ( x)

f x x 4

1 3 3 2
x x 2x
3
2

29

Concave/Convex Regions

30

Multivariate Functions

Even if a function is not entirely concave or convex

We have analogous definitions when the

functions are of more than one variable.

it still may be concave or convex over a certain

region.

The function is concave if the line connecting

any two points on the function is below the
function.

Example 4:
3
2
3
f '' x 2 x 3 0 x
2
3
3
f is concave when x and convex when x .
2
2
f '' x 2 x 3 0 x

Strictly concave if the line is strictly below the

function.
Weakly concave if the line is weakly below the
function.

For convex,
convex replace below
below with above
above .
31

32

Multivariate Functions

Second Order Conditions

Formally:

We can generalize our test for functions of

more than one variable using the Hessian
matrix.
matrix

A function g x is concave if

g x 1 y g x 1 g y

for all x y and for anyy 0 1.

Matrix of all second order partials.

A function g x is convex if

g x 1 y g x 1 g y

f all
for
ll x y andd for
f any 0 1.
1

33

Example

Hessian

f x1 , x2 x12 4 x22 kx1 x2

f x f x1 , x2 , , xn
H f '' x fij x nn
f11 x
f x

H 21

f n1 x

f12 x

f 22 x

fn2 x

34

f1n x
f2n x

f nn x
35

f1

f
2 x1 kx2 ;
x1

f2

f
8 x2 kx2
x2

f11

2 f
2;
x12

f 21

2 f
k
x2 x1

f12

2 f
k;
x1x2

f
H 11
f 21

f 22

f12 2 k

f 22 k 8

2 f
8
x22

36

Unconstrained Problems

Leading Principal Minors of H

Were
We re going to use the Hessians
Hessian s leading
principal minors to express the second
order conditions.
conditions

f11 x
f x

H 21

f n1 x

Determinants starting from the top left

element of H and gradually expanding to
cover the entire matrix.

f12 x

f 22 x

fn2 x

f1n x
f2n x

f nn x

37

Leading Principal Minors of H

f11 x

f x
H 21

f n1 x

f12 x
f 22 x

fn2 x

38

Leading Principal Minors of H

f11 x

f 21 x
H

f n1 x

f1n x

f2n x

f nn x

39

f12 x

f 22 x

fn2 x

f1n x

f2n x

f nn x

40

Leading Principal Minors of H

f11 x

f 21 x

f n1 x

f12 x

f 22 x

fn2 x

Leading Principal Minors of H

f11 x
f x
Dr x 21

f r1 x

f1n x

f2n x

f nn x

f12 x
f 22 x

f r1 x

f1r x
f 2r x

f rr x

r 1, 2, , n

41

42

Concave

Concave
r
1 Dr x 0

f11 x

f x
H 21

f n1 x

The sign of D1 x is negative.

The signs alternate as we increase r.

f12 x

f 22 x

fn2 x

f1n x

f2n x

f nn x

D1 x 0

H is negative definite.

43

44

Concave
f11 x

f 21 x
H

f n1 x

f12 x
f 22 x

fn2 x

Concave
f11 x
f x
H 21

f n1 x

f1n x

f2n x

f nn x

f12 x
f 22 x

fn2 x

f1n x
f 2n x

f nn x

Dn x 0 if n is
i odd.
dd

D2 x 0

Dn x 0 if n is even.
45

46

Example (continued)

Convex

f x1 , x2 x12 4 x22 kx1 x2

Dr x 0

2 k
H

k 8

The sign
g of each Dr x is ppositive.

D1 2

f cannott be
b convex.

D2 2 8 k 2 16 k 2 0

H is positive definite.
definite

k 2 16
4 k 4 f is concave.
If k 4 or k 4 then
th f is
i neither
ith concave nor convex.
47

48

Convex Sets

Examples 1 and 2
S1 All real numbers x | 0 x 4

Let x and y be any two points in a set.

The set is convex if the entire line connecting x and y
is also in the set.

S1 is a convex set.
S 2 All integers x | 0 x 4
2 is in the set.
3 is in the set.
2 25 is
2.25
i not.
t
S 2 is not a convex set.

If x S and y S , then for all 0 1, x 1 y S .

49

Examples 3 and 4
S3

S4

51

50