Convex

Optimization
Chapter 1 Introduction
What, Why and How
What is convex optimization
Why study convex optimization
How to study convex optimization
What is Convex Optimization?
 Mathematical Optimization

Nonlinear Optimization

Convex Optimization

Least-squares LP
Mathematical Optimization
Convex Optimization
Least-squares
Analytical Solution of Least-
squares

f 0(x) = jjAx bjj22 = (Ax b)> (Ax b)

@f 0 (x)
@x = 2A > (Ax b) = 0

x = (A > A) 1A >b
Linear Programming (LP)
Why Study Convex Optimization?
Solving Optimization
Problems
Mathematical Optimization

Nonlinear Optimization

Convex Optimization

Least-squares LP
 Mathematical Optimization

Nonlinear Optimization

Convex Optimization

Least-squares LP

Analytical solution
Good algorithms and software
High accuracy and high reliability
Time complexity: C n 2 k

A mature technology!
 Mathematical Optimization

Nonlinear Optimization

Convex Optimization

Least-squares LP

No analytical solution
Algorithms and software
Reliable and efficient
Time complexity: C n 2 m

Also a mature technology!

 Mathematical Optimization

Nonlinear Optimization

Convex Optimization

Least-squares LP

No analytical solution
Algorithms and software
Reliable and efficient
Time complexity (roughly)
max{n 3 , n 2 m, F }

Almost a mature technology!

 Mathematical Optimization

Nonlinear Optimization

Convex Optimization

Least-squares LP

Sadly, no effective methods to solve

Only approaches with some compromise
Local optimization: more art than technology
Global optimization: greatly compromised efficiency
Help from convex optimization
1) Initialization 2) Heuristics 3) Bounds

Far from a technology! (something to avoid)

Why Study Convex
Optimization

Ifnot,
thereislittlechanceyoucansolveit.

-- Section 1.3.2, p8, Convex Optimization

How to Study Convex Optimization?
Two Directions
As potential users of convex optimization

As researchers developing convex

programming algorithms
Recognizing least-squares
problems
Straightforward: verify
the objective to be a quadratic function
the quadratic form is positive semidefinite
Standard techniques increase flexibility
Weighted least-squares

Regularized least-squares
Recognizing LP problems

Example: ti = jr i j
Sum of residuals approximation

Chebyshev or minimax approximation

t = maxi ja>
i x bi j
Recognizing Convex
Optimization Problems
An Example
 10
Adding Cm linear constraints?????
P 10 1
P m
p
k =1 j k
2
p
j =1 j 8f j 1; j 2; ; j 10g
Summary
From the book, we expect to learn
To recognize convex optimization problems
To formulate convex optimization problems
To (know what can) solve them!