Inequality constrained minimization: log-barrier method

We wish to solve

min cT x

subject to

Ax b

with n = 50 and m = 100.

We use the barrier method with logarithmic barrier function
(x) =

m
X

log((aTi x bi ))

i=1

and solve a sequence of smooth unconstrained problems

x (t) = argminxRn

tcT x + (x)

Objective functional augmented with the log-barrier

function [f,g,H] = objective_barrier(t,x,A,b,c);
[m,n] = size(A);
d = A*x - b; D = diag(1./d);
f = t*c*x - log(-d)*ones(m,1);
g = t*c - A*D*ones(m,1);
H = A*D^2*A;
Problem parameters
m = 100; n = 50;
ALPHA = .1; BETA = .7;
mu = 50;
A = randn(m,n);
b = 1 + abs(randn(m,1));
c = randn(n,1);
Smooth unconstrained minimization
Start with strictly feasible point x = 0
Terminates when t = 108 (Duality gap 106 )
Centering uses Newton method with backtracking

x = zeros(n,1);
t = 1;
histobj=[];
NTTOL = 1e-10;
MAXITERS = 500;

% stop inner iteration if lambda^2/2 < NTTOL

while (t <= 1e8), % Outer loop

niter = 0;
for k=1:MAXITERS,

% Inner loop

[val,g,H] = objective_barrier(t,x,A,b,c);
v = -H\g;
lambda = g*v;

% Newton step
% Newton decrement

% Perform backtracking line search along search direction

s = 1;
while (min([b - A*(x+s*v) ]) < 0),
s = BETA*s;
end % first get feasible point ... then search minimum
while (objective_barrier_val(t,x+s*v,A,b,c) > val + ALPHA*s*lambda),
s = BETA*s;
end
x = x+s*v;
niter = niter + 1;
% Test if optimum achieved
if (abs(lambda/2) < NTTOL),
break;
end

% decrement smaller than NTTOL?

end
% Display progress
obj = c*x;
histobj=[[histobj],[obj;niter;m/t]]; % Bookkeeeping
disp([obj: ,num2str(obj,%1.6e),; PDGap: , num2str(m/t,%1.2e), ;
number iterations: ,int2str(niter)]);
% Update
t = mu*t;
end

Plot results (more Matlab commands to produce the nice plot below)
PDGap = histobj(3,:); niter = histobj(2,:);
total_iter = cumsum(niter);
figure; semilogy(total_iter,PDgap,*);

10

mu = 2
mu = 50
mu = 150

10

10

Duality gap

10

10

10

10

10

10

10

20

40

60

80
100
120
140
Total Number of iterations

160

180

200

Figure 1: Plot of duality gap vs. total number of Newton iterations for = 2, 50, 150

160

140

Total Number of iterations

120

100

80

60

40

20

20

40

60

80

100
mu

120

140

160

180

200

Figure 2: Trade-off between and the total number of Newton iterations needed to reduce the duality
gap from 100 to 104 . The optimization problem is a moderately small inequality constrained LP,
just as before. This shows that the method is not very sensitive to the value of provided 10

Figure 11.7 Progress of barrier method for three randomly generated standard form LPs of different dimensions, showing duality gap versus cumulative number of Newton steps. The number of variables in each problem is
n = 2m. Here too we see approximately linear convergence of the duality
gap, with a slight increase in the number of Newton steps required for the
larger problems.
The following figures are taken from our textbook (Boyd and Vandenberghe).

35

Newton iterations

PSfrag replacements
30
25
20
15 1
10

102
m

103

Figure 11.8 Average number of Newton steps required to solve 100 randomly
generated LPs of different dimensions, with n = 2m. Error bars show standard deviation, around the average value, for each value of m. The growth
in the number of Newton steps required, as the problem dimensions range
over a 100 : 1 ratio, is very small.

PSfrag replacements
602

11

Interior-point methods

duality gap
102

duality gap

100
102
104
106
0

PSfrag replacements

= 50 = 200
20

60
40
Newton iterations

=2
80

Figure 11.15 Progress of barrier method for an SOCP, showing duality gap
versus cumulative number of Newton steps.

Newton iterations

11.6
with
generalized
inequalities
with xProblems
R50 , m
= 50,
and Ai
R550 . The problem instance was randomly
generated, in such a way that the problem is strictly primal and dual feasible, and
has optimal value p! = 1. We start with a point x(0) on the central path, with a
duality gap of 140
100.
The barrier method is used to solve the problem, using the barrier function
120
m
!
"
#
100 (x) =
log (cTi x + di )2 #Ai x + bi #22 .
80

i=1

The centering problems are solved using Newtons method, with the same algorithm
60
parameters as in the examples of 11.3.2: backtracking parameters = 0.01, =
2
5
0.5, and a stopping
40 criterion (x) /2 10 .
Figure 11.15 shows the duality gap versus cumulative number of Newton steps.
20similar to those for linear and geometric programming, shown
The plot is very
in figures 11.4 and 11.6, respectively. We see an approximately constant number
0
of Newton steps required
per centering
step,
approximately
linear
80
120and therefore
0
40
160
200

convergence of the duality gap. For this example, too, the choice of has little
choice steps,
of the provided
parameter,isfor
small10
SOCP.
effectFigure
on the11.16
totalTrade-off
number in
ofthe
Newton
ataleast
or so. As in
The
vertical
axis
shows
the
total
number
of
Newton
steps
required
to
reduce
the examples for linear and geometric
programming, a reasonable choice of is in
the duality gap from 100 to 103 , and the horizontal axis shows .
the range 10 100, which results in a total number of Newton steps around 30 (see
figure 11.16).
with
variable
A small
SDPx R100 , and Fi S100 , G S100 . The problem instance was
generated randomly, in such a way that the problem is strictly primal and dual
Our next
example
is an
feasible,
with
p! = 1.
TheSDP
initial point is on the central path, with a duality gap
of 100.
T
minimize c$
x
We apply the barrier method with logarithmic
barrier function
(11.46)
n

603