Chapter 19

BARRIER FUNCTION METHODS

1.

Introduction

Penalty function methods generate a sequence of infeasible points {xk) which

come closer to the constraints as the iterations proceed. By contrast, barrier
function methods - which can only be applied to problems with inequalities but
no equality constraints - generate points which lie inside the feasible region.
Hence, for the remainder of the chapter, we consider the problem

2.

Minimize F ( x )

(19.1.1)

subject to ci(x)2 0 ( i = 1 , ...,m )

(19.1.2)

Barrier functions

Definition One form of barrier function for the problem (19.1.I), (19.1.2) is

Because the barrier term includes reciprocals of the constraints we see that B is
very much greater than F when any ci(x) is near zero - i.e. when x is near the
boundary of the feasible region. Similarly, B = F when all the q ( x ) are much
greater than zero and x is in the interior of the feasible region.

Definition A second (more popular) barrier function for (19.1.1), (19.1.2) is

m

B(x, r) = F (x) - r

log(cj (x)).

(19.2.2)

i= 1

When 1 > ci(x) > 0 then log(ci(x) < 0. Hence the second term on the right
of (19.2.2) implies B >> F when any of the constraint functions approaches
zero. Note, however, that (19.2.2) is undefined when any cj(x) is negative.
There is a relationship, similar to that for penalty functions, between unconstrained minima of B(x, r) and the solution if (19.1. I), (19.1.2).

Proposition Suppose that (19.1. l), (19.1.2) has a unique solution x,* ,A*. Suppose also that p is a positive constant and, for all rk < p, the Hessian matrix
v 2 ~ ( x , r kof) the barrier functions (19.2.1) or (19.2.2) is positive definite for
all x. If xk denotes the solution of the unconstrained problem
Minimize B(x, rk)
then
xk

+ X*

as rk

(19.2.3)

0.

Moreover, if B is defined by (19.2. l),

and if B is defined by (19.2.2),

We omit the main part of the proof of this result. However it is easy to justify
( 1 9.2.4) because differentiating (19.2.1) gives

By comparing (19.2.6) with the Lagrangian stationarity condition (12.2.2) as

rk + 0 we deduce (19.2.4). A similar argument justifies (19.2.5).
This proposition is the basis of the B-SUMT algorithm, stated below. (A fuller
theoretical background can be found in [42].) The convergence test for the
algorithm is based on satisfying the complementarity condition (17.2.4), using
the estimated Lagrange multipliers implied by (19.2.4) or (19.2.5). B-SUMT
can often be used successfully, in practice, for problems of the form (19.1. I),
(19.1.2) even when the conditions in the proposition cannot be verified.

Barrier function methods

Barrier Function SUMT (B-SUMT)

Choose an initial guessed solution xo
Choose a penalty parameter rl and a constant P(< 1 )
Repeat f o r k = 1,2, ...
starting from xk-1 use an iterative method to find xk to solve (19.2.3)
set rk+l = prk
if B is defined by (19.2.I ) then

A.-Ik
for
1

ci (

~ k ) ~

i = l,..,m

else, if B is defined by (19.2.2)then

2.-

1 -

ci (xk)

for i = l , . . , m

until k l c l ( x k ) .,., kmc, ( x k )are all sufficiently small.

Exercise
Obtain expressions for the Hessian matrices of barrier functions (19.2.I ) , (19.2.2).
Hence find an expression for the Newton search direction for (19.2.2). How
' available?
could this expression be modified if ( v 2 ~ ( x ) ) -were

A worked example
As an example of the use of the log-barrier function we consider the problem
Minimize F ( x ) =
subject to cl ( x ) = xl

4+ 3xi

+ 5x2

(1 9.2.7)

> 0.

(19.2.8)

The corresponding barrier function is

B(x,r ) = x;

+3 4

rlog(xl

+ 5x2 - 1 )

and hence the minimum of B(x,r ) satisfies

Eliminating the term involving r between these two equations we get

5
x2 = - X I ;
3

(19.2.11)

and substitution in (19.2.9) gives

so that

Using (19.2.1 1) we get

In these expressions the quantity under the square root is greater than 4 when
r > 0. Hence (19.2.12) gives one positive and one negative value for XI. But
(19.2.11) means that x2 must have the same sign as xl. However, a solution
with both xl and x2 negative cannot satisfy (19.2.8). Therefore the unconstrained minimum of B(x, r ) is at

&.

These values satisfy the

Hence, as r + 0 we have xl -+ and x2 4
optimality conditions for problem (19.2.7), (19.2.8).

Exercises
1. Deduce the Lagrange multiplier for the worked example above.
2. Use a log-barrier function approach to solve the problem
Minimize xl

+ x2

subject to x;

+x; 2 2

3. A log-barrier approach is used to solve the problem

Minimize

- rTy subject to yTQy 5 V,.

Suppose that the barrier parameter r is chosen so the minimum of B(y, r ) occurs where yTQy = kV,, where k < 1. Obtain an expression for y(r) which
minimizes the barrier function and hence find r in terms of T , Q, and V,.
4. Solve

Minimize

XI+

2x2 subject to x1

using the barrier function (1 9.2.1).

> 0, x2 > 1

Barrier function methods

3.

Numerical results with B-SUMT

We use B-SUMT to denote the SAMPO implementation of the barrier SUMT

algorithm using the log-barrier function (1 9.2.2). In B-SUMT the unconstrained
minimizations are done by QNw or QNp.
A safeguard is needed in the line search for the unconstrained minimization
technique used in B-SUMT. The log-barrier function is undefined if any of the
constraints ci ( x ) are non-positive and therefore the line search must reject trial
points where this occurs. This can be done within the framework of the Armijo
line-search by re-setting B(x, r ) to a very large value at any point x which violates one or more of the constraints.

In this section we consider the minimum-risk and maximum-return problems

in the forms Minrisk4 and Maxret4. Program sample11 allows us to solve
such problems using B-SUMT and we begin with Problem T l lb. We note first
that B-SUMT must be started with a feasible point. Hence the automatic initial
guess yi = l l n , i = 1 , . ..,n (suitable for the other two SUMT algorithms) will
probably not be appropriate for B-SUMT.In the present case, it is fairly easy to
find a feasible starting guess. Since we have the expected return for each asset
(4.4.4),we can set y3 = 0.95 and yi = 0.01, i = 1,2,4,5. This ensures C y i < 1
and also that the expected portfolio return exceeds R p ( l%). The progress made
by B-SUMT is shown in Table 19.1. For comparison, this table also summarises
the behaviour of P-SUMT from the same starting point. In both cases the unconstrained minimizer was QNw and the initial penalty parameter and rate of
reduction were given by ro = 0.1, P = 0.25.
B-SUMT

P-SUMT

Table 19.1. B-SUMT and P-SUMT solutions to problem Tl l b

The fact that P-SUMT converges much more quickly than B-SUMT is largely due

to the fact that the minimum of B(x,O. 1) is very much further from the optimum than the minimum of P(x, 0.1). For this problem, therefore, the initial
choice rl = 0.1 is a bad one for the barrier approach.
The solution to problem TI l b obtained with B-SUMT is approximately

with V w 1.083 and R w 1.044%. The solution from P-SUMT is

giving V w 1.083 and R w 1.026%. These both lie between the two solutions
(17.5.5) and (17.5.6). Both solutions are both equally valid because they are
feasible points giving the same value of the objective function V. The fact that
B-SUMT and P-SUMT terminate at different points seems to be due to the fact
that one works inside the feasible region while the other operates outside.
We now apply B-SUMT to the maximum-return problem T13b. We need to
find a feasible starting values for the yi, which is not so straightforward as
it was for Minrisk4. However, if we have already solved Minrisk4 then its
solution will be appropriate as an initial guess for Maxret4 provided it gives
V < V,. Alternatively, if we first solve MinriskO then we must get a solution
with V < V, (unless V, has been chosen less than Vmi,so that no portfolio with
the given data yields an acceptable risk.)
Since Problem T13b is based on the same asset data as Problem 1l b we can
use the feasible solution from B-SUMT as a starting point. Table 19.2 compares
the rates of convergence of B-SUMT and P-SUMT.
B - SUMT

P-SUMT

Table 19.2. B-SUMT and P-SUMT solutions to problem T13b

In this case the behaviour of the barrier approach is comparable with that of the
penalty method. The initial unconstrained minima of B(x, 0.1) and P(x, 0.1)

217

Barrier function methods

are at about the same distance from the true solution and hence B-SUMT and
P-SUMT need about the same number of QNw iterations to converge. B-SUMT
needs more function calls, however, because of the extra restrictions on step
length mentioned at the beginning of this section. Since problem T13b has a
unique solution, both methods converge to the same result.
Table 19.3 gives a comparison between the SUMT and SQP methods on problems T l l b - T14b.
Method

Tllb

T12b

T13b

T14b

Table 19.3. Pcrfromance of SUMT and SQP on problems Tl 1b-T14b

Because B-SUMT is included, the counts of iterations and function values are
based on a (different) feasible starting guess for each problem and so the entries
for P-SUMT,AL-SUMT and AL-SQP differ from those in Table 18.4. Specifically,
these starting guesses are as follows:
For problem T I lb: y, = 0.01 for i = l,2,4,.5; y3 = 0.95
ForproblemT12b: yi = 0.1 for i = 2 , 3 , 5 , 8 , 9 , l O ; yi = 0.001 for i = 1 , 4 , 6 , 7
For problem T13b: y is given by the feasible solution to problem T11b.
For problem T14b: y is given by the feasible solution to problem T12b.
We can see that the barrier function approach is usually the least competitive
of the SUMT methods. Hence, in the form described in this chapter, its use is
normally confined to those problems where the function cannot be calculated at
some infeasible points. A simple example would be if the expression for F(x)
included terms involving fil,
..,fi,,
which are non-computable if the any of
0 are violated. In such situations it is important
the constraints xl 2 0, ..,x,
to use a method whose iterates stay inside the constraint boundaries.

>

In spite of the relatively poor performance of B-SUMT, the ideas behind the
method are important because they are the foundation for the interior point
methods described in the next chapter.

Exercises (To be solved using sample 11 or other suitable software. In particular, B-SUMT can be implemented by applying SOLVER to minimize B(x,r)
for a sequence of values of r.)
1. Apply B-SUMT to problem T l l b using other feasible starting guesses such
=~0.0001, i = 2, ...,5. Compare performance with that of
as yl = 0 . 9 9 , ~

AL-SUMT from the same initial guess. Can you find a starting guess from which
B-SUMT converges to the same solution as AL-SUMT?
2. By performing numerical experiments using problems TI l b and T13b, investigate how the speed of convergence of B-SUMT varies with the choice of
initial parameter rl .

3. Use problems T2b and T4b to compare the performance of the log-barrier
and reciprocal-barrier forms of the B-SUMT algorithm.
4. Consider the Lagrange multiplier estimates provided by B-SUMT, P-SUMT
and AL-SUMT at the solutions to TI l b and T13b and comment on any differences you observe.