604

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 3, MAY 2013

Iterative Learning Control With Mixed Constraints

for Point-to-Point Tracking
Chris T. Freeman and Ying Tan

AbstractIterative learning control (ILC) is concerned with

tracking a reference trajectory defined over a finite time duration,
and is applied to systems which perform this action repeatedly.
However, in many application domains the output is not critical
at all points over the task duration. In this paper the facility to
track an arbitrary subset of points is therefore introduced, and the
additional flexibility this brings is used to address other control
objectives in the framework of iterative learning. These comprise
hard and soft constraints involving the system input, output and
states. Experimental results using a robotic arm confirm that
embedding constraints in the ILC framework leads to superior
performance than can be obtained using standard ILC and an a
priori specified reference.
Index TermsIterative learning control (ILC), iterative
methods, learning control systems, linear systems, motion control,
optimization methods, robot motion, test facilities.

I. INTRODUCTION

TERATIVE learning control (ILC) is a methodology applicable to systems which repeatedly track a reference,
,
defined over a finite interval
. The aim is to use past
experience to sequentially improve tracking performance over
repeated trials of the task. Over the last 25 years it has been
an area of intense research interest in both theoretical and application domains, for recent overviews of the literature see [1]
and [2]. However, rather than follow a motion profile defined at
all points, in many applications the system output is only critical at a finite set of prescribed time instants. Examples include
production line automation, crane control, satellite positioning,
and robotic pick and place tasks in which the critical points
correspond to the location of the payloads. Furthermore, ILC
has recently been used to great effect within stroke rehabilitation [3], where motion control is naturally specified in terms of a
point-to-point optimization problem in order to correspond with
results from human motor learning [4].
The standard ILC framework is able to tackle the
point-to-point problem simply by employing an arbitrary
Manuscript received August 11, 2011; revised November 17, 2011; accepted
January 22, 2012. Manuscript received in final form February 08, 2012. Date
of publication March 14, 2012; date of current version nulldate. This work was
supported by Australian Research Council Future Fellow Grant FT0991385.
Recommended by Associate Editor S. S. Saab.
C. T. Freeman is with the School of Electronics and Computer Science, University of Southampton, Southampton SO17 1BJ, U.K. (e-mail:
cf@ecs.soton.ac.uk).
Y. Tan is with the Electrical and Electronic Engineering Department, University of Melbourne, Parkville VIC 3010, Australia (e-mail: yingt@unimelb.edu.
au).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TCST.2012.2187787

reference,
, which passes through the desired points.
However superior results follow if this is coupled with strategies such as Input Shaping in order to suppress vibrations that
occur between the critical points. This approach is taken in [5]
for a high-acceleration positioning table. An alternative is to
and to employ
use a simpler feedback controller to track
ILC to update parameters within the input shaping filter applied
to the reference, as proposed by [6] for control of an industrial
robot. Another approach is to develop ILC algorithms which
have two separate components; one which ensures tracking of
, and another which reduces the amplitude of residual
vibrations occurring after the point-to-point location is reached
[7]. The drawback to all these methods is that they fail to utilize
the extra freedom available in ILC design to address additional
is designed a
performance demands. Furthermore, if
priori to meet such performance objectives, these will not be
met in practice due to the presence of model uncertainty and
noise.
Other approaches to point-to-point motion control have
broken away from the standard ILC framework of tracking a
, but have only constatic reference defined over
sidered the case where a specified position must be reached at
time
, as in [8][13], or the case of a movement between
two equilibrium points [14]. While these approaches dispense
with tracking unnecessary output points, they do not use the
resulting freedom to tackle additional performance objectives
which may be of critical concern. A further limitation is that
they only consider a single point-to-point movement, rather
than a sequence of actions needed to build up complex movements, such as is required in robotic automation and production
line assembly.
This paper addresses current drawbacks by providing
a framework that can deal with an arbitrary number of
point-to-point movements, while also addressing a general
form of performance objective which encompasses a wide
variety of practical performance concerns. Along with soft
performance constraints, this includes hard constraints which
are needed to address actuator saturation, physical workspace
limitations, or imposed safety restrictions. This framework
significantly increases the flexibility and functionality of
point-to-point ILC compared with approaches currently available. Moreover, the action of embedding both the performance
and tracking objectives within the framework of iterative
learning yields algorithms which are capable of reaching optimal solutions in the presence of model uncertainty and noise.
To achieve this, ILC is employed as an iterative optimization
paradigm which uses experimental data to tackle a general form
of cost function involving the input, output and states. A similar

1063-6536/$31.00 2012 IEEE

FREEMAN AND TAN: ILC WITH MIXED CONSTRAINTS FOR POINT-TO-POINT TRACKING

approach has recently been applied to constrained ILC in [15],

which addresses both soft and hard constraints, but without
explicit reference to, or analysis of, the point-to-point tracking
problem. In addition, the current papers focus on point-to-point
tracking requires it to be embedded in the form of an additional
equality constraint, in order that the soft constraints may not be
allowed to degrade tracking at the critical time locations. Such
separation is not addressed in [15], with tracking and additional
constraints being combined in the same soft constraint.
The paper is organized as follows. Section II introduces
and motivates the point-to-point tracking control problem.
Section III develops gradient descent ILC laws which address
both hard and mixed constraints. In Section IV, ILC algorithms
based on the Newton method are presented. Experimental
results are provided in Section V and conclusions and future
work are given in Section VI.
II. PROBLEM FORMULATION
Denote the set of real numbers as , and the set of integers
as . The symbol denotes the trial number and
. For
any vector
,
. For any matrix
,
is the induced norm of the vector norm,
denotes
the
eigenvalue of ,
is the minimum singular value
of ,
is the maximum singular value of , and
is the spectral radius of . The notation
is
the pseudoinverse of and
. The notation
is the orthogonal projection onto the nullspace
of . The
identity and zero matrices are denoted by
and , respectively.
In order to simplify presentation, the following linear timeinvariant (LTI) system is considered:

(1)
defined over the finite time interval
where the number of samples
. Here
,
,
are the state, input and output vectors,
respectively, and the input and output sequences are given by

Remark 1: The analysis framework provided in this work

can be extended to general nonlinear time-varying discrete-time
systems with a proper linearization. With a slight modification,
similar (local) results can be obtained.
The standard ILC framework constructs a series of inputs
which drives the system to track a reference sequence

605

where
is the unknown desired input sequence corresponding
to . This leads to

Over the
trial the relationship between the input and output
time-series can be expressed by
where the
matrix is

..
.

..

(3)

..
.

Here
is the response to initial conditions whose effect can
be absorbed into the reference trajectory, so that without loss of
generality it is assumed
, or equivalently
.
For some
, an ILC update of the form
(4)
can be considered as an iterative numerical method to solve the
tracking problem, and the derivation of a suitable matrix has
been the focus of significant research effort. Since
(5)
the update (4) is convergent to a solution satisfying (2) if and
only if
(6)
The convergence speed is determined by the magnitude of
and is maximum when
.
A. Point-to-Point ILC Formulation
Now suppose that the th plant output is only required to track
a reference trajectory at a fixed number,
, of sample
instants along the trial duration. These sample instants are given
by
. To define the point-topoint tracking problem it is first necessary to remove the points
that do not need to be tracked from the original reference .
This yields a reduced reference vector
whose length
is given by
(7)
It is then necessary to define a matrix transformation
such that
. This is achieved by
first introducing a row vector
whose th element
is 1 if the th element of
is required to be tracked, and 0
otherwise. The formal definition for is

Let
and
be the input and output vectors respectively on
the th trial, with
the tracking error. Then it is
necessary to find a sequence of control inputs that satisfies
(2)

..
.

if
otherwise
where
tion. The matrix

(8)

and
denotes the floor funcis then produced as follows: 1) set
,

606

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 3, MAY 2013

2) starting at the first element, increment along the bottom row

of , and whenever a non-zero element is encountered move all
subsequent bottom row entries into a newly created bottom row
that is appended to , maintaining their position along the row
and padding the remaining entries of both rows with zeros. Formally this is defined by
if

(9)

otherwise.
, when any output vector is
As seen by the relation
pre-multiplied by , it extracts the components that correspond
to prescribed point-to-point locations, while retaining the order
in which they appear.
Remark 2: Suppose that at each point-to-point location each
component of the output is required to track a reference point,
that is if
. In this case the matrix
has a simpler form given by the block-wise components
if
,
otherwise

(10)

and the reference has the form

(11)
is the prescribed output vector at sample
,
where
and
.
ILC can be re-formulated for the point-to-point case by deriving an iterative numerical solution to the problem of finding
a control input which minimizes the point-to-point error norm.
The control objective is to find a sequence of control inputs
such that
(12)
which replaces the standard requirement (2). The ILC update
(4) now assumes the form
(13)
so that the point-to-point error evolution is
(14)
and the convergence condition (6) becomes
(15)
which guarantees zero point-to-point error.
In Sections III and IV learning operators are derived to
satisfy (15), but first further motivation is provided to support
the utility of point-to-point ILC over the standard framework.
B. Point-to-Point ILC Motivation
The first result shows point-to-point ILC can enlarge the feasible region of the solution. That is, it confirms that some problems cannot be solved by the standard ILC framework, but are
feasible for point-to-point ILC.
Theorem 1: Let denote the rank deficiency of the plant matrix (the number of linearly dependent rows). If
the

standard ILC update (4) cannot force the plant to track an arbitrary reference trajectory . The point-to-point update (13) can
enforce tracking of an arbitrary reference
if and only if the
tracked points are chosen such that
(16)
Proof: A necessary and sufficient condition for an operto exist satisfying the convergence condition (15) is that
. For the standard ILC case
,
and hence
, leading to
having eigenvalues at unity. Now the
row of
is the
row of , hence if
and the
point-to-point samples are chosen to correspond to any subset
of linearly independent rows of , the convergence condition
(15) can be satisfied. If
then the additional condition
is imposed.
Remark 3: Let system (1) be written as discrete transfer-function matrix
with component
the transfer-function linking the th output with the th
input. If the relative degree of
is
, then we have
.
The ability of point-to-point ILC to employ a modified standard reference to recover feasibility is extremely important, especially as
in practice due to the delay action of a zeroorder hold. However many tasks are naturally defined only at a
small number of points, and hence additional benefits may also
be expected by not enforcing tracking of unnecessary points.
The next lemma shows how the space of feasible inputs expands
as the number of tracked points, , reduces.
Lemma 1: Assuming
, the feasible input
space which forces the system (1) to track is of dimension
, and is given by
.
The nullspace of
has an orthogonal basis given by the rows
of
, where
is such that the
ator

is full rank.
matrix
It is next illustrated how this enlarged space of feasible inputs
can be used to increase performance. In particular, the practically relevant case is addressed in which a weighted input norm
is required to be small. However, before this can be considered
a preliminary proposition is required.
Proposition 1: Let
comprise point-to-point locations
satisfying
. Let equal but with the
row removed, and hence correspond to tracking all but the
point-to-point location. Let the
eigenvalues of the matrix
be denoted
,
which also equal the singular values since is Normal. Similarly, let the
eigenvalues of the matrix
be denoted
, which also equal
the singular values since
is Normal. Then the following
relationship holds:
(17)
In particular, let equal the th column of with the th element removed. Then if the eigenvalues of are distinct and no
eigenvector of is orthogonal to then
(18)

FREEMAN AND TAN: ILC WITH MIXED CONSTRAINTS FOR POINT-TO-POINT TRACKING

607

Proof: First note that is a Hermitian matrix of order ,

and that is a principal submatrix of of order
. Then
(17) follows as an application of Cauchys Interlace Theorem
for eigenvalues of Hermitian matrices [16]. It is further proven
in [16] that (18) holds provided: 1) the eigenvalues of satisfy
and 2) the vector

of Proposition 1 means that

increases as each
point-to-point location is removed, and hence the right-hand
side of (20) decreases.
Remark 4: If each component of the plant output is only required to track a reference position at a single sample instant,
that is
, then (20) becomes

(19)

(23)

is a unitary matrix of order

has non-zero elements, where
such that
, with
.
To satisfy 2) a suitable choice for has columns that are the
eigenvectors of , and hence
only if is orthogonal
to an eigenvector of .
With the help of Proposition 1, the following result is obtained.
Theorem 2: Consider the system (1) and a point-to-point
tracking task which has a corresponding
matrix satisfying
. There then exists an input which achieves
tracking of
and has a weighted norm with upper bound
(20)
is
whose right-hand side reduces as the number of points
reduced. Here the operator
has full rank.
Proof: Suppose the input
solves the standard tracking
problem, so that
, where
contains the desired
points,
, along with additional free components that are
not associated with the point-to-point objective. Then exchange
rows in matrix and
to group the stipulated, , and free
components, , as
(21)
where
is such that
The optimal cost associated with the problem

is full rank.

subject to
is the norm of the orthogonal projection of
of
, and it follows that:

, onto the range

This also holds if the temporal distance between point locations

exceeds the time taken for the impulse response to approximately go to zero (assuming asymptotic stability).
Theorem 2 provides an example of the benefit obtained compared with the bound
corresponding to
standard ILC (if it exists). This benefit increases as the number
of tracked points is reduced, or their temporal spacing is increased.
Reduction in the number of tracked points, , hence expands
the set of feasible inputs and enables them to be chosen to address infeasibility in the tracking of a reference defined at all
samples
along the trial, as well as additional performance objectives. In the next section these objectives will be embedded in the ILC framework to enable optimal
solutions to be arrived at in the presence of model uncertainty
and noise.
III. GRADIENT DESCENT POINT-TO-POINT ILC
The gradient descent method is one of most popular numerical algorithms used to tackle nonlinear optimization
problems, and has previously been applied to the single-input,
single-output (SISO) case within the standard ILC framework
[17]. Unlike alternative approaches, it is straightforward to
embed experimental data, directly yielding updates in the ILC
framework which, through suitable step size selection, have
favorable convergence and robustness properties that can be
manipulated in a simple and transparent manner. Motivated by
(12) and the accompanying discussion, the gradient descent
method is applied to solve
(24)

(22)
Now insert the relationship

into (22) to obtain the solution

The relationship

leads to the weighted input bound (20). It follows that the input
norm is small when point-to-point locations are selected which
maximize the smallest eigenvalue of
. Application

leading to the iterative update for the control input

(25)
where
is the gradient operator with respect to and is
a positive scalar. Note that the experimental plant output,
has replaced the nominal value,
, so that the optimization
is robustly achieved within the ILC framework.
Theorem 3: Provided the point-to-point locations are chosen
such that
, the choice of gain
(26)
guarantees convergence of the update (25) to the reference .
In particular, the maximum convergence rate corresponds to
(27)

608

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 3, MAY 2013

The convergence rate using (27) increases as the number of

point locations, , is reduced.
Proof: The convergence condition for (25) is given by
(28)
which ensures a linear convergence rate to zero error [18]. This
yields (26) since

lies in the open interval

. Note
that any gain uncertainty
can be tolerated through use
of a sufficiently small .
Remark 5: The term
in (25) can be efficiently generated using the co-state representation of system
(1). More specifically, it is equal to the output of the system

(30)
with the input
and terminal state
.
Use of (30) therefore avoids calculation of the large matrix
appearing in (25) and the algorithms which follow.
Remark 6: It is shown in [20] that the gradient point-to-point
algorithm (25) applied to a linear system always converges to a
solution which minimizes
. Hence, using Theorem 2, (25)
converges to a solution with a norm satisfying

where the inequality

since
is positive definite. The solution to
corresponds to the choice (27) and the convergence rate is

(29)
Application of Proposition 1 guarantees that each point removed
increases
and reduces
.
from
Hence the convergence rate (29) increases.
Having shown point-to-point ILC increases the convergence
rate, robustness margins are next established.
Theorem 4: Let there exist a multiplicative uncertainty on each element of the plant model
, such that
. Here
is the actual plant and the
model
corresponds to the matrix
used in the update
law (25). A sufficient condition for monotonic convergence is
that each
lies in the open interval
,
demonstrating an allowable phase margin uncertainty of 90 .
Proof: This is an extension of robustness analysis for
the standard gradient algorithm (
) in [19] for the
SISO case. Suppose that the uncertainty can be expressed in
the matrix form
, and that point locations are such
that
. Then using (25) the point-to-point error
satisfies

(31)
whose upper bound decreases as the number of tracked points
is reduced.
A. Inequality Constrained Gradient Point-to-Point ILC
Consider vector inequality constraints on the system input of
the form
, where
and
, where is
the number of imposed constraints. The point-to-point problem
(24) now becomes
subject to

(32)

This can be tackled using an interior-point approach to inequality constrained minimization, termed the barrier function
[21]. A logarithmic barrier function is employed, producing the
auxiliary problem
(33)
where

where
. If is positive, the first term on the right-hand
side is strictly positive for an arbitrary non-zero
and
,
and of
. Similarly the second term is of
and strictly
negative, and hence there always exists a
which ensures monotonic reduction in error norm. This also holds if the
components of
are reordered so that the elements corresponding to the same input are grouped, resulting in a reordering
of the matrix such that
. The stipulation that the components of associated with the same input
have the same uncertainty then results in
having the block
diagonal structure
, where
corresponds to the th input. A sufficient condition for to be positive definite is that each
is positive definite. This is the same
condition as that given in [19] which goes on to show that a
sufficient condition is that each
is positive-real. Therefore a sufficient condition for monotonic convergence is that

, are the
rows of , , respectively. The scalar
is used to weight the action of the barrier and should
be gradually increased to result in a solution which satisfies the
tracking requirement. The solution via the gradient method is
(34)

are given by
,
where the elements of
and
is the value of on trial .
With appropriately chosen scalars
and , this converges
to the zero error solution as
provided there exists
which satisfies
[21], [22]. In the context
of ILC the increase in must not be too rapid in order to ensure
that the barrier component effectively engages with the ILC update. Conversely it must be fast enough not to significantly reduce overall convergence speed. The selection of and must
therefore ensure:
1) the input (34) remains feasible (
);

FREEMAN AND TAN: ILC WITH MIXED CONSTRAINTS FOR POINT-TO-POINT TRACKING

2) the constraint term in (34) comprises a significant proportion of the input over samples which are required to adapt
to the imposed hard constraint;
3) 1) and 2) are satisfied without reducing .
It follows that an appropriate update strategy is to select the
highest value of that results in a feasible input without the
constraint term, that is, on trial choose a value
which satisfies
s.t.
and then update

(35)

609

The equality constraint reflects the fact that the point-to-point

tracking requirement is an essential element of the task (e.g.,
a robot must reach the required positions during the assembly
task, whilst the soft constraint is merely desirable).
To remove the equality constraint express the vector quantity
in terms of the input vector, , using
(38)
in which

according to
if
otherwise

..
.

(36)

and use in the update (34). The multiplier

is chosen
as a compromise between convergence to the hard constraint,
and robustness [21]. By satisfying the hard constraint and ensuring
is updated slow enough to engage with the ILC update, (34) converges to a input which solves the tracking requirement as
, provided such an input exists. If it does
not exist the procedure converges to a local minimizer of (33).
In practice the designer can gain insight into the feasibility of
the problem through simulations using the nominal plant model.
Note that the simple structure of the gradient ILC update allows
transparent control over convergence in practical conditions involving model uncertainty and noise that is not possible using
many of the alternative inequality constrained minimization approaches available.
Remark 7: The convergence of iterative algorithm (34) with
respect to the given constrained optimization problem (32) belongs to a class which has been extensively studied in the optimization literature, see, for example, [22], and hence the proof
of convergence is omitted.

..
.

..
.

..

..
.

(39)
and
.
For notational simplicity, and without loss of generality, it is
assumed that
.
Now take as any solution satisfying
and
. Providing it is feasible, that is
, such an input is found
in practice through application of the approach of Section III-B.
Also introduce
as a matrix with columns
that form an orthogonal basis of the nullspace of
. From
Lemma 1, a suitable candidate is

Denote
, then the minimization (37)
is equivalent to the inequality constrained problem
s.t.
(40)
with

. The solution is then

(41)

B. Incorporation of Additional Objectives

Having achieved point-to-point tracking with inequality constraints on the input, a wide range of other performance indices that are important in practice can be addressed. These may
comprise reducing the input or output energy, or reducing the
output derivative at critical times to provide smoother movements. Theorem 2 illustrates how the expanded feasibility space
provides scope to achieve such objectives. Consider the general
case of minimizing a linear function,
, of the input,
output, and states. If the point-to-point tracking requirement is
expressed as an equality constraint, the problem can be written
as
subject to

This has split the solution into components and that minimize the soft and tracking constraints, respectively. The use
of means that updating does not affect the plant output at
the point-to-point locations which have already been forced to
follow the prescribed reference. Applying the barrier function
method to solve (40) yields the auxiliary problem

This has iterative solution via the gradient method

(37)

is a weighting matrix. It is
where
worthwhile highlighting that the formulation of point-to-point
tracking as an equality constraint that must be satisfied at each
iteration is a much stronger performance requirement compared
with that of standard ILC. Moreover, it is a requirement that
must be satisfied in the face of additional performance demands.

(42)
are given by
where the elements of
. With suitable updating of the step-sizes
and
, (42) is guaranteed to reach a local solution of
(37). As discussed in Section III-B, for the barrier function to

610

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 3, MAY 2013

effectively engage with the minimization component of (42)

is chosen to satisfy

s.t.

on trial and then (36) is applied to update . Convergence

properties of the barrier method have been extensively studied,
see, for example [22], and therefore a convergence proof for
update (42) is omitted here.
The equality constraint in (37) ensures minimization of the
soft objective does not conflict with the point-to-point tracking
task. However in practice
must also be updated to ensure it
continues to be satisfied in the presence of model uncertainty
and noise. This is done following the update of
and hence
it must ensure the plant input
satisfies
, and hence is given by
subject to

(43)

with the resulting update

initial input. However the presence of soft constraints influences

the action of the hard constraint on the point-to-point tracking
component, and may mean convergence to zero point-to-point
tracking error is no longer achieved.
In order to illustrate how to convert some performance indices into the standard formulation in (37), two examples are
provided.
1) Example 1Derivative Constraints: It is often desired
that the plant output velocity be zero at certain time instants. In
many applications this is important for vibration suppression,
or to ensure the system is momentarily stationary in order to
carry out a task (such as picking up or placing a component
on a manufacturing line). In addition, constraints on the input
velocity are useful for reducing actuator wear. This leads to (37)
becoming
subject to

(48)

where the diagonal matrices

,
select the points at which the respective derivatives
are required to be zero. Since
, where
is the differential operator of appropriate dimension, this
gives
, and (45b) becomes

(44)
where the elements of
are given by
. To ensure that the step-sizes
and
engage productively, the procedure (35), (36) is again
employed, now with elements
.
The final update sequence on each trial is
(45a)
(45b)
(45c)
is the experimentally obtained performance function
where
.
Remark 8: In the absence of inequality constraints the soft
constraints reach a global minimum with convergence criterion

(49)
In the case where vibrations are suppressed at the point-to-point
locations,
is given by
, where is defined in (8).
2) Example 2Energy Constraints: Suppose instead a combination of the input and output signal norms are required to be
minimal, giving rise to the cost
subject to
(50)
This form of constraint may be used to reduce either the input
or output norm, by setting the other weight to zero. This may
lead to an excessively impulsive action, however, which can be
addressed by instead using a small, non-zero value multiplied
by the identity matrix (or alternatively minimizing the output
derivative). The cost (50) corresponds to
and
. This gives
, and the
update (45b) becomes

(46)
(51)

which corresponds to the choice of step-size

(47)
This guarantees existence of a solution
provided that
is full, which requires
. In the
absence of inequality constraints the input to the problem (37)
converges to a solution satisfying (20) with the substitution
. The upper bound reduces as points are removed from the
tracking task.
Remark 9: Rather than using the update of Section III-B to
initially solve the equality constraint, the updates (45a)(45c)
may be applied directly to solve (37) starting from an arbitrary

within the ILC framework. Here the signals can be read directly
or observed using a suitable estimator.
IV. NEWTON METHOD-BASED ILC
While providing a high level of robustness to plant uncertainty, the gradient descent approach has only a linear convergence rate. This section shows how the previous algorithms can
be extended to deliver quadratic convergence. Consider again
the point-to-point tracking problem
(52)

FREEMAN AND TAN: ILC WITH MIXED CONSTRAINTS FOR POINT-TO-POINT TRACKING

and now apply the Newton method [18] to solve it, yielding

611

and the necessary and sufficient convergent criterion for Newton

method based point-to-point ILC (54) is replaced by
(59)

(53)
where
is the Hessian matrix. From (15) the necessary and sufficient condition for convergence to zero error in a
single trial is now

Proof: Application of
produces the signal

cycles of (57) to the plant matrix

(60)

(54)
which is satisfied if and only if
. Since its computation involves inverse and derivative operations, the update
(53) is difficult to implement, especially for large values of . It
may contain excessive amplitudes and high frequencies which
increase learning transients, and, depending on point-to-point
locations, it may be singular. However, it is shown in [23] that
is the solution, , to
subject to

(55)

where
operator which replaces

. Since
in (53) is

, the resulting

The argument of the convergence criteria with this value substituted is

which is further shown to be equal to the solution to

(56)
via the unconstrained gradient descent method of Section III
which always yields the minimum input energy solution. Substituting
and
in (24) and (25) yields
an update of
(57)
which converges to the required solution provided (26) is satisfied with the scalar gain . Between trials and
of
ILC, some techniques: updates (
) of (57) are applied
in simulation to the plant
, to yield a suitable approximation to
. The number of inter-trial updates is
chosen to affect a compromise between excessively high amplitudes/frequencies in the update, robustness, and subsequent
performance. The total update sequence is as follows.

(61)
If this relation is applied

times, (61) simplifies to

This corresponds to the convergence rate (58), and directly

yields the convergence criterion given by (59).
The necessary and sufficient convergence criterion for the
gradient algorithm (25) is given by (28), and is satisfied with
a scalar gain, , satisfying (26). Since
(62)

Algorithm 1
(a) apply input
to the real plant and record output
(b) solve (56) through repeated application of (57) to the
plant model
to obtain a suitable approximation to
(c) use the resulting input to form the next descent direction
in the Newton update (53). Go to (a)
The next theorem establishes how the number of inter-trial
updates influences the convergence rate of the overall ILC law.
Theorem 5: For some
, if inter-trials updates of (57)
are performed, the error evolution is given by
(58)

the faster Newton based method is guaranteed to converge if

the gradient method is convergent, with a rate that increases by
the power . Hence increasing the number of inter-trial updates
provides a smooth transition between the convergence rate of
the gradient algorithm (28), and the more rapid convergence rate
of the Newton method based algorithm (54). In particular, the
choice of
means that the two algorithms are equivalent.
In order to approximate the Hessian matrix term in (53) a significant number of inter-trial updates may be needed. Each however can be implemented using a state-space system of order
and hence is not computationally intensive. The parameter is
a tuning parameter chosen to affect a compromise between convergence speed, amplitude/frequency content of the input, and
resulting robustness.

612

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 3, MAY 2013

A. Inequality Constrained Newton Method-Based

Point-to-Point ILC
Consider again the case in which hard constraints alone are
required by the application, yielding the constrained problem
subject to

(63)

This can be solved via the Newton method by imposing the inequality constraint in the inter-trial calculation of the descent direction,
, given by the solution to (55). Within
this inter-trial problem, the inequality constraint translates to
and the descent direction is thus generated using

Following the procedure of Section III-C, this is equivalent to

the inequality constrained problem (40). The control input applied to the plant is
(67)
Temporarily omitting the constraint from (40), the solution
using the Newton method is
(68)
with convergence criterion
(69)

subject to

(64)

This is equivalent to applying the gradient method to solve

which is satisfied if the matrix

. The term
each trial by solving

subject to

s.t.

This is the form addressed in Section III-B, with corresponding

inter-trial update
(65)
applied in simulation to the plant model
, where the elements
of
are given by
.
The full update sequence is therefore as follows.

has full rank, requiring

is calculated between
(70)

via the gradient method. Hence to solve (40) via the Newton
method, now impose the inequality constraint on (70). In terms
of the control input, on trial
, the constraint enforces
, which, assuming
has not yet been updated,
can be written as
. In terms of the Newton
descent direction, this translates to
.
Hence, (70) becomes
s.t.

Algorithm 2
(a) apply input
to the real plant and record output
(b) between trial and
, construct suitable
approximation to
satisfying
when used in Newton update (53), through
repeated application of (65) to the simulated plant
(c) use the resulting input to form the next descent direction
in the Newton update (53). Go to (a)
The number, , of inter-trial updates is chosen heuristically
to affect a compromise between the amplitude of the descent
direction , and the overall convergence of the ILC scheme,
dictated by (58). In practice this is application specific, and is
achieved by decreasing in response to excessive amplitudes/
frequencies in the input signal, levels of fluctuation in the error
norm, and the overall convergence rate achieved.
B. Incorporation of Additional Objectives
As previously considered, having satisfied the point-to-point
tracking requirement using the algorithms of Section III-B or
Section IV-B, an additional objective function may be introduced. This is required to be minimized while continuing to satisfy the point-to-point tracking requirement with an inequality
constrained input. The problem is given by
s.t.

(66)

This is equivalent to
s.t.
with corresponding update
(71)
applied to the plant
, where the elements of
are
given by
. Similar analysis to
that used in Theorem 5 relates the number of inter-trial updates
of (71) to the convergence of the Newton update (68) whose
descent direction it approximates.
Although separation of the soft constraint and tracking error
objective is ensured by the inequality constraint in (66), in practice
must also be updated to ensure it continues to be satisfied
in the presence of model uncertainty and noise. Therefore
in
(67) is also updated using the Newton ILC update
(72)
where it has been
with the constraint
assumed that
has just been updated via (68) as discussed.
The unconstrained Newton ILC descent direction,
, in (72) is the solution, , to
s.t.

(73)

FREEMAN AND TAN: ILC WITH MIXED CONSTRAINTS FOR POINT-TO-POINT TRACKING

613

so that the corresponding required constraint is

. This gives
s.t.
which is equivalent to
s.t.

Fig. 1. Robotic manipulator system showing output angle, .

with the corresponding inter-trial gradient descent update

(74)

Using (71), the descent direction in (75) is produced after trial

by inter-trial updates of the input

applied to the simulated plant model

, where the elements of
are given by
.
The total update sequence is therefore as follows.

(76)

Algorithm 3

to the system

(a) apply input

to the real plant and record output
(b) construct a suitable approximation to the term
which satisfies
when used in Newton update (68), through repeated
application of (71)
(c) use the resulting input to form the next update (68)
(d) construct suitable approximation to
which satisfies
when used in
Newton update (72), through repeated application of (74)
(e) use the resulting input to form the next update (72)
(f) use the new
and
values to form the next
control input
using (67). Go to (a)
The number of inter-trial updates of (71) and (74) is chosen
heuristically to provide a compromise between excessive amplitudes/frequencies present in the update
, robustness, and
the subsequent convergence governed by (62). In practice is
treated as a tuning parameter which is adjusted by monitoring
plant input, output and error signals between trials.
Remark 10: Instead of initially solving the equality constraint through application of the algorithms in Section III-B or
Section IV-B, the mixed constraint updates may be applied directly to solve (66) starting from an arbitrary initial input. As in
the gradient approach, however, the soft constraints may influence the action of the hard constraint so that the point-to-point
tracking component may not be solved as accurately as when it
is tackled in the absence of the soft constraint.
Similar to Section III-C, Example 1 and Example 2 are again
used to show how to incorporate some performance indices into
the objective function (66).
1) Example 1Derivative Constraints: Again consider the
output derivative constrained problem for vibration suppression
at prescribed time instants. From Example 1 in Section III-C
and the soft constraint component (68)
becomes
(75)

2) Example 2Energy Constraints: Consider again the

mixed input and output constrained problem (Example 2 in
Section III-C) which reduces signal bounds while ensuring a
non-impulsive action. From Section II this cost corresponds to
, and the update (68) is
(77)
From (71), the descent direction in (77) is produced after trial
by inter-trial updates of the input

(78)
to the simulated system

V. EXPERIMENTAL RESULTS
The ILC approaches developed have been tested on a six degree of freedom anthropomorphic robotic arm whose five rotary
joints are composed of PowerCubes (Schunk GmbH & Co.) incorporating brushless servomotors with integrated power electronics and transmission. These communicate with a dSPACE
ds1103 control board via a CAN bus at a rate of 500 kbit/s.
Results are presented for the first joint which is aligned in the
horizontal plane as shown in Fig. 1. Each servomotor includes
cascaded current and velocity control loops, and frequency response tests have established that the linear model (79), shown
at the bottom of the next page, adequately represents the system
dynamics, with input and output in degrees. A sampling time of
200 Hz has been used in all experimental tests.

614

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 3, MAY 2013

Fig. 3. Unconstrained point-to-point compared against standard ILC with a

priori designed optimal reference (denoted standard ILC reference in Fig. 2),
for both gradient and Newton based algorithms, using optimal .

Fig. 2. Unconstrained point-to-point ILC experimental results: a) output; b)

input; and c) point-to-point tracking error. Final trial output and input are shown
. The output produced in simulation is
for the gradient update with
shown in a) and denoted standard ILC reference.

The task replicates an industrial process and comprises

moving to three angles
, at corresponding
samples
,
and
. First the unconstrained gradient approach of Section III has been applied.
The update (25) is employed for both
and
values, and error norm and tracking results are shown in Fig. 2.
Larger values of
produce overly oscillatory behavior and
ultimately error divergence. For comparison, results using the
Newton update of Section IV are also given. Here Algorithm
1 is performed using
inter-trial updates of (57) with
used to produce each Newton descent direction employed in (53). These values have been chosen heuristically to
affect a compromise between convergence and excessive input
amplitudes/frequencies which give rise to learning transients
and ultimately error divergence.
The point-to-point framework is now compared to the standard ILC framework in terms of error tracking and ability to
meet performance objectives. The unconstrained point-to-point
algorithms correspond to a minimum input energy performance
objective. Hence standard ILC implementations (
)
of both gradient and Newton-based algorithms have been conducted using a reference that is designed a priori based on the
nominal plant model, to minimize the same criterion [shown in

Fig. 2(a)]. Fig. 3 shows the corresponding point-to-point error

norm and input energy using the optimal gain choice (27). From
the input energy plot it is clear that embedding performance objectives leads to superior values since the updates reach a minimum through learning from experimental data, rather than one
purely relying on the nominal model. It is also important to note
that forcing tracking along the full trial duration also creates additional learning transients which degrade convergence as confirmed by the plot of
. These reflect the reduced convergence rate of standard gradient ILC shown in Theorem 3.
Next inequality constraints of
have
been introduced to represent actuator saturation, through
selection of
and
within (32) or (63). Results are
shown in Fig. 4 where 150 trials of both the gradient approach
of Section III-B, and the Newton approach of Section IV-B
have been performed. The gradient scheme uses (34) with
updated using (35), (36) and
. It can be seen that the
input satisfies the inequality constraints while meeting the
point-to-point tracking requirement. The inequality constrained
Newton update is implemented using Algorithm 2 with
trials of (65) using
to produce each Newton update.
These values have been chosen heuristically to provide a compromise between convergence speed and excessive amplitudes,
frequencies and learning transients. To compare point-to-point
ILC with the standard ILC framework, a reference has first been
generated by solving the constrained point-to-point tracking
problem in simulation using the nominal plant [the converged

(79)

FREEMAN AND TAN: ILC WITH MIXED CONSTRAINTS FOR POINT-TO-POINT TRACKING

615

Fig. 4. Experimental results for point-to-point ILC with hard input constraints:
a) output; b) input; and c) point-to-point tracking error. Final trial output and
. The output produced
input are shown for the gradient update with
in simulation is also shown in a) and denoted standard ILC reference. When
this is used as a reference in standard ILC, the input produced violates the hard
constrains, as shown in b).

output is shown in Fig. 4(a)]. This output is then used as the reference for standard ILC (
) applied to the experimental
system using the optimal gain choice (27). The input produced
over 150 trials is shown in Fig. 4(b) and clearly does not
satisfy the hard constraint. This confirms that the standard ILC
framework is unable to satisfy constraints since a predefined
reference is not robust to model uncertainty and noise.
Finally the case of a soft constraint in conjunction with
inequality constraints is considered. The soft constraint
comprises minimizing the output derivative over the period
, as may be required when fragile payloads or
open top containers containing liquid are handled. This corresponds to the weight
and the function
in (37) or (66). Inequality
constraints of
have also been employed
through selection of
and
. For the gradient case, updates (45a) and (45c) are used in conjunction with (49). Results
are shown in Fig. 5 using
,
. Results using the
Newton update with mixed constraints are also shown, where
Algorithm 3 of Section IV-C has been implemented using 10
iterations of (76) with
to construct the descent direction
in (75), and 20 iterations of (74) with
to approximate
the descent direction in (72). As with previous results, the
Newton approach converges to very similar input and output
signals as the gradient approach, but requires significantly
fewer trials. To facilitate comparison with the standard ILC
framework, the constrained point-to-point tracking problem
has been solved in simulation using the nominal plant [with

Fig. 5. Experimental results for point-to-point ILC with mixed constraints: a)

output; b) input; c) point-to-point tracking error; and d) derivative norm. Final
,
trial output and input are shown for the gradient update with
. The output produced in simulation is also shown in a) and denoted standard
ILC reference. When this is used as a reference in standard ILC, the input
produced violates the hard constrains, as shown in b).

converged output shown in Fig. 5(a)]. Using the optimal gain

choice (27), this output is then used as the reference over 150
experimental trials of standard ILC (
). The converged
input is shown in Fig. 5(b) and clearly violates the required
inequality constraints, again illustrating that using a predefined
reference with standard ILC framework cannot satisfy constraints in practice.
In all tests parameters are chosen to affect a compromise between convergence, input amplitudes/frequencies and learning
transients. The experimental results confirm the ability of
point-to-point ILC to robustly address both hard and soft
constraints. Results also confirm that the proposed algorithms
significantly improve on the standard ILC framework which is
not robust with respect to the imposed constraints.
VI. CONCLUSION AND FUTURE WORK
The requirement for point-to-point motion control arises in
many practical applications, including industrial automation,
robotics, and rehabilitation engineering. However, there are no
available approaches to address general point-to-point tasks and
performance objectives in a framework which uses learning to
attain optimal solutions in the presence of model uncertainty
and noise. This paper addresses this deficit, enabling multiple

616

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 3, MAY 2013

point-to-point tasks to be achieved while simultaneously tackling both hard and soft constraints of wide relevance. Experimental results confirm the practical utility and performance of
the proposed approaches and illustrate the benefit gained over
using the standard framework with an a priori generated reference. They also clearly show the benefit of point-to-point ILC
over the standard ILC framework.
Future work will consider the inclusion of prescribed
variation in the temporal point-to-point locations to provide
more flexibility and faster convergence properties. Constraints
linking two or more outputs will also be considered, allowing
coordinated movements to be performed while relaxing unnecessary temporal constraints.
REFERENCES
[1] D. A. Bristow, M. Tharayil, and A. G. Alleyne, A survey of iterative learning control a learning-based method for high-performance
tracking control, IEEE Control Syst. Mag., vol. 26, no. 3, pp. 96114,
2006.
[2] H. S. Ahn, Y. Chen, and K. L. Moore, Iterative learning control: Brief
survey and categorization, IEEE Trans. Syst., Man, Cybern. C, Appl.
Rev., vol. 37, no. 6, pp. 10991121, Nov. 2007.
[3] C. T. Freeman, E. Rogers, A. M. Hughes, J. H. Burridge, and K. L.
Meadmore, Iterative learning control in healthcare: Electrical stimulation and robotic-assisted upper limb stroke rehabilitation, IEEE Control Syst. Mag., vol. 32, no. 1, pp. 1843, Feb. 2012.
[4] D. M. Wolpert, Z. Ghahramani, and J. R. Flanagan, Perspectives and
problems in motor learning, Trends in Cognitive Sci., vol. 5, no. 11,
pp. 487494, 2001.
[5] H. Ding and J. Wu, Point-to-point control for a high-acceleration positioning table via cascaded learning schemes, IEEE Trans. Ind. Electron., vol. 54, no. 5, pp. 27352744, Oct. 2007.
[6] J. Park, P. H. Chang, H. S. Park, and E. Lee, Design of learning input
shaping technique for residual vibration suppression in an industrial
robot, IEEE/ASME Trans. Mechatron., vol. 11, no. 1, pp. 5565, Feb.
2006.
[7] J. van de Wijdeven and O. Bosgra, Residual vibration suppression
using hankel iterative learning control, Int. J. Robust Nonlinear Control, vol. 18, pp. 10341051, 2008.
[8] G. Gauthier and B. Boulet, Robust design of terminal ILC with
mixed sensitivity approach for a thermoforming oven, J. Manuf. Sci.
Eng., vol. 2008, 2008, Article ID 289391.
[9] Y. Wang and Z. Hou, Terminal iterative learning control based station
stop control of a train, Int. J. Control, vol. 84, no. 7, pp. 12631274,
Jul. 2011.
[10] J.-X. Xu and D. Huang, Initial state iterative learning for final state
control in motion systems, Automatica, vol. 44, pp. 31623169, 2008.
[11] Y. Chen and J.-X. Xu, A high-order terminal iterative learning control
scheme, in Proc. 36th Conf. Decision Control, 1997, pp. 37713772.
[12] J.-X. Xu, Y. Chen, T. Lee, and S. Yamamoto, Terminal iterative
learning control with an application to RTPCVD thickness control,
Automatica, vol. 35, pp. 15351542, 1999.
[13] G. Gauthier and B. Boulet, Terminal iterative learning control design with singular value decomposition decoupling for thermoforming
ovens, in Proc. Amer. Control Conf., 2009, pp. 16401645.

[14] P. Lucibello, S. Panzieri, and G. Ulivib, Repositioning control of

a two-link flexible arm by learning, Automatica, vol. 33, no. 4, pp.
579590, 1997.
[15] S. Mishra, U. Topcu, and M. Tomizuka, Optimization-based constrained iterative learning control, IEEE Trans. Control Syst. Technol.,
vol. 19, no. 6, pp. 16131621, Nov. 2011.
[16] S.-G. Hwang, Cauchys interlace theorem for eigenvalues of Hermitian matrices, Amer. Math. Monthly, vol. 111, pp. 157159, 2004.
[17] D. H. Owens, J. J. Htnen, and S. Daley, Robust monotone gradientbased discrete-time iterative learning control, Int. J. Robust Nonlinear
Control, vol. 19, pp. 634661, 2009.
[18] J. M. Ortega and W. C. Rheinboldt, Iterative Solution Of Nonlinear
Equations In Several Variables, 1st ed. New York: Academic Press,
1970.
[19] J. J. Htnen, Issues of algebra and optimality in iterative learning
control, Ph.D. dissertation, Dept. Process Environmental Eng., Univ.
Oulu, , Oulu, Finland, 2004.
[20] C. T. Freeman, Constrained point-to-point iterative learning control
with experimental verification, Control Eng. Practice, vol. 20, no. 5,
pp. 489498, May 2012.
[21] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge,
MA: Cambridge Univ. Press, 2005.
[22] A. Ben-Tal and M. Zibulevsky, Penalty/barrier multiplier methods for
convex programming problems, SIAM J. Optim., vol. 7, pp. 347366,
1997.
[23] C. T. Freeman, Constrained point-to-point iterative learning control,
in Proc. 18th IFAC World Congr., 2011, pp. 36113616.

Chris T. Freeman received the B.Eng. degree

in electromechanical engineering and the Ph.D.
degree in applied control from the University of
Southampton, Southampton, U.K., in 2000 and 2004,
respectively, and the B.Sc. degree in mathematical
sciences from the Open University, Milton Keynes,
U.K., in 2006.
His research currently focuses on the development, application and assessment of iterative
learning and repetitive controllers within both the
biomedical engineering domain and for application
to industrial systems.

Ying Tan received the Bachelors degree from

Tianjin University, Tianjin, China, in 1995, and
the Ph.D. degree from the National University of
Singapore, Singapore, in 2002.
She joined McMaster University in 2002 as a postdoctoral fellow with the Department of Chemical
Engineering. She has worked with the Department of
Electrical and Electronic Engineering, the University
of Melbourne, Parkville, Australia, since 2004. Her
research interests include intelligent systems, nonlinear control systems, model predictive control, real
time optimization, sampled-data distributed parameter systems and formation
control.
Dr. Tan is a Future Fellow (20102013), which is a research position funded
by the Australian Research Council.