This article was downloaded by: [University of Houston]

On: 14 January 2015, At: 19:31

Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

International Journal of Control

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/tcon20

Multivariable norm optimal and parameter optimal

iterative learning control: a unified formulation
D.H. Owens

a b c

Department of Automatic Control and Systems Engineering , University of Sheffield ,

Mappin Street, Sheffield S1 3JD, UK
b

Department of Electronics and Computer Science , University of Southampton , Highfields,

Southampton SO17 1BJ , UK
c

The Italian Institute of Technology , Via Morego, 30, 16163 Genova , Italy
Published online: 05 Apr 2012.

To cite this article: D.H. Owens (2012) Multivariable norm optimal and parameter optimal iterative learning control: a
unified formulation, International Journal of Control, 85:8, 1010-1025, DOI: 10.1080/00207179.2012.673136
To link to this article: http://dx.doi.org/10.1080/00207179.2012.673136

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions

International Journal of Control

Vol. 85, No. 8, August 2012, 10101025

Multivariable norm optimal and parameter optimal iterative learning control:

a unified formulation
D.H. Owensabc*
a
Department of Automatic Control and Systems Engineering, University of Sheffield,
Mappin Street, Sheffield S1 3JD, UK; bDepartment of Electronics and Computer Science,
University of Southampton, Highfields, Southampton SO17 1BJ, UK; cThe Italian Institute of Technology,
Via Morego, 30, 16163 Genova, Italy

Downloaded by [University of Houston] at 19:31 14 January 2015

(Received 9 February 2011; final version received 3 March 2012)

This article investigates the two paradigms of norm optimal iterative learning control (NOILC) and parameter
optimal iterative learning control (POILC) for multivariable (MIMO) -input, m-output linear discrete-time
systems. The main result is a proof that, despite their algebraic and conceptual differences, they can be unified
using linear quadratic multi-parameter optimisation techniques. In particular, whilst POILC has been naturally
regarded as an approximation to NOILC, it is shown that the NOILC control law can be generated from a
suitable choice of control law parameterisation and objective function in a multi-parameter MIMO POILC
problem. The form of this equivalence is used to propose a new general approach to the construction of POILC
problems for MIMO systems that approximates the solution of a given NOILC problem. An infinite number of
such approximations exist. This great diversity is illustrated by the derivation of new convergent algorithms based
on time interval and gradient partition that extend previously published work.
Keywords: iterative learning control; parameter optimal iterative learning control; norm optimal iterative
learning control; optimisation; optimal control

1. Introduction
Iterative learning control (ILC) originated with the
work of Arimoto, Kawamura, and Miyazaki (1984) is
concerned with the performance of systems that
operate in a repetitive manner and includes examples
such as robot arm manipulators and chemical batch
processes, where the task is to follow some specified
output trajectory r in a fixed specified time interval
with high precision. Motivated by human learning,
ILC uses information from previous executions of the
task in an attempt to improve performance from
repetition to repetition in the sense that the tracking
error (between the output and a specified reference
trajectory) is asymptotically reduced to zero (see, e.g.
the review papers Bristow, Tharayil, and Alleyne
(2006), Ahn, Chen, and Moore (2007) and Owens
and Hatonen (2005) and the applications described in
those review papers and papers such as Daley, Owens,
and Hatonen (2007), Ratcliffe, Lewin, Rogers,
Hatonen, and Owens (2006) and Kirchhoff, Schmidt,
Lichtenberg, and Werner (2008). Note that repetitions
are often called trials, passes or iterations in the
literature. Similarly, the time interval is sometimes
called the trial length or pass length.

*Email: D.H.Owens@sheffield.ac.uk
ISSN 00207179 print/ISSN 13665820 online
 2012 Taylor & Francis
http://dx.doi.org/10.1080/00207179.2012.673136
http://www.tandfonline.com

Amann, Owens, and Rogers (1996) provided a

complete (Hilbert space) analysis of m-output, -input
(MIMO) norm optimal iterative learning control
(NOILC) for linear systems based on the use of a
quadratic performance criteria that permits analytic
solution and also has a practical realisation in terms of
Riccati equation generated feedback and feedforward
descriptions. Extensions to this include the discrete
case (Amann, Owens, and Rogers 1996) and the
inclusion of prediction (Amann, Owens, and Rogers
1998). NOILC has the bonus of providing monotonic
reductions in mean square tracking error from iteration to iteration. In a more recent paper, Owens and
Feng (2003) suggested an alternative approach for the
single-inputsingle-output (SISO) case using a new
parameter optimal iterative learning control (POILC)
paradigm that introduced a simplified control law
whilst retaining the most important properties of
NOILC. A discussion of this idea can be found in
the IFAC Review paper (Owens and Hatonen 2005)
and in papers describing a multi-parameter version
(Hatonen, Owens, and Feng 2006), robust inverse
model algorithms (Harte, Hatonen, and Owens 2006)
and related gradient ILC methodologies (Owens,

Downloaded by [University of Houston] at 19:31 14 January 2015

International Journal of Control

Hatonen, and Daley 2008). The important practical
point is that POILC also guarantees monotonic
reductions in mean square tracking error but has the
advantage of potentially being much simpler than
NOILC in terms of implementation. The price paid for
this advantage can be exemplified by the simplest
algorithm introduced by Owens and Feng (2003) where
convergence to zero depends upon positivity properties
of the plant (or a compensated plant). In the absence of
positivity, monotonic error reduction to a non-zero
limit error can be expected (convergence to the
so-called limit set) unless the algorithm is modified
by introducing switching mechanisms (Owens, TomasRodriguez, and Daley 2008).
NOILC and POILC are related but there are,
currently, no published MIMO form of POILC as
distinct methodologies. The primary purpose of this
article is twofold. Following a summary of relevant
known properties of NOILC and (SISO) POILC in
Section 3, the first objective achieved in Section 4 is to
provide a rigorous demonstration that the paradigms
of NOILC and POILC for linear discrete-time systems
can be unified into a single theoretical methodology
using appropriately chosen linear quadratic optimisation techniques. The approach used is to answer the
question.
System Theoretical Question: For any linear singleinput, -input, m-output (MIMO) discrete time dynamical system and any reference signal r, does there exist a
linear multi-parameter control law and an associated
quadratic POILC problem that has a solution that is
identical to that obtained by a chosen NOILC algorithm
for that system?
The main result of this article is a proof that the
answer to this question is positive. The proof is by
construction and provides an explicit relationship
between the required parameters/weights Wk1 in the
chosen POILC objective function, the previous iteration error norm kekk and the parameters Q and R in
the NOILC objective function. As a byproduct, the
results provide a theoretical justification of the use of
error-dependent weights, as suggested by Owens and
Feng (2003). The interpretation used is that, as POILC
was originally seen as a approximation to NOILC, the
NOILC is, in fact, equivalent to POILC if the POILC
parameterisation and objective function are carefully
chosen. In this sense, NOILC is just a special case of
POILC and POILC is the more general concept. In the
authors opinion, the benefits of the result for
optimisation-based algorithm development are that it
opens up the development of a wide range of simplified
multi-parameter MIMO POILC control laws that are
capable of providing a performance that is arbitrarily

1011

close to any desired NOILC problem. This is the

second objective.
The second objective is considered in Section 5 by
the construction and solution of a new and general
multi-parameter, MIMO POILC problem based on the
form and properties of the results of objective 1. The
algorithms are constructed to be approximations to a
given NOILC problem but differ by a degree resulting
from the form and simplicity of the choice of
parameterisation used. They retain the important
property of monotonicity of the error norm and
provide simple formulae for the required weighting
matrices Wk1, feedforward formulae for input time
series updates and formulae for parameter vector
updates, the consequent error evolution and the
associated (tracking error) limit set S1.
The general algorithm derived can be applied to
any linearly parameterised multi-parameter MIMO
POILC problem. Specific examples of SISO POILC
exist in the literature (Owens and Feng 2003; Owens
and Hatonen 2005; Harte et al. 2006; Hatonen et al.
2006; Owens et al. 2008), but clearly there are an
infinite number of other choices that could be made
and appropriate structures for the MIMO case need to
be proposed. This article takes the view that, just as
classical control theory explores the benefits that could
be obtained from more general feedback laws than
simple proportional plus integral plus derivative (PID)
feedback, applied ILC could benefit from an exploration of the possibilities that the more general MIMO
POILC control theory proposed here could offer. This
article does not attempt to do a complete exploration
but begins the process by illustrating how the theory
makes simple MIMO extensions of current knowledge
easy to analyse, to use and to relate to an equivalent
or parent NOILC problem. More precisely, this
article explores the use of:
(1) Control law parametrisation by interval partition. This can be regarded as a natural
extension of Owens and Fengs basic singleparameter, SISO algorithm (Owens and Feng
2003) to a multi-parameter MIMO case by
providing a mechanism for using different
control laws (with different learning gains) on
different time intervals.
(2) Extensions of the SISO gradient methodology
described in Owens et al. (2008) to the
MIMO multi-parameter case by using a
multi-parameter linear decomposition of an
NOILC descent direction into a linear
combination of separate terms corresponding
to a linear parallel, decomposition (termed a
parallel gradient partition) of the plant
model.

Downloaded by [University of Houston] at 19:31 14 January 2015

1012

D.H. Owens

This article concludes in Section 7 with some numerical

studies of the gradient partition algorithm based on
partial fraction plant decompositions. These numerical
studies show that the multi-parameter approach is
capable of substantially improving on the simple
gradient algorithm (Owens et al. 2008) and, more
surprising perhaps, it can provide convergent behaviour very similar to that achieved by the parent
NOILC problem even if a very low number of
parameters is used. It is the authors belief that
future explorations of algorithms within this framework will increase the understanding of ILC and
provide the flexibility to meet the needs of a wide range
of applications.
Finally, Section 2 contains background material
and, in particular, constructs an appropriate lifted
matrix model of an -input, m-output state space plant.
Although algebraically similar to the formulae for the
SISO case, the MIMO version often loses the property
of invertibility either because it is non-square (i.e.
m 6 ) or because the leading Markov parameter
matrix does not have full row or column rank. The
implications of these properties are summarised and
play an important role in the following MIMO
analysis.

2. Problem definition and the lifted plant model

2.1 Problem definition
Consider a standard discrete-time, linear, timeinvariant -input, m-output dynamical system with
state-space model representation defined over a
finite,
discrete
time
interval,
t 2 [0, N]
{0, 1, 2, . . . , N} (in order to simplify notation it is
assumed that the sampling interval, ts is unity or,
equivalently, that the symbol t denotes the index of
the tth sample in the time series). The system is
assumed to be operating in a repetitive mode where,
at the end of each repetition, the state is reset to a
specified initial condition x0 for the next operation
during which a new control input signal can be used.
A reference signal r(t), t 2 [0, N] is specified, the
ultimate control objective being to find an input
function u*(t) so that the resultant output function
y(t) tracks this reference signal r(t) exactly on [0, N].
This objective may or may not be achievable in
practice. Each repetition is also called an iteration or
trial. The process model (written S(A, B, C, D)) is
written in the form:
xt 1 Axt But,
yt Cxt Dut

x0 x0

where the state x() 2 Rn, output y() 2 Rm and input

u() 2 R. The matrices A, B and C have an appropriate
dimensions and D is an m  matrix. It will be
assumed that either D 6 0 or that D 0 and

CA j1B 0 for 1  j 5 k* and CAk 1 B 6 0 for some
j k*  1 (trivially satisfied in practice) and that
the system (1) is both controllable and observable.
If D 6 0, then take k* 0. By construction, in the
SISO case, k* is the relative degree of the transfer
function
Gz CzI  A1 B D

of the system. For notational convenience fk(t) will

denote the value of a signal at time t on iteration k.
The repetitive nature of the problem permits the
iterative modification of the input vector time series
{u(t)}t0 so that, as the number of repetitions increases,
the system asymptotically learns the input function
that gives perfect tracking. More precisely, the control
objective is to find a causal recursive control law
(algorithm) with the properties that, independent of
the control input time series u0 chosen for the first trial,
limk!1 kek k 0 and

limk!1 kuk  u k 0

where k  k denotes any norm for the time series. In

what follows, attention is focussed on feedforward
algorithms of the form uk1() f(k, uk(), ek()). In
addition, the chosen norm is taken to be the
Euclidean norm of the vector of the time series
(which is proportional to the mean square magnitude
of the time series) or any topologically equivalent norm
generated by a positive definite quadratic form. Other
design issues of the rate of convergence, robustness
etc., also important to practical application, will be
pursued in future research.

2.2 Matrix representations of MIMO plant dynamics

Following previous papers that analyse the SISO case
(see, e.g. Owens and Hatonen 2005; Owens et al. 2008),
this article approaches the theoretical development for
MIMO systems by replacing the state space model by
an equivalent matrix, lifted or supervector description. Note that, as the control input has no effect on
the output response for t 0, 1, 2, . . . , k*  1, the
appropriate matrix description is of dimension
m(N 1  k*)  (N 1  k*). Although the algorithms obtained are stated in terms of this matrix
description, the associated matrix computations can
always be replaced by simulations using the underlying
state space model.
To construct this matrix model, define the time
series input, output, reference and error super-vectors

1013

International Journal of Control

(respectively) on the kth trial via
uk uTk 0, uTk 1, . . . , uTk N  k T 2 R N1k

representation of the system with m  transfer

function matrix

yk yTk k , yTk k 1, . . . , yTk NT 2 RmN1k

r rTk k , rTk k 1, . . . , rTk NT 2 RmN1k
ek

eTk k , eTk k

1, . . . , eTk NT

2R

mN1k

Downloaded by [University of Houston] at 19:31 14 January 2015

where ek( j) r( j)  yk( j), k*  j  N. Furthermore, let

u* be the input sequence (in time series or supervector
form) that gives r(t) [Gcu*](t) d0(t) and Gc is the
convolution mapping corresponding to (1).
Note: The values of r(t) in the range 0  t  k*  1
play no role in the model as the plant output y(t) on
this interval cannot be affected by any choice of input
sequence. Tracking is therefore impossible and this
sub-interval can be eliminated from the analysis in a
natural way.
With the above definitions, the usual formula for
the inputoutput response of the system can be written
in the matrix form, k  0,
yk Guk d0

where G has dimension m(N 1  k*)  (N 1  k*)

and has the lower triangular (block) band structure
Gij G(i1)( j1) that is required by causality and time
invariance of linear time-invariant MIMO convolution
systems. More precisely, if k* 0, then
2
3
D
0
0  0
6 CB
D
0  0 7
6
7
6
7
6
7
CAB
CB
D



0
6
G6
7
.. 7
.. . .
..
..
6
4
5
. .
.
.
.
CAN1 B

CAN2 B   

Alternatively, if k*  1,
2

CAk 1 B
0
0
6


k
k
1
6 CA B CA
B
0
6
6



CAk 1 B CAk B CAk 1 B
G6
6
6
..
..
..
6
.
.
.
4
CAN1 B CAN2 B











..
.

0
..
.

   CAk

3
7
7
7
7
7
7
7
7
5
1

zk Gz CAk zI  A1 B D0

B
7

In both cases, d0 CAk x0 , . . . , CAN x0 T . The blocks

CA jB of the matrix G are the so-called Markov
parameter matrices (1) or, equivalently, the unit
impulse response matrices of the plant. It is useful to
note that the matrix G is the matrix describing the

dynamics of the system SA, B, CAk , D0 where D0 D

if k* 0 or CAk 1 B otherwise. Alternatively, it is a

Note that the tracking capability of the system depends

crucially on m and and the properties of the leading
Markov parameter D0:
(1) G is typically non-square but is square if m .
(2) If 4 m, then the kernel of G (denoted ker(G))
always contains non-zero vectors and the input
generating the required reference is nonunique.
(3) The range of G, denoted by R(G), spans the
output space if, and only if, m  and the
diagonal block D0 of G has full column rank.
(4) G is invertible if, and only if, m and the
Markov parameter matrix D0 on the main
diagonal is non-singular. In this case, for an
arbitrary reference supervector r on k*  j  N,
there exists a unique vector time series u* on
0  j  (N  k*) such that r Gu* d0.
(5) More generally, even if m 4 1, the diagonal
block matrices and hence G itself can lose rank.
In these cases, R(G) again does not span the
output space so that there are reference signals
on k*  t  N that cannot be tracked. A simple
illustrative output space direct sum decomposition can be written


RmN1k RG Q1 kerGT

or replace kerGT RG? 9
where Q is arbitrary, symmetric and positive
definite and the orthogonal complement ? is
taken with respect to the Euclidean inner
product hx, yi xTy. Any component of r in
Q1ker(GT) cannot be tracked.
The precise form of R(G) or ker GT is
difficult to characterise and, given the typically
high dimension of the spaces concerned, has
little use in this article. It is of interest to note
that an initial subinterval [0, k**] of [0, N]
always plays a role. More precisely, write


10
Ok D, CB, CAB, CA2 B, . . . , CAk1
and note that it has a range that can be
identified with the achievable outputs at time
t k. Note that Ok 0 for 0  k 5 k* underpinning the observation that the outputs on
this interval cannot be influence by controls on
t  0. Note also that, if the first non-zero
Markov parameter matrix does not have full
row rank, the minimum k (denoted k**  n)
that ensures that Ok has column rank m

1014

D.H. Owens

satisfies k** 4 k*. Thus, t k** is the first time

at which any value of the output vector can be
achieved.
(6) If m 1, then the above analysis reduces to
the modelling of SISO systems summarised in,
for example, Owens et al. (2008) and Owens
and Hatonen (2005). The construction extends
this work to the MIMO case and introduces
new structural features (in the non-square case
and the case where the leading Markov
parameters lose rank) that will impinge on the
following development.

Downloaded by [University of Houston] at 19:31 14 January 2015

From now on this so-called lifted plant model will be

used as a starting point for analysis.

3. A summary of NOILC and POILC

In what follows, the published ideas and formulae for
MIMO NOILC and SISO POILC are summarised in
preparation for the main MIMO results of this article.

to zero as k ! 1, i.e. kek1k  kekk, 8k  0 and

limk!1ek 0.
(2) This result applies for all plant models G on
finite intervals, i.e. the algorithm has global
convergence properties for all MIMO linear
systems.
(3) It can be implemented using a Riccati state
feedback plus feedforward structure.
NOILC has been applied successfully in a number of
application (e.g Ratcliffe et al. 2006) and a good
understanding of its performance is currently available
(including the effects of non-minimum-phase zeros on
performance Owens and Chu 2010) in the SISO case. It
is taken to be the benchmark algorithm for the
developments in this article.

3.2 Essential structure and properties of POILC

NOILC (Amann et al. 1996a, b) constructs the control

time series supervector uk1 by minimising the
performance index

Now turning to the POILC problem introduced in

Owens and Feng (2003) and motivated by a perceived
need to simplify the control structure and, in particular, to avoid the need for Riccati feedback implementations. More precisely, their work considered
SISO systems and introduced POILC as structurally
different control laws of the feedforward form

Jk1 uk1 kek1 k2Q kuk1  uk k2R ,

uk1 uk k1 Kek

3.1 Essential structure and properties of NOILC

11

where Q, R are symmetric positive

p definite matrices.
Here kfkH denotes the norm fT Hf of the time series
vector f induced by the inner product hx, yiH xTHy,
H HT 4 0. Minimising Jk1 subject to the plant
dynamics yields the formal input and error recursions
uk1 uk R1 GT Qek1 , ek1 I GR1 GT Q1 ek
12
and hence
uk1 uk R1 GT QI GR1 GT Q1 ek
uk I R1 GT QG1 R1 GT Qek

13

Note: In Amann et al. (1996), the weighting matrices Q

(resp. R) were assumed to be block diagonal with
m  m (resp.  ) blocks to reflect the normal
structure of the optimal control theory used to
construct the Riccati solution to the problem. This
assumption is not necessary and is not assumed here
for simplicity of notation.
This NOILC algorithm has the important properties (Amann et al. 1996; Owens and Hatonen 2005),
(1) If the signal r  d0 lies in the range of G, then
NOILC guarantees monotonic error reductions

14

where the matrix K is a lifted representation of a fixed

linear operation on the data ek and k1 is a scalar
parameter (learning gain) to be chosen for each
iteration. A preliminary connection between POILC
and NOILC can be seen by noting that NOILC takes
the same form by choosing k1 1 and K R1GTQ
(I GR1GTQ)1 (or (I R1GTQG)1R1GTQ).
The SISO POILC optimisation problem originally
posed and solved by Owens and Feng (2003) assumes
that some choice of K has been made (and is not varied
from iteration to iteration) and that k1 is chosen by
minimising the following criterion subject to plant
dynamics.
Jk1 k1 kek1 k2 wk1 2k1
giving k1

eTk GKek
wk1 kGKek k2

15

The weight wk1 suggested by Owens and Feng was

wk1 w 1 w 2 kek k2 ,

w 1  0, w 2  0

w 1 w 2 4 0
16

Despite its computational simplicity, the algorithm has

several important properties (Owens and Feng 2003)
paralleling those of NOILC, namely,

1015

International Journal of Control

(1) The error norm reduces monotonically each
iteration with (possibly non-zero) cluster points
of the error sequence {ek}k0 in the so-called
limit set
S1 fe: eT GKe 0g

17

Downloaded by [University of Houston] at 19:31 14 January 2015

(2) The algorithm is guaranteed to converge to

zero limit error (independent of the starting
condition (u0, e0) if, and only if, S1 {0}.
(3) The limit set is non-empty and is the singleton
{0} if, and only if, GK (GK)T is strictly signdefinite. For example, if K GT (Owens et al.
2008) or K G1 (Harte et al. 2006).
The above algorithm shows an algebraic connection
between NOILC and POILC. The important property
of error norm monotonicity is retained, but the
convergence of the two approaches also have distinct
differences due to the need to consider the nature of the
limit set as summarised in the second and third points.

4. The main result: a unification of NOILC and

POILC
The following section unifies NOILC and a new
formulation of MIMO POILC into one algebraic
framework and demonstrates that POILC is the more
general approach in the sense that the following
question has a positive answer:

where tr[X] denotes the trace of the square matrix X

and Wk1 4 0 is an, as yet, unspecified (iterationdependent) symmetric, positive definite weight matrix.
The trace function has the well-documented properties
tr(XY) tr(YX) and tr(X) tr(XT).
Note: This expression is a natural extension of the
POILC idea of minimising the quadratic error norm
plus a weighted quadratic norm on the algorithm
gain obtained by replacing wk1 2k1 by the term
trTk1 Wk1 k1 . It is also naturally linked to NOILC
by using the replacement
kuk1  uk k2R ! trTk1 Wk1 k1 

20

in the objective function.

It is now proved that, by suitable choice of weight
Wk1, this POILC problem has the necessary structure
to produce a result identical to that of a NOILC
problem for the same r and with weighting matrices Q
and R.
Using the error update equation and letting k1 be
replaced by k1 k1, the first variation of Jk1 is
given by


Jk1 2 eTk1 Qek1 trTk1 Wk1 k1 

2 eTk I  Gk1 T QGk1 ek


tr Tk1 Wk1 k1 


2tr ek eTk I  Gk1 T QG Tk1 Wk1 k1
21

System Theoretical Question: For any linear -input,

m-output (MIMO) discrete time dynamical system and
any reference signal r, does there exist a linear multiparameter control law and an associated quadratic
POILC problem that has a solution that is identical to
that obtained by a chosen NOILC algorithm for that
system?

Setting this equal to zero for all values of  gives a

minimising k1 as a solution of the equation

The necessary form of the MIMO POILC problem

that should be used is constructed and the equivalence
is proved by showing that it leads, uniquely, to the
same control law as that derived from an MIMO
NOILC problem with weighting matrices Q and R.
The construction producing the answer to the
above question uses the following general form of
linear parameterised control structure

23
p
(where kek denotes the Euclidean norm eT e of e).
Solving for k1ek then gives the unique POILC
control update law

uk1 uk k1 ek ) ek1 I  Gk1 ek

18

where k1 is an (N 1  k*)  m(N 1  k*) matrix.

This matrix will now be regarded as a design
parameter (or, more precisely, a collection of
m(N 1  k*)2 design parameters) and chosen to
minimise the (strictly convex) objective function
Jk1 k1 kek1 k2Q trTk1 Wk1 k1 

19

Wk1 k1 GT QI  Gk1 ek eTk

22

Operating on ek then gives

Wk1 k1 ek GT Qek kek k2  GT QGk1 ek kek k2

1
T
2 1
T
uk1 uk I kek k2 W1
k1 G QG kek k Wk1 G Qek

24
This equation applies for all choices of weight
sequences {Wk1}k0. However, with the special
choice of Wk1 Rkekk2, this formula is precisely
similar to that obtained from NOILC. This proves the
following main theorem.
Theorem 1: The MIMO POILC algorithm defined by
the update law
uk1 uk k1 ek ,

ek r  yk , k  0

25

1016

D.H. Owens

where the matrix k1 is computed as the solution of

k1 arg minfkek1 k2Q kek k2 trTk1 Rk1 ;
yk1 Guk1 d0 g

26

generates an input uk1 that is exactly equal to that

produced from the NOILC algorithm
uk1 arg minfkek1 k2Q kuk1  uk k2R ;
yk1 Guk1 d0 g

27

For completeness, the optimal value of k1 is as

follows.

Downloaded by [University of Houston] at 19:31 14 January 2015

Theorem 2: The optimal value of k1 is uniquely

defined and can be computed as
1 1
T
T
T
28
k1 I kek k2 W1
k1 G QG Wk1 G Qek ek

Proof: First, by substitution, observe that this expression is a solution of the stationary point condition for
the objective function. Uniqueness follows from the
strict convexity of the objective function and the linear
constraints.

Note that:
(1) For practical purposes, this theorem indicates
that any MIMO NOILC problem is a special
case of an MIMO POILC approach obtained
by a specific choice of weight matrices and
control law parameterisation. In this sense,
there is nothing special about NOILC other
than its potentially useful realisation in terms of
Riccati state feedback and off-line feedforward
computations. If state feedback is not acceptable, then NOILC is just an example of
POILC but one that may not be appropriate
for implementation.
(2) It is interesting to note that the fact that Wk1
is necessarily quadratic in the Euclidean norm
kekk provides some justification of the empirically suggested choice of scalar weight wk1
w 1 w 2 kek k2 proposed by Owens and Feng
(2003) for SISO POILC.
In the next section these results are used to motivate
and construct a wide range of POILC algorithms
capable of approximating a given NOILC algorithm.

5. POILC as an approximation to NOILC

The results of the previous section embed NOILC
within a POILC framework and indicate that, despite
the observed power of NOILC, it is simply a very
specific example of a multi-parameter MIMO POILC
problem. This result has systems theoretical value but,
for the purposes of this article, it is also regarded as

opening up and motivating the possibility of generalising known SISO POILC concepts to the MIMO
POILC case by algorithm constructions that approximate a given MIMO NOILC problem. More precisely, consider the construction of a general approach
to approximating a given MIMO NOILC problem
(with given Q and R weight matrices) by an MIMO
POILC problem defined by Q, R and a chosen control
law parameterisation. This is done by exploiting the
structure of the POILC problem that reproduces
NOILC, but making possible a considerable reduction
in the (typically very large) number m(N 1  k*)2 of
free parameters. The formulation is linked to but
extends other approaches to POILC such as Owens
et al. (2008) and Owens and Feng (2003) to a new
general approach to the MIMO multi-parameter case.

5.1 Linear parameterisations

Consider a more general form of linearly parameterised ILC algorithm by expressing the control signal
update in the form
uk1 uk k1 ek ,

but write

k1 k1
29

where the matrix (k1) depends on a p  1 parameter

vector
h
iT
2
p
k1 1
k1 , k1 , . . . , k1

30

in a linear way. That is, for suitable choices of

(N 1  k*)  m(N 1  k*) matrices ( j), 1  j  p,


p
X

 j  j

31

j1

In particular, it is always possible to write



e
Fe, where Fe 1 e, 2 e, . . . ,  p e
32
It is easily seen that the error evolution from trial to
trial is then described by
ek1 ek  Gk1 ek ek  Mek k1 ,
where Mek GFek , k  0:
33
To eliminate any redundancy in the parameterisation,
it is assumed that {( j)}1jp are linearly independent
or, equivalently,
 0 ,  0

34

International Journal of Control

Also note that it is always possible to choose a p  p
matrix W such that


tr T R T W,


by setting elements Wsq tr s T Rq Wqs
35

Downloaded by [University of Houston] at 19:31 14 January 2015

from which it follows that WT W. In addition, W 4 0

as tr(T()R())  0 and equals zero if, and only if,
() 0 (i.e. iff  0).
Note: In providing simple POILC approximations to
NOILC, it is natural to consider situations where the
number of free parameters p  (N 1  k*)2 is relatively small. The theory provides little insight into the
viability of using small p successfully but examples
later in this article show that this can be the case.

5.2 The proposed approximating MIMO POILC

problem
From the above note, the natural POILC problem that
approximates to the NOILC problem for the -input,
m-output system G with weights Q, R is taken to be the
minimisation of the multi-parameter POILC objective
function
Jk1 kek1 k2Q kek k2 trTk1 Rk1

36

but where the search over all possible values of k1

(generating the NOILC solution exactly) is replaced by
a (suboptimal) search over all possible values of k1
obtained by replacing it by (k1). That is, the control
law is obtained by minimising
Jk1 kek1 k2Q kek k2 trT k1 Rk1
kek1 k2Q Tk1 Wk1 k1

37

with respect to k1 2 Rp subject to the plant dynamic

constraints. The weighting matrix chosen is
Wk1 kek k2 W,

38

which is symmetric and positive definite if ek 6 0.

Clearly, the use of p 5 (N 1  k*)2 parameters
limits the search space. This is expressed more precisely
as follows.
Theorem 3: The solution of the above POILC problem
approximates the solution of the NOILC problem in the
sense that, for a given value of ek,


mink1 Jk1 ; ek1 ek  Gk1 ek


mink1 Jk1 ; ek1 ek  Mek k1

39

1017

equality holding if, and only if, R1GTQ(I

GR1GTQ)1ek
(I R1GTQG)1R1GTQek lies in
the span of the search directions {( j)ek}1jm.
Note: The theorem relates the objective functions
J(k1) and J(k1) on a given iteration from the
same error ek. It does not imply any similar result for
the error norms on that iteration or for the evolution of
the error norms with trial index k from the same
starting point e0. However, it does imply that the
optimal value of the NOILC objective function can be
arbitrarily closely achieved by choice of POILC
parameterisation.
Proof: The inequality must be true as the first
problem solves the NOILC problem by a search over
all possible values of k1. The second problem differs
only in that the gain matrix is computed by a more
limited search over all values of k1 with k1
(k1). Any such value is therefore suboptimal.
The result is hence proved as, at optimality, the value
of k1ek is uniquely defined and equal to

R1GTQ(I GR1GTQ)1ek.

5.3 Computation of the optimal parameter vector

Now turning to the computational solution of this
POILC problem. Minimising J with respect to k1
subjected to error dynamics then gives the general
POILC parameter vector update algorithm
k1 Wk1 MT ek QMek 1 MT ek Qek ,

k0
40

Note that the inverse exists if ek 6 0, which is naturally

satisfied in practice as iteration is not necessary if the
error ek is already zero. The POILC algorithm is
implemented by using ek data to compute k1 from
the above equation followed by substitution into the
control update law to obtain uk1. This new control
times series is then used on the next iteration.
Note that:
(1) The computation of k1 is easily undertaken
once the m(N 1  k*)  p matrix M(ek)
GF(ek) is computed. A simulation-based
approach can use the state space model to
compute the response of the plant from zero
initial conditions to the input vector time series
described by the supervector  jek and placing
this as the j-th column of M(ek).
(2) If, as desired, limk!1 ek 0, then the inversion
has the potential to become ill-conditioned.
One potential solution is to scale the

1018

k1

D.H. Owens
equations as follows:

1

ek
ek
ek
T
T
WM
M
QM
kek k
kek k
kek k
ek
, k  0:
41
Q
kek k
The approach taken here, however, to ameliorate the problem in what follows, is to replace
Wk1 by
Wk1 ! W1 W2 kek k2

42

In particular,
(1) If E1 0 then limk!1 kekk 0 and the tracking
error converges to zero asymptotically.
(2) If E1 0 and w 4 0, then limk!1 kekk 0 and
the parameter vector sequence {k }k1 satisfies
P
1
2
k1 kk k 5 1 i.e. limk!1
P k 0.2
(3) If E1 4 0 and w  0, then 1
k1 kk k 5 1 (i.e.
limk!1 k 0) and all cluster points of the error
vector sequence {ek}k0 are non-zero and lie in
the limit set
S1 fe: eT QMe 0g:

where

Downloaded by [University of Houston] at 19:31 14 January 2015

W1 W1 T  wI,

W2 W

43

and the parameter w  0. The choice of w 4 0

ensures invertibility and will offset any illconditioning of Wk1 when kekk is small. This
construction parallels that of Owens and Feng
(2003) and extends it to the multi-parameter,
MIMO case in a natural way. One simple
choice is proposed as follows:
W1 2 W

Wk1 2 kek k2 W

44

where  4 0. This can be motivated and

interpreted by the observation that the term
2 plays little role if   kekk, but comes into
play when   kekk. This suggests that " could
be chosen to be of the order of the smallest
Euclidean error norm that is required, in
practice, for the application being considered.
The following section explores the properties of the
algorithm described above.

48

Note: S1 can be interpreted as the set of errors

generating a zero parameter vector.
(4) Finally, suppose that w  0, then a necessary and
sufficient condition for the error to converge to
zero (independent of the starting condition e0) is
that
and S1 \ RG f0g

r  d0 2 RG

49

where R(G) denotes the range of G.

Proof: Using optimality gives
kek1 k2Q Tk1 Wk1 k1  kek k2Q

50

as the choice of k1 0 is, at best, optimal.

Monotonicity of the error norm (and hence the
existence of E1  0) then follows from the positivity
of Wk1. If kek1kQ kekkQ 4 0, then Tk1 Wk1 0,
and hence k1 0 as W 4 0.
Writing
 1

W kek k2 W2 MT ek QMek k1 MT ek Qek
51
then

5.4 Convergence analysis

For completeness, in the following algorithm the
convergence and monotonicity result is stated and
proved.
Theorem 4: With the above notation and assumptions,
suppose that ek 6 0. Then the resultant POILC algorithm has the monotonic error reduction property
kek1 kQ  kek kQ

8k  0

45

with equality holding if, and only if, the computed

optimal parameter vector k1 0. Moreover, the
parameter vector sequence is bounded and satisfies
1
X

Tk1 Wk1 k1 5 1

In addition, there exists an E1  0 such that

lim kek k2Q E1  0

47

Tk1 MT

ek
ek
Q
kek k
kek k

52

and the boundedness of the parameter vector sequence

follows by the application of the well-known Cauchy
Schwarz inequality.
Proposition (1) follows from the definition of a
limit. To prove (2), use (1) and then apply induction on
(50) to give
0  kek1 k2Q
 ke0 k2Q 

k1
X
j1

46

k0

k!1

Tk1 W2 k1

Tj Wj j  ke0 k2Q  w

k1
X

kj k2

j1

which
always true that
P1 T proves that it is P1
2

W

5
1.
Also
if
j j
j1 j
k1 kk1 k 5 1
w 4 0 so that, in this case, limj!1j 0.
To prove (3), note that E1 4 0 and the topological
equivalence of the two norms kek and kekQ, implies the

International Journal of Control

existence of a scalar  4 0 such that
0  kek1 k2Q  ke0 k2Q 

k1
X

Tj Wj j

j1

 ke0 k2Q  E1

Downloaded by [University of Houston] at 19:31 14 January 2015

P1

k1
X

Tj Wj

j1

hence proving that k1 kk1 k 5 1 as W WT 4 0.

As a consequence, limk!1 k 0 and the error
sequence can only have non-zero cluster points solving
MT(e1)Qe1 0, i.e. e1 2 S1.
Finally, to prove (4), if r  d0 2
= R(G) then zero
error is unattainable, proving the necessity. The
necessity of the condition S1 \ R(G) {0} follows as:
if not satisfied it contains a point e0 r  d0  Gu0 6 0
and hence the corresponding input u0 generates the
parameter vector sequence {k}k1 {0, 0, 0, 0, . . .} and
hence ek e0, 8k  0 contradicting the convergence of
the error to zero. To prove the sufficiency, let u0 be
arbitrary and note that ek 2 R(G) 8k  0. Suppose that
E1 4 0, then, using (3), all cluster points of the error
sequence lie in S1 \ R(G) {0} (by assumption), i.e.
ek ! 0 so that E1 0, which is a contradiction. This
completes the proof of the result.

5.5 Properties of the limit set S1

The retention of the NOILC property of monotonicity
of the error norms in the MIMO POILC algorithm is a
major practical benefit, but it is noted that a possible
consequence of the use of simple parameterisations will
be that convergence of the tracking error to a non-zero
point in the limit set e1 2 S1 is possible. This may be
inevitable due to the choice of a reference r 2
= R(G) or,
if r 2 R(G), it may occur (dependent on choice of ())
but be acceptable in practice (or offset by switching
strategies Owens et al. 2008) but it is probably
more useful to avoid the problem by constructing
parameterisations where the Limit Set satisfies
S1 \ R(G) {0}. This is not difficult as is illustrated
by the single-parameter examples:
(1) An MIMO Inverse Model Algorithm: If the
matrix model G is square and invertible, then
the choice (Harte et al. 2006) of the inverse
model algorithm k1 k1G1 gives F(e)
G1e and hence eT QMe eT QGFe kek2Q ,
i.e. the limit set S1 is exactly the set e 0.
Convergence to zero error is achieved for all
reference signals in this case.
(2) An MIMO Gradient Algorithm: Consider the
following MIMO extension of the SISO error
gradient-based (Owens et al. 2008) algorithm
uk1 uk k1 R1 GT Qek

53

1019

which
gives
eTQM(e) eTQGF(e)
T
1 T
e QGR G Qe and hence the limit set S1
{e:
GTQe 0} Q1ker(GT) Q1R(G)?
where the orthogonal complement ? is taken
with respect to the natural Euclidean inner
product hx, yi xTy. It follows that S1 \
R(G) {0} so that convergence of the error to
zero is guaranteed provided that r  d0 2 R(G).
More generally, we have the following theorem.
Theorem 5: S1 always contains the vector subspace
Q1ker(GT) Q1R(G)?.
Proof: The proof follows simply from the characterisation of S1. More precisely, if e 2 S1, then
eTQGF(e) 0, which is true for all e (independent of
the parameterisation F(e)) satisfying GTQe 0, i.e.

e 2 Q1ker(GT) Q1R(G)?.
Theorem 6: For the above MIMO POILC algorithm,
(1) A sufficient condition for convergence of the
error to zero error whenever r  d0 2 R(G) is that
there exists a non-zero vector P
 [ 1, . . . ,  p]T
j
such that the symmetric part of m
is
j1 j QG
sign definite on R(G).
(2) A sufficient condition for this is that
R0GTQ 2 span{( j)}1jm for some (N 1 
k*)  (N 1  k*) matrix R0 with positive
definite symmetric part.
Proof: From the definition of S1,Pif e 2 S1 \ R(G) is
j
arbitrary, then eT QFe eT m
j1 j QG e 0
which, under the conditions of the theorem, implies
that e 0 and hence S1 \ R(G) {0}. Hence limk!1
ek 0. Part two follows easily as the existence of the  j
is guaranteed by the inclusion condition.

Note: An example of a suitable choice of R0 will be

seen to be R1 in the parallel gradient partition
methods described later.

6. Examples of parameterisations
Using the results of the previous sections, the choice of
parameterisation is a design decision aimed at achieving desired ILC performance. The original SISO
POILC algorithm suggested by Owens and Feng is
the simplest such choice and has been well studied. The
more complex SISO inverse-model-based algorithm
(Harte et al. 2006) and gradient algorithm (Owens et al.
2008) have also been well studied with results on
convergence and comparative robustness conditions
rigorously derived. In the following subsections, a
number of alternative new, multi-parameter parameterisations are explored to indicate the richness of

1020

D.H. Owens

choice available, the methods of analysis and the

potential for improved performance.

Downloaded by [University of Houston] at 19:31 14 January 2015

6.1 Parameterisations as gradients

Given the proven monotonic reduction of kekQ, the
essential choice in POILC is a good and flexible choice
of descent direction. Note that f is a descent direction
if, and only if, f 2 {f : fTQe 5 0} and hence the number
of choices is infinite within the defined open halfspace. A useful class of descent directions can be
characterised by the relation f H1GTQe where
H HT 4 0 is arbitrary. The steepest descent direction
(with H I) is one such vector but is known in
numerical analysis to lead often to slow convergence.
This is a natural motivation for searching over a higher
dimensional subspace spanned by p 4 1 linearly
independent descent directions, a construction that
will introduce greater flexibility in reducing the norm
of the error by increasing the number of descent
directions available to the search. In this context, the
general POILC update law
uk1 t uk t

p
X

j
k1
 j ek

54

j1

can be interpreted as the construction of p independent

search directions by the choice of {( j)}1jp operating
on the data ek and combined by suitable choice of
j
independent scalar learning gains fk1
gj1 .
In the above context, the two essential questions of
ILC design are
(1) What are appropriate choices of p and operations ( j) in a given case
(a) to be a defined approximation to the
NOILC problem with a specified Q and
R and
(b) to produce the desired convergence and
performance characteristics?
(2) What is the form of the Limit Set S1 {e:
eTG( j)e 0, 1  j  p} arising from this choice?
That is, what are the solutions of the p
quadratic inequalities dependent on G and the
choice of the operations ( j)?
To illustrate the potential of the idea, some examples
of parameterisations are introduced in the next sections
motivated by the notion of extending the SISO work of
Owens and Feng (2003), the inverse model approach
(Harte et al. 2006) and the gradient approach (Owens
et al. 2008). No attempt is made to be exhaustive as the
aim of the examples is solely to indicate the large
number of potential parameterisations available and
the potential for improved performance.

6.2 Parameterisation using interval partition

This section re-examines Owens and Fengs (2003)
results with a view to increase the potential for
improved performance in MIMO systems by the
choice of several parameters equivalent to applying
different learning gains to different subintervals of the
time interval [0, N  k*].
Note: A simple practical motivation for this construction is the need to track reference signals containing
significant events at known times or over known times
intervals. The ability to decouple iteration updates over
these different intervals or between these times is
intuitively beneficial.
Let K be the (N 1  k*)  m(N 1  k*) matrix
representation of an m-input, -output, causal or
noncausal known compensator mapping signals on
[k*, N] into [0, N 1  k*]. A simple multi-parameter
POILC update law generalising that of Owens and
Feng could be obtained by dividing the N 1  k*
sample points intoP
p disjoint groups of N1, N2, . . . , Np
p

samples (with
Assuming
j1 Nj N 1  k ).
that these groups are subintervals and setting
 [(1)    ( p)] suggests the following generalisation
of Owens and Feng (2003):
uk1 uk k1 ek where
2 1
 IN1
0
0 0
:
6
2
:
 IN2 0 0
6 0
6
6 :
:
: :
:
6
 6
6 :
:
: :
:
6
6
6 :
:
: 0  p1 INp1
4
0

: :

7
7
7
7
7
7K
7
7
7
7
5

:
:
:
0
p

 INp
55

which applies a learning gain ( j) to the j-th subinterval

following the operation K on ek. If
2 3
K1
6 . 7

. 7
K6
4 . 5, where Kj is Nj  mN 1  k , then
Kp
2 3
0
6 .. 7
6 . 7
6 7
6 7
607
6 7
j
7
56
 6
6 Kj 7:
6 7
607
6 . 7
6 . 7
4 . 5
0

1021

International Journal of Control

To construct the approximating POILC problem to the
NOILC problem with weighting matrices Q, R requires
the evaluation of
3
2 1
0 0 0
:
:
e
6 0 e2 0 0
:
: 7
7
6
7
6
7
6 :
:
:
:
:
:
7
Fe 6
6 :
:
: :
:
: 7
7
6
7
6
4 :
:
: 0 e p1
0 5

Downloaded by [University of Houston] at 19:31 14 January 2015

0
0

0
2

e p
31
1

e
B
6 2 7C
B
7C
6 e
B
7C
6
B
6 : 7C
B
7C
6
B
7C
6
Bwhere e Ke 6 : 7C:
B
7C
6
B
6 : 7C
B
7C
6
B
6 p1 7C
@
5A
4 e

p
e

57

with e j 2 RmNj , 1  j  p, in order to apply the

update formula on subintervals as follows:
j
u k1

u kj

j j
k1
ek ,

1  j  p,
31
u 1
B
6 2 7C
B
7C
6 u
B
7C
6
B
6 : 7C
B
7C
6
B
7C
6
Bwhere u 6 : 7C
B
7C
6
B
6 : 7C
B
7C
6
B
6 p1 7C
@
5A
4 u

58

u p
The vector k1 is computed from the approximating
POILC problem with weighting matrix
W Wij ,

Wij tri T R j 

59

Note: If R is block diagonal with Nj  Nj diagonal

blocks {R1, . . . , Rp}, then this formula simplifies to
W diagftrKTj Rj Kj g1jp .
The algorithm is guaranteed to produce monotonic
reductions in kekQ from iteration to iteration. The
Limit Set is defined by solutions of the p quadratic
equalities eTQM(e) eTQGF(e) 0 from which
Theorem 7: Using the above construction, sufficient
conditions for the error to converge to zero is that
r  d0 2 R(G) and that the quadratic form eTQGKe is
sign definite on R(G).
Proof: The proof follows from theorem 6 choosing
 [1, 1, . . . , 1] when F(e) Ke.

The above construction and results reduce to those

of Owens and Feng (2003) in the case of m p 1

and Q I. They are hence a true generalisation of

those ideas to the MIMO multi-parameter POILC case
and have the added benefit of containing explicit
formulae for the objective function that can be used to
ensure that the problem is an approximation to the
NOILC problem defined by Q, R. Increasing p will
increase convergence speeds as it increases the number
of search directions that are used in the optimisation.
In principle, p itself and the associated intervals could
be changed each iteration if desired.
The choice of K is open to the control designer.
A link to SISO inverse model control is expressed as
the following theorem for MIMO Inverse Model ILC
with Interval partition.
Theorem 8: If w  0, G is invertible and an interval
partition algorithm is chosen with K G1, then limk!1
ek 0.
Proof: Using Theorems 4 and 6, it is sufficient to
prove that S1 {0}. This follows as eTQGKe eTQe is
positive definite.

In the next subsections, interval partition is applied

to gradient ILC and a new gradient partition algorithm
is presented.

6.3 Parameterisation using interval gradient partition

Using the interval partition construction of the
previous section, another choice of K is one possible
generalisation of the gradient algorithm to the MIMO
multi-parameter case, namely
K R1 GT Q

60

which generates descent directions for kek2Q and is

linked to NOILC by noting that K R1GTQ is
an approximation to R1GTQ(I GR1GTQ)1 that
is good if R1 is small.
Again eTQGKe eTQGR1GTQe is positive definite
on R(G) as, if eTQGR1GTQe 0, then Qe 2 ker(GT)
R(G)?. The result follows as Q1R(G)? \ R(G) {0}.
The error converges to zero again if r  d0 2 R(G),
proving the following theorem.
Theorem 9: If w  0 and an interval partition algorithm is chosen with K R1GTQ, then the resultant
multi-parameter MIMO Gradient-based ILC Algorithm
with Interval Partition ensures that limk!1ek 0 for
all initial u0 and for all achievable r.
This is just one way forward to develop multiparameter MIMO POILC based on gradient concepts.
An alternative considered here is gradient partition
using a parallel decomposition of the system model G
as described below.

1022

D.H. Owens

6.4 Parameterisation using parallel gradient partition

An increase in the number of search directions can be
achieved using what will be called Parallel Gradient
Partition. More precisely, write the plant matrix as an
additive (and hence parallel) sum of subsystems in the
form
G

p
X

Finally, note that computations of k1 and the

input time series update require the following formulae
Fe GT1 e, GT2 e, . . . , GTp e,
Me GFe GGT1 e, GGT2 e, . . . , GGTp e
W Wij ,

Wij trGTi Gj Wji ,

1  i, j  p
65

Gj

61

j1

Downloaded by [University of Houston] at 19:31 14 January 2015

where each matrix representation Gj is m(N

1  k*)  (N 1  k*). A candidate parameterisation
of the gradient is then obtained by choosing the
Parallel Gradient Partition
k1

m
X

j
k1
 j ,

 j R1 GTj Q, 1  j  p

j1

62
There are infinity of such partitions, all of which
generate a p-dimensional search space containing the
descent direction R1GTQek for kek2Q at e ek obtained
j
1, 1  j  p. The best choice
by the choice of k1
of p and the Gj for a particular application is an open
question. For example, the partition of G could be
identified with a decomposition of the transfer function

matrix zk Gz by writing


zk Gz

p
X

Gj z

63

j1

where the {Gj(z)} are linearly independent. An

illustrative example of this would be a partial fraction
decomposition


zk Gz

n
X
j1

j
1  z1 pj

64

where p n (the state dimension), pj denotes one of the

n (assumed real and distinct for simplicity) poles of
G(z) and j is the m  residue matrix at the pole pj.
Decompositions with p 5 n can be generated by
grouping terms (e.g. complex conjugate pair poles) in
the partial fraction expansion but then the decomposition is non-unique. One possibility is to separate
different dominant pole or (complex) pole pairs into
different Gj(z).
In a similar manner to the interval partition
analysis, the fact that eTQGKe is positive definite on
R(G) ensures the following.
Theorem 10: Using the above notation, if w  0, then
the resultant multi-parameter Parallel MIMO Gradient
Partition ILC Algorithm ensures that limk!1 ek 0 for
all initial u0 and for all achievable r.

6.5 Note on gradient computations

The formula given above requires the computation of
Gjv for some time series vector v and GTj e for some time
series vector e. The time series represented by Gjv is the
response of the dynamical system Gj to an input time
series v and, in a similar way to the SISO case (Owens
et al. 2008), it is easily seen from the formula
2
3
0 0    0 I
6 0 0  I 0 7

6
7
GTj 6
.. 7
6 ..
7
4.
.5
I 0    0 0
2
0
DTj
6 T T
6 Bj Cj
DTj
6
6 T T T
B A C
6
6 j j j
6
..
6
.
4
0





   BTj CTj

2
3
7 0 0    Im
0 7
76 0 0    0 7
76
7
0 76 .
.. 7
76 .
7
. 5
4 .
.. 7
7
. 5 I 0  0
m
DTj
66

that, if Gj has a realisation S(Aj, Bj, Cj, Dj), then the

times series GTj e is the times series obtained from the
time reversed response of ATj , CTj , BTj , DTj to the time
reversal of the input time series e.

7. Numerical examples
To illustrate the potential of alternative parameterisations and motivate the future exploration of new
parameterisations to meet a variety of design criteria,
this section considers the application of the proposed
parallel gradient partition to SISO systems based on
transfer function decomposition. The transfer function
decomposition chosen in all cases is that of partial

fraction expansion of zk Gz.
Example 1: Consider the simple example of a
second-order SISO system
Gs

2
s2 3s 2

67

1023

International Journal of Control

101

101

NOILC
Partition
Gradient

NOILC
Partition
Gradient

100

lg(||ek||)

lg(||ek||)

100
101

101
10

20

40

60

80

102

100

Figure 1. Example 1: comparison of convergence with

Q R 1.

with sampling interval h 0.1. The trial length assumed

is T 10 so that N T/Ts 100, zero initial conditions
are assumed and the reference signal is a sampled
version of the sine-wave r(t) sin(0.1t). Partial fraction
expansion is used with p 2. The initial input is chosen
to be u0 0. The aim is to compare, using simulation,
the performance of NOILC, the new multi-parameter
parallel gradient partition method and the singleparameter gradient-based algorithm (Owens et al.
2008) over 100 iterations to show long-term algorithm
performance and to demonstrate that a low number of
parameters p can achieve very good results comparable
to the performance of the underlying parent NOILC
problem. The weights chosen for this NOILC problem
are Q R 1. The results for the evolution of the
logarithmic error norm are shown in Figure 1. Note
that the partitioned gradient performs much better than
the single-parameter gradient method with faster
convergence, particularly over the first 10 iterations
where substantial error reduction is rapidly achieved.
Also, perhaps surprisingly, the partitioned gradient
method performs better in terms of error norm
reduction than its NOILC parent although there is
no theoretical reason why this should not occur as
POILC is only suboptimal with respect to the objective
function. Clearly, despite the low number p 2 of
parameters used, the proposed partitioned gradient
algorithm is a good approximation to NOILC in
this case.
The choice of Q and R do affect the detail of these
conclusions. For example, if a smaller R 0.1 is used,
the results are as shown in Figure 2. Here, NOILC now
outperforms the gradient partition algorithm and, as
expected, the gradient partition algorithm outperforms
the single-parameter version.

20

40

60

80

100

Trials

Figure 2. Example 1: comparison of convergence with Q 1,

R 0.1.

5
4
3
2
1
uref(t)

Downloaded by [University of Houston] at 19:31 14 January 2015

Trials

0
1
2
3
4
5
0

3
Time(s)

Figure 3. Example 2: first choice of reference signal r(t).

Example 2: Now consider the example of the nonminimum-phase electromechanical experimental rig
described in Cai, Freeman, Lewin, and Rogers
(2006). The transfer function is
Gs

165:954  s
s4 21:5s3 170:28s2 368:52s 663:82
68

with sampling time h 0.01. The trial length is T 6 so

that (N T/Ts 600), zero initial conditions are
assumed and the reference signal is as shown in
Figure 3. The initial input is chosen to be u0 0. The
simulation compares the performance of NOILC, the
gradient partition method and the gradient-based
algorithm over 100 iterations. The weights chosen are
Q 5, R 1 and the results are shown in Figure 4.

1024

D.H. Owens

100.7

100.6

lg(||ek||)

lg(||ek||)

NOILC
Partition
Gradient

NOILC
Partition
Gradient

100.5

10

100.4
0

20

40

60

80

100
0

Trials

20

40

60

80

100

Trials

Figure 6. Example 2: comparison of convergence with

second reference signal.

5
4
3
2
uref(t)

Downloaded by [University of Houston] at 19:31 14 January 2015

Figure 4. Example 2: comparison of convergence with first

reference signal.

1
0
1
2
3
4
5
0

3
Time(s)

Figure 5. Example 2: second choice of reference signal r(t).

Note that the gradient partition method outperforms

the gradient method and is a good approximation to
NOILC in this case. The poor residual error here is due
to the non-minimum-phase zero, a phenomenon that
has recently been explained (Owens and Chu 2010)
using an algorithm modelling process.
The results do depend on the reference signal
chosen. For example, using the shifted reference shown
in Figure 5 gives the results shown in Figure 6 where
the separation between the single-parameter gradient
algorithm and the others is reduced, but the multiparameter algorithm does provide a much better
replication of the fast convergence speed of the
parent NOILC approach, particularly over the first
1015 iterations.

8. Conclusions
This article has demonstrated the very strong
connection between the paradigms of NOILC

(Amann et al. 1996) and the much simpler POILC

(introduced in Owens and Feng (2003) for the SISO
case) by demonstrating that the solution to any linear
MIMO NOILC problem can be obtained as the
solution to an MIMO multi-parameter POILC problem constructed from plant dynamics and the
weighting matrices Q, R. This shows that, although
POILC was introduced for the SISO case as a simple,
conceptual approximation to SISO NOILC, it can be
concluded that an MIMO version of POILC is the
more general concept. NOILC is however still the
special case which has the advantage (and complexity)
of an implementation in terms of Riccati state feedback
and feedforward control functions.
The results have been shown to open up the
possibility of the creation of a wide range of MIMO
POILC algorithms that approximate a given MIMO
NOILC problem by a multi-parameter optimisation
problem. The details of such an approach have been
presented starting from the derivation of lifted
representations G of MIMO systems, an investigation
of the implications of the new and important MIMO
dynamic possibility that ker(G) 6 {0} and the derivation of explicit formulae for parameter vector and
input time series updates. In particular, the crucial
choice of POILC p  p weighting matrix Wk1 is
deduced from the chosen parameterisation, the specified NOILC weights Q, R and the norm of the previous
trial error kekk (supporting the original empirical
proposal of Owens and Feng (2003) that these should
be previous error norm and iteration dependent). It has
been shown that the NOILC property of monotonicity
of the error norm sequence is retained, but that it is
more commonly the case that the limit set S1 6 {0}.
Necessary and sufficient conditions for the convergence of the error to zero are derived and linked to the

Downloaded by [University of Houston] at 19:31 14 January 2015

International Journal of Control

choice of reference r and the properties of the
attainable parts of the set S1 \ R(G).
This article notes the infinity of possible approaches
to ILC design that this approach makes possible. It also
suggests that an exploration of carefully chosen
parameterisations may be a useful way forward in the
development of a wider range of ILC controllers. To
begin this process, this article has presented two such
classes based on sub-interval parameterisation and
gradient partition that generalise the previous singleparameter, SISO results in Owens and Feng (2003)
(including links to generalisations of the inverse model
approach (Harte et al. 2006)) and Owens et al. (2008).
All have well-defined theoretical outcomes (in terms of
monotonic convergence and limit set characterisation)
that are worthy of further exploration but they are not
unique. Numerical simulations for a simple secondorder system and a fourth-order model of a nonminimum-phase electromechanical experimental rig
described in Cai et al. (2006) are given. The results
support the conclusions of the theory and, in particular,
support the assertions (i) that multi-parameter POILC
can improve considerably on the performance of singleparameter POILC and (ii) that MIMO POILC has the
potential to produce very similar results to its parent
NOILC problem even if only a very small number
p  N 1  k* of parameters is used.
Finally, this article has concentrated on issues of
modelling, algorithm construction and convergence for
MIMO systems. The issues of robustness and the
effects of nonlinearity, noise and disturbances or
variations in initial conditions are also of importance
and will form part of further studies and research.

Acknowledgements
Thanks are due to Dr Bing Chu, currently with the School
of Electronics and Computer Science, University of
Southampton, for his help with the computational results.

References
Ahn, H.-S., Chen, Y., and Moore, K.L. (2007), Iterative
Learning Control: Brief Survey and Categorization, IEEE
Transactions on Systems, Man and Cybernetics, Part C, 37,
11091121.
Amann, N., Owens, D.H., and Rogers, E. (1996a), Iterative
Learning Control using Optimal Feedback and
Feedforward Actions, International Journal of Control,
65, 277293.
Amann, N., Owens, D.H., and Rogers, E. (1996b), Iterative
Learning Control for Discrete-time Systems with

1025

Exponential Rate of Convergence, Proceedings of the

IEE Control Theory Applications, 143, 217224.
Amann, N., Owens, D.H., and Rogers, E. (1998), Predictive
Optimal Iterative Learning Control, International Journal
of Control, 69, 203226.
Arimoto, S., Kawamura, S., and Miyazaki, F. (1984),
Bettering Operations of Robots by Learning, Journal of
Robotic Systems, 1, 123140.
Bristow, D.A., Tharayil, M., and Alleyne, A.G. (2006), A
Survey of Iterative Learning Control, IEEE Control
Systems Magazine, 26, 96114.
Cai, Z., Freeman, C.T., Lewin, P.L., and Rogers, E. (2006),
Iterative Learning Control for a Non-minimum Phase
Plant Based on a Reference Shift Algorithm, Control
Engineering Practice, 16, 633643.
Daley, S., Owens, D.H., and Hatonen, J. (2007), Application
of Optimal Iterative Learning Control to the Dynamic
Testing of Structures, Special Issue of the Proceedings of
the Institution of Mechanical Engineers, Part I (special issue
on Dynamic Testing), 221, 211222.
Harte, T.J., Hatonen, J., and Owens, D.H. (2006), Discretetime Inverse Model-based Iterative Learning Control:
Stability, Monotonicity and Robustness, International
Journal of Control, 78, 577586.
Hatonen, J., Owens, D.H., and Feng, K. (2006),
Basis Functions and Parameter Optimisation in Highorder Iterative Learning Control, Automatica, 42,
287294.
Kirchhoff, S., Schmidt, C., Lichtenberg, G., and Werner, H.
(2008), An Iterative Learning Algorithm for Control
of an Accelerator Based Free Electron Laser,
in Proceeding of the 47th IEEE Conference on Decision
and Control, Cancun, Mexico, 911 December,
pp. 30323037.
Owens, D.H., and Chu, B. (2010), Modelling of Nonminimum Phase Effects in Discrete-time Norm Optimal
Iterative Learning Control, International Journal of
Control, 83, 20122027.
Owens, D.H., and Feng, K (2003), Parameter Optimisation
in Iterative Learning Control, International Journal of
Control, 76, 10591069.
Owens, D.H., and Hatonen, J.J. (2005), Iterative Learning
Control an Optimisation Paradigm, IFAC Annual
Reviews in Control, 29, 5770.
Owens, D.H., Hatonen, J.J., and Daley, S. (2008), Robust
Monotone
Gradient-based
Discrete-time
Iterative
Learning Control, International Journal of Robust and
Nonlinear Control, 19, 634661.
Owens, D.H., Tomas-Rodriguez, M., and Daley, S. (2008),
Limit Sets and Switching Strategies in Parameter Optimal
Iterative Learning Control, International Journal of
Control, 81, 626640.
Ratcliffe, J.D., Lewin, P.L., Rogers, E., Hatonen, J.J., and
Owens, D.H. (2006), Norm-optimal Iterative Learning
Control Applied to Gantry Robots for Automation
Applications, IEEE Transactions on Robotics, 22,
13031307.