194

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 1, JANUARY 2011

Let I1 and I2 be the sets of time-intervals such that  (t) = x(t) if 2 I1 and  (t) = x(t +  ) if t 2 I2 . Note that I1 and I2 are mutually exclusive with (01; 1) as their union. Now
t

01

x(t) (t)N ( (t)) dt =

x(t)z (t)dt+
Term1

x(t)z (t +  )dt :
Term2

Since z = xN (x) is positive valued for all positive valued x, : 2 [0; rxz (0)]. By hypothesis, z = xN (x) increases monotonically with x if x  0. Hence, by Lemma 1, Term 2 2 [0; rxz (0)]. Hence
Term 1

[4] C. Desoer and M. Vidyasagar, Feedback Systems: Input-Output Properties. New York: Academic, 1975. [5] M. G. Safonov and V. V. Kulkarni, Zames-Falb multipliers for MIMO nonlinearities, Int. J. Robust Nonlin. Control, vol. 10, no. 11/12, pp. 10251038, Sep./Oct. 2000. [6] A. Rantzer, Friction analysis based on integral quadratic constraints, Int. J. Robust Nonlin. Control, vol. 11, no. 10/11, pp. 645652, Jun. 2001. [7] J. S. Markson and E. K. OShea, The molecular clockwork of a protein-based circadian oscillator, FEBS Lett., vol. 583, no. 24, pp. 39383947, 2009. [8] G. Stan, Global analysis and synthesis of oscillations: A dissipativity approach, Ph.D. dissertation, Univ. Liege, Liege, Belgium, Mar. 2005.

i.e.

01
;

x(t) (t)N ( (t)) dt

[0; 2rxz (0)]

Stochastic Tubes in Model Predictive Control With Probabilistic Constraints

Mark Cannon, Basil Kouvaritakis, Saa V. Rakovic, and s Qifeng Cheng

21rxz (0)

1
1

01

x(t) (t)N ( (t)) dt:

AbstractStochastic model predictive control (MPC) strategies can provide guarantees of stability and constraint satisfaction, but their online computation can be formidable. This difculty is avoided in the current technical note through the use of tubes of xed cross section and variable scaling. A model describing the evolution of predicted tube scalings facilitates the computation of stochastic tubes; furthermore this procedure can be performed ofine. The resulting MPC scheme has a low online computational load even for long prediction horizons, thus allowing for performance improvements. The efcacy of the approach is illustrated by numerical examples. Index TermsConstrained control, model predictive control (MPC), probabilistic constraints, stochastic systems.

Using the above inequality, we get

(1 + 21)rxz (0) 0 rxz ( )

 rxz (0) 0 rxz ( ) 

0 (using (A:5)):

+1

01

x(t) (t)N ( (t)) dt

Hence the proof. Q.E.D. C. Proof of Theorem 1 Consider x 2 L2 such that x(t)  0 8t 2 . Let y = N (x). Now
:

hx; M 3 N (x)iT

= hxT ; (M y )T i = hxT ; gyT i 0 hxT ; (h 3 y )T i = g rx y (0) 0 = (g

I. INTRODUCTION A common framework for robust model predictive control (MPC) is based on the assumption that model uncertainty (whether multiplicative or additive) can be described in terms of bounded compact sets (e.g., [13], [22]) without any specic reference to information that may be available on the probability distribution of the uncertainty. Yet this information (if available) has a key role to play in the treatment of soft constraints of a probabilistic nature. Such constraints can be handled through the use of second order cone inequalities (see for example [8], [20], both of which deal with the case of open loop stable systems with nite impulse responses), or condence ellipsoids (e.g., [15], [21]), or through estimates of the conditional mean of the state [23], or a transformation that leads to conditions involving standard multivariate normal distributions [12], or by multivariate integration [1]. However none of these approaches considered the issues of recursive feasibility or stability of the MPC strategy in closed loop operation. Recent work [3][5] considered constraints on the average probability of constraint violation over a given horizon. This involved minimizing a
Manuscript received September 28, 2009; revised September 29; 2010 accepted September 29, 2010. Date of publication October 14, 2010; date of current version January 12, 2011. Recommended by Associate Editor R. D. Braatz. M. Cannon, B. Kouvaritakis, and Q. Cheng are with the University of Oxford, Oxford OX1 3PJ, U.K. (e-mail: mark.cannon@eng.ox.ac.uk; basil.kouvaritakis@eng.ox.ac.uk; qifeng.cheng@st-hughs.ox.ac.uk). S. V. Rakovic is with the Institute for Automation Engineering, Otto-vonGuericke University of Magdeburg D-39106, Germany. Color versions of one or more of the gures in this technical note are available online at http://ieeexplore.ieee.org. Digital Object Identier 10.1109/TAC.2010.2086553

0 (1 + 21)khk1 ) rx y
Term1 1 h(t) x y (t)dt 01 Term2

01

h(t)rx y (0t)dt
(0)

(A6)

where x y (t) = (1 + 21)rx y (0) 0 rx y (0t). Now, consider the (A6). By assumption, g 0 (1+21)khk1 > 0. Further, rx y (0)  0 8x 2 L2 since N ( ) is positive valued for all  2 . Hence, Term 1  0. By Lemma 3, x y (t)  0 for all nite energy signals x satisfying x(t)  0 8t. By assumption, h(1)  0 and x(t)  0 8t. Hence, Term 2  0. Hence the proof. Q.E.D.

REFERENCES
[1] G. Zames, On the input-output stability of time-varying nonlinear feedback systemsParts I and II, IEEE Trans. Autom. Control, vol. AC-15, no. 4/7, pp. 228-238465-476, Apr./Jul. 1966. [2] G. Zames and P. Falb, Stability conditions for systems with monotone and slope-restricted nonlinearities, SIAM J. Control Optim., vol. 6, no. 1, pp. 89108, 1968. [3] J. Willems, The Analysis of Feedback Systems. Cambridge, MA: MIT Press, 1971.

0018-9286/$26.00 2010 IEEE

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 1, JANUARY 2011

195

performance index subject to constraints that restrict the predicted plant state either to nested ellipsoidal sets ([4]) or to nested layered tubes with variable polytopic cross sections ([3], [5]). By constraining the probability of transition between tubes and the probability of constraint violation within each tube, this strategy ensures satisfaction of soft (also hard) constraints as well as recursive feasibility with respect to these constraints. Tubes have been proposed for different MPC problem formulations: [10], [14] consider linear dynamics with additive bounded uncertainty, whereas [19] and [11] address nonlinear systems with and without additive uncertainty, respectively. The main drawback of [3], [5] is that the constraints on transitional and violation probabilities were invoked using condence polytopes for model uncertainty, and this results in large numbers of variables and linear inequalities in the MPC optimization, which can limit application to low-dimensional systems with short horizons even if the number of tube layers is small. Moreover, for small numbers of tube layers the handling of probabilistic constraints becomes conservative. The current technical note uses tubes with xed cross sections (for convenience ellipsoids are used, though more general forms are possible), but the scalings and centres of these cross sections are allowed to vary with time. As in the deterministic case [18], the evolution of the tubes is described by a scalar dynamic system, which implies a signicant reduction in the number of optimization variables. In this context the dynamics governing the tube scalings are stochastic, and the computation of the predicted distributions of tube scalings is facilitated using a process of discretization. The distributions enable bounds to be imposed on the probability of violation of state constraints; these are computed ofine and are invoked online by tightening the constraints on the predictions of a nominal model. The ofine computation of distributions for the stochastic variables allows many discretized levels to be used, thus allowing for the implicit use of many layered tubes without any concomitant increase in the online computation. Unlike [21] and [23], which did not guarantee closed loop feasibility, the current technical note ensures feasibility in a recursive manner. This means that, given feasibility at initial time, the proposed approach ensures that the online constrained optimization problem is feasible at the next time instant and therefore remains feasible at all subsequent times.

bility 1 (w.p. 1). To simplify presentation, our technical notes approach is developed for the case of a single soft constraint (nc = 1)

of nc constraint sets. The control problem is to minimize, at each time k , the expected quadratic cost

Pr(gT xk  h)  p; p > 0:5 (3) and the case of nc > 1 can be treated simply by taking the intersection
Jk =

i=0

T xT+i Qxk+i + uk+i Ruk+i k

(4)

subject to Pr(g T xk+i  h)  p for all i > 0, while ensuring closed loop stability (in a suitable sense) and convergence of xk to a neighbourhood of the origin as k ! 1. We decompose the state and input of the model (1) into

xk

= zk + ek ; uk = Kxk + ck
zk+1 = 8zk + Bu ck ek+1 = 8ek + Bw wk

(5)

so that zk and ek evolve according to (6) (7)

tion at time k . This allows the effect of disturbances on the i-stepahead predicted state, xk+i , to be considered separately (via ek+i ) from the nominal prediction, zk+i , and thus simplies the handling of constraints. III. UNCERTAIN PREDICTIONS The uncertainty in the i-step-ahead prediction ek+i can be characterized using (2) and (7) in terms of the scalings i of ellipsoidal sets containing ek+i , denoted ek+i 2 E (V; i )

where 8 = A + Bu K is assumed to be strictly stable and fck+i ; i = 0; 1; . . . ; N 0 1g are the free variables in a receding horizon optimiza-

E (V; i ) = fe : eT V e  i g;

= V T  0:

(8)

II. PROBLEM DEFINITION Consider the linear time-invariant model with state xk trol input uk , and disturbance input wk

n , con-

This section constructs a dynamic system to dene the random sequence fi ; i = 1; 2; . . .g and proposes a method of approximating numerically the distribution functions of i , i  1. Given ek+i 2 E (V; i ) and wk+i 2 E (W; ) we have ek+i+1 2 E (V; i+1 ) if and only if
e w T max 2E ((V; )) (8e + Bw w) V (8e + Bw w)  i+1 : 2E W;

(9)

xk+1 = Axk + Bu uk + Bw wk :

(1)

Here wk , for k = 0; 1; . . ., are zero-mean, independent and identically distributed (i.i.d.) random variables, and it is known that wk 2 E (W; k ) where E (W; k ) is an ellipsoidal set

The problem of computing the minimum i+1 satisfying (9) is NP-complete, but sufcient conditions for (9) are as follows. Lemma 1: If i , i+1 and V satisfy

E (W; k ) = fw : wT W w  k g W = W T  0:

(2)

i+1 = i + i 01 0 1 8V 01 8T 0 Bw W 01 Bw  0 T V 

(10) (11)

Here k  0, k = 0; 1; . . ., are i.i.d. random variables. The distribution function F (a) = Pr(k  a) is assumed to be known, either in closed form or by numerically integrating the density of w , and we make the following assumptions on .  Assumption 1: F is continuously differentiable and  2 [0; ]. T The system is subject to soft constraints of the form Pr(gj xk  hj )  pj , j = 1; . . . nc , for given gj 2 n , hj 2 and given probabilities pj > 0:5. General linear constraints on states and inputs can be handled using the technical notes framework, and hard constraints can be included as a special case of soft constraints invoked with proba-

for some  > 0, then ek+i+1 2 E (V; i+1 ) whenever ek+i 2 E (V; i ) and wk+i 2 E (W; i ). Proof: Using the S-procedure [2], sufcient conditions for (9) are obtained as
T Bw

8T

[8

Bw ]  

and i+1  i + i for some  > 0 and  > 0. However  can be removed from these inequalities by scaling i , i+1 and V , and conditions (10) and (11) follow directly.

V [I

0 0] +  I

W [0 I ]

196

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 1, JANUARY 2011

The distribution of i for any i > 0 can be determined from the distributions of  and 0 using (10). In the sequel , V in (10) (11) are taken to be constants independent of i , i (a procedure for optimizing the values of  and V is described in Section IV). We further assume that the distribution function, F , of 0 is known and has the following properties. Assumption 2: F is right-continuous with only a nite number of  discontinuities and 0 2 [0; 0 ]. Note that Assumption 2 allows for xed, deterministic 0 (e.g., 0 = 0 corresponds to F (x) = 1 for all x  0). The following result shows that F is well-dened if  2 (0; 1). Theorem 2: If 0 <  < 1, then:     (i) i is bounded for all i : i 2 [0; i ] , where i+1 = i +  ,     and i   = maxf0 ; (1=1 0 ) g; (ii) F is continuously differentiable for all i  1; (iii) i converges in distribution to the random variable L = k=0 k k as i ! 1.  Proof: Assumption 1 and (10) together imply that i 2 [0; i ] i+1 = i + . Hence, if  2 (0; 1), then fi g is monotonic   with    and i ! (1=1 0 ) as i ! 1, which proves (i).  The distribution of i+1 is given for any i  0 by the convolution integral (see e.g., [6])

for i = 0; 1; . . . ; N 0 1, where the ( + 1) 2 ( + 1) elements of the matrix P are determined by the particular numerical integration scheme employed and the density, f , of . If the data points xj satisfy jxj +1 0 xj j   for j = 0; . . . ;  0 1, then the approximation error can be made arbitrarily small if  is suf^ ciently small, as we show below. We denote F ; as the piecewise ^ ; (x) = i;j for x 2 [xj ; xj +1 ), j = constant function with F 0; . . . ;  0 1. ^ Lemma 3: For any nite horizon N we have F ; ! F as  ! 0 ^ for i = 1; . . . ; N . Also F ; ! F as  ! 0, where L is the eigenvector of P associated with eigenvalue 1. Proof: By Assumptions 1 2 and Theorem 2, the integrand in (15) is piecewise continuous for i = 0 and continuous for i > 0. It can therefore be shown that, as  ! 0
x

^ max jF

(x) =

  0

(17) where m denotes the number of discontinuities in F and ,   1 are constants dictated by the numerical integration scheme employed. This ^ implies F ; ! F as  ! 0 since m and N are nite. Combining the error bounds used to derive (17) with i+1 = P i , we have, as !0

2[0 ]
;

;

(x) 0 F (x)j = O m + i  ;

i = 1; . . . ; N

F (x 0 y) f

y 

dy

(12)

^ F

;

(x) = 

  0

^ F ; (y) f (x 0 y) dy + O(  )

where f denotes the density of i . It follows from Assumption 1 is continuously and the dominated convergence principle that F differentiable, as claimed in (ii). To prove (iii), let i denote the random variable i = i + i 0 for i = 0; 1; . . . where

and since L satises L = P L , it follows that, as 

!0
(18)

^ F

;

(x) = 

  0

^ F

;

(y ) f (x 0 y ) dy + O(  ) :

i+1 = i + i i 0 = 0:
i k i k i k k i k i k i i i

(13)

But by Theorem 2, F is the unique solution of

01 +  , and since = Then  0 = =0 01   010 + 0wheref0g is i.i.d., the (10) gives of  distributions =0 and  0 are identical for all i  0. Furthermore the bounds on  and
i

F (x) = 

  0

F (y) f (x 0 y) dy :

0 in Assumptions 1 and 2 imply that for every  > 0 there exists n

such that

0 Pr(jj

0 i0 j < ) = 1

and it follows that i converges a.e. to a limit as i ! 1 [17]. From the denition of L , we therefore have i ! L a.e., and hence i converges in distribution to L [17]. The simple form of (10) facilitates the approximate computation of the distribution of i . Consider for example a numerical integration scheme based on a set of points fxj ; j = 0; 1; . . . ; g in the interval  [0;  ] with

8 i > n; j > i

 0 = x0 < x1 < 1 1 1 < x = :

(14)

Let i;j be an approximation to F (x) in the interval x 2 [xj ; xj +1 ) for j = 0; . . . ;  0 1, and let i; = 1. Then since the convolution in (12) can be written equivalently as

(x) = 

  0

F (y) f (x 0 y) dy

(15)
T

^ Therefore (18) implies F ; ! F as  ! 0. Lemma 3 shows that the eigenvector L provides an approximation to the steady state distribution of (10) despite the linear growth of the error bound in (17) with N . In the sequel we assume that  is chosen sufciently small that the approximation errors associated with i for i = 0; . . . ; N and L may be neglected. Remark 1: The matrix T 1 P T , where T is a lower-triangular matrix of 1s, is the transition matrix associated with a Markov chain. The (j; k)th element of T 1 P T gives the probability of xj 1  i+1 < xj given xk 1  i < xk . The elements of T 1 P T are therefore non-negative and each column sums to 1, so P necessarily [16] has an eigenvalue equal to 1. Remark 2: The denitions of P and 0 ensure that each i generated by (16) belongs to the set = f 2 +1 : 0  0  1 1 1  +1 = 1g. We next show how the sequence fi g can be used to construct ellipsoidal sets that contain the predicted state of (7) with a specied level of condence. For a given sequence fxj g satisfying (14), and for any  2 and p 2 [0; 1], let ind(; p) and b(; p) denote the functions that extract respectively the index and the value of xj corresponding to a condence level p

a generic quadrature method enables the vectors i = i;0 1 1 1 i; to be computed for i = 1; . . . ; N by setting 0;j = F (xj ) for j = 0; . . . ;  and using the recursion

ind(; p) = minfj : j  pg b(; p) = xj j = ind(; p):

(19a) (19b)

i+1 = P i

(16)

E V; b(i ; p)

Theorem 4: The i-step-ahead prediction ek+i satises ek+i with probability p.

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 1, JANUARY 2011

197

Proof: This follows from ek+i 2 E (V; i ), since, if 0 is determined using the known distribution for 0 , then by construction i  b(i ; p) holds with probability p. Theorem 4 implies that the predicted values of ek+i lie inside tubes with ellipsoidal cross sections (dened by V and b(i ; p)) with probability p; hence we refer to these tubes as stochastic tubes. Although the stochastic tubes apply to ek+i , they can be referred to xk+i through the use of (5), which translates their centres to the nominal state prediction zk+i . IV. MPC STRATEGY To avoid an innite dimensional optimization problem, we adopt the usual dual prediction mode MPC paradigm [13]: at time k , mode 1 consists of the rst N steps of the prediction horizon over which the control moves ck+i in (6) and (7) are free, and mode 2 comprises the remainder of the horizon, with ck+i = 0 for all i  N . Because of the additive uncertainty appearing in (1), the quadratic cost (4) is unbounded. Hence, the modied cost of [3] is used
Jk =

Note that the replacement of jg T ej by its maximum value g T V 1 g describes the process of constraint tightening whereby the constraint g T zk+i k  h on the nominal predicted state is replaced by the tighter condition given in (22). Although (22), when invoked for i = 1; . . . ; N , ensures that at time k the predicted sequence fxk+i k g satises (3), it does not ensure recursive feasibility, namely that there exists a predicted sequence at time k + 1 satisfying (3). This is because constraints of the form Pr(g T xk+i k  h)  p do not guarantee that g T xk+i k+1  h holds with probability p. However the future state is necessarily contained at all times within the robust tube associated with upper bounds on i :
b1=2

ek+i k

j 2E

V; b(i 0 ; 1)

b(i 0 ; 1) =

0 i   10

8i  1

(23)

i=0

k (Lk+i ) 0 Lss

Lss = lim
i

!1

k (Lk+i )

(20)

T where Lk+i = xT+i Qxk+i + uk+i Ruk+i . This cost is shown in [3] k to be a quadratic function of the vector ck of free variables fck+i ; i = 0; . . . ; N 0 1g at time k

with probability 1. The stochastic tubes that dene constraints such as (22) at times k + 1; k + 2; . . . are therefore based on initial distributions for 0 which can be inferred from the robust tube at time k , and this enables the construction of constraints to ensure recursive feasibility. Let i j denote the discrete approximation to the distribution of i that is obtained if j (j < i) is equal to the upper bound b(j 0 ; 1), so that F (x) = 0 for x 2 [0; b(j 0 ; 1)) and F (x) = 1 for x  b(j 0 ; 1). Then applying (16) for j; . . . ; i 0 1 gives

i j = P i j j j j j = u ind(j 0 ; 1)

0 j

(24)

Jk = [ xT k

cT k

1 ]
[ xk

cT k

1]

(21)

for a suitably dened positive denite matrix

. Constraints are invoked explicitly in mode 1 and implicitly, through the use of a terminal set, S , in mode 2. A. Constraint Handling in Mode 1 The distribution of 0 is dictated by the information available on the plant state at the beginning of the prediction horizon. Assuming that xk is known at time k , we set zk = xk , ek = 0 and 0 = 0, so that F (x) = 1 for all x  0. Then, using v k to denote the corresponding predictions of variable v at time k , we have from (7)

where u(j ) = [0 1 1 1 0 1 1 1 1 1]T denotes the j th column of the lowertriangular matrix of 1s. Lemma 6: For any i > j , the value of ek+i k+j predicted at time k (given xk ) satises

ek+i k+j

2E

V; b(i j ; p)

with probability p:

(25)

Proof: From ek+j +1 = 8ek+j + Bw wk+j and (23), the prediction at time k of ek+j +1 k+j satises

1j

ek+j +1 k+j

2 8E

V; b(j 0 ; 1)

8E

V; b(1 0 ; p) w:p: p

ek k = 0

ek+ijk = 8 01 Bw wk + 1 1 1 + Bw wk+i01
i

8i  1:

where 8 denotes Minkowski addition. But, by Lemma 1 and (24) we have

8

These predictions are independent of k , so the sets that dene the stochastic tube containing ek+i k can be computed ofine, namely ek+i k 2 E V; b(i 0 ; p) with probability p, where

V; b(j 0 ; 1)

8E

V; b(1 0 ; p)

E

V; b(j +1 j ; p)

which establishes (25) for i = j + 1. Similarly

ek+j +2 k+j

i 0 = P i 0 0 ;

i = 1; 2; . . . ;

0 0 = [ 1

111

1] :

2 8E

V; b(j +1 j ; p)

8E

V; b(1 0 ; p) w:p: p

In this setting the probabilistic constraint of (3) can be enforced as follows. Lemma 5: The constraint Pr(g T xk+i k  h)  p for p > 0:5 is ensured by the condition

g T zk+i k

j h0

b(i 0 ; q )

1=2

gT V

01 g j j

(22)

so ek+j +2 k+j 2 E V; b(j +2 j ; p) with probability p, and (25) follows by induction for all i > j . A simple way to ensure the recurrence of feasibility is to require the probabilistic constraints (3) to hold for all future predictions. Lemma 6 shows that the worst case predictions of (25) for j = 0; 1; . . . can be handled by selecting at each prediction time i the largest scaling:
i (q ) = max b( ; q ); . . . ; b( b i0 ii

where q = 2p 0 1 and zk+i+1 k = 8zk+i k + Bu ck+i k for i = 0; 1; . . ., with zk k = xk . Proof: Assume that the distribution of g T ek+i k is symmetric about 0 (note that there is no loss of generality in this assumption because of the symmetric bound employed in (8)). Then g T zk+i k + g T ek+i k  h holds with probability p > 0:5 iff g T zk+i k + jg T ek+i k j  h with probability q = 2p 0 1. The sufcient condition (22) then follows from ek+i k 2 E V; b(i 0 ; p)

j 01 ; q)g
q = 2p 0 1:

(26)

b and by replacing b(i 0 ; q ) with i (q ) in (22) g T zk+i k

j j

j j

j h0

i (q ) b

1=2

gT V

01 g;

(27)

and maxe

2E (V;b) jgT ej = b1=2

gT V

01 g .

The sets E (V; i (p)), for i = 1; 2; . . ., dene the cross sections of a b recurrently feasible stochastic tube. The preceding argument is summarized as follows.

198

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 1, JANUARY 2011

Theorem 7: If at k = 0 there exists c0 satisfying (27) for all i > 0, then for all k > 0 there exists ck satisfying (27) for all i > 0, and hence the probabilistic constraint (3) is feasible for all k > 0. Proof: Follows directly from Lemma 5 and Lemma 6. Remark 3: Note that the computation of recurrently feasible stochastic tubes does not depend on the information available at time k , and hence can be performed ofine. Consequently the online computation does not depend on the dimension of  , so this can be chosen sufciently large that the approximation errors discussed in Lemma 3 are negligible. B. Constraint Handling in Mode 2 The constraint handling framework of mode 1 can also be used to dene a terminal constraint, zk+N jk 2 S , which ensures that (27) is satised for all i > N . Since (27) constitutes a set of linear constraints on the deterministic predicted trajectory fzk+ijk g, the innite horizon of mode 2 can be accounted for using techniques similar to those deployed in [7] for the computation of maximal invariant sets. Before describing the construction of S , we rst derive some fundamental properties of the constraints (27). Lemma 8: The scaling of the tube cross section dened in (26) satb b ises i (q )  i+1 (q ) for all i and all q 2 [0; 1], and converges as i ! 1 to a limit which is bounded by
i!1

is bounded. This involves solving a sequence of linear programs to nd b1=2 the smallest integer n3 such that g T 8n +1 z  h 0 1 g T V 01 g T 8j z  h 0 1=2 g T V 01 g for j = 1; . . . ; n3 , for all z satisfying g b1 then SN in (31) is given by ^

^ SN = fz : g T 8i z  h 0 [N +i (q )]1=2 g T V 01 g; i = 1; . . . ; N b ^
b1= g T 8N +j z  h 0 12 g T V 01 g; j
^

 1; . . . ; n g:
3

(32)

Remark 4: The scalings of the tube cross sections in (31) are computed on the basis of (10); this is done for convenience given the simple form of (10). However tighter tubes would be obtained if (9) were used in place of (10) to generate the robust bounds b(ij0 ; 1) of (23) and the b probabilistic bounds i (q ) of (26). The implied computation is practicable even though (9) involves an implicit optimization because of the discretization of the range for  in (14). C. Stochastic MPC Algorithm

V and . A possible choice is given by the pair V;  that minimizes the

The ofine computation of the tube scalings requires knowledge of

steady state effect of the uncertainty on the constraints (27). This effect b is given by the steady state value of [i (q )]1=2 g T V 01 g , which by  (28) is [ =(1 0 )]1=2 g T V 01 g , and this suggests dening V;  via

b lim i (q)   b

Proof: The monotonic increase of the scaling b(ij0 ; 1) of the robust tube with i in (23) and the denition of j jj in terms of the robust tube in (24) implies (by Lemma 1) that b(P i0j j jj ; q )  b(P i0j j +1jj +1 ; q ), and hence b(ijj ; q )  b(i+1jj +1 ; q ) for b b any i > j . This implies i (q )  i+1 (q ) for all i, and since i (q)  b(ij0 ; 1) for all i, i (q) must converge as i ! 1 to a limit b b b no greater than the asymptotic value of the robust tube scaling, 1 . Lemma 9: Let S1 be the maximal set with the property that (27) is satised for all i > N whenever zk+N jk 2 S1 , then a sufcient condition for S1 to be non-empty is

 = b 1 0 :
1

(28)

(V

01

; ) = arg

h  12 g T V 01 g : b1=

(29)

For a xed value of  > 0, (33) is a semidenite program (SDP) in the variable V 01 , and since  is univariate and restricted to the interval (0; 1), (33) can be solved by solving successive SDPs for V 01 with alternate iterations of a univariate optimization (such as bisection) for . Given the denition of the cost (21), the constraints (27), and the terminal set (31), the MPC algorithm is as follows. Algorithm 1 (Stochastic MPC): Ofine: Compute V ,  via (33). ^ Determine P in (16), and hence, for i = 1; . . . ; N + N , compute i;j , i (q) in (26). Compute the j = 0; . . . ; i 0 1 in (24) and the scalings b smallest n3 such that SN in (31) can be expressed as (32). ^ Online: At each time-step k = 0; 1; . . . 1) solve the quadratic programming (QP) optimization

1 min (1 0 ) gT V ;

01

subject to (11):

(33)

Proof: Satisfaction of constraints (27) over the innite horizon of mode 2 requires that

1=2 b 8 z  h 0 N +i (q)

g T V 01 g;

i = 1; 2; . . .

(30)

where for convenience the initial state of mode 2 is denoted as z . By b Lemma 8, i (q ) increases monotonically with i, and 8 is strictly stable by assumption, so (29) is obtained by taking the limit of (30) as i ! 1 and using the upper bound in (28). b b Lemma 8 implies that N +i (q ) in (30) can be replaced by 1 for all ^ ^ i  N , for some nite N , in order to dene the terminal set. In this setting, the terminal set is given by S = SN , where ^

and the quadratic stability criterion

= arg min Jk subject to (27) for i = 1; . . . ; N 0 1 c and zk+N k 2 SN (34) ^ 2) implement uk = Kxk + ck . Theorem 10: Given feasibility at k = 0, Alg. 1 is feasible for all k  1. The closed loop system satises the probabilistic constraint (3)
c3 k
j 3

1 lim n n
!1

n k=0
0

(Lk )  Lss :

(35)

^ SN = fz : g T 8i z  h 0 [N +i (q )]1=2 g T V 01 g; i = 1; . . . ; N b ^
g T 8N +j z
^

 h 0 1=2 b
1

g T V 01 g; j

= 1; 2; . . .g:

(31)

The constraints in (31) are necessarily conservative, but the implied constraint tightening can be made insignicant with sufciently large ^ N . This follows from the assumption that 8 is strictly stable, which ^ implies that SN ! S1 as N ! 1. ^ Although (31) involves an innite number of inequalities, SN has the ^ form of a maximal admissible set. The procedure of [7] can therefore be used to determine an equivalent representation of SN in terms of a ^ nite number of inequalities, the existence of which is ensured if S1

Proof: Recurrence of feasibility follows from Theorem 7, which implies that the tail: ck+1jk = fck+1 ; . . . ; ck+N 01 ; 0g of the minimizing sequence: c3 = fck ; . . . ; ck+N 01 g of (34) at time k is feasible k for (34) at time k + 1. The feasibility of the tail also implies that the 3 3 optimal cost satises the bound: Jk 0 k (Jk+1 )  Lk 0 Lss , which implies (35). V. ILLUSTRATIVE EXAMPLES Example 1: Consider the second order system with

1:8 00:5 01:5 1:2 Pr(gT xk  h)  p;

A=

= 2 ; 1 g = [01:75 1]T ;
Bu

Bw

= I; h = 1:5;

=I p = 0:8:

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 1, JANUARY 2011

199

Algorithm 1 meets constraints with probability greater than 0.8, while allowing some constraint violations (17% at k = 1). As expected, unconstrained LQ-optimal control achieves lower average cost
1 Jcl = nlim n !1

n01 k=0

( k

T xT Qxk + uk Ruk ) 0 Lss

Fig. 1. Closed loop responses of Algorithm 1 and unconstrained LQ-optimal control for 10 disturbance sequence realizations, showing ellipsoidal tube cross-sections predicted at k = 0 for p = 1 and p = 0:8.

the average values over 200 realizations of the disturbance sequence being Jcl = 306 for LQ-optimal control, and Jcl = 330 for Algorithm 1, but this is at the expense of violating the probabilistic constraint (LQoptimal control gave 100% violations at k = 1). Conversely, robust MPC (with p = 1) ensures zero violations, but achieves this at the expense of higher average cost: Jcl = 360. The online optimization is a QP with six optimization variables and 13 constraints, requiring an average CPU time of 3.5 ms (Matlab/2.4 GHz processor). By contrast, the approach of [20] requires the solution a second order cone program (SOCP), which for the same number of free variables and constraints requires an average of 105 ms CPU time. Example 2: This example considers the dcdc converter system described in [9]. For the operating point dened by x0 = 0:389 A, x0 = 2 1 016 V, u0 = 0:516, sampling interval T = 0:65 ms and problem parameters Vin = 15 V, R = 85
, L = 4:2 mH, C = 2200 F, the linearized state space model matrices are given by

A = 001143 :

0 0075 0 996

: :

4 798 ; Bu = 0::115 ; Bw = I; W = I:

The LQG-optimal feedback gain for Q = diagf1; 10g, R = 1 is K = [ 0:211 0:0024 ]. Fluctuations in the inductor current, which arise due to random variations in the source input voltage, are subject to the probabilistic constraint dened by Prf[1 0](xk 0 x0 )  2g  0:8. For additive uncertainty with covariance W 01 =252 , and  = 0:02, the  solution of (33) is

 = 0:980; V

1 458 7 30

: :

7 30

114 8

: :

Fig. 2. Closed loop responses of Robust MPC and unconstrained LQ-optimal control for 10 disturbance sequence realizations.

The disturbance wk has a truncated normal distribution, with T zero mean, covariance (wk wk ) = W 01 =122 and bounds 02:68  (wk )1;2  2:68. The distribution of  is then a modi ed -squared distribution with  = 0:1. The cost weights are Q = I , R = 5, and K in (5) is chosen as the unconstrained LQ-optimal feedback gain: K = [1:518 0 0:645]. The ofine optimization of V;  in (33) gives

b The robust tube has asymptotic scaling 1 = 1:02, and horizons N = 0 ^ 8, N = 9 were employed. For initial condition x0 0 x = (2:5; 2:8) 3 and 10 disturbance sequence realizations, Algorithm 1 produced an average cost of Jcl = 221, as compared with Jcl = 230 for robust MPC (with p = 1) and Jcl = 166 for unconstrained LQ-optimal control in closed loop operation. Robust MPC gave no constraint violations (Fig. 2), which is clearly over-cautious given the limit of 20% violations at each sample, whereas Algorithm 1 (Fig. 1) allowed constraint violations with a frequency of 14.4% over k = 1; . . . ; 6, and LQG ignored the constraint (100% violation rate for k  3).
VI. CONCLUSION The proposed stochastic MPC strategy handles probabilistic constraints with online computational load similar to that of nominal MPC. This is achieved by xing the cross sectional shape of a tube containing predicted trajectories, but allowing the centres and scalings of the cross sections to vary. The resulting MPC law ensures recursive feasibility and convergence.

 = 0:41; V

00:348

0 782

00:348 : 0:168

b The robust tube has asymptotic scaling 1 = 0:170, and the interval [0, 0.173] is divided into 1000 equal subintervals to dene fxj g and ^ b hence compute P and i (q ). Horizons N = 6; N = 7, give n3 = 0 and the scalings of the recurrently feasible stochastic tube are

REFERENCES
[1] H. Arellano-Garcia and G. Wozny, Chance constrained optimization of process systems under uncertainty: I. Strict monotonicity, Comp. Chem. Eng., vol. 33, no. 10, pp. 15681583, 2009.

fi (q)g = f0:31; 4:41; 6:09; 6:78; 7:06; 7:18; . . . 7:26g 2 1002 : b

200

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 1, JANUARY 2011

[2] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory. Philadelphia, PA: SIAM, 1994. [3] M. Cannon, B. Kouvaritakis, and D. Ng, Probabilistic tubes in linear stochastic model predictive control, Syst. Control Lett., vol. 58, no. 10, pp. 747753, 2009. [4] M. Cannon, B. Kouvaritakis, and X. Wu, Probabilistic constrained MPC for multiplicative and additive stochastic uncertainty, IEEE Trans. Autom. Control, vol. 54, no. 7, pp. 16261632, Jul. 2009. [5] M. Cannon, D. Ng, and B. Kouvaritakis, Successive linearization NMPC for a class of stochastic nonlinear systems, in Nonlinear Model Predictive Control: Towards New Challenging Applications. New York: Springer, 2009, vol. 384, Lecture Notes in Control and Information Sciences, pp. 249262. [6] W. Feller, An Introduction to Probability Theory and Its Applications. New York: Wiley, 1971, vol. 2. [7] E. G. Gilbert and K. T. Tan, Linear systems with state and control constraints: The theory and practice of maximal admissible sets, IEEE Trans. Autom. Control, vol. 36, no. 9, pp. 10081020, Sep. 1991. [8] D. E. Kassmann, T. A. Badgwell, and R. B. Hawkins, Robust steadystate target calculation for model predictive control, AIChE J., vol. 46, no. 5, pp. 10071024, 2000. [9] M. Lazar, W. Heemels, B. Roset, H. Nijmeijer, and P. Van Den Bosch, Input-to-state stabilizing sub-optimal NMPC with an application to dcdc converters, Int. J. Rob. Nonlin. Control, vol. 18, no. 8, pp. 890904, 2008. [10] Y. I. Lee and B. Kouvaritakis, Robust receding horizon predictive control for systems with uncertain dynamics and input saturation, Automatica, vol. 36, no. 10, pp. 14971504, 2000. [11] Y. I. Lee, B. Kouvaritakis, and M. Cannon, Constrained receding horizon predictive control for nonlinear systems, Automatica, vol. 38, no. 12, 2002. [12] P. Li, M. Wendt, and G. Wozny, A probabilistically constrained model predictive controller, Automatica, vol. 38, no. 7, pp. 11711176, 2002. [13] D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert, Constrained model predictive control: Stability and optimality, Automatica, vol. 36, no. 6, pp. 789814, 2000. [14] D. Q. Mayne, M. M. Seron, and S. V. Rakovic, Robust model predictive control of constrained linear systems with bounded disturbances, Automatica, vol. 41, no. 2, pp. 219224, 2005. [15] Z. K. Nagy and R. D. Braatz, Robust nonlinear model predictive control of batch processes, AIChE J., vol. 49, no. 7, pp. 17761786, 2003. [16] J. R. Norris, Markov Chains. Cambridge, U.K.: Cambridge Univ. Press, 1997. [17] A. Papoulis, Probability, Random Variables and Stochastic Processes. New York: McGraw-Hill, 1965. [18] S. V. Rakovic and M. Fiacchini, Approximate reachability analysis for linear discrete time systems using homothety and invariance, in Proc. 17th IFAC World Congr., Seoul, Korea, 2008, pp. 1532715332. [19] S. V. Rakovic, A. R. Teel, D. Q. Mayne, and A. Astol, Simple robust control invariant tubes for some classes of nonlinear discrete time systems, in Proc. 45th IEEE Conf. Dec. Control, San Diego, CA, 2008, pp. 63976402. [20] A. Schwarm and M. Nikolaou, Chance constrained model predictive control, AIChE J., vol. 45, no. 8, pp. 17431752, 1999. [21] D. H. van Hessem and O. H. Bosgra, A conic reformulation of model predictive control including bounded and stochastic disturbances under state and input constraints, in Proc. 41st IEEE Conf. Dec. Control, Las Vegas, NV, 2002, pp. 46434648. [22] Y. J. Wang and J. B. Rawlings, A new robust model predictive control method I: Theory and computation, J. Proc. Control, vol. 14, pp. 231247, 2004. [23] J. Yan and R. Bitmead, Incorporating state estimation into model predictive control and its application to network trafc control, Automatica, vol. 41, no. 4, pp. 595604, 2005.

Output Consensus of Heterogeneous Uncertain Linear Multi-Agent Systems

Hongkeun Kim, Hyungbo Shim, and Jin Heon Seo

AbstractThis technical note studies the output consensus problem for a class of heterogeneous uncertain linear multi-agent systems. All the agents can be of any order (which might widely differ among the agents) and possess parametric uncertainties that range over an arbitrarily large compact set. The controller uses only the output information of the plant; moreover, the delivered information throughout the communication network is also restricted to the output of each agent. Based on the output regulation theory, it is shown that the output consensus is reached if the (state) consensus is achieved within the internal models among the agents controllers (even though the plants outputs, rather than the internal models outputs, are communicated). The internal models can be designed and embedded into the controller, which provides considerable exibility to designers in terms of the type of signals that are agreed on among the agents. Index TermsHeterogeneous multi-agent systems, internal models, output consensus, output feedback.

I. INTRODUCTION For a decade, the consensus problem is actively studied due to its numerous applications such as cooperative control of unmanned aerial vehicles, communication among sensor networks, and formation of mobile robots (see [2], [3], [8][12], [14] and the references therein). While most of the results have focused on the homogeneous multiagent systems, only a few papers [2], [9], [10] and a book [8] considered heterogeneous cases. In particular, Chopra and Spong [2] dened the output synchronization problem and presented a solution for weakly minimum phase nonlinear systems having relative degree one, and Qu et al. [8][10] solved an output consensus problem under a dynamically changing communication network. Although they geared up for output consensus problem about nonlinear heterogeneous systems, there are also some limitations such as: i) the agreement achieved among the agents is on a constant value [2], [9], [10] or a ramp signal [8] (using a virtual leader apporach), ii) uncertainties are not taken into account [2], [8][10], and iii) state information should be available to local-level controllers [2], [8][10]. In this technical note, although limited to linear SISO systems and xed network topology, we newly propose a different route to handle the problem. First, by introducing an internal model into the consensus problem, we extend the class of signals that will be agreed among the agents. The synchronized signal can be any time-varying one if it is _ the output of an autonomous linear system w = Sw . Next, it is shown that large uncertainties and the heterogeneity can be effectively dealt with by the output regulation theory (that are well established in [1], [4], [5], [13], [14]). In particular, all the agents can be of any order which might even be different among the agents and have parametric
Manuscript received July 27, 2009; revised May 06, 2010; accepted September 26, 2010. Date of publication October 21, 2010; date of current version January 12, 2011. This work was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology under Grant 2010-0001966. Recommended by Associate Editor I. Paschalidis. The authors are with the ASRI, School of Electrical Engineering, Seoul National University, Seoul 151-600, 151-744, Korea (e-mail: hkkim@cdsl.kr; h.shim@ieee.org, hshim@snu.ac.kr; jhseo@snu.ac.kr). Color versions of one or more of the gures in this technical note are available online at http://ieeexplore.ieee.org. Digital Object Identier 10.1109/TAC.2010.2088710

0018-9286/$26.00 2010 IEEE