Optimisation and Optimal Control: Exercises 2 (Optimal control)

Question 1: Consider the optimal control problem defined over a fixed time horizon 0 t T :
Z T
min J[u] = M (x(T )) +
L(x(t), u(t))dt
0

subject to the system dynamic constraints x = f (x(t), u(t), x(0) = x0 . Under appropriate
assumptions, a necessary condition for a minimising solution u = u? (t) is:
H
H
H
M (T )
= 0; x =
= f (x, u), x(0) = x0 ; p =
, p(T ) =
u
p
x
x
where H = H(x, p, u) = L(x, u) + pT (t)f (x, u) denotes the Hamiltonean of the system and p(t)
the co-state vector of the system (Lagrange multipliers). Consider now the LQR problem over
a fixed time horizon:
Z
1 T
1 T T
min J[u] = x (T )Sx(T ) +
(x Qx + uT Ru)dt
2
2 0
where S = S T 0, Q = QT 0 and R = RT > 0. Starting from the stated optimality
conditions for the general problem, show that the LQR problem reduces to the solution of the
following boundary-value problem:
!
! !
x
A BR1 B T
x
=
T
p
Q
A
p
subject to the boundary conditions x(0) = x0 and p(T ) = Sx(T ). By using the substitution
p(t) = P (t)x(t) (with P (T ) = S), show that the boundary value problem reduces to the solution
of the differential Riccati equation P (t) = AT P (t) + P (t)A P (t)BR1 B T P (t) + Q.
Question 2: Explain how the stabilising solution of the algebraic Riccati equation:
AT P + P A P BR1 B T P + C T C = 0
can be obtained from the eigenvalue-eigenvector decomposition of the associated Hamiltonean
matrix:
!

A
BR1 B T
H=
C T C
AT
Indicate clearly under what conditions the stabilising solution may fail to exist.
Question 3: (i) Consider the scalar differential Riccati equation:
p(t)
= ap2 (t) + bp(t) + c
1 of 3


where a, b and c are constants. Using the transformation p(t) = a1 u(t)
show that this can be
u(t)
transformed to a linear second-order differential equation.

(ii) Consider the scalar time-varying Riccati equation

(t)(p(t)
+ (t)) = p2 (t)
Using the transformation:
p(t) = (t)

u(t)

u(t)

show that this may be written as:

d
((t)u(t))

= (t)u(t)
dt
Question 4: The system x(t)

= u(t) is controlled from the initial condition x(0) = x0 so that

Z
1 2
1 1 2
u (t)dt
J[u] = x (1) +
2
2 0
is minimised. Find and solve the corresponding differential Riccati equation and hence
determine the optimal control.
Question 5: Minimise the performance index:
1
1 2
J[u] =
x1 (3) + x22 (3) +
2
2

3
0

(u21 (t) + u22 (t))dt

subject to the dynamic constraints x 1 (t) = u1 (t) and x 2 (t) = u2 (t).

Question 6: Consider the LQR minimisation problem
Z
min J[u] =
(mx2 (t) + u2 (t))dt
0

subject to the dynamics x(t)

= ax(t) + u(t) where m > 0. (i) Find the optimal state feedback.
In your calculations you need to determine which is the stabilising solution of the ARE (from
two possible solutions); (ii) Verify your conclusion by calculating the optimal closed-loop Amatrix; (iii) Explain how the response of the optimal closed-loop system varies with parameter
m. In your answer concentrate on the speed with which the state x(t) decays to zero starting
from some arbitrary initial condition x(0) = x0 (e.g. the time constant of the response) and on
the peak value maxt0 |u(t)| of the optimal control signal u(t).
Question 7: Find the optimal control of the system x 1 (t) = x2 (t), x 2 (t) = x1 (t) + u(t) which
minimises the performance index:
Z
1
1 2
(x1 (t) + u2 (t))dt
J[u] =
2 0
8
2 of 3

Question 8: Consider the algebraic Riccati equation

AT P + P A P BB T P + C T C = 0
in which

A=

0 1
0 0

, B=

0
1

; and C =

1 0

Is (A, B) controllable? Is (A, C) observable? Form the Hamiltonean matrix H corresponding

to the Riccati equation and obtain the stabilising direction of the ARE directly from the
eigenvalue-eigenvector decomposition of H. Check that your solution satisfies the ARE, that
it is positive definite and that A BB T P is asymptotically stable.
Question 9: Consider the standard (infinite-horizon) LQR problem for the double integrator
system:

!
!
!
x 1
0 1
x1
=
x 2
0 0
x2
with performance index (which must be minimised):
Z
1 2
(x1 + x22 + u2 )dt
J[u] =
2 0
Express the performance index in the standard form. By solving the corresponding Riccati
equation obtain the optimal state-feedback control law. Hence derive the resulting closed-loop
system in state-space form and verify that it is asymptotically stable.
Question 10
Consider the scalar system x = x + u + v, y = x + w where v and w are mutually uncorrelated
white noise signals with variances E(v 2 ) = E(w2 ) = 1. Obtain the optimal steady-state Kalman
filter of this system and the corresponding Kalman gain L. Derive also the corresponding
dynamics of the estimation error signal e(t) = x(t)
x(t) and show that in the absence of process
and mesument noise (v = w = 0) this converges to zero for any initial error e(0) = x(0) x(0).

3 of 3