Professional Documents
Culture Documents
Final-state constraint:
ψ(x(T ), T ) = 0, ψ ∈ Rp
Slide 6 of 69
• Minimum-time problem
Z T
J= 1dt
t0
• Minimum-energy problem
Z T
J = xT (T )Rx(T ) + xT (t)Qx(t) + uT (t)Ru(t) dt
t0
Slide 7 of 69
Leibniz’s rule: Z T
J(x(t)) = h(x(t), t)dt
x0
Z T
dJ = h(x(T ), T )dT + [hTx (x(t), t)δx]dt
t0
∂h
where hx , ∂x
.
Slide 10 of 69
ψ(x(T ), T ) = 0 (3)
Using Lagrange multipliers λ(t) and ν to join the constraints (2) and (3)
to the performance index (1):
Z T
J 0 = φ(x(T ), T )+ν T ψ(x(T ), T )+ [L(x, u, t)+λT (t)(f (x, u, t)−ẋ)]dt
t0
Note:
ν : constant, λ(t): function
Slide 11 of 69
Remember
dx(T ) = δx(T ) + ẋ(T )dT,
so
System model:
ẋ = f (x, u, t), t ≥ t0, t0 fixed
Cost function:
Z T
J(u, T ) = φ(x(T ), T ) + L(x, u, t)dt
t0
Final-state constraint:
ψ(x(T ), T ) = 0
State equation:
∂H
ẋ = =f
∂λ
Costate (adjoint) equation:
∂H ∂f T ∂L
−λ̇ = = λ+
∂x ∂x ∂x
Slide 14 of 69
Stationarity condition:
∂H ∂L ∂f T
0= = + λ
∂u ∂u ∂u
Boundary (transversality) condition:
ẋ = −ax + bu.
Slide 16 of 69
In order to control the temperature on the fixed time interval [0, T ] with
the least supplied energy, define the cost function as
Z T
1 1
J(u) = s(x(T )) + u2(t)dt,
2 2 0
0 = Hu = u + bλ (7)
Slide 17 of 69
So
ẋ = −ax − b2λ(T )e−a(T −t).
Solving the above ODE, we have
−at b2
x(t) = x(0)e − λ(T )e−aT sinh(at).
a
Now we have the second equation about x(T ) and λ(T )
−aT b2
x(T ) = x(0)e − λ(T )(1 − e−2aT )
2a
Assuming x(0) = 0◦ , λ(T ) can now be solved:
−20as
λ(T ) =
2a + b2s(1 − e−2aT )
Slide 20 of 69
∗ −10aseat
λ (t) =
aeaT + sb2sinh(aT )
Finally we obtain the optimal control
∗ 10abseat
u (t) =
aeaT + sb2sinh(aT )
Conclusion:
TPBV makes it hard to solve even for simple OCP problems. In most
cases, we have to rely on numerical methods and dedicated OCP software
package, such as RIOTS.
Slide 21 of 69
System model:
ẋ = A(t)x + B(t)u
Objective function:
Z T
1 1
J(u) = xT (T )S(T )x(T ) + (xT Q(t)x + uT R(t)u)dt
2 2 t0
where S(T ) and Q(t) are symmetric and positive semidefinite weighting
matrices, R(t) is symmetric and positive definite, for all t ∈ [t0 , T ]. We
are assuming T is fixed and the final state x(T ) is free.
State and costate equations:
ẋ = Ax − BR−1B T λ,
−λ̇ = Qx + AT λ
Slide 23 of 69
Control input:
u(t) = −R−1B T λ
Terminal condition:
λ(T ) = S(T )x(T )
Considering the terminal condition, let’s assume that x(t) and λ(t) sat-
isfy a linear relation for all t ∈ [t0 , T ] for some unknown matrix S(t):
λ(t) = S(t)x(t)
To find S(t), differentiate the costate to get
−Ṡ = AT S + SA − SBR−1B T S + Q, t ≤ T.
Now the optimal controller is given by
ẋ1 = x2,
ẋ2 = u,
Objective function (time-optimal:
Z T
J(u) = T = 1dt
0
Input constraints:
|u(t)| ≤ 1
Slide 27 of 69
End-point constraint:
x1(T )
ψ(x(T ), T ) = = 0.
x2(T )
The Hamiltonian is:
H = 1 + λ1x2 + λ2u,
where λ = [λ1 , λ2 ]T is the costate.
Costate equation:
λ̇1 = 0 ⇒ λ1 = constant
λ̇2 = −λ1 ⇒ λ2(t) is a linear function of t (remember it!)
Boundary condition:
λ2(T )u(T ) = −1
Slide 28 of 69
λ∗2(t)u∗(t) ≤ λ∗2(t)u(t)
How to make sure λ∗2 (t)u∗ (t) is less or equal than λ∗2 (t)u(t) for any
admissible u(t)?
Answer:
∗ 1, λ∗2(t) < 0
u (t) = −sgn(λ∗2(t)) =
−1, λ∗2(t) > 0
What if λ∗2 (t) = 0? Then u∗ is undetermined.
Since λ∗2 (t) is linear, it changes sign at most once. So does u∗ (t)!
Slide 29 of 69
10
Switching Curve
2 u=−1
x2
0
u=+1
−2
−4
−6
−8
−10
−50 −40 −30 −20 −10 0 10 20 30 40 50
x1
Slide 31 of 69
References:
F. L. Lewis and V. L. Syrmos, Optimal Control, John Wiley & Sons, Inc,
1997
A. E. Bryson and Y. C. Ho, Applied Optimal Control, New York: Hemi-
sphere, 1975
J. T. Betts, Practical Methods for Optimal Control Using Nonlinear Pro-
gramming, SIAM, 2001
I. M. Gelfand and S. V. Fomin, Calculus of Variations, New York: Dover
Publications, 1991
Slide 32 of 69
What is RIOTS?
RIOTS is a group of programs and utilities, written mostly in C, Fortran,
and M-file scripts and designed as a toolbox for Matlab, that provides an
interactive environment for solving a very broad class of optimal control
problems.
Slide 36 of 69
History of RIOTS:
1. RIOTS for Sun (Adam L. Schwartz)
• Compiler: Sun C compiler
• Matlab Version: Matlab 4
• MEX version: v4
Slide 41 of 69
History of RIOTS:(cont.)
3. RIOTS for Windows (rebuild) and Linux (Jinsong Liang)
• compiler: Microsoft Visual C++ (Windows version), GNU gcc
(Linux version)
• Matlab Version: Matlab 6.5
• MEX version: v6
Slide 43 of 69
Subject to:
Subject to:
ẋ1(t) = x2(t), x1(0) = −5
2
ẋ2(t) = −x1(t) + [1.4 − 0.14x2(t)]x2(t) + 4u(t), x2(0) = −5
Slide 46 of 69
sys init.m
function neq = sys_init(params)
% Here is a list of the different system information paramters.
% neq = 1 : number of state variables.
% neq = 2 : number of inputs.
% neq = 3 : number of parameters.
% neq = 4 : reserved.
% neq = 5 : reserved.
% neq = 6 : number of objective functions.
% neq = 7 : number of nonlinear trajectory constraints.
% neq = 8 : number of linear trajectory constraints.
% neq = 9 : number of nonlinear endpoint inequality constraints.
% neq = 10 : number of linear endpoint inequality constraints.
% neq = 11 : number of nonlinear endpoint equality constraints.
% neq = 12 : number of linear endpoint equality constraints.
% neq = 13 : 0 => nonlinear, 1 => linear, 2 => LTI, 3 => LQR, 4 =
% The default value is 0 for all except neq = 6 which defaults to
neq = [1, 2; 2, 1]; % nstates = 2 ; ninputs = 1
Slide 47 of 69
sys acti.m
function message = sys_acti
sys l.m
function z = l(neq,t,x,u)
% z is a scalar.
z = x(1)ˆ2 + uˆ2;
sys g.m
function J = sys_g(neq,t,x0,xf)
% J is a scalar.
J = 0;
Slide 49 of 69
sys Dh.m
function [h_x,h_u] = sys_dh(neq,t,x,u)
h_x = [0, 1; -1, 1.4-0.42*x(2)ˆ2];
h_u = [0; 4.0];
sys Dl.m
function [l_x,l_u,l_t] = sys_Dl(neq,t,x,u)
%l_t is not used currently
l_x = [2*x(1) 0];
l_u = 2*u;
l_t = 0;
Slide 50 of 69
sys Dg.m
function [J_x0,J_xf,J_t] = sys_dg(neq,t,x0,xf)
% J_x0 and J_xf are row vectors of length n.
% J_t is not used.
J_x0 = [0 0];
J_xf = [0 0];
J_t = 0;
Slide 51 of 69
mainfun.m
N = 50;
x0 = [-5;-5];
t = [0:2.5/N:2.5];
u0 = zeros(1,N+2-1); % Second order spline---in
sp_plot(t,u);
figure;
plot(t,x);
Slide 52 of 69
4
Optimal Control
−1
−2
0 0.5 1 1.5 2 2.5
Time
Slide 53 of 69
2
Optimal Trajectory
−2
−4
−6
0 0.5 1 1.5 2 2.5
Time
Slide 54 of 69
J(u) = T
Subject to:
ẋ1 = x2, x1(0) = 0, x1(T ) = 300,
ẋ2 = u, x2(0) = 0, x2(T ) = 0
Slide 56 of 69
Z a+T
min g(T, x(T )) + l(t, x, u)dt
u,T a
Subject to:
Z b
min g((b − a)xn+1, x(b)) + l(xn, x, u)dt
u,xn+1 a
Subject to:
xn+1h(xn, x, u) x(a)
ẏ = xn+1 , y(a) = a , t ∈ [a, b],
0 ξ
sys init.m
function neq = sys_init(params)
% Here is a list of the different system information paramters.
% neq = 1 : number of state variables.
% neq = 2 : number of inputs.
% neq = 3 : number of parameters.
% neq = 4 : reserved.
% neq = 5 : reserved.
% neq = 6 : number of objective functions.
% neq = 7 : number of nonlinear trajectory constraints.
% neq = 8 : number of linear trajectory constraints.
% neq = 9 : number of nonlinear endpoint inequality constraints.
% neq = 10 : number of linear endpoint inequality constraints.
% neq = 11 : number of nonlinear endpoint equality constraints.
% neq = 12 : number of linear endpoint equality constraints.
% neq = 13 : 0 => nonlinear, 1 => linear, 2 => LTI, 3 => LQR, 4 =
% The default value is 0 for all except neq = 6 which defaults to
neq = [1 3 ; 2 1 ; 12 2]; % nstates = 3 ; ninputs = 1; 3 endpoin
Slide 60 of 69
sys acti.m
function message = sys_activate
message = ’bang’;
sys h.m
function xdot = sys_h(neq,t,x,u)
global sys_params
tau = x(3);
sys l.m
function z = l(neq,t,x,u)
global sys_params
z = 0;
sys g.m
function J = sys_g(neq,t,x0,xf)
global sys_params
F_NUM = neq(5);
if F_NUM == 1
J = x0(3);
elseif F_NUM == 2
J = xf(1)/300.0 - 1;
elseif F_NUM == 3
J = xf(2);
end
Slide 62 of 69
sys Dh.m
function [h_x,h_u] = sys_Dh(neq,t,x,u)
global sys_params
% h_x must be an n by n matrix.
% h_u must be an n by m matrix.
tau = x(3);
h_x = zeros(3,3);
h_u = zeros(3,1);
h_x(1,2) = tau;
h_x(1,3) = x(2);
h_x(2,3) = u(1);
h_u(2,1) = tau;
Slide 63 of 69
mainrun.m
x0 = [0 1 0 0; 0 1 0 0;1 0 .1 10]
% The first column is the initial condition with the optimal dura
% set to 1. The zero in the second column will be used to tell
% riots() that the third initial condition is a free variable. T
% third and fourth columns represent upper and lower bounds on th
% initial conditions that are free variables.
N = 20;
u0 = zeros(1,N+2-1);
t = [0:10/N:10]; % Set up the initial time vect
% the intial time interval is
[u,x,f]=riots(x0,u0,t,-2,1,[],100,2);
Tf = x(3,1)
% The solution for the time interval is t*Tf = [0,10]*Tf. Thus,
% solution for the final time is 10*Tf = 29.9813. The actual sol
% for the final time is 30.
sp_plot(t*Tf,u)
xlabel(’Time’),ylabel(’Optimal Control’)
figure
plot(t*Tf,x)
Slide 64 of 69
1.5
0.5
u
−0.5
−1
−1.5
−2
−2.5
0 5 10 15 20 25 30
Time
Slide 65 of 69
350
300
250
200
x
150
100
50
0
0 5 10 15 20 25 30
Time
Slide 66 of 69
References:
Adam L. Schwartz, Theory and Implementation of Numerical Methods
Based on Runge-Kutta Integration for Solving Optimal Control Prob-
lems, Ph.D. Dissertation, University of California at Berkeley, 1996
RIOTS 95 web:
http://www.csois.usu.edu/ilc/riots
Slide 69 of 69
Credits:
Dr. Adam L. Schwartz, the father of RIOTS for creating the first Unix
OS4 version of RIOTS. Dr. YangQuan Chen for making the Windows
version of RIOTS 95. Dr. Jinsong Liang for making Windows-based
RIOTS MEX-6 compatible and versions for Linux and Solaris, AIX.
This set of slides were mostly prepared by Jinsong Liang under the
supervision of Prof. YangQuan Chen as a module in Dr. Chen’s
ECE/MAE7360 “Robust and Optimal Control” course syllabus. This
OCP/RIOTS module has been offered twice (2003, 2004).