1

On Hybrid State Estimation for Stochastic

Hybrid Systems
Weiyi Liu Student Member, IEEE and Inseok Hwang, Member, IEEE
Abstract
This paper considers the state estimation problem for the Stochastic Hybrid System (SHS) which has various
applications in modern control engineering. Dened on the hybrid state space, the SHS has the interacting discrete
dynamics and continuous dynamics subject to various uncertainties. The hybrid state estimation problem is to estimate
both the continuous state and the discrete state of the SHS with the information given by noisy observations. In
this paper, the hybrid state estimation problem is mathematically formulated and two dynamical equations (ltering
equations) are derived to describe the evolution of the hybrid state conditioned on the observation history. A numerical
algorithm based on stochastic approximation is proposed to solve the ltering equations. A Markov Chain (MC) is
rst constructed on the descretized hybrid state space to approximate the dynamics of the SHS and then hybrid state
estimation for the SHS is reduced to estimating the state of the MC. It is proved that the state estimation results
of the MC converge to those of the SHS as the MC converges to the SHS. The proposed numerical algorithm is
validated through an aircraft tracking scenario for air trafc control.
I. INTRODUCTION
The Stochastic Hybrid System (SHS) is a class of stochastic processes with the interacting continuous dynamics
and discrete dynamics. The switching and/or jumping dynamical behaviors of the SHS are suitable to model many
complex stochastic systems including maneuvering aircraft [1], [2], switching communication networks [3], growth
of cataract [4], stock prices with jumps [5], etc.
The hybrid state estimation problem is an important branch of the research in the SHS. The goal is to estimate
both the continuous state and the discrete state of the SHS with the information from a series of noisy observations.
The hybrid state estimation problem has a variety of applications including target tracking [6], [7], [8], [9], fault
detection and isolation [10], [11], [12], [13], information fusion [14], sensor scheduling [15], etc. Compared with
the conventional state estimation problem, hybrid state estimation is much more challenging due to the fact that the
state of the SHS is dened on the complex hybrid state space and the evolution of the SHS consists of the complex
switching continuous state dynamics and jumping discrete state dynamics.
Most of the previous research in the hybrid state estimation problem is limited to some special classes of the
SHS. For example, some SHSs have the complex discrete dynamics with the simple continuous dynamics and then
The authors are with the School of Aeronautics and Astronautics, Purdue University, West Lafayette, IN, 47907 USA. e-mail: li-
u61,ihwang@purdue.edu.
September 24, 2011 DRAFT
2
the corresponding hybrid state estimation problem can be approximately solved by estimating only the discrete state
of the SHS using nite state machines [16], [17]. On the other extreme, some other SHSs have the simple discrete
dynamics which can be regarded as disturbances to the continuous dynamics. The Luenberger observer for such
SHSs can be designed for state estimation, which estimates only the continuous state of the SHS [18]. For another
class of the SHS called the Markov Jump Linear System (MJLS), whose continuous dynamics is given by a set of
linear systems and the discrete state transitions are governed by a Markov Chain (MC), the hybrid state estimation
problem can be solved by algorithms based on a bank of Kalman Filters, e.g., the Interacting-Multiple-Model (IMM)
algorithm [7] and its variants [19], [8], [9]. Also, the hybrid state estimation problem for the SHS can be solved
as an optimization problem, which results in a moving horizon estimation algorithm [20]. For the SHS with the
nonlinear continuous and discrete dynamics, the particle lter-based approaches are used for hybrid estimation and
the research in this topic is relatively limited [21], [22], [23].
This paper is focused on the state estimation problem for a general class of nonlinear SHSs, namely, the hybrid
stochastic processes dened on the continuous time. For the SHS considered in this paper, the continuous state
evolution is governed by a set of Stochastic Differential Equations (SDEs), each matched to a discrete state, and
the discrete state transitions are Poisson jumps whose transition rate is dependent on the continuous state. The
modeling of such SHS is pioneered by Lygeros, Hu, Bojorianu and so on [24], [25], and most of the existing SHS
models can be regarded as special cases of this general SHS.
The rst contribution of this paper is to mathematically formulate the hybrid state estimation problem for the
SHS and derive ltering equations to solve it. The hybrid state estimation problem is to compute the probability
distribution of the hybrid state with the information given by noisy observations, i.e., we want to compute a
conditional distribution of the hybrid state. To describe the evolution of the conditional distribution, we derive
ltering equations similar to the Zakai equation [26]. The derived equations govern how the probability distribution
function (pdf) of the hybrid state changes over time with the noisy observations.
Another contribution of this paper is the development of a numerical algorithm which can solve the ltering
equations for the SHS. A Markov Chain (MC) is constructed on a discretized hybrid state space such that the
evolution of the MC approximates the dynamics of the SHS. Then, the hybrid state estimation problem can be
reduced to estimating the state of the approximating MC, which can be numerically computed. When the grid size
and the interpolation time interval of the approximating MC converge to zero, we prove that the MC converges to
the SHS in distribution and the state estimation result for the MC also converges to the hybrid state estimates of
the SHS.
The paper is organized as follows: In Section II, the model of the SHS and its state estimation problem are
presented. The ltering equations are derived in Section III and the numerical hybrid state estimation algorithm
is presented in Section IV. Section V presents the simulation results of the numerical algorithm via an aircraft
tracking scenario. Conclusions are given in Section VI.
September 24, 2011 DRAFT
3
II. MATHEMATICAL FORMULATION OF THE HYBRID STATE ESTIMATION PROBLEM FOR THE STOCHASTIC
HYBRID SYSTEM
In this section, the mathematical model of the Stochastic Hybrid System (SHS) and the nonlinear observation
model are presented, and the corresponding hybrid state estimation problem is formulated in two equivalent forms.
A. Notations
Denote by: R
n
, the n-dimensional Euclidean space; R
+
, the positive real numbers; Z, the integers; N, the natural
numbers; , the Euclidean norm; E[], the (conditional) expectation of a random variable; Pr{}, the (conditional)
probability of an event; I
n
, the n n identity matrix.
B. Stochastic hybrid system model
The SHS is a class of stochastic processes dened on the hybrid state space composed of several copies of the
continuous state space [25]. At each time, the evolution of the SHS may be conned in a continuous state space
indexed by the discrete state (continuous dynamics), or jumps from one discrete state to another (discrete dynamics)
and starts a new continuous evolution in another continuous state space. Many switching or jump dynamics including
the Markov Jump Linear System (MJLS) and the jump diffusion process are special cases of the SHS.
Let (, F, F
t
, P) be a complete probability space with a sample space , a -eld of the events F, a natural
ltration F
t
and a natural probability measure P : F [0, 1]. Dened in the probability space, the SHS model is
H = (X, Init, a, b, , R) which is detailed as follows:
Hybrid state space (X): z R
n
is the continuous state; q Q = {1, , n
d
} is the discrete state; a
composition of the continuous state and the discrete state, x := (z, q), is the hybrid state; and X := R
n
Q
is the hybrid state space that x belongs to.
Initial condition (Init): the evolution of the SHS x
t
= (z
t
, q
t
) starts from an initial distribution in the hybrid
state space X, Init : B(X) [0, 1], where B(X) denotes all the Borel sets in X.
Continuous dynamics (a and b): for a discrete state q Q, the continuous dynamics is described by a Stochastic
Differential Equation (SDE):
dz
t
= a(z
t
, q)dt + b(z
t
, q)dW
t
(1)
where W
t
is the standard n-dimensional Brownian motion. For q Q, a(z, q) and b(z, q) satisfy the Lipschitz
condition in the continuous state space R
n
such that the solution to the SDE exists and is unique [27].
Discrete dynamics ( and R): the discrete dynamics is described by a bounded Poisson transition rate function:
: X R
+
and a stochastic kernel: R : X B(X) [0, 1]. The Poisson transition rate function denes
the probability that a discrete state transition happens in any given time interval, and the reset condition R is
a distribution of the hybrid state x right after a discrete state transition happens.
Figure 1 illustrates one execution (sample path) x
t
() ( ) of the SHS which has three discrete states
q {1, 2, 3} and starts from x
0
as t = 0. The times when the discrete state transition happens are denoted by
September 24, 2011 DRAFT
4
x
0
x
T

1
x
T
1
x
T

2
x
T
2 x
T

3
x
T
3
q = 1
q = 2
q = 3
Fig. 1: One execution x
t
() of the SHS.
a series of random times, T
1
, T
2
, T
3
, , which can be characterized by the stopping times associated with the
execution x
t
(). On each time interval [T
i
, T
i+1
), there are no discrete state transitions, i.e., the discrete state q
t
is
constant t [T
i
, T
i+1
) and the evolution of z
t
is described by (1). Provided that a discrete state transition happens
at T
i
, the next discrete state transition time T
i+1
is a random variable dened on (T
i
, ) whose distribution is
given by:
Pr{T
i+1
< t} = 1 exp
_

_
t
T
i
(x
s
())ds
_
t (T
i
, ) (2)
which has the Poisson distribution, i.e., for any small increment of time u 0 [28],
Pr{There is a discrete state transition on the time interval [t, t + u)} = (x
t
()) u + o(u) (3)
where o(u) is the higher-order term of u. Right after a discrete state transition happens at time T
i+1
, the hybrid
state x is reset according to the reset condition (stochastic kernel) R such that B(X):
Pr{x
T
i+1
} = R(x
T

i+1
, ) (4)
where x
T

i+1
is the left-limit of x
T
i+1
, i.e., the hybrid state right before a discrete state transition happens at T
i+1
.
Remark 1: In this paper, the discrete state transitions are usually referred to as the spontaneous jumps in the SHS
[25]. The SHS could have another kind of transitions, namely, the forced jumps. However, the methods developed
in this paper can be easily extended to incorporate the forced jump case.
Lemma 1: The SHS x
t
is a well-dened Markov process.
The proof of this lemma can be found in [29].
Denition 1: Dene a class of functions C on the hybrid state space X such that: f C f : X R
1
and f
is bounded, measurable and twice-differentiable with respect to (w.r.t.) z almost everywhere.
In this paper, we always assume that f C, if it is not otherwise specied.
Denition 2: B(X), the indicator function 1

(x) C is dened such that:

1

(x) =
_
_
_
1 x

0 otherwise
where

is the closure of .
September 24, 2011 DRAFT
5
Theorem 1: (Martingale representation theorem of the SHS) x = (z, q) X and f C, dene an operator
L as:
Lf(x) :=
n

i=1
a
(i)
(z, q)
f(x)
z
(i)
+
1
2
n

i,j=1
(b(z, q)b
T
(z, q))
(i,j)

2
f(x)
z
(i)
z
(j)
+ (x)
_
X
(f(y) f(x))R(x, dy) (5)
where the superscript (i) denotes the i-th element in a vector and (i, j) denotes the element in the i-th row and the
j-th column of a matrix. Then, t R
+
, the SHS x
t
= (z
t
, q
t
) starting from x
0
at t = 0 satises:
f(x
t
) = f(x
0
) +
_
t
0
Lf(x
s
)ds +

M
t
+

M
t
(6)
where

M
t
:=
_
t
0
n

i=1

z
(i)
f(x
s
)b(z
s
, q
s
)dW
s

M
t
:=

T
k
t
(f(x
T
k
) f(x
T

k
))
_
t
0
(s)
_
X
(f(y) f(x
s
))R(x
s
, dy)ds
are local martingales.
The proof of this theorem is omitted here and can be derived using the results in [29], [30]
Remark 2: The operator L dened in (5) is the innitesimal generator, uniquely associated with the Markov
process x
t
. Specically, for any function f C,
Lf(x) = lim
t0
+
E
P
[f(x
t
)] f(x)
t
with x
0
= x.
C. Observation models
The continuous-time observation process y
t
R
m
is the output of a nonlinear observation model:
y
t
=
_
t
0
h(x
s
)ds + B
t
(7)
where h : X R
m
is a bounded and continuous function w.r.t. the continuous state z and B
t
is the m-dimensional
Brownian motion.
Remark 3: For a more general observation model: y
t
=
_
t
0
h(x
s
)ds +(x
t
)B
t
where : X R
m
R
m
is an
invertible matrix, a normalized observation process y
t
can be dened as:
y
t
:=
_
t
0

1
(x
s
)dy
s
=
_
t
0

1
(x
s
)h(x
s
)ds + B
t
(8)
which can still be written in the form of (7). Thus, the observation model (7) is general for the hybrid state estimation
problem.
Remark 4: In many computer controlled systems, the observations are taken at discrete times. The discrete-time
observation model can be obtained by taking difference on both sides of (7). Let T
s
be the sampling interval of
the sensor and dene the observation at each sampling time as y
D
kT
s
:= y
kT
s
y
(k1)T
s
. Then,
y
D
kT
s
=
_
kT
s
(k1)T
s
h(x
t
)dt + B
kT
s
B
(k1)T
s
h(x
kT
s
)T
s
+ w(k) (9)
September 24, 2011 DRAFT
6
where w(k) are independent and identically distributed (i.i.d.) Gaussian random variables with zero mean and
covariance matrix T
s
I
m
, by the properties of the Brownian motion [28]. When T
s
converges to 0, the Right-
Hand-Side (R.H.S.) of (9) converges to the Left-Hand-Side (L.H.S.) of (9). Hense, the observation model (9) is an
approximation of (7) in the discrete time.
In this paper, we use (7) to derive the ltering equations and (9) to derive a numerical hybrid state estimation
algorithm. The computation results of the numerical algorithm approximate the solution to the ltering equations.
D. Hybrid state estimation problem
In this section, we formulate the hybrid state estimation problem for the SHS in two equivalent forms. Firstly,
the state estimation problem can be regarded as computing the probability distribution function (pdf) of the hybrid
state x at time t with the information given by the observation history. Dene a -algebra:
F
y
t
:= {y
s
, 0 s t}
to be a collection of the information generated by the observation y
t
in the time interval [0, t]. We are interested
in a stochastic process x
t
which has the following property:
Pr{ x
t
} = Pr{x
t
|F
y
t
} B(X) (10)
Thus, x
t
can be regarded as an estimate of x
t
with the given information F
y
t
and thus the hybrid state estimation
problem is to compute the pdf of x
t
:
Problem 1: Given the SHS x
t
and corresponding observation information F
y
t
, then t 0 and x X, we want
to compute the pdf p
t
(x) of the conditional process x
t
in (10), such that B(X),
Pr{ x
t
} =
_

p
t
(x)dx (11)
Another problem which is equivalent to Problem 1 can be stated as:
Problem 2: For any function f C, the hybrid state estimation problem is to compute:
E
P
[f(x
t
)|F
y
t
] (12)
where the subscript P denotes that the expectation is taken w.r.t. the measure P.
Remark 5: Note that in Problem 2, Equation (12) is dened for any functions f C. By choosing f in
different forms, Equation (12) gives different information about the state estimate x
t
. For example, we dene

f := [f
(1)
f
(2)
f
(n)
]
T
with f
(i)
(x) = z
(i)
for i = 1, , n. Then, E
P
[

f(x
t
)|F
y
t
] = E
P
[z
t
|F
y
t
] gives the
mean of the estimated continuous state at each time t. For another example, we choose f(x) = 1
{x|q=1}
(x). Then
E
P
[f(x
t
)|F
y
t
] = Pr{q
t
= 1|F
y
t
} is the estimated probability that the discrete state q
t
= 1 at time t with the
observation information F
y
t
.
Theorem 2: Problems 1 and 2 are equivalent.
September 24, 2011 DRAFT
7
Proof: Suppose p
t
(x) has already been computed by solving Problem 1. Then, f C, E
P
[f(x
t
)|F
y
t
] in
Problem 2 can be computed by:
E
P
[f(x
t
)|F
y
t
] = E
P
[f( x
t
)] =
_
X
f(s)p
t
(s)ds (13)
Conversely, if E
P
[f(x
t
)|F
y
t
] has been computed by solving Problem 2, then p
t
(x) can be computed by letting
f(s) = (x s) and evaluating (13). Note that is the Dirac delta function.
III. DERIVATION OF THE FILTERING EQUATIONS
Problems 1 and 2 are formulated in this section by two dynamical equations called as the ltering equations [26],
[31]. For a given initial condition, the ltering equations describe the time evolution of p
t
(x) and E
P
[f(x
t
)|F
y
t
]
with the observations y
t
. In this section, we rst derive a ltering equation in the integral form to solve Problem
2. Then, another ltering equation similar to the Zakai equation [26] is derived to solve Problem 1.
The method we use to derive the ltering equations is to replace the natural probability measure P with a new
measure Q
t
, which is called the Change-Of-Measure (COM) approach [26].
A. Change of measure
The COM approach is to look for a new measure other than the natural measure P, such that the observations
y
t
and the state x
t
can be decoupled under the new measure. The COM approach is similar to the technique of
integration by substitution and it has been used to derive many estimation/ltering algorithms [6], [31], [32], [33].
To understand the COM approach, rst consider the following illustrative example:
Example 1: Let P
1
be the Lebesgue measure. A Gaussian random variable is dened on R
1
with the distribution
p
1

under P
1
:
(A) =
_
A
p
1

()dP
1
:=
1

2
_
A
exp
_

(x )
2
2
_
dx A B(R)
Suppose there is a new measure P
2
, such that the Radon-Nikodym derivative [28] between P
1
and P
2
is
dP
1
/dP
2
:= exp{x +
2
/2}. Then under the new measure P
2
:
(A) =
_
A
p
2

()
dP
2
dP
1
dP
1
=
1

2
_
A
exp
_

x
2
2
_
dP
2
A B(R)
where p
2

:= p
1

dP
1
/dP
2
is the distribution of under P
2
. Thus, has the standard Gaussian distribution under
the new measure P
2
, which can make further computation easier.
Similar to Example 1, at any time t, a new measure Q
t
can be constructed to make the computation of (12)
easier. For the observation model (7), Girsanovs theorem provides a guideline for choosing an appropriate new
measure Q
t
under which y
t
is the Brownian motion independent of x
t
. Dene
t
R as:

t
=
dP
dQ
t
being the Radon-Nikodym derivative between the two measures at time t. Thus, by Bayes theorem [34], (12) can
be rewritten as:
E
P
[f(x
t
)|F
y
t
] =
E
Q
t
[f(x
t
)
t
|F
y
t
]
E
Q
t
[
t
|F
y
t
]
(14)
September 24, 2011 DRAFT
8
Thus, solving Problem 2 is equivalent to solving (14).
If
t
is chosen such that

t
:= exp
__
t
0
h
T
(x
s
)dy
s

1
2
_
t
0
h(x
s
)
2
ds
_
where h
T
() is the conjugate transpose of h(), then,
By Girsanovs theorem, under the new measure Q
t
, x
t
and y
t
are independent and y
t
is the standard m-
dimensional Brownian motion [26].
By Itos rule,
t
satises the following SDE:
d
t
=
t
h
T
(x
t
)dy
t
(15)
with the initial condition
0
= 1 [27].
B. Filtering equation as an integral equation to solve Problem 2
To compute the R.H.S. of (14), an integral equation is rstly derived to describe the evolution of E
Q
t
[f(x
t
)
t
|F
y
t
].
Theorem 3: For any function f C, dene an operator:
t
(f) := E
Q
t
[f(x
t
)
t
|F
y
t
]. Then,
t
() satises:

t
(f) =
0
(f) +
_
t
0

s
(Lf)ds +
_
t
0

s
(f h
T
)dy
s
(16)
where the operator L is dened in (5)
Proof: Apply Itos rule to f(x
t
)
t
:
f(x
t
)
t
= f(x
0
)
0
+
_
t
0
f(x
s
)d
s
+
_
t
0

s
df(x
s
) +
_
t
0
df(x
s
)d
s
(17)
By (15),
_
t
0
f(x)d
s
=
_
t
0
f(x
s
)
s
h
T
(x
s
)dy
s
(18)
By (6), we obtain
_
t
0

s
df(x
s
) =
_
t
0

s
Lf(x
s
)ds +
_
t
0

s
d

M
s
+
_
t
0

s
d

M
s
(19)
Plugging (18), (19) and (15) into (17) yields:
f(x
t
)
t
= f(x
0
)+
_
t
0
f(x
s
)
s
h
T
(x
s
)dy
s
+
_
t
0

s
Lf(x
s
)ds+
_
t
0

s
d

M
s
+
_
t
0

s
d

M
s
+
_
t
0

s
h
T
(x
s
)df(x
s
)dy
s
Then, take the conditional expectation E
Q
t
[|F
y
t
] on both sides of the above equation. Since

M
t
and

M
t
are two
local martingales and y
t
is the Brownian motion independent of x
t
under the measure Q
t
, the last three terms of
the above equation vanish after taking the expectation. Then, (16) follows.
Based on (16), the following theorem solves Problem 2:
Theorem 4: For any function f C, dene an operator
t
(f) := E
P
[f(x
t
)|F
y
t
]. Then,
t
() satises the
following integral equation:

t
(f) =
0
(f) +
_
t
0

s
(Lf)ds +
_
t
0
{
s
(h f)
s
(f)
s
(h)}
T
{dy
s

s
(h)ds} (20)
September 24, 2011 DRAFT
9
Proof: Letting f(x) 1 in (16) yields:

t
(1) = 1 +
_
t
0

s
(h
T
)dy
s
Then, by Itos rule,
1
E
Q
t
[
t
|F
y
t
]
=
1

t
(1)
= 1
_
t
0

s
(h
T
)

s
(1)
3
{
s
(1)dy
s

s
(h)ds} (21)
By (14), we obtain

t
(f) = E
P
[f(x
t
)|F
y
t
] =
E
Q
t
[f(x
t
)
t
|F
y
t
]
E
Q
t
[
t
|F
y
t
]
=
t
(f)
1

t
(1)
Then, using (16) , (21) and Itos rule on the R.H.S. of the above equation yields (20).
C. Filtering equation as a Partial Differential Equation to solve Problem 1
In this section, based on (16), we derive a ltering equation which solves Problem 1. Firstly, we assume that
there exists an unnormalized pdf p
t
(x) such that for all f C:

t
(f) =
_
X
f(x) p
t
(x)dx (22)
where the operator
t
(x) is dened in Theorem 3. The derived ltering equation is a stochastic PDE which governs
the time evolution of the unnormalized pdf p
t
(x) depending on the observation y
t
. Then, the normalized pdf p
t
(x)
dened in (11) can be computed by normalizing p
t
(x):
p
t
(x) =
p
t
(x)
_
X
p
t
(x)dx
(23)
Suppose the initial distribution of x
t
is given by p
0
(x), and dene a measure J on B(X R
+
):
J(A) = E
p
0
(x)
P
_
_

k0
1
A
(x
T

k
, T
k
)
_
_
where T
k
are the stopping times dened in (2) and the superscript p
0
(x) emphasizes the dependence of J(A) on
the initial distribution of x
t
. Then, B(X), J( [t
1
, t
2
)) denotes the expected number of jumps (discrete
state transitions) starting from during the time interval [t
1
, t
2
).
Dene the mean jump intensity r
t
which is a measure on X such that:
J( [0, t)) =
_
t
0
r
s
()ds B(X)
Note that r
t
() is also dependent on the initial distribution p
0
(x). By the denition of (x) and p
t
(x) given by (3)
and (11), respectively, we can get:
r
t
() = E
p
0
(x)
P
[(x
t
)1

(x
t
)] =
_

(x)p
t
(x)dx
Thus, we can write:
r
t
(dx) = (x)p
t
(x)dx (24)
By using the above denitions, we can compute the adjoint operator of the innitesimal generator L which is
useful in deriving the ltering equation to solve Problem 1.
September 24, 2011 DRAFT
10
Lemma 2: Suppose the operator L is dened in (5). Then Ls adjoint operator L

which satises
_
X
Lf(x)p
t
(x)dx =
_
X
f(x)L

p
t
(x)dx f, p
t
C
is given by:
L

p
t
(x) =
n

i=1

z
(i)
_
a
(i)
(x)p
t
(x)
_
+
1
2
n

i,j=1

2
z
(i)
z
(j)
_
(b(x)b
T
(x))
(i,j)
p
t
(x)
_
+
_
X
{r
t
(dx)R(x, ) r
t
()I(, dx)}
(25)
where I is the identity kernel that I(y, dx) = (y x)dx.
Proof: Dene:
_
_
_
L
1
f(x) :=

n
i=1
a
(i)
(x)
f(x)
z
(i)
+
1
2

n
i,j=1
(b(x)b
T
(x))
(i,j)

2
f(x)
z
(i)
z
(j)
L
2
f(x) := (x)
_
X
(f(y) f(x))R(x, dy)
Thus, Lf = L
1
f +L
2
f and L

p
t
= L

1
p
t
+L

2
p
t
. Note that L
1
is the innitesimal generator associated with the
SDE (1) and its adjoint operator is a well-known result in probability theory [27]:
L

1
p
t
(x) =
n

i=1

z
(i)
_
a
(i)
(x)p
t
(x)
_
+
1
2
n

i,j=1

2
z
(i)
z
(j)
_
(b(x)b
T
(x))
(i,j)
p
t
(x)
_
(26)
On the other hand, using (24):
_
X
L
2
f(x)p
t
(x)dx =
_
X
(x)
_
X
(f(y) f(x)) R(x, dy)p
t
(x)dx
=
_
X
(x)p
t
(x)
_
X
f(y)R(x, dy)dx
_
X
(x)f(x)p
t
(x)dx
In the last line of the above equation, swapping the variables x and y in the rst integration and rewriting the
second integration using the identity kernel I yield:
_
X
L
2
f(x)p
t
(x)dx =
_
X
f(x)
_
X
(y)p
t
(y)R(y, dx)dy
_
X
f(x)
_
X
(x)p
t
(x)I(x, dy)dx
=
_
X
f(x)
_
X
{r
t
(dy)R(y, dx) r
t
(dx)I(x, dy)}
=
_
X
f(x)L

2
p
t
(x)dx
(27)
Combining (26) and (27) yields (25).
The following theorem characterizes the evolution of p
t
(x), which solves Problem 1.
Theorem 5: The evolution of the unnormalized pdf p
t
(x) is given by the following stochastic PDE:
d p
t
(x) = L

p
t
(x)dt + p
t
(x)h
T
(x)dy
t
(28)
With p
t
(x), the normalized pdf p
t
(x) can be computed by (23).
Proof: Using the denition of p
t
(x) given by (22) and Fubinis theorem, Equation (16) can be written as:
_
X
f(x) p
t
(x)dx =
_
X
f(x) p
0
(x)dx +
_
t
0
_
X
f(x)h
T
(x) p
s
(x)dxdy
s
+
_
t
0
_
X
Lf(x) p
s
(x)dxds
=
_
X
f(x) p
0
(x)dx +
_
X
f(x)
_
t
0
p
s
(x)h
T
(x)dy
s
dx +
_
X
f(x)
_
t
0
L

p
s
(x)dsdx
September 24, 2011 DRAFT
11
where L

is given in (25). Since f is an arbitrary function in C, the following equation holds almost surely:
p
t
(x) = p
0
(x) +
_
t
0
p
s
(x)h
T
(x)dy
s
+
_
t
0
L

p
s
(x)ds
Differentiating both sides of the above equation w.r.t. t yields (28)
IV. NUMERICAL ALGORITHM FOR HYBRID STATE ESTIMATION
In this section, a numerical algorithm based on stochastic approximation [35] is derived for hybrid state estimation.
The SHS is approximated by a discrete-time Markov Chain (MC) dened on a discretized hybrid state space. Then,
the hybrid state estimation problem can be approximated by the state estimation problem for the MC. Such grid-
based state estimation algorithms are computationally efcient and can be easily implemented [36], [37].
A. Discretization of the hybrid state space X
The state space of the MC, X
MC
, is constructed by discretizing the hybrid state space X. The evolution of the
MC is dened on discrete time with the interpolation time interval . Let R
n
be the grid size. and are
design parameters when the MC is constructed, with

2

always being constant (i.e., converges to zero faster

than ).
The MCs state
,
X
MC
can be written as
,
= (, q) where is the discretized continuous state z and q
is the discrete state. Note that the superscripts and emphasize the state
,
s dependence on the grid size and
the interpolation time interval. Let G(
,
) X be the grid in the hybrid state space X centered at
,
. Thus,
the hybrid state space X can be represented as
X =
_

,
X
MC
G(
,
)
i.e., X is covered by all of the grids. Figure 2 illustrates the discretization of the hybrid state space X which has
three discrete states q Q = {1, 2, 3}. The hybrid state space X can be written as: X = R
2
Q. In Figure 2, X
is discretized by the grids to yield the state space X
MC
for the MC. A state
,
X
MC
(the black dot) and its
corresponding grid G(
,
) (the grey square) are shown in Figure 2.

,
= (, q)
G(
,
)
q = 1
q = 2
q = 3
Fig. 2: Discretization of the hybrid state space X.
September 24, 2011 DRAFT
12
B. Construction of the approximating Markov chain
In this section, an MC, {
,
k
}, k N, is constructed such that its dynamics dened on X
MC
approximates the
evolution of the SHS dened on the hybrid state space X, i.e., if the approximating MC and the SHS start with
the same initial distribution, the distribution of the MCs state,
,
k
, converges to the the distribution of the SHSs
state, x
k
, for all k N, as the grid size and the interpolation time interval converge to zero, which is called
the weak convergence (convergence in distribution) [38].
k N, the MC {
,
k
}s transition probability at the k-th step is dened as:
P(
2
,
1
) := Pr{
,
k+1
=
2
|
,
k
=
1
}
where
1
:= (
1
, q
1
) X
MC
and
2
:= (
2
, q
2
) X
MC
are two states of the MC {
,
k
}. Note that
1
and

2
can be the same. P(, ) is a design parameter which should be chosen such that the evolution of the MC
approximates the evolution of the SHS. The approach we use to design P(, ) is called the locally consistent
approach inspired by [39], [1].
Dene two events associated with the SHS x
t
:
E
k
:= {The SHS x
t
has no discrete state transition for t [k, (k + 1))}
E

k
:= {The SHS x
t
has one discrete state transition for t [k, (k + 1))}
Thus, E
k
E

k
= . Also, by the denition of the Poission transition rate function (x) given by (3):
_

_
Pr{E
k
|x
k
=
1
} = 1 (
1
) + o()
Pr{E

k
|x
k
=
1
} = (
1
) + o()
Pr{\(E
k
E

k
)|x
k
=
1
} = o()
(29)
where o() is the higher-order term of . Note that at each step k, the MCs state
,
k
can be written as

,
k
= (
k
, q
k
). Dene two events associated with the MC {
,
k
}:

E
k
:= {The MC
,
k

s discrete state q
k
changes at the kth step}

k
:= {The MC
,
k

s discrete state q
k
does not change at the kth step}
P(, ) should be designed such that

E
k
and E
k
, and

E

k
and E

k
happen with the same probabilities. Ignoring the
higher-order terms in (29), we have:
_

_
Pr{

E
k
|
,
k
=
1
} = 1 (
1
)
Pr{

k
|
,
k
=
1
} = (
1
)
Pr{\(

E
k


E

k
)|
,
k
=
1
} = 0
(30)
By the total probability theorem and (30):
P(
2
,
1
) =Pr{
,
k+1
=
2
|

E
k
,
,
k
=
1
}Pr{

E
k
|
,
k
=
1
} + Pr{
,
k+1
=
2
|

k
,
,
k
=
1
}
Pr{

k
|
,
k
=
1
} + Pr{
,
k+1
=
2
|\(

E
k


E

k
),
,
k
=
1
}Pr{\(

E
k


E

k
)|
,
k
=
1
}
=Pr{
,
k+1
=
2
|

E
k
,
,
k
=
1
}(1 (
1
)) + Pr{
,
k+1
=
2
|

k
,
,
k
=
1
}(
1
)
(31)
September 24, 2011 DRAFT
13
To parameterize Pr{
,
k+1
=
2
|

E
k
,
,
k
=
1
} and Pr{
,
k+1
=
2
|

k
,
,
k
=
1
} in (31), we consider the
following two cases:
Case 1: (Parameterize Pr{
,
k+1
=
2
|

E
k
,
,
k
=
1
})
The SHS x
t
has no discrete state transition in the time interval t [k, (k + 1)) and then q
1
= q
2
for the
MC at the k-th step. Dene P

E
k
(
2
,
1
) := Pr{
,
k+1
=
2
|
,
k
=
1
,

E
k
} which is the transition probability
when the discrete state does not change at the k-th step. Since the dynamics of x
t
for t [k, (k + 1))
is governed by the SDE (1), the transition probability P

E
k
(
2
,
1
) should be designed to approximate the
evolution of the SDE (1), which is stated by the following theorem:
Theorem 6: Suppose an MC {
,
k
} has a transition probability P

E
k
(, ) designed such that
E
P
_

,
k+1

,
k

,
k
=
1
,

E
k
_
=

2
X
MC
_

1
_
P

E
k
(
2
,
1
)
a(
1
)
E
P
_
(
,
k+1

,
k
)(
,
k+1

,
k
)
T

,
k
=
1
,

E
k
_
=

2
X
MC
_

1
_ _

1
_
T
P

E
k
(
2
,
1
)
b(
1
)b
T
(
1
)
when and 0. Then the MC {
,
k
} converges weakly to the Markov process whose dynamics is dened
by the SDE (1).
The proof of Theorem 6 can be found in [38]. Note that the parameterization of P

E
k
(, ) that satises the
above weak convergence conditions is not unique. There are many ways to design such locally consistent
transition probabilities for the MC which approximates the SDE (1) [35].
Case 2: (Parameterize Pr{
,
k+1
=
2
|

k
,
,
k
=
1
})
The SHS x
t
has one discrete state transition in the time interval t [k, (k + 1)) i.e., q
1
= q
2
for the MC
at the k-th step. Dene P

k
(
2
,
1
) := Pr{
,
k+1
=
2
|
,
k
=
1
,

E

k
}. Thus, by the denition of the reset
condition (4),
P

k
(
2
,
1
) = R(
1
, G(
2
))
Plugging P

E
k
(, ) and P

k
(, ) designed in Cases 1 and 2 respectively into (31) yields:
1
,
2
X
MC
P(
2
,
1
) =
_
_
_
P

E
k
(
2
,
1
)(1 (
1
)) if q
1
= q
2
P

k
(
2
,
1
)(
1
) if q
1
= q
2
(32)
which is the desired transition probability P(, ) for the MC {
,
k
} that approximates the dynamics of the SHS.
Theorem 7: (Weak convergence of the approximating MC {
,
k
}) Suppose an MC {
,
k
}s transition probability
is given by (32) and it has the same initial distribution as the SHS x
t
. Then k N, the distribution of the MCs
state,
,
k
, converges to the distribution of the SHSs state, x
t
, at t = k, as and converge to zero.
Proof: To prove that
,
k
converges to x
k
in distribution, we only need to show: S
1
, S
2
B(X) and
September 24, 2011 DRAFT
14
k, u N,
Pr
_

,
k+u
S
2

,
k
S
1
_
Pr
_
x
(k+u)
S
2

x
k
S
1
_
(33)
as the grid size and the interpolation time interval converge to zero. By the design procedure of the transition
probability P(, ) given by (32),
1
,
2
X
MC
:
Pr
_

,
k+
=
2

,
k
=
1
_
Pr
_
x
(k+1)
G(
2
)

x
k
G(
1
)
_
as and converge to zero. Thus, by the Markov property of the MC {
,
k
} and the total probability theorem:
Pr
_

,
k+u
S
2

,
k
S
1
_
=

u
S
2

u1
X
MC

1
X
MC

0
S
1
Pr
_

,
k+u
=
u

,
k+u1
=
u1
_
Pr
_

,
k+u1
=
u1

,
k+u2
=
u2
_
Pr
_

,
k+2
=
2

,
k+1
=
1
_
Pr
_

,
k+2
=
1

,
k
=
0
_

u
S
2

u1
X
MC

1
X
MC

0
S
1
Pr
_
x
(k+u)
G(
u
)

x
(k+u1)
G(
u1
)
_
Pr
_
x
(k+u1)
G(
u1
)

x
(k+u2)
G(
u2
)
_
Pr
_
x
(k+2)
= G(
2
)

x
(k+1)
= G(
1
)
_
Pr
_
x
(k+2)
= G(
2
)

x
k
= G(
0
)
_
Pr
_
x
(k+u)
S
2

x
k
S
1
_
as and converge to zero. Thus, (33) is true and the MC
,
k
converges to x
k
in distribution.
Since the MC
,
k
converges to the SHS x
t
at t = k, a direct consequence of Theorem 7 is that the MC
,
k
converges to x
t
in distribution for any t R
+
as , 0.
C. Numerical state estimation algorithm for the approximating Markov chain {
,
k
}
In this section, a numerical algorithm is proposed for the state estimation of the MC {
,
k
} which is an
approximation to the solutions to the hybrid estimation problems (Problems 1 and 2). It is proved that the numerical
solution converges to the exact solution to the hybrid estimation problems as the grid size and the interpolation
time interval converge to zero. In practical applications, and can be chosen to trade-off the computational
complexity and numerical approximation errors.
To estimate the state of the MC {
,
k
}, y
t
is rstly sampled with the sampling interval . Dene a collection
of information generated by the sampled observations, y
0
, y

, y
2
, , up to the current time t as:

F
y
k
:= {y
l
, l {1, 2, , k} with k t and (k + 1) > t} (34)
Note that

F
y
k
converges to F
y
t
as converges to zero. Thus, the state estimation problem for the MC {
,
k
} can
be stated as:
Problem 3: For a given SHS x
t
, an approximating MC {
,
k
} and an observation process y
t
, X
MC
, we
want to compute

,
k
() := Pr
_

,
k
= |

F
y
k
_
(35)
where

F
y
k
is dened in (34).
September 24, 2011 DRAFT
15
Remark 6: The solution to Problem 3 dened on X
MC
is an approximation to the solutions to Problems 1 and
2 dened on X. Since

F
y
k
converges to F
y
t
, and
,
k
converges to x
t
as , 0, by Lebesgues dominated
convergence theorem [28], f C,

X
MC
f()
,
k
() E
P
[f(x
t
)|F
y
t
]
as , 0. Thus, the solution to Problem 2 can be approximated by the L.H.S. of the above equation. Letting
f(x) := 1
{x|xG()}
(x) in the above equation gives:

X
MC
f(

)
,
k
(

) =
,
k
()
_
G()
p
t
(x)dx
as , 0. Suppose p
t
(x) is continuous, then, x X, we can nd X
MC
such that x G(), and the
solution to Problem 1, p
t
(x), can be approximated by
,
k
()/|G()|, where |G()| :=
_
G()
dx is the area of
the grid G().
Dene y
D
k
:= y
k
y
(k1)
. By Bayes theorem and the total probability theorem:

,
k
() =
1
C
Pr
_
y
k

,
k
= ,

F
y
k1
_
Pr
_

,
k
=

F
y
k1
_
=
1
C
Pr
_
y
k
y
(k1)

,
k
= ,

F
y
k1
_

X
MC
Pr
_

,
k
=

,
k1
=

,

F
y
k1
_
Pr
_

,
k1
=

F
y
k1
_
=
1
C
Pr
_
y
D
k

,
k
= ,

F
y
k1
_

X
MC
P(,

)
,
k1
(

)
(36)
where C is a normalizing constant and Pr
_
y
D
k

,
k
= ,

F
y
k1
_
= Pr
_
y
D
k

,
k
=
_
is a likelihood function.
Given that
,
k
= , let T
s
= in the observation model (9). Then,
y
D
k
h() + w(k)
where w(k) are the i.i.d. Gaussian random variable with zero mean and covariance matrix I
m
. Thus, the likelihood
function in (36) can be computed explicitly:
Pr
_
y
D
k
|
,
k
=
_
= N
_
y
D
k
; h(), I
m
_
=
1

2
exp
_

1
2
_
y
D
k
h()
_
T
_
y
D
k
h()
_
_
exp
_
h
T
()y
D
k

1
2
h()
2
_
(37)
where N (; h(), I
m
) is the pdf of the Gaussian random variable with mean h() and covariance matrix
I
m
. Plugging (37) into (36) gives:

,
k
() =
1
C

exp
_
h
T
()y
D
k

1
2
h()
2
_

X
MC
P(,

)
,
k1
(

) (38)
September 24, 2011 DRAFT
16
TABLE I: Numerical algorithm for approximating Markov chain state estimation.
1. Initialization :
Compute initial distribution of {
,
k
} dened on the discretized state
space X
MC
:
,
0
() =

G()
p
0
(x)dx for all X
MC
.
2. Iteration :
At each time k
do :
Compute y
D
k
using (9) with T
s
= .
Compute
,
k
() using (38) for all X
MC
.
k = k + 1
end
where C

is a normalizing constant. Thus, for a given initial condition distribution

,
0
and a given observation
history

F
y
k
up to time t, (38) governs the time evolution of the state estimates {
,
k
}, which is the core of the
numerical algorithm to solve Problem 3 (see also Table I).
Theorem 8: (Weak convergence of the MC state estimation results
,
k
) For any k N, the
,
k
computed by
(38) converges in distribution to x
t
at t = k as and converge to zero.
Proof: Dene an unnormalized MC {
,
k
} which has the same initial condition as {
,
k
} for k = 0. The
iteration of {
,
k
} is given in the following equation:

,
k
() = exp
_
h
T
()y
D
k

1
2
h()
2
_

X
MC
P(,

)
,
k1
(

) (39)
where P(, ) is the transition probability of the MC {
,
k
} designed in (32). Thus, at each step k,
,
k
can be
computed by normalizing
,
k
(see also (38)).
Dene

,
k
:= exp
_
k

i=0
h
T
(
,
i
)y
D
i

1
2

i=0
h(
,
i
)
2
_
Then (39) can be rewritten as:

,
k
() = E
Q
k
_
1
{
,
k
=}
(
,
k
)

,
k

F
y
k
_
where the measure Q
t
(t = k) is dened in Section III-A. Under the measure Q
t
, y
D
k,
and
,
k
are independent.
By the above equation, for any function f C,
E
Q
k
_
f(
,
k
)
_
= E
Q
k
_
f(
,
k
)

,
k,

F
y
k
_
By the weak convergence result of {
,
k
} in Theorem 7 and the fact that

,
k
and

F
y
k
converge to
t
and F
y
t
at t = k respectively, the following convergence result holds: E
Q
t
_
f(
,
k
)

,
k,

F
y
k
_
E
Q
t
[f(x
t
)
t
|F
y
t
] at
t = k as , 0. Therefore, by normalization, f C, E
P
_
f(
,
k
)
_
E
P
[f(x
t
)|F
y
t
] = E
P
[f( x
t
)] at t = k
as , 0.
September 24, 2011 DRAFT
17
Since the MC state estimates {
,
k
} converges to x
t
in distribution, the approximation error of the proposed
numerical algorithm can be arbitrarily small as , 0.
V. SIMULATIONS
z
(1)
z
(2)
0

Aircraft 1
Aircraft 2
Fig. 3: Aircraft tracking scenario.
In this section, the proposed numerical algorithm is tested with an aircraft tracking scenario as shown in Figure
3. Consider the relative motion of two aircraft modeled as point masses in the vertical plane. The coordinate frame,
(z
(1)
, z
(2)
), is attached to Aircraft 1 with the z
(1)
axis pointing the heading direction of Aircraft 1 and the z
(2)
axis pointing up. In the scenario, Aircraft 1 is chasing Aircraft 2 and using its onboard radar to track the motion of
Aircraft 2. The onboard radar provides noisy information about Aircraft 2s relative position w.r.t. Aircraft 1 in the
(z
(1)
, z
(2)
) plane. We use z
t
= [z
(1)
t
z
(2)
t
]
T
R
2
to denote Aircraft 2s relative position w.r.t. Aircraft 1 at time t.
Suppose Aircraft 1 and 2 start from the same altitude at time t = 0 with the same heading direction and the speed
of Aircraft 1 is larger than that of Aircraft 2. Thus, the relative distance of the two aircraft is decreasing. However,
to avoid a potential midair conict, when the relative distance of the two aircraft is smaller than a threshold value,
Aircraft 2 starts to climb with a constant climb rate while Aircraft 1 maintains its original heading. Such a switching
behavior can be described by the SHS.
Let Q = {1, 2} be the discrete state space with q = 1 being the discrete state that both aircraft y straight with
the same cruise speed and q = 2 being the discrete state that Aircraft 2 climbs with a constant climb rate while
Aircraft 1 maintains its original speed and heading. The continuous dynamics of the SHS with the two discrete
states is given by:
q = 1 : dz
t
=
_
_
V
h
0
_
_
dt + gdW
t
q = 2 : dz
t
=
_
_
V
h
V
c
_
_
dt + gdW
t
where V
h
= 10m/s is the relative speed of the two aircraft on the horizontal axis z
(1)
; V
c
= 10m/s is the
climb rate of Aircraft 2; g = 15m and W
t
is the 2-dimensional Brownian motion modeling the uncertainties in the
continuous dynamics.
September 24, 2011 DRAFT
18
q = 1 q = 2
Guard condition: z
(1)
t
d
min
Guard condition: z
(1)
t
> d
min
Fig. 4: SHS discrete state transition model.
The discrete dynamics of the SHS models how the discrete state q changes (see also Figure 4). According to
the Federal Aviation Administration (FAA)s rule, the minimum en route separation distance of two aircraft in the
horizontal plane is d
min
= 5nm, i.e., Aircraft 2 should climb when the horizontal distance between the two aircraft
z
(1)
t
is smaller than d
min
. Thus, the discrete state q should jump to 2 when the guard condition that z
(1)
t
d
min
is satised; and q should jump to 1 when z
(1)
t
> d
min
. However, such discrete state transitions are likely but not
exactly to happen when the guard conditions are satised due to various uncertainties including the navigation error,
different pilots reactions, etc. Thus, the Poisson transitions with the transition rate function : (z
(1)
, q) R
+
can
be used to model such uncertain discrete state transition behaviors which are dependent on the continuous state z
t
.
Figure 5 shows a parameterized transition rate function : (z
(1)
, q) R
+
which is explained as follows:
Case 1: If z
(1)
> d
min
, the transition rate function (z
(1)
, 1) is small and (z
(1)
, 2) is large, i.e., the probability
of the transition from q = 1 to q = 2 is small while the probability of its inverse is large. Thus, the SHSs
discrete state is dominated by q = 1 when the horizontal distance between the two aircraft is much larger than
the threshold d
min
.
Case 2: If z
(1)
d
min
, (z
(1)
, 1) is large and (z
(1)
, 2) is small. By the analysis similar to Case 1, the SHS
discrete state is dominated by q = 2 when the two aircraft get dangerously close.
9000 9050 9100 9150 9200 9250 9300 9350 9400 9450 9500
5
10
15
z
(1)
(m)

(z
(1)
,1)
(z
(1)
,2)
d
min
Fig. 5: Transition rate function : (z
(1)
, q) R
+
.
Note that the stochastic kernel R which describes the continuous state zs distribution after discrete state transitions
is trivial, because the continuous state z
t
does not jump when a discrete state transition happens.
Aircraft 1s onboard radar provides observations of the relative position of Aircraft 2 w.r.t. Aircraft 1, i.e., the
continuous state z
t
. At each sampling time, the output of the radar is the noisy information about the relative
September 24, 2011 DRAFT
19
distance between the two aircraft and the angle between the line connecting the two aircraft and the z
(1)
axis
(see also Figure 3). The observation model (9) is given by:
y
D
kT
s
=
_
_

kT
s

kT
s
_
_
=
_

_
tan
1
_
z
(2)
kT
s
z
(1)
kT
s
_
z
kT
s

_ + w(k)
where T
s
= 0.1sec is the radars sampling time; = diag(0.028, 40) is a constant positive denite matrix
representing the radars accuracy in measuring distance and angle ; and w(k) are i.i.d. Gaussian random variables
with zero mean and covariance matrix T
s
I
2
.
Figure 6 shows one realization x
t
(), starting from x
0
= (z
0
, q
0
) with z
0
= [9480 0]
T
m and q
0
= 1, and the
corresponding noisy radar observations y
D
kT
s
. The SHSs evolution x
t
() dened on X = R
2
Q is projected onto
R
2
by showing the continuous state trajectory z
t
(). Different colors are used to show the different discrete states
at each time. When the horizontal distance of the two aircraft is larger than the threshold d
min
, the continuous
dynamics is dominated by q = 1 and when the horizontal distance is smaller than d
min
, the continuous dynamics
is dominated by q = 2.
9050 9100 9150 9200 9250 9300 9350 9400 9450 9500
200
100
0
100
200
300
z
(1)
(m)
z
(
2
)

(
m
)


Radar observations
Trajectory of z
t
(q=1)
Trajectory of z
t
(q=2)
d
min
x
0
Fig. 6: One realization of the SHS x
t
() and the radar observations.
The numerical algorithm proposed in Section IV-C is used to estimate the SHS state by using the observation
measurements y
D
kT
s
. An MC is constructed to approximate the dynamics of the SHS with = T
s
= 0.1sec and
= [5 5]
T
m. The numerical algorithm computes
,
k
at each time k, which approximates the solution to Problem
1.
By Remark 6, f C, E
P
[f(x
t
)|F
y
t
] can be approximated by

X
MC
f()
,
k
. By Remark 5, we choose
f(x) = z and then E
P
[f(x
t
)|F
y
t
] = E
P
[z
t
|F
y
t
] is the estimated aircraft position. Dene the aircraft position
September 24, 2011 DRAFT
20
0 5 10 15 20 25 30 35 40
0
100
200
300
400
Time (sec)
D
i
s
t
a
n
c
e

(
m
)


Observation error
Estimation error
Fig. 7: Observation error and state estimation error.
0 5 10 15 20 25 30 35 40
0
0.2
0.4
0.6
0.8
1
Time (sec)
E
s
t
i
m
a
t
e
d

D
i
s
c
r
e
t
e

S
t
a
t
e

P
r
o
b
a
b
i
l
i
t
y


q
t
=1
q
t
=2
Fig. 8: Estimated discrete state probabilities.
observation error e
Obs
t
and the estimation error e
Est
t
at t = kT
s
as:
e
Obs
kT
s
:=
_
_
_
_
_
_
_
_

kT
s
cos(
kT
s
)

kT
s
sin(
kT
s
)
_
_
z
kT
s
_
_
_
_
_
_
, e
Est
kT
s
:= E
P
[z
kT
s
|F
y
k
] z
kT
s

where e
Est
kT
s
can be approximately computed by the proposed algorithm. Figure 7 compares e
Obs
kT
s
and e
Est
kT
s
, and
shows that our algorithm can give accurate estimates of the true continuous state z
t
.
Also, by Remark 5, if we choose f(x) = 1
{x|q=1}
(x) or f(x) = 1
{x|q=2}
(x), then the estimated discrete state
probabilities Pr{q
t
= 1|F
y
t
} or Pr{q
t
= 2|F
y
t
} can be approximately computed by the proposed algorithm. The
estimated discrete probabilities are shown in Figure 8, which is in accordance with the previous theoretical analysis
that the discrete state is initially dominated by q
t
= 1 and later by q
t
= 2.
VI. CONCLUSIONS
This paper has considered the hybrid state estimation problem for the Stochastic Hybrid System (SHS) which
is a class of Markov stochastic processes dened on the hybrid state space. Two equivalent dynamical equations
(ltering equations) similar to the Zakai equation have been derived to formulate the hybrid state estimation problem
by describing the evolution of the probability distribution function (pdf) of the SHSs hybrid state conditioned on
the observations. A numerical algorithm based on stochastic approximation has been proposed to solve the ltering
September 24, 2011 DRAFT
21
equations. Firstly, a Markov Chain (MC) dened on the discretized hybrid state space is designed to approximate
the dynamics of the SHS. Then, the hybrid state estimation problem is approximated by estimating the state of the
MC. It has been proved in this paper that the numerical solution converges to the exact solution to the hybrid state
estimation problem as the grid size and the interpolation time interval of the MC converge to zero. The proposed
algorithm has been validated with an aircraft tracking example in which the aircraft dynamics is modeled by the
SHS and the observations are made by an onboard radar. Simulations have shown that the proposed numerical
algorithm can compute the accurate estimate of the continuous state and the discrete state of the SHS.
REFERENCES
[1] M. Prandini and J. Hu. Application of reachability analysis for stochastic hybrid systems to aircraft conict prediction. IEEE Transactions
on Automatic Control, 54(4):913 917, Apr. 2009.
[2] W. Glover and J. Lygeros. A stochastic hybrid model for air trafc control simulation. In Hybrid System: Computation and Control, pages
372386. Springer-Verlag, Mar. 2004.
[3] J. Hespanha. Stochastic hybrid systems: Applications to communication networks. In Hybrid Systems: Computation and Control, pages
387401. Springer-Verlag, Mar. 2004.
[4] D. Riley, X. Koutsoukos, and K. Riley. Verication of biochemical processes using stochastic hybrid systems. In IEEE 22nd International
Symposium on Intelligent Control, 2007. ISIC 2007., pages 100 105, Oct. 2007.
[5] J. Westman and F. Hanson. State dependent jump models in optimal control. In 38th IEEE Conference on Decision and Control, pages
2378 2384, Dec. 1999.
[6] D. Sworder and J. Boyd. Estimation Problems in Hybrid Systems. Cambridge University Press, 1999.
[7] Y. Bar-Shalom, X. R. Li, and T. Kirubarajan. Estimation with Applications to Tracking and Navigation. John Wiley & Sons, 2001.
[8] C. E. Seah and I. Hwang. Stochastic linear hybrid systems: Modeling, estimation, and application in air trafc control. IEEE transactions
on Control Systems Technology, 17(3):563575, 2009.
[9] W. Liu, C. Seah, and I. Hwang. Estimation algorithm for stochastic linear hybrid systems with quadratic guard conditions. In 48th IEEE
Conference on Decision and Control, pages 3946 3951, Dec. 2009.
[10] W. Liu and I. Hwang. Robust estimation and fault detection and isolation algorithms for stochastic linear hybrid systems with unknown
fault input. IET Jounal on Control Theory and Applications, 5(12):1353 1368, Aug. 2011.
[11] L. Mihaylova, E. Semerdjiev, and X. R. Li. Detection and localization of faults in system dynamics by IMM estimator. In In Proc. of the
2 nd Intern. Conf. on Multisource-Multisensor Information Fusion, pages 937943, 1999.
[12] S. Tafazoli and X. Sun. Hybrid system state tracking and fault detection using particle lters. IEEE Transactions on Control Systems
Technology, 14(6):10781087, Nov. 2006.
[13] E. Cinquemani, M. Micheli, and G. Picci. Fault detection in a class of stochastic hybrid systems. In 43rd IEEE Conference on Decision
and Control, volume 3, pages 3197 3203, Dec. 2004.
[14] W. Liu, J. Wei, M. Liang, Y. Cao, and I. Hwang. Multi-sensor data fusion and fault detection for aircraft tracking in air trafc control.
Submitted to IEEE Transactions on Aerospace and Electronic Systems, 2011.
[15] W. Liu and I. Hwang. Optimal sensor scheduling for hybrid estimation. In IEEE American Control Conference 2011, pages 30123017,
Jun. 2011.
[16] A. Balluchi, L. Benvenuti, M. D. Benedetto, and A. Sangiovanni-Vincentell. Design of observers for hybrid systems. In Hybrid System
Computation and Control (HSCC), volume 2289, pages 7689, 2002.
[17] M. W. Hofbaur and B. C. Williams. Mode estimation of probabilistic hybrid systems. In Hybrid Systems: Computation and Control, pages
253266. Springer Verlag, 2002.
[18] A. Alessandri and P. Coletta. Design of luenberger observers for a class of hybrid linear systems. In Proceedings of the 4th International
Workshop on Hybrid Systems: Computation and Control, pages 718, London, UK, 2001. Springer-Verlag.
[19] J. Ru and X. R. Li. Variable-structure multiple-model approach to fault detection, identication, and estimation. IEEE Transactions on
Control Systems Technology, 16(5):10291038, Sept. 2008.
September 24, 2011 DRAFT
22
[20] G. Ferrari-Trecate, D. Mignone, and M. Morari. Moving horizon estimation for hybrid systems. IEEE Transactions on Automatic Control,
47(10):1663 1676, Oct. 2002.
[21] X. Koutsoukos. Estimation of hybrid systems using discrete sensors. In 42nd IEEE Conference on Decision and Control, volume 1, pages
155 160, Dec. 2003.
[22] X. Koutsoukos, J. Kurien, and F. Zhao. Estimation of distributed hybrid systems using particle ltering methods. In Hybrid System
Computation and Control HSCC03, pages 298313, 2003.
[23] H.A.P. Blom and E.A. Bloem. Particle ltering for stochastic hybrid systems. In 43rd IEEE Conference on Decision and Control, volume 3,
pages 3221 3226, Dec. 2004.
[24] J. Hu, J. Lygeros, and S. Sastry. Towards a theory of stochastic hybrid systems. In Third International Workshop on Hybrid Systems:
Computation and Control, pages 160173. Springer, 2000.
[25] L. M. Bujorianu and J. Lygeros. Toward a general theory of stochastic hybrid systems. In Stochastic Hybrid Systems: Theory and Safety
Critical Applications, volume 337 of Lecture Notes in Control and Information Sciences, pages 330. Springer Verlag, Berlin, 2006.
[26] J. Xiong. An Introduction to Stochastic Filtering Theory. Oxford University Press, 2008.
[27] B. ksendal. Stochastic Differential Equations: An Introduction with Applications. Springer, 1998.
[28] P. Billingsley. Probability and Measure. John Wiley & Sons, 1995.
[29] M.L. Bujorianu and J. Lygeros. Theoretical foundations of general stochastic hybrid processes. In 16th International Symposium on
Mathematical Theory of Networks and Systems, 2004.
[30] M.L. Bujorianu and J. Lygeros. General stochastic hybrid systems: Modelling and optimal control. In 43th IEEE Conference of Decision
and Control, pages 18721877, 2004.
[31] A. Bain and D. Crisan. Fundamentals of Stochastic Filtering. Springer, 2000.
[32] R. J. Elliott, F. Dufour, and D. D. Sworder. Exact hybrid lters in discrete time. IEEE Transactions on Automatic Control, 41(12):1807
1810, Dec. 1996.
[33] R.J. Elliott and J. van der Hoek. A nite-dimensional lter for hybrid observations. IEEE Transactions on Automatic Control, 43(5):736
739, May 1998.
[34] R.J. Elliott, L. Aggoun, and J.B. Moore. Hidden Markov Models: Estimation and Control. Springer, 1995.
[35] H.J. Kushner and P. Dupuis. Numerical Methods for Stochastic Control Problems in Continuous Time (2nd edition). Springer, 2001.
[36] K. Holley. Applications of Multigrid Algorithm to Solving the Zakai equation of nonlinear ltering with VLSI implementation. PhD thesis,
University of Maryland, 1986.
[37] H. Zhang and D. Laneuville. Grid based solution of zakai equation with adaptive local renement for bearings-only tracking. In 2008
IEEE Aerospace Conference, pages 1 8, Mar 2008.
[38] R. Durrett. Stochastic Calculus: A Practical Introduction. CRC Press LLC, 1996.
[39] X. Koutsoukos and D. Riley. Computational methods for verication of stochastic hybrid systems. IEEE Transactions on Systems, Man
and Cybernetics, Part A, 38(2), 2008.
September 24, 2011 DRAFT