942 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 12, NO.

4, DECEMBER 2011
An Efcient Computational Architecture for a
Collision Early-Warning System for Vehicles,
Pedestrians, and Bicyclists
Daniel Greene, Juan Liu, Member, IEEE, Jim Reich, Yukio Hirokawa,
Akio Shinagawa, Hayuru Ito, and Tatsuo Mikami
AbstractWe describe a computational architecture of a col-
lision early-warning system for vehicles and other principals.
Early-warnings allow drivers to make good judgments and to
avoid emergency stopping or dangerous maneuvering. With many
principals (vehicles, pedestrians, bicyclists, etc.) coexisting in a
dense intersection, it is difcult to predict, even a few seconds
in advance, since there are many possible scenarios. It is a ma-
jor challenge to manage computational resources and human
attention resources so that only the more plausible collisions are
tracked, and of those, only the most critical collisions prompt
warnings to drivers. In this paper, we propose a two-stage collision
risk assessment process, including the following: 1) a preliminary
assessment via simple efcient geometric computations, which
thoroughly considers surrounding principals and identies likely
potential accidents, and 2) a specialized assessment that computes
more accurate collision probabilities via sophisticated statistical
inference. The whole process delivers an expected utility assess-
ment to available user interfaces (UIs), allowing the UIs to make
discriminating choices of when to warn drivers or other principals.
Index TermsCollision warning, risk assessment, road vehicle
safety.
I. INTRODUCTION
I
MPROVED sensing and communication technologies have
been incorporated in various intelligent transportation sys-
tems to provide users with signicantly better information
about their surrounding environment and the road ahead. Vehi-
cles are now equipped with better sensors, and pedestrians and
bicyclists are carrying intelligent devices. All this information
can be exploited to prevent accidents. It is extremely important
to warn principals (vehicles, bicyclists, pedestrians, etc.) early
to give them time to react to warnings and to apply their own
Manuscript received September 11, 2009; revised July 21, 2010; accepted
November 20, 2010. Date of publication February 4, 2011; date of current
version December 5, 2011. This work was supported by Fujitsu Ltd. The
Associate Editor for this paper was P. Grisleri.
D. Greene and J. Liu are with the Palo Alto Research Center, Palo Alto, CA
94304 USA (e-mail: greene@parc.com; juan.liu@parc.com).
J. Reich is with Streetline, San Francisco, CA 94105 USA (e-mail:
jreich@alum.mit.edu).
Y. Hirokawa, A. Shinagawa, and H. Ito are with Fujitsu Ltd., Tokyo
105-7123, Japan (e-mail: y.hirokawa@jp.fujitsu.com; shina@jp.fujitsu.com,
hayuru@jp.fujitsu.com, nra24256@nifty.com).
T. Mikami is a consultant in the eld of intelligent transportation, residing in
San Jose, CA (e-mail: nra24256@nifty.com).
Color versions of one or more of the gures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identier 10.1109/TITS.2010.2097594
additional judgment before action is taken to avoid accidents.
This is in contrast to immediate warning systems, which are
important but often result in emergency stopping or maneuver-
ing and can be uncomfortable or even cause other accidents.
However, early-warning poses signicant computational dif-
culty, particularly at a busy intersection where there are many
principals, and anticipating ahead, even a few seconds, involves
tracking an explosion of possible trajectories. We must take
a selective approachdevoting computation to the likely col-
lisions rather than all possible collisions. Even then, we will
process hundreds of likely collisions, and we must be more
selective with warnings to humans. Unnecessary warnings may
distract users and make the system unusable.
In this paper, we propose a computationally efcient col-
lision early-warning system (CLEWS). The architecture is
shown in Fig. 1. It consists of three main components: 1) sens-
ing and communication for signal collection from the physical
surrounding environment; 2) user interface (UI), which prompts
warnings to drivers; and 3) a reasoning layer, which is the
main focus of this paper. We do not discuss basic sensing and
communication in this paper; we assume that the principals
reasoning layer receives sensing information about other nearby
principals. The algorithms in the reasoning layer are designed
to tolerate moderately noisy sensor data. Likewise, we are not
addressing the design of in-vehicle UIs. Our system would
benet from recent UI research on low-distraction ways of
issuing warnings, e.g., in the periphery of head-up displays or
with directional sound. We have taken great care to make the
output of the reasoning layer useful to a wide variety of UIs
(e.g., audio, in-dash, heads-up, and roadside signage) by pro-
viding an expected utility assessment that includes information
necessary to make critical decisions on if, when, and how to
issue warnings. The actual decisions are customized to the UIs.
A. Dual Assessment
A central aspect of our architecture is the division of the
reasoning layer into two parts, which we will refer to as
preliminary assessment and specialized assessment. This
architecture is shown in Fig. 1. The preliminary assessor takes
a management role. Initially, the preliminary assessor ex-
amines all pairs of principals for possible collision scenarios.
For any potential collision, it creates a specialized assessor to
closely examine collision risks within a lookahead horizon. The
1524-9050/$26.00 2011 IEEE
GREENE et al.: COMPUTATIONAL ARCHITECTURE FOR CLEWS FOR VEHICLE, PEDESTRIAN, AND BICYCLIST 943
Fig. 1. Overview of the architecture.
lookahead horizon value is empirically chosen. For instance,
a timeline of 3 seconds is normally sufcient for collision
avoidance without the need for emergency maneuvering. The
preliminary assessor maintains multiple specialized assessors,
allocating computational resources to the most urgent and
serious potential collisions. It also interfaces to the UI, allowing
the UI to decide about which potential collisions to warn the
driver and when. In contrast, the specialized assessor can be
considered as a relatively computationally expensive tool for
accurate risk assessment but invoked only on demand by the
preliminary assessor.
This dual-assessment approach effectively solves three criti-
cal problems.
1) The complexity of activity that might cause accidents:
The geometric approach of the preliminary assessor is
highly optimized for covering large numbers of principals
and scenarios.
2) Managing computational resources: By rst computing
preliminary assessments of a wide range of possible
accidents, the reasoning layer is in an ideal position to
allocate computational resources to the computationally
more expensive specialized assessors.
3) Model selection: The performance of risk assessment is
highly dependent on the validity of the models describing
the principals behaviors. In real-world trafc, a large
percentage of principals follow predicable patterns; the
preliminary assessor sorts through all these patterns and
selects models for specialized assessors.
In the next two sections, we describe the design of the dual
assessment approach. Roughly speaking, the preliminary asses-
sor does a rapid geometric identication of potential accidents
and starts specialized assessors to do more detailed statistical
analyses of potential accidents when necessary.
B. Related Work
Among the collision detection/avoidance literature, CLEWS
is unique in its emphasis on providing advance warning in dense
intersections. It includes techniques to deal with the compu-
tational complexity and long-term modeling of the principals
behaviors. Other collision detection work has focused on emer-
gency braking [1], [2] and less complex (e.g., rural) intersec-
tions [3], [4]. Often, linear dynamics models are used [5], [6],
which are suitable and efcient for short-term prediction but are
limited in modeling driver maneuvering behavior over the time
scales necessary for early-warning. In contrast, the two-stage
risk assessment approach that CLEWS takes is specically
designed to handle busy intersections with complex principal
motion. This is the main differentiator. Another key aspect of
our system is the concept of risk assessment. This concept has
appeared in the EUs PROTECTOR project [7], which produces
a conict risk analysis. However, the PROTECTOR system
simply focuses on probability, rather than taking into account
the benet of issuing a particular kind of warning, and does not
consider warning timing or adaptation to different types of UI.
Statistical approaches are widely adopted in collision risk
assessment; for examples, see [8][11]. Statistical inference
methods such as parametric lters (e.g., Kalman lter) or
nonparametric techniques (e.g., Monte Carlo simulation) have
been applied to decide the collision risk based on observations.
Our specialized assessor follows the same line. The novelty of
our approach is the carefully designed computational tradeoff.
We use the preliminary assessor for quick screening and the
specialized assessor for more accurate statistical risk assess-
ment. Because the preliminary assessment will be followed
by specialized assessment, the preliminary assessment can be
optimized for fast computation, it uses a geometric form of
reachable sets but does not need to compute the reachable
sets with stochastic detail (compare [9]). Similarly, because
the specialized assessment will be preceded by a screening
with preliminary assessment, the specialized assessment can
be enhanced with richer and more computationally challenging
modeling. Here, we have included driver alertness in the model-
ing, which we believe will be valuable in avoiding unnecessary
warnings when drivers are already responding to potential col-
lisions. Furthermore, we have developed a variety of statistical
techniques for this kind of specialized assessment, varying
from a relatively simple interacting multiple model (IMM)
Kalman lter to more computationally intense particle lters.
As described in Section III, these techniques can be chosen to
obtain a suitable tradeoff between computation complexity and
risk accuracy.
II. PRELIMINARY ASSESSOR
In the dual assessment framework, the preliminary assessor
is designed to act as a fast screener; among the hundreds of
principals at a busy intersection, the preliminary assessor picks
out possible collision pairs on which the system can focus
resources. The requirement on the preliminary assessor is on
fast computation rather than high accuracy. Here, we describe a
geometric approach optimized for computation efciency.
A. Motion of Principals
Preliminary assessment uses a library of likely trajectories
for principals and applies efcient geometric tests to determine
if two principals might collide. In our architecture, the motion
of a principal is decomposed into a trajectory in the xy
plane and a dynamics along the trajectory. This is based
on the observation that principals usually follow highly pre-
dictable trajectories, e.g., the curves followed by right-turning
vehicles in an intersection are very similar, but their speed and
944 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 12, NO. 4, DECEMBER 2011
Fig. 2. Dynamics and a trajectory that, when combined, describe a vehicle
slowing as it makes a left turn.
acceleration along those trajectories can be highly variable.
A trajectory T(s) is a vector-valued function from an arc
length parameter s to the physical plane xy. For our prototype
system, T is a piecewise function of line segments and arc
segments. A dynamics D(t) is a function from time t to
distance s. For our prototype system, D is a piecewise function
of periods of steady speed (i.e., at +b) and accelerations or
decelerations (i.e., at
2
+bt +c). A motion is a composition
of a trajectory and a dynamics: T(D(t)) (see Fig. 2 for an
example). Such a decomposition of motion is used elsewhere in
vehicle safety analysis, e.g., in [8], a road coordinate system is
used to separate the trajectory from the dynamics, and in [9], a
trajectory is used as a frame of reference for a stochastic dynam-
ics model. We have chosen a simple set of geometric primitives
to represent the motion of principalsprecise delity is not
critical since all of the computations described later will make
allowance for the principals not precisely following trajectories
and dynamics.
B. Geometric Computation: Segmented Cones
The central computational element of the preliminary as-
sessor is the segmented cone. Segmented cones are worst
case descriptions of where principals might travel as they
follow scenarios. They are generated from a library of scenarios
associated with the local road geometry. For example, there
might be scenarios for vehicles stopping at an intersection,
turning right, changing lanes, etc., and scenarios for pedestrians
crossing in crosswalks or jaywalking.
A segmented cone is a region dened in 3-D space (including
the time dimension, i.e., (x, y, t)). It covers most of the likely
future locations of a principal following a particular scenario.
A segmented cone is similar to a reachability analysis of a
principals future location, except it is conditioned on a scenario
being followed, and it excludes very low probability locations.
Thus, e.g., if the scenario is vehicle makes a right turn, then
a segmented cone for that scenario describes a region S R
3
.
If the vehicle does make a right turn, then, with very high prob-
ability, the vehicles future coordinates (x, y, t) are inside of S.
As t grows larger, it is harder to predict where a principal will be
located; therefore, segmented cones will are with increasing t,
giving them the inverted cone appearance shown in Fig. 3.
We refer to these as segmented cones, because they are
assembled from prismatoids joined together at a xed set of t
values: t
0
, t
1
, t
2
, t
3
, . . .. For each prismatoid, there is a bound-
ing box, and there is an overall bounding box for the entire
segmented cone. These bounding boxes are shown in red and
Fig. 3. Segmented cone with its bounding boxes.
blue in Fig. 3, respectively. The bounding boxes are the basis of
highly efcient cone intersection. It is easy to decide whether
two bounding boxes intersect; only comparison operations are
involved. The overall bounding boxes can be compared rst to
determine if more comparisons are necessary.
The generation of cones is based on the context of the
principalthis context suggests which scenarios are likely and
cones are generated for all likely scenarios (sometimes more
than one per principal, as we shall see shortly). The context is
dened as a conjunction of trigger conditions, e.g., a vehicle
is near the center of a left-turn lane and within 40 m of an in-
tersection, so a left-turning cone should be generated. (Trigger
conditions can be decided based on considerations as the type
of the principal, the location and velocity of the principal, and
the state of nearby trafc lights.) Once a cone is triggered, the
cone itself is created by combining a priori information from
the scenario with the current state of the principal to project a
cone forward in time. Fig. 4(a) illustrates the information used
in cone generation, and Fig. 4(b) illustrates the cone generated.
In this case, a priori information includes a trajectory from the
scenario (a left turn), and the current state includes the princi-
pals location and velocity. These are combined as follows: The
principals current displacement from the trajectory is applied
to the entire trajectory. Various extreme values for the para-
meters such as maximum acceleration or maximum trajectory
deviation (a priori information associated with the scenario)
are applied to the principals current state to generate polygons
at future time values. For example, one point of the polygon
at layer t (i.e., the forward left corner) would be determined
by computing the largest possible value for the dynamics (i.e.,
s
max
= a
max
t
2
+v
current
t +s
current
) and then, at T(s
max
) on
the trajectory, adding an additional displacement, which we
will refer to as the are, that is perpendicular and tangent
to this point and adjustable with a scalar parameter f. The
are accounts for the inaccuracy in the principals following the
trajectory (e.g., lane position) and inaccuracy in the trajectory
representing the scenario. The forward left corner would be at
T(s
max
) + (T

(s
max
) +T

(s
max
))f. Other extreme points on
the polygon are computed in a similar way, and the polygons on
adjacent layers form the prismatoids that are used to create the
bounding boxes in the segmented cone. This way, the scenario
is customized to the principals, e.g., a faster moving vehicle will
generate a more extended cone.
GREENE et al.: COMPUTATIONAL ARCHITECTURE FOR CLEWS FOR VEHICLE, PEDESTRIAN, AND BICYCLIST 945
Fig. 4. Cone generation. (a) Top panel-trigger conditions and parameters. (b) Bottom panel-generated cones.
The extreme values of the parameters, e.g., the maximum
acceleration a
max
, are chosen to cover most driving behavior
with high probability, and therefore, the polygons formed by
sampling combinations of extremes (e.g., max acceleration,
with max lane deviation) will also cover the principals future
locations with high probability, assuming a linear trajectory.
However, nonlinearities such as curved trajectories and accel-
eration between time slices mean that prismatoids will not
necessarily cover all the extreme values. Because this is part
of a dual assessment, where this preliminary assessment will
be followed by a more accurate specialized assessment, we
chose to favor computational efciency in our initial prototype,
using the are parameter previously described to dilate the
prismotoids. This means that the preliminary assessment is
less precise, with higher false positives. Alternative approaches,
such as computing more perimeter points on the polygons
and/or computing ner time slices in the segmented cones,
would make the preliminary assessment more precise and re-
quire less dilation. This tradeoff needs further exploration.
C. Computation of Preliminary Assessment
Because segmented cone intersection is so efcient, it can
be frequently repeated (e.g., intersecting all cones two to ve
times per second). In our prototype system, a vehicle repeatedly
triggers cones for itself and any other principals being tracked.
The generated cones are compared with each other to determine
if they intersect. When two segmented cones intersect [e.g.,
Fig. 5(b)], this indicates a possible collision, and a preliminary
assessment of that collision is computed. Because preliminary
assessment is based on large worst case sized cones, the po-
tential collision may still be extremely unlikely, and therefore,
we will choose some scenarios for a full specialized assess-
ment that will arrive at probabilities and expected utilities that
can help determine if warnings are warranted. The preliminary
assessment will enable the system to allocate resources to those
scenarios most deserving specialized assessment.
Fig. 6 shows in a stylized manner how a preliminary as-
sessment is computed from a segmented cone intersection. We
Fig. 5. Cone scenarios. (a) Multiple cones (for stopping, proceeding straight,
and turning left or right) are triggered as a vehicle approaches an intersection.
(b) Two vehicle cones intersect. They might collide. A preliminary assessment
is computed.
Fig. 6. Deriving a preliminary assessment from the segmented cone
intersection.
estimate time-to-accident as the point where the cones rst
intersect. The probability of collision is computed by separately
considering each of the overlapping tiers level by level, esti-
mating the probability based on the amount of overlap, and
then using (conservatively) the highest probability among the
overlapping tiers. For two overlapping tiers, the probability is
estimated as
P(s
i
, s
j
) = p
s
i
p
s
j
_
o
a
i
__
o
a
j
_
(1)
where s
i
and s
j
are the two scenarios that caused the two cones
to be generated, a
i
and a
j
are the xy areas of the two bounding
boxes, and o is the xy area of the overlap of these two
946 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 12, NO. 4, DECEMBER 2011
Fig. 7. Overall ow of computation.
bounding boxes. Equation (1) includes the prior probabilities
for the two individual scenarios p
s
i
and p
s
j
. (These are xed
probabilities in the scenario library.) This formula is heuristic.
We also estimate seriousness based on the kinetic energy and
vulnerability of the principals. Once the preliminary assessment
is completed, the product of the probability and the seriousness
gives a utility estimate to rank the potential accidents for
specialized assessment and possible warning.
D. Managing Limited Computational Resources
Fig. 7 shows the operation of the preliminary assessor. A
vehicle approaching an intersection (or other region for which
early-warning is useful) will rst compute a set of triggers.
These triggers need to be computed only once on approach, and
they will be repeatedly reused as the vehicle travels through
the intersection. The triggers depend on the geometry of the
intersection and the library of scenarios that will be tracked
for early-warning. For example, if the intersection contains
crosswalks, there will be triggers that will cause segmented
cones to be generated when pedestrians enter those crosswalks.
The geometry of the intersection must be available for this
initial trigger formation. We expect that there will be triggers
for each possible major action by a principal, e.g., turn left,
stop, go straight, etc. Thus, we expect principals to occasionally
simultaneously trigger two to eight scenarios, particularly ap-
proaching choice points in intersections. This gives worst-case
combinatorics proportional to (8n)
2
; however, the preliminary
assessor is optimized to quickly compare the segmented cones
generated from these scenarios, quickly eliminating many pair-
wise scenarios (or, in rare cases, offering generalized warnings
of multiple collisions).
Preliminary assessments provide a basis for resource allo-
cation for specialized assessment. There will be multiple ap-
proaches to specialized assessment (several are described in the
next section) that will provide a tradeoff between the amount of
computation used and the accuracy of results achieved. Using
the results of preliminary assessment, we can determine which
scenarios deserve specialized assessment and, of those deserv-
ing specialized assessment, which algorithm should be used.
These choices have several objectives: 1) to avoid unnecessary
warnings because the preliminary assessor has overestimated
the collision; 2) to avoid overlooking scenarios because the
preliminary assessor has underestimated potential collisions;
and 3) when warnings are presented to the driver, the timing
of the warning (see the UI section) benetting from careful
specialized assessment.
We have explored several cost-benet approaches to allo-
cating resources for specialized assessment. Given that we
expect to have a generous amount of computation allocated to
specialized assessment, we believe that the following simple
and robust approach will work well; we use a time slice between
0.2 and 0.5 seconds. At the beginning of the time slice, roughly
10% of the time is allocated to a thorough preliminary assess-
ment of all triggered scenarios for all nearby principals. A next
portion of the time slice (e.g., 60%) is allocated to specialized
assessment using a moderately expensive and reasonable accu-
rate IMM based on Kalman ltering (described later in the next
section). This kind of tracking is fast enough that many spe-
cialized assessments can be run during this portion of the time
slice; this will almost certainly cover the most critical scenarios.
Finally, the remainder of the time slice (e.g., 30%) is devoted
to more accurate specialized assessment algorithms. This xed
division of the time slice is not precisely optimal from the cost-
benet approach but has good cost-benet characteristics, and it
is more robust because it always allocates a xed amount (60%)
to obtain thorough coverage of the more critical scenarios.
Finally, we note that not all preliminary assessments re-
quire specialized assessment. Our prototype system supports
erratic behavior patterns, where the trajectory of the principal
is not very predictable, and thus, heavily ared cones are
generated to reect the lack of good a priori information.
These erratic scenarios would include such behaviors as a driver
turning the wrong direction on a one-way street, a driver falling
asleep and drifting into oncoming trafc, or a drivers trajectory
after a collision. The erratic scenarios do not have good
models for specialized assessment, yet the need for warning
is very high that the preliminary assessment alone will be
sufcient to provide warnings to the driver.
III. SPECIALIZED ASSESSOR
As we have described in the previous sections, the pre-
liminary assessors role is to examine all pairs of principals
(vehicles, pedestrians, or bicyclists) to quickly identify possible
collision incidents at a low computation. Accuracy is not a
major concern in the preliminary assessor stage. It is left to the
specialized assessor to perform accurate risk assessment when
necessary. In this section, we will describe the basic design of
the specialized assessor.
To motivate the need for accurate risk assessment, let us con-
sider a preliminary assessor shown in Fig. 8(a). The preliminary
assessor assesses risk by intersecting two cones projected into
the future, corresponding with the primary principal and an
other principal. If the two cones intersect, it calculates risk
based on the overlapping area (marked by the red ellipsoid).
The bigger the overlapping region is, the higher possibility
that they will collide. This is intuitively correct but with the
implicit assumption that the principal lies in the cone with a
uniform distribution. This is a rough approximation. Based on
GREENE et al.: COMPUTATIONAL ARCHITECTURE FOR CLEWS FOR VEHICLE, PEDESTRIAN, AND BICYCLIST 947
Fig. 8. (a) Preliminary assessor risk assessment via cone intersection. (b) Low collision probability. (c) High collision probability.
Fig. 9. Models. (a) Slotcar. The straight red line is the desired trajectory, and the blue curve is the actual motion path. (b) Driver alertness as a Markovian process.
observed data, one may believe that the vehicle is more likely
in certain positions, e.g., on the left edge of the cone, as shown
in Fig. 8(b), resulting in a low probability of collision. Fig. 8(c)
shows the opposite situation where the primary vehicle is close
to the right edge, with a high collision probability. These two
scenarios cannot be differentiated by the preliminary assessor.
Regardless of what the data suggests, the preliminary assessor
predicts the same collision risk. This shows the limitation of
preliminary assessor and calls for ner granularity and higher
accuracy in risk assessment. The specialized assessor is de-
signed to fulll this goal.
To enable early-warning, the specialized assessor needs to
predict collision well into the future. We formulate the risk
assessment problem as a statistical inference problem based on
relatively long-term (a few seconds, as opposed to a fraction
of a second) models. This sets us apart from most existing
collision warning systems such as [4], [5], and [12][14],
which focus on providing immediate collision warnings based
on short observation data. Our specialized assessor is model
based; principals are associated with a library of models, such
as a dynamics model specifying the way the vehicle moves
and the drivers mental model specifying how the driver acts
based on the surrounding environment. With these models, the
specialized assessor ts the data and makes inference about the
underlying state (e.g., position, velocity, and whether the driver
is paying attention). Based on the inferred state, the specialized
assessor makes predictions into the future and generates a risk
assessment, which can be used to make informed decisions
about whether a warning is necessary. Model-based approaches
are exible, allowing new models to be integrated and existing
models to be improved without the need to completely redesign
the system.
A. Models
The dynamics model species the way a principal moves.
In our prototype, we use a relaxed slotcar model, as shown
in Fig. 9(a). A slotcar is a powered miniature toy automobile
that is guided by a groove or slot in a track. In our prototype
model, the principal follows a predened desired trajectory (the
red straight line in the gure) but is allowed to deviate from
the trajectory to simulate real-world motion. The principal is
subject to two model forces: 1) a perpendicular force pulling
it toward the desired trajectory if it deviates and 2) a tangential
force adjusting the speed toward some target speed. The target
speed is a parameter that may be dependent on several factors:
the driving habit of the driver, the speed limit at the current
location, and the speed of cars around the principal.
Driver (and/or pedestrian and bicyclist) alertness is also
important to risk assessment. If a driver is alert, he/she has less
chance colliding with others, even if the vehicle is currently
going fast. This aspect of the modeling will help avoid unnec-
essary warnings to otherwise alert and responding drivers. The
driver alertness is formulated as binary: 0 for nonalert and 1
for alert. The alertness model is described as Markovian [see
Fig. 9(b)]: If the driver is alert at time t, he/she has a high
probability for staying alert at the next time instant t + 1. The
same is true for the transition from nonalert back to alert. The
parameters and characterize the drift and are assumed to be
small (< 0.1). Furthermore, the transition is asymmetric and
dependent on the surrounding environment. For example, if the
driver is alert and realizes the potential danger of hitting another
vehicle/pedestrian, then he/she will pay conscious attention and
will not drift to nonalert state before the danger clears. With this
alertness model, the specialized assessor seeks to infer about the
drivers alertness from the vehicle history data.
Similar ideas of modeling driver attention can be found in
[8]. It introduces a directional visibility model, in which the
driver is more likely to pay attention to the direction front
than to the sides and back. In essence, this is an observation
model. In contrast, our Markovian alertness model is part of
the dynamics model, describing the time-varying pattern of
driver alertness. Although both models are incorporating driver
attention in collision prediction, they are quite different. One
948 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 12, NO. 4, DECEMBER 2011
can even combine the directional observation model in [8]
and the dynamic alertness model in this paper into a collision
assessment system if necessary. Another difference is that our
scheme performs statistical inference regarding driver alertness
state, and this state can be submitted to UI to avoid unnecessary
warnings.
B. Risk Assessment Using Statistical Inference
With the models previously described, the specialized as-
sessor assesses risk by inference in the state space X =
(pos, vel, alertness). Two questions need to be answered at
any time t. First is the ltering (state estimation) problem:
Given the observation history data z
t
up to time t, in what
state is the principal? The second is the prediction problem:
Given the principals current state at t, in what state will the
principal will be at time t + 1, t + 2, , t +N, where N is the
prediction horizon (3 seconds in our implementation)? Here, for
simplicity, we use the digital time notion, assuming the intervals
between neighboring time t and t + 1 is unit time. In practice,
the time step t is empirically chosen. A small value of t
means increased computation; a large value of t may result
in missed collision due to sparse sampling in time (a.k.a. the
tunneling effect, i.e., two objects cross each other in continuous
trajectory but have well-separated positions when sampled at a
large interval t). However, a tunneling effect can be detected
in the preliminary assessor, and if tunnel effect is suspected,
the preliminary assessor will run the specialized assessor at
a shorter sampling interval to avoid missed collision. In our
implementation, t around 1/4 seconds works well.
For the ltering problem, we refer to the generic formulation
of sequential Bayesian ltering, which recursively updates the
posterior of underlying state x
t+1
as
p(x
t+1
|z
t+1
) p
o
(z
t+1
|x
t+1
)
_
X
p
d
(x
t+1
|x
t
)p(x
t
|z
t
) dx
t
.
(2)
The integral makes a single step of prediction, bringing
the previous state up the current time and then applying a
new external measurement to modify the state. The prediction
step assumes an object dynamics model p
d
(x
t+1
|x
t
). In our
prototype, the dynamics obey the relaxed slotcar model and the
alertness model. The prediction is then multiplied by a likeli-
hood, reecting the contribution of observation z
t
and using an
observation model p
o
(z
t+1
|x
t+1
). The lter (2) is sequential:
the current lter distribution p(x
t+1
|z
t+1
) is computed from
the previous lter distribution p(x
t
|z
t
) at every step.
The prediction problem is simpler. Without any observation
input, it is entirely based on the car dynamics and driver alert-
ness model. From the predicted state of the primary and the
other principals fromt to time horizon t +N, the specialized
assessor evaluates risks and predicts the following: 1) whether
an accident is likely; 2) when and where the accident may be;
and 3) how severe the accident will be. This information is then
returned to the preliminary assessor, delivered to the human
machine interface module, and possibly surfaced as a warning
to the driver.
C. Particle Filtering
For the ltering and prediction problem, we have experi-
mented with several types of techniques. The rst type is the
well-known particle lter [15], which represents a continuous
probability density function as a set of weighted Monte Carlo
samples, which is referred to as a particle set {(x
i
, w
i
)}, where
x
i
is the state of the particle (e.g., 2-D location and alert-
ness), and w
i
is the weight reecting the probability in Monte
Carlo simulation (

n
i=1
w
i
= 1). Each particle (x
t
i
, w
t
i
) rst
evolves its state independently based on the dynamics models
p
d
(x
t+1
|x
t
) and then adjusts its weight based on observation
data as
w
t+1
i
= w
t
i
p
_
z
t+1
|x
t+1
i
_
.
The resulting particle set {(x
t+1
i
, w
t+1
i
)}
n
i=1
corresponds to
the updated ltered belief p(x
t+1
|z
t+1
). Since we can only
maintain a nite number of particles in practice, the last step
is a resampling step according to the weights w
i
, which is
performed on the particle set, so that the particles with small
weights are eliminated (since these particles correspond to un-
likely target states) and replaced by multiple samples of the par-
ticles with large weights, allowing the regions around the likely
states to be more thoroughly explored. The literature [15] has
shown that this resampling step is critical in order for the parti-
cle set to remain a faithful representation of the updated belief.
We refer the readers to the tutorial paper [16] for an overview
of particle ltering applying to target-tracking applications.
The advantage of particle lter is its remarkable exibility.
It can accommodate almost any form of dynamics p
d
(x
t+1
|x
t
)
and observation model p
o
(z
t+1
|x
t+1
). The drawback is its high
computational complexity; a large number of particles need to
be simulated to maintain the representativeness of the particle
set. In practical situations, the high computational complexity
may make this infeasible. We reserve particle ltering for only
the most serious potential collisions and the most complex
models.
D. IMM Kalman Filtering
The second type of approach is the commonly used Kalman
lter [17], [18]. It is a special case of sequential Bayesian
ltering under the assumption that the object dynamics and
the observation model are both linear and Gaussian. Because
a Gaussian distribution can be completely characterized by
its mean and covariance, the Kalman-ltering equations se-
quentially update the mean x

= E[x
t
|z
t
] and the variance
P
t

= E[(x
t
x)(x
t
x)
T
] as measurements are observed via
simple Kalman gain linear operations. It is computationally
efcient, but the performance is often limited by its modeling
assumptions. In the collision warning problem, we linearize
principal state (pos, vel) with respect to the principal trajectory
to comply with the linear assumption. However, in practice,
the (pos, vel) state may be nonlinear and multimodal (which is
clearly non-Gaussian). To tackle this difculty, we use a method
known as the IMM algorithm [19], [20], which is speci-
cally designed for ltering in linear systems with Markovian
transitions.
GREENE et al.: COMPUTATIONAL ARCHITECTURE FOR CLEWS FOR VEHICLE, PEDESTRIAN, AND BICYCLIST 949
IMM assumes multiple modes of operation. In our case,
the vehicle state has multiple modes, which includes various
maneuvering (stopping, accelerating, steering) and driver alert-
ness (binary). The Markovian transition assumption is natural
and reasonable; hence, IMM is well suited. Within each mode,
the principal follows a linear dynamics model. The principal
may switch from one mode to another. We assume that the
transition probabilities is known or learned a priori. IMM
formulates the ltering problem as a hypotheses management
problem, where the hypotheses {H
1
, H
2
, . . . , H
K
} are
the K different maneuvering modes, with probability
{
1
,
2
, . . . ,
K
}, respectively, and

i

i
= 1. The goal is to
update the posterior probability for each hypothesis
t
i
based on
the observation history z
t
. The question is, Can we estimate
t
sequentially as time t advances?
The answer is yes. The technical details of IMM are de-
scribed in [20]. The intuition is given as follows: At any point of
time t, we start with a hypothesis prior set {
t
j
, j = 1, . . . , K},
where
t
j
= p(H
t
j
|z
t
). Now, assume that, at time t + 1, hypoth-
esis H
i
is true. Given that the system is linear and Gaussian
under H
i
, the Kalman lter is appropriate. One can update the
state estimate and corresponding covariance and compute the
likelihood p(z
t+1
|H
i
). The hypothesis probability can then be
updated as

t+1
i

j
_

t
j
p
_
H
t+1
i
|H
t
j
_
p(z
t+1
|H
i
)

.
We think IMM are particularly suitable for collision pre-
diction applications, because they handle the two-state (alert
versus nonalert) model above very easily, and we expect that
specialized tracking of most real-world driving will contain
several distinct modes like these. The computation cost is
K-times the normal Kalman lter, but it is still much more
efcient than the particle lter.
E. Computation of Specialized Assessment
Based on the forward prediction described in earlier sections,
a specialized assessor computes an assessment for every time
slice. The assessment includes the following information.
1) s
i
and s
j
are the scenarios that the principals are fol-
lowing. (This includes trajectories, text descriptions, and
other useful information for presenting warnings.)
2) P(t) is the probability assessed now (time t) that there
will be a collision in the future, assuming that the princi-
pals follow s
i
and s
j
. (This probability is conditioned on
the scenarios.)
3) T(t) is the expected time of the collision. (This value and
all the following components of the specialized assess-
ment are conditioned on the collision occurring P(t) and
on the scenarios s
i
and s
j
.)
4) L(t) is the predicted location of the collision.
5) B(t, t

) is the predicted benet of warning assessed now

(time t) but estimating the expected benet of warning at
some future time t

relative to the current time. The ben-

et function B(t, t

) will be further discussed in the next

section. It will be used to plan warnings, i.e., delaying
warnings, to increase certainty, when there is no loss of
benet.
IV. ASSESSMENTS BECOME WARNINGS:
THE USER INTERFACE
We do not address UI design, but in this section, we describe
how to connect the assessments with UIs. The ultimate goal of
an early-warning system is to issue warnings to the principal in
a way that prevents accidents and avoids dangerous emergency
maneuvers. The decision to warn will be dependent on the
characteristics of specic UIs. Warnings displayed on head-up
displays will have different characteristics that warnings issued
with sounds. Therefore, we include in the architecture decision
adaptors UI-specic software that receives all the assessments
and makes the critical warning decisions.
We have decoupled the computation of assessment from the
UI so that diverse UIs can be supported, even those that might
be used to warn pedestrians and bicyclists, such as roadside
signage or cell phone ringers. However, these kinds of UIs
add additional complexity to the adaptors; thus, to simplify the
exposition, we will describe decision adaptors for onboard
UIs for automobile drivers.
For purposes of illustration, we describe a basic decision
adaptor that uses a few parameters that are specic to a UI;
it then lters all the assessments and passes on to the UI only
those assessments that require immediate warning. While this
general-purpose adaptor is a good starting point, it does not
cover all the possibilities that could be built into a customized
decision adaptor, such as aggregation of warnings based on
location.
A. Basic Decision Adaptor
For this decision adaptor, the UI is characterized by six para-
meters: 1) warning rate W, which is a comfortable rate at which
warnings can be absorbed by the driver; 2) maximum warning
rate M, which is the maximum rate at which warnings will be
issued in a crisis; 3) efciency of warning Q, which is a factor
between 0 and 1 that describes the effectiveness of warnings
communicating to the driver; 4) rate at which the same warning
will be repeated R, in case the driver has not headed the warn-
ing; 5) penalty for an unnecessary warning D; and 6) threshold
for warning T. These parameters form a simple UI model that
the decision adaptor uses to customize the warnings to the UI.
Fundamentally, it is best to warn the driver about the most
serious potential accidents (i.e., highest expected utility for
warning). The expected utility is roughly the probability of
collision times the benet of the warning. To incorporate the
benet of warning in the calculation, we have included B(t, t

),
as shown in Fig. 10, in the specialized assessment. Ideally,
B(t, t

) should be the currently computed (at time t) benet of

warning a driver at a future time (t

) about a potential accident

(predicted at an even later time that is not parameterized). The
benet of warning is the extent of accident avoided once the
warning is received. B(t, t

) does not include any stochastic

information on probability that the accident occurs (this is
separately provided in the assessment as P(t)), nor does it
950 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 12, NO. 4, DECEMBER 2011
Fig. 10. Example benet function.
include any inefciency in delivering the warning to the driver,
which is dependent on the UI, and is separated into the decision
adaptor, which is, in this case, parameter Q. Nevertheless,
B(t, t

) does capture that extent to which a late warning, even

if successfully received by the driver, may not result in the
accident being avoided. Needless to say, the precise computa-
tion of B(t, t

), even after careful specialized assessment of the

principals involved, is very challenging.
We decided to simplify the function B in several respects.
First, the benet of a warning is an accident avoided, and
therefore, the height of the curve in Fig. 10 is dependent on
the seriousness of the accident and can be estimated based on
the predicted speed at the predicted point of collision. However,
rather than attempting to distinguish between serious accidents,
all but the slowest predicted collisions between vehicles use the
maximum height for B. Second, to be benecial, the warning
must be delivered on time, and therefore, function B has a
knee at a time t

> t, after which the warning may be too

late. The key salient features are captured by simplifying B
to the two lines in Fig. 10 that can be represented with three
parameters for the seriousness, the knee, and the collision time.
Rather than directly using the probability of collision P(t)
and the benet of warning B(t, t

) that are provided in the spe-

cialized assessment, P and B are further customized to the UI
by taking into account some of the pragmatic characteristics
of the UI, such as missed warnings or distracting warnings. For
example, in the basic decision adaptor we are describing, the Q
and D parameters of the UI model are used to customize the
utility measure according to a ranking formula
R = PQB(0) (1 PQ)D (3)
which adds a degree of realism diagrammed in Fig. 11. (The
ideal is Q = 1 and D = 0.) We have chosen as simple a formula
as possible that still captures important characteristics for the
real utility of warning.
However, this ranking alone is not sufcient to time warn-
ings. Even when the system is predicting a serious collision, it
may not be best to immediately warn the driver. When there is
still plenty of time, waiting may resolve the problem; drivers
can react on their own, or improved accuracy of assessment can
eliminate the need for a warning. Waiting to avoid warnings
will reduce distracting warnings and avoid frustrating drivers
enough that they disable the warning system, which are prob-
lems that can cause additional accidents.
Fortunately, the benet function in Fig. 10 includes a knee
which is the critical last opportunity to warn the driver with
enough time to avoid a collision. Warnings can wait until their
Fig. 11. Rening the assessment PB(0) according to the Q and D charac-
teristics of the UI.
knees, unless there are multiple serious collisions that require
spacing out warnings, to give the driver time to absorb each
warning and form judgments about appropriate actions. (Our
basic decision adaptor assumes the UI should avoid issuing
simultaneous warnings; UIs supporting simultaneous warnings,
such as head-up displays, can customize their decision adaptors,
or use very high rates W in the basic decision adaptor described
here.) Our basic decision adaptor attempts to schedule all pend-
ing warnings before their respective knees, using a warning
rate less than W. This leaves extra space so that unforeseen
warnings can be added, if necessary, using a higher warning rate
M. The basic adaptor treats knees in the pending assessments
as deadlines and schedules the lowest ranked assessments rst
in the backward direction, thereby favoring higher ranked as-
sessments in the forward direction. After scheduling, if there is
time remaining before the rst warning in the schedule, then no
warning is issued in the current time slice, and the procedure is
repeated again in the next time slice. However, if the scheduler
cannot complete a schedule with time to spare, then a warning
is necessary and is issued from the beginning of the schedule.
It may be necessary to recompute the schedule with spacing
M to accommodate all the warnings. In extreme cases, the
scheduler may issue an overload warning, indicating that it
cannot individually present all the warnings and that the UI
should issue a more general warning.
V. BENCHMARKING OF COMPUTATIONAL
COMPLEXITY AND DISCUSSIONS
The two-stage CLEWS architecture has been prototyped in
Java. This is an early high-level language implementation that
lacks tuning and performance optimization. We report these re-
sults to demonstrate good computational performance, although
we expect that signicant improvement would still be possible.
Our performance results are measured using a single-
processor Pentium-4 central processing unit running at
2.8 GHz. For the preliminary assessor, the key computational
element is the segmented cone creation and intersection. Table I
reports the benchmarks for computation speed. For a 15-tier
cone, creation takes approximately 70 s, and intersecting two
cones takes between 0.19 s, depending on the overlap. For
a 24-tier cone, we have improved accuracy with about 50%
slowdown in computation. This is still extremely efcient. In
our current prototype, when there is only a small number
of principals, the dominant cost is cone creation. We do a
GREENE et al.: COMPUTATIONAL ARCHITECTURE FOR CLEWS FOR VEHICLE, PEDESTRIAN, AND BICYCLIST 951
TABLE I
BENCHMARK OF THE PRELIMINARY ASSESSOR
TABLE II
BENCHMARK OF THE SPECIALIZED ASSESSOR. THE FIRST ASSESSMENT
IS THE INITIALIZATION OF IMM KALMAN FILTER OR PARTICLE FILTER
USING 3 SECONDS OF PRINCIPAL MOTION HISTORY SAMPLED AT
0.25-SECONDS INTERVAL. THE PARTICLE FILTER BENCHMARK
IS OBTAINED WITH A 200-PARTICLE SIMULATION
back-of-the-envelope calculation here: fty principals with an
average of three active scenarios would create 150 cones in
10.5 ms, using roughly 2% of the processor if the preliminary
assessment is performed two times per second. For larger num-
bers of principals, the dominant cost is a pairwise intersection
of cones, most of which will be highly efcient at 0.1 s, e.g.,
200 principals with an average of three active scenarios would
spend about 600
2
0.1 10
6
36 ms in pairwise cone inter-
section, using roughly 7% of the processor if the preliminary
assessment is performed two times per second. For implemen-
tations beyond the current prototype, these running times can
be signicantly improved, e.g., in the current prototype, cone
creation cost is due primarily to memory allocation, which can
be avoided by reusing preallocated data structures. In addition,
for larger numbers of scenarios, the n
2
scaling of pairwise
intersection can be reduced using spatial data structures such
as octtrees.
The specialized assessor is more computation intensive. For
this, we have benchmarked the IMM Kalman lter (second
column of Table II). A tracking step takes around 37 ms. This
means that a processor dedicated to specialized assessment can
handle 140300 tracking updates per second. A particle lter
is much more expensive and can only be run on dedicated fast
processors.
Combining the preceding benchmarking results, the collision
early-warning architecture shows the potential to analyze com-
plex intersections (with a few hundreds principals) and warn
drivers of the most serious collisions, using only a modest
amount of computation power.
Fig. 12 shows an example scenario of an intersection. The
rectangular boxes represent vehicles, and the dots represent
pedestrians on the crosswalk (marked with thick parallel lines).
While the CLEWS design can accommodate busy intersections,
we choose this simple one to illustrate the risk assessors. Fig. 12
shows the risk assessment for a vehicle marked with a big circle.
Risk is assessed by a specialized assessor using IMM Kalman
lter and shown in the gure in color; red means high risk, and
blue means lowrisk. For the marked vehicle, the car going north
is not of collision concern, and hence, the preliminary assessor
decides to ignore it. The marked vehicle has potential collision
risk with the vehicle in front of it (rear ending). Depending on
Fig. 12. Intersection example. The boxes represent cars, and the dots represent
pedestrians on the crosswalk. The assessed risks are plotted for the car marked
with a big circle. Color varies from blue to red, indicating risk from low to high.
Fig. 13. Pedestrian grouping. Patches on the crosswalk are pedestrian group
convex hulls.
its motion trajectory (e.g., going straight, turning left, or turning
right), it may run over pedestrians on all four crosswalks. The
risks are shown as rays emitted from the vehicle, and the width
of the ray shows the risk; thicker lines mean higher risks.
Risk assessment capability has been integrated with a
directional-audio UI developed by Fujitsu. Based on the as-
sessed risk, the UI generates directional sound as if it comes
from the estimated collision location. This gently directs the
drivers attention to avoid a potential collision.
A. Discussions
The risk assessment approach illustrated so far is focus-
ing on pairwise interaction between principals. We have also
developed grouping methods that aggregate pedestrians into
groups and treat groups as pseudoprincipals. Fig. 13 shows
the pedestrian groups. (The dots on the crosswalk are individual
pedestrians, and the patches are the groups.) The collision
risk is then assessed between vehicles and pedestrian groups.
Each pedestrian group is represented using the convex hull of
its member positions. Grouping greatly and easily improves
risk assessment efciency by a magnitude order in busy in-
tersections. We have developed dynamic algorithms for group
partitioning, merging/splitting, and convex hull tracking. Since
952 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 12, NO. 4, DECEMBER 2011
the grouping algorithms are specic to pedestrian modeling, we
leave them out of the scope of this paper to keep the focus on
risk assessment.
While pairwise risk assessment is intuitive and sound, there
are multiple ways to further cut computation if computation
resources are tight. For instance, one can focus on the most
immediate collision danger. For a vehicle waiting in line to
cross the intersection, it should worry about rear-ending vehi-
cles in front before it worries about running over pedestrians
on the crosswalk. Another way is to exploit the similarity
between principals, e.g., vehicles on the same lane tend to have
similar trajectories. Such similarities can be leveraged to save
computation. These remain as topics for future research.
VI. CONCLUSION
We have described the computational architecture for a
CLEWS that uses a two-stage reasoning layer. The reasoning
layer lies between sensor data and the UI. It applies a library
of scenarios, rst using highly efcient geometric comparisons
and then selectively applying more elaborate statistical analysis
to compute an assessment of potential accidents that contains
enough information for the UI to effectively decide if, when,
and how to warn the driver. This two-stage assessment is
computationally efcient and can work in real time for complex
intersections.
ACKNOWLEDGMENT
This is joint work of the Palo Alto Research Center and
Fujitsu Ltd. The authors would like to thank T. Aoki,
Y. Aragane, K. Asakawa, and T. Takebayashi for their fore-
sight and direction in formulating this joint project and Palo
Alto Research Center for the support, particularly T. Lunt, for
her guidance, and D. Thavarajah, for his contribution to the
implementation.
REFERENCES
[1] A. Broggi, P. Cerri, S. Ghidoni, P. Grisleri, and H.-G. Jung, A new ap-
proach to urban pedestrian detection for automatic braking, IEEE Trans.
Intell. Transp. Syst., vol. 10, no. 4, pp. 594605, Dec. 2009.
[2] T. Wada, S. Doi, N. Tsuru, K. Isaji, and H. Kaneko, Characterization of
expert drivers last-second braking and its application to a collision avoid-
ance system, IEEE Trans. Intell. Transp. Syst., vol. 11, no. 2, pp. 413
422, Jun. 2010.
[3] K. Funderburg (2004). Update on Intelligent Vehicles and Intersections.
[Online]. Available: http://tinyurl com/y4c26v .
[4] J. Misener, R. Sengupta, and H. Krishnan, Cooperative collision warn-
ing: Enabling crash avoidance with wireless technology, in Proc. ITS
World Congr., 2005.
[5] O. Aycard, A. Spalanzani, M. Yguel, J. Burlet, N. D. Lac,
A. D. L. Fortelle, T. Fraichard, H. Ghorayeb, M. Kais, C. Laugier,
C. Laurgeau, G. Michel, D. Raulo, and B. Steux, PUVAMENew
French approach for vulnerable road users safety, in Proc. IEEE Intell.
Vehicles Symp., 2006, pp. 27.
[6] S. Atev, O. Masoud, R. Janardan, and N. Papanikolopoulos. (2004)
Real-time collision warning and avoidance at intersections, Univ.
Minnesota, Minneapolis, MN, Tech. Rep. MN/RC 2004-45.
[Online]. Available: http://www.lrrb.org/PDF/200445.pdf
[7] G. D. Nicolao, A. Ferrara, and L. Giacomini, A collision risk assessment
approach as a basis for the on-board warning generation in cars, in Proc.
IEEE Intell. Vehicle Symp., Jun. 2002, pp. 436441.
[8] A. Eidehall and L. Petersson, Statistical threat assessment for general
road scenes using Monte Carlo sampling, IEEE Trans. Intell. Transp.
Syst., vol. 9, no. 1, pp. 137147, Mar. 2008.
[9] M. Althoff, O. Stursberg, and M. Buss, Model-based probabilistic colli-
sion detection in autonomous driving, IEEE Trans. Intell. Transp. Syst.,
vol. 10, no. 2, pp. 299310, Jun. 2009.
[10] A. Broadhurst, S. Baker, and T. Kanade, Monte Carlo road safety rea-
soning, in Proc. IEEE Intell. Vehicle Symp. (IV), Vienna, Austria, 2005,
pp. 319324.
[11] M. Althoff, O. Stursberg, and M. Buss, Safety assessment of driving
behavior in multi-lane trafc for autonomous vehicles, in Proc. IEEE
Intell. Vehicle Symp. (IV), St. Louis, MO, 2009, pp. 893900.
[12] D. M. Gavrila, J. Giebel, and S. Munder, Vision-based pedestrian de-
tection: The protector system, in Proc. IEEE Intell. Vehicle Symp. (IV),
Parma, Italy, Jun. 2004, pp. 1318.
[13] K. Stubbs, A. H. O. Masoud, C. MicMillen, H. Veeraraghavan,
R. Janardan, and N. Papanikolopoulos, A real-time collision warn-
ing system for intersections, in Proc. Intell. Transp. Syst. America,
May 2003.
[14] H. Veeraraghavan, O. Masoud, and N. Papanikolopoulos, Computer vi-
sion algorithms for intersection monitoring, IEEE Trans. Intell. Transp.
Syst., vol. 4, no. 2, pp. 7889, Jun. 2003.
[15] A. Doucet, N. de Freitas, and N. Gordon, Sequential Monte Carlo
Methods in Practice. New York: Springer-Verlag, 2001.
[16] S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, A tutorial on
particle lters for on-line non-linear/non-Gaussian Bayesian tracking,
IEEE Trans. Signal Process., vol. 50, no. 2, pp. 174188, Feb. 2002.
[17] R. E. Kalman, A new approach to linear ltering and prediction
problems, Trans. ASME, J. Basic Eng., vol. 82, pp. 3545, 1960.
[18] H. Sorenson, Kalman Filtering: Theory and Application. Piscataway,
NJ: IEEE Press, 1985.
[19] H. Blom and Y. Bar-Shalom, The interacting multiple model algorithm
for systems with Markovian switching coefcients, IEEE Trans. Autom.
Control, vol. 33, no. 8, pp. 780783, Aug. 1988.
[20] M. K. Kalandros, L. Trailovic, L. Pao, and Y. Bar-Shalom, Tutorial on
multisensor management and fusion algorithms for target tracking, in
Proc. Amer. Control Conf., Boston, MA, Jul. 2004, pp. 47344748.
Daniel Greene received the A.B. degree in applied
mathematics from Harvard University, Cambridge,
MA, in 1977 and the Ph.D. degree in computer
science from Stanford University, Stanford, CA,
in 1983.
He is currently a Senior Researcher with the Palo
Alto Research Center, Palo Alto, CA where his re-
search has focused on the design and analysis of
algorithms applied to problems in data compres-
sion, computational geometry, and communication
protocols. He has developed the concept of epidemic
algorithms for the dissemination of information in networks, including vehicle-
to-vehicle networks. His current research interest is the application of op-
timization and control algorithms to energy management and to intelligent
transportation systems.
Juan Liu (M02) received the M.S. and Ph.D. de-
grees in electrical engineering from the University of
Illinois, Urbana, in 1998 and 2001, respectively.
She is currently a member of Research Staff with
the Palo Alto Research Center. Her research interests
include signal processing, statistical modeling and
inference, distributed sensor networks, and applica-
tions such as intelligent transportation systems.
Dr. Liu was the recipient of the 2002 IEEE Signal
Processing Best Young Author Paper Award.
GREENE et al.: COMPUTATIONAL ARCHITECTURE FOR CLEWS FOR VEHICLE, PEDESTRIAN, AND BICYCLIST 953
Jim Reich received the B.S. degree, focusing on
space avionics, from the Massachusetts Institute of
Technology, Cambridge, and the M.S. degree in elec-
trical and computer engineering, with specialization
in control systems, fromCarnegie Mellon University,
Pittsburgh, PA.
He has been doing research and development on
both the systems and signal processing aspects of
sensor networks, Smart Matter, and, more generally,
in sensing and control for 20 years. He is currently
the Vice President of Engineering for Streetline, San
Francisco, CA, which is a startup company building 10 000+ node sensor
networks for parking, trafc, and urban infrastructure. Previously, he led the
Embedded Collaborative Computing area at Palo Alto Research Center, focus-
ing on sensor networks and sensing applications, including a signicant thrust
in intelligent transportation systems. Earlier work included launch vehicle and
spacecraft control systems and control hardware.
Yukio Hirokawa received the P.E.Jp (Professional
Engineer) degree in information engineering
in 1995.
He has been engaged in system planning and
construction in the telematics eld with Fujitsu Ltd.,
Tokyo, Japan, since 1998. He is currently involved
in the design and development of ideas for safety
improvement and environmental load reduction with
automotive networking.
Akio Shinagawa received the B.E. degree in elec-
trical engineering from Waseda University, Tokyo,
in 1977.
In 1977, he joined Fujitsu Ltd., Tokyo, Japan,
where he has been engaged in research and devel-
opment of the mainframe machine FACOM M-360,
the lisp machine FACOM Alpha, the fth-generation
parallel inference machine, the vehicle route plan-
ning system, and large-scale integration and printed
circuit board computer-aided design layout systems,
as a researcher, developer, and manager. He is cur-
rently involved in telematics services.
Hayuru Ito received the B.A. degree in behavioral
science from Hokkaido University, Sapporo, Japan,
in 1986.
Prior to joining the intelligent transportation
systems (ITS) project with Fujitsu Ltd., he was a re-
searcher in articial reality, articial life, and speech
conversation systems with the HumanMachine
Interface Project of Fujitsu Laboratories. He is cur-
rently with the Telematics Services Unit, Fujitsu
Ltd., Tokyo, Japan. His work domain is telematics
services in ITS, as well as humancomputer interac-
tion systems.
Tatsuo Mikami received the M.Sc. degree in com-
puter science from Nagoya University, Nagoya,
Japan, in 1978.
Prior to joining the intelligent transportation sys-
tems (ITS) project with Fujitsu, he was engaged with
the space robotics project of Fujitsu as a Project
Leader and the deep space navigation system project.
He is currently a Consultant in the eld of infor-
mation technology, ITS, and space development in
California.