360 L. Mussone: l16. Jg.(1999),H.

6 ~,~~i]

A review of feedforward neural networks

in transportation research
L. Mussone ~

The paper reports applications of neural networks in transportation research and

shows the author's specific experience and applications of neural networks for many
fields concerning transport and traffic theory. By using the interpolation capability of
neural networks it is possible to extract relationships present in data according to
particular conditions and to the features of used variables. A comparison between
applications anda discussion about problems and difficulties arising with the use of
feedforward neural networks are carried out. The aim is to underline that some
results are obtainable exactly by using neural network capabilities and can be trans-
lated to other fields without great difficulties.
Keywords: neural networks; feedforward NN; MLP; transport; traffic flow; accidents

• ª vorausfª neurale Netzwerke in der Fortbewegungsforschung.

Dieser Bericht bezieht sich auf Wendungen neuraler Netzwerke auf dem Gebiet der
Verkehrsforschung und zeigt die spezifische Erfahrung des Autors und Anwendun-
gen neuraler Netzwerke auf vielen Gebieten, die sich mit Transport- und Verkehrs-
theorie besch~iftigen. Durch Verwendung der Interpolationskapazit~it neuraler Netz-
werke ist es m6glich, bestimmten Bedingungen und den Kennzeichen verwendeter
Variabler entsprechende Datenbeziehungen zu extrahieren. Es wird ein Vergleich
zwischen Anwendungen angestellt, und es findet eine Diskussion ª Probleme und
Schwierigkeiten statt, die mit der Verwendung vorw~irtsgerichteter neuraler Netz-
werke verbunden sind. Ziel dabei ist die Unterstreichung der Tatsache, dass einige
Ergebnisse genau durch die Verwendung neuraler Netzwerkskapazit~iten gezeitigt
werden und ohne grolAe Schwierigkeiten auf andere Gebiete ª werden k6n-
nen.
Schlª neurale Netzwerke; vorausfª neurale Netzwerke; MLP;
Fortbewegung; Verkehrsfluss; Unf~ille

1. A p p l i c a t i o n s to t r a n s p o r t recognition of samples, recognition of signs, analysis

The term neural networks refers to a very wide cat-
egory of neural network structures, of learning algo-
rithms and paradigms and of neural activation func-
tions; their performance are very different: their flexi-
bility in learning very complex processes is ah interest-
ing issue especially with regard to their possibility of
computing parallelism.
In a first overview [4], neural networks have been
regarded as ah altemative to traditional methods, such
as linear programming, discrete optimization, statisti-
cal discriminant analysis, regression and cluster analy-
sis. The application fields are several and multifaceted:
Lorenzo Mussone, DSTM, Politecnico di Milano, Pza L. Da
Vinci 32, 1-20133, Italy, E-mail: mussone@mail.polimi.it.
and recognition of the voice, making decisions, map-
ping. Some years later Dougherty [11] listed a wider
range of applications to transport and other appli-
cations are reported in a special issue of Transportation
Research C [12].
Neural networks may also be used as an integrative or
substantial support to other techniques, such as expert
systems or fuzzy logic [37]. The potential of neural
networks in the field of non-linear dynamic systems,
statistical analyses, and modelling in general, has been
investigated by several authors [42], for example. A
preliminary review of the advantages connected with
the use of neural networks is in [26].
The transportation sectors in which artificial neural
networks (AAN) have been successfully applied are
the following:
 '~,'~)~i 116.Jg. (1999),H. 6 A review of feedforward neural networks in transportation research
-- identification of driver behaviour with advanced
traveller information systems [9, 45],
-- freeway incident detection [39, 40, 47],
-- recognition of images to detecting flow [2],
-- prediction and recognition of urban traffic condi-
tion [8, 10, 3],
-- prediction of freeway flow data [7] and prediction
of flow state [46],
-- evaluation of gap (area of dilemma) at signalized
intersections [38],
-- transportation/Land use [41, 14],
-- prediction of atmospheric and acoustic pollution
[25].
2. Applications experienced
The author applied ANNs in many fields of transpor-
tation research. The feature which links all those appli-
cations together is the use of feedforward multi-layer
perceptron (MLP) networks, even though different
learning techniques and descent algorithms were
adopted to improve minimisation of error. MLPs
showed to be more adapt to many fields in which a non
linear regression with many parameters is needed.
Many theorems state that feedforward multi-layered
perceptron networks (with at least one hidden layer),
by using neuron transfer functions of a sigmoidal type
and linear input combinations, can approximate any
continuous function, or more precisely which belongs
to L 2 (the Lebesgue two space) with a small margin of
error [6, 20, 19, 21, 22].
Many authors, such as [44, 43, 24] underline, on the
other hand, the difficulties of training, and in particular
the problems of overfitting or overtraining which
adversely affect the performance of a neural network.
Many techniques to ah optimum exploitation of data
must be kept into account: cross-validation, Jacknife
and Bootstrap [13, 44].
Many learning techniques are proposed and applied
both to improve the search of absolute minimum in the
error function and to reduce computing time: the most
famous, other than backpropagation, are the Gauss-
Newton method or its derived Levenberg-Marquardt
technique [43] and the radial basis function (RBF)
approach.
The carried out applications are:
-- mapping from a R" to R 2 for reconstruction of
vehicular flow-density and speed-density relation-
ship [15, 16, 35]; lane occupation models [17]; rec-
ognition of the state of traffic flow [15],
-- time series prediction [15, 27, 5, 30],
-- function approximation for accident risk evalu-
ation [31, 33, 34, 28], number of violations at sig-
nalised intersections [32], analysis of saturation
flow at signalized intersections [23]; driver behav-
361
iour [36]; user behaviour in transport modal choice
[1],
prediction of flow data in a traffic network [27].
3. Results
Results are grouped by application type. For each type
of problem, difficulties arisen in working out models
are described; in some cases particular properties of
the application and comparisons with other techniques
are proposed.
3.1 M a p p i n g
The reconstruction of flow-density and flow-speed
curves is one the most innovative result. What is more
interesting is the possibility to see how this curve
changes with parameters such as clearance or visibil-
ity, percentage of heavy vehicles, brightness, weather
conditions and obviously with flow, density and speed;
these variations are not linear like supposed or
accepted till now by theory (Fig. 1).
Effect of
1 Clearance meters [m]
0.9
l 0.8 10
0.7
...... 50
0.6
0.5 200
0.4
la
0.3 >52
0.2
7., 0.1 ]
o
0 10 20 30 40 50 60 70 80 9 0 1 0 0 1 1 0 1 2 0 1 3 0 1 4 0 1 5 0
Density [veh./km]
Fig. 1. Exampleof flow-densityrelationship reconstruction by
changing clearance [16]
A first practical application of these approaches is used
to calculate the PCU (passenger car unit) equivalent
values [35, 36] by analysing difference in flow-density
relationships for different percentage of heavy
vehicles. The difficulties encountered with this appli-
cations concern data extraction to obtain a good data
uniform distribution. For this reason and because it is
necessary that the neural network learns any real situ-
ation with the same frequency, an extraction rule must
be defined both for the training and test set.
Another application concerns reconstruction of percen-
tage of motorway lane occupation by changing both
parameters as the above mentioned application and
others affecting driver behaviour such as presence or not
of VMS (variable message signs) (Fig. 2). Its basic con-
cept is the same as the previous one but models are more
362 L. Mussone: 116. Jg.(1999),H. 6 .~~

1 .........Fast weight set) and comparison with ARIMA (with and

__~,o~d; n without missing data) are carried out with positive
0.8
results. Data used were collected on the M25 motor-
0.6 way in England. The effect of missing data is not deter-
minant on performance of neural networks while for
8 0.4
ARIMA models can be a crucial handicap; normalised
~'~ 0.2 RMSE (root mean square error) produced by using dif-
/ Standard without VMS
ferent initial weight sets has a variance less than 0.001.
10 20 30 40 50 60 70 80 90 Highway and rural traffic prediction was experienced
De nsity [veh/km] with good results both for short and long term horizon
[27], Fig. 3 shows an example of rural traffic prediction
Fig. 2. Examplesof lane occupationfor a percentage of heavy on 6 minutes basis.
vehicles of 40 % withpresence of VMS [18]
Applications to predict urban traffic were experienced
robust with respect to data extraction probably because by other researchers with a lesser success probably
relationships between variables are simpler. because of uncontrollable and variable delays in flow
due to local congestions: this fact makes traffic flow
Also reconstruction of the user choice of transport
non linear but also non continuous. It must be
mode was built up and choice mechanisms can be
remarked that multilayer feedforward can approximate
studied by the NN model [1].
only continuous functions.

3.2 Recognition
By using a definition of flow status (stable/critical/
unstable) based on analysis of flow curve (for example
worked out by models obtained with the previous
application) a model, capable of identifying the state of
flow, was carried out. Only flow data (combined with
stability information) are used to learn the network
without any explicit information about the flow-den-
sity or flow-speed relationships. Results are very
encouraging and show a very low error. Extrapolation
to cases never seen is very good and error in density
value is about 2-3 veh/km. This application is particu-
larly suitable for real time control of traffic where it is
important to know immediately the flow status for dif-
ferent flow and weather conditions.
3.3 Time series prediction
This field is very promising: advantages of neural net-
works are flexibility and robustness; especially in the
paper [5] a special investigation on robustness (initial
3.4 Function approximation
Many fields can be investigated by using non linear
regression capability: the proposed papers are limited
in respect of possible applications. In the field of acci-
dent evaluation they show good results and the more
complete available data the better the applications are.
In Fig. 4 an example of accident index calculation is
shown. Both urban and motorway/extra-urban environ-
ments are considered with interesting results. They
concern mainly the importance a variable assumes in
increasing the accident index. Previous studies based,
on linear or non linear regression of disjoint variables,
were not capable of explaining the very complicated
relations existing in accident data. Main difficulties
consist instead in good quality of data themselves
which sometimes do not include all variables necess-
ary to explain the process: for example the flow when
the accident occurred it is rather difficult to collect.
This leads to an intrinsic uncertainty during the model
work out; without any other information it is difficult

Normalised Output One step - 6 minutes Prediction

0.9
Congested Flow
Vector state

'1~"
~~~.~~~~~~~r
0.8 l i ; ,
r IV state
0,7

0.6

0.5

0.4
,' 'I' '[' …
0.3

0.2

0.1

0
0 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Time [h]

Fig. 3. Predictionfor congestedflowconditions [27]

 ~~ 116.Jg. (1999),H.6 A review of feedforward neural networks in transportation research 363

D a y - Veh.B Flow = 250 veh./h

1 - - Veh. A Flow
0.9- [veh./h]

0.8 - [] 25

0.7- [] 50
9 100
~ 0.6-
,x 250
~ o.5-
,x 500
9~ 0.4
9 1000
"~ 0.3
o 1500
0.2
o 2000
0.1
9 3000
I + 9 + I I i
20(1 400 600 800 1000 1200 1400 1600 1800 2000
A r e a o f t h e I n t e r s e c t i o n [ m 2]

Fig. 4. Accident Index calculation in an urban environment: rear-side accident in a "give way" intersection with a dry road bed and
calm weather [29]

r
09 ~ 1 lane 2 lanes 3 lane S
~o.~ ~
--- 07
t
,.=
~ = 0.6
0.8 ~e__~e!!!~

~ o.4 ~ th

~~,. o.2 J
0.3

~ 0.1
0 . . . . . . . . . .
16 24 32 40 48 56 64 72 16 24 32 40 48 56 64 72 16 24 32 40 48 56 64 72

Green phase length [s]

Fig. 5. Saturationflow [veh/min] by varying green phase length and queue length [23]

to ascribe the error to the model or to the data and translated to other fields without a great difficulty
therefore it is more difficult to single out the best net- being significant the concept of the application itself.
work (in the sense of performance).
Models worked out by the author are feedforward type
Fig. 5 shows the relationship between saturation flow so as to use their capability of approximating any con-
and not only with green time and the number of lanes tinuous function.
(for the latter an analogy with the HCM utilisation fac-
Main properties and advantages encountered by using
tor can exist) but also with the length of the queue.
When the queue increases, maintaining equal the these models are:
length of green time, the saturation flow increases the -- no analytical model of process must be defined;
smaller the green time is. this is particularly useful when relationships
The interest of this result, though it is limited to only one between variables are very complex and not linear;
network of urban traffic and requires further research, - the NN model is a powerful tool to investigate real-
consists in charging to the driver behaviour the differences ity, and to discover the most important relation-
(or fluctuations) in saturation flow. Till now the drive ships so as to improve data set, training and test
behaviour has been considered only in planning prob- extraction for a further improvement, if necessary,
lems, such as route choice, but nothing prevents extending of an analytical model and theory;
this approach also to microscopic driving choices -- non linearity of variables and their combination
(more or less systematic) induced by traffic conditions. can be obtained by other different methods with
great difficulty,
4. Conclusions
- interpolation capability of feedforward neural net-
works is irreplaceable particularly in time series
The paper presents several applications of neural net- prediction when a lot of missing data can be pres-
works in the transportation field. Some of them can be ent.
 364 L. Mussone:
On the other hand, main difficulties are:
- only deterministic problems (or problems where
the average value more significant than the vari-
ance) can be tackled successfully by feedforward
neural networks,
- optimisation problems cannot be solved easily by
M L F and should need other type of networks,
- because there ate no rules to build up the optimum
network, it is difficult to ascribe the model error to
the model itself or to the stochasticity of the pro-
cess: this makes more difficult the build up of the
optimum network especially in singling out the
best structure (number of hidden layers and neur-
ons),
-- robustness can be evaluated empirically but the
procedure is time-consuming and sometimes may
be not exhaustive,
- there are no tools to estimate whether a data set is
complete (that is it represents well, in a statistical
sense, the phenomenon) or not, when the phenom-
enon is unknown,
- neural networks do not furnish any analytical
model; this is n o t a limit in practical applications
when N N model is a part of a more complex oper-
ative system but it limits theoretical discussion.
Other types of networks (such as associative memory)
are very intriguing and computationally more powerful
thanks to their capability of auto-adaptive learning not
too sensitive to the damage and corruption of input
data but their applications to transport are till now very
limited. New important developments in transport
research could be carried on especially by associative
neural networks: for examples in spatial interaction
problems, driver behaviour analysis (also combined
with neuro-fuzzy models) and, above all, in optimisa-
tion problems.
Theoretical solutions to the problem of representation
of topological networks (for road, transport or commu-
nication networks, for example) by neural networks
could constitute a strategic improvement not only in
transport field research.
R e f e r e n c e s
[1] Arganese, S., Florio, L., Mussone, L.: An user behav-
iour model for the choice of public transport mode (in
italian). Paesaggio Urbano (1998), 3-4, pp. 67-71.
[2] Belgaroui, B., Blosseville, J. M.: Techniques neuron-
ales appliques au trafic routier. Recherche Transports
Securit~, RTS 36 (1992), pp. 35-46.
[3] Burattini, E., De Gregorio, M.: A neural network to
evaluate congestion levels. In: Mussone, L., Marescotti,
L. (eds.): Urban traffic: control possibility, tools and
their effectiveness. CNR, PFT2, I1 Rostro, Milano.
1998.
116. Jg. (1999), H. O ~ , ~
[4] Burke, L., Ignizio, J. E: Neural networks and oper-
ational research: ah overview. Computer & Operational
Research 19 (1992), 3/4, pp. 179-189.
[5] Chen, H., Mussone, L., Montgomery, F., Grant-Muller,
S.: Effects of missing data on neural network perform-
ance in forecasting flow. ICTTS 98. Beijing, China.
1998.
[6] Cybenko, G.: Approximation by superpositions of sig-
moidal functions. Mathematics Control, Signals and
Systems 2 (1989), pp. 303-314.
[7] Dochy, T., Danech-Pajouh, M.: Pr› ?acourt terme
du trafic routier par r› de neurons. Recherche
Transports S›233 RTS. 42 (1994), pp. 35--44.
[8] Dougherty, M., Kirby, H., Boyle, R.: The use of neural
networks to recognize and predict traffic congestion.
Int. Congress on Transport Research. Lyon. 1992.
[9] Dougherty, M., Joint, M.: A behavioural model of
driver route choice using neural networks. International
Conference on Artificial Intelligence Applications in
Transportation Engineering. San Buenaventura, Cali-
fomia. 1992, pp. 99-110.
[10] Dougherty, M., Kirby, H., Boyle, R.: The use of neural
networks to recognize and predict traffic congestion.
Traffic Engineering and Control (1993), pp. 311-314.
[11] Dougherty, M.: A review of neural networks applied to
transport. Transpn. Res.-C 3.4 (1995), pp. 247-260.
[12] Dougherty, M.: Guest applications of neural networks
in transportation. Transpn. Res.-C 5.5 (1997), pp. 225-
257.
[13] Efron, B.: The Jacknife, the Bootstrap and other resam-
pling plans. Society for Industrial and Applied Mathe-
matics. 1982.
[14] Fischer, M. M., Gopal, S.: Artificial neural networks: a
new approach to modelling interregional telecommuni-
cation flows. Journal of Regional Science 4 (1994), 34,
pp. 503-527.
[15] Florio, L., Mussone, L.: Neural network models for
classification and forecasting of highway traffic flow
stability. Control Engineering Practice 4 (1996), No.2,
pp. 153-164.
[16] Florio, L., Mussone, L.: Freeway section capacity for
different meteorological conditions using neural net-
works. 7th WCTR, World Conference on Transport
Research, Vol. 2., pp. 3-18. Amsterdam: Elsevier
Science. 1996.
[17] Florio, L., Mussone, L.: Lane occupation models in
motorway sections before a toll-gate station. 8th IFAC
97. Creta. 1997, pp. 1472-1477.
[18] Florio, L., Mussone, L.: A model to analyse VMS
effects on flow in motorway. 8th WCTR. Antwerp.
1998.
[19] Girosi, E, Poggio, T.: Networks for leaming. In Antog-
netti E, Milutinovi6, V. (eds.): Neural networks: con-
cepts, applications and implementation. New York:
Prentice Hall, Inc., pp. 110-154.
[20] Hornik, K., Stinchombe, M., White, H.: Universal
approximation of an unknown mapping and its deriva-
tive using multilayer feedforward networks. Neural
Networks 3 (1990), pp. 551-560.
[21] Homik, K.: Approximation capabilities of multilayer
feedforward networks. Neural Networks 4 (1991),
pp. 251-257.
 ~~~ 116.Jg. (1999), H. 6 A review of feedforward neural networks in transportation research
[22] Leshno, M., Lin, V. Ya., Pinkus, A., Schocken, S.:
Multilayer feedforward networks with a non polyno-
mial activation function can approximate any function.
Neural Networks 6 (1993), pp. 861-867.
[23] Marescotti, L., Mussone, L.: A flow saturation model
for signalized intersections. Urban Transport and the
environment. Lisbon, Portugal: WIT Press. 1998,
pp. 63-72.
[24] Masters, T.: Practical neural network recipes in C++.
New York: Academic Press, Inc. 1993.
[25] Moseholme, L., Silva, J., Larson, T.: Forecasting carbon
monoxide concentrations near a sheltered intersection
using video traffic surveillance and neural networks.
Transpn. Res.-D. Vol. 1.1 (1996), pp. 15-28.
[26] Mussone, L.: Artificial neural networks in transport (in
italian). Trasporti & Trazione 2 (1994), pp. 56-72.
[27] Mussone, L.: Artificial intelligence techniques to pre-
dict urban traffic indicators. AIENG 97. Capri. 1997.
[28] Mussone, L., Reitani, G.: A road safety analysis in
urban enviroments. ISATA 98. Dª 1998,
pp. 389-396.
[29] Mussone, L., Rinelli, S.: An accident analysis for urban
vehicular flow, URBAN 96. Barcelona. 1996.
[30] Mussone, L., Rinelli, S.: An analysis of urban accidents
in Milan by AI techniques, 8th WCTR. Antwerp, Bel-
gium. 1998.
[31] Mussone, L., Reitani, G., Rinelli, S.: Estimating the
accident probability of a vehicular flow by means of an
artificial neural network. Environment and Planning B:
Planning and Design. Pion Limited. London. 23 (1996),
pp. 667-675.
[32] Mussone, L., Reitani, G., Rinelli, S.: Definition and
building of a database for studying driver behaviour in
urban vehicular flows. 7th WCTR, World Conference
on Transport Research. Vol. 2., pp. 241-256. Amster-
dam: Elsevier Science. 1996.
[33] Mussone, L., Reitani, G., Rinelli, S.: A cross-sectional
analysis of vehicular accidents to work out a risk expo-
sure model. ISATA 96. Florence. 1996, pp. 213-220.
[34] Mussone, L., Reitani, G., Rinelli, S.: Effect of road
peculiarities and length on accident risk exposure.
ISATA 97. Firenze. 1997, pp. 201-208.
[35] Mussone, L., Chen, H., Montgomery, E: Calculating
PCU values in motorway environments - a method
using the MLR EIS Intemational ICSC Symposium on
365
Engineering of Intelligent Systems. Tenerife, Spain.
1998.
[36] Mussone, L., Reitani, G., Rinelli, S.: A methodology to
modelling driver behaviour in signalised urban intersec-
tions using artificial neural networks. In: Himanen, V.,
Nijkamp, P., Reggiani, A., Raitio, J., (eds.): Neural net-
works in transport applications, Ashgate. 1998.
pp. 171-189.
[37] Nanda, R., Kikuchi, S.: Fuzzy regression based trip
generation model by using of neural networks. Int. Con-
gress on Transport Research. Abs. No. 563. Lyon. 1992.
[38] Pant, E D.: Neural network for gap acceptance at stop-
controlled intersection. Journal of Transportation
Engineering 120 (1994), No. 3.
[39] Ritchie, S. G., Cheu, R. L., Recker, W. W.: Freeway
incident detection using artificial neural networks. Int.
Conference on Artificial Intelligence Applications in
Transportation Engineering. San Buenaventura, Cali-
fornia. 1992, pp. 215-231.
[40] Ritchie, S. G., Cheu, R. L.: Simulation of freeway inci-
dent detection using artificial neural networks. Transpn.
Res.-C 1, 3 (1993) pp. 203-217.
[41] Rodrigue, J.: Parallel modelling and neural networks:
an overview for transportation/land use systems.
Transpn. Res. -C5, 5 (1997), pp. 259-271.
[42] Sartori, M. A., Antsaklis, R J.: Implementation of learn-
ing control systems using neural networks. IEEE Con-
trol Systems 1992, pp. 49-57.
[43] Sj0berg, J., Hjalmarsson, H., Ljung, L.: Neural net-
works in system identification. In: From identification
to learning Course. Milan, Italy. NATO ASI and CNR.
1994.
[44] Weiss, S. M., Kulikowski, C. A.: Computer Systems
that leam. San Mateo, California: Morgan Kaufmann
Pub. 1991.
[45] Yang, H., Kitamura, R., Jovanis, R E, Vaughn, K. M.,
Abdel-Aty, M. A.: Exploration of route choice behavior
with advanced traveler information using neural net-
work concepts. Transportation 20 (1993), No. 2,
pp. 199-223.
[46] Yang, B., Qiao, E: Neural network approach to classifi-
cation of traffic flow states. Joumal of Transportation
Eng. 124 (1998), No. 6, pp. 521-525.
[47] Zhang, H. M., Ritchie, S. G.: Freeway ramp metering
using artificial neural networks. Transpn. Res.-C 5. 5
(1997) pp. 273-286.