Proc. IEEE-INNS-ENNS Intl. Joint Conf. on Neural Networks, July 24-27, 2000, Como, Italy, Vol. 3, pp. 207-212.

STORAGE AND RECALL OF COMPLEX TEMPORAL SEQUENCES THROUGH A

CONTEXTUALLY GUIDED SELF-ORGANIZING NEURAL NETWORK
Guilherme de A. Barreto

Aluizio F. R. Arajo

Universidade de So Paulo
Departamento de Engenharia Eltrica
C. P. 359, 13560-970, So Carlos, SP, BRASIL
{gbarreto, aluizioa}@sel.eesc.sc.usp.br
Abstract. A self-organizing neural network for learning and recall of complex temporal sequences is proposed. we
consider a single sequence with repeated items, or several sequences with a common state. Both cases gives rise to
ambiguities during recall of such sequences which is resolved through context input units. Competitive weights
encode spatial features of the input sequence, while the temporal order is learned by lateral weights through a timedelayed Hebbian learning rule. Repeated or shared items are stored as a single copy resulting in an efficient memory
use. In addition, redundancy in item representation improves the network robustness to noise and faults. The model
operates by recalling the next state of the learned sequences and is able to solve potential ambiguities. The model is
simulated with binary and analog sequences and its functioning is compared to other neural networks models.
Keywords: Self-organization, context-based learning, spatiotemporal sequences, Hebbian learning, robotics.

1. INTRODUCTION
It is well known that natural data are usually dynamic or, equivalently, sequential in the sense that temporal
dependencies between consecutive patterns should be taken into account (Kohonen, 1997). Recognizing such
dynamic patterns is crucial in hearing and vision, and reproducing them underlies processes like motor pattern
production, speech and singing. Aware of this, a long-standing problem in neural network learning theory has been
the mathematical and computational characterization of the so-called list learning or temporal sequence learning
(Grossberg, 1969a, 1969b; Amari, 1972).
Two mechanisms are essential for these models to learn and recall a temporal sequence efficiently. (i) For
purpose of learning, a mechanism must be implemented to extract and store temporal relationship between patterns.
Such a mechanism is usually called short-term memory (STM). In the STM, individual items are stored in such a
way that the pattern of STM activity encodes both the items that have occurred and the temporal order in which they
have occurred. In the cognitive literature, such a mechanism is often said to store both item information and order
information (Bradski et al., 1992). (ii) For purpose of recall, the network dynamics must also include a mechanism
for reading out items in the stored temporal order.
The basic idea involved in learning the temporal order of sequential patterns is known as the temporal chaining
hypothesis: A sequence is seen as a chain of temporally linked items in which the set of associations between
consecutive components (state transitions) must be learned for total or partial reproduction of the memorized
sequence. Most of the artificial neural network (ANN) models that implement this hypothesis are based on either
multilayer perceptron trained with a temporal version of gradient-based learning algorithms or based on the Hopfield
model (see Mozer, 1993; Wang, 1995; Herz, 1995 and references therein). Nevertheless, it is important to emphasize
that self-organization play a major role in temporal sequence learning, and specially the field of robot learning has
gained relevant contributions. The vast majority of models is involved in either solving inverse kinematics for
visuomotor coordination (Kuperstein & Rubistein, 1989; Martinetz et al., 1990; Gaudiano & Grossberg, 1991;
Walter & Schulten, 1993) or route learning (Denham & McCabe, 1995; Gaudiano et al., 1996; Heikkonen &
Koikkalainen ,1997).
A robot task that has received contributions from the field of unsupervised neural network is trajectory
tracking (Hytyniemi, 1990; Althfer & Bugmann, 1995; Bugmann et al., 1998; Barreto & Arajo, 1999a, b). This
task aims at converting a description of a desired motion to a trajectory defined as a time sequence of intermediate
configurations between an origin and a destination. The desired motion is that of an industrial robot arm consisting
of joints driven by individual actuators. The robot is required to follow a prescribed path, so that its controllers must

Proc. IEEE-INNS-ENNS Intl. Joint Conf. on Neural Networks, July 24-27, 2000, Como, Italy, Vol. 3, pp. 207-212.

coordinate the movements of the individual joints of the robot to achieve a desired overall movement from point to
point along the path.
In the models by Hytyniemi (1990), Althfer & Bugmann (1995) and Bugmann et al (1998), the state
transitions are hard-wired. Two layers of connections store exactly the same components, and the temporal links are
established by the network designer since the trajectory is known beforehand. In addition, for sequences with
repeated or shared states, the first two models are unable to reproduce the stored trajectories correctly. The third one
can recall a single trajectory with repeated states but is unable to deal with multiple trajectories with shared states.
Recently, Barreto & Arajo (1999a,b) have proposed a different approach in which temporal order of items is
learned without supervision through a temporal version of Hebbs rule (Hebb, 1949) as the input sequence is being
read. The first model (Barreto & Arajo, 1999a) can handle trajectories that shared states with others. The shared
states are stored at different neurons. The second model (Barreto & Arajo, 1999b) also handle shared states but this
time, shared states are stored as single copy in a single neuron. This work extends this second model by allowing it
to handle closed sequences such as figure eight trajectories and mixed sequences (part open, part closed), while store
repeated or shared states as a single copy.
In this paper, we extend our previous self-organizing network (Barreto & Arajo, 1999b) in order to handle
closed sequences with repeated or shared items and to use less memory resources through the use of a single neuron
to every repeated item. We will consider binary and analog sequences in the form of closed curves such as circles
and figure eight, as well as open ones such as straight lines. For the binary case, a sequence of letters are considered,
and for the analog case we evaluate the network on the problem of trajectory tracking. The rest of the paper is
organized as follows. In Section 2, we present the model. In Section 3, we evaluate the performance of the model
through computer simulations and discuss the main results. We conclude the paper in Section 4.

2. THE NEURAL NETWORK MODEL

The architecture of the proposed model is shown in Figure 1. It is simple and akin to standard architectures of
unsupervised neural networks. The basic differences is in the input layer, divided into three parts, and the presence
of delayed signals at the network output. The model has feedforward and feedback connections that play different
roles in its dynamics. Each component of the network is detailed described in the following.
--z-1

z-1

z-1

Feedback
weights M

z-1

z-1

z-1

---

Feedforward
weights W
Sensory inputs

trajectory states

Global

Local

context units

Figure 1. The architecture of the our model.

Each input pattern v(t) Rp+q+d comprises sensory units s(t) Rp, global context units gc(t) Rq, and local context
units lc(t) Rd. Sensory units receive the trajectory state at time step t and propagates it towards the output. Context
units are of two types, global and local, and do not perform any processing but have an essential role in resolving
ambiguities. Global context is time-invariant and is usually set to the initial or final item of a sequence. Global
context acts as a kind of a global sequence identifier. In robotic tasks, this global information is usually supplied as
task specification and the behavior of the robot can understood as goal-oriented. This identifier is crucial in
situations where multiple sequences have to be learned, especially when the learned trajectories have items in
common (Barreto & Arajo, 1999a). Furthermore, if these sequences are open (final item initial item) or closed
(final item = initial item) without repeated items, it is shown that global context is sufficient to recall the stored
sequences in the correct temporal order even in the presence of shared items.
However, for a closed single sequence with repeated items, additional information is necessary since the
repeated states have the same global context. Hence, a local, time-variant, context is added to the network input.

Proc. IEEE-INNS-ENNS Intl. Joint Conf. on Neural Networks, July 24-27, 2000, Como, Italy, Vol. 3, pp. 207-212.

These type of context is formed by the sequence items (past history) that precedes the current state. As will be
shown in the simulations, the inclusion of time-varying context units allows the network to encode both closed and
open sequences, increasing the model applicability.
The synaptic weights consists of feedforward weights and feedback weights. The interlayer and intralayer
weights are updated by competitive and Hebbian learning rules respectively. The feedforward weights connect each
input units to each output neuron. They encode the spatial configuration of a sequence item at a specific time step.
The intralayer coupling structure encodes the temporal order of the patterns in a sequence by diffusing its activation
through a non-zero lateral connection in order to trigger its successor in the current sequence being recalled.
The two groups of synaptic weights are updated during a single pass of an entire sequence in which each state
is read once. This means that a sequence with Nc components requires Nc training steps. The input state is compared
with each feedforward weight vector through Euclidean distance. The weight vectors closest to the input vector is
selected to be updated. We define a sensor distance, D sj (t ) R, a global context distance D gc
j (t ) R, and local
context distance, Dlcj (t ) R, as follows:
D sj (t ) = s(t ) w sj (t ) ,

gc
D gc
j (t ) = gc(t ) w j (t )

D lcj (t ) = lc(t ) w lcj (t )

and

(1)

where || x ||2 = x12 + + x 2p + q . The distance D sj (t ) is used to find the winners of the current competition, while D gc
j (t )
and

Dlcj

(t ) are used to solve ambiguities during recall.

For good tracking performance, every item of a sequence should be memorized and recalled. That is, if the
input sequence has 20 items, every 20 items should appear at the network output. Standard competitive networks
tend to cluster the input patterns and can not be used for tracking purpose. To overcame this situation, the network
should penalize a neuron, by excluding it from subsequent competitions, when it tries to encode more than one
pattern. This can be accomplished by defining a function Rj(t), called responsibility function, that indicates if a
neuron is already responsible for encoding a given sequence item. If Rj(t) >> 0, neuron j is excluded from
subsequent competitions for sequence components. If Rj(t) = 0, neuron j is allowed to compete.
According to the definition of Rj(t), if a given item occurs again it will be encoded by other neuron, and many
copies of the same item will exist in the network (Barreto & Arajo, 1999a). In order to efficiently use memory
resources, every time an item occurs, it should be encoded by the neuron that encoded it for the first time. It can be
accomplished by defining a similarity radius 0 < << 1 that accounts for the repeated items in the following way
(Barreto & Arajo, 1999b):
D s (t ), if D sj (t ) < or R j (t ) = 0
f j (t ) = j
otherwise
R j (t ),

(2)

According to Eq. (2), a neuron can only be used more than one time if (1) it never was used before, i.e, Rj(t) = 0, or
(2) its weight vector lies at a distance of less than of the current input item. Thus, the output neurons are ranked as
follows:
f1 (t ) < f 2 (t ) <

< f

n 1

(t ) <

f n (t )

(3)

where i (t), i = 1,..., n, is the index of the i-th closest output neuron to s(t). We choose K neurons, (t) = (1(t),
2(t),...,K(t)), K n, as winners of the current competition. They will represent the current input vector v(t). The
corresponding activation values decay linearly from a maximum value amax R, for 1(t), to a minimum amin R,
for K(t), according to the following equation:
a min
a
(i 1) ,
ai (t ) = a max max
max(1, K 1)

i = 1, ..., K

(4)

Proc. IEEE-INNS-ENNS Intl. Joint Conf. on Neural Networks, July 24-27, 2000, Como, Italy, Vol. 3, pp. 207-212.
where amax and amin are user-defined. For i > K, ai (t ) = 0. The responsibility function Rj(t) is updated every time a
new activation pattern a(t) = (a1(t),..., an(t))T is computed: Rj(t+1) = Rj(t) + aj(t), where >> 0 is an exclusion
constant. Following the selection of the winning neurons and the determination of their activations and outputs, the
weight vectors wj(t) are updated according to the following competitive learning rule:

wj(t+1) = wj(t) + aj(t)[v(t)- wj(t)]

(5)

where 1 is the learning rate. For t=0, wj(0), for all j, is initialized with random numbers between 0 and 1.
The successive winners are linked in the correct temporal order through a lateral coupling structure. The
feedback weights are updated according to the following learning rule (Barreto & Arajo, 1999b):

mjr(t+1) = mjr(t) + aj(t)ar(t-1) or M(t+1) = M(t) + a(t)aT(t-1) (in matrix form) (6)
where 0 < < 1 is the feedback learning rate. The activities of the previous competition, ar(t-1), are made available
through time delays, and the matrix form in Eq. (6) lends itself to a simple matrix analysis of temporal associative
memory (Amari, 1972; Barreto & Arajo, 1999a). Equation (6) is a Hebbian learning rule (Hebb, 1949) that creates
temporal associations between consecutive states of the input trajectory, encoding the temporal order of the input
sequences (see Figure 2). For t=0, Mj(0) = 0 for all j, which indicates that no temporal associations exist.

t=0

t=1
s(1)

t=2

t=2
cg

cl

s(2)

cg

cl

s(2)

cg

cl

Figure 3. A sketch of how consecutive winners are temporally linked through lateral connections. Initially (t=0), the
network has no lateral connections. At t=1, the neuron on the left is the winner for pattern s(1). At t=2, the neuron on the
right is the winner for pattern s(2). Still at t = 2, a lateral connection is created from the neuron on the left to the neuron
on the right through Eq. (6), indicating the temporal order. cg stands for global context and cl for local context.

It is well known that time in Hebbian learning rules plays an essential role in psychology (Tesauro, 1986;
Montague & Sejnowski, 1994), object recognition (Wallis, 1996), route learning and navigation (Schlkopf &
Mallot, 1995) and blind source separation (Girolami & Fyfe, 1996).
To start the recall of a stored sequence the network must be supplied with any item belonging to that sequence.
This item will then activate the neuron whose weight vector is the most similar (Eq. 4). Then, the active neuron will
trigger its successor through the lateral connections through the following output equation:

D gc
Dlcj (t ) n

j (t )

y j (t ) = 1 n

1
g m jr (t )ar (t )

n
gc
lc
Dr (t ) Dr (t ) r =1

r =1
r =1

(7)

where g(u) 0 and dg(u)/d u > 0. The weight vector of the neuron with highest value of yj(t) is used to represent the
next sequence item. This item is then fed back to the network input and the process continues until the end of the
stored sequence. For t=0, the activation and output values are set to aj(0) = yj(0) = 0, for all j.

3. SIMULATIONS
The first test evaluates the network on binary sequences of letters with repeated and shared items (see Fig. 4a). Each
letter is represented as a 108 matrix which forms an 80-dimensional input vector. We consider two types of
sequences, namely, closed with a repeated item (figure eight) and open with shared items. The network parameters
were set to: p = 80, q = 80, d = 160, n = 250, K = 2, amax = 1, amin = 0.98, = 10-4, = 100, = 1, = 0.8. The
sequences are trained one after the other, and the global context is set to any sequence item in the case of the figure
eight sequence and to the letter U in the case of the open sequence. The local context is lc(t) = {v(t-1), v(t-2)}. Note

Proc. IEEE-INNS-ENNS Intl. Joint Conf. on Neural Networks, July 24-27, 2000, Como, Italy, Vol. 3, pp. 207-212.

that item K is a repeated one for the figure eight sequence, and is also an item shared with the open sequence.
Typical results are shown in Fig. 4b for both sequences. This simple example elucidates the need of both global and
local context. In the figure eight case, when the network arrives at the item K it passes to the correct next item X
because of the local context and it does not jump to the item Z because of the global content acting as an identifier.
D

...

P
U

This letter was given

X
K

This letter was given

Z

(a)

(b)

Figure 4. Sequences of binary patterns. (a) The input sequences, (b) typical recall for the two sequences.
Figure 5 shows a typical result when the given starting item is corrupted by noise. In this test, 10% of the
elements of the starting item were chosen at random and had their values changed from 0 to 1 or from 1 to 0.
The noisy L was given

...
The noise E was given

Figure 5. Recall of binary sequences with a noisy starting item.

Figure 6 illustrates the test with analog sequences in the form of a figure eight robot trajectory constructed with
the toolbox Robotics of Matlab (Corke, 1996) for a PUMA 560 with six degree of freedom. The trajectory has 50
states, so that each state vector s(t) R15 consists of the end-effector spatial position z = (x,y,z), six joint angles and
six joint applied torques at time step t. The repeated state is at position (0.5, 0.5, 0.5). The following parameters
were changed: p = 15, q = 3, d = 6, = 10-6. The global context is set to any point of the any state of the input
trajectory, and the local context is lc(t) = {z(t-1), z(t-2)}. In Figure 6a, the recall results for the end-effector positions
is shown. In Figure 6b, a test of tolerance to faults is performed, in which the winners 1(t) for each sequence item is
eliminated after training. Due to the redundancy, the recall is now performed by the winners 2(t). The positioning
mean squared errors were 0.0 (Fig. 6a) and 7.710-5 (Fig. 6b). The previous results show that the proposed model
can produce very reliable recalls, even in the presence of noise and faults

(a)

(b)

Figure 6. Sequences of analog patterns. (a) Typical recall, (b) a fault-tolerance test. In (a) and (b) the actual
trajectory is represented by circles and the recalled one by asterisks. Arrows show the direction of movement.

Proc. IEEE-INNS-ENNS Intl. Joint Conf. on Neural Networks, July 24-27, 2000, Como, Italy, Vol. 3, pp. 207-212.

4. CONCLUSION AND FURTHER WORK

An unsupervised model for learning and recall of robot trajectories is developed. The neural network is simple yet
powerful, since it accurately stores and retrieves open and closed complex temporal sequences even in the presence
of noise and faults. Compared with the self-organizing models of Grossberg (1969a) and Wang & Arbib (1990)
which can also handle repeated patterns, our model do the same using less neurons since they use multiple copies of
the repeated state. A very similar approach using the concept of spatial items linked via time-delayed Hebbian rule
was proposed by Kopecz (1995), but his model cannot handle repeated patterns. Further tests are being conducted in
order to evaluate how the number of items (sampling rate) in a sequence influences the network robustness to noise
and the value of the similarity radius. Also, further comparison with other models such as the network proposed by
Wang & Yuwono (1996) and Srinivasa & Ahuja (1999), and the implementation of the model in a real PUMA 560
robot system are being pursued.
Acknowledgements: The authors thank FAPESP for its financial support (Project # 98/12699-7).
References
Althfer, K. & Bugmann, G. (1995). Planning and learning goal-directed sequences of robot arm movements. Proc. of the Int. Conf. on Artificial
Neural Networks (ICANN95), Paris, France, vol. 1, pp. 449-454.
Amari, S. (19972). Learning patterns and pattern sequences by self-organizing nets. IEEE Trans. Computers, 21:1197-1206.
Barreto, G.A. & Arajo, A.F.R. (1999a). Unsupervised learning and recall of temporal sequences: An application to robotics. Int. Journal of
Neural Systems, 9(3):235-242.
Barreto, G.A. & Arajo, A.F.R. (1999b). Unsupervised context based learning of multiple temporal sequences. Proc. IJCNN99, Washington,
D.C., paper #575.
Bradski, Carpenter, G.A. & Grossberg, S. (1994). STORE working memory networks for storage and recall of arbitrary temporal sequences. Biol.
Cybern., 71:469-480.
Bugmann, G., Koay, K. L., Barlow, N., Phillips, M. & Rodney, D. (1998). Stable encoding of robot trajectories using normalized radial basis
functions: Application to an autonomous wheelchair. Proc. of the 29th Int. Symp. on Robotics (ISR'98), Birmingham, UK, pp. 232-235.
Chen, C.Y, Mills, J.K. & Smith, K.C. (1996). Performance improvement of robot continuous-path operation through iterative learning using
neural networks. Machine Learning, 23:75-105.
Corke, I. (1996). A Robotics toolbox for MATLAB. IEEE Robotics and Automation Magazine, 3(1):24-32.
Denham, M. J. & McCabe, S. L. (1995). Robot control using temporal sequence learning. Proc. of the World Congress on Neural Networks, vol.
2, 346-348.
Gaudiano, P. & Grossberg, S. (1991). Vector associative maps: Unsupervised real-time error-based learning and control of movement trajectories.
Neural Networks, 4:147-183.
Gaudiano, P., Zalama, E. & Coronado, J.L. (1996). An unsupervised neural network for low-level control of a wheeled mobile robot: noise
resistance, stability, and hardware implementation. IEEE Trans. Syst., Man, and Cybern. - Part B, 26:485-496.
Girolami, M. & Fyfe, C. (1996). A temporal model of linear anti-Hebbian learning. Neural Proc. Lett. 4:139-148.
Grossberg, S. (1969a). Some networks that can learn, remember, and reproduce any number of complicated space-time patterns, I. Journal of
Math. and Mechanics, 19:53-91.
Grossberg, S. (1969b). On the serial learning of lists. Mathematical Biosciences, 4:201-253.
Hebb, D.O. (1949). The organization of behavior. Wiley.
Heikkonen, J. & Koikkalainen, (1997). Self-organization and autonomous robots. In: Neural Systems for Robotics, O. Omidvar and van der
Smagt (Eds.), Academic Press, 297-337.
Herz, A. V. M. (1995). Spatiotemporal association in neural networks. In: The Handbook of Brain Theory and Neural Networks, M. A. Arbib
(ed.), pp. 902-905, MIT Press.
Kohonen, T. (1997). Self-organizing maps. 2nd edition, Springer-Verlag.
Kopecz, K. (1995). Unsupervised learning of sequences on maps with lateral connectivity. Proc. ICANN95, vol. 2, pp. 431-436.
Kuperstein, M. & Rubinstein, J. (1989). Implementation of an adaptive neural controller for sensory-motor coordination. IEEE Control Systems
Magazine, 9(3):25-30.
Martinetz, T.M., Ritter, H.J. & Schulten, K.J. (1990). Three-dimensional neural net for learning visuomotor coordination of a robot arm. IEEE
Trans. Neural Networks, 1(1):131-136.
Montague, R. & Sejnowski, T. J. (1994). The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms.
Learning & Memory, 1:1-33.
Mozer, M. C. (1993). Neural net architectures for temporal sequence processing. In: Predicting the Future and Understanding the Past, A.
Weigend & N. Gershenfeld (Eds.), Redwood City, CA: Addison-Wesley, 243-264.
Owen, C. & Nehmzow, U. (1996). Route learning in mobile robot through self-organization. Proc. Eurobot 96, IEEE Computer Society Press.
Schlkopf & Mallot (1995). View-based cognitive mapping and path planning. Adaptive Behavior, 3:311-348.
Srinivasa, N. & Ahuja, N. (1999). A topological and temporal correlator network for spatiotemporal pattern learning, recognition, and recall.
IEEE Trans. on Neural Networks, 10(2):356-371.
Tesauro, G. (1986). Simple neural models of classical conditioning. Biol. Cybern. 55:187-200.
Wang, D.L. & Arbib, M.A. (1990). Complex temporal sequence learning based on short-term memory. Proc. of the IEEE, 78(9):1536-1543.
Wang, D. L. (1995). Temporal pattern processing. In: The Handbook of Brain Theory and Neural Networks, M. Arbib (ed.), 967-971, MIT Press.
Wang, D. L. & Yuwono, B. (1996). Incremental learning of complex temporal patterns. IEEE Trans. on Neural Networks, 7(6):1465-1481.
Wallis, G. (1996). Using spatio-temporal correlations to learn invariant object recognition. Neural Networks, 9(9):1513-1519.
Walter, J.A. & Schulten, K.J. (1993). Implementation of self-organizing neural networks for visuo-motor control of an industrial robot. IEEE
Trans. on Neural Networks, 4(1):86-95.