COMPUTER SCIENCE REVIEW 3 (2009) 127–149

available at www.sciencedirect.com

journal homepage: www.elsevier.com/locate/cosrev

Survey

Reservoir computing approaches to recurrent neural

network training

Mantas Lukoševičius ∗ , Herbert Jaeger

School of Engineering and Science, Jacobs University Bremen gGmbH, P.O. Box 750 561, 28725 Bremen, Germany

A R T I C L E I N F O A B S T R A C T

Article history: Echo State Networks and Liquid State Machines introduced a new paradigm in artificial
Received 17 October 2008 recurrent neural network (RNN) training, where an RNN (the reservoir) is generated
Received in revised form randomly and only a readout is trained. The paradigm, becoming known as reservoir
27 March 2009 computing, greatly facilitated the practical application of RNNs and outperformed classical
Accepted 31 March 2009 fully trained RNNs in many tasks. It has lately become a vivid research field with
numerous extensions of the basic idea, including reservoir adaptation, thus broadening
the initial paradigm to using different methods for training the reservoir and the readout. This
review systematically surveys both current ways of generating/adapting the reservoirs
and training different types of readouts. It offers a natural conceptual classification of
the techniques, which transcends boundaries of the current “brand-names” of reservoir
methods, and thus aims to help in unifying the field and providing the reader with a
detailed “map” of it.
c 2009 Elsevier Inc. All rights reserved.

1. Introduction in the absence of input. Mathematically, this renders an

Artificial recurrent neural networks (RNNs) represent a large and
varied class of computational models that are designed by
more or less detailed analogy with biological brain modules.
In an RNN numerous abstract neurons (also called units or
processing elements) are interconnected by likewise abstracted
synaptic connections (or links), which enable activations to
propagate through the network. The characteristic feature of
RNNs that distinguishes them from the more widely used
feedforward neural networks is that the connection topology
possesses cycles. The existence of cycles has a profound
impact:
• An RNN may develop a self-sustained temporal activation
dynamics along its recurrent connection pathways, even
RNN to be a dynamical system, while feedforward networks
are functions.
• If driven by an input signal, an RNN preserves in its
internal state a nonlinear transformation of the input
history — in other words, it has a dynamical memory, and
is able to process temporal context information.
This review article concerns a particular subset of RNN-
based research in two aspects:
• RNNs are used for a variety of scientific purposes,
and at least two major classes of RNN models exist:
they can be used for purposes of modeling biological
brains, or as engineering tools for technical applications.
The first usage belongs to the field of computational
neuroscience, while the second frames RNNs in the realms

∗ Corresponding author.
E-mail addresses: m.lukosevicius@jacobs-university.de (M. Lukoševičius), h.jaeger@jacobs-university.de (H. Jaeger).
1574-0137/$ - see front matter
c 2009 Elsevier Inc. All rights reserved.
doi:10.1016/j.cosrev.2009.03.005
128 COMPUTER SCIENCE REVIEW
of machine learning, the theory of computation, and
nonlinear signal processing and control. While there are
interesting connections between the two attitudes, this
survey focuses on the latter, with occasional borrowings
from the first.
• From a dynamical systems perspective, there are two
main classes of RNNs. Models from the first class
are characterized by an energy-minimizing stochastic
dynamics and symmetric connections. The best known
instantiations are Hopfield networks [1,2], Boltzmann
machines [3,4], and the recently emerging Deep Belief
Networks [5]. These networks are mostly trained in some
unsupervised learning scheme. Typical targeted network
functionalities in this field are associative memories,
data compression, the unsupervised modeling of data
distributions, and static pattern classification, where the
model is run for multiple time steps per single input
instance to reach some type of convergence or equilibrium
(but see e.g., [6] for extension to temporal data). The
mathematical background is rooted in statistical physics.
In contrast, the second big class of RNN models
typically features a deterministic update dynamics and
directed connections. Systems from this class implement
nonlinear filters, which transform an input time series into
an output time series. The mathematical background here
is nonlinear dynamical systems. The standard training
mode is supervised. This survey is concerned only with
RNNs of this second type, and when we speak of RNNs later
on, we will exclusively refer to such systems.1
RNNs (of the second type) appear as highly promising
and fascinating tools for nonlinear time series processing
applications, mainly for two reasons. First, it can be shown
that under fairly mild and general assumptions, such
RNNs are universal approximators of dynamical systems [7].
Second, biological brain modules almost universally exhibit
recurrent connection pathways too. Both observations
indicate that RNNs should potentially be powerful tools for
engineering applications.
Despite this widely acknowledged potential, and despite
a number of successful academic and practical applications,
the impact of RNNs in nonlinear modeling has remained
limited for a long time. The main reason for this lies in the
fact that RNNs are difficult to train by gradient-descent-based
methods, which aim at iteratively reducing the training error.
While a number of training algorithms have been proposed (a
brief overview is given in Section 2.5), these all suffer from the
following shortcomings:
• The gradual change of network parameters during learn-
ing drives the network dynamics through bifurcations [8].
At such points, the gradient information degenerates and
may become ill-defined. As a consequence, convergence
cannot be guaranteed.
• A single parameter update can be computationally
expensive, and many update cycles may be necessary. This
results in long training times, and renders RNN training
feasible only for relatively small networks (in the order of
tens of units).
1 However, they can also be used in a converging mode, as
shown at the end of Section 8.6.
3 (2009) 127–149
• It is intrinsically hard to learn dependences requiring
long-range memory, because the necessary gradient
information exponentially dissolves over time [9] (but see
the Long Short-Term Memory networks [10] for a possible
escape).
• Advanced training algorithms are mathematically in-
volved and need to be parameterized by a number of global
control parameters, which are not easily optimized. As a
result, such algorithms need substantial skill and experi-
ence to be successfully applied.
In this situation of slow and difficult progress, in
2001 a fundamentally new approach to RNN design and
training was proposed independently by Wolfgang Maass
under the name of Liquid State Machines [11] and by
Herbert Jaeger under the name of Echo State Networks [12].
This approach, which had predecessors in computational
neuroscience [13] and subsequent ramifications in machine
learning as the Backpropagation-Decorrelation [14] learning rule,
is now increasingly often collectively referred to as Reservoir
Computing (RC). The RC paradigm avoids the shortcomings
of gradient-descent RNN training listed above, by setting up
RNNs in the following way:
• A recurrent neural network is randomly created and
remains unchanged during training. This RNN is called
the reservoir. It is passively excited by the input signal and
maintains in its state a nonlinear transformation of the
input history.
• The desired output signal is generated as a linear
combination of the neuron’s signals from the input-
excited reservoir. This linear combination is obtained by
linear regression, using the teacher signal as a target.
Fig. 1 graphically contrasts previous methods of RNN
training with the RC approach.
Reservoir Computing methods have quickly become
popular, as witnessed for instance by a theme issue of
Neural Networks [15], and today constitute one of the basic
paradigms of RNN modeling [16]. The main reasons for this
development are the following:
Modeling accuracy. RC has starkly outperformed previ-
ous methods of nonlinear system identification, predic-
tion and classification, for instance in predicting chaotic
dynamics (three orders of magnitude improved accuracy
[17]), nonlinear wireless channel equalization (two or-
ders of magnitude improvement [17]), the Japanese Vowel
benchmark (zero test error rate, previous best: 1.8% [18]),
financial forecasting (winner of the international forecast-
ing competition NN32 ), and in isolated spoken digits recog-
nition (improvement of word error rate on benchmark
from 0.6% of previous best system to 0.2% [19], and further
to 0% test error in recent unpublished work).
Modeling capacity. RC is computationally universal
for continuous-time, continuous-value real-time systems
modeled with bounded resources (including time and
value resolution) [20,21].
2 http://www.neural-forecasting-competition.com/NN3/index.
htm.
 COMPUTER SCIENCE REVIEW
Fig. 1 – A. Traditional gradient-descent-based RNN training methods adapt all connection weights (bold arrows), including
input-to-RNN, RNN-internal, and RNN-to-output weights. B. In Reservoir Computing, only the RNN-to-output weights are
adapted.
Biological plausibility. Numerous connections of RC
principles to architectural and dynamical properties
of mammalian brains have been established. RC (or
closely related models) provides explanations of why
biological brains can carry out accurate computations
with an “inaccurate” and noisy physical substrate [22,23],
especially accurate timing [24]; of the way in which visual
information is superimposed and processed in primary
visual cortex [25,26]; of how cortico-basal pathways
support the representation of sequential information; and
RC offers a functional interpretation of the cerebellar
circuitry [27,28]. A central role is assigned to an RC
circuit in a series of models explaining sequential
information processing in human and primate brains,
most importantly of speech signals [13,29–31].
Extensibility and parsimony. A notorious conundrum of
neural network research is how to extend previously
learned models by new items without impairing or
destroying previously learned representations (catastrophic
interference [32]). RC offers a simple and principled solution:
new items are represented by new output units, which
are appended to the previously established output units
of a given reservoir. Since the output weights of different
output units are independent of each other, catastrophic
interference is a non-issue.
These encouraging observations should not mask the
fact that RC is still in its infancy, and significant further
improvements and extensions are desirable. Specifically, just
simply creating a reservoir at random is unsatisfactory. It
seems obvious that, when addressing a specific modeling
task, a specific reservoir design that is adapted to the task
will lead to better results than a naive random creation. Thus,
the main stream of research in the field is today directed
at understanding the effects of reservoir characteristics on
task performance, and at developing suitable reservoir design
and adaptation methods. Also, new ways of reading out
from the reservoirs, including combining them into larger
structures, are devised and investigated. While shifting from
the initial idea of having a fixed randomly created reservoir
and training only the readout, the current paradigm of
reservoir computing remains (and differentiates itself from
other RNN training approaches) as producing/training the
2 
reservoir and the readout separately and differently.
y(n) − ytarget (n)
This review offers a conceptual classification and a
2  , (1)
comprehensive survey of this research.
D
ytarget (n) − ytarget (n)
As is true for many areas of machine learning, methods in
reservoir computing converge from different fields and come
3 (2009) 127–149 129
with different names. We would like to make a distinction
here between these differently named “tradition lines”, which
we like to call brands, and the actual finer-grained ideas on
producing good reservoirs, which we will call recipes. Since
recipes can be useful and mixed across different brands, this
review focuses on classifying and surveying them. To be fair,
it has to be said that the authors of this survey associate
themselves mostly with the Echo State Networks brand, and
thus, willingly or not, are influenced by its mindset.
Overview. We start by introducing a generic notational frame-
work in Section 2. More specifically, we define what we mean
by problem or task in the context of machine learning in Sec-
tion 2.1. Then we define a general notation for expansion
(or kernel) methods for both non-temporal (Section 2.2) and
temporal (Section 2.3) tasks, introduce our notation for re-
current neural networks in Section 2.4, and outline classical
training methods in Section 2.5. In Section 3 we detail the
foundations of Reservoir Computing and proceed by naming
the most prominent brands. In Section 4 we introduce our
classification of the reservoir generation/adaptation recipes,
which transcends the boundaries between the brands. Fol-
lowing this classification we then review universal (Section 5),
unsupervised (Section 6), and supervised (Section 7) reservoir
generation/adaptation recipes. In Section 8 we provide a clas-
sification and review the techniques for reading the outputs
from the reservoirs reported in literature, together with dis-
cussing various practical issues of readout training. A final
discussion (Section 9) wraps up the entire picture.
2. Formalism
2.1. Formulation of the problem
Let a problem or a task in our context of machine learning be
defined as a problem of learning a functional relation between
a given input u(n) ∈ RNu and a desired output ytarget (n) ∈ RNy ,
where n = 1, . . . , T, and T is the number of data points in
the training dataset {(u(n), ytarget (n))}. A non-temporal task is
where the data points are independent of each other and the
goal is to learn a function y(n) = y(u(n)) such that E(y, ytarget )
is minimized, where E is an error measure, for instance, the
normalized root-mean-square error (NRMSE)
v 
u
u
u
E(y, ytarget ) = E
u
u 
t
where k·k stands for the Euclidean distance (or norm).
130 COMPUTER SCIENCE REVIEW
A temporal task is where u and ytarget are signals in a
discrete time domain n = 1, . . . , T, and the goal is to learn
a function y(n) = y(. . . , u(n − 1), u(n)) such that E(y, ytarget )
is minimized. Thus the difference between the temporal and
non-temporal task is that the function y(·) we are trying to
learn has memory in the first case and is memoryless in
the second. In both cases the underlying assumption is, of
course, that the functional dependence we are trying to learn
actually exists in the data. For the temporal case this spells
out as data adhering to an additive noise model of the form
ytarget (n) = ytarget (. . . , u(n − 1), u(n)) + θ(n), where ytarget (·) is
the relation to be learned by y(·) and θ(n) ∈ RNy is a noise term,
limiting the learning precision, i.e., the precision of matching
the learned y(n) to ytarget (n).
Whenever we say that the task or the problem is learned
well, or with good accuracy or precision, we mean that
E(y, ytarget ) is small. Normally one part of the T data points is
used for training the model and another part (unseen during
the training) for testing it. When speaking about output errors
and performance or precision we will have testing errors in mind
(if not explicitly specified otherwise). Also n, denoting the
discrete time, will often be used omitting its range 1, . . . , T.
2.2. Expansions and kernels in non-temporal tasks
Many tasks cannot be accurately solved by a simple linear
relation between the u and ytarget , i.e., a linear model y(n) =
Wu(n) (where W ∈ RNy ×Nu ) gives big errors E(y, ytarget )
regardless of W. In such situations one has to resort to
nonlinear models. A number of generic and widely used
approaches to nonlinear modeling are based on the idea of
nonlinearly expanding the input u(n) into a high-dimensional
feature vector x(n) ∈ RNx , and then utilizing those features
using linear methods, for instance by linear regression or
computing for a linear separation hyperplane, to get a
reasonable y. Solutions of this kind can be expressed in the
form
y(n) = Wout x(n) = Wout x(u(n)), (2)
Ny ×Nx
where Wout ∈ R are the trained output weights.
Typically Nx  Nu , and we will often consider u(n) as included
in x(n). There is also typically a constant bias value added
to (2), which is omitted here and in other equations for
brevity. The bias can be easily implemented, having one of
the features in x(n) constant (e.g., = 1) and a corresponding
column in Wout . Some models extend (2) to
y(n) = fout (Wout x[u(n)]), (3)
where fout (·) is some nonlinear function (e.g., a sigmoid
applied element-wise). For the sake of simplicity we will
consider this definition as equivalent to (2), since fout (·) can
be eliminated from y by redefining the target as y0target =
−1
fout (ytarget ) (and the error function E(y, y0target ), if desired).
Note that (2) is a special case of (3), with fout (·) being the
identity.
Functions x(u(n)) that transform an input u(n) into a
(higher-dimensional) vector x(n) are often called kernels (and
traditionally denoted φ(u(n))) in this context. Methods using
kernels often employ the kernel trick, which refers to the
3 (2009) 127–149
option afforded by many kernels of computing inner products
in the (high-dimensional, hence expensive) feature space of
x more cheaply in the original space populated by u. The
term kernel function has acquired a close association with the
kernel trick. Since here we will not exploit the kernel trick, in
order to avoid confusion we will use the more neutral term of
an expansion function for x(u(n)), and refer to methods using
such functions as expansion methods. These methods then
include Support Vector Machines (which standardly do use the
kernel trick), Feedforward Neural Networks, Radial Basis Function
approximators, Slow Feature Analysis, and various Probability
Mixture models, among many others. Feedforward neural
networks are also often referred to as (multilayer) perceptrons
in the literature.
While training the output Wout is a well defined and
understood problem, producing a good expansion function
x(·) generally involves more creativity. In many expansion
methods, e.g., Support Vector Machines, the function is
chosen “by hand” (most often through trial-and-error) and is
fixed.
2.3. Expansions in temporal tasks
Many temporal methods are based on the same principle.
The difference is that in a temporal task the function
to be learned depends also on the history of the input,
as discussed in Section 2.1. Thus, the expansion function
has memory: x(n) = x(. . . , u(n − 1), u(n)), i.e., it is an
expansion of the current input and its (potentially infinite)
history. Since this function has an unbounded number
of parameters, practical implementations often take an
alternative, recursive, definition:
x(n) = x(x(n − 1), u(n)). (4)
The output y(n) is typically produced in the same way as
for non-temporal methods by (2) or (3).
In addition to the nonlinear expansion, as in the non-
temporal tasks, such x(n) could be seen as a type of a spatial
embedding of the temporal information of . . . , u(n − 1), u(n).
This, for example, enables capturing higher-dimensional
dynamical attractors y(n) = ytarget (. . . , u(n−1), u(n)) = u(n+1)
of the system being modeled by y(·) from a series of lower-
dimensional observations u(n) the system is emitting, which
is shown to be possible by Takens’s theorem [33].
2.4. Recurrent neural networks
The type of recurrent neural networks that we will consider
most of the time in this review is a straightforward
implementation of (4). The nonlinear expansion with
memory here leads to a state vector of the form
x(n) = f (Win u(n) + Wx(n − 1)), n = 1, . . . , T, (5)
where x(n) ∈ RNx is a vector of reservoir neuron activations at
a time step n, f (·) is the neuron activation function, usually
the symmetric tanh(·), or the positive logistic (or Fermi)
sigmoid, applied element-wise, Win ∈ RNx ×Nu is the input
weight matrix, and W ∈ RNx ×Nx is a weight matrix of internal
network connections. The network is usually started with the
initial state x(0) = 0. Bias values are again omitted in (5) in
 COMPUTER SCIENCE REVIEW
the same way as in (2). The readout y(n) of the network is
implemented as in (3).
Some models of RNNs extend (5) as
x(n) = f (Win u(n) + Wx(n − 1) + Wofb y(n − 1)),
n = 1, . . . , T, (6)
where Wofb ∈ R Nx ×Ny
is an optional output feedback weight
matrix.
2.5. Classical training of RNNs
The classical approach to supervised training of RNNs,
known as gradient descent, is by iteratively adapting all
weights Wout , W, Win , and possibly Wofb (which as a whole
we denote Wall for brevity) according to their estimated
gradients ∂E/∂Wall , in order to minimize the output error
E = E(y, ytarget ). A classical example of such methods is
Real-Time Recurrent Learning [34], where the estimation of
∂E/∂Wall is done recurrently, forward in time. Conversely,
error backpropagation (BP) methods for training RNNs, which
are derived as extensions of the BP method for feedforward
neural networks [35], estimate ∂E/∂Wall by propagating
E(y, ytarget ) backwards through network connections and
time. The BP group of methods is arguably the most
prominent in classical RNN training, with the classical
example in this group being Backpropagation Through
Time [36]. It has a runtime complexity of O(Nx 2 ) per weight
update per time step for a single output Ny = 1, compared to
O(Nx 4 ) for Real-Time Recurrent Learning.
A systematic unifying overview of many classical gradient
descent RNN training methods is presented in [37]. The same
contribution also proposes a new approach, often referred
to by others as Atiya–Parlos Recurrent Learning (APRL). It
estimates gradients with respect to neuron activations ∂E/∂x
(instead of weights directly) and gradually adapts the weights
Wall to move the activations x into the desired directions. The
method is shown to converge faster than previous ones. See
Section 3.4 for more implications of APRL and bridging the
gap between the classical gradient descent and the reservoir
computing methods.
There are also other versions of supervised RNN
training, formulating the training problem differently, such
as using Extended Kalman Filters [38] or the Expectation-
Maximization algorithm [39], as well as dealing with special
types of RNNs, such as Long Short-Term Memory [40] modular
networks capable of learning long-term dependences.
There are many more, arguably less prominent, methods
and their modifications for RNN training that are not
mentioned here, as this would lead us beyond the scope of
this review. The very fact of their multiplicity suggests that
there is no clear winner in all aspects. Despite many advances
that the methods cited above have introduced, they still have
multiple common shortcomings, as pointed out in Section 1.
3. Reservoir methods
Reservoir computing methods differ from the “traditional”
designs and learning techniques listed above in that they
make a conceptual and computational separation between
3 (2009) 127–149 131
a dynamic reservoir — an RNN as a nonlinear temporal
expansion function — and a recurrence-free (usually linear)
readout that produces the desired output from the expansion.
This separation is based on the understanding (common
with kernel methods) that x(·) and y(·) serve different
purposes: x(·) expands the input history u(n), u(n − 1), . . . into
a rich enough reservoir state space x(n) ∈ RNx , while y(·)
combines the neuron signals x(n) into the desired output
signal ytarget (n). In the linear readout case (2), for each
dimension yi of y an output weight vector (Wout )i in the same
space RNx is found such that
(Wout )i x(n) = yi (n) ≈ ytarget i (n), (7)
while the “purpose” of x(n) is to contain a rich enough
representation to make this possible.
Since the expansion and the readout serve different
purposes, training/generating them separately and even
with different goal functions makes sense. The readout
y(n) = y(x(n)) is essentially a non-temporal function, learning
which is relatively simple. On the other hand, setting
up the reservoir such that a “good” state expansion x(n)
emerges is an ill-understood challenge in many respects.
The “traditional” RNN training methods do not make the
conceptual separation of a reservoir vs. a readout, and train
both reservoir-internal and output weights in technically
the same fashion. Nonetheless, even in traditional methods
the ways of defining the error gradients for the output y(n)
and the internal units x(n) are inevitably different, reflecting
that an explicit target ytarget (n) is available only for the
output units. Analyses of traditional training algorithms
have furthermore revealed that the learning dynamics of
internal vs. output weights exhibit systematic and striking
differences. This theme will be expanded in Section 3.4.
Currently, reservoir computing is a vivid fresh RNN
research stream, which has recently gained wide attention
due to the reasons pointed out in Section 1.
We proceed to review the most prominent “named”
reservoir methods, which we call here brands. Each of them
has its own history, a specific mindset, specific types of
reservoirs, and specific insights.
3.1. Echo State Networks
Echo State Networks (ESNs) [16] represent one of the two
pioneering reservoir computing methods. The approach is
based on the observation that if a random RNN possesses
certain algebraic properties, training only a linear readout
from it is often sufficient to achieve excellent performance
in practical applications. The untrained RNN part of an ESN
is called a dynamical reservoir, and the resulting states x(n) are
termed echoes of its input history [12]—this is where reservoir
computing draws its name from.
ESNs standardly use simple sigmoid neurons, i.e., reservoir
states are computed by (5) or (6), where the nonlinear function
f (·) is a sigmoid, usually the tanh(·) function. Leaky integrator
neuron models represent another frequent option for ESNs,
which is discussed in depth in Section 5.5. Classical recipes of
producing the ESN reservoir (which is in essence Win and W)
are outlined in Section 5.1, together with input-independent
properties of the reservoir. Input-dependent measures of the
132 COMPUTER SCIENCE REVIEW
quality of the activations x(n) in the reservoir are presented
in Section 6.1.
The readout from the reservoir is usually linear (3), where
u(n) is included as part of x(n), which can also be spelled out
in (3) explicitly as
y(n) = fout (Wout [u(n)|x(n)]), (8)
where Wout ∈ RNy ×(Nu +Nx ) is the learned output weight
matrix, fout (·) is the output neuron activation function
(usually the identity) applied component-wise, and ·|· stands
for a vertical concatenation of vectors. The original and most
popular batch training method to compute Wout is linear
regression, discussed in Section 8.1.1, or a computationally
cheap online training discussed in Section 8.1.2.
The initial ESN publications [12,41–43,17] were framed in
settings of machine learning and nonlinear signal processing
applications. The original theoretical contributions of early
ESN research concerned algebraic properties of the reservoir
that make this approach work in the first place (the echo state
property [12] discussed in Section 5.1) and analytical results
characterizing the dynamical short-term memory capacity of
reservoirs [41] discussed in Section 6.1.
3.2. Liquid State Machines
Liquid State Machines (LSMs) [11] are the other pioneering reser-
voir method, developed independently from and simultane-
ously with ESNs. LSMs were developed from a computational
neuroscience background, aiming at elucidating the princi-
pal computational properties of neural microcircuits [11,20,
44,45]. Thus LSMs use more sophisticated and biologically re-
alistic models of spiking integrate-and-fire neurons and dy-
namic synaptic connection models in the reservoir. The con-
nectivity among the neurons often follows topological and
metric constraints that are biologically motivated. In the LSM
literature, the reservoir is often referred to as the liquid, fol-
lowing an intuitive metaphor of the excited states as ripples
on the surface of a pool of water. Inputs to LSMs also usually
consist of spike trains. In their readouts LSMs originally used
multilayer feedforward neural networks (of either spiking or
sigmoid neurons), or linear readouts similar to ESNs [11]. Ad-
ditional mechanisms for averaging spike trains to get real-
valued outputs are often employed.
RNNs of the LSM-type with spiking neurons and more
sophisticated synaptic models are usually more difficult to
implement, to correctly set up and tune, and typically more
expensive to emulate on digital computers3 than simple ESN-
type “weighted sum and nonlinearity” RNNs. Thus they are
less widespread for engineering applications of RNNs than
the latter. However, while the ESN-type neurons only emulate
mean firing rates of biological neurons, spiking neurons are
able to perform more complicated information processing,
due to the time coding of the information in their signals
(i.e., the exact timing of each firing also matters). Also findings
on various mechanisms in natural neural circuits are more
3 With a possible exception of event-driven spiking NN
simulations, where the computational load varies depending on
the amount of activity in the NN.
3 (2009) 127–149
easily transferable to these more biologically-realistic models
(there is more on this in Section 6.2).
The main theoretical contributions of the LSM brand to
Reservoir Computing consist in analytical characterizations
of the computational power of such systems [11,21] discussed
in Sections 6.1 and 7.4.
3.3. Evolino
Evolino [46] transfers the idea of ESNs from an RNN of
simple sigmoidal units to a Long Short-Term Memory type
of RNNs [40] constructed from units capable of preserving
memory for long periods of time. In Evolino the weights of
the reservoir are trained using evolutionary methods, as is
also done in some extensions of ESNs, both discussed in
Section 7.2.
3.4. Backpropagation-Decorrelation
The idea of separation between a reservoir and a readout
function has also been arrived at from the point of view of
optimizing the performance of the RNN training algorithms
that use error backpropagation, as already indicated in
Section 2.5. In an analysis of the weight dynamics of an
RNN trained using the APRL learning algorithm [47], it was
revealed that the output weights Win of the network being
trained change quickly, while the hidden weights W change
slowly and in the case of a single output Ny = 1 the changes
are column-wise coupled. Thus in effect APRL decouples the
RNN into a quickly adapting output and a slowly adapting
reservoir. Inspired by these findings a new iterative/online
RNN training method, called BackPropagation-DeCorrelation
(BPDC), was introduced [14]. It approximates and significantly
simplifies the APRL method, and applies it only to the output
weights Wout , turning it into an online RC method. BPDC
uses the reservoir update equation defined in (6), where
output feedbacks Wofb are essential, with the same type of
units as ESNs. BPDC learning is claimed to be insensitive to
the parameters of fixed reservoir weights W. BPDC boasts
fast learning times and thus is capable of tracking quickly
changing signals. As a downside of this feature, the trained
network quickly forgets the previously seen data and is highly
biased by the recent data. Some remedies for reducing this
effect are reported in [48]. Most of applications of BPDC in
the literature are for tasks having one-dimensional outputs
Ny = 1; however BPDC is also successfully applied to Ny > 1,
as recently demonstrated in [49].
From a conceptual perspective we can define a range of
RNN training methods that gradually bridge the gap between
the classical BP and reservoir methods:
1. Classical BP methods, such as Backpropagation Through
Time (BPTT) [36];
2. Atiya–Parlos recurrent learning (APRL) [37];
3. BackPropagation-DeCorrelation (BPDC) [14];
4. Echo State Networks (ESNs) [16].
In each method of this list the focus of training gradually
moves from the entire network towards the output, and
convergence of the training is faster in terms of iterations,
with only a single “iteration” in case 4. At the same time the
potential expressiveness of the RNN, as per the same number
of units in the NN, becomes weaker. All methods in the list
primarily use the same type of simple sigmoid neuron model.
 COMPUTER SCIENCE REVIEW
3.5. Temporal Recurrent Networks
This summary of RC brands would be incomplete without
a spotlight directed at Peter F. Dominey’s decade-long
research suite on cortico-striatal circuits in the human brain
(e.g., [13,29,31], and many more). Although this research is
rooted in empirical cognitive neuroscience and functional
neuroanatomy and aims at elucidating complex neural
structures rather than theoretical computational principles,
it is probably Dominey who first clearly spelled out the RC
principle: “(. . . ) there is no learning in the recurrent connections
[within a subnetwork corresponding to a reservoir], only
between the State [i.e., reservoir] units and the Output units.
Second, adaptation is based on a simple associative learning
mechanism (. . . )” [50]. It is also in this article where Dominey
brands the neural reservoir module as a Temporal Recurrent
Network. The learning algorithm, to which Dominey alludes,
can be seen as a version of the Least Mean Squares discussed
in Section 8.1.2. At other places, Dominey emphasizes the
randomness of the connectivity in the reservoir: “It is worth
noting that the simulated recurrent prefrontal network relies on fixed
randomized recurrent connections, (. . . )” [51]. Only in early 2008
did Dominey and “computational” RC researchers become
aware of each other.
3.6. Other (exotic) types of reservoirs
As is clear from the discussion of the different reservoir
methods so far, a variety of neuron models can be used for the
reservoirs. Using different activation functions inside a single
reservoir might also improve the richness of the echo states,
as is illustrated, for example, by inserting some neurons
with wavelet-shaped activation functions into the reservoir
of ESNs [52]. A hardware implementation friendly version
of reservoirs composed of stochastic bitstream neurons was
proposed in [53].
In fact the reservoirs do not necessarily need to be neural
networks, governed by dynamics similar to (5). Other types
of high-dimensional dynamical systems that can take an
input u(n) and have an observable state x(n) (which does
not necessarily fully describe the state of the system) can
be used as well. In particular this makes the reservoir
paradigm suitable for harnessing the computational power
of unconventional hardware, such as analog electronics [54,
55], biological neural tissue [26], optical [56], quantum, or
physical “computers”. The last of these was demonstrated
(taking the “reservoir” and “liquid” idea quite literally) by
feeding the input via mechanical actuators into a reservoir
full of water, recording the state of its surface optically,
and successfully training a readout multilayer perceptron
on several classification tasks [57]. An idea of treating a
computer-simulated gene regulation network of Escherichia
Coli bacteria as the reservoir, a sequence of chemical stimuli
as an input, and measures of protein levels and mRNAs as an
output is explored in [58].
3.7. Other overviews of reservoir methods
An experimental comparison of LSM, ESN, and BPDC reservoir
methods with different neuron models, even beyond the
standard ones used for the respective methods, and different
parameter settings is presented in [59]. A brief and broad
3 (2009) 127–149 133
overview of reservoir computing is presented in [60], with
an emphasis on applications and hardware implementations
of reservoir methods. The editorial in the “Neural Networks”
journal special issue on ESNs and LSMs [15] offers a short
introduction to the topic and an overview of the articles in
the issue (most of which are also surveyed here). An older and
much shorter part of this overview, covering only reservoir
adaptation techniques, is available as a technical report [61].
4. Our classification of reservoir recipes
The successes of applying RC methods to benchmarks (see
the listing in Section 1) outperforming classical fully trained
RNNs do not imply that randomly generated reservoirs
are optimal and cannot be improved. In fact, “random”
is almost by definition an antonym to “optimal”. The
results rather indicate the need for some novel methods of
training/generating the reservoirs that are very probably not
a direct extension of the way the output is trained (as in
BP). Thus besides application studies (which are not surveyed
here), the bulk of current RC research on reservoir methods is
devoted to optimal reservoir design, or reservoir optimization
algorithms.
It is worth mentioning at this point that the general “no
free lunch” principle in supervised machine learning [62]
states that there can exist no bias of a model which would
universally improve the accuracy of the model for all possible
problems. In our context this can be translated into a claim
that no single type of reservoir can be optimal for all types of
problems.
In this review we will try to survey all currently
investigated ideas that help producing “good” reservoirs. We
will classify those ideas into three major groups based on
their universality:
• Generic guidelines/methods of producing good reservoirs
irrespective of the task (both the input u(n) and the desired
output ytarget (n));
• Unsupervised pre-training of the reservoir with respect to
the given input u(n), but not the target ytarget (n);
• Supervised pre-training of the reservoir with respect to both
the given input u(n) and the desired output ytarget (n).
These three classes of methods are discussed in the
following three sections. Note that many of the methods to
some extend transcend the boundaries of these three classes,
but will be classified according to their main principle.
5. Generic reservoir recipes
The most classical methods of producing reservoirs all fall
into this category. All of them generate reservoirs randomly,
with topology and weight characteristics depending on some
preset parameters. Even though they are not optimized for
a particular input u(n) or target ytarget (n), a good manual
selection of the parameters is to some extent task-dependent,
complying with the “no free lunch” principle just mentioned.
5.1. Classical ESN approach
Some of the most generic guidelines of producing good
reservoirs were presented in the papers that introduced
134 COMPUTER SCIENCE REVIEW
ESNs [12,42]. Motivated by an intuitive goal of producing a
“rich” set of dynamics, the recipe is to generate a (i) big, (ii)
sparsely and (iii) randomly connected, reservoir. This means
that (i) Nx is sufficiently large, with order ranging from tens
to thousands, (ii) the weight matrix W is sparse, with several
to 20 per cent of possible connections, and (iii) the weights
of the connections are usually generated randomly from a
uniform distribution symmetric around the zero value. This
design rationale aims at obtaining many, due to (i), reservoir
activation signals, which are only loosely coupled, due to (ii),
and different, due to (iii).
The input weights Win and the optional output feedback
weights Wofb are usually dense (they can also be sparse like
W) and generated randomly from a uniform distribution. The
exact scaling of both matrices and an optional shift of the
input (a constant value added to u(n)) are the few other free
parameters that one has to choose when “baking” an ESN. The
rules of thumb for them are the following. The scaling of Win
and shifting of the input depends on how much nonlinearity
of the processing unit the task needs: if the inputs are close
to 0, the tanh neurons tend to operate with activations close
to 0, where they are essentially linear, while inputs far from
0 tend to drive them more towards saturation where they
exhibit more nonlinearity. The shift of the input may help to
overcome undesired consequences of the symmetry around
0 of the tanh neurons with respect to the sign of the signals.
Similar effects are produced by scaling the bias inputs to the
neurons (i.e., the column of Win corresponding to constant
input, which often has a different scaling factor than the
rest of Win ). The scaling of Wofb is in practice limited by
a threshold at which the ESN starts to exhibit an unstable
behavior, i.e., the output feedback loop starts to amplify (the
errors of) the output and thus enters a diverging generative
mode. In [42], these and related pieces of advice are given
without a formal justification.
An important element for ESNs to work is that the
reservoir should have the echo state property [12]. This
condition in essence states that the effect of a previous state
x(n) and a previous input u(n) on a future state x(n + k)
should vanish gradually as time passes (i.e., k → ∞), and
not persist or even get amplified. For most practical purposes,
the echo state property is assured if the reservoir weight
matrix W is scaled so that its spectral radius ρ(W) (i.e., the
largest absolute eigenvalue) satisfies ρ(W) < 1 [12]. Or, using
another term, W is contractive. The fact that ρ(W) < 1
almost always ensures the echo state property has led to an
unfortunate misconception which is expressed in many RC
publications, namely, that ρ(W) < 1 amounts to a necessary
and sufficient condition for the echo state property. This is
wrong. The mathematically correct connection between the
spectral radius and the echo state property is that the latter
is violated if ρ(W) > 1 in reservoirs using the tanh function as
neuron nonlinearity, and for zero input. Contrary to widespread
misconceptions, the echo state property can be obtained even
if ρ(W) > 1 for non-zero input (including bias inputs to
neurons), and it may be lost even if ρ(W) < 1, although it is
hard to construct systems where this occurs (unless f 0 (0) > 1
for the nonlinearity f ), and in practice this does not happen.
The optimal value of ρ(W) should be set depending on
the amount of memory and nonlinearity that the given
3 (2009) 127–149
task requires. A rule of thumb, likewise discussed in [12], is
that ρ(W) should be close to 1 for tasks that require long
memory and accordingly smaller for the tasks where a too
long memory might in fact be harmful. Larger ρ(W) also have
the effect of driving signals x(n) into more nonlinear regions
of tanh units (further from 0) similarly to Win . Thus scalings
of both Win and W have a similar effect on nonlinearity of
the ESN, while their difference determines the amount of
memory.
A rather conservative rigorous sufficient condition of the
echo state property for any kind of inputs u(n) (including zero)
and states x(n) (with tanh nonlinearity) being σmax (W) < 1,
where σmax (W) is the largest singular value of W, was proved
in [12]. Recently, a less restrictive sufficient condition, namely,
infD∈D σmax (DWD−1 ) < 1, where D is an arbitrary matrix,
minimizing the so-called D-norm σmax (DWD−1 ), from a
set D ⊂ RNx ×Nx of diagonal matrices, has been derived
in [63]. This sufficient condition approaches the necessary
infD∈D σmax (DWD−1 ) → ρ(W)− , ρ(W) < 1, e.g., when W
is a normal or a triangular (permuted) matrix. A rigorous
sufficient condition for the echo state property is rarely
ensured in practice, with a possible exception being critical
control tasks, where provable stability under any conditions
is required.
5.2. Different topologies of the reservoir
There have been attempts to find topologies of the
ESN reservoir different from sparsely randomly connected
ones. Specifically, small-world [64], scale-free [65], and
biologically inspired connection topologies generated by
spatial growth [66] were tested for this purpose in a careful
study [67], which we point out here due to its relevance
although it was obtained only as a BSc thesis. The NRMS
error (1) of y(n) as well as the eigenvalue spread of the cross-
correlation matrix of the activations x(n) (necessary for a
fast online learning described in Section 8.1.2; see Section 6.1
for details) were used as the performance measures of the
topologies. This work also explored an exhaustive brute-
force search of topologies of tiny networks (motifs) of four
units, and then combining successful motives (in terms of the
eigenvalue spread) into larger networks. The investigation,
unfortunately, concludes that “(. . . ) none of the investigated
network topologies was able to perform significantly better than
simple random networks, both in terms of eigenvalue spread as
well as testing error” [67]. This, however, does not serve as
a proof that similar approaches are futile. An indication of
this is the substantial variation in ESN performance observed
among randomly created reservoirs, which is, naturally, more
pronounced in smaller reservoirs (e.g., [68]).
In contrast, LSMs often use a biologically plausible
connectivity structure and weight settings. In the original
form they model a single cortical microcolumn [11]. Since
the model of both the connections and the neurons
themselves is quite sophisticated, it has a large number of
free parameters to be set, which is done manually, guided
by biologically observed parameter ranges, e.g., as found
in the rat somatosensory cortex [69]. This type of model
also delivers good performance for practical applications of
speech recognition [69,70] (and many similar publications
 COMPUTER SCIENCE REVIEW
by the latter authors). Since LSMs aim at accuracy of
modeling natural neural structures, less biologically plausible
connectivity patterns are usually not explored.
It has been demonstrated that much more detailed
biological neural circuit models, which use anatomical
and neurophysiological data-based laminar (i.e., cortical
layer) connectivity structures and Hodgkin–Huxley model
neurons, improve the information-processing capabilities
of the models [23]. Such highly realistic (for present-day
standards) models “perform significantly better than control
circuits (which are lacking the laminar structures but are otherwise
identical with regard to their components and overall connection
statistics) for a wide variety of fundamental information-processing
tasks” [23].
Different from this direction of research, there are also
explorations of using even simpler topologies of the reservoir
than the classical ESN. It has been demonstrated that the
reservoir can even be an unstructured feed-forward network
with time-delayed connections if the finite limited memory
window that it offers is sufficient for the task at hand [71].
A degenerate case of a “reservoir” composed of linear
units and a diagonalized W and unitary inputs Win was
considered in [72]. A one-dimensional lattice (ring) topology
was used for a reservoir, together with an adaptation of
the reservoir discussed in Section 6.2, in [73]. A special
kind of excitatory and inhibitory neurons connected in a
one-dimensional spatial arrangement was shown to produce
interesting chaotic behavior in [74].
A tendency that higher ranks of the connectivity matrix
Wmask (where wmaski,j = 1 if w i,j 6= 0, and = 0 otherwise,
for i, j = 1, . . . , Nx ) correlate with lower ESN output errors
was observed in [75]. Connectivity patterns of W such that
W ∞ ≡ limk→∞ W k (W k standing for “W to the power
k” and approximating weights of the cumulative indirect
connections by paths of length k among the reservoir units)
is neither fully connected, nor all-zero, are claimed to give
a broader distribution of ESN prediction performances, thus
including best performing reservoirs, than random sparse
connectivities in [76]. A permutation matrix with a medium
number and different lengths of connected cycles, or a
general orthogonal matrix, are suggested as candidates for
such Ws.
5.3. Modular reservoirs
One of the shortcomings of conventional ESN reservoirs is
that, even though they are sparse, the activations are still
coupled so strongly that the ESN is poor in dealing with
different time scales simultaneously, e.g., predicting several
superimposed generators. This problem was successfully
tackled by dividing the reservoir into decoupled sub-
reservoirs and introducing inhibitory connections among all
the sub-reservoirs [77]. For the approach to be effective,
the inhibitory connections must predict the activations of
the sub-reservoirs one time step ahead. To achieve this the
inhibitory connections are heuristically computed from (the
rest of) W and Wofb , or the sub-reservoirs are updated in a
sequence and the real activations of the already updated sub-
reservoirs are used.
3 (2009) 127–149 135
Fig. 2 – Signal flow diagram of the standard ESN.
The Evolino approach introduced in Section 3.3 can also
be classified as belonging to this group, as the LSTM RNN
used for its reservoir consists of specific small memory-
holding modules (which could alternatively be regarded as
more complicated units of the network).
Approaches relying on combining outputs from several
separate reservoirs will be discussed in Section 8.8.
5.4. Time-delayed vs. instantaneous connections
Another time-related limitation of the classical ESNs pointed
out in [78] is that no matter how many neurons are contained
in the reservoir, it (like any other fully recurrent network
with all connections having a time delay) has only a single
layer of neurons (Fig. 2). This makes it intrinsically unsuitable
for some types of problems. Consider a problem where the
mapping from u(n) to ytarget (n) is a very complex, nonlinear
one, and the data in neighboring time steps are almost
independent (i.e., little memory is required), as e.g., the
“meta-learning” task in [79].4 Consider a single time step
n: signals from the input u(n) propagate only through one
untrained layer of weights Win , through the nonlinearity
f influence the activations x(n), and reach the output y(n)
through the trained weights Wout (Fig. 2). Thus ESNs are not
capable of producing a very complex instantaneous mapping
from u(n) to y(n) using a realistic number of neurons, which
could (only) be effectively done by a multilayer FFNN (not
counting some non-NN-based methods). Delaying the target
ytarget by k time steps would in fact make the signals coming
from u(n) “cross” the nonlinearities k+1 times before reaching
y(n + k), but would mix the information from different time
steps in x(n), . . . , x(n + k), breaking the required virtually
independent mapping u(n) → ytarget (n + k), if no special
structure of W is imposed.
As a possible remedy Layered ESNs were introduced in [78],
where a part (up to almost half) of the reservoir connections
can be instantaneous and the rest take one time step for
the signals to propagate as in normal ESNs. Randomly
generated Layered ESNs, however, do not offer a consistent
improvement for large classes of tasks, and pre-training
methods of such reservoirs have not yet been investigated.
The issue of standard ESNs not having enough trained
layers is also discussed and addressed in a broader context
in Section 8.8.
5.5. Leaky integrator neurons and speed of dynamics
In addition to the basic sigmoid units, leaky integrator
neurons were suggested to be used in ESNs from the point
of their introduction [12]. This type of neuron performs a
4 ESNs have been shown to perform well in a (significantly)
simpler version of the “meta-learning” in [80].
136 COMPUTER SCIENCE REVIEW
leaky integration of its activation from previous time steps.
Today a number of versions of leaky integrator neurons are
often used in ESNs, which we will call here leaky integrator
ESNs (LI-ESNs) where the distinction is needed. The main two
groups are those using leaky integration before application
of the activation function f (·), and after. One example of the
latter (in the discretized time case) has reservoir dynamics
governed by
x(n) = (1 − a1t)x(n − 1) + 1tf (Win u(n) + Wx(n − 1)), (9)
where 1t is a compound time gap between two consecutive
time steps divided by the time constant of the system and a
is the decay (or leakage) rate [81]. Another popular (and we
believe, preferable) design can be seen as setting a = 1 and
redefining δt in Eq. (9) as the leaking rate a to control the
“speed” of the dynamics,
x(n) = (1 − a)x(n − 1) + af (Win u(n) + Wx(n − 1)), (10)
which in effect is an exponential moving average, has only
one additional parameter and the desirable property that
neuron activations x(n) never go outside the boundaries
defined by f (·). Note that the simple ESN (5) is a special
case of LI-ESNs (9) or (10) with a = 1 and 1t = 1. As a
corollary, an LI-ESN with a good choice of the parameters
can always perform at least as well as a corresponding simple
ESN. With the introduction of the new parameter a (and 1t),
the condition for the echo state property is redefined [12].
A natural constraint on the two new parameters is a1t ∈
[0, 1] in (9), and a ∈ [0, 1] in (10) — a neuron should neither
retain, nor leak, more activation than it had. The effect
of these parameters on the final performance of ESNs was
investigated in [18] and [82]. The latter contribution also
considers applying the leaky integrator in different places of
the model and resampling the signals as an alternative.
The additional parameters of the LI-ESN control the
“speed” of the reservoir dynamics. Small values of a and
1t result in reservoirs that react slowly to the input. By
changing these parameters it is possible to shift the effective
interval of frequencies in which the reservoir is working.
Along these lines, time warping invariant ESNs (TWIESNs)
— an architecture that can deal with strongly time-warped
signals — were outlined in [81,18]. This architecture varies 1t
on-the-fly in (9), directly depending on the speed at which the
input u(n) is changing.
From a signal processing point of view, the exponential
moving average on the neuron activation (10) does a simple
low-pass filtering of its activations with the cutoff frequency
a approximation property measures the capability of the readout
fc = , (11)
2π(1 − a)1t
where 1t is the discretization time step. This makes the
neurons average out the frequencies above fc and enables
tuning the reservoirs for particular frequencies. Elaborating
further on this idea, high-pass neurons, that produce their
activations by subtracting from the unfiltered activation
(5) the low-pass filtered one (10), and band-pass neurons,
that combine the low-pass and high-pass ones, were
introduced [83]. The authors also suggested mixing neurons
with different passbands inside a single ESN reservoir,
and reported that a single reservoir of such kind is able
3 (2009) 127–149
to predict/generate signals having structure on different
timescales.
Following this line of thought, Infinite Impulse Response
(IIR) band-pass filters having sharper cutoff characteristics
were tried on neuron activations in ESNs with success in
several types of signals [84]. Since the filters often introduce
an undesired phase shift to the signals, a time delay for the
activation of each neuron was learned and applied before the
linear readout from the reservoir. A successful application of
Butterworth band-pass filters in ESNs is reported in [85].
Connections between neurons that have different time
delays (more than one time step) can actually also be used
inside the recurrent part, which enables the network to
operate on different timescales simultaneously and learn
longer-term dependences [86]. This idea has been tried for
RNNs trained by error backpropagation, but could also be
useful for multi-timescale reservoirs. Long-term dependences
can also be learned using the reservoirs mentioned in
Section 3.3.
6. Unsupervised reservoir adaptation
In this section we describe reservoir training/generation
methods that try to optimize some measure defined on the
activations x(n) of the reservoir, for a given input u(n), but
regardless of the desired output ytarget (n). In Section 6.1 we
survey measures that are used to estimate the quality of
the reservoir, irrespective of the methods optimizing them.
Then local, Section 6.2, and global, Section 6.3 unsupervised
reservoir training methods are surveyed.
6.1. “Goodness” measures of the reservoir activations
The classical feature that reservoirs should possess is the
echo state property, defined in Section 5.1. Even though this
property depends on the concrete input u(n), usually in
practice its existence is not measured explicitly, and only
the spectral radius ρ(W) is selected to be <1 irrespective of
u(n), or just tuned for the final performance. A measure of
short-term memory capacity, evaluating how well u(n) can be
reconstructed by the reservoir as y(n + k) after various delays
k, was introduced in [41].
The two necessary and sufficient conditions for LSMs to
work were introduced in [11]. A separation property measures
the distance between different states x caused by different
input sequences u. The measure is refined for binary ESN-
type reservoirs in [87] with a generalization in [88]. An
to produce a desired output ytarget from x, and thus is not an
unsupervised measure, but is included here for completeness.
Methods for estimating the computational power and
generalization capability of neural reservoirs were presented
in [89]. The proposed measure for computational power, or
kernel quality, is obtained in the following way. Take k different
input sequences (or segments of the same signal) ui (n), where
i = 1, . . . , k, and n = 1, . . . , Tk . For each input i take the
resulting reservoir state xi (n0 ), and collect them into a matrix
M ∈ Rk×Nx , where n0 is some fixed time after the appearance
of ui (n) in the input. Then the rank r of the matrix M is
 COMPUTER SCIENCE REVIEW
the measure. If r = k, this means that all the presented
inputs can be separated by a linear readout from the reservoir,
and thus the reservoir is said to have a linear separation
property. For estimating the generalization capability of the
reservoir, the same procedure can be performed with s (s  k)
inputs uj (n), j = 1, . . . , s, that represent the set of all possible
inputs. If the resultant rank r is substantially smaller than
the size s of the training set, the reservoir generalizes well.
These two measures are more targeted to tasks of time series
classification, but can also be revealing in predicting the
performance of regression [90].
A much-desired measure to minimize is the eigenvalue
spread (EVS, the ratio of the maximal eigenvalue to the
minimal eigenvalue) of the cross-correlation matrix of the
activations x(n). A small EVS is necessary for an online
training of the ESN output by a computationally cheap
and stable stochastic gradient descent algorithm outlined in
Section 8.1.2 (see, e.g., [91], chapter 5.3, for the mathematical
reasons that render this mandatory). In classical ESNs the EVS
sometimes reaches 1012 or even higher [92], which makes
the use of stochastic gradient descent training unfeasible.
Other commonly desirable features of the reservoir are small
pairwise correlation of the reservoir activations xi (n), or a
large entropy of the x(n) distribution (e.g., [92]). The latter is
a rather popular measure, as discussed later in this review. A
criterion for maximizing the local information transmission
of each individual neuron was investigated in [93] (more in
Section 6.2).
The so-called edge of chaos is a region of parameters of
a dynamical system at which it operates at the boundary
between the chaotic and non-chaotic behavior. It is often
claimed (but not undisputed; see, e.g., [94]) that at the
edge of chaos many types of dynamical systems, including
binary systems and reservoirs, possess high computational
power [87,95]. It is intuitively clear that the edge of chaos in
reservoirs can only arise when the effect of inputs on the
reservoir state does not die out quickly; thus such reservoirs
can potentially have high memory capacity, which is also
demonstrated in [95]. However, this does not universally
imply that such reservoirs are optimal [90]. The edge of chaos
can be empirically detected (even for biological networks)
by measuring Lyapunov exponents [95], even though such
measurements are not trivial (and often involve a degree of
expert judgment) for high-dimensional noisy systems. For
reservoirs of simple binary threshold units this can be done
more simply by computing the Hamming distances between
trajectories of the states [87]. There is also an empirical
observation that, while changing different parameter settings
of a reservoir, the best performance in a given task correlates
with a Lyapunov exponent specific to that task [59]. The
optimal exponent is related to the amount of memory needed
for the task, as discussed in Section 5.1. It was observed in
ESNs with no input that when ρ(W) is slightly greater than
1, the internally generated signals are periodic oscillations,
whereas for larger values of ρ(W), the signals are more
irregular and even chaotic [96]. Even though stronger inputs
u(n) can push the dynamics of the reservoirs out of the
chaotic regime and thus make them useful for computation,
no reliable benefit of such a mode of operation was found in
the last contribution.
3 (2009) 127–149 137
In contrast to ESN-type reservoirs of real-valued units,
simple binary threshold units exhibit a more immediate tran-
sition from damped to chaotic behavior without interme-
diate periodic oscillations [87]. This difference between the
two types of activation functions, including intermediate
quantized ones, in ESN-type reservoirs was investigated more
closely in [88]. The investigation showed that reservoirs of bi-
nary units are more sensitive to the topology and the connec-
tion weight parameters of the network in their transition be-
tween damped and chaotic behavior, and computational per-
formance, than the real-valued ones. This difference can be
related to the similar apparent difference in sensitivity of the
ESNs and LSM-type reservoirs of firing units, discussed in Sec-
tion 5.2.
6.2. Unsupervised local methods
A natural strategy for improving reservoirs is to mimic biology
(at a high level of abstraction) and count on local adaptation
rules. “Local” here means that parameters pertaining to some
neuron i are adapted on the basis of no other information
than the activations of neurons directly connected with
neuron i. In fact all local methods are almost exclusively
unsupervised, since the information on the performance E at
the output is unreachable in the reservoir.
First attempts to decrease the eigenvalue spread in ESNs
by classical Hebbian [97] (inspired by synaptic plasticity
in biological brains) or Anti-Hebbian learning gave no
success [92]. A modification of Anti-Hebbian learning, called
Anti-Oja learning, is reported to improve the performance of
ESNs in [98].
On the more biologically realistic side of the RC research
with spiking neurons, local unsupervised adaptations are
very natural to use. In fact, LSMs had used synaptic
connections with realistic short-term dynamic adaptation,
as proposed by [99], in their reservoirs from the very
beginning [11].
The Hebbian learning principle is usually implemented
in spiking NNs as spike-time-dependent plasticity (STDP) of
synapses. STDP is shown to improve the separation property
of LSMs for real-world speech data, but not for random inputs
u, in [100]. The authors however were uncertain whether
manually optimizing the parameters of the STDP adaptation
(which they did) or the ones for generating the reservoir
would result in a larger performance gain for the same effort
spent. STDP is shown to work well with time-coded readouts
from the reservoir in [101].
Biological neurons are widely observed to adapt their
intrinsic excitability, which often results in exponential
distributions of firing rates, as observed in visual cortex
(e.g., [102]). This homeostatic adaptation mechanism, called
intrinsic plasticity (IP), has recently attracted a wide attention
in the reservoir computing community. Mathematically, the
exponential distribution maximizes the entropy of a non-
negative random variable with a fixed mean; thus it enables
the neurons to transmit maximal information for a fixed
metabolic cost of firing. An IP learning rule for spiking model
neurons aimed at this goal was first presented in [103].
For a more abstract model of the neuron, having a
continuous Fermi sigmoid activation function f : R →
138 COMPUTER SCIENCE REVIEW
(0, 1), the IP rule was derived as a proportional control that
changes the steepness and offset of the sigmoid to get an
exponential-like output distribution in [104]. A more elegant
gradient IP learning rule for the same purpose was presented
in [93], which is similar to the information maximization
approach in [105]. Applying IP with Fermi neurons in reservoir
computing significantly improves the performance of BPDC-
trained networks [106,107], and is shown to have a positive
effect on offline trained ESNs, but can cause stability
problems for larger reservoirs [106]. An ESN reservoir with
IP-adapted Fermi neurons is also shown to enable predicting
several superimposed oscillators [108].
An adaptation of the IP rule to tanh neurons (f : R →
(−1, 1)) that results in a zero-mean Gaussian-like distribution
of activations was first presented in [73] and investigated
more in [55]. The IP-adapted ESNs were compared with
classical ones, both having Fermi and tanh neurons, in the
latter contribution. IP was shown to (modestly) improve the
performance in all cases. It was also revealed that ESNs
with Fermi neurons have significantly smaller short-term
memory capacity (as in Section 6.1) and worse performance
in a synthetic NARMA prediction task, while having a slightly
better performance in a speech recognition task, compared to
tanh neurons. The same type of tanh neurons adapted by IP
aimed at Laplacian distributions are investigated in [109]. In
general, IP gives more control on the working points of the
reservoir nonlinearity sigmoids. The slope (first derivative)
and the curvature (second derivative) of the sigmoid at the
point around which the activations are centered by the IP
rule affect the effective spectral radius and the nonlinearity
of the reservoir, respectively. Thus, for example, centering
tanh activations around points other than 0 is a good idea if
no quasi-linear behavior is desired. IP has recently become
employed in reservoirs as a standard practice by several
research groups.
Overall, an information-theoretic view on adaptation of
spiking neurons has a long history in computational neuro-
science. Even better than maximizing just any information
in the output of a neuron is maximizing relevant informa-
tion. In other words, in its output the neuron should encode
the inputs in such a way as to preserve maximal informa-
tion about some (local) target signal. This is addressed in a
general information-theoretical setting by the Information Bot-
tleneck (IB) method [110]. A learning rule for a spiking neuron
that maximizes mutual information between its inputs and
its output is presented in [111]. A more general IB learning
rule, transferring the general ideas of IB method to spiking
neurons, is introduced in [112] and [113]. Two semi-local train-
ing scenarios are presented in these two contributions. In the
first, a neuron optimizes the mutual information of its output
with outputs of some neighboring neurons, while minimizing
the mutual information with its inputs. In the second, two
neurons reading from the same signals maximize their in-
formation throughput, while keeping their inputs statistically
independent, in effect performing Independent Component
Analysis (ICA). A simplified online version of the IB training
rule with a variation capable of performing Principle Compo-
nent Analysis (PCA) was recently introduced in [114]. In addi-
tion, it assumes slow semi-local target signals, which is more
3 (2009) 127–149
biologically plausible. The approaches described in this para-
graph are still waiting to be tested in the reservoir computing
setting.
It is also of great interest to understand how different
types of plasticity observed in biological brains interact when
applied together and what effect this has on the quality of
reservoirs. The interaction of the IP with Hebbian synaptic
plasticity in a single Fermi neuron is investigated in [104]
and further in [115]. The synergy of the two plasticities is
shown to result in a better specialization of the neuron that
finds heavy-tail directions in the input. An interaction of
IP with a neighborhood-based Hebbian learning in a layer
of such neurons was also shown to maximize information
transmission, perform nonlinear ICA, and result in an
emergence of orientational Gabor-like receptive fields in [116].
The interaction of STDP with IP in an LSM-like reservoir of
simple sparsely spiking neurons was investigated in [117].
The interaction turned out to be a non-trivial one, resulting in
networks more robust to perturbations of the state x(n) and
having a better short-time memory and time series prediction
performance.
A recent approach of combining STDP with a biologically
plausible reinforcement signal is discussed in Section 7.5, as
it is not unsupervised.
6.3. Unsupervised global methods
Here we review unsupervised methods that optimize
reservoirs based on global information of the reservoir
activations induced by the given input u(x), but irrespective of
the target ytarget (n), like for example the measures discussed
in Section 6.1. The intuitive goal of such methods is to
produce good representations of (the history of) u(n) in x(n)
for any (and possibly several) ytarget (n).
A biologically inspired unsupervised approach with a
reservoir trying to predict itself is proposed in [118]. An
additional output z(n) ∈ RNx , z(n) = Wz x(n) from the reservoir
is trained on the target ztarget (n) = x0 (n + 1), where x0 (n) are
the activations of the reservoir before applying the neuron
transfer function tanh(·), i.e., x(n) = tanh(x0 (n)). Then, in
the application phase of the trained networks, the original
activations x0 (n), which result from u(n), Win , and W, are
mixed with the self-predictions z(n − 1) obtained from Wz ,
with a certain mixing ratio (1 − α) : α. The coefficient α
determines how much the reservoir is relying on the external
input u(n) and how much on the internal self-prediction z(n).
With α = 0 we have the classical ESN and with α = 1 we
have an “autistic” reservoir that does not react to the input.
Intermediate values of α close to 1 were shown to enable
reservoirs to generate slow, highly nonlinear signals that are
hard to get otherwise.
An algebraic unsupervised way of generating ESN
reservoirs was proposed in [119]. The idea is to linearize
the ESN update equation (5) locally around its current state
x(n) at every time step n to get a linear approximation
of (5) as x(n + 1) = Ax(n) + Bu(n), where A and B are
time (n)-dependent matrices corresponding to W and Win
respectively. The approach aims at distributing the predefined
complex eigenvalues of A uniformly within the unit circle on
the C plane. The reservoir matrix W is obtained analytically
 COMPUTER SCIENCE REVIEW
from the set of these predefined eigenvalues and a given input
u(n). The motivation for this is, as for Kautz filters [120] in
linear systems, that if the target ytarget (n) is unknown, it is
best to have something like an orthogonal basis in x(n), from
which any ytarget (n) could, on average, be constructed well.
The spectral radius of the reservoir is suggested to be set
by hand (according to the correlation time of u(n), which is
an indication of a memory span needed for the task), or by
adapting the bias value of the reservoir units to minimize the
output error (which actually renders this method supervised,
as in Section 7). Reservoirs generated this way are shown
to yield higher average entropy of x(n) distribution, higher
short-term memory capacity (both measures mentioned in
Section 6.1), and a smaller output error on a number of
synthetic problems, using relatively small reservoirs (Nx =
20, 30). However, a more extensive empirical comparison of
this type of reservoir with the classical ESN one is still lacking.
7. Supervised reservoir pre-training
In this section we discuss methods for training reservoirs
to perform a specific given task, i.e., not only the concrete
input u(n), but also the desired output ytarget (n) is taken into
account. Since a linear readout from a reservoir is quickly
trained, the suitability of a candidate reservoir for a particular
task (e.g., in terms of NRMSE (1)) is inexpensive to check.
Notice that even for most methods of this class the explicit
target signal ytarget (n) is not technically required for training
the reservoir itself, but only for evaluating it in an outer loop
of the adaptation process.
7.1. Optimization of global reservoir parameters
In Section 5.1 we discussed guidelines for the manual choice
of global parameters for reservoirs of ESNs. This approach
works well only with experience and a good intuitive grasp on
nonlinear dynamics. A systematic gradient descent method
of optimizing the global parameters of LI-ESNs (recalled
from Section 5.5) to fit them to a given task is presented
in [18]. The investigation shows that the error surfaces in the
combined global parameter and Wout spaces may have very
high curvature and multiple local minima. Thus, gradient
descent methods are not always practical.
7.2. Evolutionary methods
As one can see from the previous sections of this
review, optimizing reservoirs is generally challenging, and
breakthrough methods remain to be found. On the other
hand, checking the performance of a resulting ESN is
relatively inexpensive, as said. This brings in evolutionary
methods for the reservoir pre-training as a natural strategy.
Recall that the classical method generates a reservoir
randomly; thus the performance of the resulting ESN varies
slightly (and for small reservoirs not so slightly) from one
instance to another. Then indeed, an “evolutionary” method
as naive as “generate k reservoirs, pick the best” will
outperform the classical method (“generate a reservoir”) with
3 (2009) 127–149 139
probability (k − 1)/k, even though the improvement might be
not striking.
Several evolutionary approaches on optimizing reservoirs
of ESNs are presented in [121]. The first approach was to
carry out an evolutionary search on the parameters for
generating W: Nx , ρ(W), and the connection density of W.
Then an evolutionary algorithm [122] was used on individuals
consisting of all the weight matrices (Win , W, Wofb ) of small
(Nx = 5) reservoirs. A variant with a reduced search space was
also tried where the weights, but not the topology, of W were
explored, i.e., elements of W that were zero initially always
stayed zero. The empirical results of modeling the motion of
an underwater robot showed superiority of the methods over
other state-of-art methods, and that the topology-restricted
adaptation of W is almost as effective as the full one.
Another approach of optimizing the reservoir W by a
greedy evolutionary search is presented in [75]. Here the same
idea of separating the topology and weight sizes of W to
reduce the search space was independently used, but the
search was, conversely, restricted to the connection topology.
This approach also was demonstrated to yield on average
50% smaller (and much more stable) error in predicting the
behavior of a mass–spring–damper system with small (Nx =
20) reservoirs than without the genetic optimization.
Yet another way of reducing the search space of the
reservoir parameters is constructing a big reservoir weight
matrix W in a fractal fashion by repeatedly applying Kronecker
self-multiplication to an initial small matrix, called the
Kronecker kernel [123]. This contribution showed that among
Ws constructed in this way some yield ESN performance
similar to the best unconstrained Ws; thus only the good
weights of the small Kronecker kernel need to be found
by evolutionary search for producing a well-performing
reservoir.
Evolino [46], introduced in Section 3.3, is another example
of adapting a reservoir (in this case an LSTM network) using a
genetic search.
It has been recently demonstrated that by adapting only
the slopes of the reservoir unit activation functions f (·) by a
state-of-art evolutionary algorithm, and having Wout random
and fixed, a prediction performance of an ESN can be achieved
close to the best of classical ESNs [68].
In addition to (or instead of) adapting the reservoirs,
an evolutionary search can also be applied in training the
readouts, such as readouts with no explicit ytarget (n), as
discussed in Section 8.4.
7.3. Other types of supervised reservoir tuning
A greedy pruning of neurons from a big reservoir has been
shown in a recent initial attempt [124] to often give a (bit)
better classification performance for the same final Nx than
just a randomly created reservoir of the same size. The effect
of neuron removal to the reservoir dynamics, however, has
not been addressed yet.
140 COMPUTER SCIENCE REVIEW
7.4. Trained auxiliary feedbacks
While reservoirs have a natural capability of performing com-
plex real-time analog computations with fading memory [11],
an analytical investigation has shown that they can ap-
proximate any k-order differential equation (with persistent
memory) if extended with k trained feedbacks [21,125]. This
is equivalent to simulating any Turing machine, and thus
also means universal digital computing. In the presence of
noise (or finite precision) the memory becomes limited in
such models, but they still can simulate Turing machines with
finite tapes.
This theory has direct implications for reservoir comput-
ing; thus different ideas on how the power of ESNs could
be improved along its lines are explored in [78]. It is done
by defining auxiliary targets, training additional outputs of
ESNs on these targets, and feeding the outputs back to the
reservoir. Note that this can be implemented in the usual
model with feedback connections (6) by extending the orig-
inal output y(n) with additional dimensions that are trained
before training the original (final) output. The auxiliary tar-
gets are constructed from ytarget (n) and/or u(n) or some ad-
ditional knowledge of the modeled process. The intuition is
that the feedbacks could shift the internal dynamics of x(n) in
the directions that would make them better linearly combin-
able into ytarget (n). The investigation showed that for some
types of tasks there are natural candidates for such auxiliary
targets, which improve the performance significantly.
Unfortunately, no universally applicable methods for produc-
ing auxiliary targets are known such that the targets would be
both easy to learn and improve the accuracy of the final out-
put y(n). In addition, training multiple outputs with feedback
connections Wofb makes the whole procedure more compli-
cated, as cyclical dependences between the trained outputs
(one must take care of the order in which the outputs are
trained) as well as stability issues discussed in Section 8.2
arise. Despite these obstacles, we perceive this line of re-
search as having a big potential.
7.5. Reinforcement learning
In the line of biologically inspired local unsupervised adapta-
tion methods discussed in Section 6.2, an STDP modulated
by a reinforcement signal has recently emerged as a pow-
erful learning mechanism, capable of explaining some fa-
mous findings in neuroscience (biofeedback in monkeys), as
demonstrated in [126,127] and references thereof. The learn-
ing mechanism is also well biologically motivated as it uses
a local unsupervised STDP rule and a reinforcement (i.e.,
reward) feedback, which is present in biological brains in a
form of chemical signaling, e.g., by the level of dopamine. In
the RC framework this learning rule has been successfully ap-
plied for training readouts from the reservoirs so far in [127],
but could in principle be applied inside the reservoir too.
Overall the authors of this review believe that reinforce-
ment learning methods are natural candidates for reservoir
adaptation, as they can immediately exploit the knowledge
of how well the output is learned inside the reservoir without
the problems of error backpropagation. They can also be used
in settings where no explicit target ytarget (n) is available. We
expect to see more applications of reinforcement learning in
reservoir computing in the future.
3 (2009) 127–149
8. Readouts from the reservoirs
Conceptually, training a readout from a reservoir is a common
supervised non-temporal task of mapping x(n) to ytarget (n).
This is a well investigated domain in machine learning, much
more so than learning temporal mappings with memory. A
large choice of methods is available, and in principle any of
them can be applied. Thus we will only briefly go through the
ones reported to be successful in the literature.
8.1. Single-layer readout
By far the most popular readout method from the ESN
reservoirs is the originally proposed [12] simple linear
readout, as in (3) (we will consider it as equivalent to (8),
i.e., u(n) being part of x(n)). It is shown to be often sufficient,
as reservoirs provide a rich enough pool of signals for
solving many application-relevant and benchmark tasks, and
is very efficient to train, since optimal solutions can be found
analytically.
8.1.1. Linear regression
In batch mode, learning of the output weights Wout (2) can be
phrased as solving a system of linear equations
Wout X = Ytarget (12)
with respect to Wout , where X ∈ RN×T are all x(n) produced
by presenting the reservoir with u(n), and Ytarget ∈ RNy ×T
are all ytarget (n), both collected into respective matrices over
the training period n = 1, . . . , T. Usually x(n) data from the
beginning of the training run are discarded (they come before
n = 1), since they are contaminated by initial transients.
Since typically the goal is minimizing a quadratic error
E(Ytarget , Wout X) as in (1) and T > N, to solve (12) one
usually employs methods for finding least square solutions
of overdetermined systems of linear equations (e.g., [128]), the
problem also known as linear regression. One direct method
is calculating the Moore–Penrose pseudoinverse X+ of X, and
Wout as
Wout = Ytarget X+ . (13)
Direct pseudoinverse calculations exhibit high numerical
stability, but are expensive memory-wise for large state-
collecting matrices X ∈ RN×T , thereby limiting the size of the
reservoir N and/or the number of training samples T.
This issue is resolved in the normal equations formulation
of the problem5 :
T T
Wout XX = Ytarget X . (14)
A naive solution of it would be
T T
Wout = Ytarget X (XX )−1 . (15)
T T
Note that in this case Ytarget X ∈ RNy ×N and XX ∈ RN×N
do not depend on the length T of the training sequence,
and can be calculated incrementally while the training data
are passed through the reservoir. Thus, having these two
5 Note that our matrices are transposed compared to the
conventional notation.
 COMPUTER SCIENCE REVIEW
matrices collected, the solution complexity of (15) does not
depend on T either in time or in space. Also, intermediate
values of Wout can be calculated in the middle of running
through the training data, e.g., for an early assessment of the
performance, making this a “semi-online” training method.
The method (15) has lower numerical stability, compared
to (13), but the problem can be mitigated by using the
T T
pseudoinverse (XX )+ instead of the real inverse (XX )−1
(which usually also works faster). In addition, this method
enables one to introduce ridge, or Tikhonov, regularization
elegantly:
T T
Wout = Ytarget X (XX + α2 I)−1 , (16)
where I ∈ RN×N is the identity matrix and α is a regularization
factor. In addition to improving the numerical stability, the
regularization in effect reduces the magnitudes of entries
in Wout , thus mitigating sensitivity to noise and overfitting;
see Section 8.2 for more details. All this makes (16) a
highly recommendable choice for learning outputs from the
reservoirs.
Another alternative for solving (14) is decomposing the
T
matrix XX into a product of two triangular matrices via
Cholesky or LU decomposition, and solving (14) by two steps
of substitution, avoiding (pseudo-)inverses completely. The
Cholesky decomposition is the more numerically stable of the
two.
Weighted regression can be used for training linear
readouts by multiplying both x(n) and the corresponding
ytarget (n) by different weights over time, thus emphasizing
some time steps n over others. Multiplying certain recorded
√
x(n) and corresponding ytarget (n) by k has the same
emphasizing effect as if they appeared in the training
sequence k times.
When the reservoir is made from spiking neurons and thus
x(n) becomes a collection of spike trains, smoothing by low-
pass filtering may be applied to it before doing the linear
regression, or it can be done directly on x(n) [11]. For more
on linear regression based on spike train data, see [129].
Evolutionary search for training linear readouts can
also be employed. State-of-art evolutionary methods are
demonstrated to be able to achieve the same record levels of
precision for supervised tasks as with the best applications
of linear regression in ESN training [68]. Their much higher
computational cost is justifiable in settings where no explicit
ytarget (n) is available, discussed in Section 8.4.
8.1.2. Online adaptive output weight training
Some applications require online model adaptation, e.g., in
online adaptive channel equalization [17]. In such cases one
typically minimizes an error that is exponentially discounted
going back in time. Wout here acts as an adaptive linear
combiner. The simplest way to train Wout is to use stochastic
gradient descent. The method is familiar as the Least Mean
Squares (LMS) algorithm in linear signal processing [91], and
has many extensions and modifications. Its convergence
performance is unfortunately severely impaired by large
T consider the feedback loop connections Wout and Wofb as
eigenvalue spreads of XX , as mentioned in Section 6.1.
An alternative to LMS, known in linear signal processing as
the Recursive Least Squares (RLS) algorithm, is insensitive to the
3 (2009) 127–149 141
detrimental effects of eigenvalue spread and boasts a much
faster convergence because it is a second-order method. The
downside is that RLS is computationally more expensive
(order O(N 2 ) per time step instead of O(N) for LMS, for Ny = 1)
and notorious for numerical stability issues. Demonstrations
of RLS are presented in [17,43]. A careful and comprehensive
comparison of variants of RLS is carried out in a Master’s
thesis [130], which we mention here because it will be helpful
for practitioners.
The BackPropagation-DeCorrelation (BPDC) algorithm dis-
cussed in Section 3.4 is another powerful method for online
training of single-layer readouts with feedback connections
from the reservoirs.
Simple forms of adaptive online learning, such as LMS,
are also more biologically plausible than batch-mode training.
From spiking neurons a firing time-coded (instead of a
more common firing rate-coded) output for classification
can also be trained by only adapting the delays of the
output connections [101]. And firing rate-coded readouts
can be trained by a biologically-realistic reward-modulated
STDP [127], mentioned in Section 6.2.
8.1.3. SVM-style readout
Continuing the analogy between the temporal and non-
temporal expansion methods, discussed in Section 2, the
reservoir can be considered a temporal kernel, and the
standard linear readout Wout from it can be trained
using the same loss functions and regularizations as in
Support Vector Machines (SVMs) or Support Vector Regression
(SVR). Different versions of this approach are proposed and
investigated in [131].
A standard SVM (having its own kernel) can also be used as
a readout from a continuous-value reservoir [132]. Similarly,
special kernel types could be applied in reading out from
spiking (LSM-type) reservoirs [133] (and references therein).
8.2. Feedbacks and stability issues
Stability issues (with reservoirs having the echo state
property) usually only occur in generative setups where a
model trained on (one step) signal prediction is later run
in a generative mode, looping its output y(n) back into the
input as u(n + 1). Note that this is equivalent to a model with
output feedbacks Wofb (6) and no input at all (Nu = 0), which
is usually trained using teacher forcing (i.e., feeding ytarget (n)
as y(n) for the feedbacks during the training run) and later
is run freely to generate signals as y(n). Win in the first
case is equivalent to Wofb in the second one. Models having
feedbacks Wofb may also suffer from instability while driven
with external input u(n), i.e., not in a purely generative mode.
The reason for these instabilities is that even if the model
can predict the signal quite accurately, going through the
feedback loop of connections Wout and Wofb (or Win ) small
errors get amplified, making y(n) diverge from the intended
ytarget (n).
One way to look at this for trained linear outputs is to
part of the reservoir W. Putting (6) and (2) together we get
x(n) = f (Win u(n) + [W + Wofb Wout ]x(n − 1)), (17)
142 COMPUTER SCIENCE REVIEW
where W + Wofb Wout forms the “extended reservoir”
connections, which we will call W ∗ for brevity (as in [78]
Section 3.2). If the spectral radius of the extended reservoir
ρ(W ∗ ) is very large we can expect unstable behavior. A
more detailed analysis using Laplace transformations and a
sufficient condition for stability is presented in [134]. On the
other hand, for purely generative tasks, ρ(W ∗ ) < 1 would
mean that the generated signal would die out, which is not
desirable in most cases. Thus producing a generator with
stable dynamics is often not trivial.
Quite generally, models trained with clean (noise-free)
data for the best one-time-step prediction diverge fast in
the generative mode, as they are too “sharp” and not noise-
robust. A classical remedy is adding some noise to reservoir
states x(n) [12] during the training. This way the generator
forms a stable attractor by learning how to come to the
desired next output ytarget (n) from a neighborhood of the
current state x(n), having seen it perturbed by noise during
training. Setting the right amount of noise is a delicate
balance between the sharpness (of the prediction) and the
stability of the generator. Alternatively, adding noise to x(n)
can be seen as a form of regularization in training, as it in
T
effect also emphasizes the diagonal of matrix XX in (16). A
similar effect can be achieved using ridge regression (16) [135],
or to some extent even pruning of Wout [136]. Ridge regression
(16) is the least computationally expensive to do of the three,
since the reservoir does not need to be rerun with the data to
test different values of the regularization factor α.
Using different modifications of signals for teacher forcing,
like mixing ytarget (n) with noise, or in some cases using pure
strong noise, during the training also has an effect on the final
performance and stability, as discussed in Section 5.4 of [78].
8.3. Readouts for classification/recognition
The time series classification or temporal pattern detection
tasks that need a category indicator (as opposed to real
values) as an output can be implemented in two main ways.
The most common and straightforward way is having a
real-valued output for each class (or a single output and
a threshold for the two-class classifier), and interpreting
the strengths of the outputs as votes for the corresponding
classes, or even class probabilities (several options are
discussed in [18]). Often the most probable class is taken as
the decision. A simple target ytarget for this approach is a
constant ytarget i (n) = 1 signal for the right class i and 0 for
the others in the range of n where the indicating output is
expected. More elaborate shapes of ytarget (n) can improve
classification performance, depending on the task (e.g., [81]).
With spiking neurons the direct classification based on time
coding can be learned and done, e.g., the class is assigned
depending on which output fires first [101].
The main alternative to direct class indications is to
use predictive classifiers, i.e., train different predictors to
predict different classes and assign a class to a new example
corresponding to the predictor that predicts it best. Here
the quality of each predictor serves as the output strength
for the corresponding class. The method is quite popular in
automated speech recognition (e.g., Section 6 in [137] for an
overview). However, in Section 6.5 of [137] the author argues
3 (2009) 127–149
against this approach, at least in its straightforward form,
pointing out some weaknesses, like the lack of specificity, and
negative practical experience.
For both approaches a weighting scheme can be used for
both training (like in weighted regression) and integrating the
class votes, e.g., putting more emphasis on the end of the
pattern when sufficient information has reached the classifier
to make the decision.
An advanced version of ESN-based predictive classifier,
where for each class there is a set of competitively trained
predictors and dynamic programming is used to find the
optimal sequence of them, is reported to be much more noise
robust than a standard Hidden Markov Model in spoken word
recognition [138].
8.4. Readouts beyond supervised learning
Even though most of the readout types from reservoirs
reported in the literature are trained in a purely supervised
manner, i.e., making y(n) match an explicitly given ytarget (n),
the reservoir computing paradigm lends itself to settings
where no ytarget (n) is available. A typical such setting is
reinforcement learning where only a feedback on the model’s
performance is available. Note that an explicit ytarget (n) is not
required for the reservoir adaptation methods discussed in
Sections 5 and 6 of this survey by definition. Even most of the
adaptation methods classified as supervised in Section 7 do
not need an explicit ytarget (n), as long as one can evaluate
the performance of the reservoir. Thus they can be used
without modification, provided that unsupervised training
and evaluation of the output is not prohibitively expensive
or can be done simultaneously with reservoir adaptation. In
this section we will give some pointers on training readouts
using reinforcement learning.
A biologically inspired learning rule of Spike-Time-
Dependent Plasticity (STDP) modulated by a reinforcement
signal has been successfully applied for training a readout
of firing neurons from the reservoirs of the same LSTM-type
in [127].
Evolutionary algorithms are a natural candidate for
training outputs in a non-supervised manner. Using a genetic
search with crossover and mutation to find optimal output
weights Wout of an ESN is reported in [139]. Such an ESN
is successfully applied for a hard reinforcement learning
task of direct adaptive control, replacing a classical indirect
controller.
ESNs trained with a simple “(1 + 1)” evolution strategy for
an unsupervised artificial embryogeny (the so-called “flag”)
problem are shown to perform very well in [140].
An ESN trained with a state-of-art evolutionary continuous
parameter optimization method (CMA-ES) shows comparable
performance in a benchmark double pole balancing problem
to the best RNN topology-learning methods in [68,141].
For this problem the best results are obtained when the
spectral radius ρ(W) is adapted together with Wout . The
same contributions also validate the CMA-ES readout training
method on a standard supervised prediction task, achieving
the same excellent precision (MSE of the order 10−15 ) as
the state-of-art with linear regression. Conversely, the best
results for this task were achieved with ρ(W) fixed and
 COMPUTER SCIENCE REVIEW
training only Wout . An even more curious finding is that
almost as good results were achieved by only adapting slopes
of the reservoir activation functions f (·) and having Wout
fixed, as mentioned in Section 7.2.
8.5. Multilayer readouts
Multilayer perceptrons (MLPs) as readouts, trained by error
backpropagation, were used from the very beginnings of
LSMs [11] and ESNs (unpublished). They are theoretically
more powerful and expressive in their instantaneous
mappings from x(n) to y(n) than linear readouts, and are
thus suitable for particularly nonlinear outputs, e.g., in [142,
143]. In both cases the MLP readouts are trained by error
backpropagation. On the other hand they are significantly
harder to train than an optimal single-layer linear regression,
thus often giving inferior results compared to the latter in
practice.
Some experience in training MLPs as ESN readouts,
including network initialization, using stochastic, batch, and
semi-batch gradients, adapting learning rates, and combining
with regression-training of the last layer of the MLP, is
presented in Section 5.3 of [78].
8.6. Readouts with delays
While the readouts from reservoirs are usually recurrence-
free, this does not mean that they may not have memory. In
some approaches they do, or rather some memory is inserted
between the reservoir and the readout.
Learning a delay for each neuron in an ESN reservoir x(n)
in addition to the output weight from it is investigated in [84].
Cross-correlation (simple or generalized) is used to optimally
align activations of each neuron in x(n) with ytarget (n), and
then activations with the delays xdelayed (n) are used to find
Wout in a usual way. This approach potentially enables
utilizing the computational power of the reservoir more
efficiently. In a time-coded output from a spiking reservoir
the output connection delays can actually be the only thing
that is learned [101].
For time series classification tasks the decision can be
based on a readout from a joined reservoir state xjoined =
[x(n1 ), x(n2 ), . . . , x(nk )] that is a concatenation of the reservoir
states from different moments n1 , n2 , . . . , nk in time during
the time series [18]. This approach, compared to only using
the last state of the given time series, moves the emphasis
away from the ending of the series, depending on how the
support times ni are spread. It is also more expressive, since
it has k times more trainable parameters in Wout for the same
size of the reservoir N. As a consequence, it is also more prone
to overfitting. It is also possible to integrate intervals of states
Pn1
in some way, e.g., use x∗ (n1 ) = n −n1 +1 m=n 0
x(m) instead
1 0
of using a single snapshot of the states x(n1 ).
An approach of treating a finite history of reservoir
activations x(n) (similar to X in (12)) as a two-dimensional
image, and training a minimum average correlations energy
filter as the readout for dynamical pattern recognition is
presented in [144].
Even though in Section 1 we stated that the RNNs
considered in this survey are used as nonlinear filters,
3 (2009) 127–149 143
which transform an input time series into an output time
series, ESNs can also be utilized for non-temporal (defined
in Section 2.1) tasks {(u(n), ytarget (n))} by presenting an ESN
with the same input u(n) for many time steps, letting the
ESN converge to a fixed-point attractor state xu(n) (∞) (which
it does if it possesses echo state property) and reading the
output from the attractor state y(n) = y(xu(n) (∞)) [145,146].
8.7. Combining several readouts
Segmenting of the spatially embedded trajectory of x(n) by
k-means clustering and assigning a separate “responsible”
linear readout for each cluster is investigated in [147]. This
approach increases the expressiveness of the ESN by having k
linear readouts trained and an online switching mechanism
among them. Bigger values of k are shown to compensate
for smaller sizes Nx of the reservoirs to get the same level
of performance.
A benchmark-record-breaking approach of taking an
average of outputs from many (1000) different instances of
tiny (N = 4) trained ESNs is presented in Section 5.2.2 of [18].
The approach is also combined with reading from different
support times as discussed in Section 8.6 of this survey.
Averaging outputs over 20 instances of ESNs was also shown
to refine the prediction of chaotic time series in supporting
online material of [17].
Using dynamic programing to find sequences in multiple
sets of predicting readouts for classification [138] was already
mentioned at the end of Section 8.3.
8.8. Hierarchies
Following the analogy between the ESNs and non-temporal
kernel methods, ESNs would be called “type-1 shallow ar-
chitectures” according to the classification proposed in [148].
The reservoir adaptation techniques reviewed in our article
would make ESNs “type-3 shallow architectures”, which are
more expressive. However, the authors in [148] argue that any
type of shallow (i.e., non-hierarchical) architectures is inca-
pable of learning really complex intelligent tasks. This sug-
gests that for demandingly complex tasks the adaptation of
a single reservoir might not be enough and a hierarchical ar-
chitecture of ESNs might be needed, e.g., such as presented
in [149]. Here the outputs of a higher level in the hierarchy
serve as coefficients of mixing (or voting on) outputs from
a lower one. The structure can have an arbitrary number of
layers. Only the outputs from the reservoirs of each layer are
trained simultaneously, using stochastic gradient descent and
error backpropagation through the layers. The structure is
demonstrated to discover features on different timescales in
an unsupervised way when being trained for predicting a syn-
thetic time series of interchanging generators. On the down-
side, such hierarchies require many epochs to train, and suf-
fer from a similar problem of vanishing gradients, as deep
feedforward neural networks or gradient-descent methods
for fully trained RNNs. They also do not scale-up yet to real-
world demanding problems. Research on hierarchically struc-
tured RC models has only just begun.
144 COMPUTER SCIENCE REVIEW
9. Discussion
The striking success of the original RC methods in
outperforming fully trained RNNs in many (though not all)
tasks, established an important milestone, or even a turning
point, in the research of RNN training. The fact that a
randomly generated fixed RNN with only a linear readout
trained consistently outperforms state-of-art RNN training
methods had several consequences:
• First of all it revealed that we do not really know how
to train RNNs well, and something new is needed. The
error backpropagation methods, which had caused a
breakthrough in feedforward neural network training (up
to a certain depth), and had also become the most popular
training methods for RNNs, are hardly unleashing their full
potential.
• Neither are the classical RC methods yet exploiting the full
potential of RNNs, since they use a random RNN, which is
unlikely to be optimal, and a linear readout, which is quite
limited by the quality of the signals it is combining. But
they give a quite tough performance reference for more
sophisticated methods.
• The separation between the RNN reservoir and the readout
provides a good platform to try out all kinds of RNN
adaptation methods in the reservoir and see how much
they can actually improve the performance over randomly
created RNNs. This is particularly well suited for testing
various biology-inspired RNN adaptation mechanisms,
which are almost exclusively local and unsupervised, in
how they can improve learning of a supervised task.
• In parallel, it enables all types of powerful non-temporal
methods to be applied for reading out of the reservoir.
This platform is the current paradigm of RC: using different
methods for (i) producing/adapting the reservoir, and (ii)
training different types of readouts. It enables looking for
good (i) and (ii) methods independently, and combining the
best practices from both research directions. The platform
has been actively used by many researchers, ever since the
first ESNs and LSMs appeared. This research in both (i) and
(ii) directions, together with theoretical insights, like what
characterizes a “good” reservoir, constitutes the modern field
of RC.
In this review, together with motivating the new paradigm,
we have provided a comprehensive survey of all this RC
research. We introduced a natural taxonomy of the reservoir
generation/adaptation techniques (i) with three big classes of
methods (generic, unsupervised, and supervised), depending
on their universality with respect to the input and desired
output of the task. Inside each class, methods are also
grouped into major directions of approaches, taking different
inspirations. We also surveyed all types of readouts from
the reservoirs (ii) reported in the literature, including the
ones containing several layers of nonlinearities, combining
several time steps, or several reservoirs, among others. We
also briefly discussed some practical issues of training the
most popular types of readouts in a tutorial-like fashion.
The survey is transcending the boundaries among several
traditional methods that fall under the umbrella of RC,
generalizing the results to the whole RC field and pointing
out relations, where applicable.
3 (2009) 127–149
Even though this review is quite extensive, we tried
to keep it concise, outlining only the basic ideas of each
contribution. We did not try to include every contribution
relating to RC in this survey, but only the ones highlighting
the main research directions. Publications only reporting
applications of reservoir methods, but not proposing any
interesting modifications of them, were left out. Since this
review is aimed at a (fast) moving target, which RC is, some
(especially very new) contributions might have been missed
unintentionally.
In general, the RC field is still very young, but very active
and quickly expanding. While the original first RC methods
made an impact that could be called a small revolution,
current RC research is more in a phase of a (rapid) evolution.
The multiple new modifications of the original idea are
gradually increasing the performance of the methods. While
with no striking breakthroughs lately, the progress is steady,
establishing some of the extensions as common practices to
build on further. There are still many promising directions to
be explored, hopefully leading to breakthroughs in the near
future.
While the tasks for which RNNs are applied nowadays
often are quite complex, hardly any of them could yet be
called truly intelligent, as compared to the human level of
intelligence. The fact that RC methods perform well in many
of these simple tasks by no means indicates that there is
little space left for their improvement. More complex tasks
and adequate solutions are still to meet each other in RC. We
further provide some of our (subjective, or even speculative)
outlooks on the future of RC.
The elegant simplicity of the classical ESNs gives many
benefits in these simple applications, but it also has some
intrinsic limitations (as, for example, discussed in Section 5.4)
that must be overcome in some way or other. Since the RNN
model is by itself biologically inspired, looking at real brains
is a natural (literally) source of inspiration on how to do
that. RC models may reasonably explain some aspects of how
small portions of the brain work, but if we look at the bigger
picture, the brain is far from being just a big blob of randomly
connected neurons. It has a complex structure that is largely
predefined before even starting to learn. In addition, there
are many learning mechanisms observed in the real brain, as
briefly outlined in Section 6.2. It is very probable that there
is no single easily implementable underlying rule which can
explain all learning.
The required complexity in the context of RC can be
achieved in two basic ways: either (i) by giving the reservoir
a more complex internal structure, like that discussed in
Section 5.3 or (ii) externally building structures combining
several reservoirs and readouts, like those discussed in
Section 8.8. Note that the two ways correspond to the
above-mentioned dichotomy of the RC research and are not
mutually exclusive. An “externally” (ii) built structure can also
be regarded as a single complex reservoir (i) and a readout
from it all can be trained.
An internal auto-structuring of the reservoir (i) through an
(unsupervised) training would be conceptually appealing and
nature-like, but not yet quite feasible at the current state of
knowledge. A robust realization of such a learning algorithm
would signify a breakthrough in the generation/training
 COMPUTER SCIENCE REVIEW
of artificial NNs. Most probably such an approach would
combine several competing learning mechanisms and goals,
and require a careful parameter selection to balance them,
and thus would not be easy to successfully apply. In addition,
changing the structure of the RNN during the adaptive
training would lead to bifurcations in the training process, as
in [8], which makes learning very difficult.
Constructing external architectures or several reservoirs
can be approached as more of an engineering task. The
structures can be hand-crafted, based on the specifics of the
application, and, in some cases, trained entirely supervised,
each reservoir having a predefined function and a target
signal for its readout. While such approaches are successfully
being applied in practice, they are very case-specific, and not
quite in the scope of the research reviewed here, since in
essence they are just applications of (several instances of) the
classical RC methods in bigger settings.
However, generic structures of multiple reservoirs (ii) that
can be trained with no additional information, such as
discussed in Section 8.8, are of high interest. Despite their
current state being still an “embryo”, and the difficulties
pointed out earlier, the authors of this review see this
direction as highly promising.
Biological inspiration and progress of neuroscience in
understanding how real brains work are beneficial for both
(i) and (ii) approaches. Well understood natural principles of
local neural adaptation and development can be relatively
easily transfered to artificial reservoirs (i), and reservoirs
internally structured to more closely resemble cortical
microcolumns in the brain have been shown to perform
better [23]. Understanding how different brain areas interact
could also help in building external structures of reservoirs
(ii) better suited for nature-like tasks.
In addition to processing and “understanding” multiple
scales of time and abstraction in the data, which hierarchical
models promise to solve, other features still lacking in the
current RC (and overall RNN) methods include robustness
and stability of pattern generation. A possible solution to
this could be a homeostasis-like self-regulation in the RNNs.
Other intelligence-tending features as selective longer-term
memory or active attention are also not yet well incorporated.
In short, RC is not the end, but an important stepping-
stone in the big journey of developing RNNs, ultimately
leading towards building artificial and comprehending
natural intelligence.
Acknowledgments
This work is partially supported by Planet Intelligent Systems
GmbH, a private company with an inspiring interest in
fundamental research. The authors are also thankful to
Benjamin Schrauwen, Michael Thon, and an anonymous
reviewer of this journal for their helpful constructive
feedback.
REFERENCES
[1] John J. Hopfield, Hopfield network, Scholarpedia 2 (5) (2007)
1977.
3 (2009) 127–149 145
[2] John J. Hopfield, Neural networks and physical systems with
emergent collective computational abilities, Proceedings of
the National Academy of Sciences of the United States of
America 79 (1982) 2554–2558.
[3] Geoffrey E. Hinton, Boltzmann machine, Scholarpedia 2 (5)
(2007) 1668.
[4] David H. Ackley, Geoffrey E. Hinton, Terrence J. Sejnowski,
A learning algorithm for Boltzmann machines, Cognitive
Science 9 (1985) 147–169.
[5] Geoffrey E. Hinton, Ruslan Salakhutdinov, Reducing the
dimensionality of data with neural networks, Science 313
(5786) (2006) 504–507.
[6] Graham W. Taylor, Geoffrey E. Hinton, Sam Roweis,
Modeling human motion using binary latent variables,
in: Advances in Neural Information Processing Systems 19,
NIPS 2006, MIT Press, Cambridge, MA, 2007, pp. 1345–1352.
[7] Ken-ichi Funahashi, Yuichi Nakamura, Approximation of
dynamical systems by continuous time recurrent neural
networks, Neural Networks 6 (1993) 801–806.
[8] Kenji Doya, Bifurcations in the learning of recurrent neural
networks, in: Proceedings of IEEE International Symposium
on Circuits and Systems 1992, vol. 6, 1992, pp. 2777–2780.
[9] Yoshua Bengio, Patrice Simard, Paolo Frasconi, Learning
long-term dependencies with gradient descent is difficult,
IEEE Transactions on Neural Networks 5 (2) (1994) 157–166.
[10] Felix A. Gers, Jürgen Schmidhuber, Fred A. Cummins,
Learning to forget: Continual prediction with LSTM, Neural
Computation 12 (10) (2000) 2451–2471.
[11] Wolfgang Maass, Thomas Natschläger, Henry Markram,
Real-time computing without stable states: A new frame-
work for neural computation based on perturbations, Neu-
ral Computation 14 (11) (2002) 2531–2560.
[12] Herbert Jaeger, The “echo state” approach to analysing
and training recurrent neural networks, Technical Report
GMD Report 148, German National Research Center for
Information Technology, 2001.
[13] Peter F. Dominey, Complex sensory-motor sequence learn-
ing based on recurrent state representation and reinforce-
ment learning, Biological Cybernetics 73 (1995) 265–274.
[14] Jochen J. Steil, Backpropagation-decorrelation: Recurrent
learning with O(N) complexity, in: Proceedings of the IEEE
International Joint Conference on Neural Networks, 2004,
IJCNN 2004, vol. 2, 2004, pp. 843–848.
[15] Herbert Jaeger, Wolfgang Maass, José C. Príncipe, Special
issue on echo state networks and liquid state machines —
Editorial, Neural Networks 20 (3) (2007) 287–289.
[16] Herbert Jaeger, Echo state network, Scholarpedia 2 (9) (2007)
2330.
[17] Herbert Jaeger, Harald Haas, Harnessing nonlinearity:
Predicting chaotic systems and saving energy in wireless
communication, Science (2004) 78–80.
[18] Herbert Jaeger, Mantas Lukoševičius, Dan Popovici, Udo
Siewert, Optimization and applications of echo state
networks with leaky-integrator neurons, Neural Networks
20 (3) (2007) 335–352.
[19] David Verstraeten, Benjamin Schrauwen, Dirk Stroobandt,
Reservoir-based techniques for speech recognition, in:
Proceedings of the IEEE International Joint Conference on
Neural Networks, 2006, IJCNN 2006, 2006 pp. 1050–1053.
[20] Wolfgang Maass, Thomas Natschläger, Henry Markram, A
model for real-time computation in generic neural mi-
crocircuits, in: Advances in Neural Information Processing
Systems 15, NIPS 2002, MIT Press, Cambridge, MA, 2003,
pp. 213–220.
[21] Wolfgang Maass, Prashant Joshi, Eduardo D. Sontag,
Principles of real-time computing with feedback applied
to cortical microcircuit models, in: Advances in Neural
Information Processing Systems 18, MIT Press, Cambridge,
MA, 2006, pp. 835–842.
146 COMPUTER SCIENCE REVIEW
[22] Dean V. Buonomano, Michael M. Merzenich, Temporal
information transformed into a spatial code by a neural
network with realistic properties, Science 267 (1995)
1028–1030.
[23] Stefan Haeusler, Wolfgang Maass, A statistical analysis
of information-processing properties of lamina-specific
cortical microcircuit models, Cerebral Cortex 17 (1) (2007)
149–162.
[24] Uma R. Karmarkar, Dean V. Buonomano, Timing in the
absence of clocks: Encoding time in neural network states,
Neuron 53 (3) (2007) 427–438.
[25] Garrett B. Stanley, Fei F. Li, Yang Dan, Reconstruction of
natural scenes from ensemble responses in the lateral
genicualate nucleus, Journal of Neuroscience 19 (18) (1999)
8036–8042.
[26] Danko Nikolić, Stefan Haeusler, Wolf Singer, Wolfgang
Maass, Temporal dynamics of information content carried
by neurons in the primary visual cortex, in: Advances in
Neural Information Processing Systems 19, NIPS 2006, MIT
Press, Cambridge, MA, 2007, pp. 1041–1048.
[27] Werner M. Kistler, Chris I. De Zeeuw, Dynamical working
memory and timed responses: The role of reverberating
loops in the olivo-cerebellar system, Neural Computation 14
(2002) 2597–2626.
[28] Tadashi Yamazaki, Shigeru Tanaka, The cerebellum as a
liquid state machine, Neural Networks 20 (3) (2007) 290–297.
[29] Peter F. Dominey, Michel Hoen, Jean-Marc Blanc, Taïssia
Lelekov-Boissard, Neurological basis of language and
sequential cognition: Evidence from simulation, aphasia,
and ERP studies, Brain and Language 86 (2003) 207–225.
[30] Jean-Marc Blanc, Peter F. Dominey, Identification of prosodic
attitudes by atemporal recurrent network, Cognitive Brain
Research 17 (2003) 693–699.
[31] Peter F. Dominey, Michel Hoen, Toshio Inui, A neurolinguis-
tic model of grammatical construction processing, Journal
of Cognitive Neuroscience 18 (12) (2006) 2088–2107.
[32] Robert M. French, Catastrophic interference in connection-
ist networks, in: L. Nadel (Ed.), Encyclopedia of Cognitive Sci-
ence, Volume 1, Nature Publishing Group, 2003, pp. 431–435.
[33] Floris Takens, Detecting strange attractors in turbulence,
in: Proceedings of a Symposium on Dynamical Systems and
Turbulence, in: LNM, vol. 898, Springer, 1981, pp. 366–381.
[34] Ronald J. Williams, David Zipser, A learning algorithm for
continually running fully recurrent neural networks, Neural
Computation 1 (1989) 270–280.
[35] David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams,
Learning internal representations by error propagation,
in: Neurocomputing: Foundations of research, MIT Press,
Cambridge, MA, USA, 1988, pp. 673–695.
[36] Paul J. Werbos, Backpropagation through time: What it does
and how to do it, Proceedings of the IEEE 78 (10) (1990)
1550–1560.
[37] Amir F. Atiya, Alexander G. Parlos, New results on recurrent
network training: Unifying the algorithms and accelerating
convergence, IEEE Transactions on Neural Networks 11 (3)
(2000) 697–709.
[38] Gintaras V. Puškorius, Lee A. Feldkamp, Neurocontrol of
nonlinear dynamical systems with Kalman filter trained
recurrent networks, IEEE Transactions on Neural Networks
5 (2) (1994) 279–297.
[39] Sheng Ma, Chuanyi Ji, Fast training of recurrent networks
based on the EM algorithm, IEEE Transactions on Neural
Networks 9 (1) (1998) 11–26.
[40] Sepp Hochreiter, Jürgen Schmidhuber, Long short-term
memory, Neural Computation 9 (8) (1997) 1735–1780.
[41] Herbert Jaeger, Short term memory in echo state networks,
Technical Report GMD Report 152, German National
Research Center for Information Technology, 2002.
3 (2009) 127–149
[42] Herbert Jaeger, Tutorial on training recurrent neural
networks, covering BPTT, RTRL, EKF and the “echo
state network” approach, Technical Report GMD Report
159, German National Research Center for Information
Technology, 2002.
[43] Herbert Jaeger, Adaptive nonlinear system identification
with echo state networks, in: Advances in Neural Informa-
tion Processing Systems 15, MIT Press, Cambridge, MA, 2003,
pp. 593–600.
[44] Thomas Natschläger, Henry Markram, Wolfgang Maass,
Computer models and analysis tools for neural microcir-
cuits, in: R. Kötter (Ed.), A Practical Guide to Neuroscience
Databases and Associated Tools, Kluver Academic Publish-
ers, Boston, 2002 (chapter 9).
[45] Wolfgang Maass, Thomas Natschläger, Henry Markram,
Computational models for generic cortical microcircuits,
in: J. Feng (Ed.), Computational Neuroscience: A Compre-
hensive Approach, CRC-Press, 2002.
[46] Jürgen Schmidhuber, Daan Wierstra, Matteo Gagliolo,
Faustino J. Gomez, Training recurrent networks by Evolino,
Neural Computation 19 (3) (2007) 757–779.
[47] Ulf D. Schiller, Jochen J. Steil, Analyzing the weight dynam-
ics of recurrent learning algorithms, Neurocomputing 63C
(2005) 5–23.
[48] Jochen J. Steil, Memory in backpropagation-decorrelation
O(N) efficient online recurrent learning, in: Proceedings
of the 15th International Conference on Artificial Neural
Networks, in: LNCS, vol. 3697, Springer, 2005, pp. 649–654
(chapter 9).
[49] Felix R. Reinhart, Jochen J. Steil, Recurrent neural autoasso-
ciative learning of forward and inverse kinematics for move-
ment generation of the redundant PA-10 robot, in: Proceed-
ings of the ECSIS Symposium on Learning and Adaptive
Behaviors for Robotic Systems, LAB-RS, vol. 1, 2008, 35–40.
[50] Peter F. Dominey, Franck Ramus, Neural network processing
of natural language: I. Sensitivity to serial, temporal and
abstract structure of language in the infant, Language and
Cognitive Processes 15 (1) (2000) 87–127.
[51] Peter F. Dominey, From sensorimotor sequence to grammat-
ical construction: Evidence from simulation and neurophys-
iology, Adaptive Behaviour 13 (4) (2005) 347–361.
[52] Se Wang, Xiao-Jian Yang, Cheng-Jian Wei, Harnessing
non-linearity by sigmoid-wavelet hybrid echo state net-
works (SWHESN), in: The 6th World Congress on In-
telligent Control and Automation, WCICA 2006, 1, 2006,
pp. 3014–3018.
[53] David Verstraeten, Benjamin Schrauwen, Dirk Stroobandt,
Reservoir computing with stochastic bitstream neurons,
in: Proceedings of the 16th Annual ProRISC Workshop,
Veldhoven, The Netherlands, November 2005, pp. 454–459.
[54] Felix Schürmann, Karlheinz Meier, Johannes Schemmel,
Edge of chaos computation in mixed-mode VLSI - A
hard liquid, in: Advances in Neural Information Processing
Systems 17, MIT Press, Cambridge, MA, 2005, pp. 1201–1208.
[55] Benjamin Schrauwen, Marion Wardermann, David Ver-
straeten, Jochen J. Steil, Dirk Stroobandt, Improving reser-
voirs using intrinsic plasticity, Neurocomputing 71 (2008)
1159–1171.
[56] Kristof Vandoorne, Wouter Dierckx, Benjamin Schrauwen,
David Verstraeten, Roel Baets, Peter Bienstman, Jan Van
Campenhout, Toward optical signal processing using
photonic reservoir computing, Optics Express 16 (15) (2008)
11182–11192.
[57] Chrisantha Fernando, Sampsa Sojakka, Pattern recognition
in a bucket, in: Proceedings of the 7th European Conference
on Advances in Artificial Life, ECAL 2003, in: LNCS, vol. 2801,
Springer, 2003, pp. 588–597.
 COMPUTER SCIENCE REVIEW
[58] Ben Jones, Dov Stekelo, Jon Rowe, Chrisantha Fernando, Is
there a liquid state machine in the bacterium Escherichia
coli?, in: Proceedings of the 1st IEEE Symposium on Artificial
Life, ALIFE 2007, 1–5 April 2007, pp. 187–191.
[59] David Verstraeten, Benjamin Schrauwen, Michiel D’Haene,
Dirk Stroobandt, An experimental unification of reservoir
computing methods, Neural Networks 20 (3) (2007) 391–403.
[60] Benjamin Schrauwen, David Verstraeten, Jan Van Campen-
hout, An overview of reservoir computing: Theory, appli-
cations and implementations, in: Proceedings of the 15th
European Symposium on Artificial Neural Networks, ESANN
2007, 2007, pp. 471–482.
[61] Mantas Lukoševičius, Herbert Jaeger, Overview of reservoir
recipes, Technical Report No. 11, Jacobs University Bremen,
2007.
[62] David H. Wolpert, The supervised learning no-free-lunch
theorems, in: Proceedings of the 6th Online World
Conference on Soft Computing in Industrial Applications,
WSC 2006, 2001, pp. 25–42.
[63] Michael Buehner, Peter Young, A tighter bound for the echo
state property, IEEE Transactions on Neural Networks 17 (3)
(2006) 820–824.
[64] Duncan J. Watts, Steven H. Strogatz, Collective dynamics of
‘small-world’ networks, Nature 393 (1998) 440–442.
[65] Albert-Laszlo Barabasi, Reka Albert, Emergence of scaling in
random networks, Science 286 (1999) 509.
[66] Marcus Kaiser, Claus C. Hilgetag, Spatial growth of real-
world networks, Physical Review E 69 (2004) 036103.
[67] Benjamin Liebald, Exploration of effects of different net-
work topologies on the ESN signal crosscorrelation ma-
trix spectrum, Bachelor’s Thesis, Jacobs University Bre-
men, 2004 http://www.eecs.jacobs-university.de/archive/
bsc-2004/liebald.pdf.
[68] Fei Jiang, Hugues Berry, Marc Schoenauer, Supervised and
evolutionary learning of echo state networks., in: Proceed-
ings of 10th International Conference on Parallel Problem
Solving from Nature, PPSN 2008, in: LNCS, vol. 5199,
Springer, 2008, pp. 215–224.
[69] Wolfgang Maass, Thomas Natschläger, Henry Markram,
Computational models for generic cortical microcircuits,
in: Computational Neuroscience: A Comprehensive Ap-
proach, Chapman & Hall/CRC, 2004, pp. 575–605.
[70] David Verstraeten, Benjamin Schrauwen, Dirk Stroobandt,
Jan Van Campenhout, Isolated word recognition with the
liquid state machine: A case study, Information Processing
Letters 95 (6) (2005) 521–528.
[71] Michal Čerňanský, Matej Makula, Feed-forward echo state
networks, in: Proceedings of the IEEE International Joint
Conference on Neural Networks, 2005, IJCNN 2005, vol. 3,
2005, pp. 1479–1482.
[72] Georg Fette, Julian Eggert, Short term memory and pattern
matching with simple echo state network, in: Proceedings
of the 15th International Conference on Artificial Neural
Networks, ICANN 2005, in: LNCS, vol. 3696, Springer, 2005,
pp. 13–18.
[73] David Verstraeten, Benjamin Schrauwen, Dirk Stroobandt,
Adapting reservoirs to get Gaussian distributions, in:
Proceedings of the 15th European Symposium on Artificial
Neural Networks, ESANN 2007, 2007, pp. 495–500.
[74] Carlos Lourenço, Dynamical reservoir properties as network
effects, in: Proceedings of the 14th European Symposium on
Artificial Neural Networks, ESANN 2006, 2006, pp. 503–508.
[75] Keith Bush, Batsukh Tsendjav, Improving the richness of
echo state features using next ascent local search, in:
Proceedings of the Artificial Neural Networks In Engineering
Conference, St. Louis, MO, 2005, pp. 227–232.
3 (2009) 127–149 147
[76] Márton Albert Hajnal, András Lőrincz, Critical echo
state networks, in: Proceedings of the 16th International
Conference on Artificial Neural Networks, in: LNCS, vol.
4131, Springer, 2006, pp. 658–667.
[77] Yanbo Xue, Le Yang, Simon Haykin, Decoupled echo state
networks with lateral inhibition, Neural Networks 20 (3)
(2007) 365–376.
[78] Mantas Lukoševičius, Echo state networks with trained
feedbacks, Technical Report No. 4, Jacobs University
Bremen, 2007.
[79] Danil V. Prokhorov, Lee A. Feldkamp, Ivan Yu. Tyukin,
Adaptive behavior with fixed weights in RNN: An overview,
in: Proceedings of the IEEE International Joint Conference on
Neural Networks, 2002, IJCNN 2002, 2002, pp. 2018–2023.
[80] Mohamed Oubbati, Paul Levi, Michael Schanz, Meta-
learning for adaptive identification of non-linear dynamical
systems, in: Proceedings of the IEEE International Joint
Symposium on Intelligent Control, June 2005, pp. 473–478.
[81] Mantas Lukoševičius, Dan Popovici, Herbert Jaeger, Udo
Siewert, Time warping invariant echo state networks,
Technical Report No. 2, Jacobs University Bremen, 2006.
[82] Benjamin Schrauwen, Jeroen Defour, David Verstraeten, Jan
M. Van Campenhout, The introduction of time-scales in
reservoir computing, applied to isolated digits recognition,
in: Proceedings of the 17th International Conference on
Artificial Neural Networks, in: LNCS, vol. 4668, Springer,
2007, pp. 471–479.
[83] Udo Siewert, Welf Wustlich, Echo-state networks with band-
pass neurons: Towards generic time-scale-independent
reservoir structures, Internal Status Report, PLANET intel-
ligent systems GmbH, 2007. Available online at http://snn.
elis.ugent.be/.
[84] Georg Holzmann, Echo state networks with filter neurons
and a delay&sum readout, Internal Status Report, Graz
University of Technology, 2007. Available online at http://
grh.mur.at/data/misc.html.
[85] Francis wyffels, Benjamin Schrauwen, David Verstraeten,
Stroobandt Dirk, Band-pass reservoir computing, in: Z. Hou,
and N. Zhang (Eds.), Proceedings of the IEEE International
Joint Conference on Neural Networks, 2008, IJCNN 2008,
Hong Kong, 2008, pp. 3204–3209.
[86] Salah El Hihi, Yoshua Bengio, Hierarchical recurrent neural
networks for long-term dependencies, in: Advances in
Neural Information Processing Systems 8, MIT Press,
Cambridge, MA, 1996, pp. 493–499.
[87] Nils Bertschinger, Thomas Natschläger, Real-time compu-
tation at the edge of chaos in recurrent neural networks,
Neural Computation 16 (7) (2004) 1413–1436.
[88] Benjamin Schrauwen, Lars Buesing, Robert Legenstein, On
computational power and the order-chaos phase transition
in reservoir computing, in: Advances in Neural Information
Processing Systems 21, NIPS 2008, 2009, pp. 1425–1432.
[89] Wolfgang Maass, Robert A. Legenstein, Nils Bertschinger,
Methods for estimating the computational power and gen-
eralization capability of neural microcircuits, in: Advances
in Neural Information Processing Systems 17, NIPS 2004,
MIT Press, Cambridge, MA, 2005, pp. 865–872.
[90] Robert A. Legenstein, Wolfgang Maass, Edge of chaos and
prediction of computational performance for neural circuit
models, Neural Networks 20 (3) (2007) 323–334.
[91] Behrouz Farhang-Boroujeny, Adaptive Filters: Theory and
Applications, Wiley, 1998.
[92] Herbert Jaeger, Reservoir riddles: suggestions for echo state
network research, Proceedings of the IEEE International
Joint Conference on Neural Networks, 2005, IJCNN 2005, vol.
3, 2005, pp. 1460–1462.
148 COMPUTER SCIENCE REVIEW
[93] Jochen Triesch, A gradient rule for the plasticity of a
neuron’s intrinsic excitability, in: Proceedings of the 13th
European Symposium on Artificial Neural Networks, ESANN
2005, 2005, pp. 65–70.
[94] Melanie Mitchell, James P. Crutchfield, Peter T. Hraber,
Dynamics, computation, and the “edge of chaos”: A re-
examination, in: G. Cowan, D. Pines, D. Melzner (Eds.),
Complexity: Metaphors, Models, and Reality, Addison-
Wesley, Reading, MA, 1994, pp. 497–513.
[95] Robert Legenstein, Wolfgang Maass, What makes a
dynamical system computationally powerful? in: S. Haykin,
J. Príncipe, T. Sejnowski, J. McWhirter (Eds.), New Directions
in Statistical Signal Processing: From Systems to Brain, MIT
Press, 2007, pp. 127–154.
[96] Mustafa C. Ozturk, José C. Príncipe, Computing with
transiently stable states, in: Proceedings of the IEEE
International Joint Conference on Neural Networks, 2005,
IJCNN 2005, vol. 3, 2005, pp. 1467–1472.
[97] Donald O. Hebb, The Organization of Behavior: A Neuropsy-
chological Theory, Wiley, New York, 1949.
[98] Štefan Babinec, Jiří Pospíchal, Improving the prediction ac-
curacy of echo state neural networks by anti-Oja’s learn-
ing, in: Proceedings of the 17th International Conference on
Artificial Neural Networks, in: LNCS, vol. 4668, Springer,
2007, pp. 19–28.
[99] Henry Markram, Yun Wang, Misha Tsodyks, Differential
signaling via the same axon of neocortical pyramidal
neurons, Proceedings of National Academy of Sciences USA
95 (9) (1998) 5323–5328.
[100] David Norton, Dan Ventura, Preparing more effective liquid
state machines using Hebbian learning, in: Proceedings of
the IEEE International Joint Conference on Neural Networks,
2006, IJCNN 2006, 2006, pp. 4243–4248.
[101] Hélène Paugam-Moisy, Regis Martinez, Samy Bengio, Delay
learning and polychronization for reservoir computing,
Neurocomputing 71 (7–9) (2008) 1143–1158.
[102] Roland Baddeley, Larry F. Abbott, Michael C.A. Booth, Frank
Sengpeil, Toby Freeman, Edward A. Wakeman, Edmund
T. Rolls, Responses of neurons in primary and inferior
temporal visual cortices to natural scenes, Proceedings of
the Royal Society of London B 264 (1997) 1775–1783.
[103] Martin Stemmler, Christof Koch, How voltage-dependent
conductances can adapt to maximize the information
encoded by neuronal firing rate, Nature Neuroscience 2 (6)
(1999) 521–527.
[104] Jochen Triesch, Synergies between intrinsic and synaptic
plasticity in individual model neurons, in: Advances in
Neural Information Processing Systems 17, MIT Press,
Cambridge, MA, 2005, pp. 1417–1424.
[105] Anthony J. Bell, Terrence J. Sejnowski, An information-
maximization approach to blind separation and blind
deconvolution, Neural Computation 7 (6) (1995) 1129–1159.
[106] Jochen J. Steil, Online reservoir adaptation by intrinsic
plasticity for backpropagation-decorrelation and echo state
learning, Neural Networks 20 (3) (2007) 353–364.
[107] Marion Wardermann, Jochen J. Steil, Intrinsic plasticity for
reservoir learning algorithms, in: Proceedings of the 15th
European Symposium on Artificial Neural Networks, ESANN
2007, 2007, pp. 513–518.
[108] Jochen J. Steil, Several ways to solve the MSO problem, in:
Proceedings of the 15th European Symposium on Artificial
Neural Networks, ESANN 2007, 2007, pp. 489–494.
[109] Joschka Boedecker, Oliver Obst, Norbert Michael Mayer,
Minoru Asada, Studies on reservoir initialization and
dynamics shaping in echo state networks, in: Proceedings
of the 17th European Symposium on Artificial Neural
Networks, ESANN 2009, 2009 (in press).
3 (2009) 127–149
[110] Naftali Tishby, Fernando C. Pereira, William Bialek, The
information bottleneck method, in: Proceedings of the 37th
Annual Allerton Conference on Communication, Control
and Computing, 1999, pp. 368–377.
[111] Taro Toyoizumi, Jean-Pascal Pfister, Kazuyuki Aihara,
Wulfram Gerstner, Generalized Bienenstock–Cooper–Munro
rule for spiking neurons that maximizes information
transmission, Proceedings of National Academy of Sciences
USA 102 (2005) 5239–5244.
[112] Stefan Klampfl, Robert Legenstein, Wolfgang Maass,
Information bottleneck optimization and independent
component extraction with spiking neurons, in: Ad-
vances in Neural Information Processing Systems 19,
ICANN 2007, MIT Press, Cambridge, MA, 2007, pp. 713–720.
[113] Stefan Klampfl, Robert Legenstein, Wolfgang Maass, Spiking
neurons can learn to solve information bottleneck problems
and to extract independent components, Neural Computa-
tion 21 (4) (2008) 911–959.
[114] Lars Buesing, Wolfgang Maass, Simplified rules and
theoretical analysis for information bottleneck optimization
and PCA with spiking neurons, in: Advances in Neural
Information Processing Systems 20, MIT Press, Cambridge,
MA, 2008, pp. 193–200.
[115] Jochen Triesch, Synergies between intrinsic and synaptic
plasticity mechanisms, Neural Computation 19 (4) (2007)
885–909.
[116] Nicholas J. Butko, Jochen Triesch, Learning sensory rep-
resentations with intrinsic plasticity, Neurocomputing (70)
(2007) 1130–1138.
[117] Andreea Lazar, Gordon Pipa, Jochen Triesch, Fading memory
and time series prediction in recurrent networks with
different forms of plasticity, Neural Networks 20 (3) (2007)
312–322.
[118] Norbert M. Mayer, Matthew Browne, Echo state networks
and self-prediction, in: Revised Selected Papers of Biologi-
cally Inspired Approaches to Advanced Information Tech-
nology, BioADIT 2004, 2004, pp. 40–48.
[119] Mustafa C. Ozturk, Dongming Xu, José C. Príncipe, Analysis
and design of echo state networks, Neural Computation 19
(1) (2007) 111–138.
[120] William H. Kautz, Transient synthesis in the time domain,
IRE Transactions on Circuit Theory 1 (3) (1954) 29–39.
[121] Kazuo Ishii, Tijn van der Zant, Vlatko Bečanović, Paul
Plöger, Identification of motion with echo state network,
in: Proceedings of the OCEANS 2004 MTS/IEEE – TECHNO-
OCEAN 2004 Conference, vol. 3, 2004, pp. 1205–1210.
[122] John H. Holland, Adaptation in Natural and Artificial
Systems: An Introductory Analysis with Applications to
Biology Control and Artificial Intelligence, MIT Press,
Cambridge, MA, USA, 1992.
[123] Ali Ajdari Rad, Mahdi Jalili, Martin Hasler, Reservoir
optimization in recurrent neural networks using Kronecker
kernels, in: Proceedings of IEEE International Symposium on
Circuits and Systems 2008, IEEE, 2008, pp. 868–871.
[124] Xavier Dutoit, Hendrik Van Brussel, Marnix Nutti, A first
attempt of reservoir pruning for classification problems, in:
Proceedings of the 15th European Symposium on Artificial
Neural Networks, ESANN 2007, 2007, pp. 507–512.
[125] Wolfgang Maass, Prashant Joshi, Eduardo D. Sontag,
Computational aspects of feedback in neural circuits, PLoS
Computational Biology 3 (1) (2007) e165+.
[126] Robert Legenstein, Dejan Pecevski, Wolfgang Maass, Theo-
retical analysis of learning with reward-modulated spike-
timing-dependent plasticity, in: Advances in Neural In-
formation Processing Systems 20, NIPS 2007, MIT Press,
Cambridge, MA, 2008, pp. 881–888.
 COMPUTER SCIENCE REVIEW
[127] Robert Legenstein, Dejan Pecevski, Wolfgang Maass,
A learning theory for reward-modulated spike-timing-
dependent plasticity with application to biofeedback, PLoS
Computational Biology 4 (10) (2008) e1000180.
[128] Åke Björck, Numerical Method for Least Squares Problems,
SIAM, Philadelphia, PA, USA, 1996.
[129] Andrew Carnell, Daniel Richardson, Linear algebra for time
series of spikes, in: Proceedings of the 13th European
Symposium on Artificial Neural Networks, ESANN 2005,
2005, pp. 363–368.
[130] Ali U. Küçükemre, Echo state networks for adaptive filtering,
University of Applied Sciences Bohn-Rhein-Sieg, Germany,
April 2006. http://www.faculty.jacobs-university.de/hjaeger/
pubs/Kucukemre.pdf.
[131] Zhinwei Shi, Min Han, Support vector echo-state machine
for chaotic time-series prediction, IEEE Transactions on
Neural Networks 18 (2) (2007) 359–372.
[132] Jürgen Schmidhuber, Matteo Gagliolo, Daan Wierstra,
Faustino J. Gomez, Evolino for recurrent support vector
machines. Technical Report, 2006.
[133] Benjamin Schrauwen, Jan Van Campenhout, Linking non-
binned spike train kernels to several existing spike train
metrics, in: M. Verleysen (Ed.), Proceedings of the 14th
European Symposium on Artificial Neural Networks, ESANN
2006, d-side publications, Evere, 2006, pp. 41–46.
[134] Jochen J. Steil, Stability of backpropagation-decorrelation
efficient O(N) recurrent learning, in: Proceedings of the 13th
European Symposium on Artificial Neural Networks, ESANN
2005, 2005, pp. 43–48.
[135] Francis wyffels, Benjamin Schrauwen, Dirk Stroobandt,
Stable output feedback in reservoir computing using
ridge regression, in: Proceedings of the 18th International
Conference on Artificial Neural Networks, ICANN 2008,
in: LNCS, vol. 5163, Springer, 2008, pp. 808–817.
[136] Xavier Dutoit, Benjamin Schrauwen, Jan Van Campen-
hout, Dirk Stroobandt, Hendrik Van Brussel, Marnix Nut-
tin, Pruning and regularization in reservoir computing: A
first insight, in: Proceedings of the 16th European Sym-
posium on Artificial Neural Networks, ESANN 2008, 2008,
pp. 1–6.
[137] Joe Tebelskis, Ph.D. Thesis, Speech Recognition using Neural
Networks, School of Computer Science, Carnegie Mellon
University, Pittsburgh, Pennsylvania, 1995.
[138] Mark D. Skowronski, John G. Harris, Automatic speech
recognition using a predictive echo state network classifier,
Neural Networks 20 (3) (2007) 414–423.
3 (2009) 127–149 149
[139] Dongming Xu, Jing Lan, José C. Príncipe, Direct adaptive
control: An echo state network and genetic algorithm
approach, in: Proceedings of the IEEE International Joint
Conference on Neural Networks, 2005, IJCNN 2005, vol. 3,
2005, pp. 1483–1486.
[140] Alexandre Devert, Nicolas Bredeche, Marc Schoenauer,
Unsupervised learning of echo state networks: a case
study in artificial embryogeny, in: Proceedings of the 8th
International Conference on Artificial Evolution, in: LNCS,
vol. 4926, Springer, 2008, pp. 278–290.
[141] Fei Jiang, Hugues Berry, Marc Schoenauer, Unsupervised
learning of echo state networks: Balancing the double
pole, in: Proceedings of the 10th Genetic and Evolutionary
Computation Conference, ACM, 2008, pp. 869–870.
[142] Keith Bush, Charles Anderson, Modeling reward functions
for incomplete state representations via echo state
networks, in: Proceedings of the IEEE International Joint
Conference on Neural Networks, 2005, IJCNN 2005, vol. 5,
2005, pp. 2995–3000.
[143] Štefan Babinec, Jiří Pospíchal, Merging echo state and
feedforward neural networks for time series forecasting,
in: Proceedings of the 16th International Conference on
Artificial Neural Networks, in: LNCS, vol. 4131, Springer,
2006, pp. 367–375.
[144] Mustafa C. Ozturk, José C. Príncipe, An associative memory
readout for ESNs with applications to dynamical pattern
recognition, Neural Networks 20 (3) (2007) 377–390.
[145] Mark Embrechts, Luis Alexandre, Jonathan Linton, Reservoir
computing for static pattern recognition, in: Proceedings
of the 17th European Symposium on Artificial Neural
Networks, ESANN 2009, 2009 (in press).
[146] Felix R. Reinhart, Jochen J. Steil, Attractor-based com-
putation with reservoirs for online learning of inverse
kinematics, in: Proceedings of the 17th European Sym-
posium on Artificial Neural Networks, ESANN 2009, 2009
(in press).
[147] Keith Bush, Charles Anderson, Exploiting iso-error path-
ways in the N, k-plane to improve echo state network per-
formance, 2006.
[148] Yoshua Bengio, Yann LeCun, Scaling learning algorithms
toward AI, in: L. Bottou, O. Chapelle, D. DeCoste, J. Weston
(Eds.), Large Scale Kernel Machines, MIT Press, Cambridge,
MA, 2007.
[149] Herbert Jaeger, Discovering multiscale dynamical features
with hierarchical echo state networks, Technical Report
No. 9, Jacobs University Bremen, 2007.