IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL 5 . NO. 3.
MAY 1994
A Communication Architecture Tailored for
Analog VLSI Artificial Neural Networks:
Intrinsic Performance and Limitations
Alessandro Mortara, Student Member, IEEE and Eric A. Vittoz, Fellow, IEEE
Abstract-An architecture for interchip communication among
analog VLSI neural networks is proposed. Activity is encoded in
a neuron’s pulse emission frequency. Information is transmitted
through the non-arbitered, asynchronous access of pulses to a
common bus. The impact of collisions when the bus is accessed
by more than one user is investigated. The information-carrying
capability is assessed and the trade-off between accuracy of the
transmitted information and attainable dynamic range is brought
out in terms of simple global parameters that characterize the
application. It is found that the proposed architecture is well
suited for the kind of communication requirements associated to
neural computation systems. A coding scheme aimed at pushing
the system towards its theoretical performance is also presented
and evaluated.
I. INTRODUCTION
T HE OBJECTIVE of this paper is to propose an architec-
ture for interchip communication among artificial neural
networks realized in analog VLSI. Typical neural processing
requires a front end performing some sort of useful preprocess-
ing of the raw “sensory” data; e.g., sound or images. Typically,
this kind of preprocessing is very efficiently implemented by
a fully analog chip, and several examples can be found in [ 11.
Surface constraints, however, hinder higher-level computation
on the same chip that performs preprocessing. Single-chip
systems are thus limited in both computational ability and
sensory resolution and an efficient communication scheme be-
tween a fully analog interface and subsequent, possibly digital,
higher-level computation parts would be most welcome. If,
on the one hand, the fully parallel communication technique
implemented in nervous systems is unapplicable in the VLSI
context because of the prohibitive number of output pins
the connections would need, a purely systematic sequential
communication scheme (scanning) requires a very high clock
rate for large numbers of cells in order to follow activity
variations at sufficient speed. This is incompatible with the
need to limit the power dissipated on the chip and takes no
advantage of the way activity is most often distributed over a
Manuscript received September 2, 1992: revised November 30, 1992. This
work was supported by the FSRM (Fondation Suisw pour la Recherche en
Microtechnique).
A. Mortara and E. A. Vittor are with the Electronics Laboratory, Department
of Electrical Engineering, Swiss Federal Institute of Technology, CH-I015
Lausanne, Switzerland.
E. A. Vittoz is with the Electronics Laboratory. Department of Electrical
Engineering, Swiss Federal Institute of Technology. CH-I015 Lausanne,
Switzerland and also with the Centre Suisse d’Electronique et de Microtech-
nique, Ch-2007 Neuchltel, Switzerland.
IEEE Log Number 9207 152.
1045-9227/94$04.00 0 1994 IEEE
459
neural network: concentrated in a small number of cells. The
solution proposed and evaluated in this paper takes advantage
of this circumstance and of other characteristics of neural
computation networks. In an interchip communication system,
of possible guidelines could be the following:
Since the available bandwidth of a VLSI communication
channel is typically several orders of magnitude larger
than that of its biological counterpart (lo7 versus lo3
Hz) [ 2 ] , the high parallelism present in biological com-
munication structures (for example axon bundles) could
conceivably be replaced by faster shared structures in
an analog neural system, provided collisions (two or
more cells trying to access the channel at the same
time) have negligible occurrence probability or can be
properly handled.
The communication overhead should not complicate
or severely influence the structure of the computing
elements.
The communication scheme should be tailored on, or at
least take some advantage of the particular way data are
processed by a neural system. For example it is well
known that retinal response is faster to the variation of
high-activity signals or that the response of a Kohonen
network localizes in a few units while the others are
inactive [3]. A simple sequential scanning of the cells
would not capture these particularities.
Communication should take place by means of pulses
carrying information in their interpulse time [4], the
frequency of pulse emission increasing with increasing
activity.
For the sake of speed, any coding intended to either
identify a cell or to reduce collision errors should be
wired.
These guidelines point to a system that is as “natural” as
possible, namely pulsed, unclocked, and as wired as possible.
Since it is not possible to equip each neuron needing to
communicate information to another chip with an individual
“wire,” that uniquely identifies it, some sort of binary coding
of the neuron’s identity over the shared channel is used. Thus
the general structure evoked could look like that of Fig. 1.
All cells needing to output their activity have access to a
parallel bus on which their identity is wired in encoded form.
When activity manifests itself by a pulse emission, the bus’
configuration carries the identity of the emitting cell for the
duration of a pulse [ 5 ] . Pulses are decoded and accumulated
460
converter’soutput
7
I1111111111 I IIIIII I
n
\
\
reconstructed
c
L
E /-I
oulse
neuron cell to
. .
frepuencv
converter
- U
Fig. 1, General structure of the communication system. Neurons’ analog
outputs are transformed in a frequency modulated train of pulses by an
activity-to-frequency converter. For the duration of a pulse, the encoded
identity of the emitting cell appears on the bus. At the receiver side, a decoder
activates the receiver corresponding to the active cell. The selected receiver
accumulates the incoming pulses during an observation time short enough to
follow activity variations and produces, by averaging, its estimate of the sender
cell’s activity on the receiving network in the form of an analog quantity such
as current or voltage.
by the target cells over a time window T limited by the speed
required by the application (say 50 ms for a retinal network).
In this paper the issue of mapping the activity distribution
of the sender chip onto the receiver chip is addressed with no
reference to the possibility of a direct use of the pulses, without
accumulation, for specific applications such as a cochlear
auditory processor. The coding scheme, depicted in Fig. 1
as the encoder-decoder pair, is not optional since it must
compensate for the most notable difference between such
scheme and the biological implementation: the possibility of
collisions when two or more cells attempt to access the channel
at the same time. This coding scheme is an alternative to
the use of arbiter circuits [ 5 ] . An arbiter decides which one
of a number of colliding pulses has the right to access the
transmission channel and allows a second chance to the losers
of this competition. This very desirable property is offset
by increased circuit complexity (an arbitration between
alternatives needs O ( N log N ) elementary arbitration circuits
[5] arranged in a binary tree) and signalling complexity in
the form of request and acknowledge signals propagating
through the arbiter and the body of the circuit [6]. The
possibility also exists that the variable time the arbiter needs
to grant access to the channel introduces excessive alterations
in the time structure of the spike train to be communicated.
The alternative we propose and evaluate in this paper stems
from an error detection philosophy in which collision events
are detected through coding and ignored (pulse loss). The
encoding hardware consists of just connected wires so that
speed and design simplicity are favored. In particular speed
offsets the lack of a “record” of the colliding parties in that it
allows a prompt repetition of the information lost in a collision.
The paper discusses quantitatively this basic trade-off.
In Section I1 we concentrate on the intrinsic limitations of
the architecture assuming that the effect of collisions is just the
loss of a pulse and not its reception by an unintended target.
This, as mentioned, requires that an efficient collision detection
IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL. S, NO. 3 . MAY 1994
scheme be available. One such scheme, easily wired at the
expense of tolerable redundancy, is presented and analyzed
in Section 111. Section IV presents a discussion of the system
performance.
11. INTRINSIC PROPERTIES AND LIMITATIONS
A . Statistical Model
The model we present here is based on the two assumptions
of complete asynchronicity of the pulses generated by any two
cells in the network and of activity variations occurring at a
time scale much larger than the pulse duration. As opposed to a
clocked system, where transitions occur at predictable instants,
here we assume that the cells emit pulses in such a way that
coherent phenomena between pulses do not take place. This
is a way of taking advantage of the intrinsic limitations of an
analog VLSI system: if mismatches between components exist
and if the pulse duration is of the order of the RMS value
of the unavoidable phase jitter of the activity-to-frequency
converter, the possibility of two cells emitting synchronously
and colliding systematically is ruled out. Thus only the average
number of pulses per unit time is fixed by the average activity
of the network. More precisely, consider a generic network
consisting of N cells that emit pulses of duration S at a rate
H , f o where H i is the activity of cell i and 0 < Hi < 1, f o is the
frequency corresponding to maximum activity. If the average
activity N is defined as
N=-ZH,
-
1
3
N
/=1
we model the point process of the beginning of a pulse
emission anywhere in the network by a Poisson process [7]
with rate X = N a f ~ We . can then calculate the probability
that a cell trying to put a pulse through the channel succeeds
without undergoing collision. If the cell starts firing at time t ,
there will be a collision if any other cell fires any time between
+
t - h and t h. The probability of this event is:
Thus the probability of safe emission is
The receiver observes the pulses coming from cell .L and
retrieves the information contained in the number k , of pulses
coming from cell i in the observation time T (short enough
to follow the time scale of variation of 0;): it estimates the
observed activity by k , /Tfo. Collisions are responsible of the
loss of part of the emitted pulses, thus I;, is a random variable.
We now derive its probability distribution.
Dropping the index i , in a time window T , the number
of times the cell fires is also a random variable V with 2
possible values: 71 = IHTfo, the bar denoting the largest
+
integer smaller than HTfo, and rl, 1. These values occur with
respective probability (1-.c) and .I’ where .I‘ = HT,fo-n. These
definitions are illustrated in Fig. 2. Using the total probability
MORTARA AND VI’ITOZ: COMMUNICATION ARCHITECTURE FOR ANALOG VLSl ARTIFICIAL NEURAL NETWORKS 46 I

~ TI:x=On=4
U m
T2 : x = I ; n = 4

Fig. 2. Illustrating the derivation of (3). If the receiver observation time is

T I and the frequency observed is 0 f o , there is a very small probability .I’ of
observing five pulses [corresponding to i f +
1 in (3)]. If the observation time
is T L ,this probability is close to 1. The pulse duration is 6.
0.0003 1’ I
10
I
100
I
1000
theorem: pr(lc) = pr(klV = 7 ~ )pr(V = n ) pr(k/V = + Normalized activity: Wa(TnN6)

71+ +
1) pr(V = n l ) ,the probability of k can be written as Fig. 3. Error-activity relationship. Curve a: exact value curve b: expression
a combination of binomial distributions: (8). For a given maximum-activity frequency, the curve provides the receiver
error as a function of transmitted local activity. Larger activities are estimated
with a smaller error.

Using mean value and second moment of the binomial distri- K is the cell’s activity normalized to the average activity and
bution with parameters U (number of trials) and T (probability multiplied by the scale factor (Tl2N6). The scale factor is the
of favorable event), given respectively by U T and u27r2 +
ratio of the observation time, determined by the application,
u ~ ( -l T ) , the mean m, second moment r n 2 and variance to the characteristic time 2NS (twice the time needed to scan
u2 = 771712 - m2 of the distribution ( 3 ) are calculated: the network spending a time 6 in each cell). Assuming for
T the maximum value allowed by the application, a system
+ +
m = 4 7 1 1 ) p (1 - x ) n p = p(n : E ) + ( 4 4 based on pure scanning will simply not work (i.e., will exhibit
+
m2 = z [ ( n 1)’y2 + + (71 +
~ ) p q ] (1 - :r;)(n2p2 n p q ) +aliasing) unless the scale factor is > l / 2 . The system we are
(4b) examining, conversely, shows what Mead [ 11 calls a “graceful
+
fT2 = pfJ(7L :J;) + p2.C(l - :J;) (4c) degradation” of performance as the scale factor diminishes:
there will not be aliasing but just a noisier reproduction of the
The relative error c2 = a 2 / m 2is thus: transmitted data as K decreases.
q/p 2(1 - ). E is the average frequency cyfa normalized to the inverse
( 5 ) of the characteristic time. The first term in the right-hand
& 2 = - + -
n, :r + (n : E ) ~ + side of (7) can be termed a “floor” noise and rewritten
B . Performance and Limitations ( 1 / K ) f ( E )where f([) = [exp(E) - 1]/E converges to 1
According to (4a), a measurement of activity for E << 1
<
for + 0. Hence performance is ultimately limited by 1/K,
which sets the absolute minimum relative error. The second
yields pulse counts narrowly distributed around 1)(71+2). Thus
term contributes a lobed, or spiky structure whose envelope
every cell seen from the outside seems to have an activity
decreases as l / ( < K ) * .
approximately reduced to a fraction p of its real activity.
Degradation relative to floor noise occurs at the onset of
This, for instance in an image, corresponds only to a general
the spiky behavior of error, as shown in Fig. 3, which plots
darkening but not to contrast reduction or noise addition. Noise
c2 as a function of the normalized activity K for a particular
+
is described by the spreading around p ( n : E ) of the pulse
value = 0.1. Note, looking back to (6), that for 71 sufficiently
count coming from the observed cell or equivalently by ( 5 ) .
larger than 1 the maxima of the lobes are ~ 1 / 4 / ( < Ksince)~
c2 consists of 2 terms:
the maximum of x ( 1 - z) is 1/4 for 0 < z < 1. Using this
+
z(1 - 2)/(7171 : c ) ~dominates at low pulse counts and fact, the envelope of the relative error, also plotted in Fig. 3,
describes the blurring of the frequency measurement can be rewritten:
when the observation time is just a few periods of the
observed frequency. 2NaS
(8)
+
(q/p)/(n, x) more specifically reflects the uncertainty
introduced by collisions, even when the observation time
which can be solved for 6’ to yield:
is long compared to the interpulse spacing. Using ( l ) ,
(2) and the expression of X in (5) we can write:
(9)

Equation (9) provides the relationship between the minimum

activity H that can be observed with a given error and
the normalized maximum-activity frequency <. It is illustrated
in Fig. 4 for several values of E ~ The
. minimum activity is
expressed in terms of the minimum attainable K : Kminand
the maximum-activity frequency is expressed by E. Fig. 4
362
3.105
U
8
2 3.10’
5
I I I J
0.001 0.01 0.1 1 Fig. 5 .
Normalized frequency. 5
Fig. 4. Dynamic range-maximum activity frequency relationship. Each curve
shows how to choose the best <
to attain the largest dynamic range (i.e..
the smallest minimum activity detectable with a given tolerable error).
The optimum (minimum of the curves) occurs at slightly different value\
of normalized frequency but \atisfactory. although suboptimal. behavior is
<
obtained in all cases for % 0.1.
shows that a best exists that optimizes the dynamic range
for a given tolerable error level, the minimum is achieved at
slightly different values of the optimal for different values
of E * . However, since none of the minima corresponding to
different E ’ s is sharp, setting to about 0.1, thus the ,f0 value
to about 0.1(1/2Ntrb), guarantees operation to be very close
to the optimum for every practical error range. The order of
magnitude of the minimum activity measured at a given error
level is Nrub/TE2. In order to be around the optimal operating
frequency, the value of f o should be controlled by the average
activity according, in a non-critical manner, to the relationship
tYf” % O.l/’2iVh (10)
For the results just derived to be applicable. an efficient
collision detection scheme must be available because the effect
of receiver errors due to the appearance after a collision of a
wrong address on the bus has been ignored. One such scheme
is discussed in the next section.
111. CODINGSTRATEGY
A . Motivations
During time intervals where the communication channel is
accessed by more than one cell, the information it carries about
the senders may be incorrect. We assume that the bus performs
the bitwise wired OR operation because the simultaneous
presence of two or more pulses on a wire of the bus should
naturally correspond to a pulse on the wire. Two cases can then
occur: either the result of OR-ing the two (or more) identities
leaves one of them unaltered (Self-Arbitered Collision: SAC)
or the result differs from all colliding senders’ identities (Non-
Self-Arbitered Collision: NSAC: these are the only acronyms
used here.) In the latter case it should be ensured that the
wrong identity carried by the bus does not correspond to a
possible sender on the network so that the NSAC is detected.
The coding strategy that suggests itself is thus to systematically
eliminate, by not wiring them on the bus, those sender identity
codes that most frequently occur as NSAC results. In the
following we first examine how the probability of SAC varies
with the number N of bits used to encode identities. This
is an important quantity to know because SAC’S introduce an
IEEE TRANSACTIONS O N NEURAL NETWOKKS. VOL. 5. NO. 3, MAY 1994
Collision matrix for .I-= 3 bits
unwanted bias. some cells having a better probability of getting
through after a collision. They also constitute a problem in a
two dimensional array where independent encoding of lines
and columns is implemented in that they can produce “image”
events: if cell (11. c1) collides with cell ( b 2 . c 2 ) the result can be
a pair of valid codes (ll. c 2 ) or ( 1 2 . c l ) . Although potentially
“nasty,” this problem can be made quantitatively irrelevant
as shown in Section 111-B. We then derive the distribution of
NSAC results to quantify the efficiency of eliminating a sender
code and finally discuss the trade-off between redundancy and
collision detection efficiency.
B . Prohahilit?. of Self-Arbirered Co1lision.s
To obtain a formula giving the number of SAC using N bits
we consider a “collision matrix” TI‘ as shown in Fig. 5 for
N = 3. In this matrix lines and column indexes are given in
their binary expression and the matrix entries give the result
of bitwise OR-ing the line and column indexes and underlined
font indicates a NSAC. We call upper (lower) triangle of the
matrix the set of entries with column index larger (smaller)
than line index. It is shown in Appendix A that if an N-bits
coding is used, the number K ( N ) of possible NSAC is:
K ( N )= 2 , 41\--1 - 3.1- + 2.’-1 (1 1)
Since total number of possible collisions is 2K-1(2’y - 1)
(number of entries in the upper triangle of T 7 ) the conditional
probability of SAC given a collision occurs is:
and is plotted in Fig. 6. We are now in a position to evaluate
the problem of the appearance of “image events” evoked in
Section 111-A. Suppose we are encoding independently the
line and column values in a two-dimensional array using 8
bits for the lines and 8 bits for the columns. What is the
probability that an “image event” appears during a collision? If
a collision occurs, the probability that a SAC results in both the
line and column buses is, according to Fig. 6 and since lines
and columns are encoded independently, about (0. The
possible outcomes of a collision where cell ( 1 1 . (:I) collides
with cell ( L z . c . 2 ) and a SAC occurs in the line and column
address are (11. q ) .( 1 2 . ~ 2 ) (. / I . Q) and ( l 2 . c l ) : the last two
correspond to an image event. The resulting probability of an
image event is thus 0.5.(0.18)2 = 0.016. It decreases rapidly
(as the square of the probability of SAC) with increasing
MORTARA AND VITTO/ COMMUNlCArlON ARCHITECTLRE FOR A\ALOC; VLSl ARTIFICIAL NEURAL NETWORKS
00 , I , , , , , , , ,
1 2 3 4 5 6 7 8 9 1 0 1 1
Number of bits used to encode
Fig. 6. Probability of a SAC vcrws nunibcr of hit\ u v d to encode
number of bits used to encode the line and column values
as shown in Fig. 6.
C . Distribution o j NSAC Kesirlts
Since the result of NSAC at the bit level is 1, addresses
containing many 1’s in their expression appear more frequently
as NSAC results. Moreover addresses with the same number
of 1’s appear the same number of times as NSAC results since
the significance of bits at different positions is only a matter
of convention. More precisely, it is shown in Appendix B that
the number of times addresses containing X. I ‘s in their N-bits
expression appear as NSAC result is:
It can be readily verified, as it is necessary. that
The coding strategy is best illustrated by noting that by simply
not wiring the address 11 . . . 1 a number (3-‘/2 - 2.‘ 1/2)
of NSAC becomes detectable. Of course efficiency decreases
continuing along this line: more addresses must be eliminated,
and redundancy increased, to detect the same number of
collisions. The exact relationship between the redundancy
introduced by encoding less than 2.‘ identities over bits
and the NSAC detection capability is examined in the next
section.
D. Redundancy versus Detwtuhility
In this section we are concerned with the following problem:
by renouncing to wire all the addresses with 1V. N-1. . . . . AV-
I; 1’s in their expression:
1) What is the probability J ( S k. ) that a collision among
the remaining addresses remains undetected?
2 ) What redundancy is introduced?
To answer 1) it is shown in Appendix C that if all the addresses
having N - k to N ones are removed, then the number S ( X . k )
463
3 4 5 6 1 8 9 10 11
Number of bits med to encode: N
Fig. 7. Probability of undetected collision versus number of bits used to
encode. By renouncing to wire all the addresses with .V. .V - 1. . . . . .\-- k
ones in their expression, the probability of undetected collision is given by
the ordinate of the point specified on the curve by .\-and A..
of collisions that disappear is recursively given by:
S(.I-.k ) = S(s - 1. k ) + 2S( -1-- 1.1. - 1 ) + (-1- - 2) ( 2 \ -i -
k-1
(15)
and the answer to 1) is:
Using (1 5 ) the values of S ( N . k ) can be computed and inserted
in (16) together with K ( N ) and C ( N .k) given respectively
by ( 1 1) and (13). The results are reported in Fig, 7 that plots
f ( N . k ) as a function of N, with k as a parameter.
The answer to 2 ) can be read from Table I, which gives the
fraction of addresses with at least N - k ones in their binary
expression. As an example of how to use these results, consider
a network of 350 cells: a minimum of 9 bits is required
to encode their identities. Suppose we want more than 95%
collision detection rate. Fig. 7 shows that this is achievable
with a 10 bits coding by not wiring addresses with five or more
ones in their expression. Table I tells us that those addresses are
+ 62.3% of the total; we are thus left with 21°(1 - 0.623) = 386
addresses: enough to encode 350 identities.
IV. CONCLUSION
How does the communication system presented match the
guidelines proposed in Section I? The communication over-
head consists of an activity-to-frequency converter per cell,
a wired encoder, a bus, a decoder of standard design, and a
pulse-accumulation block per cell (implemented for example
by a low-pass filter, [SI). Note that only the activity-to-
frequency converter and the pulse-accumulation block need
to be physically located near the computing elements.
Consider now the dependency of the relative error on aver-
age activity, for a cell whose activity is fixed. If the condition
for maximum dynamic range (2NtrJbS = 0.1) is fulfilled, an
increase of 0 corresponds to a decrease in fo. Referring to
(6). this reduction in the maximum-activity frequency reduces
both denominators in the right-hand side. Thus the system
performs better for low average activity distributions, likely
464 IEEE TRANSACTIONS O N NEURAL NETWORKS, VOL. 5. NO. 3. MAY 1994

.Y = 2 3 4 5 6 7 8 9 10 I1

.v - k =0 1 1 1 1 1 1 I I I I

1 0.75 0.875 0.9375 0.9687 0.9844 0.9922 0.0061 0.998 0.999 0.999
2 0.25 0.5 0.6875 0.8125 0.8906 0.9375 0.9648 0.9805 0.9893 0.9937
3 0. I25 0.3125 0.5 0.6562 0.7734 0.8555 0.9 102 0.9453 0.9668
4 0.0.625 0.187.5 0.3437 0.5 0.6367 0.7461 0.828 1 0.8862
5 0.03 I 2 0. I094 0.2266 0.3633 0.5 0.623 0.725 1
6 0.0156 0.0625 0.1445 0.2.539 0.377 0.5
7 0.0078 0.0352 0.0898 0.1719 0.2739
8 0.0039 0.0 195 0.0547 0.1 128
9 0.002 0.0107 0.0322
10 0.001 0.0054
II 0.0005

to be found in neural computation networks. For example, in
a Kohonen map, the number of active cells is clustered in a
bubble that represents only a very small fraction of the total
number of cells, setting the network’s average activity to a
small value CY << 1. Other examples of this kind of “sparse”
activity distribution are associative memories and edge or spot
detection systems.
On the local scale, again referring to (6), for a fixed average
activity ( 2 , the relative error increases with decreasing local
activity 19.Precision is better for larger activities in conformity
to the principle of allowing those neurons that have “something
to say” to communicate it quickly and precisely.
In some cases it may be interesting to trade dynamic range
for speed and the condition 2Ntrf”G zz 0.1 can be relaxed. If,
for instance, it is desired to detect a bright spot on a grid of
detectors or to quickly report the output of an edge detecting
system a rough distinction between active and inactive regions
is sufficient . The inactive background’s activity can be known
with a large relative error since its absolute value is small.
Thus f o can be increased and accordingly T , the observation
time, can be reduced and it is enough to observe for a very
short time to draw a significant conclusion on where the spot
or the edges are located.
As a last point, it may be appropriate to emphasize that
we have examined a communication system rather than a
visualization system [91. There is no better way of presenting
a moving image to the eye than to show a succession of
frames “frozen” about 50 times per second. But some features
conveying important information may be varying much faster
than a sequential visualization system can ever be able to
follow. This is the motivation for the event-driven approach
N . T-‘, is divided into 4 ( 2A’-’ ) x (2“’ ) submatrices:
T;- and TA’s upper triangles and T;”;contain all the informa-
tion. Tfi(T.2)is simply T5-’ to which all the entries have
been concatenated with the same MSB 0 ( l ) ,each of them
contributes the same number of NSAC as TX-’ and so does
the upper triangle of T;”;.We now show that all the entries
of TG’s lower triangle correspond to NSAC. Notice first that
the result of a SAC must be the larger of the two colliding
addresses. Indeed, in TC, the lower triangle entries have the
line index larger than the column index, thus:
I ) if an entry chosen in TG’s lower triangle corresponds to
a SAC, it must coincide with the line index. The entry at
the corresponding location in T A , however, corresponds
to an NSAC because the MSB of its line (0) collides
with the MSB ( I ) of its column so that the N - 1 LSB’s
are those of the line index, and the MSB is that of the
column index giving an NSAC.
2) If an entry in T;:’s lower triangle corresponds to a
NSAC, the entry at the corresponding location in Ti‘;
also represents a NSAC because the collision takes place
among the N - 1 LSB’s of the colliding addresses.
All entries in the lower triangle of T12correspond therefore
to NSAC and the proof is complete. This result provides the
relation:
K ( N ) = 3 K ( N - 1) + 2-’-2(2A’-1 - 1) (Al)
where the 3 K ( N - 1) term comes from the contributions of
TA and a-’-* (p’
the upper triangle of Ti:. Ti’;. - 1) is

proposed in this paper. We have been mainly concemed with the number of entries in T z ’ s lower triangle. (AI) is easily
the viability of such scheme, thus the focus of the discussion recast in closed form: let R.y = 2~’~-*(2~’-’ - 1) we can then
has been on the fundamental limitations and advantages. rewrite:
Future and ongoing work will entail the realization of a neural
communication system based on the principles and methods
presented here. (A2)
(A2) is transformed with the help of the summation of a
APPENDIXA
geometric series formula and a little algebra into:
DERIVATION
OF ( 1 1)
K ( N ) = 2 . 4-1 - ‘J.\-+ 2-V-1. (‘43)
To derive ( 1 1) we first seek a relation between K(N ) and
K(N - 1). Referring to Fig. 6 , the collision matrix of order (A3) is (11).
MORTARA A N D VlTTOZ COMMUNICATION ARCHITFCTLRE FOR ANA1 OG VLSl ARTIFICIAL N E U R A L NETWORKS 465

APPENDIXB ) the contribution from the upper triangles of TG and

2 d ( ~ r 1 is
DERIVATION
OF (13) T.2 column 2.v-' + v i , while 22Y-' - 1 - 71))is the number of
Consider an N-bits coding and an integer r r i . 0 5 7ri 5 entries in the lower triangle of Ti'; in column 2"-' v i , , all +
2jV - 1. Let f ( m ) be the number of times rrl appears as the NSAC as shown in Appendix A. Consider now a string with
result of a NSAC. We first determine the number f ( 2 " ni) + k 1's. It can either:
of times that 2" +
711 appears as result of NSAC with an 1) be an index of T A , hence a number <ZA'--', and an
(N +1)-bits coding. 2.'- +
711 appears at the locations of (AV- 1)-bit string with k 1's concatenated with an MSB
T.V+1
12 and TZ+"s upper triangles corresponding to locations of 0.
of TA+' upper triangle where rri appears as NSAC result. It 2) be obtained from an ( N - 1)-bit string with ( k - 1) 1's
also appears in TL'l's lower triangle locations corresponding concatenated with an MSB of I . This translates into the
to TlY+1 locations where rrr appears as NSAC result. This first following recursion for the sum of NSAC that disappear
11
contribution adds thus to 3,f( 711). There are other appearances by not wiring addresses with k 1's in their N-bits code,
+
of 2lV v i , namely so many times as rn appears in T:+' where ( C l ) has been used for the type 2) contributions:
as a SAC result. As argued in Appendix A this can happen
only in line m thus any appearance of entry 7r1 in line rri>
of TC+' corresponds to the appearance of 2.' 'rr1 as NSAC + si -1-.x.1 = si .\-- 1. k ) + 2si -1-- I. 1. - 1) + E(2 ' -1 - 1 - .r<)
result in the corresponding location of TA+'. We now evaluate (C2)
this second contribution. It is the same for all addresses with where : r l < 2.'-l has k - 1 ones in its ( A -l)-bits expression.
the same number u ( m ) of 1 ' s in their expression, hence it Note that
is the same as for the string 00..011..1, with u ( r r / , ) LSB set
to I . This string is the binary expression of 2"((") - 1. and it
appears as SAC result exactly 2"('") - 1, times when it collides S, Yz
with any address smaller than 2"("') - 1. Summing the two where is the 1's complement of :cl and y1 < 21V-' has
contributions we obtain: ( k - 1) 0's in its ( N - 1)-bits expression. To evaluate the sum
consider first the example N - 1 = 5 et k - 1 = 2 . We must
f(2" + m )= : j f ( ' r r r ) + z ~ ~- (1 ~ ~ ~ (B1)
)
calculate the sum of all the 5-bit integers with two zeros in
as f ( m )and f ( 2 " - + rri,) depend only on X. = u(7ri3), (Bl) can their binary expression, as shown below:
be rewritten: 00111t
f(X: + 1) = 3 f ( k )+ 2 k - 1 (B2) 0101 1t
01101+
(B2) can be recast in closed form by successively substituting
0111ot
f(1) in f ( 2 ) . f ( 2 ) in f ( 3 ) and so on. The result is:
10011t
k-2

f ( k )= 3;(2'-'-; - +
1) 3"'f( 1) with f'( 1) = 0 (B3) 10101t
J =U 1011ot

thus 11001+
1101ot
11100
Note that each column of the sum has the same number of
Since using an N-bits coding there are (;) addresses with k 1's: in the example 6 and in general:
1's in their expression, ("-1) ~ ("-2)
k-1 k-2
or (number of (Ar- 1)-bits strings with ( k - 1) zeros)-(number
of ( Y - 2)-bits strings with (X - 2) zeros). Each column con-
(B.5) is (13). tributes to the sum according to the power of 2 corresponding
to its position: thus in the example:
APPENDIXC i
OF (15)
DERIVATION
!JI = ci 2J
Consider first an N-bits code and an integel r r t 0 5 rr/ 5 Y> J =o
2' -' - 1. The number of NSAC that dirappear by removing r / i
and in general:
I S denoted by d ( m ) With the help of the rewltu of Appendix
A, we can calculate d(2' rri) -' +
d Q - 1 +7r1) = 2tl(rr,) +(p-1 - 1 - rrc) (Cl)
466 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 5, NO. 3, MAY 1994

-1 N-1 N-2 Alessandro Mortara (S’92) was born in Rome,

- I ) [ ( k - 1) - (k-2)] Italy, in 1963. He obtained the “Laurea” degree in
electronic engineering in 1988 from the University
-1 of Rome, “La Sapienza,” and a Master’s degree
from the Massachusetts Institute of Technology in
1991. He has held positions in the research de-
Where the partment of Olivetti S.P.A., Ivrea, Italy: the MIT
National Magnet Laboratory; the CSEM (Centre

(;I;) (”,
Suisse d’Electronique et de Microtechnique) and is
-
- now working toward the Ph.D. degree at the Swiss
+ l) Federal Institute of Technology in Lausanne under
the supervision of Professor E. A. Vittoz. His research interests are in the
has been used. Substituting (C3) in (C2) we obtain: field of analog VLSI implementation of neural systems with special regard
to communication issues
S ( N ,IC) = S ( N - 1. IC) +2S(N - 1. k - 1)
+ (”
IC-1
- 2)(2“ - 1); (C4)

(C4) is (15).

REFERENCES
C. Mead, Analog VLSI and Nerrrul Systems. Reading, MA: Addison-
Wesley, 1989.
E. Seevinck, P. J. van Beers, and H. Ontrop, “Current mode techniques
for high speed VLSI circuits with application to current sense amplifier
for CMOS SRAM’s,” IEEE .ISolid-Stare
. Circuits. vol. 26, no. 4, pp.
525-536, 1991.
T. Kohonen, Self Organizatiori and Associutiw Memor-y. Berlin:
[31
Springer Verlag, 1989, 3rd edition.
A. F. Murray, D. Del Corso, and L. Tarassenko, “Pulse-stream VLSI
[41
neural networks mixing analog and digital techniques,” IEEE Trans.
Neural Nerworks. vol. 2, no. 2, 1991, pp. 193-204.
151 M. Mahowald, “VLSI analogs of neuronal visual processing: a synthesis
of form and function,” Ph.D. Thesis, Computation and Neural Systems,
California Institute of Technology, 1992.
I61 J. Lazzaro, J. Wawrzynek. M. Mahowald, M. Sivilotti, and D. Gillespie,
“Silicon auditory processors as computer peripherals,” Proceedings
NIPS 1992, Denver, CO, USA, to appear.
A. Papoulis, Prohahility. Random Variahles. atid Stochastic Processes.
New York: McGraw-Hill, 1981.
E. J. Bayly, “Spectral analysis of pulse frequency modulation in the
nervous systems,” IEEE Trans. on Biomed. EnL?..vol. 15, no. 4, pp.
257-265, 1968.
[91 C. A. Mead and T. Delbriick, “Scanners for visualizing activity of analog
VLSI circuitry,” California Institute of Technology, CNS memo I I , June
1991.
Eric A. Vittoz (A’63-M’72-SM’87-F’89) was born
in Lausanne, Switzerland in 1938. He received the
M.S. and Ph.D. degrees in electrical engineering
from the Swiss Federal Institute of Technology in
Lausanne (EPFL) in 1961 and 1969, respectively.
After spending one year as a research assistant,
he joined the Centre Electronique Horloger S.A.
(CEH), Neuchltel, in 1962, where he became in-
volved in micropower integrated circuit develop-
ment for watches, while working on a thesis in
the same field. In 1971, he was appointed Vice
Director, supervising advanced developments in electronic watches and other
micropower systems. In 1984, he took the responsibility of the Circuits
and Systems Research Division of the Swiss Center for Electronics and
Microtechnology (CSEM) in Neuchltel, where he was appointed Executive
Vice-president, Circuit and System Design, in 1991. His field of personal
research interest is the design of low-power analog CMOS circuits, with an
emphasis on their application to neural networks. Since 197.5, he has been
lecturing and supervising student projects in analog circuit design at EPFL,
where he became a Professor in 1982.
Dr. Vittoz has published more than 70 papers and holds 25 patents.