Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.

NPL Symposium on the Mechanization of Thought .

\ Process. No. 10,pp 535-539, HM Stationery Office, London

NATIONAL PHYSICAL LAEORATORY

TEDDINGTON, MIDDLESEX, ENGLAND

PAPER 4-1

SKNSORY MECHANIS118, THE REDUCTION OF

REDUNDANCY ,· ·AND INTELLIGENCE

e
r''
(_,

by

DR. H. B. BARLOW

To be presented at. a Symposlum on

The Mechan!zatlon or rnought Process,
wh!ch wlll be held at the Nat1onal Phys!cal
Laboratory, Teddlngton, M1ddlesex, from 24th-
27th November 1958. The papers and the dlscuss1ons
are to be publlshed by H.M.S.O. ln the Proceed1ngs
ot the Symposlum. Thls paper should not be repro-
ctuced wlthout the permlss ton of the author an.d or
the Secretary, Nat1onal Physical Laboratory.
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought
\ .Process. No. 10,pp 535-539, HM Stationery Office, London
.·

e
4-1. SENSORY MECHANISMS, THE REDUCTION OF
REDUNDANCY, AND INTELLIGENCE
by

DR. H. B. BARLOW

SUMMARY

PSYCHQ-PHYSICAL and phys1olog1cal 1nvest1gat1ons have shown that the eye

and the ear are remarkably eff1c1ent instruments: consequently the amount
of 1nformat1on being fed into the central nervous system must be enormous.
After a delay, which may vary frorn about 100 msec. to about 100 years,
this informatlon plays a part in determining the actlons of an individual:
therefore sorne of the incoming 1nformat1on ls stored for long perlods,
The argurnent ls put forv1ard that the storage and ut111zat1on of th1s
enormous sensory lnflow would be made easier 1f the redundancy o! the
incom1ng messages was reduced. Sorne phys1olog1cal mechanlsms which wauld
start to do this are already known, but these appear to have arlsen by
evolutionary adaptation of the organlsm to types of redundancy whlch are
always present in the environment of the specles. Hucb of the sensory in-
put 1s not shared by all 1nd1v1duals of a spec1es (eg. st1mul1 provided by
parents, language, and geographlcal local! ty) so a dev1ce for "learnlng 11
to reduce redundancy ls required. Psychologlcal experiments give 1nd1ca-
t1ons of such mechanisms opera.ting a t low levels in sensory pa th1.vays, and
"1ntell1gence 11 may lnvolve the capaclty to do.the same at h1gh levels.
In arder to exempllfy the operations contemplated, a device whlch
reduces the correlated actlvity of a pair of binary channels 1s descr1bed.

THE usual mechanlstic approach to the higber nervous system begins with a
cons1derat1on of the factors whlch can be shown to have an 1IT~ed1ate effect
on the output of the nervous system. The commonest starting po1nt ls the
simple monosynapt1c reflex in whlch a single sensory input controls a
single motor output, as shown d1agrammat1c.ally 1n !ig. l(a). The next stage
ls to elaborate this by taklng into account other sensory modallties,
1nh1b1t1on, 1ntcrnunc1al neurones, and controlllng neurones from elsewhere

(94009) 4-1. p3

,r· \
\ i
Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

Se.nsory
Fibre.\
Spinol
Cord .

. -Synapse.

Fibre.

F1g.1(a)

Po in

Controlling
neurone.

rl
Flg.. 1 {b)

B~ Condjtional neurone;
~ acttvated by Bell if

· L_f
· P(Food in mouth 1 Bell)
Food in mouth is high enough.
Effector neurone.:
invariably octivated
Sol~ by Food -in-mouth or
· conditionor neurone.

Flg. l(c)
F!g.l. Dlagram showlng appraach to function rrommotor {effector) slde.
{a) monosynaptic stretch same with addl tion of 1nternunc1al
neurones, con trolllng neurones other par ts or the central ne rvous
system, and 1nh1b1t1on by pa1n endlng.s; {e) condltloned reflex.

{94009) 4-1. P4
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London
)

1n the nervous systern, as shown infig-.J(b). Wlth allits tr1mm1ngs tl1is

gets one toa stage of complexlty perhaps comparable to that of an automat1c
tracklng radar set, or the automatic p11ot of an aeroplane. It i..Jlll· show
none of the plasticity or adaptab111ty to new surroundlngs wh1ch is charac-
ter1st1c of the higher nervous system, so the Pavlovian conditioned reflex
1s next lntroduced. The prlnciple here 1s that 1f there are two sensory
stlmu11 (Bell and food ln mouth), one or whlch (food in mouth) always pro-
duces a response (sallvation), then lf they occur jointly wlth sufflcient
frequency, the one which, to begln wlth, did not cause a response, begins
to do so (Bell alone causes saliva tion}. Thls is shown d1agrammat1cally in
fig-. J(c), and ls perhaps the slmplest type of learning behaviour that has
been studied in anlmals, thou~1 it has not been investigated in a simple
1solated preparation as the diagram might suggest. Uttley (1954, refs. 22
and 2~ has clarifled the pr1nc1ples of operation of such mechanisms and
bu1lt cond1t1onal probabllity devices which shov.1 the same propertles of
learnlng and lnference.
Now the simple feedback dlagram in fig-. J(a) has a single input channel,
fig-. l(b) and (e) have t~tiO 1nputs, and Uttley 1 s machine has up to five ln-
puts; but a human bra1n has something llke 3 x 106 sensory nerve fibres
lead!ng lnto 1t. If it could be supposed that a m1111on or so devlces l!ke
that of fig-. J(c) would deal w1 th t.~.e sensory 1nflow one would be 'tlell
satlsf1ed w!th the understanding galned from thls approach: but th!s 1s not
so. The essential operatlon in a cond!tlonal probab111ty devlce 1s to L __ _
measure the frequency of occurrence of combinations of activlty ln the in-
put. Now 1f the number of blnary lnputs is lncreased from two to a rn1ll1on
the numter of poss1ble comb1nat1ons 1s lncreased from z2 to z(milllon); an
arrangement llke that or Jie. J(c) takes one less far than at first sight
appears. I thlnk 1t follm-vs from thls considera tlon that condi tional pro-
bab111ty machipes cannot be fed wlth raw sensory lnformatlon, and the
problem of d1gest1ng or process!ng the sensory 1nformat1on entering the
bra!n 1s an lmportant one. Furthermore, modern electrophys1olog1cal
techn!ques are maklng 1t posslble to record from nerve cells at var1ous
levels in the sensory pathways, so this ls a problem wh!ch !s beco~ing
accesslble to experimental 1nvest1gat1on.
In thls paper r have flrst trled to make rough estimatas of the rate at
wh1ch information flows lnto the human braln. It is then suggested that an
essent!al step in organ1s1ng thls vast lnflow is to derive slgnals of hlgh
relat1ve entropy from the hlghly redundant sensory messages. For this some-
thing similar to the optimal codes d1scussed by Shannon ( 1949, re!. 19)
needs to be dev!sed for the sensory input, and the steps requ1red to do
th1s are consldered. Finally, a mod1f1ed form of such recod1ng is proposed,
sorne ev1dence that 1t occurs 1s brought forward, and 1t 1s suggested that
the ldea may be extended to cover sorne of the processes go1ng on in con-
sciousness and called reason1ng or 1ntell1gence.

(94009) 4-1.p5
Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

1. THE SENSORY INF'LOW

(a) Properties of Nerve F~ bres

We are equlpped vlith sensory instruments of astonishlng sensltlvity and
versat111ty ~11ch supply 1nformat1on about the environment to the central
nervous system. This 1nfonnat1on ls carried along nerve flbres, and s1~ce a
good deal 1s kno~vn about what these flbres can and cannot do, one can derive
an approxlma te upper llmi t to the ra te a t whlch in forma ti on en ters tlle
braln. If the simple assumptions are made that (1) the maxlmum frequency of
impulses is 700¡sec, and (11) in 1/700th.sec a nerve can only be used to
lndlcate the presence or absence of an impulse, then the maxtmum rate at
whlch it can transmlt lnformation is 700 bits¡sec" Mackay and McCulloch
(1952, ref. 16) po1nt out that the nerve mlght be used more efflclently lf,
lnstead of detecttng the presence or absence of an impulse, the lntervals
between impulses are used to convey informatlon. Using such pulse lnterval
modulation, and assum1ng {1) accuracy of est1mat1on of lntervals of
0.05 rosee, (11) a mlnimum lnterval of 1 rosee, they glve the maxlmum capaclty
as 2880 bits¡sec. This would requlre a mean frequency of 670 1mpulses¡sec,
but at a mean frequency of so¡sec, such pulse 1nterval modulatlon still
allows 500 blts¡sec to be transrnitted. These figures are actuallY too low,
because Mackay and rlcCulloch lncorrectly assumed that the optlmum d1str1bu- )
t1on of lntervals was unlform lnstead of exponentlal: however, lf the other 1

L ..-
assumpt!ons are granted, they show clearly that a single nerve fibre could
be used to transmit lnformation at a rate well above 1000 bits¡sec. )
The total capaclty of the sensory lnflow appears t.o be above 3 x 10 9
blts¡sec, but 1t ls certaln that nothlng llke the full capaclty is utilised.
The mean frequency of impulses mus t be far below the op tünum; p erlpheral
nerves appear to use pulse frequency rather than pulse 1nterval modulatlon,
so that there wlll be high serial correlations between the values of 1nter-
vals; furthermore, there are generallY considerable overlaps in the pick-up
areas of ne1ghbour1ng flbres, which are therefore bound to shoH correlated
actl vl ty. Flnally, the figure for the performance of a nerve f1 bre g1 ven
above m1ght be approxlmately true for the large dlameter fibres, but those
of smaller dlameter, whlch make up a iarge fract1on of the total number,
must have a smaller capaclty. It would be pure guesswork to trj to allow
for these ractors, but one can get 1ndlcat1ons of the utlllsed capacitY from
two other sources.

(b~ Sensory Ability

Jacobson (1950, 19fi1 rcfs. 13 ,li.J} has made estima tes or the infonnatlonal
capaclty of the ear and the eye. For the ear he calculated 50,000 blts¡sec
f~om the number or discrlmlnable pltches (about 1450), the number of dlscri-
mlnable lntensltles at each pitch (average about 230), and the time required

(94009} 4-1. p6

•.
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

to make such dlscrlmlnatlons (1/4 sec). Th!s does not make any alloHance
for masking - the observed fact that the presence of one tone interferes
w1th the perceptlon o f o ther tones. Jacobson calcula ted tha t th!s Hould
reduce the 1nformat1on capacl ty by a factor of about six, ·or!ng!ng 1t
down to 8,000 blts¡sec. Now there are 30,0CO nerve f!bres from the ear, so
each flbre must carry an average of about 0.3 bits per sec.
For the eye he calculated from publlshed data of central and perlpheral
acu!ty that there were 240,000 resolvable elements in the visual f!eld (he
seems to omlt a factor of two in the 1ntegratlon, but thls ls perhaps com-
pensated by the rather high f15ure for aculty wh!ch he uses). He supposes
that each element can be d1scrim1nated at two lntensitles, vlith an average
temporal resolut!on of 1/18 sec. These figures glve 4.3 x 10 6 b!ts¡sec. In
the optlc nerve there are just under a mlllion f!bres, so about 5 b!ts¡sec
are conveyed on the average by each flbre,
These are crude es tima tes. For !ns tan ce, no accoun t has be en taken of
colour d1scr1m1nat1on, or of the abillty to localise a sound by b!naural
effect and judge depth by stereoscopic vlslon. Nevertheless, they are pro-
bably of the r!ght arder of magnitude and they are probably good enough to
justify the clalm that optic nerve flbres carry much more 1nformat1on than
those of the audltory nerve. Thls may be s1gnlf1cant and wlll be referred

o to later ..
These figures suggest that total sensory inflow along the three mlll!on
sensory flbres is rather under 10 7 b1ts¡sec.
C> (e) Communication bandwidths
The capacity of the communlcat!on channels englneers need to transmlt
audl tory and visual s1gnals ls clearly re la ted to the capac 1ty o f trte
sensory pathvmys. Engineers, in the !nterests of economy, may be expected
to try to use the narrowest bandwidths whlch will sat1sfactor11Y lo&d up
1
the sense organs lnvolved, and reclp!ents may be expected to 1ns1st that
'! such satisfactory load1ng 1s not too far short of normal load!ng.
Ten k. c. bandv.Jidth at 40 d. b. s!gnal no1se ratio g!ve a good qual1ty
auditory signal, and has a capac!ty of 133,000 b1ts¡sec. Th1s 1s more than
ten times Jacobson's final figure for the capacity of the ear (8,000 bits¡
sec), and the d1screpancy is presumably due to (1) the transmlss1on of
relative phases of the frequency components, which gives 1nformatlon not
ut111sed by the ear - at least in the type of d1scr!m1natlon taken account
of by Jacobson; (11} the failure of the engineer to explo1 t the loss of
efflc!ency of the ear which results from masking.
A satisfactory 400 11ne televlsion plcture requ1res three megacycle
bandwidth at about 10 d .. b. signal-nolse ratio, and this corresponds to
1.2 x 107 bits¡sec. one 1s much more aware that such a telev1s1on plcture
falls short of one•s normal visual s1gnals than one ls in the case of a

(94009} 4-1. p7

\
Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London
~

10 k.c. 40 d.b. auditory slgnal because lt does not flll the visual fleld,
and lacks detall and colour, but 1t ts st111 more than double Jacobson •s •~'
estimate of the eye•s capac1ty. In th!s case the most notable matchlng
errors are the failure to explo1t (1) low per!pheral acutty of the eye,
(11) reduced temporal and spat1al resolv1ng pm·,rer in low 1ntens1ty regions
of the lmage.
Englneers seem to requtre 5- 10 b1ts¡sec channel capacity per nerve
flbre to load up our sensory pathways, but the d1screpancles between thls
figure and those obtained from direct estlmates of sensory ab111t1es can
probably be attrlbuted to poor matching.

{d) Time of storage

Not only 1s the input to the nervous system enormous, but somef at
least, of the messages received are stored for very long per1ods. Most
people would agree that sensory 1mpresslons can be recalled after a lapse
or, say, 70 years, and somet1mes a person can produce object1ve ev1dence of
the accuracy of hls recollectlons. In add1t1on there are, of course, many
sensory 1mpress1ons whlch cannot be recalled, but which have, none the
less, left their mark: we do not remember the successes and fallures by
which we acquired the correc.t usage of •yes• and •no•, but this correct
'L ___ _
usage 1s often retained beyond the ret1r1ng age. If one allows for fifty
years or wak!ng life, the total sensory input is someth!ng 11ke 10 16 bits.
Complete storage of all this informatlon is neither 11kely to be poss1ble
nor, of course, 1s 1t what ls needed.

(e) Fate o! Sensory In!onnat1.on

The rest of this paper ls about a suggested plan of storlng and d1splay-
1ng thls enormous sensory input, but one must flrst have some idea of the
use that 1s made of the sensory 1nformat1on and the neural equipment wh1ch
1s avallable for aeal1ng wlth it. Accordlng to Cralk (1942, ref.~) the sen-
sory 1nformatlon is used to bu1ld up a model of the externa! world wh1ch
provides a bas1s for determ1ning what course of actlon 1s most likely to
lead to the survlval of the 1nd1v1dual and his species. That is a br1ef
answer to the flrst quest1on, and 1t also glves the answer to another fact
whlch mlght otherwlse be puzzllng. A man can only make dec1s1ons on the
basis of sensory information at a maxlrnum rate of about 5 to 25 b1ts¡sec.
(Hick, 1954 ref. 11 Quastel, 1956, re/. 18}: why, then, does he need a
sensory input of 10 7 b1ts¡sec.? Craik 1 s answer would probably have been
that the greater the sensory input the more complete and accurate the
model, and hence the surer its bas1s for plann1ng surv1va1.
Yne question of the equ1pment available can also, because of our ignor-
rance, be answered brlefly. There are sorne tolO 1nterconnect1ng nerve cells
in the central nervous system, and quite a large proportlon of therrr must be

(94009) 4-1. P8

·J
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

avallable for the task of deallng wlth the sensory input and building up
the model. He are only beg1nn1ng to determine the properties of these
cells; 1 t has be en knmm tha t thei r long processes transmi t informa ti on as
all-or-none impulses for more than flfty years, but how information ls
stored ls not yet understood. In what follows I shall be talking about
what the nervous system does rather than how lt does it, so our lgnorance
of the method of storage of 1nformat1on ls not too serious. The problem
might be discussed abstractly, but for the sake of a definlte model one can
think of each nerve cell havlng "excltation laws" which determine the con-
ditlons under whlch it becomes active, and suppose that these laws can be
changed so that 1t becomes active in response toa dlfferent set of
patterns of actlvity in the nerve cells in contact with it. The excltation
laws for all the neurones would then form a store of 1nformat1on and the
current display would conslst of the pattern of nerve cells wh1ch are
actually transm1tt1ng impulses down their long processes at any g1ven
mamen t.
W1th this model in m1nd the problem is: what should the exc1tat1on laws
of the neurones be, and how should they be alterable, in arder that the
display of actlvlty shall help the individual and species to survlve in
the sltuatlon givlng rise to the current sensory input? To avoid baslng
the argument on uncertaln preconceptlons of what the brain does, one could
(.~; /
put !t in more general terms in this way. The barrage of nervous lmpluses
reach1ng the nervous system seems to be unmanageably large; how should a
··(1
' .....,.·
selectlon of this activity be made for current display and future
reference?

1 -·
1 2. ORGANISATION OF THE SENSORY INPUT
!
The propos1t1on 1s that the 1n1t1al selectlon is performed according to
those stat1st1cal properties of the past sensory messages whlch determine
-how much 1nformat1on particular impulses convey. It ls supposed t.hat the
sensory messages are submltted to a succession of re-codlng operations
which result in reduct1on of redundancy and lncrease of relative entropy
-or the messages which get through. Ideally one mlght imagine that an
optlmal code 1s constructed, so that the output, or "display" of current
input, has no redundancy, relative entropy 1, and carrles all the 1nforma-
t1on o! the input. Thls ideal obvlously cannot be reached, but the re-
cod1ng operatlons are supposed to tend towards the ideal: that ls, outputs
are derlved from the input, whlch have h1gh relative entropy and carry as
much of 1ts 1nformat1on as possible.
Shannon has shmm that lt is possible in principie to obtain near
optlmal codlng lf a suff1c1ent number of messages of a g1 ven length have
occurred to gi ve l:mc\vledge of the sta tistical structure o f the mcssages,

(94009)
Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

,' )

and 1f delays are permitted between input and output. Fano and Huffman
(1953, ref. 1~ have described procedures for constructlng such codes. The )
f1rst steps are to define v.Jhat shall constitute a single message and then
to measure the frequency of occurrence of all posslble messages o f thls
class. Clearly the class cannot be the whole of the sensory input to the
bra1n up to a particular moment, for this message has only occurred once.
The input must be sub-divlded in time, and first consider the operation
required to re-cade messages of duratlon, say, one second. The capacity or
the input channel has been shown to be about 3 x 10 9 bits¡sec. wl:'llch
corresponds to lO(thousand milllon) possible messages per second. If one
takes accollilt of the res trictlons whic!1 reduce the ut111sed capac1 ty to
sorne 107 bits¡sec., and considers messages of one-tenth second duration, ·
there are still sorne 10 300 • 00° possible messages. It would clearly be hope-
less to devote neural equipment to the countlng of each possible message,
for lt ls hi~~lY improbable that any single message will be exactly
repeated and most of such equipment would be unused at death. Thls is,
essen tlally, the same di f ficul ty tha t \-Jas levelled agatnst the idea tha t
conditlonal probab111ty devlces could be served w1th unprocessed sensory
data, but when one considers optlmal codlng there ls a posslble solution.
Because the code is reversible, no 1nformation 1s lost by re-codlng small
sect1ons of the sensory input independently, and such preliminary re-coding L ..
will enable the whole message to be passed down a channel of smaller
capacity, and thus facilltate subsequent steps.
The idea 1s bes t 1llus tra ted by consldering the arder in wh1ch di fferen t (,:
types of redundancy mlght be encountered, and ellmlnated, during the
successlve re-codlng operat1ons. Flrst there is the very large amount which
results from the 1neffic1ent utlllsation of perlpheral nerve fibres. Look-
1ng only at the nerve impulses as they arrlve, 1t would be found that
·impulses occurred at different mean rates in different fibres and in all
of them at rates well below th.e optimal frequency for informatlon trans-
misston. Thls type of 1neff1cient utilisatton of a set of co~municatlon
channels is a fonn of redundancy, but for reasons dlscussed later (Section
4) lt may be less 1mportant to el1m1nate than other forms: for the moment
one can conslder the capacity of a nerve flbre as determined, not by max1-
mum frequency of impulses, but by the mean frequency at whlch they occur.
Next, stlll looking only at the impulses as they reach the central
nervous system, it would be found that impulses do not occur completely at
ran9,om 1n time but tend to follow one another in sequences and bursts: the
flrst re-codlng operatlon mlght be a mechan1sm w:tüch reduced the serial
correlations so that the same amount of lnfonnatlon was carried by fewer
impulses. In addl ti on 1 t would be found tha t cer tain groups of nerve
flbres tended to become active at the sarne time. These would be fibres
whose receptlve flelds on the sensory surface overlapped, so that th1s
particular form of redw'ldancy resul ts from the anatomlcal propert1es of

(94009) 4:...1. P10

 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

f1 bres and sense organs, jus t as the serial e orrela ti ons 1n t 1m e resul t from
the fact the lntensity of a stlmulus 1s coded as frequency of impulses at
the sense organs.
These flrst steps, then, would reduce the orderl1ness in the sensory
messages which results from character1st1cs of the sensory apparatus. But 1f
th1s orderllness can be ellminated, so can that rcsulting from the charac-
terlst1cs of the env1ronment whlch is providlng these st1mul1. For 1nstance,
1t wlll often happen that a st1mulus covers more than a sin~le po1nt on the
sensory surface and therefore causes act1v1ty ln a group of flbres larger
than those whose recept1ve f1elds overlap. Advantage could be taken of th1s
to reduce the number of impulses required to convey 1nformat1on about such a
st1mulus. Again, a s t1mulus vJ1ll often be moved across a sensory surface
caus1ng exc1tat1on in sequences of nerve flbres. Such repeated, ordered,
sequences of act1v1ty would be a form of redundancy which could be reduced
by suitable re-codlng. In fact, any pattern of st1mu11 whlch represents a
departure from complete rand~ess- such as simultaneous stlmul1 at different
points on the sensory surrace, st1mul1 whlch are ma1nta1ned for long dura-
tion of time, ordered sequences or cycles of st1mu11 - present an opportun-
1ty of reduc 1ng the magn1 tude of the sensory lnflmv by sui table re- coding.
It is clear that many of the complex reatures of our envirow~ent wlll come
1nto thls category. ~or lnstance, the stlmull whlch result from an an1mal•s
parents or 1ts hab1tat are repe~ted frequently, and economles could be
effected by reducing the space 1n' the sensory representat1on occupled by
these familiar stlmuli and allowing more Epace for the 1nfrequent and L ._
unexpected stlmul1.
It 1s suggested, then, that the process!ng or organ1sat1on of sensory
messages 1s carrled out by dev!s1ng a successlon of opt1mal or near-opt1mal
codes adapted to the messages which have been recelved. In the early stages
the total inflow will be sub-divlded 1nto many small sections, presumably
taking in each sectlon the messages coming along ne1ghbour1ng fibres dur1ng
a short interval of time. In the later stages the coded outputs Hlll be
re-mixed, possibly with the additlon of delayed lnputs (as ut111sed by
Uttley ln conditional probab111ty devlces) to allow detect1on of movement
and other ordered sequences of act1V1ty, and then wlll be sub-dlvlded aga1n
1nto small sectlons. Tims in the later stages the nerve messages be!ng
re-coded may be derlved from more ~nd more remate parts of the sensory 1n-
flow and may also come from sensory stimul1 more and more separated from
each other In time of occurrence. It will be seen that at each stage storage
or sorne or the sensory 1nformat1on is requlred in arder to construct the
optlmal code, and thus the code 1tself forms a klnd of memory.
Now the idea that our brains detect arder in the environment 1s not ne\<I.
Emp1r1c1st ph1losophers have talked of percepts be1ng assoc1ated sense
1mpress1ons, and of causal1ty corresponding to 1nvar1ant successlon or
sense 1mpress1ons. Behavlourists have emphasised the lmportance of

(94009) 4-1. P11

Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

) '

assoc1at1on, and Gestalt psychologlsts talk of orderlng sensatlon accord!ng

to certa1n schernata (though here there seems to be sorne confuslon as to ))
whether the ordered schemata are dertved from sensatlons or 1mposed upon
them). Thus the fact that our h1gher centres are much concerned wlth the
redundancy of the sensory messages has often been polnted out, but two
aspects of thls fact have not, I th1nk, been so widely recognlzed. F1rst,
the detectlon of redundancy enables the sensory messages to be represented
or d1splayed in a more compact fonn; and second, the reduct1on of redundancy
ls a task whlch can be subd1vlded and performed ln stages. Figure 2 sh0\1S
dlagramrna tlcally hovl the sugges ted se heme of s torage and display compares
wi th more ,othodox representa t1ons o r memo ry and con se 1ousness. I t w111 be
seen that 1n the present scheme a large part of the storage of 1nformat1on
occurs befare the display- that 1s befare the level or re-cod1ng which
mi gh t correspond to con se 1ous a~r1areness of sensory s t1mul1. The re-coding
1s supposed to continue at consclous levels, so sorne of the 1nformat1on
reachlng consclousness is also stored, but th1s would only be suff1-
c1ent, f1rst, to enable the process of building up the code to cont1nue,
and second to enable "useful" assoc1at1on to be made between motor acts and
reatures of the current sensory input (e.g. between sallvatlon and bell}.

'l.\
j/

Vorious filters
tronsformattons etc

Flg.~ D!agram contrastlng memory ajter consc1ousness ln orchodox scheme w1th

storage bejore display In optlmal codlng scheme.

(94009) 4-1. :p 12
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

It seems a help to consider the process1ng of sensory lnformation as

o optimal or near-optlmal cod1ng for tv:o reasons. Pract1cally, 1 t enables the
subject to be approached along the f1rm path of sensory physiology lnstead
of through the sh1ft1ng sands of conscious 1ntrospect1on and phllosophy.
And conceptually it shows a way in which complete mental acts, wh1ch seem
appalling in their compllcation and perfectlon, may be sub-d1Vlded 1nto a
success1on of much slmpler operatlons; this is clearly a prerequisi te for
ga1n1ng an understanding of the physiological basis of mental function.
It ls worth notlng that the possib111ty of sub-d1v1s1on rests on
Shannon's proof of the poss1b111ty of near-optlmal coding; 1f the early
tr.ansformat1ons of the sensory 1nformat1on were not reversible, redundant
features which are detected later m1ght be lost: and if the earller trans-
formatlons did not lncrease the relatlve entropy of the messages, they
would not fac111tate the detectlon of hlgher arder redundancy.

3. DESIRABILITY OF OPTIMAL CODING

In the last section an outllne scheme ror deal1ng w1 th the enormous

¡
sensory 1nflow ~tJas suggested. In. t.h1s sec tion sorne reasons for the des1r- ¡

(o ab111ty of optlmal coding are put forward. It will be argued that it is

desirable on the grounds or access1b111ty, stab111ty, and economy, and
l_~
because 1t requires storage of informatlon sufficlent to form a model of
~e·~ the anlmal•s environment. Of course, such arguments for 1ts des1rab111ty are
not suff1c1ent reasons for bel1ev1ng that 1t actually occurs.

(a) Accessibiltty.
i
,....
Opt1mal cod1ng wlll 1mprove the access1b111ty of 1nformatlon in two ways.
Flrst, the capaclty of the display required for the current sensory Input
w111 be decreased. Thls s1mpl1f1es the task of f1nd1ng useful assoclatlons
., justas reduc1ng the s1ze of a haystack s1mpl1f1es the task of f1nd1ng
needles. The second way 1s less obvlous. In ~essages of h1gh relatlve
entropy, the probablllty of a given message occurr1ng ls clase to the pro-
duct of the probab111t1es of the individual slgns wh1ch make lt up. Now a
dog feeds once or twlce a day, and when looking for sensory correlates of
sal1vat1on lt would not be worthwhile to search among comblnat1ons of
1ndlv1dual s1gns whose probab111ty of joint occurrence was so low that they
would be expected only, say, once a week, nor amongst those whose probabillty
was so hlgh that they would be expected, say, once an hour. If the input to
a condltlonal probab111ty device is known to be or high relat1ve entropy,
great econom1es or des1gn are posslble.

(94009) 4-1. p13

Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

~~·, ¡

(b) Stability
It 1s sometimes argued that redundancy 1s a good th1ng because 1 t pro- ' ))
te e ts a message from nolse. T'nere may v..'ell be random e r fe e ts lnslde the
nervous system agalnst wh1ch the storage and display of sensory 1nformat1on
needs protect1on, but the of the lnternal representatlon wh1~h
would ach1 eve this 1s no t 1n general the same as the redundancy vJhl eh
occurs In the sensory input. 1Ahen dr1v1ng at night the tnternal representa-
t1on of a pedestrian cross1ng the road requ1res as much protect1on as the
representation of tlle bllndlng glare frorn an oncomlng car, but 1n the 1n-
com1ng sensory rnessage the fonner may be represented by a barely s1gn1 fl cant
dlsturbance in the pattern of nerve impulses, the latter by hlgh frequency
volleys of impulses 1n many f1bres. Stab111ty of storage and display
requlre, at least, a re-adjustrnent of the redundanc,y of the sensory
messages.

(e) Economy 1.n transmissi on and storaf!e.

~ensory 1nformat1on has to be transmltted rrom place to place ln:the
central nervous system and the reductlon or redundancy befare th1s 1s done
would enable the number of .1nternal connecting f1bres to be reduced. An
example where the economy so effected seems to be part1cularly deslrable ls 1
1

the connectlon between the eye and the braln. It would clearly interfere '-:.-.
wi th the mob111 ty o f the eye 1 f the opt1c nerve ~.;as very much larger than
1t 1s, and accordlng to Jacobsen•s estlmates it would have to be fifteen
times larger lf the nerve flbres were ut111sed as 1neff1c1ently as are
1n the ear. The attainment of this 15-fold economy may, as Jacobsen
suggests, be the main runctlon of the nervous layer of the retina whlch
llnks receptors to optic nerve fibres. Squlds and octopuses forrn an 1nte-
. rest!ng compar1son, for they have eyes wh1ch are comparable opt1callY to
those or vertebrates, but the!r retina 1s much simpler with no synaptlc
layer- the optlc nerve comes dlrect from the receptor cells. It 1s bulky,
conta1nlng a vast number of fibres, and seems likely to be a factor res-
tr!ctlng the mob111ty of their eyes.
The same argument mlght be to storage of 1nformat1on, slnce lt
1s clearly more economlcal to store messages after their redundancy has
been reduced. Here, however, there ls a compllcation. The dev1s1ng of a
redundancy-reduclng code requ1res storage of certa1n propert1es of the
sensory message, and 1t has not been shown that more capac1ty would be
saved by storlng messages after re-codlng than would be utlllsed in devls-
1ng the code. The cond1t1on that thls should be so depends upon the number
of the times that the code, once devls~d, ls subsequently utlllsed, but a
discusslon of thls polnt cannot go far w1thout knmving what parts of the
sensory inflow are In fact stored: the argument of the next section ls that
the coder itself stores suff1c1ent 1nformat1on to form a worklng model of

(94009) 4-l.p 14

(
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought1 Process.
• No. 10,pp 535-539, HM Stationery Office, London

e
the animal's environment, and therefore represents a large fractlon of the
total storage the animal needs.

(d) Nodelltng the Envtronment.

Cralk suggested that sensory 1nformat1on was used to form a model of the
anlmal•s env!ronment. By a model one does not mean a simple copy of those
as pe e ts o f 1t v.tli eh have gl ven rl se to sensory s t1muli: 1 t mus t al so m1m1 e
the structure of the environment, so that an operation performed on the
model will give the same result as the analogous operation per!ormed on
the environment. When the schoolboy turns his model engine round, he
recelves visual st1mul1 similar to those he would have received if a real
eng1ne had been turned round in front of him. The model imposes restric-
tlons on the sensory st1mul1 whlch are recelved in certaln s!tuatlons, thesc
restrlctlons being the same as those 1nherent in the properties of the
object modelled. Now 1t !s preclsely these restrlct1ons- the departures
from comp~ete randomness of the sensory input- which the coder ut111ses to
1ncrease the relatlve entropy of the s1gnals. The particular code adopted
1s related to the particular rest~!ct1ons of past sensory inputs and 1s
therefore, in a sense, a model of the an!mal•s env1ronment. In the example
above, the model was statlc, but the restrlctions must often be dynamlc;
l_
sets of sensory stlmuli rrequently follow one another ln a repeated
sequence, and such repeated sequences w111 also be reflected in the parti-
cular code adopted. Thus the code contains a work1ng model or the envlron-
ment.
If the code stores suff1c1ent 1nformat1on to fonn a model of the environ-
ment, lts potential use in a1d1ng surv1val ls not conflned to the provls1on
of a more compact display of the sensary input. But to make full, pred1c-
t1ve, use of these potent1al1tles sorne add1t1onal fac111ty for gettlng at
._thls stored 1nformat1on seems to be needed. To return to the earller
example, what fac!lity do we have for turnlng round the model engine ln our
braln so that we can look at the other aspect?

4. MODIFIED RE-CODING

so rar the type of optimal codlng envlsaged has been that described by
Shannon, Fano, and Huffman, in which the output ls the smallest number of
blnary s1gnals capable of carrylng the lnformatlon of the input. At first
slght th1s seems to be what is needed in the nervous system, for nerve
l f1bres transmlt all or nothlng impulses and thus seem to use a blnary
sys tem. However, 1 t has already be en poln ted out tha t the mean frequency o f
t impulses ls well below the op timal for in fonna ti on transm1 ss ion even ln
per1pheral nerve flbres, and there ls sorne ev1dence whlch suggests that the

(94009) 4-1.p15

('
Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process.
1 • No. 10,pp 535-539, HM. Stationery Office, London
\

1
.·

mean frequency ls even lower in the more central neurones (Gal~~bos, 1954,
re!. 6'}. Furthermore 1 r the Shannon type o f re-coding was occur~ing, one )
would expect to f1nd the sensory pathvJays becomlng more and more compact as
the sensory 1nformat1on y.;as coded on to fewer and fewer elements. Th1s cloes
occur ln the retina, where sorne 10 8 sensory elements are connected to 106
nerve f!bres, but as one follows the optic nerve into the braln there 1s no
evidence of further compression on to a smaller numtJer of nerves, bu t rather
the reverse. The strlate reglan of the cerebral cortex whlch !s mainly,
perhaps exclusi vely, concerned wi th vislon, contains sorne 108 nerve cells;
ln other reg!ons of the cortex there are about 6.5 x 109 cells (Sholl 1956,
ref. ro) many of ~.,h!ch must be partlally concerned w1 th visual 1nformatlon.
Galambos (1954, ref.6) glves strlklng figures showlng how the number of
nerve fibres avallable for audltory 1nformat1on increases as one follows
the sensory pathway from ear to cortex.
These facts do not f1t in •v'/ith the idea that codlng in the h1gher nervous
system compresses 1nformat1on 1nto a smaller number of nerve f!bres, and
·suggest that, lf optlmal codlng occurs, the output 1s not in the rorm of
b1nary s1gnals at the optlmum frequency for information transmisslon.
For an engineer designing a cowmunlcatlon link, the capacity of the
channel is one of the factors under hls control, and he can effect economles l __
by cod1ng hls signals so that they requlre a smaller capac1ty. In the ner- ..,.
vous system the number of nerve flbres avallable ror a particular task must, ·) )J

to a large extent, be determlned genetically. One may expect evolut!onary_

adaptatlon to have performed part of the engtneer•s job in selectlng suit- \
l' _)
able codes ror the sensory slgnals, but such 1nher1ted cedes obviously
cannot be adapted to the redundancy of sensory input whlch 1s peculiar to
each individual. Now althou~1 the number of nerve cells ava1lable 1s prob-
, .,

ably determ1ned genet1cally, the number of impulses in the nerve cells 1s

not, and sorne or the advantages of optlmal coding would apply 1f the 1ncorrr
1ng 1nformat1on were coded- not onto the smallest posslble number of nerve
f1bres each worklng at 1ts optlmal mean frequency- but 1nto the smallest
_possible number o f impulses in a rela tively flxed number o f nerves. Thl s type
of codlng can be ep1tom1sed as economy of impulses: the nervous system w111
tend to code sensory messages so that they are represented, on the average,
by the smallest number of impulses ln the nerve cells avallable. There ls
an lmportant dlfference between this type or re-codlng and the Sha~~on-
Fano - Huffman type; the latter does not d1st1ngulsh betv.;een redundancy
caused by non-optlmal frequency of ut111sat1on of the 1nd1v1dual slgns of
the input message, and that whlch 1s caused by correlatlon bet\.veen signs.
If impulses rather than nerve flbres are economlsed, mean impulse frequen-
cies of the output w1ll be as low as the rate of 1nflow of 1nfonnat1on
pennlts, and wlll thus possess maximum redundancy or the flrst type and
~n1mum redundancy or the second type.

(94009) 4-1.p16

.{
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

l, A reversible codlng dev1ce 1s descrlbed in the appendlx \<lhich decreases

the frequency of occurrence of a pa1r of blnary output sl~ts by gett1ng
rld of sorne of the redundancy caused by correlatlons betvveen a pa1r of
b1nary input slgnals.

5. EVIDENCE

So far sorne grounds for bellevlng that the optimal coding of sensory
1nformat1on would be deslrable have been g1 ven, argulng from the enormous
quantity of lnformation pourlng in and from rather vague ideas about what
the brain does w1 th 1 t, In this section sorne of the evldence ln favour of
the v1ew that 1t does actually occur 1s sketched, but thls 1s 1ntended to
show the klnd of consequences of optimal codlng wh1ch rnay be found experi-
mentally, and 1s nelther a cla1m that lt has been preved to occur, nora
cr1t1cal rev1ew of the evidence for and against lt. The evldence comes
rrom a number of sources.

{~) Introspection of sense impressions

This ls a notor1ously unrellable way of obta1n1ng sclentifically valid
evldence, but lt 1s 1rr~ed1ately accesslble to everyone, so lt comes flrst.
Ir the hypo thes1 s 1s corree t, the sensory messages reachlng consclousness
will have been partlally re-coded, and w111 therefore have hlgher relatlve
entropy and lower redundancy than the raw sensory messages. Th1s seems to
me l1kely to be true of the furn1ture of rny own consciousness, and others
rnay feel it is true also: if, however, somebody d1d not agree I don•t think
I could persuade hlm by verbal arguments. More objectlve evldence can be
ob talned by looklng at sorne messages vvhl eh do not reacl1 conscl ousn ess bu t
whlch are knohn to be lmpressed on the sense organs. Examples of th1s are
the shadows of the blood vessels ~vh1ch run on the ret1na in front of the
sens1t1ve elements; the fact that if dlstortlng or lnvertlng spectacles are
worn, after sorne days one ceases to be aware of the dlstortlon or 1nvers1on;
adaptation to the curlous tone quallty imposed on all sounds by the average
· domes t1 e wl re le ss set and so on. In all o f these examples there are fea tu res
of the sensory rnessages whlch are constantly repeated and are therefore
redundant; a code whlch increased the relatlve entropy of the messages
mlght be expected to reduce their promlnence, and the fact that we cease to
be consc1 ous o f them suggests tha t thl s re-codlng do es take place be rore
sensory messages reach consclousness.
An experimental approach to thls problern may be poss1ble througl1 the 1n-
vest1gat1on of threshold sensatlons. These are perhaps the slrnplest ele-
•. ' J> roen ts o r our consci ousness, and accordlng to the hypothesi s they shoul d
tend to possess the hlghest posslble relatlve entropy ora b1nary signal

(94009) 4-1. P17

\
Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought1Process.
• No. 10,pp 535-539, HM . ' Stationery Office, London
'!

) """

after the physical l1ml tations of the st1mulus and or t ..'r¡e sense organs
have been taken 1nto account, and they should show a tendency to retain
this property in a great diversity of stimulus cond1t1ons.

(b) From sensory neuro-physiolofy.

Dur!ng the past th1rty years various types of relation have been
observed betvleen an applled physical stimulus and the resultlng pattern of
nerve impulses. Phys1olog1sts have perhaps got used to these transforma-
tions and no longer th1nk of them as somethlng requirlng further 1nterpre-
tatlon, but poss1bly they can be looked upon as examples or the prlnciple
ot economy of Impulses: the relation between the physlcal stimulus and the
occurrence of impulses 1s such that the number of impulses used to convey
1nfonnat1on about the st1mulus is lower than 1t would be with other, more
stra1ghtforward, relat1ons.
(i) Adaptat1on. When a sustained physlcal stimulus 1s applied to a
.sense organ the nerve flbre often responds w1 th brlef burst of impulses
which rapldly decreases in frequency and is not sustalned for the duratlon
or the phys1cal stimulus. In tr1e left half of /tf. 3 comparison of the
trains of impulses shows the economy brought about by adaptation. But 1t
can, of course, only be thought of as an economy when compared to a non-
adap t1ng endlng, and e ven then only When the physical S tlmul1 na turally
applled to the sense organ are frequently of a long-sustalned type.
e
Nevertheless, where 1 t occurs, adaptatlon would lead to economy of impulses ',~) )
and Adrlan (1928, ref. 1), suggested that lts functlon mlght be to prevent
an excesslve number of impulses reachlng the nervous system.
(ii) Inue~sion. It can be seen.from the r1ght half of !ir. 3 that an
adaptlng nerve flbre fails to slgnal the end of a sustalned stlmulus. This
derect could be remedled by hav1ng one whlch discharged as sho\\n 1n the
bottom l1ne, and such nerve flbres are found. In the eye of the scallop
(Pecten) Hartl!ne {1938, re/.9) sho~ed that one group of fibres discharged
when a 11 gh t was swl tched on and ano ther group o f f1 bres di scharged a t
•orr•. A similar, but rather more complex, situatlon 1s found in the
vertebrate eye (Hartllne 1928, ref, a; Gran1 t, 1947, re!. 7). This arrange-
ment mlght be thought of as making good sorne of the loss of information
caused by adaptat1on.
(iii) Lateral inhibition. Adaptatlon lncreases the relatlve entropy of
the nerve message by preventlng many impulses be1ng used to signal a phys1-
cal stlmulus whlch 1s constant in t~me. It 1s clear that physical st1mul1
w111 o ften be appll ed to many ne1 ghbourlng receptors s1mul ta.fleously, so
there is a place for a spat1al analogue of adaptatlon. The best worked out
example of this occurs in the lateral eye of Llmulus, wr1ere the arrangemen t
shown 1n fig. 4 has been deduced by Hartllne and hls cO-h'Orkers (re!. lO}.

..(94009) 4-1. P18

 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London
1 •

e
o
Time.
-->
~.S_t_im__u_lu_s_.____~f ~~-----
No adaptation. lllffl((fllllll
Adaptation. JlllLJ
nversi on.
o J
111111
tf'
"'/ F1g.3. D1agram showlng that adaptatlon leads to economy or impulses when a
physlcal stlmulus 1s of long duratlon, and that 1nvers1on replaces
lnformatlon lost by adaptatlon.

Apparen tly each receptor ln tlle array exerts an lnfluence, graded ac;;ord-
1ng to the number of impulses 1 t ls 1 tself producing, \oJhlch reduces the
number of impulses given by nelghbourlng receptor unlts. It wlll be seen
that the effect is to decrease the number of impulses coming from a
unlformly lllumlnated area, while the number comlng from the borders of
the area are relatlvely unaffected. A similar situation ex~sts in the frog
(Barlow 1953, re f. 2) and ca t retina (Ku ffler, 1953, re!. 15) and 1 t has
al so been descr1 bed in the audl tory (Galambo s 1944, ref. 5) and tac tlle
(Mountcastle, 1957 ref. 17; Amassian, 1958, re/.21) pathways.
One reature of lateral 1nh1b1t1on in the ma111Iflalial1 retina ls of speclal
lnterest: it ls found when the retina ls adapted to a un1form background
11ght, but is absent after complete dark adaptatlon (Barlow, FltzHugh,
and Kuf fler, 1957, re!. 3). now 1 t 1s only when the un1 form background is
present that the correlated discharge or ne1ghbourlng receptors w111 tend
to occur, so lt looks as though lateral 1nh1b1tlon 1s notan 1nvar1ant
feature of the retlnal organ1sat1on, but develops in the condl t1ons '.Nhere
1 t can 1ncrease the relat1 ve en tropy of the opti e nerve s1 gnals. Perl1aps

(94009) 4-1.p19

1
Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

F!g.4. Dlagram showlng that lateral 1nh1bltlon leads to economy of Impulses In a

un1 formly 1llum !na ted area.

1 t 1s a simple example of "l.earn t 11 re-coding adapted to the redundancy

which 1 s presen t.
Adaptatlon, inverslcn and lateral 1nll1b1t1on may thus be devices used in
the perlpheral parts of sensory pathways to obtain signals of h1~~er rela-
t1ve entropy. It ls now posslble to record the act1v1ty of more centrally
placed neurones, but the nervous system has outwltted the physlologlst who
has so far be en unable to de tenn!ne trte func t1on o r the cells whose nervous
responses he has recorded. The model descrlbed ln the appendix does a
simple re-codlng operatlon on two b1nary 1nputs, but 1t would be a d1ff1cult
task to relate the output to past and present 1nputs w1thout sorne h1nt
about the purpose or the device. The reason, then, for puttlng forward the

{94026) 4-1.p20
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

()
. optlmal re-coding hypothesls ls the hope that 1 t may be better mat.ched to
o the subtlety or the nervous system than the simpler hypotheses at pres~nt
entertalned in physlology.
!

t 6. INTELLI GENCE
¡
6
t Thls word was added to the title in an 1ncaut1ous moment, but there are
reasons just1fy1ng 1ts 1nclus1on. If it is accepted that the large size of
thesensory 1nflow precludes 1ts dlrect ut111sat1on in the control of
learnt motor actions, then the mechanlsrn wl11ch organlses thls !nfonnatlon
rrrust play an important part in the production of 1ntell1gent behaviour.
In add1t1on, when one cons!ders the two maln operations requlred for
opt1mal coding there ls a striklng parallel wlth the two types of reasoning
whlch underlie in telligence.
The outputs of a code can be thought of as logical statements about the
input, and, if the code is reversible, these logical statements, taken
together, are suff1c1ent to determine the exact input. Formlng these state-
ments and ensurlng that they fulfil this condition are stralghtforward pro-
blems of deduct1ve logic. If ·the code is optimal, the output statements
must be chosen so that they fulfil the additional condltion that, on the
average, they are the smallest possible number which sufflces to detennine
the input ( for the type o f madi f1ed optimal code sugges ted in Se ct1 on 4,
the additional cond1t1on ls that a fixed number of posslble statements are
chosen fo r the ou tpu t 1n su eh a way tha t th e small es t num be r, on the
average, are asserted as true). The ful filllng of these addi tional cond1-
t1ons 1s not exactly inductive reasoning, but it 1s closely related to lt, ---·>
1

for both depend on counting frequencies of occurrence of events. Hav1ng

been presented w1 th 1000 whi te swans and no black ones, the relevan t parts
of a code would say "henceforch regard all sv.1ans as whl te unless told
otherw1se". Induct1vely one would say "all swans are white". Tne tools of
logj cal reason 1ng appear to be the same as those needed for op t1mal cod 1ng,
so perhaps they can be regarded as the verbal expression of rules for
handling facts v:hich our nervous system uses constan tly and au toma ti cally
to reduce the barrage of sensory impulses to useable proport1ons.

Flnally lt should be made clear that the transformations of sensory

messages taking place in the nervous system must, in fact, fall a long way
short of true optlmal coding: lnformation must be lost, and the final
"dl splay 11 mus t s till con tain redundancy. r-:ov1ever, the rae t tha t the 1mage
cast on the retina is uot always sharp does not mean that the focussing of
llght by the eye 1s unlmportant, and the suggestlon is that opt1mal coding

{94009) 4-1. P21

Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process.
1 • No. 10,pp 535-539, HM Stationery Office, London

plays a part ln the organ1sat1on of sensory 1nformat1on comparable wlth

lmage format1on in the worklng of the eye. Hm>Jever, even 1 f thls conj ecture
ls correct, the means by whlch 1 t 1s achieved, and such matters as the
classes of redundancy whlch are easily and naturally ut111sed, and the
classes which are not, rema1n largely undeterm1ned.

APPENDIX
(In collaboratlon w1 th P. F. K. Dona.ldson)

Object of device. To code reversibly and w1thout delay a pa1r of blnary

1nputs (A and 8) onto a pair of b1nary outputs (X &nd YJ so that the redun-
dancy of the output due to correla.tlons is less than the same type or
redundancy in the input.

Principle used. The 1nformat1on carried by the lnputs wlll, in general, be

less than the capacity of the input channels first because of redundancy
due to correlat1ons between them (P(ABJ f- P(A). P(B)) .· second because the
frequency o f s ignals ln the ind1 vi dual channel s is no t optimum (P( AJ -:f i and
P(BJ i ~). The pr1nc1ple used !s to lncrease the redundancy of the second
type~ and so decrease that of the first type. A palr of outputs are sought
L
Wh1ch are revers1bly related to the inputs, and one or which occurs wlth
probablll ty further from the optlmum (~) than one, or bo th, of the inpu ts.
The outputs carry the same 1nformat1on as the 1nputs, so that 1f such a pair l
can be round, the redundancy due to correlat1on between them must be less
than 1s presen t in the 1nputs.

Possible Codes. There are four possible input states (AB,iii. As, and AB),
and rour possible output states (XY,xY, Xr, and Xi?. If the code ls rever-
sible these must be related to each other in a one-to-one manner, wh1ch can
be done ln 24 ways. Now slnce X corresponds to a pair of output states
(XY + X11, the condltion for activity ln X must be the occurrence of either
or a pair of the poss1ble input states, and 11kew1se for ~ There are slx
such palrs:- AB + A.B::: A, AB + .48 = }{ AB + AB = B, AB + AB = B, AB + AB =
(A and B the same), and i8+ As= (A and B different}. In add1t1on, for
revers1bil1ty, the two palrs chosen must have a common member, for if this
was not so X would always be active when Y was not active, and vice versa.
After a llttle cog1tat1on 1t wlll be found ·that there are 24 possible
codes, whlch fall 1nto 3 groups each contain1ng 8 codes, the groups dlffer-
lng from each other in the respect which lnterests us, namely the divislon
or redundancy between correlation-type and non-optimal-frequency-type. one
group does not d1ffer from the input ln th1s respect. The other two groups
do differ, and they are made up of those 16 codes for \~lch one or other of

(94009) 4-1. P22

 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
.
NPL Symposium on the Mechanization of Thought\ Process. No. 10,pp 535-539, HM Stationery Office, London

¡·
1

the outputs corresponds to elther AB + AB (A and 8 alike) or AB + AB (4 ;md

() B di fferent).

Condition for succt:ss, then, ls that elther P(AB + ALJ}, or P(AB + AEJ, should
d1ffer from ~ by more than one or other or both of P(A) and P(B). Thls 1s
not, of course, the same as the conditlon that A and B are correlated, so
the recodlng does not always reduce correlation redundancy vJhen r,:nls ls
presen t. Success t.'ul recodlng occurs for the smallest departures from zero
correlatlon when el ther P(A) or P(BJ 1s clase to -k·

Method. The devlce 1s made up of 6 similar units each of whlch compares two
probab111t1es and operates a relay accordlng to \<lhlch 1s greater (see cir-
cul t dlagram, fiJ,r. 5}.
(a) Probab111t1es are measured by charglng a leaky condenser when A (or
B etc.) = 1; hence they are we!ghted for recent events, the we1gr1ts
decPeaslng exponentlally w1 th lapse of time. These time weighted probabi-
11t1es are called P' (A), P' (AB + ABJ, etc.
(b) P' (AJ is compared vli th P' (ji), P' (BJ wl th P'BJ, and P' (AB + ABJ vll th
P 1 ( AB + As). In ea eh case a. ?1gnal correspondlng to the srnaller o f the
palr 1s selected. Call these slgnals K, L, N. '
L
) o
r·
1
!
'

Ftg. 5. e1rcu1t dlagram or recod!ne; de vice.

(94009) 4-1. p23

Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

(e) P'(KJ is compared w1th P1 (L), P1 (L) w1th P'(N), and P 1 (N) with
P' (K). Sw1 tching is performed accordlng to the resul t of these compar1- /"
sons so that '
X= smallest of K, L. N.
=
Y next smallest of K, L, H.
Result. The result of these operations 1s more spec!flc than the original
objectlve in that one particular code ls chosen from a group or 8, any one
of Whlch would have met the requlrements. 'Ihe added spec1f1c1ty results
from the fact that we have chosen outputs which occur least frequently, not
most frequen tly, and ha ve arranged that P(X) shall be less than P(Y).
Note that lf there is any log!cal relatlon ln the 1nputs (e.g. AB =O),
then the outputs become mutually exclusive (P(XYJ :: O}. If there 1s a
double relatlon (e.g. AB =o and AB O), then only one output channel
operates (P(XJ =
O). The devlce might be roughly described as one wh1ch
determines lnductlvely what log1cal relatlons, lf any, are obeyed by lts
input. If two such relations are found, one output channel !s not used;
1 f one 1s found, the two ou tpu ts become mu tually exclusl ve; 1 f non e 1s
·round, but there 1s statlstlcal correlatlon between the lnputs, lt w111
sometimes find outputs which are less correlated.

REFERENCES

1. ADRIAN, E. D. The Basis o f Sensat!on. Christophers, London, ( 1928) •

2. BARLOW, H. B. Summat1on and Inh1b1t1on in the frog•s retina. J.Physiol.,
1953, 119, 68.
3. BARLOW, H. B. FITZHUGH, R, and KUFFLEB, S. W. Change of organ1sat1on in
~he receptlve flelds of the cat's retina durlng dark adaptat!on.
J.Physiol., 1957, 137, 338.
4. CRAIK, K. J. w. The Nature of Explanatton. University Press, Cambridge,
(1943} •
5. GALAMBOS, R. Inh1b1tlon of actlvity in single audltory nerve fibres by
acoust!c stlmulatlon. J.Neurophysíol., 1944, 7, 287.
6. GALAMBOS, R. Neural mechanisms of aud1t1on. Physiol. Rev., 1954, 34,
497.
7. GRANIT, R. The Sensory Mechanlsms of the retina. Oxford University
Press, Oxford, (1947).
8. HARTLINE, H. K. The response of single optlc nerve f1bres of the verte-
brate eye to 1llumlnatlon of the retina. Amer.J.Physiol., 1938, 121,
400.
9. HARTLINE, H. K. The dlscharge of impulses In the opt!c nerve of Pecten
1n response to 1llum1nat1on of the eye. J.Cell.Comp.Physiol., 19.;e,
11, 465.

(94009) 4-1.p24
 Barlow HB (1959). Sensory mechanisms, the reduction of redundancy, and intelligence.
NPL Symposium on the Mechanization of Thought Process. No. 10,pp 535-539, HM Stationery Office, London

{'•

10. HARTLINE, H. K. and RATLIF~, F. Inhlbltory 1nteract1on of receptor

í" J~--~- unlts in the eye of limulus. J.Gen.Physiol., 1957, 40, 357.
1
~·
·"" 11. HICK, W. E. Why the human operator? Trans.Soc.Inst.Technol., 1952, 4,
67.
12.
·- HUFF11AN, D. A. A me thod o f cons tructlon o f m1n1mum redundancy codes.
Symposium on Communlcatlon Theory pp.102-110. Editor w. Jackson.
Butterworths, London, ( 1953} ..
;{
1 13. JACOBSEN, H. 'Ihe informatlonal capac1 ty of the Human Ear. Science,
1950, 112, 143.
14. JACOBSEN, H. The lnformational capaclty of the Human Eye. Science,
19 51, 1 13' 29 2.
15. KUF'Fl.ER, S. H. Dlscharge pattems and functlonal organisation of
mammallan retina. J.Neurophysiol., 1953, 16, 37.
16. MACKAY, D. M. and McCULLOCH, W. S. The 11m1t1ng 1nformat1on capac1ty
of a neuronal link. Bul.Nath.Biophys., 1952, 14, 127.
17. MOUNTCASTLE, V. B. Modal1ty and topographlc propert1es of single
neurones of the cat•s somatlc sensory cortex. J.Neurophystol., 1957,
20,. 408.
18. QUASTLER, H. Studies of human channel capacity. Th1rd London Symposium
on Informatlon Theory. pp.361-371. Editor c. Cherry Butterworths, '
London, ( 1956) • l_
~.
...!
'C\ ,•
19. SHA\ll'WN, c. E. and WEAVER, w. Tl1e mathematical theory of
communlcatlon. The University of Illinois Press, Urbana, (1949).
~· 20. SHOLL, D. A. 'Ihe organlzatlon of the cerebral cortex. Xe thuen,
London, ( 1956) •
21. TOWE, A. L. and AMASSiki, v. E. Patterns of act1v1ty in single
cortical un1 ts follow1ng stimulatlon of the d1g1 ts in monkeys.
J.Neurophys iol., 1958, 21, 292.
22. UTTLEY, A. M. The cond1t1onal probablllty of signals in the nervous
system. R. R. E. Nemorandum 1109, (1954). M1n1stry of SUpply,
Gt. Mal vern.
23. UTTLEY, A. !1. The classi f1cat1on of slgnals 1n the nervous system.
E.E.G. Clin. Neurophysiol., 1954, 6, 479·.

(94009) 4-1.p25