Vergence Control in a Binocular Vision System using

Weightless Neural Networks

Karin Satie Komati and Alberto Ferreira De Souza
Departamento de Informática – Universidade Federal do Espírito Santo
Av. Fernando Ferrari, S/N, 29060-970 – Vitória – ES – Brazil, Tel. +55 27 3335 2641, Fax +55 27 3335 2650
{karin, alberto}@inf.ufes.br

Abstract movements of the eyes, which allow us to inspect the

This paper presents a methodology that we have
developed and used to implement an artificial binocular
vision system capable of emulating the vergence eye
movements. This methodology involves using Weightless
Neural Networks (WNNs) as building blocks of artificial
vision architectures. Using the proposed methodology, we
have designed several WNN based architectures in which
images captured by virtual cameras are used for
controlling the position of these cameras’ “fovea” (high
resolution region of the cameras’ image). Our best
architecture is able to control the foveae vergence
movements with average error of only 3.6 image pixels,
which is equivalent to an angular error of approximately
0.615o.
environment for creating this representation. So, we have
started our research effort modeling the oculomotor
system. We have chosen WNNs as building blocks for
modeling because they are simple, but powerful, highly
flexible and have fast learning algorithms, which allow
the study of large networks using standard computers
currently available.
In this paper we present a system that models the
human vergence oculomotor subsystem [2]. Vergence
refers to the disjunctive eye movements whereby the eyes
move symmetrically in opposite directions to ensure that,
for the object of interest, the image in each eye falls on
the fovea (central part of the retina) [1].

1. Introduction
Our visual perception of the world is egocentric, i.e.,
centered in our perceived position in relation to the
environment. Normally, we do not even notice that. It
would be awful, however, if our perception were
oculocentric, i.e., centered in the position of our eyes in
relation to the environment. In such case, we would
perceive the world moving with the movements of our
eyes. Nevertheless, in an apparent paradox, the visual
brain areas that first analyze the attributes of the visual
brain input possess an accurate oculocentric neural Figure 1. Simple neural architecture
organization [1]. Still, when moving our eyes, we feel that
In our implementation, images captured by virtual
the exterior world is steady. This shows that the human
cameras are used for controlling the position of these
visual perception depends not only on the image on the
cameras’ “fovea”. The control signals for positioning the
retina, but also on the knowledge of the position of the
foveae are generated by WNNs, which receive as input
eyes, head, and body. Therefore, the oculomotor system
the images captured by the cameras. To implement this
influences the visual perception and information
vergence system, we have used current knowledge about
regarding its status is important for visual perception of
the neurophysiology of the biological visual system and a
the world [2].
software tool we have developed to emulate WNNs
This paper presents the first step of a large research
architectures. This tool, named MAE (Portuguese
effort whose main objective is to implement a system,
acronym for Event Associative Machine), allows
based on artificial Weightless Neural Networks (WNNs
simulation of complex WNN architectures with hundreds
[3, 4, 10]), capable of emulating the human biological
of thousands of neurons in a standard PC running Linux.
visual system capacity for creating a steady internal
An example architecture of vergence control system we
representation of the environment. We believe that the
have studied using MAE is shown in Figure 1, while
oculomotor system plays a significant role on building
Figure 2 shows a screen shot of MAE running the neural
this internal representation, since it is responsible for the
architecture of Figure 1.
 In Figure 1, Image_left and Image_right are images
captured by two virtual cameras positioned in distinct
places in the virtual space to emulate the physical
separation of the eyes. The symbol ƒ denotes two-
dimension filters, denotes two dimension neural layers
and denotes outputs that allow the MAE user to
observe the status of a neural layer.
Figure 2. MAE screen shot of a simple neural
architecture
We have implemented and tested several neural
architectures like the one shown in Figure 1, using filters
and trained neural layers to emulate neurophysiological
properties observed in the human biological visual
system. Our best performing architecture produces
vergence movements with average error of only 3.6 image
pixels, which is equivalent to an angular error of
approximately 0.615o.
Figure 3. Eye, LGN, V1
2. Biological Vision
Vision starts with two external sensors, the eyes. The
retina, situated at the back of the eyeball, is where the
image of the outside world is captured. The neurons of the
retina transform light energy into electrical signals that
are transmitted to the brain via the optic nerve. The retina
ganglion cells, which drive the optical nerve, can be
separated into two categories: parvocellular (small cells)
and magnocellular (large cells). These two categories of
cells are the starting point of parallel pathways that can be
identified throughout most parts of the brain devoted to
visual processing [1].
2.1. Visual Pathways
The optical nerve connects to the lateral geniculate
nucleus (LGN) of the thalamus and this to the visual
cortex. The visual cortex is usually divided into 5 separate
areas, V1-V5 [1]; the LNG connects to the primary visual
cortex, or V1 (also called striate cortex, Figure 3). V1
possesses cells that detect several basic attributes of the
visual stimuli. There are neurons that detect stimulus that
have a certain angular orientation, others are sensitive to
color, others to disparities between stimulus coming from
each eye, etc.
From V1, the visual neuronal impulses are directed to
V2. V2 projects to V3, V4, and V5, and each of these
areas sends information to any of 20 or more other areas
of the brain that process visual information. This general
arrangement can be subdivided into three parallel
pathways, which start in the parvocellular and
magnocellular cells of the retina (in V1, the parvocellular
pathway is divided into two, parvocellular interblob and
parvocellular blob, as can be seen in Figure 4). Although
each pathway is somewhat distinct in function, there is
intercommunication between them.
The magnocellular cortical pathway starts in the layer
4Cα of V1 (Figure 4). This projects to layer 4B, which
then projects to the thick stripes of V2. Area V2 then
projects to V3, V5 (middle-temporal cortex or MT), and
to the medial superior temporal cortex (MST). This
pathway is an extension of the magnocellular pathway
from the retina and LGN, and processes visual detail
leading to the perception of shape in area V3, and
movement or motion in areas V5 and MST. Retina’s
magnocellular ganglion neurons show a high sensitivity to
contrast, low spatial resolution, and high temporal
resolution or fast transient responses to visual stimuli.
These characteristics make the magnocellular branch of
the visual system especially suitable to quickly detect
novel or moving stimuli.
In the second visual pathway, neurons of the
parvocellular interblob areas of V1 project to the pale
stripes of V2 and these to the inferior temporal cortex
(Figure 4). This pathway possesses several feature
detectors (simple, complex and hypercomplex cells) and
is responsible for complex form recognition.
Finally, in the third and mixed visual cortical pathway,
neurons in the blobs of V1 project to the thin stripes of
V2. The thin stripes of V2 then project to V4. Area V4
receives input from both the parvo- and magnocellular
pathways of the retina and LGN. This parvocellular
portion of V4 is particularly important for the perception
of color and maintenance of color perception regardless of
lighting (color constancy).
 Retina’s parvocellular ganglion neurons show a low
sensitivity to contrast, high spatial resolution, and low
temporal resolution or sustained responses to visual
stimuli. These cellular characteristics make the
parvocellular visual pathways especially suitable for the
analysis of detail in the visual world.
These separated parallel pathways decode/extract the
elementary properties of the objects in the visual field,
such as color, form and orientation, producing a rather
complex internal representation of what we see. However,
this representation is not captured instantly: we examine
our surroundings all the time, focusing our attention in
different parts of the field of view through the movement
our eyes. We believe that the mind creates this internal
representation in a way similar to a painter working on his
canvas, where the eyes serve as the painter’s brush.
Figure 4. Visual parallel pathways
2.2. Oculomotor System
Several image-preprocessing operations are performed
in the retina, and two are of interest for this work. First,
the retina’s ganglion cells act like contrast discriminators,
being rather insensitive to uniform light. Second, the
amount of ganglion cells devoted to different areas of the
retina is larger at the fovea and decreases in the direction
of the borders following approximately a gaussian density
distribution. This is perhaps one of the main reasons for
the existence of ocular movements.
We have five types of eye movements that put the
foveae on a target and keep they there. They are listed in
Table 1 [2]. The first four are conjugate movements: both
eyes move the same amount in the same direction. The
fifth – vergence – is disconjugate: the eyes move in
different directions and sometimes by different amounts.
Thanks to the vergence oculomotor subsystem, when
we look to an object that is near or far from us, each eye
moves differently to keep the image of the object aligned
precisely on each fovea. If the object is near, the eyes
converge; if the object is far, the eyes diverge. The
difference in retinal position of an object in one eye
compared to its position in the other is referred to as
retinal disparity, and the detection of this disparity is
important for vergence control.
Table 1. A Functional Classification of Eye
Movements
Movement Function
Movements that stabilize the eye when the head moves
Vestibulo-ocular for brief or rapid head rotation
Optokinetic for sustained or slow head rotation
Movements that keep the fovea on a visual target
Saccade Bring new objects of interest onto the fovea
Smooth Pursuit Holds the image of a moving target on the
fovea
Vergence Adjusts the eyes for different viewing
distances in depth
2.3. Disparity Sensitivity in the Visual System
Numerous studies made on monkeys showed that a
large population of visual cells is capable of encoding the
amplitude and sign of horizontal disparities. Horizontal
disparity detectors were found in cortical visual areas V1,
V2, V3, V5 and MST. According to their disparity tuning
functions, which describe how these cells respond to
stimulus with different disparities, these cells can be
classified into several groups [5]. Figure 5 shows six such
groups: tuned near (TN), tuned excitatory (TE), tuned far
(TF), near (NE), tuned inhibitory (TI), and far (FA). We
will only discuss in more details the TN, TE and TF cells
because we have created models of them and used these
models in our vergence control system.
Figure 5. Disparity detection cells [5]

TE cells have a disparity tuning function that displays
binocular facilitation over a narrow range of disparities
around zero and binocular suppression for negative or
positive disparities (see Figure 5). That is, they have a
positive peak at zero disparity, are symmetrical around
zero disparity and have a narrow tuning width. TN and TF
cells have disparity tuning functions similar to TE cells
but peak at negative or positive disparities, respectively
(Figure 5).
In addition to helping control vergence movements,
disparity sensitive cells, together with cells that report the
position of the eyes and others responsible for visual
memory, are possibly important for creating internal
representations of form and distance of objects in the
visual field [6].
3. Weightless Neural Networks
WNNs [3] are based on artificial neurons that have
binary inputs and outputs and no modifying weight
between neurons. Each such neuron has a look-up table,
which can be implemented using Random Access
Memory (RAM). A neuron’s binary input is the address
of an entry of its look-up table, and the look-up table
entry value is used to compute the neuron’s output.
Instead of adjusting weights, WNN training consists of
changing the contents of look-up tables’ entries, which
allows highly flexible and fast learning algorithms. The
flexibility comes from the WNN neuron capacity of
implementing any logical function of its inputs and the
expedite learning from the mutual independence between
look-up table entries, which permits changing an entry
without interfering with others.
In the middle of the 60s, Igor Aleksander [7] suggested
universal logic circuits, which can be implemented with
RAM, as neurons of WNN. After that, other weightless
models were proposed but, until the late 80s, all of them
only generalized at the network level (individual neurons
did not have generalization properties). To overcome that,
Aleksander [8] suggested the inclusion of a spreading
phase, responsible for generalizing information from the
training set through storing clusters of extra slightly
changed input patterns having each training input as
centroid. Generalizing neurons are known as Generalizing
RAMs (GRAMs).
GRAMs are often implemented as Virtual GRAMs
(VGRAMs) [9]. A VGRAM is a GRAM in which space is
not allocated for patterns that are never found during the
training phase and generalization is achieved through
calculation. During the training phase, a VGRAM neuron
stores each input pattern and the correspondent desired
output into its look-up table. During the recall phase,
input patterns are compared with each input of the stored
input-output pairs. If there is an exact mach, the neuron
responds with the stored output; otherwise, one of two
possible outcomes occurs: (i) the neuron responds with
the output of the nearest stored pattern, or (ii) with a
random value if there is more than a single stored pattern
at the same Hamming distance of the input.
VGRAM neurons can be made sensitive to color or
shades of gray [10]. This is achieved using neuron inputs
dedicated to a color or gray level. These inputs respond
with a binary 1 when receiving a value equal or near its
color or gray level, and with a binary 0 otherwise. The
binary outcomes of these inputs are the VGRAM neurons
input patterns.
In this work, we use VGRAM neurons to implement
our WNN based architectures for controlling vergence.
4. The MAE Tool
The Event Associative Machine (MAE) is an open
source framework for modeling VGRAM WNNs we have
developed. MAE is similar to the Neural Representation
Modeller (NRM), developed by the Neural Systems
Engineering Group at Imperial College London and
commercialized by Novel Technical Solutions [11]. MAE
differs from NRM on three main aspects: is open source,
runs on UNIX (currently, Solaris and Linux), and uses a
textual language to describe WNNs.
MAE allows designing, training and analyzing the
behavior of modular WNNs whose basic modules are two
dimension neural layers and two dimension filters. Neural
layers’ neurons can have several characteristics (a type,
input sensitivity, memory size, etc) and the user, using C
programming language, can freely design filters. The user
specifies these modular WNNs using Neural
Architectures Description Language (NADL). NADL
source files are compiled into MAE applications, which
have a built in graphical user interface and a Control
Scripts Language (CDL) interpreter. The user can train,
recall, and alter neural layers’ contents using the graphical
interface or scripts in CDL. NADL and CDL allow the
creation of integer variables and the use of for, while and
do-while C like loops, which enable the creation of
elaborate WNNs and powerful scripts for analyzing them.
5. Methodology
5.1. The Retina Model
To implement our vergence control system we have
created a simple retina model to provide visual input
preprocessing. An artificial retina that follows this model,
as the biological counterpart, is sensitive to contrast and
provides high resolution at its center (fovea) and low
resolution at its borders. This retina model was built using
a MAE filter named “gauss” filter (Figure 1), which
receives as input the image that enters the virtual cameras
(Image_left and Image_right in Figure 1) and produces as
output an image that maps the input image following a
gaussian distribution centered in a point on the input
image. Our model produces a very distorted version of the
input image, as shown in Figure 6, but possibly similar to
the parvocellular output of the biological retina. We use
two retinas to allow binocular vision.
Figure 6. Test image and the corresponding gauss
filter output
Our gauss filter was implemented using a 9x9
difference of gaussians (DOG [12]) kernel. The
application of this kernel to a region of the input image
results in an approximation of the DOG model of retinal
ganglion cells. However, the convolution of the kernel
with the input image alone would not produce the result
shown in Figure 6, since this image processing technique
(convolution) is applied linearly over the image [13]. So,
instead of applying the kernel linearly, in order to emulate
the correlation between ganglion cells and different areas
of the retina, we have applied the DOG kernel according
to a gaussian distribution centered at the point in the input
image where to each virtual camera’s fovea is pointing.
Figure 7. TE filter input images
a) b) c)
Figure 8. TE filter output images. a) Output image
with displacement parameter equal zero. b) Output
image with displacement parameter equal 2. c) Output
image with displacement parameter equal –2.
5.2. The TE Filter
We have also modeled TE, TN and TF disparity
detection cells (see Subsection 2.3). They have been
developed as the MAE TE filter, which receives as input
the left and right retinas’ images produced by the gauss
filter. According to an integer displacement parameter,
the TE filter emulates a TE, TN or TF layer of cells. The
operation performed by this filter is very simple: each
output pixel is the average of corresponding pixels of
each input image. If the integer parameter is zero, the
input images are aligned and the filter emulates TE cells.
If the displacement parameter is positive, the input images
are displaced a number of pixels – the left image to the
left and the right image to the right – equal to the
parameter value. This emulates the TN cells, since points
of objects near to the virtual cameras then the point to
where their foveae are aimed to tend to coincide if the
images are displaced this way. If the parameter is
negative, the input images are displaced a number of
pixels equal to the parameter value but in the opposite
direction. This emulates the TF cells.
Figure 7 shows the outputs of the left and right virtual
retinas obtained from the input images shown in Figure 2
(please note where to each fovea is pointing), while
Figure 8 shows examples of TE, TN and TF neuron
layers.
5.3. The Minus Filter
Another filter we have used was the minus filter. It
receives two images and return one, where each pixel is
calculated in following way: if the value of a pixel f in the
first image is equal or larger than the value of the
corresponding pixel in the second image s, the output is
equal to f – s, otherwise it is equal to zero.
5.4. The Virtual World
The virtual world we have used in our experiments
was developed using OpenGL [14]. The initial code was
downloaded from www.linux.org/apps/AppId4310.html,
and was then modified to produce an image stereo pair as
the one shown the Image_right and Image_left windows
of Figure 2. The images are two-dimensions projections
of a 3D model of a robot arm. These images are
approximately equivalent to those that would be seem by
a human observer (7 cm eyes separation) one meter away
from a one meter high similar robot arm.
5.5. The WNN Vergence Control System
Our WNN vergence control system receives as input
several filtered versions of the left and right input images
and generates as output a value to be added to the current
horizontal position of a single virtual camera’s fovea. In
our model, the right virtual camera is the dominant
camera and decides (actually, a human operator) where to
look, while the left virtual camera is controlled by the
WNN vergence control system. The only “motor neuron
function” emulated is the horizontal eye movement. So,
the controlled virtual camera’s fovea goes to the left or to
the right of its previous position depending on the control
system output signal. However, in order to generate the
correct output, the WNN must be trained.
The WNN training was performed according to the
following simple algorithm:
1. The operator chooses a point and aims the
dominant (right) camera’s fovea to it (using a
mouse click).
2. The system automatically moves the controlled
(left) camera’s fovea the by the same amount,
possibly positioning it in an incorrect point due to
a wrong vergence angle for the chosen point.
3. The operator informs the training system (using a
mouse click) what should be the correct position
of the controlled camera’s fovea.
4. The system trains the WNN to output a pattern
suitable for moving the controlled camera’s fovea
to the correct position.
5. The system allows the WNN to move the
controlled camera’s fovea to the correct position.
6. Go back to step 1.
We have implemented this algorithm using a CDL
script (Section 4) and used it to train several WNN based
vergence control system architectures. All networks were
trained using 32 selected points.
In the recall mode, the system acquires images,
calculates the filters’ outputs, the WNNs’ outputs, and
then generates the control signals that move the controlled
camera’s fovea. Note that the fovea movement modifies
the output of the controlled camera’s retina, closing the
control loop. The recall mode procedure was implemented
with a CDL script and repeated 50 times for each testing
point. 24 unknown points (not used during training) were
used for testing the performance of the system.
5.6. Performance Parameters
We have used two performance parameters: average
error and the number of unstable points. The average
error is the average number of pixels the fovea is away
from the correct point for 40 out of the 50 recall script
iterations – we have excluded the first 10 iterations for the
system uses it to incrementally move the controlled fovea
to the target point. With some testing points, an
architecture may not be able to stabilize vergence and
may move the controlled fovea to the limit of the input
image. We count these points as unstable points.
5.6. Architectures Characteristics
Many architectures were implemented, but all of them
can be described by three main characteristics: type of
neuron, size and type of the inputs to the control neural
layers, and type of control strategy.
We have used two types of neuron: the default and
rnd_mem_zrizro neurons. The default neuron responds
exactly as in the literature, but the rnd_mem_zrizro
neuron outputs zero when all inputs are zero.
The virtual cameras’ images, after being pre-processed
by the retinas, can be filtered in five different ways,
producing five possible inputs for the vergence control
neural layers: i) TE, or the output of the TE filter with
displacement equal to zero; ii) TE1, the output of the TE
filter with displacement 2; iii) TE2, the output of the TE
filter with displacement -2; iv) right_minus_left, or the
output of the minus filter with the right retina minus the
left retina as inputs; and v) left_minus_right, which is the
opposite of right_minus_left. The number of connections
between each vergence control neuron and each of these
inputs can vary. In addition, the way these connections
are distributed throughout the input can also vary. We
have implemented two types of connection distributions:
gaussian (G) and random (R). In the gaussian connection
type, the neuron’s inputs are connected according to a
gaussian distribution centered in the point of the input that
corresponds to the neuron position in the control layer –
this type of connection tries to emulate a retinotopic
mapping. In the random connection type, the neuron’s
inputs are connected to the corresponding input randomly.
Two types of control strategies were implemented:
ResetY and Set2Layers. In ResetY strategy, a single
vergence control neuron layer is used. This layer is
divided in two halves, left and right. The number of active
neurons on the right minus the number of active neurons
on the left is the amount added to the current horizontal
fovea position. In Set2Layers strategy, two neuron layers
are used; one is trained to positive movements (to the
right) and the other to negative movements (to the left).
The number of active neurons on the right layer minus the
number of active neurons on the left layer is the amount
added to the current horizontal fovea position. On both
strategies, during training the control neural layers are
recalled “seeing” the image of the wrong vergence
position before training. Previous learned information or
random values are output according to the VGRAM
neurons behavior (see Section 3). This output is then
changed via turning off a percentage of the appropriate
neurons in order to produce the correct fovea movement.
The control neural layers are then trained “seeing” the
wrong image and outputting the corrective control action.
5.7. Implemented Architectures
Table 2 summarizes the characteristics of eight
architectures we have implemented. The architecture 1 of
Table 2 is, actually, the architecture depicted in Figure 1,
and its NADL specification is show in Figure 9.
The line 1 of Figure 9 (we have numbered the lines
only to facilitate the description) is a global command that
sets the neuron memory size of all neurons to 32. We
have used this neuron memory size in all architectures
because the number of training points is equal to 32.
Lines 3 to 8 specify the right and left input images sizes Finally, line 40 of Figure 9 specifies that the system
(512x512 pixels), their type of pixel (grayscale: 8 shades should train the control neural layer using its output as
of gray), and their generator and controller C functions. target pattern.
These two images are filtered by our gauss filter, lines 13
and 14, producing the neural layers gauss_right and Table 2. Implemented Architectures
gauss_left, which are created by lines 10 and 11. Note that Architecture 1 2 3 4 5 6 7 8
the size of these neural layers is only 64x64 neurons. Neuron rnd_mem_zrizro X X X
Lines 16 and 17 create two output windows that allow the Type default X X X X X
human operator to monitor gauss_right and gauss_left. Control right_minus_left 32 G32 G 32 G32 G 32 G32 R
These outputs are connected to gauss_right and gauss_left Layer left_minus_right 32 G 32 G
Inputs
by lines 19 and 20. These windows are the two top right TE, TE1 e TE2 32 G 32 G16 G16 R
windows of Figure 2 (both foveae are pointing to the Control ResetY X X X X X X X
center of the image). Strategy Set2Layers X

6. Experimental Results
1. set NEURON_MEMORY_SIZE = 32;
2.
3. input image_right[512][512] with greyscale outputs
4.
5.
produced by input_generator()
controled by input_controler_right(); Table 3 and Table 4 summarize the experimental
6.
7.
input image_left[512][512] with greyscale outputs
produced by input_generator()
results obtained with the 8 architectures represented in
8. controled by input_controler_left(control_out); Table 2, according to the performance parameters
9.
10. neuronlayer gauss_right[64][64] with greyscale outputs; presented in Subsection 5.6. We have tested the
11.
12.
neuronlayer gauss_left[64][64] with greyscale outputs; architectures with the points used during training (Table
13. filter image_right with gauss_filter()producing gauss_right; 3) and with unknown points (Table 4). The best results in
14. filter image_left with gauss_filter() producing gauss_left;
15. Table 3 and Table 4 are shown in gray.
16.
17.
output gauss_right_out[64][64];
output gauss_left_out[64][64];
By comparing Table 3 and Table 4, one can conclude
18. that the architecture 8 presents the best results, showing
19. outputconnect gauss_right to gauss_right_out;
20. outputconnect gauss_left to gauss_left_out; no unstable points for known and unknown points, the
21.
22.
smaller average error for known points and the runner-up
23. neuronlayer control[64][64] of rnd_mem_zrizro neurons average error for unknown points. Our best performing
24. with b&w outputs;
25. neuronlayer right_minus_left[64][64] with greyscale outputs; architecture produces vergence movements with average
26.
27. output right_minus_left_out[64][64]; error of only 3.6 image pixels, which is equivalent to an
28.
29.
output control_out[64][64]; angular error of approximately 0.615o.
30. outputconnect right_minus_left to right_minus_left_out; The model of our best architecture is depicted in
31.
32.
outputconnect control[@][@] to control_out;
Figure 10. This architecture uses default neurons
33.
34
filter gauss_right, gauss_left with minus_filter()
producing right_minus_left;
randomly connected to four filters outputs:
35. right_minus_left, with 32 connections per neuron; and the
TE, TN and TF filters with 16 connections per neuron (a
36. connect right_minus_left to control
37. with 32 random inputs per neuron and
38.
39.
gaussian distribution with radius 3.0; total of 32+3x16, or 80 connections per neuron).
40. associate control with control; Architecture 8 has Set2Layers as motor control strategy.
Figure 9. NADL specification of Figure 1 architecture
Lines 1 to 20 of Figure 9 are common to all Table 3. Results for known points
Architecture 1 2 3 4 5 6 7 8
architectures we have implemented and describe an
Average Error 19,20 23,59 4,48 11,64 10,02 -3,84 3,730,26
architecture up to its 2 retinas. However, lines 23 to 40
are specific to architecture 1. The control neural layer of Unstable Points 3 4 3 7 4 7 0 0
this architecture is created in line 23 and is composed by
rnd_mem_zrizro neurons – the first X of the architecture Table 4. Results for unknown points
1 column in Table 2 represents this characteristic. Note Architecture 1 2 3 4 5 6 7 8
that the function input_controler_left receives this neural Average Error 28,12 3,77 43,181,77 9,11 8,02 14,46 -3,58
layer as a parameter (line 8 of Figure 9). Unstable Points 0 3 9 7 8 5 0 0
The control neural layer receives input from a
right_minus_left filter only, and each of its neurons has
32 gaussian distributed connections to this filter output.
7. Discussion
This is specified by lines 36 to 38 of Figure 9 and, in In our system, vergence control is achieved using data
Table 2, represented by the 32G in the architecture 1 coming from most of the input images pixels but with a
column. Lines 27, 28, 30 and 31 allow monitoring the greater attention devoted to the fovea region. Our system
control neural layer and the right_minus_left filter allows calculating the distance of different points of
outputs.
objects in the 3D space by moving the dominant camera’s
fovea to it and calculating the vergence angle after the
system achieves vergence. With this information and the
fovea image of parts of the 3D objects, we believe it is
possible to build a representation of these objects internal
to the system. However, we are still a long way from that,
since we first need to build the parts of the system capable
of: choosing the interesting points of the 3D objects
(saccadic movement control), recognizing these
interesting points, building a 3D representation of the
objects, and grouping all these objects in a consistent
internal representation of the 3D scene.
Several works dealing with vergence in the context of
binocular tracking can be found in the literature [15, 16,
17]. Our work is similar to these in various aspects;
however, to our knowledge, there is no similar work in
the literature using WNN to control vergence.
Figure 10. The best architecture
8. Conclusion
This paper presents the methodology we have used to
develop a vergence control system for an artificial stereo
vision system based on Weightless Neural Networks
(WNNs). The system was developed using a tool we have
created for modeling WNN architectures, named Event
Associative Machine (MAE). We have used current
knowledge about the human biological vision system to
devise several architectural models of vergence control
systems. Our best architecture produces vergence
movements with average error of only 3.6 image pixels,
which is equivalent to an angular error of approximately
0.615o.
As future work, we plan to use real cameras to test our
system, and to develop a tool to create and test neural
architectures automatically using genetic programming
techniques. This tool will allow searching the best
parameters of our WNN architectures in a straightforward
way and, we believe, will be import for our research
towards implement a WNN system capable of emulating
the human biological visual system capacity for creating a
steady internal representation of the environment.
References
[1] E. R. Kandel, J. H. Schwartz, T. M. Jessell, “Principles of
Neural Science, 4th Edition”, Prentice-Hall International, 2000.
[2] S. M. Ebenholtz, “Oculomotor Systems and Perception”,
Cambridge University Press, 2001.
[3] T. B. Ludermir, A. Carvalho, A. P. Braga, and M. C. P.
Souto, “Weightless Neural Models: A Review of Current and
Past Works”, Neural Computing Surveys 2, p. 41-61, 1999.
[4] A. P. Braga, A. P. L. F. Carvalho e T. B. Ludermir,
“Fundamentos de Redes Neurais Artificiais”, Rio de Janerio:
DCC/IM, COPPE/Sistemas, NCE/UFRJ, 11ª Escola de
Computação, 1998.
[5] F. Gonzalez and R. Peres, “Neural Mechanisms Underlying
Stereoscopic Vision”, Progress in Neurobiology, Elsevier
Science Ltd., Vol 55, pp. 191-224, 1998.
[6] J. Semmlow, G. Hung, J. Horng and K. Ciuffreda, “Disparity
Vergence Eye Movements Exhibit Preprogrammed Motor
Control”, Vision Research, 34(10): 1335-1343, 1994.
[7] I. Aleksander, “Self-Adaptative Universal Logic Circuits”,
IEE Electronic Letters, 2:231, 1966.
[8] I. Aleksander, “Ideal Neurons for Neural Computers”, In R.
Eckmiller, G. Hartmann and G. Hauske - editors, Parallel
Processing in Neural Systems and Computers, pp. 225-228,
Elsevier Science, 1989.
[9] J. Mrsic-Flogel, “Approaching Cognitive System Design”,
Proceedings of the International Conference on Artificial Neural
Networks (ICANN’91), Vol. 1, pp. 879-883, 1991.
[10] James Austin (editor), “RAM-Based Neural Networks”,
World Scientific, 1998.
[11] Novel Technical Solutions, “NRM: Neural Representation
Modeller Version 1.2.01”, Novel Technical Solutions, 1998.
http://www.sonnet.co.uk/nts
[12] R. W. Rodieck, “Quantitative Analysis of Cat Retinal
Ganglion Cell Response to Visual Stimuli”, Vision Res. 5: 583-
601, 1965.
[13] R. C. Gonzalez and R. E. Woods, “Digital Image
Processing”, Addison-Wesley Publishing Company, Inc., 1992.
[14] T. McReynolds and D. Blythe, “Advanced Graphics
Programming Techniques Using OpenGL”, SIGGRAPH’99
Course, 1999, Available at
http://www.opengl.org/developers/code/sig99/advanced99/notes
/notes.html
[15] A. Bernardino and J. Santos-Victor, “Visual Behaviors for
Binocular Tracking”, Robotics and Autonomous Systems 25,
Elsevier, 1998.
[16] M. M. Marefat and L. Wu, “Purposeful Gazing and
Vergence Control for Active Vision”, Robotics & Computer-
Integrated Manufacturing, Vol. 12, No. 2, pp. 135-155, 1996.
[17] N. Kita, S. Rougeaux, Y. Kuniyoshi, and F. Chavand,
“Binocular Tracking Based on Virtual Horopters”, Proceedings
of IROS´94, pp. 2052-2057, 1994.