Visual Awareness

Jan Koenderink

D E C LOOTCRANS P RESS
Visual Awareness
Jan Koenderink

D E C LOOTCRANS P RESS , MMXII

Front cover: Caspar David Friedrich (1774–1840), Wanderer above the Sea of Fog Foreword
1818; Oil on canvas, 94 x 74.8 cm; Kunsthalle, Hamburg.

This short E–Book is intended as preliminary reading material for a sum-

mer course in visual perception. It is one of a series of short introductions.
The book was prepared in PDF–LATEX, using the movie15 and hyperref
packages. In order to make use of its structure use the (free!) Adobe Reader.
Make sure that all options are enabled! Video and sound clips will start by
clicking (or double clicking) the image, they are embedded in the file.
Make sure you are online. Clicking the light blue text will get you to Inter-
net sites with additional material (try this page!). Be sure to read some of that,
if good material is available elsewhere I’ll skip it in the text. Most references
occur only once, saving both you and me time. However, it means that you
may have to backtrack at times.
All links were active when I checked last. I will check them again with the
next reprint.

De Clootcrans Press Utrecht, february 12, 2012 — Jan Koenderink

Utrecht The Netherlands
jan.koenderink@telfort.nl

Copyright © 2012 by Jan Koenderink

All rights reserved. Please do not redistribute this file in any form without my
express permission. Thank you!
Jan Koenderink
Katholieke Universiteit Leuven
pax / jan koenderink Laboratory of Experimental Psychology
Tiensestraat 102 – bus 3711
3000 Leuven
Belgium
First edition, 2012 jan.koenderink@ppw.kuleuven.be
Personal page
10 9 8 7 6 5 4 3 2 1

i
 WHAT IS “AWARENESS”’?
In this chapter I introduce the concept of “visual awareness”. It is of
basic importance to understand the categorical difference between
visual awareness and visual cognition in the study of “visual per-
ception”.
Awareness from the inside
Here is a basic experiment:
Open your eyes in broad daylight, notice what happens.
The result may hardly prove amazing to you:
you are likely to see the scene in front of you,
at least, when you are not blind, inattentive, hallucinating, and so forth. Yet
this experimental result should appear amazing in view of the fact that science
has no clue of how to account for it!
Science can only repeat the long familiar story involving the physics of
electromagnetic radiation and its interaction with matter, the optics of the
eye, the neurophysiology of the brain, and so forth. Such stories allow you
to understand causal connections between the scene in front of you, and elec-
trochemical events inside you skull. This is by no means a minor feat, it took
centuries of intellectual labor. So far so good. But beyond that anything else
that might be said is mere mumbo jumbo to the honest scientist. The visual
awareness of the scene in front of you is literally magick, in the sense of being
beyond any (including future!) scientific understanding.
1
The doll scene from Il Casanova di Federico Fellini. (Video clip.)
Why is science unable to deal with awareness? Well, that is because aware-
ness is not an objectively established fact. Sure enough, you have little
doubt that you are aware. Is there any other alternative? Some philosophers
deny to suffer from either awareness or consciousness (which includes self-
awareness), but these are obviously in need of treatment. To deny to be aware
appears self-contradictory, though — of course — one could imagine some
machine (say a tape recorder) to announce loudly to lack awareness. (QUES -
TION would the tape recorder be right about that?) But apart from you, all
others might as well be dressed up robots, or zombies, without any awareness
at all. After all, it is easy enough to be deceived. The female doll Rosalba that
Casanova takes to dance in Fellini’s (1976, Il Casanova di Federico Fellini)
movie is actually a human actress1 , yet looks convincingly like a wooden
manichino. Mature people have been known to be reduced to tears on watch-
ing Bambi’s adventures in Disney’s (1942, directed by David Hand) movie,
1
Adele Angela Lojodice
yet Bambi isn’t even a machine, but only a pile of cartoon drawings displayed
in rapid sequence. Clearly one should mistrust one’s personal convictions
here. Science necessarily insists on publicly verifiable facts (so called “third
person accounts”). Awareness obviously fails to qualify, since it is necessarily
a “first person account”. First person accounts are views from the inside so to
speak. They can’t (strictly spoken) be shared with other people.
Disney’s Bambi. (Video clip)
In science this had led to the most embarrassing extremes. For instance
behaviorists have been known to hold the view that “speech is the move-
ment of air molecules” (seriously!). Of course, it is hard to dispute the point.
However, this definition hardly reaches the crux of the matter. One feels that
something essential is lacking. Not being false is not the same as being right.
What is lacking is the difference between speech as an intentional act (both
in its production and in its understanding), and the Brownian motion of air
molecules, or gusts of wind. But intentional acts are not proper topics of the
exact sciences. The movements of air molecules are.
2
Franz Clemens Honoratus Hermann
Brentano (Marienberg am Rhein (near
Boppard) 1838 – Zürich 1917)
What does it mean to be “visually aware”? One thing, due to Franz
Brentano (1838-1917), is that all awareness is awareness of something. One
says that awareness is intentional. This does not mean that the something ex-
ists otherwise than in awareness. For instance, you are visually aware in your
dreams, when you hallucinate a golden mountain, remember previous visual
awareness, or have pre-visions. However, the case that you are visually aware
of the scene in front of you is fairly generic.
The mainstream account of what happens in such a generic case is this: the
scene in front of you really exists (as a physical object) even in the absence of
awareness. Moreover, it causes your awareness. In this (currently dominant)
view the awareness is a visual representation of the scene in front of you. To
the degree that this representation happens to be isomorphic with the scene
in front of you the awareness is veridical. The goal of visual awareness is
to present you with veridical representations. Biological evolution optimizes
veridicality, because veridicality implies fitness. Human visual awareness is
generally close to veridical. Animals (perhaps with exception of the higher
primates) do not approach this level, as shown by ethological studies.
J UST FOR THE RECORD these silly and incoherent notions are not
something I ascribe to!
But it neatly sums up the mainstream view of the matter as I read it.
God’s Eye (“The Eye of Provi-
dence”) on the Dollar Bill. It is
the reverse of the Great Seal of the
United States.
The mainstream account is incoherent, and may actually be regarded as un-
scientific. Notice that it implies an externalist and objectivist God’s Eye view
(the scene really exists and physics tells how), that it evidently misinterprets
evolution (for fitness does not imply veridicality at all), and that it is embar-
rassing in its anthropocentricity. All this should appear to you as in the worst
of taste if you call yourself a scientist. I will touch upon these topics later.
Since awareness is a first person fact, its proper study is through phe-
nomenological analysis. Such analysis has been pursued by mystics (not nec-
essarily in a religious context — although that has often been the case, for
instance Zen can hardly be typified as “religious”), and only recently became
a topic in philosophy, and still later psychology. Throughout the second half
of the twentieth century “phenomenology” has been in ill repute in psychol-
ogy, although more recently the tide seems to be turning. Modern exponents
are Edmund Husserl and Maurice Merleau-Ponty. It is hard to say how one
3
might integrate phenomenology with generic “science”. Yet it is clear enough
that psychology will not be able to make real progress in the absence of such
a move. The wait is for the next courageous person!
Edmund Gustav Albrecht Husserl (Prossnitz 1859 – Freiburg im Breis-
gau 1938) and Maurice Merleau-Ponty (Rochefort-sur-Mer 1908 –
Paris 1961).
Experimental phenomenology involves a description of one’s mental states
as observed by oneself. This is evidently problematic. “Introspection” is an
unspeakable i-word in cognitive science. A certain degree of “intersubjectiv-
ity” (instead of the properly scientific “objectivity”) may be sought for after
“comparing notes” so to speak. Fortunately, it turns out that such an inter-
subjective agreement can grosso modo be achieved. Here we only consider
the most superficial description. However, even this is very different from the
mainstream account.
The first thing to notice is that awareness happens to you. There is nothing
you can do about it. Awareness comes like a sneeze. If you want to get rid of
visual awareness of the scene in front of you, all you can do is close your eyes,
or turn your head. Merely willing the awareness to go away has no effect. You
may be visually aware of a configuration that you know to be not factually “sense” as in “good horse sense”. Sense is not “meaning” as in thought, sense
present, or even formally impossible in your reflective thought. Knowing that is different from meaning in the logical sense. Meaning is part of thought,
you are aware of an “illusion” does not cause it to disappear. You do not by and thought is something you do. Thought implies the self. Thought can be
any (mental) means control the awarenesses that happen to you. That is why true of false, good or evil, and so forth, whereas awareness is beyond such
awareness is most aptly described as presentation (Vorstellung), rather than as dichotomies. It simply is.
representation as the mainstream would have it. Because awareness cannot
be controlled mentally, it is pre-personal, and pre- (or perhaps proto-)rational.
It is more correct to say “there is awareness” than to say “I have awareness”.
The “I” in “I have awareness” only makes sense in the same way as the elusive
“it” in “it rains”. Did you ever meet the “it”? (The mystic would ask: “when
did you last meet the I”?)

Many people conceive of this as an

“impossible figure”. Yet it doesn’t
prevent them from seeing it.

A flying carpet? It certainly looks like it! Scrutiny reveals that the “shadow
of the carpet” is something else altogether. It doesn’t keep you from seeing
a flying carpet though.

That awareness is pre-personal makes that you cannot know it. It is not part To sum up,
of your experience in the same sense as that your birth or death cannot be part
there is visual awareness,
of your experience. Thus awareness is not temporal. Yet it cannot be said to
though there is no one that sees,
be momentary either, because moments are temporal. Perhaps paradoxically
nor is there something seen,
(at least in reflective thought), presentations always involve a presence of the
past, as well as a presence of the future, and also a presence of the presence, a summary that many will consider paradoxical, although it certainly hits the
although not “in time”. nail on the head. The observer and its objects belong to thought. The pre-
The contents of awareness is characterized by quality and sense. Here I use sentations supply thought with material to work on, but in the process it is

4
stripped of qualities and senses. What is left are visual facts that can be ma-
nipulated in reflective thought. For instance, you can name objects in your
ken. When you do so, you are at some remove from immediate awareness.
This also implies that presentations cannot be illusory, only visual cogni-
tion can be “illusory” in the sense of not being veridical. Visual reality proper
is the same as visual awareness. In a more extensive sense it is frequently
taken to include visual cognition. But caution is needed because of the on-
tological chasm between awareness and thought. Whereas visual awareness
is beyond truth or falsehood, right or wrong, good or evil, these are the very
foundations of visual cognition. The scene in front of you as you know it
may well be illusory. Visual awareness is the only visual reality in the strict
sense. In the strict sense, all else is illusion, or perhaps delusion. This includes
“physical reality”.
The picture at left is aimed at your immediate visual awareness, that at right
is aimed at your visual cognition. Spend a moment to figure out what in
these picture is interesting or funny. How long did it take you to “get it”?
(Be honest!) Can you explain them to others?
5
The distinction between visual awareness and visual cognition is absolutely
crucial. It is nothing less but a tragedy that mainstream science has lost this
distinction completely. I’ll come back to it later. For the moment it may help
to consider two “visual puns” (figure at left). One is aimed at your awareness.
It works momentarily. It cannot be explained. The other is aimed at your
cognition. For most people it takes at least a few seconds to figure out why
the picture might be funny, or otherwise interesting. The pun is not purely
visual, it can be explained. If you fail to see the difference, you may as well
stop reading. The awareness–cognition dichotomy is absolutely fundamental.
The scene is experienced not only visually, but through your whole system
of senses (you are aware of sounds, weights, warmth, smells, locatedness and
orientedness, an so forth — in many cases you’re not aware of “which sense”
is involved in awareness), as well as in your actions. Here “action” is not
something like “premeditated action” (which may be compared to cognition
and reflective thought), but automatic actions, that “just happen”.
You have an intention in action, that is to say, the actions are about some-
thing (like walking, or speaking). You don’t even know which muscles you
use. That is because you don’t initiate the actions. They simply happen.
Without such automatic actions you couldn’t exist, in that they are similar to
awareness.
It is best to consider actions as part of awareness. The qualities and senses
apply equally to the actions. Distinctions that you may try to make in reflec-
tive thought turn out to be irrelevant from the phenomenological perspective.
Awareness from the outside
Whereas the scene in front of you is known to you directly through your
awareness, it is known publicly (that implies indirectly) in terms of physics.
Indeed, in the mainstream account the content of your awareness is simply a
representation of that physical object. What is the meaning of this?
A general consensus appears to be that there is a physical object (the scene
in front of you) even if you close your eyes, and that someone next to you is
likely to have pretty much the same visual awareness. So much is obvious
because both of you then happen to look at the same physical objects, so both
of you should come up with the same representation. If not, then some rep-
resentation has to be less than veridical, that is to say, illusory. The physical
object is something we all agree on, and it is the cause that our representations
are the same. This is the mainstream account of the matter.
A stationary state of the hydrogen
atom. Of course, this is mere pop-
science. The actual wave function is
a certain solution of the Schrödinger
equation for the hydrogen system.
This is a mere pretty picture, the real
physicist doesn’t need it. This is an
attempt at a picture of “physical re-
ality”.
What exactly is this physical object? Although it is generally supposed that
the answer is known (ask a physicist!), or even obvious, this is actually quite
problematic. An honest physicist will not be able to give you an answer. An
attempt at an answer would be a long story starting with (speculative) theories
involving symmetry groups in general, high dimensional spaces, touching on
elementary particles, quantum theoretical paradoxes, atoms, molecules, var-
ious aggregative states of matter, interaction with the electromagnetic field
from the sun, and so forth. Suppose the scene in front of you is your office,
containing (among more) your chair and desk. Then the physicist is very
unlikely to mention those. Chairs are not objects that modern physics has
anything to say about. A chair is an object of the life world (Umwelt, see
below), not of the physical world.
What non-physics scientists and philosophers mean when they refer to
“physics” is more like what the physicist would call “classical physics”. Clas-
6
sical physics is an odd collection of phenomenological theories2 that allow
one to predict events on the macro-scale, including the human scale. These
phenomenological theories allow you to design useful things like submarines,
bras, guns, chewing gum, and sun glasses. However, they do by no means
tell you what the scene in front of you is. Different from what non-physicists
believe, neither physics proper, nor classical physics give you “ground truth”.
If there is any “ground truth” of the matter it would be the spectral radiance
at the corneas of the eyes. However, it is obvious that visual awareness “rep-
resents” nothing like that.
In cognitive science it is not uncommon to
downplay awareness and understand percep-
tion as a cognitive activity. Language plays
an important role. The main use of percep-
tion is assumed to be recognition, which is
operationalized by naming. Thus “house”,
“tree’, “dog”, . . . would roughly exhaust the
contents of awareness. This is evidently not
the view I propose here.
There are no meanings (in the sense of “sense”) or qualities (like the “red-
ness” of a rose) in physics, only “pointer readings” and speculative theories
(the best one has for the moment, always ready to be dropped for a better one)
that allow you to predict the results of potential pointer readings from ac-
tual ones (By all means read Eddington’s rightly famous “The Nature of the
2
Classical physics was developed mainly throughout the course of the 20th c. Nowadays
it is mainly known as “engineering”. Needless to say that to the real physicists science and
engineering are very different endeavors.
Physical World”3 of 1928, where he applies methods aimed at the electron on
an elephant). The whole idea that visual awareness might “represent” some-
thing like a description in terms of physics is nonsensical. Visual awareness
is presentation, presentations are visual reality.
The notion that there is a way the scene in front of you really is—I will
refer to it as the “God’s Eye View”—is a metaphysical hypothesis that you
are free to adopt. However, don’t draw conclusions from it that force your
understanding of awareness. If you really must, then adopt Kant’s notion of
the “Ding an sich”: you will forever be unable to reach the “real” thing! It
seems more practical to adopt the attitude that reality is what you experience.
Of course, the experience reflects the way you are, just as it reflects the way
the “world” is. But this makes sense, simply consider the way a traffic sign
pole is to you, your dog, or a pigeon. Who has it right? Why?
Awareness in science
“Vision science” is a very broad field that ranges from physics and phys-
iology to psychology and philosophy.4 An attempt at objectivity limits the
methodology to third person accounts though. Thus awareness proper lies
outside science’s reach. So what can one do?
Well, anything having to do with physics (meaningless pointer readings!)
is fine. When considering a person, it is essentially one physical object in
the midst of many others. This object, specifically related to the person, is
3
A great book to read. The text is available on the Internet. Every philosopher keen to
refer to physics as the ground truth of the matter should read it. Use “Find” for “elephant” to
get at the passage mentioned in the text.
4
Of course, there is much useful science that goes by the name of “vision science”, but has
no relation whatsoever with visual awareness. Nothing wrong with that, but you have to take
notice. “Vision science” in the (narrow) sense of the “science of visual awareness” is virtually
non-existent. One common idea is that we should pursue brain science, then the pseudo-
problem of visual awareness will (given time) be forgotten. Awareness, consciousness, and
so forth are at best epiphenomena. I have a colleague who tells the world that he has no free
will, yet is very happy. He’s not responsible for anything he does, his brain makes him do it.
He expects us to see the light eventually (these people tend to be incoherent).
7
the person’s body. Thus one starts with the dichotomy body-world, where
“world” is anything physical that is not the body. This is quite different from
the look from the inside. In awareness proper (which includes actions) there
is no distinction to be made between body and world. There is only a single
“cosmos”. One might perhaps say that the body enables a person to be di-
rectly aware of physical reality from the inside perspective. If you take this
seriously, the bulk of AI (“artificial intelligence”) research loses its sense.
All perception is
perspectival. It
is not different
with science. A
perspectival view
is a limited view.
This is perfectly
illustrated with
the ancient para-
ble of the “seven
blind men and the
elephant”.
Thus one may consider the physics of the world (that is physics proper),
the physics of the body (that is physiology), and the physical interactions
between the body and the world. The latter is the more interesting (and con-
fusing!) aspect. In science proper one uses only pointer readings. Thus you
cannot ask somebody what is in visual awareness. Well, you can, but both the
question and the answer have to be understood in terms of pointer readings.
In a “psychological” experiment the scientist may utter a certain sound wave
(that I would hear as “raise your left hand if the light is red, right hand if it is
green”, but of course I’m not a very useful pressure transducer) and the ob-
server (person being studied) may be seen to raise the left arm “in response”.
The fact that the left arm goes up is easily recorded as a pointer reading, so
is the spectral composition of the source of radiation the observer has been.
“Responses” may be recorded as a function of the spectral composition of the
source.
A “primate chair”. Notice the “reward system”.
Notice that there is some fudge factor here: the instruction of the observer is
evidently not something that can be incorporated in the framework of physics.
This can be avoided by not instructing the observer, but to punish or reward
him in case he “doesn’t understand the task”. This is usually done with mon-
keys. One dehydrates them to the point that they have lost a certain fraction
of body weight (to the point that a human would go mad with thirst), so as to
ensure that they will work their shirt off in order to “be rewarded” with some
droplets of liquid. Humans can often be motivated in a more humanely man-
ner by paying or charging them money. This works best with poor students.
In the ideal case (the best schemes come close) neither the scientist nor the
observer really know what they’re doing, so the whole procedure is perfectly
objective. Does this address the observer’s awareness? Most obviously not!
What this amounts to is what I will denote “non-invasive physiology”. (Inva-
sive physiology involves sawing off the top of your skull, inserting electrodes
8
in your brain tissue, and so forth. It is usually reserved for monkeys.)
The field of “psychophysics” has been founded on such methods. It has
proven extremely useful in numerous applications. For instance, neither
television, nor audio hifi installations would exist as they do without psy-
chophysics. Usually the credits go to electronics, psychophysics being a
highly underrated discipline. However, the fact remains that psychophysics
is essentially non-invasive physiology. This can hardly be otherwise due to
the ontological gap. Science has no way to address awareness proper.
Of course, this is to be regretted. There is little doubt that animals (in-
cluding people) are not fully exhausted by their physical properties. Even if
you could predict the future course of the complete molecular structure of the
body, which will be forever impossible, you would only be able to predict
physical phenomena. Speech would indeed be exhausted by the movements
of air molecules. This might be useful enough, for instance, it would enable
you to distinguish between Maria Callas and Madonna on the basis of their
acoustical emissions. It wouldn’t enable you to get the gist of the State of the
Union (in acoustical terms) though. Something is missing.
If animals are not fully exhausted by their physical properties, then what is
missing? It is their often remarkably effective, and often surprising action–
in–the–world. The fact that they often occur as actors, pursuing certain goals,
rather than physical systems reacting to physical loads.
This is a complicated topic, because the animal’s body cannot really be
separated from its world. The two parts fit like a glove. The body of fishes
reflects the Laws of Hydrodynamics, those of birds the Laws of Aerodynam-
ics. Eyes (of a variety of builds) reflect the Laws of Optics. The spider’s web
reflects many of the physical properties of flies, as well as of the spider’s body
itself. The hedge hog goes into hibernation, apparently fitting the course of
the earth’s orbit around the sun. And so forth, and so forth. The animal world
appears as a play in which each of the many players plays its own part, yet all
together make the play. Nature appears like Indra’s Net, each jewel reflect-
ing all others. It may also remind one of Leibniz’s Monads. Although “the
monads have no windows5 ”, each living in its private world, they somehow
act in perfect synchrony. In Leibniz this is due to God’s Design, the modern
5
Monadology # 7.
scientist will refer to Darwinian evolution.
Gottfried Wilhelm von Leibniz (1646–
1716).
Remember that the space outside your body is full of radio waves, enabling
you to connect to the Internet wherever you are. How important are these
for awareness? They have no importance whatsoever! Thus it must be the
case that much physical structure is completely irrelevant to the issue. Only
physical structure that actually interacts with the sensitive body surface so as
to lead to possible changes in experience need to be taken into account.
In order to deal with such issues we draw on distinctions introduced by the
biologist Jakob von Uexküll (1864-1944):
◦ The Umgebung (environment) is the part of the world to which the body
is exposed (thus including radiowaves and neutrinos). It is reality as
known from the outside;
◦ The Merkwelt is the part of the world that causes changes of state in the
body’s sensitive surface;
◦ The Wirkwelt is the part of the world (including the body itself!) that the
body is ably to change;
◦ The Umwelt (lifeworld) includes the Merkwelt and the Wirkwelt, includ-
ing their Funktionskreisen (interrelations);
9
◦ The Innenwelt is the lifeworld from the perspective of awareness, that is
to say, it is reality as experienced from the inside.
All these distinctions are useful. However, here I will mainly use environment
and lifeworld.
Jakob von Uexküll (1864–1944).
Jakob von Uexküll is not too popular nowadays because he was not too
keen on Darwinism. Of course, that was Darwinism as it was understood
in his time. He had a decisive influence6 on the first generation ethologists
(Lorenz and Tinbergen). Lorenz was strongly influenced by von Uexküll, it
was mainly von Uexküll opposition to Darwinism that made him hide it.
Notice that science can only approach the lifeworld from the outside. Its
inner perspective, the qualities and senses of awareness must remain a closed
book. This has been explored in the well known paper by the philosopher
Thomas Nagel “What is it like to be a bat?”7
6
Perhaps surprisingly, von Uexküll’s work also was important in early behaviorism and
was highly appreciated by advocates of that persuation.
7
Philosophical Review 84(4) october 1974, pp. 435-50; see also this.
 This has important consequences. Using the tools of science it will forever
remain unclear what the nature of visual awareness is. No brain scanning is
going to help either, after all, those are just physiological techniques.
The endeavor in biology that relates to animal visual awareness is called
ethology. It is from fairly recent origin, mid twentieth century say. I cannot
spend to much time in ethology. However, it is of prime importance, and
extremely interesting. I’ll only give two examples.
Indra’s net. Each droplet reflects all others.
When female stickleback fish look for males in order to have them fertilize
their eggs, they use their eyes. “Good” males are recognized by their red bel-
lies. As Niko Tinbergen discovered, the red bellies trigger their behavior, he
called the red belly a “releaser” (of certain stereotypical behavior). Tinbergen
found many more examples of such releasers. What is interesting is that the
releasers are very specific and evidently skip any (proto–)rational processes.
10
In the case of the stickleback females the simplest “fake males” are strongly
preferred over actual males, as long as their undersides are painted a vivid
red. It is a case of super-pornography. The females have apparently no choice
in the matter. They are triggered optically.
A stickleback female is attracted by a fake male.
Here is another famous example, due to Konrad Lorenz. The Northern
Lapwing Goose builds nests on the ground. If the bird spots an egg near the
nest it rolls it to the nest. This egg rolling routine appears to be a rational
action: of course the bird wants the eggs to be in the nest! But the egg rolling
routine is also triggered by a tennis ball. Maybe tennis balls look much like
eggs? It is also triggered by a brick. The bird will try hard to “roll it back to
the nest”. This is evidently not rational behavior at all. It is known as a ”fixed
action pattern”. Is the bird visually handicapped in any way? No, its visual
acuity is as good as yours. This is a case of “soul blindness” (Seelenblindheit).
What is in the bird’s visual awareness when it does stupid things like that? Are
humans different in kind? I would doubt that. Just watch small children.8
Ethology has provided us with numerous examples of similar phenomena.
It would seem that animals often react in stereotypical ways on what is in their
optical ken. Their visions appear to be idiosyncratic user interfaces. These vi-
sual systems are evidently shaped over eons of evolution. The chicken that has
barely hatched already has a repertoire of such optically triggered behaviors.
Microgenesis
In this chapter I introduce the concept of microgenesis. Microgen-
esis is a hypothetical subconscious process that continually comes
up with visual presentations. It is a systolic process, a single systole
taking less than a tenth of a second.
How awareness comes to be
So presentations just happen to you. Moreover, they are not “caused” by the
scene in front of you in terms of physical (including physiological) causation.
But what “causes” the magick?
Notice that you can pose the question only from an external perspective.
From the internal perspective there is not something like “cause”. What “just
happens” has no prior cause.
However, one feels that “something was cooking” before you had the pre-
sentation. Reason to think this is that you frequently (once you start to no-
tice) have sequences of presentations that are different, yet relate to the “same
scene”. (Notice the tension between the inner and outer perspectives!) It is
8
Grandchildren are best for the purpose. You have more time to notice. If you’re a young-
ster you’re at a disadvantage here. You’ll have to wait.
11
not unlike the occasion of “having a good idea”. Good ideas also happen to
you, they are not something you do. Yet you have often a feeling that the
good idea “had been cooking” for some time. In this case it is clear why this
might happen. Good ideas only occur to people who spend effort pondering
some question. The good idea comes as a reward, after you’ve given up on
the problem. People who never struggled with a problem will never have a
good idea that solves it. Thus the good idea is apparently “due to” some sub-
conscious process. Although such subconscious processes are not available to
phenomenological analysis, it is thinkable that one might find physiological
processes (brain activities) related to them. It is also thinkable that one might
be able to design tools of experimental psychology that might succeed in dis-
covering changes of mental state that accompany the subconscious “cooking”.
Presentations might not be unlike good ideas in this respect. The hypothetical
process that cooks up presentations is conventionally denoted “microgenesis”.
This is a somewhat odd term, I would have preferred something like “psycho-
genesis” (already taken, and for a different topic), or “iconogenesis” (people
hate to pronounce that). But there you are. Microgenesis it is.
What does one know about microgenesis? Not a whole lot. Most ideas
have been derived from studies of various abnormal mental conditions. A
good source is the work by Jason Brown, a neurologist9 . What is most inter-
esting in Brown’s conclusions is that microgenesis works from the inside out
so to speak, in contrast to the mainstream conviction that presentations are
computed from the optical data, thus from the outside in.
The notion that microgenesis originates in the deep parts of the brain makes
excellent biological sense. Any biological system is the result of building
structure on structure. Older structures always remain functional, the newer
additions tend to add novel capacities or (more frequently) fine-tune and di-
versify the coarser core processes. The brain is such a layered structure. Of
course, this implies that one has to understand how the presentations come to
be useful. In one sense or other, they have to be constrained by the optical
structure that impinges on the eyes, otherwise the observer would only dream
or hallucinate. That is the reason why the mainstream wants to reverse the
natural direction. However, it has to be remembered that optical structure is
9
Here are some interesting interviews.
totally meaningless, there is neither sense nor quality in it. Sense, nor quality
are in nature, they are in the mind. You can’t “pick up” sense and quality, you
have to supply it! That is why the inside out approach (”top down”) makes
sense and the outside in (“bottom up”) doesn’t.
How do shapes enter awareness? An ancient
riddle.
An ancient idea is that objects shed flimsy
skins that travel through space an literally
enter the eye. Although evidently naive, it
is hardly more so than the contemporary no-
tion of “inverse optics”.
Thus the idea is that presentations start out as the gut level basic emotive
feelings, that they develop over dreamlike states, finally reaching the level
of clearcut hallucinations. However, not anything goes, at least if you are of
sound body and mind, have your eyes open, and face the world heroically
(that is to say: ready to deal with things as they occur to you). If so, then the
proto-presentations will be checked against the optical structure impinging on
the eyes. Again, not anything goes! Your presentations somehow have to “fit”
the structure that the world imposes upon you. Even stronger, you will seek
out confrontation! Your presentations will be the best you can muster to deal
with the world on its own terms. Of course, it will still be your world. All you
can ever deal with is the world as you are able to deal with.
How does microgenesis manage to come up with presentations that “ac-
count for” (to the extent that you are able) the complexity that your body (that
is you!) has to deal with? Most likely (and neurology/neuropsychiatry tend
to back this up) the process is not unlike Darwinian evolution. Microgene-
sis launches numerous threads that are subject to pruning (thus termination
12
of sub-threads) or diversification (the sprouting off of sub-threads). It is like
the biological evolution of species on a vastly faster time scale. Eventually a
certain thread dominates and determines the next presentation. This happens
when no further evolution is possible. Thus the presentations are end-states,
like the solidified crust of a living process.
The moment a presentation occurs to you it is already being replaced with
another one. Microgenesis is thus like a systolic process. The life time of a
presentation (a systole) is somewhat less than a tenth of a second. It is the time
it has before being dominated by its successor. Thus you experience about
a dozen presentations in a heart beat (about a second). The systolic period
is so short that the eye maintains fixation during that period. Subsequent
presentations may well be based on different fixations though.
There is no “flow of time” in the Newtonian sense during a systolic period.
Even subsequent presentations do not necessarily respect temporal order in
the sense of physical clock time. It is not evident how the body (or mind)
keeps anything like physical clock time. One possibility is that the electri-
cal cardiac activity, which is available throughout the body through electrical
volume conduction of the blood volume) acts as a synchronization signal. It
is certainly the case that temporal order in awareness has a grain of about a
heart beat, whereas temporal resolution has a much finer grain.
Thus the relevant time scales in vision are roughly like this:
◦ the shortest relevant scale is that of the systolic period of presentations,
somewhat less than a tenth of a second;
◦ a “glance” may include a few presentations, in many cases just one, or a
few involuntary (automatic!) fixations. Glances are still in the realm of
pure visual awareness, that is to say, they are pre-cognitive;
◦ a “good look” involves several fixations, some of these voluntary. It may
last from one to maybe a dozen seconds. Cognition already kicks in. A
good look is at some remove from pure “visual reality”;
◦ “scrutiny” involves any number of good looks. It is a mixture of cogni-
tive integration and visual awareness. Thus the result of “witnessing” is
thought, it may be right or wrong.
What is known as a “specious present” is something in the realm of good looks
and glances. The specious present does not develop over time, but is like a
“thick moment”. Anything that is temporally articulated within a specious
present is still a unitary spatio-temporal configuration. Examples are taking a
step, a golf swing, putting a fork-full of food in your mouth.
The contents of a glance, or good look, even a specious present, tend to be
“coherent stories”, they make sense. Even if you scramble spacetime in the
optical input (as you may do in the laboratory), the visual experiences remain
making sense, even if this implies “perceiving the wrong temporal order”.
This is close to the worst user interface I can possibly imagine! Ideally, a GUI
(Graphical User Interface) should be crystal clear without any explanation.
What I have said here about visual perception equally applies to (visual)
memory. Memories are like perceptions, in the sense that they are constructed
by some microgenetic process. They are not constrained by the optical struc-
ture, but are generated so as to fit the occasion on the basis of prior experi-
ence. Memories also act as constraints on presentations, this is roughly John
Searle’s “background”, Marvin Minsky’s “frames”, and so forth. The upshot
is that you are only able to see what you are ready to see. Getting ready to
13
see means having gained experiences, through your evolutionary past, dur-
ing your maturation, and during your recent mistakes. Your evolutionary past
contributes the basic fundaments, of which all else is mere diversification.
This essential grounding is usually underestimated. We share the bulk with
our animal friends, the vertebrates (and even farther back than that).
The basic principle was most clearly outlined by Jakob von Uexküll. An-
imals have distinct Umwelts and distinct Innenwelts. These determine their
capacity to experience. The meaning of anything can only be related to your
experiences (Merkwelt-Wirkwelt Funktionskreis (functional cycle)), it cannot
be related to the “world as it is”. There is no world as it is (God’s Eye view),
thus veridicality is an empty concept. The mainstream notion that perception
yields the most likely inference on the basis of the physical stimulation of the
sensitive body surface (most clearly posed by Hermann von Helmholtz) is in-
coherent. Evolution optimizes fitness, not veridicality. Perception yields the
most useful user interface for your particular physiological make up (which
is again determined by your generic environment, life style, and so forth).
We shouldn’t forget how we started. Of
course the complete story goes back much
further in time. Many of our mental abilities
we share with all vertebrates.
It is good to remember that the most effective user interfaces screen the
user from unnecessary complexity. Veridicality is counter-productive! Good
interfaces let you act efficaciously on the basis of idiosyncratic senses and
qualities. Just consider what happens when you delete a text document on a
generic computer desktop interface. What do you visually experience, what
do you think is happening, what actually occurs on the level of the program,
the file and window manager, the operating system, the hardware, and so
forth? Pondering such questions will give you some perspective. Moreover,
don’t forget the releasers and fixed action patterns of the ethologists (the fe-
male stickleback fish, the Northern Lapwing Goose, and so forth). How do
you fare (try to view yourself from a God’s Eye perspective!)?
Sparks of enlightenment
A perennial problem in the philosophy of mind is known as the “mind-
body problem”. It is about the ontological gap between awareness and the
physical description of the world. Science has banned the observer from the
description of phenomena. No first person accounts are permitted! Then, after
removing the observer, science complains that the mind (the observer) cannot
be found. Small wonder!
There is a cottage industry of Nobel Prize laureates who aim to gain a
second distinction by “solving the hard problem of consciousness”. So far no
one has succeeded, although there is no shortage of original ideas. Indeed,
some are so far fetched as to be beyond any reasonable belief. A majority
searches for the NCC, the “neural center of consciousness”. We may wish
them happy hunting! Those who join the hunt apparently don’t understand
the ontological gaps between the psychical and the physical realms. I’ll not
go into such endeavors any further, I consider them embarrassing.
Since there can be no causal relations between the physical and psychical
realms, at least not of the scientifically respectable kind, one may search for
“bridging hypotheses” that at least allow one to draw hypothetical inferences.
This may be useful in guiding empirical research. A common example of
such a bridging hypothesis is that of an isomorphism between certain neural
excitations and certain presentations.
14
For instance, it has been said that there exists such an isomorphism between
the firing of C–fibers and the awareness of pain. This has led philosophers to
hold that pain is the firing of C–fibers, the awareness being a mere epiphe-
nomenon. According to these, given time, we will come to recognize pain for
what it really is (the firing of C–fibers). Such reasonings are common enough.
They are supposed to be in accord with good science. I would say that when
interpreted in such a way as to be not obviously false, such notions fail to be
interesting, and when interpreted in any interesting way they are obviously
nonsensical.
Another bridging hypothesis was suggested by the Berlin School of Gestalt
psychology. They explained the awareness of certain visual presentations as
due to an isomorphism with certain fields of brain activity. Of course, this in-
troduces the old homunculus problem. At best (that is to say: if there actually
were such brain patterns) it would be a non-explanation. Indeed, since the
retina itself is part of the brain, there actually is such an isomorphism! But
noticing this gets you nowhere.
Erwin Schrödinger (1887–1961) is
best known for the “Schrödinger
equation” of quantum mechanics. It
is less well known that he devoted
much effort on visual perception, es-
pecially color vision. His work on
the geometry of color space is still
state of the art.
A bridging hypothesis that makes some intuitive sense to me is due to Er-
win Schrödinger (of quantum mechanical fame). It is found in his popular
book “Mind and matter” (1958). Schrödinger fully understands the root of
the problem:
The material world has only been constructed at the price of taking the
self, that is, mind, out of it, removing it; mind is not part of it. . .
He considers awareness to be pre-personal:
The only possible alternative is simply to keep to the immediate experience
that consciousness is a singular of which the plural is unknown; that there
is only one thing and that what seems to be a plurality is merely a series
of different aspects of this one thing. . .
He understands fully that awareness is visual reality:
No single man can make a distinction between the realm of his perceptions
and the realm of things that cause it, since however detailed the knowledge
he may have acquired about the whole story, the story is occurring only
once and not twice. The duplication is an allegory suggested mainly by
communication with other beings. . .
Here is his amazing bridging hypothesis:
The ensuing organic development begins to be accompanied by conscious-
ness only inasmuch as there are organs that gradually take up interaction
with the environment, adapt their functions to the changes in the situa-
tion, are influenced, undergo practice, are in special ways modified by the
surroundings. We higher vertebrates possess such an organ mainly in our
nervous system. Therefore consciousness is associated with those of its
functions that adapt themselves by what we call experience to a changing
environment. The nervous system is the place where our species is still
engaged in phylogenetic transformation; metaphorically speaking it is the
“vegetation top” ( Vegetationsspitze) of our stem. I would summarize my
general hypothesis thus: consciousness is associated with the learning
of the living substance; its knowing how ( Können) is unconscious.
15
This is indeed very intuitive. When you are involved in some automatic ac-
tion, like descending a staircase, you experience a sudden jolt of awareness
when you have (subconsciously!) “miscounted” a step. Automatic actions
remain subconscious. Awareness arises when you run into an unexpected
constraint. This is exactly the moment where you learn something. It might
be said that “the world lights up”, or that you experience a spark of enlight-
enment. The learning implies a gain of information, not information in Shan-
non’s structural sense, but information in terms of “meaning”, of good horse
sense.
Consciousness then, may be understood as a sequence of such micro-
enlightenments. The principle can be applied to all organic life, from the
simplest unicellular mechanisms to man.
Schrödinger’s bridging hypothesis is, of course, just that. It does not imply
physical causation. That is why you may accept it (as a bridging hypothesis)
and still remain an honest scientist. This is more than can be said of the
familiar contenders.
In the next sections I explore the consequences of Schrödinger’s astonish-
ing hypothesis.
Questions and answers
The Schrödinger bridging hypothesis has to do with learning by mistake.
This is an important issue in visual perception. Trivially, all learning is learn-
ing by mistake. More interestingly, in order to spot a mistake you need to
have an expectation. That is to say, the situation is perhaps best described as
questioning.
“Questioning” should be interpreted broadly. In visual awareness “ques-
tioning” involves checking the available optical structure for the existence of
a certain substructure. The “checking” is an action intended to check whether
a certain hypothesis might work. The hypothesis is due to some thread of the
microgenetic process. If the outcome is that the structure is there, the thread
carries on its checking. If the outcome is that the structure is ruled out, the
thread is killed. If the outcome is uncertain, the thread may diversify. The frogs. That is why humans see things that frogs are blind to, although frogs
checking is much like poking, like a blind man pokes with his cane. may receive similar optical structure.
The all important issue is that poking is poking for something. The meaning
of the outcome depends upon the intention of the poking. The meaning of an
answer is not a property of the answer, but a property of the question. A
question is like a format (in the sense of formal algorithmic languages), ready
to make sense of the answer.
Here is an example. You hit the keys ”F”, ”1”, ”0”, ”A”, ”1”, ”Enter” on
your keyboard. What does it mean to the computer? Well, it depends upon
the program that happens to be listening to the keyboard. Given that program,
it depends upon the state of the program, more precisely, on the format the
program is currently maintaining. Thus the meaning of ”F10A1” (assuming
the ”Enter” key was interpreted as “message completed”) might be anything
like
◦ a hexadecimal number
◦ a string
◦ a password
◦ gibberish (a certain type of error)
◦ and so forth
in principle there are infinitely many possible meanings. The point is that
“the” meaning does not depend on the keys you hit on the keyboard. The
meaning depends upon the current format, that is to say, upon the question
asked.
So what is the meaning of the optical structure currently impinging upon The frog’s microgenesis recognizes only a few formats. “Bug” is one. The
your eyes? The meaning depends entirely upon you. Taken by itself, the op- frog immediately eats it. There is no way it could do otherwise.
tical structure (the mainstream talks of “optical data”, evidently a misnomer)
is entirely meaningless. If you are unconscious, inattentive, or asleep, then
no meaning is evident. Any meaning will be different for you and your dog,
even if the optical structure is identical. The Sherlock Model
Thus asking questions is what enables microgenesis to come up with pre-
sentations that serve your (or your dog’s) fitness. Knowing “how to see” im-
plies being able to frame (subconsciously!) relevant questions. This may be How to ask good questions, even if you are at a loss as to what might be the
likened to playing the game of “Twenty Questions” with nature. Some peo- case? This is the perennial problem of forensic investigation. The optimum
ple are better at the game than others. Likewise humans are better on it than way to handle the problem (as far as we know) is the method made famous

16
by Sir Arthur Conan Doyle’s fictitious investigator Sherlock Holmes. That’s
why I refer to the “Sherlock Model” of microgenesis.
Consider a generic instance in British country life. The count is found dead,
with a kitchen knife sticking out of his chest. The village policeman arrives
on the scene and starts an investigation. What to do?
Portrait of Sherlock Holmes by Sid-
ney Paget. Sydney Paget was Sir
Arthur Conan Doyle’s preferred il-
lustrator.
There are some obvious possibilities to be dealt with first. The village
MD is called to figure out the probable cause of death, time, and so forth.
In case foul play is a possibility (the count may have stabbed himself, fell
into the knife by accident, have died by a heart attack before being stabbed
by some unknown person, and so forth), the policeman checks whether some
vagabond was recently spotted in the environment. If so, the villain is arrested
and the case considered closed, to everyone’s satisfaction. If not, then a full
forensic investigation is called for. The village policeman collects anything
even vaguely out of order (footprints, discarded cigarette butts, broken flower
pots, you name it). Nowadays he would check for DNA traces. The whole
scene is thoroughly photographed, searched for trace evidence, and otherwise
documented. Anyone within a mile is questioned (more or less discretely,
according to standing). Possible discrepancies are noticed. All of this is col-
lected into a file. The file is important as a record, because the scene will
soon change, e.g., the body is moved, and so forth. The volume of the file is
17
only limited by the eagerness of the policeman. Collectable items range from
the physical environment up to a mile from the body (say), going all the way
to the molecular scale (think of DNA traces). It is potentially infinite. Most
of this documentation will eventually prove irrelevant, but there is no way to
find out what will prove relevant. This is why the brute “bottom up” method
generally fails. The village policeman knows when he is beaten, and calls the
experts from the nearest large town.
Let’s say Sherlock Holmes takes on the case. He will not spend much
time on the file, he knowns most of its contents will prove irrelevant anyway.
Of course, he keeps a keen eye open for anything out of the extraordinary
though. But the main thing he does is frame one or more likely plots. It is not
too important what a plot is or where it comes from. Almost any plot is good
for a start. Remembering similar cases known from the literature, one likely
plot might be “the butler did it”.
Having a plot makes a huge difference. It immediately generates plenty of
questions:
◦ Where was the butler at the probable time of death?
◦ Did the butler have easy access to kitchen knifes?
◦ Since butlers tend to have flat feet, check the footprints!
◦ Did the butler hold a grudge against the count (e.g., took the count ad-
vantage of his daughter)?
◦ Did the butler stand to gain from the count’s death? (How about the
count’s most recent will?)
◦ In case cigarette butts were found, does the butler smoke? If so, do the
brands match?
◦ and so forth.
Each question can be checked. Each answer leads to further questions. Even-
tually it becomes possible to estimate the probability that the butler was actu-
ally involved. (You might use Bayesian reckoning for this.) In case the prob-
ability is low, Sherlock Holmes cheerfully adopts another plot (did the count-
ess do it?). For some plots the probability may amount to virtual certainty,
remember that the probability of a number of unlikely events is the product of
the individual probabilities. Coincidence of a few improbable events goes a
long way towards “beyond reasonable doubt”. Certainty cannot be achieved,
but “probability one” possibly can.
The method of “plots” is evidently a top down procedure. Its advantages
are obvious:
◦ One may search selectively in the records, the bulk of the file may be
ignored. This is a huge advantage
◦ One is led to novel questions, causing one to search for certain coinci-
dences, perhaps additional investigations. Diversification is conveniently
guided by ambiguous answers
◦ The more specific a plot is, the easier it is to rule out. Pruning can be fast
◦ The number of likely plots is low, one soon zooms in to ones that are
potentially rewarding, even when starting from scratch
The only way the method of plots will fail is when the murder turns out to be
a random event. However, the probability of that being the case is very low.
Thus the Sherlock Holmes method is ideally suited to handle cases that
are essentially “open”, but occur in environments that are well understood by
the investigator. In such cases the top down method is likely to succeed with
reasonable effort, even where the bottom up method is essentially hopeless.
Just imagine writing an algorithm that would compute the history of the crime
from the data (the potentially infinite file)! This is never going to succeed.
Indeed, all forensic investigations use the Sherlock Model.
Perhaps surprisingly, mainstream vision science has decided differently. It
purports to implement “inverse optics”, that is to say, to compute the scene
in front of the observer from the optical “data”. This is almost certain to be
a disaster, even in the best of worlds. More importantly, it is impossible to
compute meaning from structure. Algorithms convert structure into structure.
Structure cannot be computed, it has to be imposed. In the Sherlock Model
this occurs through the questioning on the basis of plots. The meaning is in
the questions, and derives from the plot. The “data” is simply garbage. It is
anything the dumb village policeman had occasion to collect. Any single item
is of relevance with probability zero.
All this goes to say that the Marrian “vision as inverse optics” is an im-
possibility, even an incoherent concept. Vision is more like “controlled hal-
lucination” (or “analysis by synthesis”), based on the Sherlock Model. It
is essentially a machine to generate Schrödinger–type micro-enlightenments,
18
eventually leading to presentations that are likely to increase fitness. The
evolution-like process of hallucination (generation of plots), followed by mer-
ciless pruning, and endless diversification is a machine that implements the
Sherlock Model.
Consequences for the visual front end
The term “visual front end” is usually reserved for the LGN, and the pri-
mary visual cortex, that is to say, the brain tissue nearest to the retina. The
task of the front end is usually made out to be “early vision”. The term early
vision is evidently based on the bottom up approach. In the perspective of the
Sherlock Model this is an obvious misrepresentation. “Early vision” (the “hal-
lucination”) would occur in the oldest parts of the brain, very different from
the neo-cortex. The cortical activity would be associated with the checking of
answers and thus be involved in the latest stages of microgenesis. We appar-
ently have to completely rethink these standard concepts from the mainstream
account.
One way to proceed is to remember that all visual awareness is awareness
of something. With “something” we mean an aspect of visual reality, not some
neural activity. You are not aware of optical structure, nor of neural activity.
You are aware of things like chairs, trees, colored patches, happenings, and so
forth.
Here is an example: you are marooned on a tropical island, and walk the
beach looking for signs of human presence. At one time you notice series of
regular depressions in the sand. You freeze, so there are humans here after
all! You try to be less optically conspicious, and you start to look for signs of
(hopefully!) civilized behavior. And so forth.
How to analyze the causes of the initial visual awareness? You noticed
“footprints”, but — of course — there were only depressions in the sand.
Any single depression might have been due to a multitude of distinct physical
causes. These never occurred to you because of the regular pattern of de-
pressions. Of course, animals leave tracks too. But it never occurred to you,
perhaps because of the geometrical configuration. You simply saw “human
presence”.
Foot prints in the sand. What else could they be? (Consider dust devils,
lightning, snakes, . . . .) Does any single depression look like a foot print?
Of course, a foot print is not a human. Spotting human would be the most
certain sign of human presence. Spotting part of a human might do, although
you sometimes read about stray feet found around the roads in the newspa-
pers. Foot prints will do as a surrogate. The optical structure is just as good
as the foot prints themselves. So is the retinal activity evoked by such optical
structure. So is the activity in primary visual cortex evoked by such retinal
activity. And so forth, and so forth. Eventually, the presentation will do. It is
many, many steps removed from an actual human presence though.
What is true for human presence is true for virtually anything. To be vi-
sually aware of something involves numerous layers of indirection. There is
19
not likely to be anything like a “foot print detector” in your visual cortex10 . It
is more like the visual cortex is able to take over the function of the sand of
the beach. The beach does not “represent” human presence, and neither does
your visual cortex. The “human presence” is an aspect of your presentations.
It is not any neural activity. Neural activity is just that, an electrochemical
field. There is nothing a neural activity, taken by itself, might “represent”. It
has no possible relation to “human presence” being in a completely different
ontological stratum.
Slightly wet beaches yield “better” foot prints than extremely dry beaches.
Even though the beach does not in any way represent human presence, some
beaches are better suited to detect human presence than others. This is the
way to think of the “front end visual system”: it is a volatile, read-only buffer
(the “writing” just happens from some outside source), continually overwrit-
ten due to novel optical structure (we’re dealing with physiology here, thus
physical causation applies). It stores the optical structure in a way that is es-
pecially revealing (like wet sand versus dry sand) and it tries to discard a pri-
ori irrelevant structure as much as possible. In a way, it is like the ideal file
made by the village policeman, continually updated. It contains meaningless
structure, most of it irrelevant, but it optimizes the record of the environment
in brain readable form. The visual cortex enables fast and accurate checks
on hallucinatory guesses, or (this sounds more serious) it enables answering
serious questions in the most effective manner.
Notice that such a notion is entirely different from the mainstream account.
The cortical activity is meaningless, and nothing is being “computed” (no
“inverse optics”). Yet the possession of such a fine tuned volatile buffer must
offer a tremendous evolutionary advantage. It is a huge step forward as com-
pared to the frog’s brain. Yet it is not different in essence. Only more effective
and articulate.
E MPIRICAL EVIDENCE showing that visual awareness is a mere user inter-
face, without any attempt at “veridical representation” abounds. I consider a
10
Or, for that matter, neither are there “edge detectors”, or “line detectors” in the visual
cortex. There are no edges or lines in the optical structure, these geometrical entities are
imposed by the microgenetic process, not “detected” or “found”.
few obvious examples.
The first example involves “linear perspective”. The simplest case usually
considered is that of “one point perspective”.
From this vantage point the cube, as pro-
jected on your retina, looks particularly sim-
ple. It is known as “one point perspec-
tive”. Both the front and back faces project
to squares, albeit of different size. The size
ratio depends on viewing distance. The “in-
verse optics” assumption would lead to be-
lieve that people would be well aware of
such relations.
People apparently are not aware of
the one point perspective relations at
all! Observers prefer a “template”
as “looking correct”, and complain
that the veridical renderings “look
distorted”. Apparently they take no
account of the “correct viewing dis-
tance” at all.
A cube seen in one point perspective is a planar pattern involving two con-
centric squares of different size. Their size ratio can be predicted on the basis
20
of the viewing distance relative to the edge length of the cube. In one exper-
iment we showed such figures on a screen to observers positioned at various
distances. It was the observer’s task to set the size ratio to a value such that the
figure appeared as a “perfect rendering of a cube”. If the ratio is close to one
the object looks like a flattish slab, if too different from one it looks like a long
corridor. Different from the naive prediction (on the basis of inverse optics)
all observers prefer a “template” rendering of the cube, quite independent of
viewing distance. Thus “veridical” looks “distorted”! This can be understood
as the result of a simplified interface.
Another example was suggested to me by reports from Helmholtz and
Kepler. Both men knew the visual field to subtend about a halfspace, but
noticed—to their amazement—that it appeared much narrower than that.
This is a view of the inside of a
hemispherical dome. The dome is
white, and covered with a statisti-
cally uniform distribution of iden-
tical black polka dots. Observers
are asked to draw (in a strictly
formalized protocol) the horizontal
cross-section of their pictorial relief
seen when they view the inside of
the dome with their eye (other eye
closed) at the center of the sphere.
In an experiment I had observers look into a patterned hemisphere and had
them draw a horizontal cross section of their impression of the relief. (Vi-
sion was monocular, and all kinds of technical precautions were in place.)
From these results I obtained the apparent visual field for several dozens
of observers. The results revealed a striking agreement with the reports by
Helmholtz and Kepler.
The larger group has an apparent visual field with a diameter of about ninety
degrees. A small group sees everything in front (very small apparent visual
field), and a really small group experiences a huge visual field, extending
Apparently most observers have a rather
“behind the ears”.
non-veridical experience of the diver-
gence of visual directions from the eye.
Their awareness is more like B than A.
Of course, this is perfectly fine in terms
Here are typical examples of the of the “awareness as user interface” no-
result. Most observers produce a tion. Case B makes a perfectly fine in-
drawing much like the figure at terface, and has the advantage of being
center. Apparently the full visual simpler than A.
field appears as much more narrow
to them. Some observers see the
whole visual field in front of them
The painting by Ferdinand Hodler is a striking example. The Lebensmüden
(top figure), a few see it as extend-
(“the weary of life”)) are positioned in strict military order, making for a
ing “behind their ears” (bottom fig-
striking painting. However, notice the edges of the bench that run into depth.
ure). The typical case has been no-
The persons at the left and right clearly are positioned at an angle, it can be
ticed spontaneously by a few experi-
estimated to about forty degrees.
enced observers (Helmholtz and Ke-
pler). Both report on their surprise,
because they understood the optics
quite well.

Not only the major result (most people experience a visual field that has less
than half the geometrical diameter), but also the fact that people can be very
different is highly interesting. It shows clearly that people have idiosyncratic
user interfaces. It also shows that people can still communicate easily, even
if their interfaces are diverse. After all, nobody is aware of the diameter of
another one’s apparent visual field. It is much like the old “is your red my Ferdinand Hodler’s elderly men don’t look particularly happy. They are lined
green?” —irrelevant for daily interaction. up in perfect military order. But notice the left and right sides of the bench,
The fact that most people do not experience the divergence of visual direc- compare it to the saggital planes of the figures. Hodler has rotated the figures
tions as they fan out from the eye does lead to observable effects in products by as much as forty degrees (when you do the perspective construction) in
of the visual arts. However, such effects have largely remained unnoticed. At order to make the scene “look right”!
least, I know of no literature on that specific topic.

21
 How can it be that such a huge rotation goes unnoticed? You can easily
reproduce the effect in photographs. A photograph of people in (actual!) strict
military order evidently “looks wrong”. In order to rectify it you have to
introduce the same type of rotations Hodler did.
Here is a simulation of the Hodler scene. Notice that the impression is not
the intended one (perfect military order) at all! The outermost figures look
rotated with respect to the fiducial straight-ahead direction. This impression
doesn’t go away if you put the eye at the “correct” perspective center. In fact,
you see it in the real scene when taking the photograph.
The optical user interface of most observers apparently refers the spatial
attitude of objects to the local visual direction, not to some absolute Euclidean
direction (the North-by-Northwest say).
We have turned these observations into “objective11 ” empirical data, with
11
That is to say: in the sense of experimental phenomenology. It is obviously impossible
to approach awareness by physiological means.
22
striking results. People easily commit errors in comparing apparent spatial
attitude of a hundred degrees and more! Perhaps surprisingly, the textbooks
are silent on this. Reason is—no doubt—their fear of first person accounts,
making them report on “non-invasive physiology” only.
This is how to photograph “strict military order”: Line up (or rather “cir-
cle up”) the people in a circular arc centered on the camera. Have them all
face the camera and look into the lens. Of course this is not “strict military
order” at all, but it certainly looks like it!
There can be little doubt that Hollywood directors are aware of such facts
and take measures to counteract adverse effects. After all, their aim is to con-
trol the visual awareness of their audience, not to produce a veridical record
of some scene (that is probably fake anyway!). Unfortunately, I could find no
records on the topic. Maybe the insights are kept as professional secrets, or
maybe they are simply considered obvious and not worth talking about.

The inside and the outside
A common objection against the “awareness as controlled hallucination”
concept is that this would prevent the agent to notice novel or unexpected
objects and happenings in the scene. The preferred mainstream example is
this: you are blindfolded. Some djinn flies you to the Sahara (or the North
pole, Mars, a harem, the bottom of the ocean, you name it), and rips off the
blindfold. People are reputed to instantly recognize the scene in front of them.
This is supposed to prove without reasonable doubt that vision is a bottom up
process, essentially simply inverse optics. Moreover, this is understood as
essentially the generic situation as human visual awareness goes.
Of course, there is much that might well be doubted here. It is after all only
a “thought experiment”, it would be of some interest to actually try it. There
might well be surprises. However, my major points relate to the conclusions.
First of all, such an experiment evidently fails to stand for the generic case.
In real life the scene in front of you rarely changes completely (from the Sa-
hara to the North pole say) in the blink of an eye. As a consequence, microge-
nesis typically has a head start. If the hallucination starts off with the previous
presentation, you’re generally in good shape. Secondly, you’re almost bound
to win any twenty questions game with nature, even when you have to start
from scratch. Your Umwelt is sufficiently structured for that. (Here we meet
Searle’s background and Minsky’s frames again.) If you found yourself in a
completely alien environment, things might well look different. This is indeed
the case, there are plenty of reports of “non-veridical awareness” in novel cir-
cumstances, both in our Western cultural environment (reports of micropsia
after the introduction of the railway, or (in the modern world) views from
an airplane), and in various non-Western populations (forest hunters first ex-
posed to open country seeing distant antelopes as ants). Thus the standard
mainstream account is perhaps largely a myth.
However, there is evidently some force in the standard argument. Of course,
it is the case that “bottom up processes” occur. I don’t doubt that, merely draw
different conclusions. Consider an example. Suppose you are lazily enjoying
the sunshine, generally feeling good, when you are suddenly stung by a wasp
23
(the description as viewed from the outside). You jump, perhaps yell. Here
we have stimulation of the sensitive body surface eliciting mechanical and
acoustical action. It is important to analyze the case in some detail though.
What happened in your awareness (the view from the inside)? Well, you
were aware of your body jumping and crying out loud, followed by an intense
feeling of pain. Then (but now we’re in the cognitive domain) you figured out
to have been stung by a wasp. Perhaps you looked for it, saw it, and killed
it. The important point is that the bottom up process ran essentially outside of
your awareness. One calls such stimulations “protopathic”.
This is typical for the bottom up stimulations, they belong to your sub-
conscious “zombie nature”. In the light of Schrödinger’s bridging hypothesis
this makes perfect sense. Protopathic stimulation does not depend upon pok-
ing, there is no question asked, thus no format. You are unable to learn in
such a way, there can be no micro-enlightment. Thus we come to the follow-
ing conclusion: there are both bottom up and top down processes involved,
the former are not accompanied by awareness, the latter are. Both are im-
portant to maintain your biological fitness, though in categorically different
ways.
In the case of visual awareness one may think of bottom up processes as
triggering (or merely enabling) certain classes of hallucinations. For instance,
one might expect the coarse, overall structure of the radiance to preferentially
enabling certain “gists”, like open or closed space, earth/sky, and so forth.
This would certainly give the microgenesis a head start, even if very general
and vague, a bit like in the twenty questions game opening questions like
“concrete or abstract?”, “mineral or vegetable?”. Having ideas before the
game starts is obviously useful.
From the perspective of biological implementation one would expect such
bottom up/top down processes to run simultaneously at many different levels
in the system. Biology always builds structure on structure. Moreover, the
mainstream dichotomy body–world is unfortunate. The body is part of the
world, and the world is part of the body. Thus there is not something like a
single, well defined body–world interface. Instead, there are numerous inter-
faces. Each interface may be associated with a microgenetic process. Quali-
ties and senses at the various levels are of course very diverse. Awareness as
you experience it is actually deeply rooted in many layers of complexity. As
you peel off layers you are always obtaining complete systems, albeit more
and more “sub-human”. The deep (say “reptilian”) core contributes only over-
all, emotive coloring to your awareness. One believes to observe signs of this
in cases of senile dementia.

Consequences for empirical research

The distinction between “visual awareness” and “scrutiny” or even

“reflective thought” should have important consequences for empir-
ical research. I just mention some basics.
Here is a basic experiment: I show two patches made up of arbitrarily
placed dots. The task is to “see” which of the patches has more dots (or, per-
haps, notice that these numbers are the same). I can tell you right away that
the left patch has a hundred red dots and the right patch a hundred-and-one
yellow dots. If I didn’t have to tell you (you saw that right away!), you most
likely have a certain kind of autistic mind. (This is one thread from Barry
Levinson’s drama (or road) movie “Rain man” of 1988.) Most people are like
me in that they have no clue. So most people are struck by the dialogue from
Rain man (see video clip, scene after Ray spills a box of toothpicks on the
floor):
Raymond: 82, 82, 82.
Charlie: 82 what?
Raymond: Toothpicks.
Charlie: There’s a lot more than 82 toothpicks, Ray.
Raymond: 246 total.
Charlie: How many?
Sally Dibbs: 250.
Charlie: Pretty close.
Sally Dibbs: There’s four left in the box.
This is beyond most of our mind frames.
Which cloud has more dots?
Most people cannot do the task in immediate visual awareness. Again,
most people can do the task easily with scrutiny in reflective thought. This is
because a simple algorithm immediately occurs to them:
A LGORITHM
— count the dots in each pattern
(assume the results are n and m)
— evaluate the expressions n ≥ m and m ≥ n
(assume the results are nGm and mGn
— if nGm ∧ mGn answer equal
— or else if nGm answer left, else answer right
The algorithm is certain to work. Thus smart observers can certainly do the
task easily when granted sufficient time.
A cognitive scientist would grant the time and obtain the obvious result.
The experiment would have shown beyond reasonable doubt that human ob-
servers can see dot number up to at least a hundred or so. Someone in ex-
perimental phenomenology would limit the exposure duration to a second or
so, thus ruling out scrutiny and reflective thought. Again, the result of the
experiment is obvious. The experiment would have shown beyond reasonable

24
doubt that human observers are blind as a bat to dot numbers in the region of
a hundred or so.
The “tooth pick count scene” from Barry Levinson’s drama film “Rain Man”
(1988). (Video clip.)
This is an experiment I made up to show you how important the role of the
experimenter is. The cognitive scientist and the experimental phenomenolo-
gist mean completely different things when they say that someone can “see”
or “is blind to” certain optical configurations. The literature is full of cases
where it is left up to you what the experimenter meant. It is most important
that you exercise your common sense here!
In my view “seeing” is a matter of momentary visual awareness, not of
reflective thought. The definition commonly used in main stream vision re-
search is often different though.
25
OTHER E B OOKS FROM T HE C LOOTCRANS P RESS :
1. Awareness (2012)
2. MultipleWorlds (2012)
3. ChronoGeometry (2012)
4. Graph Spaces (2012)
5. Pictorial Shape (2012)
6. Shadows of Shape (2012)
(Available for download here.)
A BOUT T HE C LOOTCRANS P RESS
The Clootcrans Press is a selfpublishing initiative of Jan Koenderink. No-
tice that the publisher takes no responsibility for the contents, except that he
gave it an honest try—as he always does. Since the books are free you should
have no reason to complain.
T HE “C LOOTCRANS ” appears on the front page of Simon Stevin’s
(Brugge, 1548–1620, Den Haag) De Beghinselen der Weeghconst, published
1586 at Christoffel Plantijn’s Press at Leyden in one volume with De Weegh-
daet, De Beghinselen des Waterwichts, and a Anhang. In 1605 there appeared
a suplement Byvough der Weeghconst in the Wisconstige Gedachtenissen. The
text reads “Wonder en is gheen wonder”. The figure gives an intuitive “eye
measure” proof of the parallelogram of forces. The key argument is
de cloten sullen uyt haer selven een eeuwich roersel maken, t’welck
valsch is.
Simon Stevin was a Dutch genius, not only a mathematician, but also an
engineer with remarkable horse sense. I consider his “clootcrans bewijs” one
of the jewels of sixteenth century science. It is “natural philosophy” at its
best.
26
27