Lev Manovich

The Engineering of Vision from Constructivism to Computers

INTRODUCTION 1
CHAPTER 1. VISUAL ATOMISM 8
1. INTRODUCTION 8
2. EXPERIMENTAL PSYCHOLOGY AND THE SCIENCE OF ART 11
3. VISUAL ATOMISM: A CODE FOR MASS COMMUNICATION 16
4. VISUAL ATOMISM AND THE MIND-BODY PROBLEM 22
5. VISUAL ESPERANTO 27
6. CONCLUSION 31
CHAPTER 2. I SEE, THEREFORE I THINK 35
1. INTRODUCTION 35
2. I SEE, THEREFORE I THINK 39
3. FIRST SIGNS OF REVOLT: VENN AND GALTON 50
4. "TO TEACH THE WORKER TO THINK DIALECTICALLY." 54
5. FREUD'S THEORY OF VISUAL REASONING 61
6. THE RISE OF THE DIAGRAM 68
7. VISUAL TECHNOLOGIES AND THE MIND 77
8. ANALOG ENGINE 85
CHAPTER 3. MAPPING SPACE 90
1. VISUAL NOMINALISM 90
2. "THE MOST IMPORTANT EVENT OF THE RENAISSANCE." 99
3. RADAR: SEEING WITHOUT EYES 115
4. 3-D COMPUTER GRAPHICS: INTERACTIVE PERSPECTIVALISM 125
5. COMPUTER VISION: AUTOMATION OF SIGHT 132
6. CONCLUSION 146
CHAPTER 4: THE ENGINEERING OF VISION FROM INKHUK TO MIT 150
1. NOT ARTISTS BUT ENGINEERS 150
2. INFORMATION THEORY: AN ENGINEER ANALYZES COMMUNICATION 156
3. THE INFLUENCE OF INFORMATION THEORY OR THE IDEOLOGY OF THE CODE 162
4. FROM "HUMAN MOTOR" TO "HUMAN INFORMATION PROCESSING" 173
5. COMMUNICATION ENGINEER ANALYZES HUMAN VISION 177
6. HUMAN ENGINEERING 186
7. CONCLUSION: THE LABOR OF PERCEPTION 193
CONCLUSION 196
BIBLIOGRAPHY 199
FIGURES 209


Introduction

The dissertation presents a history oI modern ideas about vision. I believe that vision is
not a timeless concept; rather, each period understands vision diIIerently depending on how it is
used. In the twentieth century, vision acquired new roles as the medium oI mass communication
and the instrument oI labor, and, as any other productive tool, it was subjected to engineering,
rationalization and automation. Such new disciplines as applied experimental psychology and
cognitive science, communication engineering and Iilm, robotics, and advertising design
continue to search Ior ways to utilize vision productively. In the process, they generate new
knowledge about vision, at the same time reducing it to a Iew disjoined and limited models. The
dissertation chapters Iollow the development oI Iour such models: vision as a code, vision as a
means oI logical reasoning, vision as a way to capture spatial inIormation, and vision as
inIormation processing.

Let us consider a Iew deIinitions oI vision that are representative oI entire research paradigms
and that were unthinkable beIore the middle oI this century.
David Marr's Vision, published in 1980, summarized a decade oI investigations on
human perception carried out at the MIT ArtiIicial Intelligence Laboratory. This book has been
the most inIluential account oI the computational approach to vision, shared by computer
scientists and psychologists. It opens with this statement:

What does it mean, to see? The plain man's answer (and Aristotle's, too) would be, to know what
is where by looking. In other words, vision is the process oI discovering Irom images what is
present in the world, and where it is.
1




1
David Marr, Vision (New York: W.H. Freeman and Company, 1982), 3.

There is nothing "plain" about this deIinition oI vision; it is Iunctional and pragmatic. Visual
perception is reduced to a number oI computational processes Ior the recovery oI limited
inIormation about the world: the identity oI objects and their positions. In Iact, this is the only
kind oI inIormation which may be required Ior a robot or an automatic missile to perIorm its
task. Marr projects these goals oI machine vision onto human vision. Here, vision is reduced to
the common denominator shared by humans and low level organisms: to detect an obstacle, a
predator, a prey.
In a special 1984 issue oI Cognition (the leading periodical oI cognitive science) devoted
to visual representations, Steven Pinker outlined the understanding oI vision commonly held by
cognitive psychologists:

Certain abstract problems could be best solved by translating their entities into imagined objects,
transIorming them using available image transIormations, detecting the resulting spatial relations
and properties, and translating those relations and properties back to the problem domain.
2


In this deIinition, vision is valued merely Ior its topological properties -- the ability to represent
such relations as inclusion, proximity, and relative positions. According to many cognitive
scientists, these properties make it a more eIIicient medium Ior problem-solving and abstract
thinking than language. And since the mind itselI is imagined as an exemplary model oI
computational eIIiciency that evolved through evolution, scientists postulate the centrality oI
visual (read: topological) representations Ior various mental processes. To think productively and
without waste, we might dispense with language and instead think through images, submerging
ourselves into the silent movie theater oI our minds.
Here is another model oI vision that narrows its deIinition to a particular productive
property. In a 1951 overview oI applied experimental psychology, the Iield which is today
known as human Iactors, Paul Fitts writes:

2
Steven Pinker, "Visual Cognition: An Introduction," Cognition 18 (1984): 66.


Each sense modality has certain inherent advantages and disadvantages Ior the detection and
analysis oI diIIerent kinds oI inIormation. Audition is more nearly a continuous sense than vision;
vision is basically selective and intermittent. As a consequence, audition is well adapted Ior the
detection oI warning stimuli that may arise at any moment Irom one oI a variety oI sources,
whereas vision is well suited to the selection oI and concentration on particular stimuli to the
exclusion oI others.
3


What is the purpose oI this research into the relative properties oI the senses? In contrast to a
manual worker, the job oI an operator in a modern human-machine system, be it an airplane
cockpit, radar display or an automated production line, is primarily perceptual in nature. His or
her work is to monitor the displays and to detect those signals that require intervention. Starting
in World War II, experimental psychologists have collaborated with engineers in the design oI
displays and perceptual strategies Ior their operators. One oI the important questions in this
research has been the relative advantages oI visual, auditory and tactile displays Ior the detection
and analysis oI diIIerent kinds oI signals. Consequently, Fitts describes vision as a sense which is
most reliable and eIIicient Ior the constant surveillance oI a single source -- it is "basically
selective and intermittent." According to this model, in order to use vision most productively,
one's eyes must be literally "glued to the screen."
In each oI these models, vision is understood in terms oI its eIIiciency in perIorming
speciIic tasks: recovery oI three-dimensional inIormation, logical reasoning, detection oI signals.
In Iact, there is no single meaning oI "vision" which is shared among them -- "vision" as such
does not exist. The only thing that unites these modern approaches to vision is the quest Ior the
eIIicient, reliable, and eIIective instruments oI labor. In this respect, vision is just a set oI
separate tools which can be employed by either a human or a machine to get work done.
It is this emphasis on the productive uses oI vision that distinguishes my project Irom a
number oI recent histories oI vision. In 1988 Hal Foster edited the anthology Vision and

3
Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook oI Experimental
Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1314.

Visuality which brought together the groundbreaking essays united by the common theme oI
historicising modern vision, that is, rejecting it as a natural or a cultural constant.
4
In the last Iew
years, each oI the Iive contributors -- Martin Jay, Jonathan Crary, Rosalind Krauss, Norman
Bryson, and Jacqueline Rose -- have developed their essays into books which today largely
deIine the direction oI the discourse on the visual in humanities.
5

At the center oI this discourse is the experience oI a seeing subject: how is what one sees
inIluenced by psychic structures, by gender and power positions, and by the constraints oI the
body; how is one's subjectivity determined by the experience oI seeing and being seen? At its
boundaries are works oI art and other cultural representations, which are interpreted as the reiIied
traces oI this changing experience.
The accounts oI these writers greatly expanded the horizons oI our knowledge about
modern vision. Yet, what remained unexamined is the employment oI vision Ior work. Indeed, it
is remarkable that the grounding philosophical and psychoanalytic models oI vision, inIluential
in this discourse, are theorized Irom the experience oI a "vacationer." For Sartre, it was the
person wandering in the deserted Parisian park; Ior Lacan, it was the young man (Lacan
himselI), "being observed" by a shining sardine can, while he was in a boat with some Iishermen
near a small port; Ior Freud, it was a young woman conIined to isolation in a Viennese apartment
or sent to a resort. Not only are these subjects Iar Irom a modern workplace, but they are also
away Irom the prototypical perceptual spaces oI modernity: the city street, the movie theater, the
shopping arcade.

4
Hal Foster, ed., Vision and Visuality (Seattle: Bay Press, 1988).
5
Martin Jay, Downcast Eyes: the Denigration oI Vision in Twentieth-century French Thought
(Berkeley: The University oI CaliIornia Press, 1993); Jonathan Crary, Techniques oI the
Observer: on Vision and Modernity in the Nineteenth Century (Cambridge: The MIT Press,
1990); Rosalind Krauss, The Optical Unconscious (Cambridge: The MIT Press, 1993); Norman
Bryson, Looking at the Overlooked: Four Essays on Still LiIe Painting (Cambridge: Harvard
University Press, 1990); Jacqueline Rose, Sexuality in the Field oI Vision (London: Verso,
1986).

It is precisely the experience oI these spaces that gave rise to a very diIIerent kind oI
theorizing oI vision in the writings oI Walter Benjamin. Scrutinizing these new spaces oI
modernity, Benjamin was among the Iirst to notice the contiguity between the perceptual
experiences in the workplace and outside oI it:

Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless, today's
pedestrians are obliged to do so in order to keep abreast oI traIIic signals. Thus technology has
subjected the human sensorium to a complex kind oI training. There came a day when a new and
urgent need Ior stimuli was met by the Iilm. In a Iilm, perception in the Iorm oI shocks was
established as a Iormal principle. That which determines the rhythm oI production on a conveyer
belt is the basis oI the rhythm oI reception in the Iilm.
6


Here Benjamin already notices that vision became a Iunctional and pragmatic activity rather than
a disinterested or contemplative one. The eye is trained to keep pace with the rhythm oI
industrial production at the Iactory and to navigate through the complex visual semiosphere
beyond the Iactory gates.
Among the contributors to Vision and Visuality, Jonathan Crary is the most sensitive to
the primacy oI work Ior the history oI vision in the modern period. In Techniques oI the
Observer, he examines how such nineteenth century disciplines as physiology and
psychophysics, and such techniques as optical apparatuses and illusions prepared vision Ior its
later productive deployment. This Iar-reaching archeology is extremely revealing, yet, because
the account ends in the middle oI the nineteenth century, it can outline the Iuture deployment oI
vision only in very broad terms: as participating in the process oI the rationalization oI labor and
as adapting to the new Ilux oI dematerialized images and signs. In order to understand what
really happened to vision in the twentieth century, how it was actually deployed and investigated,
Crary's Techniques oI the Observer is necessary but not suIIicient. This century has seen the
emergence oI disciplines and techniques oI vision that simply cannot be traced to the period

6
Walter Benjamin, "On Some Motives in Baudelaire," in Illuminations, ed. Hannah Arendt
(New York: Schochen Books, 1969), 175.

analyzed by Crary. In order to understand their current directions, we need to consider social
phenomena oI more recent origin such as automation, computerization, and "the inIormation
revolution." Most importantly, what could not have been predicted Irom Crary's account is the
radical shiIt in the nature oI labor itselI. Benjamin described the rhythm oI perceptual shocks oI
the assembly line worker oI the industrial era, who was nevertheless paid to perIorm physical
work: liIting, hammering, Iiling, and molding. Today, however, in the post-industrial era, the
same worker is paid Ior "perceiving" -- watching, detecting, scanning, and monitoring.
This dissertation is an archeology oI the currently prominent disciplines and techniques
oI vision -- computer vision and cognitive science, virtual reality and other new human-computer
interIaces. In this archeology, I try to uncover the recently accumulated layers -- 1920s, 1940s,
1960s. Rather than postulating a single paradigm shiIt, I describe a number oI distinct
approaches to the rationalization oI vision, which developed in parallel. Each chapter
investigates one oI these approaches.
The Iirst chapter is concerned with the model oI vision as code oI communication.
Investigations oI the psychological eIIects oI basic colors and elemental Iorms, conducted in
experimental psychology since the second halI oI the nineteenth century, made possible the idea
oI a rational visual language composed Irom simple elements. This idea acquired new
importance in the 1920s, when artists Iound themselves in the role oI the designers oI mass
communication. In Soviet Russia, artists collaborated with psychologists in order to create a Iully
rationalized visual code oI mass communication where each visual element is capable oI
communicating a meaning, producing an emotion or causing a behavioral response.
Visual reasoning is the topic oI the second chapter. While classical philosophy
considered vision unsuitable Ior reasoning, in the modern period this attitude is reversed. Freud's
model as deIined in The Interpretation oI Dreams, Galton's composite photography, Eisenstein's
intellectual montage, and the adaptation oI graphic diagrams in logic and in popular visual

culture are all symptoms oI this emerging understanding oI vision: as the most eIIicient medium
Ior logical thinking. With the centrality oI cognition Ior the post-industrial workplace, the notion
oI visual reasoning acquired new importance. Consequently, the work on the models, techniques,
and technologies oI visual reasoning, which was previously pursued by a Iew individuals, now
became a matter oI systematic research on the industrial scale in such Iields as cognitive
psychology and scientiIic visualization.
The third chapter addresses the role oI vision as a way to capture spatial inIormation.
Historically, this role has been played by the techniques oI perspectival representation. Between
1940 and 1960, these techniques were automated (computer vision, computer graphics) and also
expanded beyond the realm oI the visible with radar and other remote sensing instruments. The
automation oI these techniques necessitated a new paradigm oI vision research, which deIined
vision as a computational system Ior the reconstruction oI object positions in three-dimensional
space.
The last chapter takes up the understanding oI vision in terms oI inIormation processing,
which emerged aIter World War II. In the post-industrial society, vision became the principal
instrument oI labor, the most productive organ oI an operator in a human-machine system. In
order to monitor and improve the system's productivity, engineers and experimental
psychologists have adopted a common Iramework oI inIormation theory Ior both human and
machine components.
Today, the number oI new disciplines which study vision continues to expand: image
science, computer graphics, image processing, computer vision, research on human-computer
interIaces and so on. Why? Modern society relies on vision not only as a means Ior the
production oI subjectivity but Iirst oI all as a means oI economic production. II the Iirst use oI
vision has been extensively theorized, the second so Iar remained largely unexamined. I hope
that this dissertation will begin Iilling this missing gap in histories oI vision.

Chapter 1. VisuaI Atomism

1. Introduction

Let us compare two statements both related to the question oI visual eIIectiveness but made in
diIIerent periods.
Charles Henry was a French writer who is mainly remembered today Ior his theory oI
scientiIic aesthetics which greatly inIluenced Georges Seurat and Paul Signac.
7
Writing in the
1880s, he advocated that aesthetic responses to simple perceptual elements (which Ior him were
brightness, color, and line) need to be studied scientiIically:

Art pursues the expression oI the physiognomy oI things, and aesthetics studies the conditions in
which these things are satisIying; that is, when they are represented gay or sad, agreeable or
disagreeable, beautiIul or ugly...aesthetic things Ior us are reduced to Iorms, to colors, and to
sounds.
8


HalI a century later, in 1927, in the article Photography in Advertisement, Lszl Moholy-Nagy
states:

A modern engineer, iI his goal and the Iunctional purpose oI his work are clear, can without any
great eIIort make a product that is Iormally adequate and perIect in its economic construction. But
the photographic advertisements oI our time are not so easy to deIine. They don't come with
"user's instructions." Research into the physiological and psychological laws oI visual
eIIectiveness is still Iar behind the times, compared to the study oI the physical laws.
9


Both Henry and Moholy-Nagy argue that the creation oI visual artiIacts should be rationalized
and grounded in scientiIic psychological knowledge. However, the impetus Ior this
rationalization is Iundamentally diIIerent. For Henry, the Iunction oI visual artiIacts is artistic

7
See JosZ ArgYelles, Charles Henry and the Formation oI a Psychophysical Aesthetics
(Chicago: University oI Chicago Press, 1972).
8
Qtd. in Paul C. Vitz and Arnold B. Glimcher, Modern Art and Modern Science. The Parallel
Analysis oI Vision (New York: Praeger, 1984), 87. My analysis oI experimental psychology and
scientiIic aesthetics in this chapter is indebted to this groundbreaking book.
9
Lszl Moholy-Nagy, "Photography in Advertising," in Photography in the Modern Era, ed.
Christopher Phillips (New York: Aperture, 1989), 87.

expression; Ior Moholy-Nagy, it is to induce eIIects in the viewer. For Henry, the artiIacts are
situated in the realm oI the aesthetic, producing pleasure or displeasure, divorced Irom practical
liIe; Moholy-Nagy is concerned with everything practical -- changing belieIs, attitudes, behavior.
Henry calls Ior the rationalization oI beauty while Moholy-Nagy wants to rationalize the process
oI communication. He believes that modern designers have to grasp "the laws oI visual
eIIectiveness" to be able to control the viewer's response -- precisely and predictably. And
Iinally, while Henry's "scientiIic aesthetics" deals with the artistic expression and aesthetic
response oI an individual spectator, Moholy-Nagy's "laws oI visual eIIectiveness" are to be
employed by the newly emerged institutions oI mass communication, such as advertising.
In this chapter I will discuss one solution to the problem oI the rationalization oI vision
advanced by the 1920s artistic avant-garde. Investigations oI the psychological eIIects oI basic
colors and elemental Iorms, conducted by psychologists since the second halI oI the nineteenth
century, made possible the idea oI a rational visual language composed Irom simple elements --
"atoms" oI visual communication. This idea was already pursued in the nineteenth century by
such artists as Seurat and such theoreticians as Henry. However, in the 1920s, when artists Iound
themselves in the role oI the designers oI mass communication, the idea oI an atomistic visual
language acquired new urgency and importance. The comparison between the statements oI
Henry and Moholy-Nagy makes it clear that it was no longer a question oI scientiIic aesthetics;
oI a work oI art producing an aesthetic eIIect in an individual spectator. Now it became the
question oI rationalizing mass communication -- the question oI economic and political
importance. ThereIore, Moholy-Nagy writes about the need to precisely "engineer" photographic
advertisements, and Bauhaus welcomes experimental psychologists. The trend reaches its
extreme in Soviet Russia, where artists thought that they could control not just the consumer
habits oI segments oI society (as in the West), but the consciousness oI the whole country. Here
the program oI research into the "atoms" oI visual communication became most systematic,

translating into the establishment oI a number oI psychological laboratories at various art
institutions.
10
Their goal was to create a Iully rationalized visual language oI mass
communication where each visual element is capable oI communicating a meaning, producing an
emotion or causing a behavioral response.


10
These institutions included INKhUK (Institute oI Artistic Culture, Moscow 1920-24);
GAKhN (State Academy oI Artistic Sciences, Moscow 1921-30); VKhUTEMAS (State High
Art-Technical Studios).

2. ExperimentaI PsychoIogy and the Science of Art

As other nineteenth century sciences, experimental psychology approached its subject, the
human mind, by postulating the existence oI Iurther indivisible elements, the combination oI
which would account Ior perceptual or mental experience. II chemistry and physics postulated
the levels oI molecules and atoms and so Iorth, and iI biology saw the emergence oI the concepts
oI cell and chromosome, experimental psychology applied the same reductive logic to the human
mind. Psychologists divided sensory consciousness into diIIerent modalities: vision, hearing, and
tactility and proceeded to enumerate the elementary sensations oI each.
11

The Iirst psychological laboratory was Iounded by Wilhelm Wundt at the University oI
Leipzig in 1879. The general program oI Wundt's laboratory was to deIine the Iundamental
elements oI each modality and to Iormulate the laws according to which these elements are
combined. In 1896 Iormer student oI Wundt, E.B. Titchener, who brought experimental
psychology to the U.S., proposed that there are 32,800 visual sensations and 11,600 auditory
sensory elements, each just slightly distinct Irom the rest. Titchener summarized his research
program as Iollows: "Give me my elements, and let me bring them together under the
psychophysical conditions oI mentality at large, and I will guarantee to show you the adult mind,
as a structure, with no omissions and no superIluity."
12

This analysis oI visual experience into its elements seemed to have no practical utility. In
Iact, the new discipline oI experimental psychology developed within philosophy, both
intellectually and institutionally. In German universities oI the 1870s, more lecture courses were
devoted to psychology than any other branch oI philosophy except logic.
13
Wundt, Iormerly a

11
Julian Hochberg, "Sensation and Perception," in The First Century oI Experimental
Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates, 1979), 94.
12
Qtd. in Eliot Hearst, "One Hundred Years: Themes and Perspectives," in The First Century oI
Experimental Psychology, 25.
13
Kurt Danziger, Constructing the Subject: Historical Origins oI Psychological Research
(Cambridge: Cambridge University Press, 1990), 211.

proIessor oI philosophy, was quite content with the allocation oI psychology to philosophy and
had no interest in its practical applications.
14

Yet, even beIore the discipline oI experimental psychology was recognized oIIicially, it
had already Iound its Iirst eager users -- artists and aestheticians. This was not simply a question
oI creating works oI art which directly illustrated the new scientiIic theory, in this case,
psychologist's atomistic theory oI perception -- such as the works oI Impressionists and
Neoimpressionists which represented reality with dots and dashes, the painterly equivalents oI
sensory elements. Rather than illustrating an atomistic theory oI perception, some art theorists
and artists realized that they could adopt its approach as a basis oI a truly scientiIic aesthetic and
art.
What made this idea possible? As experimental psychologists split visual experience into
separate aspects (color, Iorm, depth, motion) and subjected these aspects to a systematic
investigation, Irom the outset they had been asking two questions. First, what physiological and
psychological mechanisms are responsible Ior the perception oI each aspect oI vision? Secondly,
what is the psychological eIIect on the subject oI the stimuli, such as elemental colors or Iorms?
It is this second question which was oI great interest to aestheticians and artists. By relying on
elemental colors or Iorms with known psychological eIIect, artists could now predict and
anticipate viewers' emotions. Conversely, aestheticians could now develop an objective aesthetic,
grounded in the scientiIic laws oI the emotional eIIects oI pictorial elements.
As an example oI this research, consider an aspect oI visual experience such as Iorm --
"visual conIiguration considered as distinct Irom such things as color and pictorial content and
Irom the representation oI objects."
15
While some psychologists were analyzing human
perception oI simple Iorms such as squares, circles, and straight lines oI diIIerent orientations,

14
Ibid., 39.
15
Vitz and Glimcher, Modern Art and Modern Science, 144.

others were investigating their psychological eIIects. In the 1870s a French proIessor oI art,
Charles Blanc, developed a theory oI the intrinsic psychological signiIicance oI vertical, straight,
and oblique lines.
16
Blanc popularized the ideas oI the Dutch painter and theorist Humbert de
Superville who around 1830 Iirst claimed that simple lines conveyed emotions by having an
inherent psychological response (Iig. 1):

Blanc argued that the straight line is a "symbol oI unity" and the curved line oI "variety." OI the
straight lines, the most important is the vertical -- since man stands perpendicular to the horizon
and since he is bilaterally symmetric. The horizontal line is next in importance and then the two
obliques. Blanc discusses three types oI human Iaces characterized by three orientations and by
the associated eIIects oI happiness, rest and sadness.
17


In the 1880s, Charles Henry Iurther advanced Blanc's ideas, developing the "aesthetic
protractor" to measure the harmony oI line angles (Iig. 2). Henry's device was designed to
measure "whether the angles between lines radiating in diIIerent directions Irom a single point
are harmonious."
18

It is signiIicant that the Iirst laboratory research on the aesthetics oI simple Iorms was
conducted in 1876 by Gustav Fechner -- the Iounder oI mathematical methods Ior the
measurement oI sensations which became the basis Ior experimental psychology. Later, at the
turn oI the century, many proto-Gestalt psychologists started to investigate the aesthetics oI
simple Iorms, isolating preIerences Ior squares, rectangles, ellipses, and triangles as well as
diIIerent lines.
19
Their publications became Iilled with "abstract" pictures consisting oI simple
Iorms.
This psychological research into the eIIects oI simple Iorms inIluenced Seurat, Signac,
Wassily Kandinsky, Paul Klee, and Piet Mondrian, among others. For instance, Seurat, who was

16
Ibid., 170.
17
Ibid., 169.
18
Ibid., 170.
19
Ibid., 155.

Iamiliar with the works oI Blanc and Henry, advanced a similar theory oI the intrinsic
psychological eIIects oI lines oI diIIerent orientation (Iig 3).
20
In another example, Kandinsky,
in Point and Line to Plane, advocated "microscopic" analysis oI three basic elements oI Iorm
(point, line, and plane) claiming that there exists reliable emotional responses to simple visual
conIigurations.
21
Equally telling oI Kandinsky's program are the titles oI the articles he
published in 1919: "Small Articles About Big Questions. I. About Point," and "II. About
Line."
22

At Iirst, artists would embed simple lines or Iorms in their otherwise representational
compositions. A typical case is Seurat who based the orientations oI lines in his major paintings,
such as Le Cirque (1890-91) and La Parade (1887-88), on the theory which he derived Irom
Blanc: "Gaiety...oI line, lines above the horizontal; calmness...the horizontal...Sadness...oI line,
downward directions."
23
Gradually, however, artists gave up representation altogether and began
to compose works which would consist solely oI the simple elements already studied by
psychologists. It is, thereIore, not accidental that the paintings oI Mondrian, Klee, and Kandinsky
look remarkably similar to the visual stimuli already widely used by psychologists in previous
decades. They are also experiments, a result oI a systematic investigation into what Kandinsky
called "the science oI art," the science which would allow the reliable communication oI any
emotional experience.

Proposing scientiIically based aesthetics, Henry wrote:


20
Ibid., 170-71.
21
Wassily Kandinsky, (1926), Point and Line to Plane (New York: Solomon R. Guggenheim
Foundation, 1947).
22
Yu. A. Molok, "'Slovar simvolov' Pavla Florenskogo. Nekotorye margonalii" (Pavel
Florensky's 'dictionary oI symbols.' A Iew margins), Sovetskoe Iskusstvoznanie 26 (1990): 328.
23
Qtd. in Vitz and Glimcher, 170.

That which science can and must do is to expand the agreeable within us and outside oI us, and
Irom this point oI view its social Iunction is immense in this time oI oppression and blind
conIlicts. It ought to spare the artist hesitations and useless attempts in assigning or indicating the
way in which he can Iind ever more rich aesthetic elements; it ought to Iurnish the critic a rapid
means oI discerning the ugly, so oIten inIormulable, however much it is Ielt.
24


Henry equates an artist with a scientist whose vocation is to provide the viewers with the means
oI escaping the pressures and the unpleasant experiences oI everyday liIe through the beauty oI
art. The artist is the therapist oI a society, and because oI the importance oI this role, he ought to
Iollow a scientiIic method in order to make his works most eIIective in helping the viewers to
Iorget the "oppression and blind conIlicts" oI modern existence.
Moholy-Nagy, in the already quoted statement Irom the 1920s, also compares the artist to
an applied scientist -- an engineer. The artist, however, is no longer removed Irom the realm oI
everyday liIe. Just as the modern engineer shapes every aspect oI material reality -- the machines
Ior living (architecture), the machines Ior transportation (airplanes, cars, trains), the machines to
produce other machines (the tools oI industrial production) -- an artist has the crucial role oI
shaping the psychical reality, controlling people's emotions and ideas through advertisements,
publicity, posters, and other visual propaganda.
Moholy-Nagy's appeal to designers and photographers brings Iorth the new purpose oI
the research into the elemental units oI vision -- the need to optimize the process oI mass
communication. This new purpose became crystallized at the moment when modernist artists
claimed the position oI designers oI mass propaganda in Soviet Russia in the 1920s. At this
moment the two lines oI inquiry -- artistic exploration oI the visual elements and research in
experimental psychology -- explicitly converge in Soviet discourses and institutions. In a number
oI Soviet art institutes oI the 1920s such as INKhUK (Institute oI Artistic Culture, Moscow
1920-24), GAKhN (State Academy oI Artistic Sciences, Moscow 1921-30), and VKhUTEMAS
(State High Art-Technical Studios), El Lissitsky, Alexander Rodchenko, Osip Brik, and others

24
Qtd. in Ibid., 87-88.

collaborated with experimental psychologists to investigate the eIIectiveness oI visual elements
and their combinations.
25




The paradigm oI research into the elements oI visual communication and their eIIects can be
conveniently traced through the history oI INKhUK, which during the Iour years oI its existence
united the key groups oI avant-garde artists and theoreticians most committed to this research.
In the Iirst year oI INKhUK's existence the group led by Kandinsky was the most active.
His program called Ior the cataloging oI basic elements oI art and to describe their emotional
eIIects.
26
Following Kandinsky's proposals, INKhUK established its own psychophysical
laboratory in the Fall oI 1920 but Kandinsky and his supporters were Iorced to leave a Iew
months later.
27
Kandinsky then became vice-president oI the newly Iormed GAKhN and the
head oI its largest physical-psychological division. The method oI Kandinsky's division was
characterized as "the observation oI and experimentation with the content and processes oI
artistic creation and reception as well the work oI art in all its complexity, understood as a

25
Other Soviet research institutes oI the 1920s whose programs included the development oI a
scientiIic approach to art and art history include: The State Institute oI Artistic Culture (headed
by Kasimir Malevich, Leningrad 1923-27); The Institute oI Literature, Art and Language, a part
oI Communist Academy (1918-36); The State Institute Ior the History oI the Arts (Leningrad
1921-31); The Institute oI Art History, a part oI Moscow University; The Russian Association oI
Research Institutes oI Social Sciences (Moscow 1924-31).
26
Wassily Kandinsky, (1920), "Programma raboty Instituta khudozhestvennoy kultury (The
Program oI the Institute oI Artistic Culture)," in Sovetskoe iskusstvo za 15 let. Materialy i
dokumentatsiia (FiIteen years oI Soviet art: materials and documentation), ed I. Matsa (Moscow,
IZOGIZ: 1933), 126-139.
27
The work oI INKhUK during Iirst year oI its existence has been documented in S.O. Khan-
Magomedov, "INKhUK: vozniknovenie, Iormirovanie i pervyy period raboty. 1920" (INKhUK:
appearance, Iormation and Iirst period oI its work, 1920), Sovetskoe Iskusstvoznanie (1981):
332-368. 3. VisuaI Atomism: a Code for Mass Communication

psycho-physical object."
28
The members oI the academy included art historians, practicing
artists, physicists, psychologists, and physiologists.
Meanwhile, Alexander Rodchenko became the new president oI INKhUK. Rodchenko
and his supporters Iormulated a new program in total opposition to Kandinsky. Against
Kandinsky's "subjective" psychologism they emphasized the materiality oI objects. They also
opposed Kandinsky's "individualistic" composition in Iavor oI construction and promoted the
making oI objects rather than the creation oI art.
29
However, the project oI a science oI visual
elements was not abandoned in practice, only the emphasis shiIted Irom the "subjective" analysis
oI individual aesthetic reactions to the "objective" analysis oI the elements as maniIested in
material objects. INKhUK members, who started to teach at the newly Iormed VKhUTEMAS
(State High Art-Technical Studios), had the opportunity to put these ideas into practice.
30
For
instance, in his course on graphics Rodchenko would start his students with systematic exercises
to combine simple Iorms (circle, triangle, and square) within a given Iormat in order to test the
visual eIIectiveness oI various combinations (Iig. 4).
31

In its third stage INKhUK became the center oI activity oI the promoters oI the idea oI
production art -- Osip Brik, Boris Arvatov, Nikolai Tarabukin, and others.
32
In 1923, this group
Iounded the journal LEF (The Journal oI the LeIt Front oI Art, Moscow 1923-25) the goal oI
which was to unite artists oI the LeIt dedicated to radical revolutionary activity in the Iace oI
what they perceived as the new Ilourishing oI "bourgeois" art made possible by the NEP.

28
P.C. Kogan, "O zadachakh akademii i yeyo zhurnala" (About the goals oI the academy and its
journal), Iskusstvo 1, no. 1 (1923): 9.
29
A number oI early constructivist maniIestos are now available in English translation in
Nicholas H. Allison, ed., Art Into LiIe: Russian Constructivism 1914-1932 (New York: Rizzoli
International Publications, 1990), 61-82.
30
It was not accidental that the artists reIerred to their activities at INKhUK as "laboratory
work." "Laboratory" connoted scientiIic labor, rather than artistic experimentation. "Institut
Khudozestvennoy Kultury" (The Institute oI Artistic Culture), Russkoe Iskusstvo 2-3 (1923): 85.
31
A.H. Lavrent'ev, "Propedevticheskaya distsiplina 'GraIika.' Vkhutemas. 1920-22 gody" (The
discipline 'Graphics.' VKhUTEMAS. 1920-22), Tekhnicheskaya Estetika 7 (1984): 19.
32
"Institut Khudozestvennoy Kultury," 86.

What should this radical revolutionary activity be? Western art historians, critics, and
artists gloriIied the constructivists' desire to give up the traditional roles oI artists and instead
submerge themselves in industry, to leave the artist's studio Ior the Iactory -- in other words, to
become industrial designers.
33
What received practically no attention was another role which
constructivists and many other Soviet artists were claiming Ior themselves in the 1920s -- the
role oI theoreticians and practitioners oI mass visual communication, be it propaganda or
advertisement. And it was in this role, particularly as advertising designers, that VKhUTEMAS
artists, as well as their counterparts Irom the Bauhaus, were able to accomplish the transition
"Irom an easel to a machine" (to quote the title oI a book by Tarabukin, one oI the theoreticians
oI production art).
34
These machines were not a lathe or an engine but a printing press, a Iilm
camera, a light projector, a radio transmitter -- the machines oI mass communication rather than
mass production.
LeIt artists were united in rejecting art in Iavor oI mass propaganda. In the Iirst issue oI
LEF Sergei Tretyakov writes: "Revolution put Iorward new practical tasks |Ior artists| --
aIIecting mass psyche, organizing the will oI the class"; "The work oI Iuturism is parallel to the
work oI communism -- Iuturism is Iighting Ior the same dynamic organization oI personality
without which the movement toward communism is impossible."
35
For LEF, this task oI the
revolutionary education oI the masses required a scientiIic method and in the same article
Tretyakov called Ior the creation oI a "science oI art as a means oI emotional and organizing

33
In 1923, Lunacharsky wrote: "They |constructivists| play at being engineers, but they don't
know as much oI the essence oI machinery as a savage." Qtd. in Brandon Taylor, Art and
Literature under the Bolsheviks (London: Pluto Press, 1991), 1: 177.
34
Between 1923 and 1925 Rodchenko designed advertising Ior many Soviet government
agencies; Lissitsky, who was sent to Germany to publicize the art oI the Iirst Socialist state,
created brilliant designs Ior many German Iirms; so did Moholy-Nagy and other Bauhaus
Iaculty.
35
Sergei Tretyakov, "Otkuda i kuda? (Perspektivu Iuturizma)" |Where Irom and where to?
(Perspectives oI Iuturism)|, LEF 1 (1923): 197, 203.

inIluence on psyche, in connection with the task oI class struggle."
36
In Iact, artists should
become as scientiIic in their work as Marxists are in their political activity. To do so, they should
rationalize the creative process and put art education on a scientiIic basis.
37
This idea was most
systematically promoted by Proletcult (Organization oI the Representatives oI Proletarian
Culture). Thus, Tarabukin organized the artist's workshop aiming to bring the principles oI
Taylorism and Fordism into artistic practice: "We are analyzing processes oI work, experiment
with compressing work periods, aim to rationalize work and leisure. In its practice the workshop
tries to get away Irom bohemian habits so common to artists."
38
Artists should also rationalize
viewers' reactions by developing a science which would enable them to produce predictable and
measurable eIIects in the viewer. Thus, the author oI a 1927 article which summarized the
activities oI the Art Section oI Leningrad's Proletcult writes:

Artist-communist, artist-revolutionary came to understand the necessity to perIectly master his
weapons to be able to strike without a miss. The striving to conscious mastery was put Iorward as
the main task ("it is necessary to know in order to produce"). Artists oI Proletcult were sure that at
the basis oI art's eIIect on perception are to be Iound completely objective laws. From this came
the task to master the "mechanics" oI artistic work, in order to orient it toward new utilitarian tasks
oI revolutionary art.
39


This dual goal oI rationalization was symbolized by the popular metaphor oI artist as
"psycho-engineer," Irequently encountered in the discourse oI constructivism, Proletcult and

36
Ibid., 199.
37
LEF required the same Irom poets and writers who could make their work scientiIic by
relying on the advances made by Russian Formalists in studying the devices oI literature. For
LEF, these devices became the tools oI verbal propaganda. In the Iirst issue oI LEF Osip Brik
writes: "OPOYAZ studies the laws oI poetic production... OPOYAZ will help the art oI the
proletariat, but not with vague conversations about the 'proletariat's spirit' and 'communist
consciousness,' but with precise technical knowledge oI the devices oI contemporary poetic
work." Osip Brik, "T.n. 'Iormalnyi metod'" (So called 'Iormal method'), LEF 1 (1923): 214-215.
The meaningless experimental poetry oI Futurists which originally made Formalists recognize
the "independent value oI the poetic sound" now became Iull oI meaning; the techniques oI
"poetic instrumentation" were now seen as the techniques oI organizing the consciousness oI the
masses.
38
I. Matsa, Sovetskoe iskusstvo za 15 let, 279.
39
Ibid., 281.

LEF. Aleksey Gastev, a leading Iigure oI Proletcult as well as oI the Soviet Taylorist movement
who in 1921 organized the Central Institute oI Labor coined the phrase "social engineering" to
describe the process oI the radical reconstruction oI the human psyche.
40
Tretyakov applied the
metaphor to art: "Along with man oI science the art worker should become psycho-engineer,
psycho-constructor."
41

The crucial question then becomes how to rationalize visual eIIectivity. In principle,
diIIerent approaches are possible: to create particular conditions oI reception; to limit the number
oI available images in the public sphere (this was done later in the period oI Socialist Realism);
and Iinally, to create a particular code, a new visual language guaranteed to communicate
eIIectively and eIIiciently to mass audiences (oIten, in the case oI Soviet Russia oI the 1920s,
illiterate). It is the last solution which was explored by LeItist artists such as Rodchenko or
Lissitsky.
Why did the notion oI a visual language consisting oI simple abstract Iorms become
attractive? First, abstract geometric Iorms were neither associated with bourgeois Iigurative
representation nor did they look like the "subjective" abstract improvisations oI Kandinsky.
Secondly, as Igor Golomstock points out, abstraction was justiIied by emphasizing its stylistic
origin in truly popular decorative and religious art (Ior instance, icons).
42
Thirdly, simple Iorms
were proclaimed to be eIIicient in a situation oI mass communication. Thus, Lissitsky argued:
"The most unambiguous and immediately recognizable Iorms are geometric Iorms. No one will
conIuse a rectangle with a circle, or a circle with a triangle."
43
Finally and most importantly,
such Iorms were imagined to be perIectly suited Ior the purpose oI controlling the eIIect in the
viewer -- both the idea and the imagery being prepared by decades oI investigations in the

40
Igor Golomstock, Totalitarian Art (New York: HarperCollins Publishers, 1990), 26.
41
Tretyakov, "Otkuda i kuda?," 202.
42
Golomstock, Totalitarian Art, 26.
43
Qtd. in Golomstock, Totalitarian Art, 24.

psychology oI perception and in scientiIic aesthetics. The tradition oI research into the
eIIectiveness oI visual elements transIormed into the dream oI mass "psychophysical culture"
advanced in Soviet Russia in the 1920s.
The most striking record oI this dream is the paper "Engineerism" which was presented at
one oI INKhUK's meetings. The paper starts with an analysis oI modernism as a reduction oI
visual experience to its elements: "In the Iinal stage oI its development art has renounced
representation and has moved toward visual sensations as such."
44
The trend continues with
abstract art, the aim oI which, according to the author, is "the organization oI visual perceptions."
Bourgeois art cannot go beyond this stage oI analysis, but in the Iirst proletarian state, this
analytic work will become the basis oI a new visual culture oI social control: "Visual sensations
as such will concertedly shape the human being as an organized unit, with the tempo oI
something which belongs only to its own time."
45

From this perspective, we also should not be surprised to Iind in the same period the
recurrent attempts to establish various graphic alphabets and dictionaries. The poet Velemir
Khlebnikov in his 1919 article "Artists oI the World" postulated two types oI a universal visual
code -- graphic signs and color equivalents. In the already mentioned INKhUK program,
Kandinsky proposed to create a Iull dictionary oI colors, Iorms and their combinations with
descriptions oI the psychological eIIect oI each element. Pavel Florensky together with his
students at VKhUTEMAS started to compose "Symbolarium" ("The Dictionary oI Symbols") in
1922.
46
This "international and ahistorical dictionary" should have contained all cultural
meanings oI every graphic symbol. In the introductory article oI "Symbolarium" Florensky
regretted that while whole academies were busy preparing and preserving the dictionaries oI

44
Allison, Art Into LiIe, 79.
45
Ibid., 81.
46
P.A. Florensky, (1922), "Symbolarium (Slovar simvolov). Predislovie" |Symbolarium (The
dictionary oI symbols). Introduction|, Trudy po znakovym sistemam V (Tartu: Uchebnye zapiski
Tarturskogo Universiteta 284, 1971): 521-527.

national languages, no work yet has been done in composing a similar dictionary oI ideograms --
the "universal language oI mankind."
47
Similar projects oI a universal graphic alphabet were
advanced in the same years by Ivan Puni, Petr Miturich, and other Russian artists.
48




To summarize: early Soviet attempts to theorize and put into practice a scientiIically grounded
visual language oI mass communication bring together artistic exploration oI visual elements and
research in experimental psychology. While the desire oI Soviet artists to emulate "scientiIic"
Marxism and to adopt Taylorism and Fordism were already suIIicient reasons to pursue the
science oI elements oI art, the ultimate justiIication Ior such a science was provided by
experimental psychology and scientiIic aesthetics (Blanc, Henry, and others). Their
investigations oI the psychological eIIects oI basic colors and simple Iorms prepared the idea oI
a language oI mass communication composed Irom simple elements. In theory, this model called
to enumerate visual elements and to describe their eIIects in order to compose dictionaries oI
"visual eIIectiveness." In practice, it translated into the creation oI works composed Irom distinct
and simple abstract Iorms.
From 1921 to 1929 the psychological studies oI viewers' responses to visual Iorms were
actively conducted in the central art institutes. In 1926 GAKhN created a psychophysiology
laboratory, a psychophysics department, and laboratories Ior experimental aesthetics and art
theory.
49
In order to arrive at the general laws oI the eIIectiveness oI visual Iorms the
investigators have systematically analyzed the basic dimensions oI volume, line, color, and

47
Ibid., 527, 523.
48
Molok, "'Slovar simvolov' Pavla Florenskogo," 322.
49
E.L. Beliajeva, Arkhitekturno-prostranstvennaya sreda goroda kak obyekt zritelnogo
vospriyatiya (Architectural and spacial city environment as an object oI visual perception)
(Moscow: Stroyizdat, 1977), 13. 4. VisuaI Atomism and the Mind-Body ProbIem

texture. It was hoped that the results oI the analysis would put visual communication on a
scientiIic basis. Jack Chen, a Communist British artist trained in the Soviet Union during the
1920s, wrote about this program oI art production based on psychological research:

Art must be a science, an industry. Pictures and sculptures should be constructed according to
exact scientiIic principles aIter colors and Iorms had been classiIied according to their human
reaction values...A picture according to the LeIt was really nothing but a "machine" Ior generating
certain predetermined human reactions. Artists should be engineers oI Iorm and color.
50


There does not appear to be a disagreement among the LeIt as to the Iunction oI art -- to
create predetermined viewer responses. But how was this to be achieved? In other words,
assuming that we have isolated the elements oI a visual code, how should these elements aIIect
the viewer? Do they induce emotions, do they communicate ideas or do they control behavior in
a more direct way?
To answer this question, Soviet artists oI the 1920s relied heavily on the psychological
theories oI the day.
51
It is possible to discern two distinct trends in how psychology was used. In
the Iirst trend, autonomy was granted to the human psyche. The second trend privileged
physiology, considering the mind a neurological organ not diIIerent Irom the rest oI the
organism. This trend was represented by such inIluential scientists as Bekhterev and Pavlov,
with their studies oI physiology, reIlexology, and conditioning.
The latter model was also prevalent in Soviet Marxist psychology in the 1920s. In 1924 it
was challenged by Lev Vygotsky, a young psychologist and a Iormer literary critic. Recognized
today as the Iorerunner oI cognitive psychology, he argued that the higher cognitive Iunctions
are qualitatively diIIerent Irom the lower physiological processes and require diIIerent

50
Jack Chen, Soviet Art and Artists (London: The Pilot Press, Ltd., 1945), 58.
51
For the history oI Soviet psychology in the 1920s, I have consulted David Joravsky, Russian
Psychology: A Critical History (Cambridge, Mass.: Basil Blackwell, 1989); Alex Kozulin,
Psychology in Utopia: Toward a Social History oI Soviet Psychology (Cambridge: The MIT
Press, 1984).

investigative approaches. In his Iamous address to the Second Russian Psychoneurological
Congress, Vygotsky declared human consciousness to be a Iundamental problem in the
psychology oI behavior, claiming that it cannot be understood through the study oI reIlexes.
SigniIicantly, the turning point oI Vygotsky's career Irom literary critic to psychologist was his
1924 dissertation Psychology oI Art. Through the analysis oI Hamlet Vygotsky attempted to
demonstrate that art is a product oI the capacities oI the speciIically human mind -- imagination,
emotion, and symbolic representation.
Crucial Ior Vygotsky's theory was the study oI the Iormal organization oI works oI art:

...every work oI art is considered to be a system oI stimuli, which are organized consciously and
intentionally in order to cause an aesthetic reaction. This way we reconstruct the structure oI the
reaction while analyzing the structure oI the stimuli...The general direction oI this method can be
summarized in the Iollowing Iormula: Irom the Iorm oI the art work through the Iunctional
analysis oI its elements and structure to the reconstruction oI the aesthetic reaction and the
discovery oI its general laws.
52


In Vygotsky's Iormula the given was the existing works oI art and the unknown was the laws oI
aesthetic reaction. As we have already seen, Ior the artists, designers, and Iilm directors oI the
time the Iormula was reversed. II Ior the psychologist Vygotsky, visual works represented a
reservoir oI knowledge about the human mind, artists, on the contrary, were hoping to utilize the
objective psychological knowledge in order to create visual works which would produce pre-
determined responses in the viewer.
Contemporary psychologists -- the supporters oI Pavlov and Bekhterev on the one hand,
and younger psychologists such as Vygotsky and A. Luria on the other hand, argued over
whether the mind could be reduced to the laws oI physiology. Similarly, the artists in theory and
in practice drew on these two alternative psychological models. Sergei Eisenstein's attempts to
ground his Iilmmaking methods in diIIerent psychological theories epitomizes the two models.
In one oI his latest written works Eisenstein summed up his work in Iilm:

52
Lev Vygotsky, Psikhologija Iskusstva (Psychology oI art) (Moscow: 1968), 39-41.


I never intended to "reIlect" the existing reality. I had one task -- using the means oI its inIluence
-- to aIIect the Ieelings and thoughts, to inIluence the psyche and to shape the consciousness oI
the viewer in the desired, required, and chosen direction.
53


Although Eisenstein's goal remained the same, the means oI achieving the desired eIIect were
conceived oI diIIerently throughout his liIe.
On the one hand, Eisenstein developed the concept oI "intellectual montage," privileging
a purely intellectual response. The goal oI cinema, in this view, was thought to induce
dialectical reasoning. The structure oI the Iilm itselI was conceived oI as dialectical thinking in
visual Iorm with montage being the means oI representing the dialectical process through the
contrast and juxtaposition oI images. This preoccupation oI Eisenstein with cinema as the
visualization oI the work oI the intellect culminated in his project to create a screen adaptation oI
Marx's Capital.
54

On the other hand, Eisenstein's second important concept developed in the Montage oI
Attractions takes the viewers' physiological reaction as a point oI departure.
55
Eisenstein based
his concept on the psychological theories oI Ludwig Klages and William James.
56
According to
Klages, in a human being emotional states are expressed through bodily movements. Klages also
insisted that human expressivity is characterized by a unique quality -- the muscular contractions
oI one person are involuntarily repeated by the observer. James' theory was related to Klages' but
causally reversed. He postulated that emotions were the eIIect oI muscular contractions -- one
does not cry because he is sad, but one becomes sad due to crying. Eisenstein combined the two
theories: the emotional state oI the actor translates into his muscular movements; these

53
Qtd. in Yu. A. Vasilieva, "Eisenstein: Kontseptsija 'Agressivnogo Iskusstva'" (Eisenstein: the
concept oI 'aggressive art'), Kinovedcheskije Zapiski 3 (1989): 207.
54
I will return to Eisenstein's "intellectual montage" in chapter 2.
55
Sergei Eisenstein, "Montazh atraktsionov" (Montage oI attractions), LEF 3 (1923): 70-75.
56
Olga Bulgakova, "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu
psikhoanalizom i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin':
between psychoanalysis and structural psychology), Kinovedcheskie Zapiski 2 (1988): 178-80.

movements are involuntarily repeated by the viewer causing him to experience similar emotions.
The important issue, then, was the training oI actors in simulating precise gestures and Iacial
expressions in order to produce a desired emotional response in the viewer. Eisenstein proposes
to develop a collection oI emotional stimuli which would be strung together in a Iilm --
"montage oI attractions." The Iilm becomes a script oI the emotional responses oI viewers.
The two concepts -- the intellectual montage and montage oI attractions -- are the rare
summits in Eisenstein's intellectual career where the opposing tendencies oI his thought
concerning the modes oI cinematic inIluence on the viewer can be clearly distinguished. Most oI
the time they seem to have been intertwined, appearing simultaneously in his changing
conception oI cinematic eIIectivity. Similarly, in the cultural imagination oI the 1920s, the two
tendencies appear to be running side by side. Is there a mind, a speciIically human autonomous
cognitive apparatus, with operational laws not reducible to simple reIlexes? Or can human
behavior be controlled by aIIecting the organism through a set oI predictable physiological
stimuli?
This second position received its extreme elaboration in the theory developed by
Emmanuel Enchmen, an early Socialist revolutionary.
57
Enchmen's "theory oI new biology" was
based on Pavlovian reIlexology and proposed a purely proletarian culture based directly on
conditioned reIlexes, without the necessity oI conventional language or thought. The new
proletarian language would consist oI conditioned grunts and gestures and replace the verbal
culture oI bourgeois society.
All in all, the co-existence oI the two schools in Soviet psychology lead to the co-
existence oI the two kinds oI models oI artistic aIIectivity: aIIecting consciousness by
communicating ideas and bypassing consciousness altogether. This dichotomy is reIlected, Ior
instance, in the Iollowing statement Irom "New Artistic Program oI Proletcult" published in

57
Joravsky, Russian Psychology, 212.

1923: "Art should become the necessary part oI everyday liIe both in its active-representational
Iorms (poster, advertisement, agitational theater, cinema) and in its material-organizational Iorms
(psychophysical culture, organization oI mass happenings, processions and demonstrations,
construction oI things)."
58
The opposition "representational" -- "organizing" reIlects two views
oI aIIectivity: aIIecting the mind and aIIecting the body.
These diIIerent models oI aIIectivity were also used to understand the roles oI visual
elements which were being "isolated" and "puriIied" in the laboratories oI the 1920s. The
elements could have aIIected the viewer in more than one way. No longer just catalysts oI basic
emotional states, as in the tradition oI experimental psychology and scientiIic aesthetics, they
were now re-interpreted as capable oI carrying complex messages addressed to the mind, and as
simple signals designed to trigger physiological and behavioral responses.





I have discussed diIIerent models oI aIIectivity which underlined the research into the elements
oI a visual code. But what kind oI code can be Iormed Irom such elements?
Semiotician Luis Prieto distinguished between codes with and without articulation.
59
A
code with articulation consists oI a number oI elements combined to produce messages. The
examples are human language, playing cards, and maritime Ilag codes. The crucial Ieature oI any
oI these codes is that they allow Ior the generation oI a large or even inIinite number oI messages
Irom a small set oI elements. For instance, in the case oI human language, its speaker can
generate an inIinite number oI sentences. In contrast, in codes without articulation there are as
many meanings as elements. For example, in the language oI Ilowers, each Ilower signiIies a

58
Matsa, Sovetskoe iskusstvo za 15 let, 261.
59
Luis Prieto, Messages et signaux (Paris, PUF, 1966). 5. VisuaI Esperanto

distinct meaning (Ior instance,"red rose" "victory is yours"); a combination oI Ilowers does not
create any new meanings.
In view oI this distinction, the research into visual elements could justiIy both a code with
articulation as well as a code without articulation. In other words, it is possible to understand the
elements in two ways: as carrying a complete message (a verbal meaning, an emotion or a
particular behavioral response), or as just a part oI the message, like a word in a sentence,
combined with other words through grammar.
The artistic tradition oI visual atomism, Irom Blanc to Kandinsky, appears to Iavor the
Iirst interpretation. In their writings, Blanc, Henry, and Seurat correlated the orientation oI
straight lines with diIIerent emotions such as gaiety or sadness. Kandinsky's Iamous
questionnaire, compiled in 1920 while he was active in INKhUK, also indicates that he
considered the elements to directly correspond to emotions or verbal meanings: "How do you see
a triangle -- do you Ieel that it is moving, where, do you see it as more witty than square; is the
sensation Irom a square similar to the sensation Irom a lemon; what is a canary's song more like -
- a triangle or a square."
60

A code without an articulation requires as many signs as there are potential messages and
thereIore is not eIIicient. Yet, it is this ineIIiciency which makes it ultimately suitable Ior new
communicative situations Iaced by artists and designers in the earlier decades oI this century.
TraIIic signs, the labels on instrument panels, trademarks had to be unambiguous and instantly
recognizable by an untrained eye. Not surprisingly, the designers in every Iield have adopted
simple abstract Iorms. Thus, in 1909, when Mondrian and Kandinsky were moving toward their
Iirst abstractions, the Iirst International ConIerence on the Regulation oI Automobile TraIIic took

60
Qtd. in Khan-Magomedov, "INKhUK," 346.

place in Paris. Following its recommendations, in the 1910s-20s, universal pictograms signiIying
main road situations were developed and employed.
61

For the Soviet "psycho-engineers," the notion oI a visual code without articulation, which
correlates each message with a separate sign, had a particular attraction. The behaviorist
psychologies oI Pavlov and Bekhterev popularized the notion oI directly controlling behavior
through conditioned reIlexes. Simple visual Iorms could have acted as such stimuli, controlling
"the proletarian zombie" as the lights and whistles already controlled a dog in Pavlov's
laboratory. In this interpretation, the dictionary oI visual elements would contain not a set oI
meanings or emotions but a set oI commands -- previously learned responses to stimuli. This is
what the author oI the "Engineerism" paper seems to have in mind when he writes oI the Iuture
where "visual sensations as such will concertedly shape the human being as an organized unit."
In other words, just as the motorists in the 1920s were beginning to memorize the international
traIIic signs, learning how to respond to them automatically (today, this is the closest most oI us
come to experience conditioning through a visual code), artists took this as a paradigm Ior
controlling the masses through visual communication.
Yet, while suitable Ior communication oI basic emotional states or a set oI behavioral
stimuli, the code without articulation is not practical Ior communicating an arbitrary large
number oI messages. II visual language is to compete with a verbal language, and even to replace
it, visual Iorms have to be interpreted as equivalents oI letters or words. They also have to be
supplemented by grammar, the rules oI how these visual "words" are combined to Iorm
sentences.
Indeed, contemporary debates on whether such animals as dolphins or primates can use
human language center around the attempts to teach them to use visual signs in this way. It is

61
A.S. Sardarov, "5000 let evolutsii doroznogo znaka" (5000 years oI the evolution oI a traIIic
sign), Tekhnicheskaya Estetika 9 (1984): 14-19.

easy to train animals to respond to visual Iorms as distinct messages: a square means "take Iood,"
a circle means "jump," and so on. The question is whether animals can string such symbols
together to Iorm new messages, thus treating the individual symbols as words rather than as
complete messages. At least a Iew researchers have claimed that they have taught some animals
to do so, thus teaching them a visual code with articulation.
What about humans? The idea oI replacing particular verbal language by a universal
visual language -- more eIIicient, capable oI reaching new masses oI immigrants (U.S.) or
"proletarians oI the whole world" (U.S.S.R.) was oIten expressed in the 1920s. Film, in
particular, was seen as the prime candidate Ior a visual Esperanto. In a 1921 interview D.W.
GriIIith said: "A picture is the universal symbol, and a picture that moves is a universal
language. Moving pictures, someone suggests, 'might have saved the situation when the Tower
oI Babel was built.'"
62
Miriam Hansen writes about the metaphor oI Iilm as visual Esperanto in
the U.S.:

GriIIith participated in the widespread celebration oI Iilm as a new "universal language" which
accompanied its Iormation as an institution. Used by journalists, intellectuals, social workers,
clergy, producers, and industrial apologists alike, the metaphor oI Iilm as a universal language
drew on a variety oI discourses (Enlightenment, nineteenth-century positivism, Protestant
millennialism, the Esperanto movement, and the growing advertising industry) and oscillated
accordingly between utopian and totalitarian implications.
63


However, it was across the Atlantic, in the U.S.S.R., where the comparison between Iilm and
verbal language was explored most systematically by Victor Shklovsky and other literary
theorists associated with the so-called Iormalist school as well as by Iilm directors and theorists
such as Eisenstein, Vsevolod Pudovkin, and Lev Kuleshov. In their writings we encounter

62
Qtd. in Miriam Hansen, "The Hieroglyph and the Whore: D.W. GriIIith's Intolerance," The
South Atlantic Quarterly 88, no. 2 (1989): 363.
63
Ibid., 362.

Irequent comparisons oI a shot and a word, or a sequence and a sentence as well as reIerences to
a "grammar oI Iilm."
64

Can we Iind similar kinds oI analogies proposed Ior the elements oI a single image? Just
as the artists were comIortable with diIIerent mutually exclusive models oI aIIectivity, they were
equally at ease with two contradictory models oI a code. II the visual elements were imagined to
be capable oI carrying complete messages, they were also thought oI as the elements oI a
message. Thus, art historian S.O. Khan-Magomedov writes about the INKhUK program: "The
point was to discover certain primary elements oI artistic expressiveness, in themselves without
signiIication and understood as an alphabet oI artistic-compositional system."
65

What we don't Iind in the 1920s, however, is equally serious thinking about the grammar
oI a still image. It was only in the 1960s, when the analogy between a verbal and a visual
language was taken with new rigor in structural semiotics, that coherent theorizing about the
grammar oI visual elements began to appear.
66




The understanding oI vision in modernity is characterized by attempts to think oI vision as a
language. This is reIlected in the popularity oI the term "visual language," this term which is so
Iamiliar to us today that we tend not to ask about its historical speciIicity. What this term points

64
Paradoxically, the classical Hollywood Iilm style, which matured by the 1920s, did become
the universal visual language oI the twentieth century, while the Iilms oI Soviet avant-garde
directors, who were most systematic in theorizing Iilm as a language, were rejected by mass
audiences and continue to play art houses, as they did in the 1920s.
65
Khan-Magomedov, "INKhUK," 343.
66
The best critical review oI visual semiotics available in English is Gsran Sonesson, Pictorial
Concepts. Inquiries into the Semiotic Heritage and its Relevance Ior the Analysis oI the Visual
World (Lund, Sweden: Lund University Press, 1989). 6. ConcIusion

to, in my view, is the new social role oI vision as the medium oI mass communication which it
acquired in the earlier decades oI this century.
The attempts to rationalize vision in this role, to conceive oI it as a code which can
Iunction independently oI verbal language and which can be eIIective in the new communicative
situations created by modernization have centered around the quest Ior the elements, the "atoms"
oI visual communication. This quest reaches its culmination in the work oI a number oI art
institutes in Soviet Russia in the 1920s, where artists collaborate with experimental psychologists
to isolate these elements and study their eIIectiveness.
The search Ior the essential elements oI each art which preoccupied early twentieth
century culture is usually seen in the context oI artistic modernism. This search is interpreted in
the context oI the rhetoric oI the purity oI medium. Indeed, especially in the 1910s-20s, artists
tried to reduce every medium to its unique qualities. To do so, they gave up representation and
concentrated on the material elements thought to be unique to each medium. Poets, such as
Russian Iuturists, were experimenting with sounds; Iilmmakers proposed that the essence oI
cinema was movement and temporal rhythm (French Iilm theory oI the 1920s)
67
or montage
(Kuleshov's group in Russia); and painters were exploring pure colors and geometric Iorms. The
Iollowing statement made in 1924 by Jean Epstein, a French avant-garde Iilmmaker and
theoretician, is typical oI modernist rhetoric oI purity; countless statements like it appeared on
the pages oI avant-garde publications oI the time:

For every art builds its Iorbidden city, its own exclusive domain, autonomous, speciIic and hostile
to anything that does not belong. Astonishing to relate, literature must Iirst and Ioremost be
literary; the theater, theatrical; painting, pictorial; and the cinema, cinematic. Painting today is
Ireeing itselI Irom many oI its representational and narrative concerns...And any literature worthy
oI the name turns its back on those twists and turns oI plot which lead to the detective's discovery

67
See especially Jean Epstein, "On Certain Characteristics oI PhotogZnie," in French Film
Theory and Criticism, ed. Richard Abel (Princeton: University oI Princeton Press, 1988), 1: 314-
318; Germaine Dulac, "Aesthetics, Obstacles, Integral CinZgraphie," in French Film Theory and
Criticism, 1: 389-397.

oI the lost treasure...The cinema must seek to become, gradually and in the end uniquely,
cinematic; to employ, in other words, only photogenic elements.
68


Yet, as this chapter has suggested, more was at stake in the quest Ior the visual "table oI
elements" than just the demand Ior purity oI medium. While many artists searched Ior "ever
more rich aesthetic elements" (Henry), trying to make their art into a "Iorbidden city" (Epstein),
others welcomed the opportunity to take their art to the streets. For these artists, the science oI
visual elements has provided hope Ior the creation oI a Iully rationalized visual language oI mass
communication.
The dream to rationalize communication was not unique to this project -- the proposals
Ior a universal language go back to the seventeenth century. Descartes suggested that the lexicon
oI a universal language should be composed oI primitive elements. By systematically combining
these elements, one can generate "an inIinity oI diIIerent words."
69
In the early eighteenth
century Leibniz outlined a language in which grammatical and logical structure would coincide,
thus making possible the automation oI thinking. The basic elements oI his ideal language were
characters representing unambiguously a limited number oI elementary concepts. Leibniz called
the inventory oI these concepts "the alphabet oI human thought." Through the algebra oI thought
primitive concepts would be combined to Iorm any complex idea.
70
The proposal oI a universal
language composed Irom graphic symbols and thereIore suitable only Ior writing Iollowed soon
with the Iirst pure pasigraphy ("writing Ior all") invented by Joseph de Maimieux in 1797.
71

However, until the twentieth century these remained isolated philosophical ideas advanced by
single individuals. With the rise oI mass communications, the Iantasy oI a universal rational
language attains new signiIicance, extending into the realm oI vision. Thus, what we now

68
Ibid., 314-15.
69
Qtd. in WinIried Nsth, Handbook oI Semiotics (Bloomington and Indianapolis: Indiana
University Press, 1990), 272.
70
Ibid., 274.
71
Ibid., 269.

witness are systematic and extensive attempts to establish Iully rationalized visual languages oI
mass communication. At the same time, the studies oI behavior and conditioning in physiology,
behaviorism and reIlexology, lead to new interpretations oI the Iunctioning oI the elements oI a
visual Esperanto. Not only would they be capable oI communicating ideas thus aIIecting the
intellect, but they would also directly control behavior bypassing the mind altogether, thus
making possible an international "psychophysical culture."




In the sixth Meditation Descartes deIined man "as a thing that thinks," to whom reasoning comes
naturally. Imagination, on the other hand, requires a special eIIort and is in no way necessary Ior
a human being. Distinguishing between imagination and pure intellection, Descartes
demonstrated the inIeriority oI vision to reasoning:

For example, when I imagine a triangle I not only conceive (intelligo) that it is a Iigure
comprehended by three lines, but at the same time also I look upon (intueor) these three lines as
present by the power and internal application oI my mind (acie mentis), and this is what I call
imagining. But iI I desire to think oI a chiliagon, I indeed rightly conceive that it is a Iigure
composed oI a thousand sides, as easily as I conceive that a triangle is a Iigure composed oI only
three sides; but I cannot imagine the thousand sides oI a chiliagon as I do the three sides oI a
triangle, nor, so to speak, view them as present |with the eyes oI my mind|.
72


Indeed, how can humans possibly reason through visual representations, iI the latter are not
capable oI representing general concepts to begin with? In Treatise Concerning the Principles oI
Human Knowledge, George Berkeley insisted, Ior instance, that it is impossible to have a mental
image oI an idea, such as "man," as a generality; it is only possible to visualize a tall or a short
man, but not man as such. Thinking deals with generalities and does not tolerate particular
things. II a philosopher tries to reason about the nature oI "man," any image oI a particular man
would mislead the reasoning process.
73

The philosophical arguments against visual reasoning still continue in this century, but
they are out oI step with the modern productive use oI vision: not only the concepts oI "visual

72
RenZ Descartes, "Meditations on the First Philosophy," in The Rationalists, trans. John Veitch
(Garden City, NY: Doubleday & Company, 1960), 160-161.
73
RudolI Arnheim, "A Plea Ior Visual Thinking," in New Essays on the Psychology oI Art (Los
Angeles and London: University oI CaliIornia Press, 1986), 136. Chapter 2. I See, Therefore I Think 1. Introduction

thinking," "thinking through images," etc. have become respectable and commonplace,
74
but
they are also put into practice daily through new techniques and technologies oI visual
representation. While Descartes thought that it was impossible (and, at any rate, unwise) to
imagine the thousands oI sides oI a chiliagon, today scientists stare at computer monitors, where
computer imaging brings beIore their eyes not only chiliagons but geometric objects oI n-
dimensions, objects composed not Irom thousands but Irom millions oI sides, objects
representing weather systems, diIIraction patterns, atomic surIaces, and other processes
inconceivable to the bare imagination (Iig. 5).
75
"ScientiIic visualization" is the name oI the new
Iield which claims to transIorm the nature oI scientiIic reasoning in biology, chemistry, and even
mathematics.
Likewise, Berkeley's insistence that images cannot convey "man" as a generality has been
superseded. Since the 1960s, numerous space stations sent to the Moon, Venus, Mars or simply
into the inIinity oI the Universe have carried images destined to communicate to intelligent
beings (should they happen to Iind the stations) the basics oI human civilization and the "essence
oI man" in a "universal language oI images." Long aIter the sun explodes and man disappears,
space stations will continue to Ily on their trajectories, carrying such images as the Iamous
drawing by Leonardo da Vinci oI an "ideal man." Intelligent (or perhaps, not so intelligent)
beings will eventually discover these images, learning Irom them iI not about human civilization
in general, then at least about the artistic tastes oI administrators oI space programs, assuring
Leonardo oI inter-galactic Iame.
The images oI general or abstract concepts can be encountered not only on space
missions but on Earth, everyday and everywhere, in numerous logotypes and designs (Ior

74
As witnessed, Ior instance, by such titles as Thinking Eye (Paul Klee) or Visual Thinking
(RudolI Arnheim).
75
Examples oI scientiIic visualizations presented at the Showcase at SIGGRAPH 1992.
SIGGRAPH '92 Final Program (New York: The Association Ior Computing Machinery, 1992),
33-40.

instance, in symbols which distinguish men's Irom women's lavatories), or in the representation
oI unimaginable data sets. However, these examples do not tell us anything about the speciIicity
oI the new Iunction oI vision in modernity. AIter all, Ior centuries pictures have been employed
as visualizations oI the holy stories. Michael Baxandall cites a late thirteenth-century text which
summarizes this purpose oI images:

Know that there were three reasons Ior the institution oI images in churches. First, Ior the
instruction oI simple people, because they are instructed by them as iI by books. Second, so that
the mystery oI the incarnation and the examples oI the Saints may be more active in our memory
through being presented daily to our eyes. Third, to excite Ieelings oI devotion, these being
aroused more eIIectively by things seen than by things heard.
76


ThereIore, what is diIIerent about the modern Iunction oI vision is not simply the use oI images
to represent abstract concepts concretely. What is diIIerent is the idea oI reasoning through
images, the idea unthinkable Ior Descartes or Berkeley. Thus, the Iield oI scientiIic visualization
is based on the assumption that visual observation oI computer generated images oI data sets and
processes leads to scientiIic breakthroughs otherwise impossible. Richard Mark FriedhoII and
William Benson propose that "using a computer simulation...even a beginning student oI
chemistry might be able to deduce benzene's structure" (a reIerence to Friedrich KebulZ's 1865
discovery oI the molecular stucture oI benzene supposedly made aIter he saw a dream involving
a snake biting its own tale).
77
Similarly, Jaron Lanier, one oI the most visible
researchers/promoters oI virtual reality, maintains that virtual reality will lead to the age oI "post-
symbolic communication," the communication oI ideas and arguments solely through images.
78

Finally, a prominent cognitive psychologist Philip Johnson-Laird claims that everyday logical

76
Michael Baxandall, Painting and Experience in FiIteenth Century Italy (OxIord: OxIord
University Press, 1972), 41.
77
Richard Mark FriedhoII and William Benson, The Second Computer Revolution:
Visualization (W.H. Freeman and Company, 1991), 13.
78
See, Ior instance, Timothy Druckrey, "Revenge oI the Nerds. An Interview with Jaron
Lanier," AIterimage (May 1991): 5-9.

reasoning takes place through images: when we are engaged in reasoning, we construct a mental
model oI a situation which represents its essential Ieatures in the Iorm oI topological relations.
79


BeIore it became possible to send Leonardo's images to remote corners oI the Universe to
represent "humanity," beIore scientists accepted that computer-aided visualization would
dramatically speed up scientiIic reasoning, beIore psychologists could claim that reasoning in
humans is a matter oI manipulating mental images, a proIound change in the cultural attitude
toward vision had to take place. This change occurred between the 1870s and the 1920s.
Logical thinking entails the representation oI abstract ideas and, also, oI the logical
relations between them. During this period, Galton, Venn, Freud, Eisenstein, and others put
vision to use Ior both purposes. More importantly, Ior the Iirst time, we can Iind in their work the
explicit justiIications Ior the very notion oI reasoning through vision.
In 1877 Galton's composite portraits visualized universal human types; in 1880, John
Venn made public a method Ior solving problems in logic by using graphic diagrams (section 3).
In The Interpretation oI Dreams, published in 1900, Sigmund Freud developed a systematic
theory oI how abstract notions and logical Iorms can be visualized (section 5). In the late 1920s
Eisenstein proposed that Iilm can be used "to encourage and direct the whole thought process"
(section 4).
These diIIerent models oI how vision can be used in reasoning represent the Iirst stage in
the reversal oI attitude towards the inadequacy oI vision. The second stage arrived when, in post-
industrial society, the concern with the eIIiciency oI the mind overwhelmed the concern with the
eIIiciency oI the body. Now, the questions oI how inIormation can be coded, stored, retrieved,
and processed more eIIiciently give direction to the study oI the mind, including the use oI visual

79
Philip Johnson-Laird, Mental Models: Towards a Cognitive Science oI Language, InIerence,
and Consciousness (Cambridge: Cambridge University Press, 1983).

representations in mental processes. Cognitive scientists, among others, begin to approach vision
Irom this new perspective, and they postulate the centrality oI visual representations Ior human
reasoning, because oI their eIIiciency (sections 6 and 7).



To be able to appreciate the novelty oI the modern paradigm oI visual reasoning, I will begin by
discussing the classic philosophical positions on the relationship between vision and reason.
Many philosophers thought vision to be unsuitable Ior reasoning, Iirst oI all because throughout
the history oI philosophy, reason was closely associated with logic. II, Iollowing its
Iormalization by George Boole and Gottlob Frege in the nineteenth century, logic has developed
into a sophisticated branch oI mathematics, today seemingly removed Irom our everyday
discourse, originally it was conceived by Aristotle as a systematization oI principles according to
which rational debates should be conducted.
80
ThereIore, it was Aristotle who originated the
tradition oI discussing human reasoning and logic interchangeably. Locke, Ior instance deIines
reason as "that Iaculty whereby man is supposed to be distinguished Irom beasts, and wherein it
is evident he much surpasses them." Claiming that the highest order oI reason consists in "the
discovering and Iinding out oI prooIs," Locke writes:

Sense and intuition reach but a very little way. The greatest part oI our knowledge depends upon
deductions and intermediate ideas: and in those cases where we are Iain to substitute assent instead
oI knowledge, and take propositions Ior true without being certain they are so, we have need to
Iind out, examine, and compare the grounds oI their probability.
81



80
Antony Flew, ed., A Dictionary oI Philosophy (London: Pan Books Ltd., 1984), 208-212.
81
John Locke, An Essay Concerning Human Understanding, ed. A.S. Pringle-Pattison (OxIord:
Clarendon Press, 1924), Book IV, Ch. 17. 2. I See, Therefore I Think

The equation oI reason with logic persisted late into the twentieth century. Piaget
assumed that the Iinal stage in the development oI human thought processes was a kind oI Iormal
logic analysis. Equally, Levi-Strauss's binary oppositions and Greimas's semiotic square are
based on stripped down Iormal logic. This logic is postulated as the structure oI the "native
mind" (Levi-Strauss) or the structure oI all signiIication (Greimas).
82
Only in the last decades
have cognitive psychologists discovered that human judgments do not Iollow statistical
probabilities and tend to ignore inIormation about the prior probability oI an event;
83
that rather
than having some uniIorm "laws oI thought," humans employ a variety oI heterogeneous
heuristics; and that in general the normative rules oI logic do not describe human thinking.
84

However, the idea oI a separation between logic as a normative or prescriptive discipline and
human reasoning, which can be described and studied empirically and which does not Iollow any
strict rules oI logic, is quite recent, and would be unimaginable Ior Aristotle or Locke.
85

ThereIore, in classical philosophy to reason is to carry out logical operations. How then
can one possibly reason through images? How can one represent visually "all," "some," "any" or

82
What is diIIerent about reason as imagined by Levi-Strauss and Greimas in contrast to Locke
is its new industrial eIIiciency: the concept oI binary oppositions was brought by Jakobson Irom
MIT, the birthplace oI inIormation theory which demonstrated that binary code is the most
economical oI all. According to GeoIIrey Sampson, "what led Jakobson to the hypothesis that all
parameters are 'binary' seems to have been the mathematical notion that a transmission-code is
more eIIicient when it uses only independent binary choices." GeoIIrey Sampson, Schools oI
Linguistics (StanIord, CA: StanIord University Press, 1980), 250.
83
Gillian Cohen, The Psychology oI Cognition, 2nd ed. (London and New York: Academic
Press, 1983), 190.
84
Michael I. Posner, and Gordon L. Shulman, "Cognitive Science," in The First Century oI
Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979), 393.
85
A recent dictionary oI philosophy still equates thinking with logic: "The mental activity oI (a)
theoretical contemplation directed toward some object with a view to reaching a propositional
conclusion; or (b) practical deliberation directed toward some object with a view oI reaching a
decision to act." Flew, A Dictionary oI Philosophy, 352. Emphasis mine - L.M. An encyclopedia
oI philosophy is equally categorical: "particular thoughts have some kind oI logical Iorm; they
may be categorical, hypothetical, disjunctive, universal, particular, and the like." Paul Edwards,
ed., Encyclopedia oI Philosophy (New York: Macmillan, 1967), 8: 101.

other logical constants? How to represent truth and Ialsity? How to distinguish particular Irom
general?
Reason, oI course, cannot proceed without the raw materials, collected Ior it by vision.
However, the role oI vision is limited to the passive and unintelligent collection oI data
(sensations), later to be interpreted by reason. Experimental psychology, emerging Irom
empiricist philosophy in the nineteenth century, gave additional support to this conceptual
hierarchy by providing a topological scheme: the outside world enters through the Iilters oI the
eyes and then is carried through visual pathways inside the head, to the brain, the site oI "higher
mental Iaculties." Even today, psychological textbooks are organized around the same hierarchy,
inherited Irom empiricist philosophy: they start with chapters on sensation (vision and other
senses) and end with cognition (thinking, memory, text comprehension, etc.)

These are some oI the ways in which philosophy has demoted vision in relation to reason: it can't
represent logical Iorms; it can't generalize; its only job is to collect the raw material reason
needs.
86
However, this philosophical tradition is counterbalanced and upset by other traditions.
For, while subordinating vision to reason, philosophers also oIten claimed that reason in Iact was
born oI vision. Moreover, it can be said that they could only conceive oI reason in visual terms.
For instance, while in terms oI the temporality oI everyday mental Iunctioning, vision
serves reason, its task being simply the collection oI data Ior reason to work with, in terms oI a
much longer temporality oI phylogenetic development oI the human race, vision is oIten
postulated as reason's origin. In particular, Locke, Herder, Vico, Nietzsche, and others claimed
that abstract categories oI language evolved Irom concrete visual (and also bodily) experience.
Locke: "I doubt not but, iI we could trace them |words| to their sources, we should Iind, in all

86
In his Downcast Eyes, unavailable to me at the time oI this writing, Martin Jay discusses the
role oI vision in modern French philosophy. Martin Jay, Downcast Eyes: the Denigration oI
Vision in Twentieth-century French Thought (Berkeley: The University oI CaliIornia Press,
1993).

languages, the names which stand Ior things that Iall under our senses to have had their Iirst rise
Irom sensible ideas."
87
Nietzsche: "Everything which makes man stand out in relieI against the
animal depends on this Iaculty oI volatilizing the concrete metaphors into a schema and thereIore
resolving perception into an idea."
88
Vico: "It is noteworthy that in all languages the greater part
oI the expressions relating to inanimate things are Iormed by metaphor Irom the human body and
its parts and Irom the human senses and passions."
89

Vico's ideas, in particular, have been recently revived in the work oI the linguist George
LakoII. LakoII argues that most, iI not all, abstract semantic concepts and abstract human
reasoning are based on metaphorical mapping oI spatial concepts (such as inside-outside, part-
whole, and others) which are inherently meaningIul to human beings since they have bodies.
Like Vico beIore him, LakoII challenges three traditional hierarchies -- between concrete
and abstract thought, between vision and reasoning, and between literal and metaphorical
meanings. In contrast to the objectivist-computational paradigm, which underlies much oI
Anglo-American philosophy oI language and cognitive science, in which thinking is
conceptualized as the algorithmic manipulation oI symbols meaningless in themselves, LakoII
claims that in thinking we project the structures oI our sensory and bodily experience to abstract

87
Qtd. in Tzvetan Todorov, Theories oI the Symbol (Ithaca: Cornell University Press, 1984),
237.
88
Friedrich Nietzsche, "On Truth and Falsity in their Ultramoral Sense," in Early Greek
Philosophy and Other Essays (New York: The Macmillan Company, 1924), 181.
89
Thomas Goddard Bergin and Max Harold Fisch, The New Science oI Giambattista Vico
(Ithaca: Cornell University Press, 1968), 129. Discussing the metaphorical origin oI abstract
concepts, Vico goes on to claim that the common assumption "that prose speech is proper
speech, and poetic speech improper" reverses the true history oI human mind: "Iirst nations"
have spoken and thought in poetry, i.e. through the metaphorical tropes (131).

situations.
90
The semantics oI natural language is built upon the meaningIul "image-schemes"
which are the generalizations oI what LakoII calls our "basic human experience." We live inside
our bodies; our bodies consist oI parts which are linked. ThereIore, we conceptualize all
situations in terms oI containers (Ior instance, an enormous number oI expressions use in and
out), part-whole relations (divorce is splitting up) and links (we make connections, break
relations). For LakoII, metaphors such as these are not the exception but the norm in language,

90
George LakoII, "Cognitive Linguistics,"
Versus 44/45 (1986): 119-154.

which involves the extension oI the bodily and sensory experience to the domain oI abstract
concepts. Because spatial (more precisely, topological) relations are an important part oI our
experience, spatial metaphors are at the core oI the semantics oI human languages; reaching
goals, moving out, walking over an enemy, etc. Not only does the semantics oI natural languages
turn out to be based on spatial metaphors but abstract reason itselI is the manipulation oI
spatially represented inIormation: "natural language reasoning makes use oI at least some
unconscious and automatic image-based processes such as superimposing images, scanning
them, Iocusing on part oI them, etc."
91
In Iact, reason is a special case oI more general mental
operations which are needed Ior perception: "Abstract reasoning is a special case oI image-based
reasoning. Image-based reasoning is Iundamental and abstract reasoning is image-based
reasoning under a metaphorical projection to an abstract domain."
92

With this claim, LakoII transcends the earlier views oI Locke, Vico, and Nietzsche
according to whom abstract reason phylogenetically evolved Irom vision. It is one thing to
suggest that abstract concepts have evolved Irom "sensible ideas" (Locke) or perceptions
(Nietzsche). It is quite diIIerent to start with the assumption, as LakoII does, that abstract ideas
are nothing but a limited set oI visual (more precisely, topological) Iorms (such as inside-outside,
part-whole) extended into the abstract domain.
Here the modern obsession with the visualization oI reasoning leads to the most dramatic
break with the traditional conceptualization oI the relation between reason and vision. Modern
writers such as LakoII or Johnson-Laird, claim is that there is only one kind oI cognitive
operations, which reason and vision share. These operations are Iundamental, original to vision:
reason borrows these operations; it does not invent any oI its own.

91
Ibid., 149.
92
George LakoII, "The Invariance Hypothesis: Is Abstract Reason Based on Image-Schemas?"
Cognitive Linguistics 1, no. 1 (1990), 65. Perhaps even this very brieI presentation oI LakoII's
ideas already makes apparent a substantial problem oI his theory -- the rather unquestioned
notion oI "basic human experience."

Johnson-Laird has proposed that when we are engaged in reasoning, we construct a
mental model oI a situation which represents its essential Ieatures in the Iorm oI topological
relations.
93
The conclusion then simply can be "read oII" Irom the representation. For instance,
how can we see the truth oI a modus ponens: "II X is an A, and iI all A's are B's, then X is a B"?
We construct a mental model: X is inside A; A is inside B. II the model is mentally "scanned,"
the right conclusion is invariably arrived at. According to Johnson-Laird, we don't have to

93
Philip Johnson-Laird, Mental Models:
Towards a Cognitive Science oI Language, InIerence, and Consciousness (Cambridge:
Cambridge University Press, 1983).

assume the existence oI any special rational logical competence unique to humans. Logical
reasoning is just one consequence oI vision.
But do these views oI LakoII and Johnson-Laird really represent a break? Perhaps this
revolt against the condemnation oI vision in the philosophical tradition was prepared by this
tradition itselI? For, while declaring vision unsuitable Ior reason, philosophers conceived oI
reason in visual terms all along: witness the persistence oI the metaphors oI eye, light, mirror,
revelation and unveiling, Irom Plato to Descartes, Hegel, Heideger, and Lacan as standing in Ior
logos, knowledge, and truth.
94

"Idea" comes Irom the Greek verb "to see" and is Irequently linked to ancient optics and
theories oI perception. According to Aristotle, in thinking ideas Iunction to substitute Ior the
absent objects as their images: "The reasoning mind thinks its ideas in the Iorm oI images; and as
the mind determines the objects it should pursue or avoid in terms oI these images, even in the

94
For gender implications oI this tradition,
see Evelyn Fox Keller and Christine R. Grontkowski, "The Mind's Eye," in Discovering Reality:
Feminist Perspectives on Epistemology, Metaphysics, Methodology, and Philosopohy oI
Science, ed. Sandra Harding and Merrill B. Hintikka (London: D. Reidel Publishing Company):
207-224. For a discussion oI Lacan's concept oI the mirror stage in relation to the Western
philosophical tradition with its concepts oI knowledge and subjectivity grounded in vision see
"The Statue Man," in M. Borch-Jakobson, The Absolute Master (StanIord: StanIord University
Press, 1991).

absence oI sensation, so it is simulated to action when occupied with them."
95
For Berkeley and
Hume thinking involved a sequential series oI images; these images are tied to certain habits oI
the mind to move Irom one image to another. Reason was equated with an eye which compared
all objects oI thought laid out beIore it. As summarized by Jonathan Crary,

But whether it is Berkeley's divine signs oI God arrayed on a diaphanous plane, Locke's sensations
'imprinted' on a white page, or Leibniz's elastic screen, the eighteenth century observer conIronts a
uniIied space oI order...on which contents oI the world can be studied and compared, known in
terms oI a multitude oI relationships. In Rorty's words, "It is as iI the tabula rasa were perpetually
under the gaze oI the unblinking Eye oI the Mind..."
96



95
Qtd. in Jean Matter Mandler and George Mandler, eds., Thinking: From Association to
Gestalt (New York: John Wiley & Sons, Inc., 1964), 9.
96
Jonathan Crary, Techniques oI the Observer: on Vision and Modernity in the Nineteenth
Century (Cambridge: The MIT Press, 1990), 55.

It was Heidegger who, in What is Called Thinking?, insisted on the visual basis oI
Western philosophy (both epistemology and ontology) Irom Plato to Hegel most strongly.
97

"The Greeks...conceived knowledge as a kind oI seeing and viewing...Because Being means
presence and permanence, 'seeing' is especially apt to serve as an explanation Ior the grasping oI

97
Martin Heidegger, What is Called
Thinking? (New York: Harper & Row, Publishers,
1968).

what is present and what is permanent."
98
For Plato, Being is idea, present as outward
appearance. However, already in Plato the seeds oI the split between object and subject are
implanted: Being is presence but at the same time is what man brings beIore his eyes. From this
moment on, idea is gradually transIormed into perception and representation, that, according to
Heidegger, characterizes the Cartesian cogito. Descartes laid a new Ioundation in privileging the
thinking subject, but the certitude oI the cogito derives Irom its visibility: as Heidegger explains,
cogitatio means Vor-stellung, i.e. "the bringing-beIore-itselI and what-is-brought-beIore-itselI
and made 'visible' in the widest sense."
99

Indeed, Cartesian rationality is based on the notion oI a subject which is outside himselI,
observing himselI contemplating the subject. This is the price Descartes pays Ior his
condemnation oI images and imagination in relation to reason: he can only conceive oI reason in
visual terms. BeIore "I think thereIore I am" comes "I see thereIore I think."



98
Qtd. in Borch-Jakobson, The Absolute Master,
53.
99
Qtd. in Ibid., 54.


Like Siamese twins, vision and reason are inseparable in classical philosophy, being connected
in a complex network oI dependencies, only a Iew oI which have been discussed above. On the
one hand, vision is but a servant oI reason, unable to represent logical relations and abstract
concepts and thereIore, clearly unsuitable Ior reasoning. On the other hand, vision achieves its
revenge, consistently slipping in as the phylogenetic origin oI reason and as the basis oI
metaphors in which operations oI reason are conceived.
Regardless oI whether Heidegger's brilliant account oI some oI these dependencies
represents a break with this classical tradition or is still a part oI it, he clearly lives in a diIIerent
age. Vision and reason are no longer thought oI as antagonists; on the contrary, "visual
reasoning" is now taken Ior granted and even promoted as a new highly eIIicient means oI
decision making and/or propaganda, suitable Ior the age oI mass mobilization, mass education,
industrialized science, and bureaucracy. Heidegger is a contemporary oI Eisenstein who, in line
with Stalin's plans to industrialize Russia overnight, dreams oI turning workers into dialectical
thinkers by passing them through the darkness oI special incubators oI cognitive skills -- movie
houses, thus shortening the duration oI the learning process to hours. Heidegger is also a
contemporary oI Benjamin who wants to teach German workers dialectical thinking even
without any external aids (such as Iilm) -- by simply juxtaposing images, in montage-like
Iashion, in the mind. Finally, Heidegger's younger compatriot, RudolI Arnheim, becomes the
most outspoken champion oI thinking in images, arguing that artistic skills should become the
3. First Signs of RevoIt: Venn and GaIton

basis oI all education, since all, even the most abstract reasoning, involves representation oI
inIormation in visual Iorm.
100

The signs oI the arrival oI the new age oI visual reasoning start to appear in the last
decades oI the nineeteenth century. In 1880 English logician John Venn published an article "On
the Diagrammatic and Mechanical Representation oI Propositions and Reasoning."
101
The

100
See Arnheim, "A Plea Ior Visual
Thinking."
101
Martin Gardner, Logical Machines and Diagrams, 2nd ed. (Chicago: The University oI
Chicago Press, 1982), 32.

article presented a method to use graphic diagrams to represent complex logical relations and it
soon became the standard Ior logicians. Venn's method not only made it possible to illustrate the
rules oI inIerential logic in visual Iorms; but much more importantly, now one could reason
automatically by simply drawing pictures (Iig. 6). Consider, Ior instance, the Iollowing
inIerence: "II all A is contained in B, and iI C is contained in A, then C must also be contained in
B." Its validity becomes obvious once the terms are represented in a diagrammatic Iorm.
Likewise, such inIerences as "All men are mortal. Socrates is a man. ThereIore, Socrates is
mortal" can be arrived at through similar diagrams. Images thought to be unsuitable Ior
representing logical relations and thus unsuitable Ior reasoning turn out to be its perIect indeed
its natural medium since they can represent topological relations with suIIicient precision and
complexity.
II Venn's diagrams showed that vision can be used to arrive at logical relations, Galton's
composite photographs proved that abstract concepts can be represented visually. Sir Francis
Galton, a statistician and a cousin oI Charles Darwin, a Iounder oI eugenics (a project oI social
betterment through planned breeding), and the author oI highly inIluential psychological texts,
pioneered in 1877 a procedure oI making composite photographs which proliIerated widely in
the next three decades.
102
Fabricated by a process oI successive registration and exposure oI
portraits onto a single plate, Galton's composites were thought to constitute true statistic
averages, representing human types -- a criminal, a prostitute, an Englishman, a Jew, and others
(Iig. 7). Galton wrote about his composite pictures that they are "much more than averages; they
are rather the equivalents oI those large statistical tables whose totals, divided by the number oI
classes and entered on the bottom line, are the averages. They are real generalizations, because
they include the whole oI the material under consideration."
103


102
Allan Sekula, "The Body and the Archive," October 39 (1987): 40.
103
Qtd. in Ibid., 47.

Galton not only claimed that "the ideal Iaces obtained by the method oI composite
portraiture appear to have a great deal in common with...so-called abstract ideas" but in Iact he
proposed to rename abstract ideas "cumulative ideas." In contrast to the human mind, "a most
imperIect apparatus Ior the elaboration oI general ideas," Galton championed his composite
photographs, which, being mechanical and precise, were much more reliable Ior arriving at
abstract representations.
104

With his photographs, Galton not only proposed that universals may be represented
through generic images; he actually objectiIied and materialized them. Plato's ideas were given
concrete Iorm: they could now be touched, copied, Iabricated, multiplied, distributed, etc.
II Venn's and Galton's inventions overturned the traditional claims about the impossibility
oI visual reasoning, they did that not through new philosophical arguments about vision but Iirst
oI all by inventing new ways to use it. The latter invented a new visual technology: representing
universals through composite photographs. The Iormer introduced a new visual technique, a new
way to employ visual representations, better yet, a new graphic sign system -- diagrams which
signiIy not any concrete objects, but logical Iorms.
105

In eIIect, Venn's technique and Galton's technology, which gave logical Iorms and
abstract ideas material tangible Iorm, externalized reasoning. What beIore was a mental process,
a uniquely individual state, now became part oI a public sphere. Unobservable and interior
processes and representations were taken out oI individual heads and put outside -- as drawings,

104
Qtd. in Ibid., 51.
105
I should clariIy the distinction between the terms visual techniques and visual technologies
as they are used in this chapter. By "techniques" oI visual representation, I mean particular
graphic and/or pictorial signiIying systems which do not depend in any crucial way on any one
medium or technological base. For example, Euler's diagrams can be drawn on paper, etched on
metal, generated with computer graphics, etc. The term "visual technologies" reIers to certain
representational devices, which do depend on the particular technological conIiguration and
would be hard or impossible to achieve without it. The examples include dissolve or traveling
shots in Iilm, which are only possible given a sequence oI images in time generated by a Iilm
projector, video tape player or computer. Galton's composite photography is such visual
technology, dependent, both in its design and its artiIacts, on particular photographic technology.

photographs and other visual Iorms. Now they could be discussed in public, employed in
teaching and propaganda, standardized, and mass-distributed. What was private became public.
What was unique became mass-produced. What was hidden in an individual's mind became
shared.



We should not be surprised, then, that the revolutionary new medium, the medium oI mass
society par excellence -- cinema -- was immediately proclaimed to be the machine Ior the
externalization oI private mental Iunctions and states. In 1916 Hugo MYnsterberg, a ProIessor oI
Psychology at Harvard University and one oI the Iounders oI the Iields oI industrial and applied
psychology, published The Film: A Psychological Study, today canonized as one oI the earliest
4. "To teach the worker to think diaIecticaIIy."

theoretical treatments oI cinema.
106
According to MYnsterberg, the essence oI the new medium
lies in its ability to reproduce, or "objectiIy" various mental Iunctions on the screen: "The
photoplay obeys the laws oI the mind rather than those oI the outer world."
107
In contrast to the
theater, where the action is constrained by the limitations oI physical reality, Iilm is Iree to shape
arbitrarily its material, closely approximating Ilashes oI memory, the Ilights oI imagination, and

106
Hugo MYnsterberg, The Photoplay: A
Psychological Study (New York: D. Aplleton & Co., 1916).

107
Ibid., 41.

other mental acts. For instance, while in theater events have to Iollow each other corresponding
to the progression oI time, in Iilm the action can suddenly jump back and Iorth, just as in an act
oI imagination.
MYnsterberg was not content to point out the analogy between Iilm and mental liIe; in an
astounding analysis, he correlated the main cinematic techniques to diIIerent mental Iunctions
such as attention and memory, one-to-one. For example, in the close-up, "everything which our
mind wants to disregard has been suddenly banished Irom our sight and has disappeared,"
analogous to how our attention selects a particular object Irom the environment. Similarly, the
"cut-back" technique objectiIies the mental Iunction oI memory.
"In both cases," MYnsterberg wrote, "the act which in the ordinary theater would go on
in our mind alone is here in the photography projected into the pictures themselves."
108
The
psychological laboratory became indistinguishable Irom the movie house; the textbook oI
experimental psychology -- Irom the cinematographer's manual. The mind was projected on the
screen; the inside became the outside.
MYnsterberg admired the power oI Iilm to externalize the Iunctions oI consciousness.
The next logical step was taken by a German psychologist Kurt Levin who, in 1924-25, was the
Iirst to use Iilm in his experiments. He wrote that "Iiction Iilm attempts to objectiIy certain
psychological processes Ior the viewer. Psychological (scientiIic) Iilm studies to what extent
these psychological processes can be objectiIied."
109
Soviet psychologist A.P. Luria, who
planned to establish a psychological laboratory in Moscow in cooperation with the State Film
Academy, acquainted Levin with Eisenstein, who attended the shooting oI one oI Levin's Iilms
and advised him.
110


108
Ibid., 41.
109
Qtd. in Olga Bulgakova, "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu
psikhoanalizom i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin':
between psychoanalysis and structural psychology), Kinovedcheskie Zapiski 2 (1988): 187.
110
Ibid., 177.

The Iigure oI Eisenstein is particularly important because it reveals the historical
connection between the interest in visual reasoning and the rise oI mass communication, oI
which Iilm was a major vehicle. The emergence oI new mass societies in the earlier part oI this
century dictated the necessity to communicate ideological concepts to mass populations which
were oIten illiterate.
111
In the 1920s Eisenstein boldly conceived a screen adaptation oI Marx's
Capital as a way to eIIiciently bring about the political enlightenment oI Russian audiences,
especially the peasants who would not sit through a political lecture but, attracted by the
"novelty" oI a movie projector, would come to see movies, regardless oI what was shown.
112

Unprecedented as his project was, its radicalism lay not only in the decision to visualize the
abstract notions and logic oI Capital but in the method employed, which, according to Eisenstein,
would directly provoke dialectical thinking in audiences.
113
Jacques Aumont concludes that Ior
Eisenstein, "the object privileged in Marx's work is not a theoretical one, like any oI the key
concepts Irom Capital. It is at another level entirely that Eisenstein selects his true object -- the
Marxist method itselI."
114
Thus it was not simply a matter oI the modern redeployment oI the
directions oI the 1492 sermon: "...Images oI the Virgin and the Saints were introduced...on
account oI the ignorance oI simple people, so that those who are not able to read the scriptures
can yet learn by seeing the sacraments oI our salvation and Iaith in pictures."
115
The viewers oI
Capital were not only to learn the scriptures oI the new atheistic religion; they were to learn the
process oI reasoning.

111
For instance, according to the 1926 census, out oI every 1,000 citizens oI the U.S.S.R., only
445 were literate. Peter Kenez, The Birth oI the Propaganda State. Soviet Methods oI Mass
Mobilization, 1917-1929 (Cambridge: Cambridge University Press, 1985), 157.
112
Kenez, The Birth oI the Propaganda State, 220.
113
The pioneering work oI Annette Michelson was important in bringing my attention to
Eisestein's Capital project. Annette Michelson, "Reading Eisenstein Reading 'Capital'," October
2 (1976): 27-38; October 3 (1977): 82-88.
114
Jacques Aumont, Montage Eisenstein (London and Bloomington: BFI Publishing and
Indiana University Press, 1987), 163.
115
Qtd. in Baxandall, Painting and Experience in FiIteenth Century Italy, 41.

It is signiIicant that the most categorical statement by Eisenstein on the possibility oI
"Iilmic reasoning," reasoning through images, appears in the context oI his discussion oI the
sequence known as For God and Country Irom October (1929):

Maintaining the denotation oI "God," the images increasingly disagree with our concept oI God,
inevitably leading to individual conclusions about the nature oI all deities...a chain oI images
attempted to achieve a purely intellectual resolution, resulting Irom a conIlict between a
preconception and a gradual discrediting oI its purposeIul steps.
Step by step...power is accumulated behind a process that can be Iormally identiIied with that oI a
logical deduction...The conventional descriptive Iorm Ior Iilm leads to the Iormal possibility oI a
kind oI Iilmic reasoning. While conventional Iilm directs emotions, this suggests an opportunity to
encourage and direct the whole thought process as well.
116


Far Irom simply representing God or deities, as they did Ior centuries, here images serve a totally
new Iunction -- to provoke and direct reasoning, reasoning oI a particular kind -- "Marxist
dialectics." In accordance with its principles, as canonized by the oIIicial Soviet philosophy,
Eisenstein wants to present the viewer with the visual equivalents oI thesis and anti-thesis so that
the viewer can then proceed to arrive at synthesis, i.e. the correct conclusion, pre-programmed by
Eisenstein.
"The content oI CAPITAL (its aim) is now Iormulated: to teach the worker to think
dialectically," Eisenstein writes enthusiastically in April oI 1928.
117
Schooled by the Iilm,
viewers would become selI-suIIicient thinkers, learning the skill oI "Communist decoding oI the
world," each a walking camera, snapping pictures oI visual thesis and anti-thesis, the brain
automatically executing cognitive operations oI montage, thinking through images, eIIiciently
and eIIectively.
Eisenstein claims the radical novelty oI his concept oI "Iilmic reasoning":


116
Sergei Eisenstein, "A Dialectical Approach to Film Form," in Film Form: Essays in Film
Theory, ed. Jay Leyda (New York: Harcourt Brace and World, 1949), 62. Emphasis mine -- L.M.
117
Sergei Eisenstein, "Notes Ior a Film oI 'Capital,'" trans. Maciej Sliwowski, Jay Leuda, and
Annette Michelson, October 2 (1976): 10.

The proclamation that I'm going to make a movie oI Marx's Das Kapital is not a publicity stunt. I
believe that the Iilms oI the Iuture will be Iound going in this direction (or else they'll be Iilming
things like The Idea oI Christianity Irom the bourgeois point oI view!) In any case, they will have
to do with philosophy...the Iield is absolutely untouched. Tabula rasa.
118


Yet, Einstein's theory was not an isolated development. Many in the artistic leIt oI the 1920s
shared a similar belieI in the cognitive power oI new visual Iorms such as montage. In the late
1920s Alexander Rodchenko promoted the use oI montage sequences in graphic design and, like
Eisenstein, he saw montage as being equivalent to "dialectical" reasoning. In this Iormulation, an
individual image corresponded to a single concept, and thinking was thought to be provoked
when a number oI images were juxtaposed in a series.
119
Walter Benjamin's notion oI
"dialectical seeing," central in his unIinished Passagen-Werk project, also depends on montage,
but within a single Irame, so to speak. "Dialectical seeing" was conceived by Benjamin as a way
to grasp the Iorms oI the present by looking to the past and to the Iuture in the same instant,
juxtaposing them in the same mental image.
120


118
Michelson, "Reading Eisenstein," October 2: 28.
119
While at Iirst Rodchenko practiced juxtaposition oI many separate photographs and
Iragments within the space oI a single image, at the end oI the 1920s his photomontages became
multi-page layouts composed oI a number oI more "traditional" photographs, more like a Iilm
montage sequence.
120
On Benjamin's concept oI "dialectical seeing" see
Susan Buck-Morss, The Dialectics oI


Venn, Galton, MYnsterberg, Eisenstein. Logical diagrams, composite photographs, cinematic
devices oI close-up, cut-back, and montage. DiIIerent, in eIIect, as they are, these developments
are symptoms oI a single social imaginary at work: to make reasoning more eIIicient and at the
same time more controllable by externalizing it and rendering it visible. This imaginary can be
glimpsed in the new techniques oI visual representation, such as Venn's diagrams. It also governs
the development and the promotion oI new visual technologies, such as Galton's composites or
Eisenstein's dialectical montage. Finally, this imagination can be also read in theoretical
speculations about how abstract concepts and logical Iorms can be represented as images in
principle, such as in Freud's psychoanalysis.
Is there a single person whose work brings into Iocus the modern preoccupation with
visual reasoning -- whether in the Iorm oI new theoretical models, new techniques oI visual
representation or new visual technologies? It is not Venn, or Galton, or MYnsterberg. This
person must be Sigmund Freud. He is not only more explicit in his deIense oI the possibility oI
visual reasoning; he develops a systematic theory oI how abstract notions and logical Iorms can

Seeing: Walter Benjamin and the Arcades Project (Cambridge: The MIT Press, 1989).


be visualized. This theory becomes one oI the cornerstones oI a new science, devoted, we should
note, to externalization, bringing into the open, into public view the most intimate, private, and
unobservable psychic phenomena -- the unconscious ideas, unreachable Ior consciousness. In
this, the new science goes well beyond MYnsterberg's notion oI the externalization oI memory
and attention by the techniques oI Iilm -- aIter all, these mental Iunctions can be voluntarily
controlled and commanded by consciousness. Indeed, MYnsterberg and his Iellow experimental
psychologists at the turn oI the century scrutinized the mind through the technique oI
introspection -- which, by deIault, limited them to the study oI only those mental acts and
Iunctions that could be voluntarily controlled and consciously observed. Instead, the new science
postulates the existence oI unobservable, uncontrollable psychic Iorces, which, lying deep below
the level oI consciousness, nevertheless control it.
The latter hypothesis, it is oIten said, constitutes the most dramatic attack on Cartesian
rationality launched in this century. However, it should be noted Ior the record that the new
science also gives rationality and reason even more power than beIore -- and not only because it
tamed the unconscious Ior the consciousness but also because it gained another ground Ior
reason -- vision. Having postulated the existence oI subterranean unconscious ideas and
unconscious reasoning, this science immediately proposes a technique by which these
phenomena can be brought to the surIace, and rendered visible and harmless. It claims that
unconscious ideas, groups oI ideas, and even arguments are transIormed into the Iorm oI dream-
images. By analyzing these images, an analyst can recover these ideas and arguments.



5. Freud's Theory of VisuaI Reasoning

Freud's The Interpretation oI Dreams, published in the Iirst year oI this century, inaugurated the
rise oI visual reasoning.
121
This work exempliIies a new understanding oI vision -- as not
inIerior to reason but, on the contrary, as its equal, as a medium quite capable oI representing
abstract ideas and logical arguments. According to Freud, dreams can translate into visual Iorm
such concepts as "a high-placed oIIicial" (Freud 378) or "superIluous" (441). They can also
represent logical connection (349), causality, negation (361, 372) or contradiction (470). In short,

121
All subsequent reIererences in this chapter to Freud, unless overwize indicated, are to
Sigmund Freud, The Interpretation oI
Dreams (New York: Avon Books, 1965).

the whole range oI reasoning operations can be encoded in visual images. With this theory,
vision is completely subsumed into the realm oI logical thinking.
According to Freud, the mind's contents, ideas and logical arguments, are securely hidden
Irom its owner, Irom the psychoanalyst and Irom the public, lurking, invisible and unreachable,
in the archaic cave oI the unconscious. They travel through the darkness oI association networks,
like newly Iormed urban masses, riding to and Irom work in the underground subways, which in
the end oI the nineteenth century were rapidly constructed in major metropolitan centers. Like
these urban masses, always a danger, potentially lurking in the darkness oI the labyrinths oI city
streets at night, ideas are Iormed, dissolved and rearranged in the total darkness, which the
limited "spotlight oI attention" (Wundt) cannot reach.
This, oI course, cannot be tolerated. Streets are streamlined and, even more importantly,
well lit. Indeed, the turn oI the century is oIten called "the age oI electricity" due to its obsession
with electrical light. But does not this light provide the best way oI Iighting invisibility and
assuring externalization? Similarly, ideas and reasoning need to be externalized, brought into the
open, rendered visible. Luckily, Freud assures us, the psyche already possesses a process which
transIorms the invisible contents oI the unconscious into images within our reach. All the
psychoanalyst needs to do is to properly interpret these dream images by tracing them back to
their underlying ideas.
So how can dreams represent ideas visually? In the section "Consideration oI
Representability," aIter he establishes the requirement Ior visualization -- "considerations oI
representability in the peculiar psychical material oI which dreams make use -- Ior the most part,
that is, representability in visual images," (379) Freud provides a number oI examples oI how
dreams transIorm abstract concepts and thoughts into pictures.
122
Here are a Iew oI them:

122
Mieke Bal's seminar on "Interpreting Culture" at the University oI Rochester brought my
attention to this point. For the original discussion oI considerations oI representability see
Samuel Weber, The Legend oI Freud (Minneapolis: University oI Minnesota Press, 1982).


"A high-placed oIIicial" is represented by a high tower (378).
"Working my way through" is represented by "a long kniIe under a cake, as though to liIt
out a slice" (380).
"SuperIluous" is represented by water everywhere -- "overIlowing," "Ilowing over"
(441).
Legs (part oI the body) are represented through pillars or columns (382).

Is there a single logic these examples Iollow? Yes, and we have already encountered it in LakoII.
LakoII, as we have seen, claims that abstract concepts and abstract reasoning take shape in the
process oI extending the structures oI human sensory and bodily experience to abstract
situations; and that even the most abstract categories are metaphors, based on such topological
relations as part-whole and inside-outside. To use Freud's examples, such concepts as "a high-
placed oIIicial" or "working my may through" are meaningIul in the Iirst place because they
involve the conceptualization oI abstract situations ("position oI people in society"; "solving a
problem") in spatial terms.
LakoII postulates that behind all seemingly abstract concepts oI natural language lie
concrete "image-schemes." II we accept LakoII's theory Ior a moment and read Freud's examples
through its terms, it appears that in dreams abstract concepts and ideas turn back into the "image-
schemes" Irom which they evolved in the Iirst place. Thus, "a high-placed oIIicial" becomes
again a concrete spatial representation, "high tower."
OI course, the point is not that LakoII's theory "explains" Freud or that Freud
"anticipates" LakoII; rather, it is that both implicitly rely on the notion which has always Iigured
centrally in the speculations about the origin oI language and reason (including, as we have seen,
oI such thinkers as Locke, Herder, Vico, and Nietzsche): that abstract concepts have evolved

Irom concrete ones, the latter directly connected with perceptual experience. Freud, however,
adds a new and important notion: regression. II in dreams abstract concepts and abstract thinking
itselI return to their "origin" as images and "image-based reasoning" (LakoII's term) this
transIormation would be in perIect correspondence with the regressive character oI dreams.
The notion oI regression is the key Ior understanding Freud's theory oI visual thinking.
Not only does it explain why in dreams abstract concepts return to their visual origins, it also
partly clariIies why dreams invariably rely on visual means oI representation at all: "what is
older in time is more primitive in Iorm and in psychical topography lies nearer to the perceptual
end" (587). Regression to the visual mode in dreams, then, operates in three ways: regression to
an older representational mode (Iormal regression); topographic regression (to the perceptual
system); and temporal regression to the older (in the evolutionist sense) psychical structures.
There is also another, Iourth meaning oI regression to the visual mode: regression to the visual
scenes witnessed in childhood ("primal scene"). ThereIore, the visual means oI representation
employed by dreams are characterized by Freud as an archaic, "primitive mode oI representation
and expression" (587). Vision, in Freud's diagnosis, is seen as the primordial, original language
oI mankind.
Should we be surprised that, according to Freud, Vienna's inhabitants circa 1900, modern
and respectable by day, at night totally lose their modernity, shed away progress, and instead
immerse themselves in archaic visual illusions? No, iI we recall the view, Irequently voiced in
the earlier part oI the twentieth century, that the new visual Iorms oI mass culture represent a
return, indeed, a regression to humanity's archaic stage. Most Irequently, it was cinema which
was seen (Ior instance, by Nabokov) as a time machine, where the metropolitan masses, stripped
oI their individuality, subjectivity, or even simply basic interiority, in total darkness (like that oI
prehistoric caves, even darker than the cave described by Plato), gave up the light oI
Enlightenment, Civilization and Reason, and instead were chanting in unison...a collective body,

mesmerized by visual Iorms, moving beIore them on the screen, made possible through the
magic oI the cinematograph.
As his contemporaries, Freud believes that the visual cinematograph oI dream-images
represents the regression to the original language oI mankind -- the language oI pictures;
however, he does not think that this entails the loss oI rationality, logic, and reason. Freud not
only claims, as we have just seen, that the dream-work has the means to represent singular ideas,
even abstract ones, but he also conIidently describes the mechanisms by which the dream-work
visualizes the logical relations between dream-thoughts.
"What representation do dreams provide Ior 'iI,' 'because,' 'just as,' 'although,' 'either-or,'
and all the other conjunctions without which we cannot understand sentences or speeches?," asks
Freud. His initial answer is negative: "In the Iirst resort our answer must be that dreams have no
means at their disposal Ior representing these logical relations between the dream-thoughts"
(347). But astonishingly, contradicting his own statement, Freud then proceeds to elaborate the
mechanisms by which the whole range oI logical relations can be represented in images:

Logical connection is represented by simultaneity in time (349).
Spatial proximity is used to represent logical relation: "Whenever they |dreams| show us
two elements close together, this guarantees that there is some specially intimate connection
between what corresponds to them among the dream-thoughts" (349).
Causality is reproduced by a time sequence (349) or by a metamorphosis.
An alternative ("either-or") may be represented by dividing the dream into two pieces oI
equal length (353).
Negation is usually discarded (353) -- but later Freud gives examples to the contrary
(361, 372).

Similarity ("just as") is the "Iavorite" oI dreams -- "this relation, unlike any other, is
capable oI being represented in dreams in a variety oI ways" (354), including "composition" -- the
creation oI composites. In principle, composition is similar to metamorphosis: "The psychical
process oI constructing composite images in dreams is evidently the same as when we imagine or
portray a centaur or a dragon in waking liIe" (359).
"The temporal repetition oI an act is regularly shown in dreams by the numerical
multiplication oI an object" (407).
Contradiction is represented by absurdity (470).

Freud does not explicitly connect his descriptions oI the dream-work's mechanisms oI the
visualization oI logical relations with his view oI the dream-work as regression to archaic mental
structures. However, his characterization oI these mechanisms only makes sense iI we assume
that, like everything connected with the dream-work, they owe their character to regression. The
connection is provided in the work oI another writer, Lucien LZvy-Bruhl, a leading French
anthropologist and Freud's contemporary. Like Freud, LZvy-Bruhl was concerned with the
nature oI archaic mental structures. But while Freud located these structures in the unconscious
oI every modern civilized individual, LZvy-Bruhl claimed to have Iound them in the mental
language oI present-day "savages."
In a number oI books LZvy-Bruhl advanced the notion oI "primitive" logic which
enjoyed wide popularity in the Iirst decades oI the twentieth century. According to LZvy-Bruhl,
the thinking oI primitive people Iollows a logic quite diIIerent Irom our own "normal" logic.
First oI all, primitive people conIuse temporal succession with causality. II they witness the two
events Iollowing each other, they assume that the two are causally related. Thus, LZvy-Bruhl
recounts that iI one oI them sees a snake Iall down Irom the tree in Iront oI him and learns soon

thereaIter that his son is dead, he will relate these two Iacts.
123
Second, primitive people believe
that iI two objects have been physically connected, they continue to inIluence each other when
separated. For instance, some primitive people are careIul to hide a babies' milk-teeth -- Ior iI
somebody Iinds them, then the new teeth will grow improperly. Summing up this Ieature oI
primitive logic, LZvy-Bruhl writes: "Primitive people conIuse the antecedent with the cause.
This would be a very common error in reasoning that is known as post hoc, ergo propter hoc
sophism."
124

These "laws" oI "primitive thinking," as described by LZvy-Bruhl, are the same ones that
Freud claims govern the representation oI the relations between dream-thoughts. Savages think
that two events Iollowing each other are causally related (LZvy-Bruhl) -- dreams represent
causality by temporal sequence (Freud). For savages, two objects physically connected are also
logically related -- dreams signiIy a logical connection by the spatial closeness oI elements or by
the simultaneity oI time (Freud, 349). Moreover, as "savage mind," the dream-work Iinds
similarity everywhere, seeing logical relations where there are none: "One and one only oI these
logical relations is very highly Iavored by the mechanism oI dream-Iormation; namely, the
relation oI similarity, consonance or approximation -- the relation oI 'just as.' This relation,
unlike any other, is capable oI being represented in dreams in a variety oI ways" (354). The
visual language oI the dream-work -- the archaic language oI the psyche, as described by Freud -
- is the same as the primordial "mental language" still at work in modern savages -- as claimed
by LZvy-Bruhl.




123
Lucien LZvy-Bruhl, Les Iunctions mentales dans les sociZtZs primitives (Paris: 1910), 72.
124
Ibid., 73. 6. The Rise of the Diagram

Freud does not just insist that abstract ideas and logic can be visualized or provide a theoretical
model oI how this can be done in principle. He practices what he preaches. What makes The
Interpretation oI Dreams a truly modern text, compatible with the age oI illustrated magazines,
color printing, scientiIic documentaries, dialectical montage, and other techniques and
technologies oI visual communication is not the abundance oI illustrations or Iigures (in Iact,
there are only a couple). Freud does not have to include illustrations, because many oI his
theories, and indeed his overall theory oI the psyche (the Iamous topographic model), are based
on simple visual models, easy to grasp and easy to remember. And this is what makes his text
modern. The reader is invited to disregard the complexities oI real anatomical data and instead to
conceptualize the psyche "as resembling a compound microscope or a photographic apparatus, or
something oI the kind" (574). According to Freud, this apparatus is made up oI just a Iew
components (called by Freud agencies or systems), which are arranged in spatial order (574-
583). This "spatial analogy" (653) allows Freud to visualize any psychic process as a simple
movement between a Iew parts oI the model.
It is as though Freud senses that the construction oI such models is very eIIective in
pushing the reader to "see" the selI-evidence oI the appropriate conclusions. Thus, aIter
presenting the typographic model oI the psyche, Freud uses it to explain the newly introduced
concept oI regression: "I believe that the name 'regression' is oI help to us in so Iar as it connects
a Iact that was already known to us with our schematic picture, in which the mental apparatus
was given a sense or direction. And it is at this point that picture begins to repay us Ior having
constructed it. For an examination oI it, without any Iurther reIlection, reveals a Iurther
characteristic oI dream-Iormation" (582. Emphasis mine -- L.M.)
To clariIy the logical relation between the conscious and the unconscious, Freud again
relies on the same technique oI constructing a visual diagram. From the point oI view oI the
ontogenetic and phylogenetic explanation oI the Iormation oI the psychical apparatus, primary

processes, as their name implies, "are present in the mental apparatus Irom the Iirst" (642) while
secondary processes appear only later and never achieve Iull domination. Thus, concludes Freud,
the primary processes (and corresponding to them the system oI the unconscious) constitute the
essence oI psychical liIe: "The unconscious is the true psychical reality" (651). To emphasize
this point and to make sure the reader remembers it, Freud "visualizes" it in the Iorm oI a simple
image oI a sphere within a larger sphere: "The unconscious is the larger sphere, which includes
within it the smaller sphere oI the conscious" (651). He then "reads oII" his conclusion Irom this
visual model: "Everything conscious has an unconscious preliminary stage" (651).
Freud's skillIul use oI visual models, analogous to Venn's diagrams, to represent the
logical relations between the terms in his theories is unusual in that Freud painstakingly
describes a visual model verbally, rather than directly presenting the reader (or should we say,
the viewer?) oI his text with a picture. It is as though Freud Iully realized the power oI diagrams
to represent complex concepts, models and arguments but, circa 1900, he was still hesitant to
"vulgarize" his scientiIic opus with images.
In 1933 Freud Iinally includes a real diagram in The New Introductory Lectures on
Psychoanalysis to illustrate the relation between two triads oI concepts: id, ego, and superego,
and unconscious, preconscious, and conscious. Freud's diagram, which eIIectively signiIies a
complex set oI relations by just a Iew lines, is representative oI a whole Iamily oI new techniques
oI visual representation, new to the modern period (Iig. 8). These techniques have risen to Iill
what can be called the cognitive needs oI modernization: to quickly communicate more and more
complicated relations between various entities, to manipulate huge sets oI data (whether in
science -- which became more and more the problem oI managing collected data -- or in the
bureaucratic management oI people), to depict in manageable Iorms endless hierarchies oI
relations -- in short, to represent reasoning visually and to reason with the help oI vision. These
techniques oI visual representation comprise what semioticians call the graphic sign systems --

signiIying systems, employing a particular set oI graphic conventions to represent particular
kinds oI objects.
125

We have already encountered one oI these new graphic systems -- Venn's diagrams.
These diagrams rely on the unique property oI vision, which, along with touch, as RudolI
Arnheim points out, is "the only sensory medium that conveys such spatial properties as
inclusion, overlap, parallelism, size, etc., with some precision."
126
This property makes possible
the precise representation oI the logical relations between classes in graphic Iorm. But, while in
principle visual images are capable oI representing spatial properties, not every image can
eIIectively do so. For instance, continuously varying tonal gradients are useless, whereas discrete
elements are eIIective. And these elements, to be easily readable, must have enough visual
diIIerence among them. ThereIore, Venn's diagrams are typically drawn using simple lines. The
classes are represented through circles (a representation in the Iorm oI rectangles is possible, but
it would be ambiguous, leading the viewer to believe that the shapes oI Iigures code some
additional logical relation). II circles overlap, the common area (representing exclusive "and") is
clearly distinguished through shading or color.
In short, we are dealing with a distinct graphic system, employing particular kinds oI
graphic marks, optimized Ior the representation oI particular reIerences -- a logic oI classes.
This graphic system accentuates one property oI images (the ability to represent spatial
properties such as inclusion) at the expense oI other properties, Ior instance, the ability to
represent continuously varying quantities through continuously varying tone or color.
Venn's diagrams is an example oI a specialized graphic sign system. Their use is limited
to a narrow proIessional domain oI logic. In the same period, another graphic sign system Ior
visualization oI logical relations comes into use (Iig. 9). In contrast to Venn's diagrams, the

125
The most comprehensive study oI graphic sign systems is Jacques Bertin, Semiology oI
Graphics (Madison, WI: The University oI Wisconsin Press, 1983).
126
Arnheim, "A Plea Ior Visual Thinking," 143.

graphic marks oI this sign system are now everywhere, seemingly the very air oI which the
contemporary visual semiosphere is composed.
What I have in mind is the gradual emergence oI greatly simpliIied graphic marks which
came to Iorm a notation, a sort oI an erector construction set, whose parts can be used to quickly
assemble a visualization oI a network oI relations, a representation oI a process or a scheme oI
dependencies. This construction set includes boxes, arrows, vectors, solid and dotted lines, and
so on. Arrows can denote such relations as "belongs to," "relates to," "is a member oI a class oI,"
"causes," etc. Boxes, on the other hand (or simply any area separated by lines) can represent a
physical object, the state oI a system, a stage in a process or any abstract entity. Freud's drawing
circa 1933, where a Iew encircled areas together with dotted and solid lines represent a complex
relationship between psychical agencies, illustrates the power oI this graphic system, which I
will call diagrammatic.
My comparison oI the diagrammatic graphic system with a construction set is not
accidental. A construction set includes some parts with more or less Iixed Iunctions (Ior instance,
wheels Ior moving models), while others (bars with holes) can be used Ior a variety oI purposes -
- to hold other parts, to represent parts oI a model, to be stationary or to move. Similarly, while
some elements oI the diagrammatic system have relatively Iixed meanings (Ior instance, wavy
brackets), others, such as arrows or dotted lines, can have numerous meanings, depending on
their particular arrangement.
This leads us to the second Ieature. In diagrammatic language, the same Iew symbols can
represent anything: a box, Ior instance, may stand Ior an idea, a theory, a building, 200 buildings,
a citizen, an army, or a state oI a physical system. Unlike the users oI verbal language who learn
particular semantics, the users oI the diagrammatic system learn the general conventions oI
syntax. It is enough to know that a box represents an entity -- regardless oI what the entity may
be. In this, diagrammatic language, used to represent a set oI entities and the relations between

them, Iunctions exactly like the symbolic language oI logic: a properly executed logic diagram is
isomorphic with a logical statement.
127
But this power is achieved through the standardization
oI graphical conventions themselves.
A diagrammatic system is a metalanguage, whose symbols represent not any particular
ideas or objects, but rather the abstract relations evoked in any process oI reasoning: "a part oI;"
"leads to;" "associated with." It is not accidental, thereIore, that this language oI arrows and
boxes is what recent generations oI computer programs, aiming to help reasoning or to imitate it
(outliners, hypertext, project management tools, idea generators, and the like) provide their users
with. The mental construction set became the graphic toolbox, always just a mouse click away.
This computer implementation oI already existing conventions oI visual culture is accompanied
by a Iurther standardization oI graphic symbols. This standardization is justiIied by the
argument, popularized since the adaptation oI Macintosh user interIace, that it helps the user to
recognize Iamiliar symbols and commands regardless oI the program she is using (Iig. 10).
The standardization oI diagrammatic symbols went hand in hand with their
simpliIication. While drawings were used Ior centuries to represent schematic inIormation rather
than simply to symbolize particular objects or to record their shapes, modernization is
accompanied by the simpliIication oI graphic symbols -- boxes, arrows, and the like. (This was
also accompanied by the streamlining oI typography.) Now, it becomes much easier to "read oII"
logical relations represented by the diagrams, since there are Iewer graphical marks to scan.
When do the symbols oI diagrammatic language, as it exists today, Iirst appear?
Obviously, we would not Iind a sharp "break" in graphic paradigms, with a new graphic system
appearing overnight. According to Ernst Gombrich, the arrow does not acquire its meaning as
pointer or vector until the eighteenth century. And even then, he could only Iind it in a couple oI
specialized proIessional contexts: it appears in a French treatise on Hydraulic Architecture oI

127
Gardner, Logic Machines and Diagrams, 28.

1737 and on some maps, where it was used to show the direction oI the river's Ilow.
128

Similarly, we can Iind other abstract graphic marks employed Ior centuries in graphic systems
used in particular proIessions -- Ior instance, engineering drawing. Gradually, however, these
symbols leave these proIessional contexts and enter visual culture at large.
Finally, what is the relation between the development oI the diagrammatic graphic
system and the overall movement in modern visual culture towards the simpliIication,
streamlining, and standardization oI all visual Iorms?
In the opening chapter oI Techniques oI the Observer, Jonathan Crary points out an
inherent contradiction present in most standard accounts oI modernism. On the one hand, these
accounts postulate the emergence oI new radical models oI perception and representation,
summed up in the notion oI abstraction, whose Iirst signs appear with Manet, impressionism or
post-impressionism and which Iully blossoms with Kandinsky, Kupka, Mondrian, and others. On
the other hand, it is also postulated that in the same historical period new popular visual Iorms
developed (Iirst photography and then cinema), Iurther solidiIying realism. As Crary notes,

128
Ernst Gombrich, "Pictorial Instructions," in Images and Understanding, ed. Barlow Horace,
Colin Blakemore and Miranda Weston-Smith (Cambridge: Cambridge University Press,
1990) , 28.

"Classical space is overturned, so it seems, on one hand, but it persists on the other. This
conceptual division leads to the erroneous notion that something called realism dominated
popular representational practices, while experiments and innovations occurred in a distinct (iI
oIten permeable) arena oI modernist art making."
129

Crary's way to solve this contradiction is to claim, throughout his book, that the so-called
realistic Iorms oI popular visual culture, based on photography, and abstract art are both
grounded in the emergence oI "subjective vision" in the early nineteenth century, in which the
link between representational signs and external reIerents is severed. "I argue," writes Crary,
"that some oI the most pervasive means oI producing 'realistic' eIIects in mass visual culture,
such as stereoscope, were in Iact based on a radical abstraction and reconstruction oI optical
experience, thus demanding a reconsideration oI what 'realism' means in the nineteenth
century."
130
What his anti-realist argument leaves out, however, is any consideration oI the
representational power oI new visual technologies such as photography. With photography,
based on mechanized perspective, the ability to establish a "reciprocal, metrical relationship
between the shapes oI objects as deIinitely located in space and their pictorial representations"
(William Ivins) became greatly ampliIied. In Ivins' words, photography is "a Iorm oI
picturemaking that is not only precisely duplicable but one in which geometrical perspective is
so inherent that today the camera is used as a surveying and measuring instrument."
131
Rather
than being anti-realistic, photography became the perIect technology oI instrumental realism
(Allan Sekula's term), literally allowing its users to have physical power over the represented
subjects and objects. As Sekula points out in his important account oI the instrumental uses oI
photography at the turn oI the century, criminal identiIication photographs, Ior instance, "are

129
Crary, Techniques oI the Observer, 4.
130
Ibid., 9.
131
William M. Ivins, On the Rationalization oI Sight (New York: Da Capo Press, Inc., 1975),
12.

designed quite literally to Iacilitate the arrest oI their reIerent" and, similarly, military
reconnaissance photographs make possible identiIication and destruction oI the objects they
capture.
132

A diIIerent way to solve the contradiction, noted by Crary, between the opposition
abstraction/high art versus realism/popular visual culture is to point out that in Iact the
development oI abstract visual Iorms was not limited to modernist art but equally took over the
realm oI the popular -- Iirst oI all, in the adaptation oI what I have called here diagrammatic
visual language. But in order to see this it is necessary to include in the discussion more than
images Irom the realms oI art, or entertainment or leisure -- the realms to which the problematic
oI high-low still limits itselI, and whose borders in Crary's argument are represented by
modernist painting, on the one side, and photographs, stereoscopes, phenakistiscopes and other
popular nineteenth century optical toys, on the other. Rather, it is necessary to consider the whole
range oI images employed in modern culture -- images employed not only Ior leisure but also Ior
work; images through which, literally speaking, modern work gets done: aerial surveillance
photographs and criminal identiIication shots, engineering and architectural drawings and plans,
maps and typography, instrumentation displays, and scientiIic imagery. And, last but not least,
we should consider the abstract graphic marks oI diagrammatic visual language, designed to
Iacilitate cognitive and communicative workloads oI modernity, in the oIIice, at school, in the
control room, in the boardroom, and on the assembly line -- which is today replaced by equally
monotonous rows oI computer terminals, outputting not Model T, but bits oI inIormation; this
steel and coal oI the inIormation -- or shall we better say, cognitive age.

InIormation age, knowledge economy, post-industrial society. All these terms come to designate
the changing nature oI society aIter World War II: the gradual displacement oI manual labor by

132
Allan Sekula, "The Body and the Archive," 7.

cognitive labor. With the centrality oI cognition Ior the post-industrial workplace, the notion oI
visual reasoning acquired new importance. Consequently, the work on the models, techniques,
and technologies oI visual reasoning, which was previously pursued by a Iew isolated individuals
(Galton, Freud, Eisenstein), has now become a matter oI systematic research on the industrial
scale in such Iields as cognitive psychology, scientiIic visualization, and virtual reality. I will
consider this research in the next two sections.


What is the relation between the psychological theories oI the mind and their contemporary
visual technologies? The recurrent claims that new visual technologies externalize and objectiIy
reasoning, and that they can be used to augment or control it, are based on the assumption oI the
isomorphism oI mental representations and operations with external visual eIIects such as
dissolves, composite images, and edited sequences. This assumption, which I so Iar have not
questioned, on a closer examination appears to be highly problematic. Whatever mental
representations and operations really are, the mind surely does not contain pictures, photographs
or Iilm clips which some mental homunculus looks at. The external images presented to the mind
are not magically transplanted inside it as ready-made ideas and arguments. Regardless oI what
visual Iorms can be presented beIore the eye -- diagrams, photographs, Iilm images -- as any
other visual input, they are subjected to the complicated processing by the nervous system which
constructs its own internal representations.
Yet, the assumption oI such an isomorphism continues to persist in modern thinking
about vision, ignited by every new round oI visualization technology: photography, Iilm,
computer animation, and virtual reality. Consider the claims which surround the new Iield oI
scientiIic visualization -- visualization oI data sets, their relationships and their dynamic behavior
using computer graphics. Richard Mark FriedhoII and
7. VisuaI TechnoIogies and the Mind

William Benson proclaim that computer visualization techniques constitute the second computer
revolution because they act as the direct "extension oI preconscious visual processes."
133
They
assume that the images on a computer screen do not simply Iunction as an aid Ior reasoning but
that they are equivalent to the mental representations the mind may construct while thinking --
and this is the source oI their power.
Or consider the technology, which, even more so than scientiIic visualization, is seen as
capable oI completely objectiIying, better yet, transparently merging with mental processes --
virtual reality (VR). Again, the descriptions oI its capabilities do not distinguish between internal
mental Iunctions, events and processes, and externally presented images. This is how, according
to Jaron Lanier, VR can take over human memory: "You can play back your memory through
time and classiIy your memories in various ways. You'd be able to run back through the
experiential places you've been in order to be able to Iind people, tools."
134
Lanier also claims
that VR will lead to the age oI "post-symbolic communication," communication without
language or any other symbols. Indeed, why should there be any need Ior linguistic symbols, iI
everybody, rather than being locked into a "prison-house oI language" (Jameson), will happily
live in the ultimate nightmare oI democracy -- the single mental space which is shared by
everybody, and where every communicative act is always ideal (Habermas). This is Lanier's
example oI how post-symbolic communication will Iunction: "you can make a cup that someone
else can pick when there wasn't a cup beIore, without having to use a picture oI the word
"cup."
135
Here, as with the earlier technology oI Iilm, the Iantasy oI objectiIying and
augmenting consciousness, extending the powers oI reason, goes hand in hand with the desire to
see in technology a return to the primitive happy age oI pre-language, pre-misunderstanding.

133
FriedhoII and Benson, The Second Computer Revolution: Visualization, 13.
134
Druckrey, "Revenge oI the Nerds," 9.
135
Ibid., 6.

Locked in virtual reality caves, with language taken away, we will communicate through
gestures, body movements, and grimaces, like our primitive ancestors...
136

What can one make oI this apparently unsound, yet irresistible, assumption oI
isomorphism between the mental process oI reasoning and external, technologically generated
visual Iorms, haunting us at least since the end oI the nineteenth century? The conIlation oI
outside and inside is, oI course, symptomatic oI the desire to project the inside onto the outside,
to make it objective and public. But is this all? To really understand the persistance oI this
assumption, we should turn to the history oI ideas about the nature oI mental processes.
It is well known that technologies have historically provided and continue to provide the
models according to which people imagine the mind. In the seventeenth century it was the clock,
in the nineteenth -- the motor, in the second halI oI the twentieth -- digital computers. More
precisely, the paradigms were provided not by the technologies themselves, but by theories
which made them possible. To take the last paradigm as an example, cognitive psychology, born
in the 1950s and gaining prominence ever since, approaches the mind as an inIormation-
processing system, as soItware which runs on the hardware oI the brain. But what gave cognitive
psychology its epistemological basis was not the new technology itselI (computers), but the
inIormation theory accompanying it. It was this paradigm which substituted the discussions oI
mind and brain by the notion oI "human inIormation processing." And beIore inIormation

136
At SIGGRAPH 1992, the premier annual conIerence on Computer Graphics and Interactive
Techniques in the U.S., attended by nearly 30,000 people, about a dozen VR exhibits always had
long lines oI visitors. However, the lines to two oI these exhibits were much longer than any oI
the others. One was called Dome, the other -- Virtual Reality Cave; in both cases, to see the
show, the visitors had to go inside some cave-line structures. It did not matter that one oI the
exhibits Ieatured a scientiIic visualization display, oI interest only to specialists. Clearly, the Iact
that in order to see the spectacle one had to go inside a dark, cave-like space, diIIerent Irom
normal space, provided enough attraction at the end oI this century, just as it did at its beginning,
when millions Ilocked into the dark caves oI movie theaters.

theory, theories oI the mind were inIluenced by thermodynamics (as in Freud) and mechanics
(Hobbes).
137

It can be also claimed, however, that human imagination about the mind's operations is
limited by the current visual technologies. LakoII's assertion that "natural reasoning makes use oI
at least some unconscious and automatic image-based processes such as superimposing images,
scanning them, Iocusing on part oI them" (1986: 149) as well as Johnson-Laird's proposal that
logical reasoning is a matter oI scanning visual models, would have been impossible beIore
television and later, computer graphics. These visual technologies made operations on images
such as scanning, Iocusing, and superimposition seem natural. Even more telling are the models
oI cognitive psychologists who, in the last two decades, have systematically scrutinized the role
played by mental images in reasoning. These models, which deIine mental images in terms oI
such characteristics as spatial resolution, speed oI access, basic graphic operations (rotation,
translation, copy), seem to be describing Iirst oI all computer imaging systems. Psychologists
argue among themselves which imaging systems better resemble mental processes, but they do
not doubt the basic metaphor. As Paul Virilio notes, "now the virtual images oI the computer
screen seem to conIirm not only the existence oI certain Iorms oI representation but, more
immediately, the objective presence oI mental images."
138


137
While this theory is well known and widely accepted, other Iacts suggest that, at least,
sometimes, the inIluence runs in the opposite direction -- biological and psychological theories
oI body and mind providing paradigms Ior theories oI mechanisms. For instance, it appears that
Norbert Wiener's cybernetics was inspired by the concept oI homeostasis developed in biology:
"Physiologist Walter B. Cannon viewed the animal body as a selI-regulating machine. Building
on the work done by Claude Bernard in the nineteenth century, Cannon developed the concept oI
'homeostasis' -- the process by which the body maintains itselI in a state oI internal equilibrium.
Cannon's ideas were well known to Norbert Wiener. In Iact, Cannon's Wisdom oI the Body
(1932) may be read as sort oI an introduction to Wiener's Cybernetics (1948)." Charles Eames
and Ray Eames, A Computer Perspective: Background to the Computer Age (Cambridge:
Harvard University Press, 1990), 99. Another example is provided by the turn oI ArtiIicial
Intelligence (AI) in the 1980s Irom trying to simulate the disembodied mind to the simulation oI
a collective oI primitive organisms, having the Iunctionality oI insects. Drawing directly on
research in biology, the researchers in AI hope that intelligence will emerge as a product oI the
collective behavior oI machines simulating simple biological organisms.
138
Paul Virilio, Lost Dimension (New York: Semiotext(e), 1991), 114.

Similarly, in the earlier period, when Freud, in The Interpretation oI Dreams, described
the mechanisms by which dream-thoughts and the logical relations between them are represented
in dreams, he and his Iellow psychologists relied on available visual technology Ior their
understanding oI the mind. Not surprisingly, Galton's composites, the earliest Iorm oI image
processing beIore digital computers, provided a particularly attractive model. Freud compared
the process oI condensation with one oI Francis Galton's procedures which became especially
Iamous: making Iamily portraits by overlaying a diIIerent negative image Ior each member oI the
Iamily and then making a single print.
139
Writing in the same decade, the American
psychologist Edward Titchener opened the discussion oI the nature oI abstract ideas in his
textbook oI psychology by noting that "the suggestion has been made that an abstract idea is a
sort oI composite photograph, a mental picture which results Irom the superimposition oI many
particular perceptions or ideas, and which thereIore shows the common elements distinct and the
individual elements blurred."
140
He then proceeds to consider the pros and cons oI this view. We
should not wonder why Titchener, Freud and other psychologists take the comparison Ior granted
rather than presenting it as a simple metaphor -- contemporary cognitive psychologists also do
not question why their models oI the mind are so similar to the computer workstations on which
they are constructed.
Thus, not only reason has always been conceived by philosophers in visual terms; as
psychologists begin to Iuriously take on the questions philosophers only wondered about,
subjecting mental processes to controlled, scientiIic, laboratory study, their models begin to
reIlect, more and more, the external visual Iorms made possible by whatever visual technology
dominates the period. As beIore, vision is the mirror reason looks into.

139
Sigmund Freud, Standard Edition oI the Complete Psychological Works (London: Hogarth
Press, 1953), 4: 293.
140
Edward BradIord Titchener, A Beginner's Psychology (New York: The Macmillan
Company, 1915), 114.

Given the reliance oI psychological theories oI the mind on contemporary visual
technologies, is there any "progress" between the turn oI the century and today, except that the
imagination oI contemporary psychologists depends on the more sophisticated visual
technologies oI computer graphics? In Iact, during this century, the assumption oI an
isomorphism between the mental and the objective became even more prominent; and
externalization oI reasoning has been taken much Iurther, both technologically and theoretically.
On the one hand, the reIinement oI various medical imagining techniques in the 1980s
made possible an increasingly precise imaging oI brain activity, including the visualization oI
reasoning -- in a literal sense. It is now possible to ask the experimental subject to concentrate
on solving a problem and to see which parts oI the cerebral cortex are active. The question oI
whether reasoning in Iact depends on the operations normally involved in perception becomes
more and more the question which, according to a number oI researchers, can be answered
through experimentation -- it is enough to show that the part oI a cortex normally dedicated to
the processing oI visual inIormation is activated in the process oI reasoning.
141

More importantly, in their theories, many cognitive psychologists have accepted as given
the equivalence between internal mental processes and the operations which can be done with
external, objectively existing visual representations and objects. Consider the debates about the
nature and role oI mental imagery, which have constituted one oI the most active areas oI
research in cognitive psychology in the last two decades.
142
On the one hand, there are those
(such as Zenon Pylyshyn) who argue that mental imagery simply consists oI the use oI general
thought processes to simulate physical perceptual events. In this view, iI the subjects report the
presence oI mental imagery during reasoning and problem solving, this is simply a side eIIect, a

141
Martha Farah, "Is Visual Imagery Really Visual? Overlooked Evidence Irom
Neuropsychology," Psychological Review 95, no. 3 (1988): 307-317.
142
For a summary oI diIIerent positions, see Ronald A. Finke, Principles oI Mental Imagery
(Cambridge: The MIT Press, 1989).

by-product oI real mental computations which do not involve visual representations. On the other
hand, there are those psychologists and neurophysiologists who, through experiments and direct
imaging oI brain activity, want to prove that reasoning takes place through the construction and
manipulation oI mental images (Alan Pavio, Roger Shepard, Stephen Kosslyn, Martha Farah).
One oI the most well-known experiments in deIense oI the latter view has been done by
Roger Shepard and Jacqueline Metzler oI Harvard University.
143
They presented subjects with

143
S. Shepard and J. Metzler, "Mental
Rotations oI Three-dimensional Objects," Science 171 (1971): 701-
703.

pairs oI perspective line drawings oI three-dimensional Iorms constructed Irom small cubes. The
subjects' task was to determine whether or not the Iorms were identical in shape, despite the
diIIerence in orientation. Shepard and Metzler have Iound that the reaction time was proportional
to the degree oI rotation which is required to bring the two objects into a similar position. These
results were taken as prooI that in solving the problem, the subjects mentally rotated
representations oI three-dimensional objects, and that imagined rotations corresponded to actual
physical rotations oI objects: "Imagined rotations and physical transIormations exhibit
corresponding dynamic characteristics and are governed by the same laws oI motion."
144
Thus, a
mental process was equated with an operation one would perIorm with real, objectively existing
objects.
Other experiments in deIense oI the position that many kinds oI reasoning involve
manipulation oI mental imagery entail the comparison oI abstract qualities. When subjects were
asked to recall two animals and to judge which one was larger, the reaction time decreased
proportionally to the diIIerence in estimated size. In another experiment, one group oI subjects
was asked to rate animals in intelligence on a scale Irom one to ten, while another group had to
compare the intelligence oI pairs oI animals. Again, the reaction time decreased as the distance
in rated intelligence increased. It was concluded that when the subjects tried to discriminate
between two objects, reaction time was shorter the greater the diIIerence between two objects,
regardless oI whether the objects were really presented (Ior instance, two lines oI diIIerent
length), or were imagined (size oI animals), or whether the qualities to be compared were
abstract (intelligence oI animals).
145

We shall leave to psychologists the debates whether these and numerous related
experiments indeed prove that internal mental processes involve the manipulation oI pictures

144
Finke, Principles oI Mental Imagery, 93.
145
John Robert Anderson, "Mental Imagery," in Cognitive Psychology and Its Implications,
(W.H. Freeman and Company, 1980).

similar in their qualities to real images. But Ior my purpose it is signiIicant in itselI that in
imagining what mental processes are like, contemporary psychologists have assumed, without
any reservations, an equivalence between the internal and the external, between the mental
objects and the real ones.

Modernization, visualization, externalization. In order to externalize the internal, the invisible, it
was Iirst equated with the visual. Once this was accomplished, it was simple and only logical to
equate the visual inside with the visual outside, the objectivity and standardization oI images
drawn on a classroom blackboard, on the screen oI a movie theater, or, most recently, on the
computer terminal.



In this chapter I am concerned with the answer to the problem oI how to represent, communicate,
induce or perIorm reasoning more eIIiciently. This problem Iirst became important with the
emergence oI mass society which required an eIIicient means to communicate abstract ideas to
large audiences. It became even more important with the shiIt to post-industrial society when
logical reasoning and other cognitive skills attained central economic importance.
The answer was to adopt vision as a more eIIicient medium. It can be perceived in
Freud's theoretical speculations oI how ideas and logical arguments can be visualized, in the
development oI new techniques oI visual representation (Venn's pictorial logic, diagrammatic
graphic system), and in the claims made about new visual technologies (Galton, MYnsterberg,
Eisenstein).
Vision, traditionally considered to be inIerior to reason, is now seen as the proper, indeed,
as the ideal medium Ior it. And this is not only a question oI the eIIiciency
8. AnaIog Engine

and eIIectiveness oI communication, although, with visual reasoning, these two demands are
Iully satisIied. Indeed, in Eisenstein's montage sequences, reasoning is speeded up -- to twenty-
Iour Irames a second. And, with pictorial statistics and diagrammatic visual language, reasoning
becomes more eIIective: complex conceptual hierarchies are reduced to boxes and arrows, huge
sets oI data become marks on paper which can be easily manipulated. But what is equally
important is that visual reasoning Iits perIectly with the demand oI mass society Ior
standardization. The subjects have to be standardized, and the means by which they are
standardized need to be standardized as well. Hence the objectiIication oI internal, private mental
processes, including ideas and logical connections, and their equation with external visual Iorms
which can be easily manipulated, mass produced, and standardized on its own. Whether we are
dealing with the conventions oI Venn's diagrams, oI signs oI diagrammatic sign system, or oI
cinematic techniques, the private and individual is translated into the public and becomes
regulated.
This new use oI vision as a means oI reasoning is accompanied by its rationalization on a
conceptual level. II previously vision was thought to be conIused, incapable oI Iorming
generalizations and unsuitable Ior representing logical Iorms, now this is all Iorgotten. Vision is
accepted as being Iully capable oI representing logical relations (Freud), abstract ideas (Galton),
and even dialectics (Eisenstein).
This equation oI vision with logic, accomplished by the beginning oI the twentieth
century, is the Iirst stage in vision's rationalization. When industrial society turned into the
society oI inIormation, computers became the dominant technology, both economically and
intellectually, and behaviorism (eIIiciency oI the body) was replaced by cognitive psychology
(eIIiciency oI the mind), this rationalization entered into a second stage. Now, with the questions
oI how inIormation can be coded, stored, retrieved, and processed more eIIiciently becoming a
matter oI economic survival, cognitive psychologists, together with computer scientists, began to

scrutinize vision Irom this new perspective. In the process, they came up with very precise
descriptions oI what vision is. However, what they describe is not some ahistorical, essential
vision, but vision already conceived in a historically and socially speciIic way -- as an eIIicient
computational system.
Why is it that when cognitive psychologists started to wonder about the properties oI data
structures involved in human reasoning, vision began to take a central place in their models?
Because every mental process is conceived as a computation, and it is assumed that this
computation has to be eIIicient. The criterion oI eIIiciency is a very general and pervasive notion
in contemporary cognitive psychology. This criterion is the main Iactor in deciding which oI the
competing models should be Iavored. The criterion is justiIied by a reIerence to biological
evolution -- it is said that in the process oI evolution the most eIIicient method, or the most
eIIicient representational code had to be adopted. However, the real justiIication oI this criterion,
in my view, is economic, coming Irom the close connection between cognitive psychology and
its master discipline, to which cognitive psychology ultimately serves -- computer science, and
behind it, the computer industry. Just as computer scientists would not want to build an
ineIIicient computer, psychologists cannot imagine an ineIIicient mental process.
Once psychologists realized that images possess some unique properties, which make
them a very eIIicient medium Ior computation, it was only natural that they entered into their
theories. How does Johnson-Laird justiIy his theory oI mental models, according to which
logical reasoning involves the construction and manipulation oI spatial models, analogous to
Venn's diagrams? A model represented in a dimensional space can be constructed, manipulated
or scanned in any direction, while propositional representations can only be processed in one
direction, making them much less eIIicient.
Why do some cognitive psychologists argue that human long-term memory stores not
only knowledge in the Iorm oI propositions, but also as images? Because iI we store appearances

oI objects then later we can compute those properties that could not be anticipated when we saw
the objects initially.
146
In contrast to propositional (linguistic) representation, images turn out to
be capital which can be used over and over in the mind's economy.
Finally, why all the debates in cognitive psychology about the role oI mental imagery in
reasoning in the last two decades? Because many problems can be solved more eIIiciently by
analog rather than digital computer, and vision has unique properties that enable it to Iunction as
such a computer:

II images are representations in a medium with certain Iixed properties, and can be subjected to
transIormations such as rotation and scaling, then imagery could be used as an analog computer, to
solve problems whose entities and relations are isomorphic to objects and spatial relations. That is,
certain abstract problems could be best solved by translating their entities into imagined objects,
transIorming them using available image transIormations, detecting the resulting spatial relations
and properties, and translating those relations and properties back to the problem domain.
147


What are these "certain properties" which make vision into an analog engine? On the one hand,
images can represent continuously varying inIormation. On the other hand, images can represent
topological relations such as inclusion, "next to," and others. Taken together, these two
properties make vision a very powerIul computer, better suited Ior solving many problems than
language. As Steven Pinker explains, in a visual medium, "it is impossible to represent the Iact
that one object is next to another without also committing oneselI to which is to the leIt. (This is
not true Ior abstract propositional representations, where the two-place predicate 'next to' can be
asserted oI a pair oI objects with no such commitment.)"
148

In principle, it is always possible to solve analog problems using propositional
representations but this simply would not be as eIIicient. So eIIiciency becomes the Iinal reason
why vision, in spite oI the arguments oI Descartes, Locke or Leibniz, is today equated with

146
Steven Pinker, "Visual Cognition: An Introduction," Cognition 18 (1984): 65.
147
Ibid., 66.
148
Ibid., 66-67.

reasoning. However, in this equation, vision has little to do with what these philosophers meant
by it -- it became an analog code, a "data structure" more proIicient Ior reasoning than language.
What remained the same, although recast in new terms, is the idea that vision came
beIore verbal language. II Ior Freud vision was the original, primordial language oI humankind,
today cognitive psychologists speculate that the continuous, analog representational system oI
mental images emerged Iirst, Iollowed by the system Ior propositional representation (language).
I once saw, thereIore I think.





The year 1991 saw two events, oI diIIerent importance and seemingly unrelated. One was the
long awaited publication in English oI one oI the most inIluential essays oI modern art history --
Chapter 3. Mapping Space 1. VisuaI NominaIism

Erwin PanoIsky's Die Perspektive als 'symbolische Form'.
149
The interest generated around the
re-emergence oI this legendary essay, written in 1924-1925,
150
demonstrates that the problem oI

149
Erwin PanoIsky, Perspective as
Symbolic Form (New York: Zone Books,
1991) .

perspective is still Ielt to be relevant to contemporary culture. Joseph Leo Koerner starts his
review oI Perspective as Symbolic Form with the Iollowing claim: "Perspective is again the
rage."
151
The second event was the GulI War, the outcome oI which was largely predetermined
by Western superiority in the techniques oI perspectival representation.

150
Erwin PanoIsky, "Die Perspektive als
'symbolische Form'," Vsrtrage der Bibliothek Warburg (Leipzig & Berlin: 1927), 258-330.
151
Joseph Leo Koerner, "The Shock oI the View," The New Republic (April 26, 1993): 32.

The images extensively televised during the GulI War, conIirmed Paul Virilio's thesis
that modern warIare has become a matter oI the "logistics oI perception."
152
True, broadcasts
have included more traditional views oI soldiers, planes, and tanks as seen Irom the outside, by a
reporter's video camera. But what we also saw were not just images oI the war, but endless
images through which the war was carried out: video images Irom an inIrared camera mounted
on a plane; video images Irom a camera installed on a weapon guided by a laser sensor; video in
its role as "battle damage assessment" where a weapon equipped with an imaging device Iollows
a weapon oI destruction and records details oI the damage. This was no longer a traditional
reporter's view oI a battle. We saw what the soldiers themselves saw: the images that were their
only inIormation about their targets. More oIten, in a strange case oI identiIication, we witnessed
what was "seen" by a machine, a bomb, or a missile.
Television reports became catalogs oI various vision technologies used to plan a battle, to
Iight it, and to assess the results. They included satellite imagery oI Iraqi territory; diagrams oI
Iraqi radar activity over the course oI a week; 3-D visualizations oI weapons; a 3-D animation
computed Irom satellite data oI a mountain region oI Iraqi thought to contain a secret sight oI

152
Paul Virilio, War and Cinema: the
Logistics oI Perception (London: Verso, 1989).

biological weapons production. We saw U.S. soldiers equipped with night vision devices as well
as U.S. pilots and soldiers training in Ilight and battle simulators. We also saw military advisors
scrutinizing and interpreting aerial photographs and satellite imagery. We were also shown Iraq's
weak, antiquated resistance against the superior Western technologies oI vision, image making,
and image analysis. Iraq Iought back with camouIlage weapons, decoys, and Iinally, according to
Western journalists, with oil Iires set to conIuse U.S. surveillance satellites and planes.
The GulI War was the combat oI surveillance against camouIlage, visibility against
invisibility, human eye against computer eye. This warIare was indeed based on the "logistics oI
perception," but we can describe its visual techniques even more precisely. Vision was employed
in a limited way as an instrument to capture and represent inIormation about shapes oI objects
and their positions in three-dimensional space.
The productive employment oI this Iunction oI vision can be seen most dramatically in
the example oI radar. A radar screen shows nothing but distances between the radar and other
objects in its Iield oI vision. The colors and textures are not important. The guiding laser sensor
oI a "smart" missile tracks the same inIormation -- the distance between the missile and the
target. Similarly, it is the layout oI a scene, distances between points, dimensions oI objects
which are recorded by a satellite photograph.
The multitude oI vision technologies which only "see" distances and dimensions are
paralleled by the reduction oI human vision to the same single Iunction. Thus, the images Irom
traditional or inIrared video cameras, which contain inIormation about various Ieatures oI the
visual world, are used by a soldier or a pilot in a much more limited way -- as a source oI
inIormation about distances and shapes in a scene. What is the shape oI an object on the horizon
-- is it an enemy or a Iriend? What is the distance between me and the enemy?
Finally, the technologies (such as radar) and techniques (using human sight in a Iunction
oI a radar) oI obtaining inIormation about shapes and distances in the real world are

complemented by 3-D computer graphics that simulate the real world on the basis oI such
inIormation. In their turn, simulated images generated with 3-D computer graphics are used in
Ilight and battle simulators, where the military eye undergoes its training to learn how to reduce
the world to geometry and topology.
The use oI technologies which capture and visualize inIormation about shapes and
distances today extends beyond warIare into all spheres oI industry and science. The pilots oI
commercial airlines utilize the same radar screens and are trained in the same Ilight simulators as
Air Force pilots.
153
Engineers and architects use 3-D graphics to visualize their designs. Oil-rich

153
As I write this in early 1993, many companies which yesterday supplied very expensive
simulators to the military are busy converting them into entertainment arcade-based systems. In
Iact, one oI the Iirst such systems already commercially operating in a number oI major cities,
including Chicago and Tokyo -- Battletech Center Irom Virtual World Entertainment, Inc. -- is
directly modeled on SIMNET (Simulation Network) developed by DARPA (DeIense Advanced
Research Projects Agency). SIMNET can be thought oI as the Iirst model oI cyberspace, the very
Iirst collaborative VR environment. SIMNET consists oI a number oI individual simulators,
networked together, each containing a copy oI the world database and the virtual representation
oI all other participants in the conIlict such as the Kuwaiti theater oI operations. Similarly, a
Battletech Center comprises a networked collection oI Iuturistic cockpit models with VR gear.
For $7 each, seven players can Iight each other in a simulated environment. In another example,
in 1992 Lucas Arts has teamed up with Hughes AircraIt, combining the expertise in computer
games oI the Iormer with the expertise in building actual Ilight simulators oI the latter, in a joint
venture aimed at theme-park type rides. On the connection between SIMNET and Battletech
Centers, see Tony Reveaux, "Virtual
Reality Gets Real," New Media (January 1993): 36-41. On VR entertainment systems in the

countries pay a high price Ior satellite images oI their territories to help them discover potential
oil Iields. And, in automated Iactories, tireless robotic arms move parts under the control oI
computerized vision systems trained to recognize the shapes oI these parts.

In this chapter I will discuss twentieth century automation oI the use oI vision to record and
visualize geometric and topographic inIormation -- to chart the distance and shape oI real and
imaginary scenes and objects. This Iunction oI vision complements another Iunction which was
the subject oI the previous chapter: the visual representation oI abstract ideas and their logical
relations. Here we will deal with the opposite problem -- how to represent a single, concrete
object or concrete scene. How, Ior instance, to represent not an idea oI a chair but a particular
chair with a unique shape and dimensions. These two kinds oI reIerents -- abstract ideas and
arguments, on the one hand, and concrete three-dimensional shapes, on the other hand, can be
related to two long-standing philosophical positions on what constitutes reality. For Plato,
sensible particulars were but pale reIlections oI Ideas or Forms. Aristotle criticized Plato,
declaring the primary substances were not Ideas but the individual things such as particular men
or animals. These opposing views have continued to be debated in scholastic philosophy, with
Plato's view giving rise to realism and Aristotle's to nominalism.
154


context oI location-based entertainment -- arcades and theme parks -- see Richard Cook,
"Serious Entertainment," Computer Graphics World (May 1992): 40-48.
154
Allan Sekula relies on this distinction between the philosophical positions oI realism and
nominalism in his discussion oI diIIerent systems oI police photography in the end oI the

In scholastic philosophy the terms "realism" and "nominalism" reIer to two opposing
ontological positions: realism claiming the primacy oI Ideas and nominalism -- the primacy oI
individual things.
155
Although in contemporary thought the meanings oI "realism" and
"nominalism" are somewhat diIIerent, the original meanings are helpIul in distinguishing two
Iundamental kinds oI representations: abstract ideas versus concrete objects.
I have suggested in the previous chapter that the desire to visualize abstract ideas and
logical relations was new to modernity. But it would not be reasonable to propose that the
Iunction oI vision which can be called visual nominalism -- to capture the identity oI individual
objects and spaces by recording distances and shapes -- originates in the twentieth century as
well. AIter all, maps were used Ior many centuries; and the sophisticated graphic language oI
engineering drawings was well developed by the nineteenth century. Yet, as this chapter will try
to demonstrate, in this century the automation oI visual nominalism, which began with the
Renaissance perspective, entered a new stage. The sign oI this automation is the multitude oI
new technologies used to capture and visualize three-dimensional reality that have emerged since

nineteenth century. Allan Sekula, "The
Body and the Archive," October 39 (1987): 3-64.
155
Antony Flew, ed., A Dictionary oI Philosophy (London: Pan Books Ltd., 1984), 250, 299.

the middle oI the twentieth century, such as radar, inIrared imaging, laser sensors, CAT scans,
magnetic resonance imagining, 3-D computer graphics, and computer holography. Finally, since
the early 1960s work has been under way to automate vision completely, to create computer
vision systems that will recognize objects and interpret scenes automatically.
The development oI these technologies has been accompanied by massive research into
the general problems oI visual nominalism in computer science, experimental psychology, and
neuroscience. New Iormal mathematical techniques were developed to analyze images as a
source oI depth inIormation and, vice versa, to transIorm this inIormation into realistic images.
The work on the automation oI visual nominalism has also lead to new attention to particular
aspects oI human perception. A new paradigm Ior the study oI human vision emerged during the
1970s at MIT associated with the name oI David Marr.
156
Within this paradigm, the goal oI
human perception is taken to be the recognition oI shapes, leading researchers to study
algorithms by which the brain "computes" shapes oI objects Irom retinal input in the hope that
these algorithms can then be used by computer vision systems.
This chapter brings together seemingly unrelated visual technologies and techniques:
radar, 3-D computer graphics, and computer vision. I will suggest that they are ultimately related
by the Iunction they all serve -- visual nominalism. Until this century, this Iunction has been
perIormed by techniques oI perspectival representation such as perspective and descriptive
geometry and, later, photography. Rather than thinking oI them as artistic techniques, such
writers as William Ivins and Bruno Latour emphasize their signiIicance Ior the development oI
science, modern technology, and the economy. What these techniques allow best is to represent
precisely three-dimensional inIormation on a two-dimensional plane, thus recording
quantitatively and eIIiciently the knowledge oI the surrounding space. New twentieth century
techniques Iurther extend the useIulness oI these kinds oI representations. Radar allows to obtain

156
David Marr, Vision (New York: W.H. Freeman and Company, 1982).

spatial inIormation without the limitations oI visibility (section 3); 3-D computer graphics add
the dimension oI interactivity (section 4); and computer vision automates visual nominalism
altogether (section 5).



While there is no history oI the burgeoning techniques oI visual nominalism in the twentieth
century, the earlier history oI these techniques has been studied in detail by art historians. The
problem oI representing three-dimensional space upon a two dimensional plane has been a
subject oI art historical research Irom at least the beginning oI the century. It is quite remarkable
that the people who are today recognized as the Iounders oI modern art history (Alois Riegl,
Heinrich WslIIlin, and Erwin PanoIsky), all deIined the Iield as the history oI the representation
oI space. Working within the paradigms oI cyclic cultural development and racial topology, they
related the representation oI space in art to the spirit oI entire epochs, civilizations and races. In
his 1901 Die SpStrsmische Kunstindustrie, Riegl characterized humankind's cultural
development as the oscillation between two extreme poles, two ways to understand space, which
he called "haptic" and "optic." Haptic perception isolates the object in the Iield as a discrete
entity, while optic perception uniIies objects in a spatial continuum.
157
Riegl's contemporary,
Heinrich WslIIlin, similarly proposed that the temperament oI a period or a nation expresses
itselI in a particular mode oI seeing and representing space. WslIIlin's Principles oI Art History
(1913) plots the diIIerence between Renaissance and Baroque on Iive dimensions: linear --
painterly, plane -- recession, closed Iorm -- open Iorm, multiplicity -- unity, and clearness --
unclearness. Finally, another Iounder oI modern art history, Erwin PanoIsky, contrasted the

157
Michael Ann Holly, PanoIsky and the Foundations oI Art History (Ithaca: Cornell University
Press, 1984), 73. 2. "The most important event of the Renaissance."

"aggregate" space oI the Greeks with the "systematic" space oI the Italian Renaissance in the
essay Perspective as a Symbolic Form (1924-1925).
158
PanoIsky established a parallel between
the history oI spatial representation and the evolution oI abstract thought. The Iormer moves
Irom the space oI individual objects in antiquity to the representation oI space as continuous and
systematic in modernity; in PanoIsky's neologisms, Irom "aggregate" space to "systematic"
space. Correspondingly, the evolution oI abstract thought progresses Irom ancient philosophy's
view oI the physical universe as discontinuous to the post-Renaissance understanding oI space as
inIinite, ontologically primal in relation to bodies, homogeneous, and isotropic -- in short, as
"systematic."
PanoIsky's 1924-25 essay has been recognized as the Iirst in what became an ever-
growing series oI interpretations oI perspective.
159
These interpretations related perspective to
every known characteristic oI the modern period: economic, social, and philosophical. Some oI
these interpretations acquired the character oI dogmas, selI-evident truths. For instance, the
correlation between Cartesian ideas oI rational subjectivity in philosophy and Renaissance
perspective in visual art has been presented as one oI the central metaphors in interpreting

158
PanoIsky, "Die Perspektive als 'symbolische Form.'
159
The recent very important work on perspective is Hubert Damisch, L'origine de la
perspective (Paris: Flammarion, 1987).

Western modern culture. This concept became so prevalent that in his critical analysis Martin Jay
Ieels justiIied to name it "Cartesian perspectivalism."
160

Most interpreters have singled out an aspect oI perspective, which then become Ior them
the core oI its symbolic Iunction. For instance, Pierre Francastel, and Iollowing him, John
Berger, and many Iilm theorists oI the 1960s and 1970s, Iocused on the mechanical character oI
perspective and its capacity to organize the representation oI reality around a single point oI
view. They theorized a connection between Renaissance perspective and the development oI
early capitalism, the emergence oI instrumental consciousness which measures everything,
which is concerned with solidity and extension, with numbers and equivalents -- the
consciousness oI market and accounting. Representations oI cosmic, religious, unmeasurable
space were replaced by the pictures oI cultivated, appropriated landscapes or oI rooms cluttered
by precisely drawn objects signiIying wealth. Perspective presented the world as ready to be
mastered, consumed, colonized -- the world originating in the eye oI the spectator.
161


160
Martin Jay, "Scopic Regimes oI
Modernity," Vision and Visuality, ed. Hal Foster (Seattle: Bay Press, 1988), 4.
161
Andrew Dudley, Concepts in Film Theory (OxIord: OxIord University Press, 1984), 31.

Other writers Ioregrounded diIIerent characteristics oI perspective in their
interpretations. For instance, Ior RudolI Arnheim, a pyramidal world portrayed by perspective
signiIies a hierarchical conception oI human existence.
162
On the other hand, Ior Fernande
Saint-Martin, perspectival representation which can only show objects at a distance, corresponds
to the "conceptual rather than sensorial, 'ideological' rather than concretely experienced" relation
oI a subject to the world.
163

PanoIsky's own account oI perspective is more subtle than many oI the consequent
interpretations it inspired. PanoIsky correlates a mode oI visual representation with a
philosophical system, but he is careIul not to read one as an expression oI the other. His
conclusion is open: perspective can be equally interpreted as subjective and objective, since it
presents space as an objective phenomenon but inevitably perceived Irom a subjective point oI
view. Calling perspective "a two-edged sword," PanoIsky writes:

Perspective subjects the artistic phenomenon to stable and even mathematically exact rules, but on
the other hand, makes the phenomenon contingent upon human beings, indeed upon the
individual: Ior these rules reIer to the psychological and physical conditions oI the visual
impression, and the way they take eIIect is determined by the Ireely chosen position oI a
subjective "point oI view."
164


162
RudolI Arnheim, Art and Visual Perception. The New Version (Berkeley: University oI
CaliIornia Press, 1974), 295.
163
Fernande Saint-Martin, Semiotics oI Visual Language (Bloomington and Indianapolis:
1990), 141.
164
And while modern perspective, as a coherent unity, is contrasted by PanoIsky to the modes
oI representation in antiquity and the Middle Ages, he also suggests that within the perspective
mode there exist many distinct variations. Working within the same mode oI perspective,
diIIerent cultures and epochs take its meaning to be diIIerent: the Dutch represent "near space,"
Germans Iavor "oblique space," Italians explore "high space."


While PanoIsky's essay had to wait almost six decades beIore its publication in English,
another equally important essay on perspective, "Obratnaya Perspektiva" (The Inverted
Perspective) still remains untranslated and unknown to Western art historians.
165
In his article
Joseph Leo Koerner brieIly reIers to the essay and regrets its unavailability in English (this is the
only reIerence I am aware oI).
166
The essay was written by the Russian philosopher, linguist,
mathematician and art historian Father Pavel Florensky, who headed the department oI the

Ervin PanoIsky, Perspective as Symbolic
Form, 68-70.
165
Pavel Florensky, "Obratnaya Perspektiva (The Inverted Perspective)," Sobranie Sochineniy,
ed. N.A. Struve (Paris: YMCA-PRESS, 1985), 1: 117-192.
166
Koerner, "The Shock oI the View," 37-38.

Analysis oI Spatial Representation in the Arts at VKhUTEMAS Irom 1921-1924.
167
Florensky's
essay can be justly considered as the counterpart to PanoIsky's work, completed in 1922, two
years beIore PanoIsky's work. Both PanoIsky and Florensky interpret any perspectival system as
convention and oppose the view that any particular system is natural.
168
At the same time, their
arguments tend to Iavor antithetical perspectival systems and world views: linear perspective and
Renaissance individualism in the case oI PanoIsky; inverted perspective and medieval religious
collectivism in the case oI Florensky.
PanoIsky narrates the evolution oI the representation oI space as an inevitable
progression towards the "systematic" yet subjective space oI the Renaissance; thus, the
Renaissance perspective is given a privileged status, its Iinal triumph as inevitable as the
progress oI history itselI. The discussion oI medieval representation oI space becomes Ior

167
VKhUTEMAS (State High Art-
Technical Studious) was the leading school oI art and design in the USSR in the 1920s and the
center oI avant-garde artistic culture.
168
Florensky writes: "Is it true that perspective, as it is claimed by its supporters, expresses the
true nature oI things and thereIore should be everywhere and always understood as the absolute
condition oI artistic truth? Or is it only a scheme, only one among many schemes oI
representation, corresponding not to the universal world view but only to one possible
understanding oI the world, connected with a particular sensibility and cognizance?" Florensky,
"Obratnaya Perspektiva," 123.

PanoIsky nothing more than a brilliant narrative device, a dramatic detour in what otherwise is
presented as a linear course Irom Antiquity to Renaissance: "When work on certain artistic
problems has advanced so Iar that Iurther work in the same direction, proceeding Irom the same
premises, appears to bear Iruit, the result is oIten a great recoil, or perhaps better, a reversal oI
direction."
169
In PanoIsky's interpretation, in other words, medieval representation oI space
serves as a dialectical antithesis between ancient and Renaissance representations. Thus,
PanoIsky claims, while it may appear that medieval representation gave up the advance oI
Antiquity in repressing three-dimensionality, in Iact it approached the Renaissance
understanding oI space as inIinite and homogeneous: "For iI Romanesque painting reduced
bodies and space to surIace, in the same way and with the same decisiveness, by these very
means it also managed Ior the Iirst time to conIirm and establish the homogeneity oI bodies and
space."
170

Florensky, on the other hand, is mostly interested in the medieval representation oI space
as maniIested by the spatial constructions oI Russian icons between the Iourteenth and sixteenth
centuries. According to Florensky, seemingly heterogeneous spatial constructions oI icons were
neither a result oI the lack oI artistic skill nor a step towards the Renaissance perspective.
171


169
PanoIsky, Perspective as Symbolic Form, 47.
170
Ibid., 51.
171
The view oI medieval art as unsystematic representation oI space, against which Florensky
revolted, continues to persist. Thus, Samuel Edgerton writes in 1975: "Unlike the Renaissance
painter depicting his scene in perspective, the medieval artist viewed his world quite
subjectively. He saw each element in his composition separately and independently, and thus

Rather, they are result oI a conscious and coherent system Ior representing reality which
corresponds to a coherent world view. It is not hard to see that Ior the orthodox and slavophile
Florensky, this world view is superior to Western post-Renaissance individualism. Russian
artists, like the artists oI ancient Egypt and China, were aware oI perspective but consciously
reIused its power, choosing instead "religious objectivity and super-personal metaphysics." For

paid little attention to anything in the way oI systematic spatial relationship between objects."

Samuel Edgerton, The Renaissance
Rediscovery oI Linear Perspective (New York: Basic Books, 1975), 21.

Florensky, the Western Renaissance represents the beginning oI the Fall: "When religious
stability oI the world view begins to disintegrate, and the sacred metaphysics oI collective
popular consciousness is segmented by individual vision oI an individual subject Irom an
individual point oI view, and only in a particular moment oI time -- only then perspectivalism,
symptomatic oI disunited consciousness, emerges."
172

Florensky's Slavophilism strikes us today as being politically biased; yet, this bias makes
us realize that PanoIsky's seemingly more objective account is also biased in its privileging oI
what Florensky labeled "bourgeois individualism." While Florensky oIIered a theoretical critique
oI linear perspective, Ior his compatriots at the time this critique became a practical problem:
what new visual Iorms shall be suitable Ior the "Iirst people's state"? Similarly identiIying linear
perspective with "bourgeois individualism," El Lissitsky and Kazimir Malevich relied in their
designs on other types oI perspectival construction such as parallel projection. In this type oI
projection, all projection rays are parallel to each other, instead oI originating Irom a single point
oI view. Parallel projection, with its absence oI a single point oI view, was presented as a more
suitable way to symbolize the vision and knowledge oI a collective. Paradoxically, this
"objective" point oI view oI a collective was expressed in the image (commonly encountered in

172
Florensky, "Obratnaya Perspektiva," 125-126. JustiIying the value oI reverse perspective as a
symbol oI "super-personal metaphysics," Florensky oIIered a critique oI the arguments later to
be Iound in PanoIsky's "Die Perspektive als 'symbolische Form.'" For instance, he precisely
articulates the notion oI "systematic" space, which, according to PanoIsky, represented the
triumph oI the Renaissance and was still absent in Medieval thought: "II we summarize
everything which is said in a Iormal sense against Medieval art, they are reduced to one
criticism: 'There is no understanding oI space.' This criticism means that there is no spatial unity,
no scheme oI Euclid's and Kant's space, the latter reduced, within the Iramework oI painting, to
linear perspective and proportionality."

Malevich's writings) oI the individual artist looking at the Earth Irom an inIinitely Iar away point
in outer space, so that the convergence oI rays is replaced by a parallel projection.
173



173

A.G. Rappoport, "El Lissitsky i ego 'Pun-
geometiya'" (El Lissitsky and his 'Pan-geometry'), Sovetskoe Iskusstvoznanie 25 (1989): 118-
119. Four decades later, LANDSAT satellites start bringing back images oI Earth objects in
axonometric projections (because they were shot Irom high orbits) thus IulIilling Malevich's
designs.

The reIusal oI linear perspective by such artists as Malevich or Lissitsky is usually seen as just
one episode in what by now became a widely accepted narrative -- the shattering and negation oI
perspectival space by modernist artists. According to this narrative, perspective was already
dead by the time art historians such as PanoIsky had begun writing its history. Such narrative is
announced, Ior instance, in the very title oI Pierre Francastel's Painting and Society. Birth and
Destruction oI Plastic Space Irom the Renaissance to Cubism (1952). The opening section oI
The Production oI Space by Henri LeIebvre is equally authoritative:

The Iact is that around 1910 a certain space was shattered. It was a space oI common sense, oI
knowledge (savoir), oI social practice, or political power...a space, too, oI classical perspective and
geometry, developed Irom the Renaissance onwards on the basis oI the Greek tradition (Euclid,
logic) and bodied Iorth in Western art and philosophy, as in the Iorm oI the city and town.
174


Yet, iI perspective disappeared Irom modern art, it survived as one oI techniques oI the
visual nominalism, a method Ior precisely representing the three-dimensional world on a two-
dimensional surIace. In this role, it extended into many new domains and became the Ioundation
oI new kinds oI automated technologies oI remote sensing and image synthesis.
To consider perspective in its role as a technique oI visual nominalism we should turn to
another interpreter oI perspective -- William Ivins and his short 1939 essay On the
Rationalization oI Sight.
175
II PanoIsky connects the development oI perspective with the idea
oI inIinite space, abstract space existing prior to objects, Ivins on the contrary emphasizes that
perspective allows the creation oI precise maps oI three-dimensional reality, to record the shapes
oI concrete objects and the layout oI concrete spaces. It is the tool oI a businessman and a
scientist rather than an artist.
In Ivins' deIinition, perspective is "a practical means Ior securing a rigorous two-way, or
reciprocal, metrical relationship between the shapes oI objects as deIinitely located in space and

174
Henri LeIebvre, The Production oI Space (OxIord: Blackwell Publishers, 1991), 25.
175
William M. Ivins, On the Rationalization oI Sight (New York: Da Capo Press, Inc., 1975).

their representations."
176
This deIinition sums up a number oI unique characteristics oI
perspectival projection. First, the size oI graphic marks in a perspectival image is proportionally
related to the dimensions oI represented objects. For instance, a Ioot-long object would be
represented by a line oI three inches, while an object twice as long would become a line oI six
inches, and so on. Second, the ratio oI sizes between these lines is proportional to the ratio oI
sizes between the corresponding objects. Third, the perspectival image does not only record the
shapes and dimensions oI objects but also contains inIormation about their distance Irom one
another (in contrast to parallel projection, Ior instance). Finally, all these relationships between
the perspectival symbols on the one hand, and between the symbols and the corresponding
objects on the other hand, are controlled by a single rule or, to use the language which would
become common within a decade aIter the publication oI On the Rationalization oI Sight, by a
single algorithm. Given the coordinates oI the point oI view and the projection surIace, the
projection oI any point in space can be determined by mechanically Iollowing a sequence oI
steps.
The most important quality oI perspective Ioregrounded in Ivins' deIinition is the precise
and reciprocal relationship it establishes between objects and their symbols. We can go Irom
objects to symbols (two-dimensional representations); but we can also go Irom such symbols to
three-dimensional objects. Using the rules oI perspective, an architect could create a visual
representation oI an already existing building, change this representation, and the workers would
execute the speciIied changes in the building itselI. Better yet, our architect can design a brand
new building -- not by painstakingly constructing a small model, where each change takes time
to implement -- but by creating a perspectival drawing, where each Iorm is just a line and change
requires only a single movement oI an eraser (or a single press oI a key on a computer keyboard,
iI the architect is working with a CAD system).

176
Ibid., 9.

Bruno Latour extended Ivins' idea by pointing out that a reciprocal relationship made
possible by perspective allows us not only to represent reality but also to control it.
177
Latour
sees perspectival representations as the "most powerIul instrument oI power," deIined as the
ability to mobilize resources across space and time, to manipulate these resources at a distance.
For instance, we cannot measure the sun in space directly, but we only need a small ruler to

177
Bruno Latour, "Visualization and
Cognition: Thinking with Eyes and Hands," Knowledge and Society: Studies in the Sociology oI
Culture Past and Present 6 (1986): 1-40.

measure it on a photograph (the perspectival image par excellence).
178
And even iI we could Ily
around the sun, we would still be better oII studying the sun through its representations which we
can bring back Irom the trip -- because now we have unlimited time to measure, analyze, and
catalog them. We can also represent absent things and plan our movement through space by
working on representations: "One cannot smell or hear or touch Sakhalin Island, but you can
look at the map and determine at which bearing you will see the land when you send the next
Ileet."
179
We can move objects Irom one place to another by simply moving their
representations: "You can see a church in Rome, and carry it with you in London in such a way
as to reconstruct it in London, or you can go back to Rome and amend the picture." Finally, as
Latour points out, "the two ways become a Iour-lane Ireeway! Impossible palaces can be drawn
realistically, but it is also possible to draw possible objects as iI they were utopian ones." Real
and imagined objects can meet on a Ilat space oI perspectival representation. All in all,
perspective is more than just a sign system, reIlecting reality -- it makes possible the
manipulation oI reality through the manipulation oI its symbols.

178
Ibid., 22.
179
Ibid., 8.

Citing perspective as a perIect example oI a new instrument oI power and domination,
Latour proposes that the history oI science and technology should be understood as a cascade oI
such innovations in representation. Yet, his argument is already anticipated by Ivins. Ivins
concludes his essay by stating that the beginning oI the rationalization oI sight through the
discovery and the development oI perspective "was the most important event oI the
Renaissance."
180
It was more important than the Iall oI Constantinople or the ReIormation or the
Counter ReIormation because it made all other discoveries possible by launching a constantly
accelerating series oI innovations.
181
The invention oI perspective propelled modern empirical
science, Ior instance biology, which could now represent Iorms oI nature with mathematical
precision. It also stimulated the rise oI modern engineering and manuIacturing by making
Ieasible the distribution oI identical designs to Iar away places.
182

Ivins' approach stands in sharp contrast to the more traditional art historical analyses oI
perspective by PanoIsky and Francastel. They are concerned with perspective as an artistic Iorm
and do not look beyond its history in art. Ivins, on the contrary, is concerned with visual culture -
- the techniques and technologies oI visual representation available to a society at a given
moment and the Iundamental role they play in shaping every aspect oI society. Because he sees
these techniques and technologies
183
as ultimately determining social development, his approach
is deeply materialistic. Perspective, Ior Ivins, is not an artistic Iorm reIlecting social or
philosophical modernity; it is the very condition which makes modernity and modernization
possible.
184
For instance, he writes: "Many reasons are assigned Ior the mechanization oI liIe

180
Ivins, On the Rationalization oI Sight, 13.
181
Ibid., 7, 12.
182
Ibid., 13.
183
Ivins Iocuses on two Iundamental developments: Iirst, the invention oI perspective and its
development into descriptive and perspective geometry; second, the invention oI technique Ior
printing pictures which made possible easy duplication oI exact copies oI the same image.
184
This is the theme which Latour also greatly extends, proposing "a new kind oI materialism"
which will account Ior social and scientiIic history through the analysis oI available Iorms oI
representation.

and industry during the nineteenth century, but the mathematical development oI perspective was
absolutely prerequisite to it."
185
Ivins is also a materialist in the most extreme Marxist sense
because he suggests that the available Iorms oI representation, such as perspective, determine the
development oI abstract thought. He points out that as important as the development oI
perspective was Ior picture making, "it is doubtless even more important to general thought,
because the premises on which it is based are implicit in every statement made with its aid."
186

This idea is repeated again at the end oI the essay:

The constant extensions oI the Iields oI useIulness oI the pictorial symbol that is precisely
duplicable |through printing| and oI the grammars oI its use |development oI perspective and
descriptive geometry| have had a most astonishing eIIect not only upon knowledge but upon
thought and its basic assumptions or intuitions...Relativity, which now in one Iorm or another runs
throughout contemporary thought and practice, is in large measure a development oI ideas that
were evolved through the study and use oI projective transIormations.
187


It is Ivins' general Iocus on visual culture, rather than art, that makes him attentive to the
current use oI perspective and its continuing development and expansion. While PanoIsky or
Francastel write about perspective in the past tense aware oI its demise in modern art, Ivins on
the contrary notes "that today there are Iew sciences and technologies that are not predicated in
one way or another upon this power oI invariant pictorial symbolization."
188
Modern designers,
scientists or engineers, oI course, do not simply use perspective as it was Iormulated by Alberti
in the IiIteenth century; they use much more sophisticated techniques. According to Ivins, the
rationalization oI perspectival sight proceeded in two directions. On the one hand, perspective
became the Ioundation Ior the development oI the techniques oI descriptive and perspective
geometry which became the standard visual language oI modern engineers and architects (Iig.
11). On the other hand, the photographic technologies automated the creation oI perspectival

185
Ivins, On the Rationalization oI Sight, 12.
186
Ibid., 9. Emphasis mine - L.M.
187
Ibid., 13.
188
Ibid.

images. Both were accomplishments oI the nineteenth century; in Iact, both were developed
more or less simultaneously. Indeed, as Ivins points out, Ni pce and Talbot, the Iounders oI
photography, were contemporaries oI Monge and Poncelet, decisive Iigures in the development
oI descriptive and perspective geometry.



Writing On Rationalization oI Sight between 1936 and 1938, Ivins mentions such examples oI
the contemporary use oI perspective as aerial photographic surveillance, classiIication in the
Iield oI archeology, and criminal detection.
189
However, all these applications oI perspectival
techniques already existed in the nineteenth century and, by the 1930s, did not represent the
"latest" developments.
While photo reconnaissance was Iirst employed systematically on a mass scale during
World War I, the interest in using photography Ior aerial surveillance existed since its invention.
FZlix Tournachon Nadar, one oI the most eminent photographers oI the nineteenth century,
Iamous Ior his portraits, succeeded in exposing a photographic plate at 262 Ieet over Bi vre,
France in 1858. He was soon approached by the French Army to attempt photo reconnaissance
but rejected the oIIer. In 1882, unmanned photo balloons were already in the air; a little later,
they were joined by photo rockets both in France and in Germany. The only innovation oI World
War I was to combine aerial cameras with a superior Ilying platIorm: the airplane.
190


189
Ibid., 12, 13.
190
Beaumont Newhall, Airborne Camera (New York: Hastings House, Publishers, 1969). For
critical histories oI photo reconnaissance see Allan Sekula, "The Instrumental Image: Steichen at
War," in Photography against the Grain: Essays and Photo Works, 1973-1983 (HaliIax: The
Press oI the Nova Scotia College oI Art and Design, 1984); Paul Virilio, War and Cinema: the
Logistics oI Perception (London: Verso, 1989); 3. Radar: Seeing Without Eyes

In 1858, Albrecht Meydenbauer, a director oI the German Government Building OIIice,
published a proposal to use photographs Ior scale measurement. His proposal was based on the
existence oI a geometrical relationship between the photographic image and the object being
photographed. Why, Ior instance, climb a Iacade oI a cathedral in order to measure it (as
Meydenbauer had to do, nearly once getting killed) when it is much saIer to measure a
photograph? Additionally, wrote Meydenbauer, "some may Iind it hard to believe, but

Manuel De Landa, "Policing the
Spectrum," in War in the Age oI Intelligent Machines (New York: Zone Books,
1991) .

experience has proven than one can see, not everything, but many things, better in scale
measurement than on the spot." In 1885 the Royal Prussian Institute Ior Scale Measurement was
Iounded and the measurement oI photographs oI historic monuments became a Irequent
practice.
191

In the Iield oI criminal detection, perspectival technology oI photography was routinely
employed decades beIore the time oI Ivins' writing. In his 1844 Pencil oI Nature William Fox
Talbot, one oI photography's Iounders, comments on a beautiIul calotype depicting several
shelves oI china: "should a thieI aIterwards purloin the treasures -- iI the mute testimony oI the
picture were to be produced against him in court -- it would certainly be evidence oI a novel
kind."
192
In 1883, Alphonse Bertillon, director oI the IdentiIication Bureau oI the Paris
PreIecture oI Police, developed a system Ior classiIication and identiIication oI criminals that
relied on photographs. By 1893, Bertillon's system was already employed in the United States,
Belgium, Switzerland, Russia, much oI South America, Tunisia, the British West Indies, and
Romania.
193


Modern perspective and descriptive geometry were Iully developed in the Iirst halI oI the
nineteenth century. Photography was already utilized by a number oI proIessions Ior
measurement, identiIication, and classiIication by the end oI the century. What was the next step
in the "rationalization oI sight"?
In Iact, while Ivins was writing his essay on perspective, across the Atlantic, in England,
work was already underway to install twenty radar stations on the east and southeast coasts to
provide surveillance oI these air approaches. These radar installations turned out to be absolutely
essential in the coming war, allowing Ior the severely outnumbered Royal Air Force to deIeat the

191
Harun Farocki, "Reality Would Have to Begin," Documents 1/2 (1992): 136-146.
192
Qtd. in Sekula, "The Body and the Archive," 6.
193
Sekula, "The Body and the Archive," 25, 35.

LuItwaIIe in the Battle oI Britain. Radar, the latest technology oI visual nominalism, became
Britain's most important weapon.
194

The Iollowing anecdote about the "discovery" oI radar, regardless oI its historical
accuracy, is useIul in understanding how radar works.
195
In 1922, two civilian scientists
employed by the Navy set out to conduct a routine communication experiment along the arm oI
the Potomac river. As usual, they set a short-wave radio receiver and transmitter along the two
banks oI the river. This time, however, a passing steamer interrupted the signal, creating an
electronic echo. The scientists immediately realized that such echoes could be used to deduce the
locations oI ships entering a harbor.


This anecdote illustrates the main principle oI radar operation. Radar is an acronym Ior
Radio Detection and Ranging. Like sound waves, radio waves create echoes when they are
interrupted by objects in their path. Radar transmits a radio wave in a particular direction. The
signal reIlected back Irom the objects is picked up by an antenna. The time between the
transmission and the reception oI the echo indicates the distance to the object; the direction the
antenna is pointing when the echo is received reveals the object's position in relation to the radar.
Detected objects appear as bright spots on a display watched by a radar operator.
196

Radar exempliIies the Iurther rationalization oI visual nominalism. All it sees and all it
shows are the positions oI objects, 3-D coordinates oI points in space, points which correspond
to submarines, aircraIt, birds or missiles. Color, texture, even shape are disregarded. Instead oI
Alberti's window, opening onto the Iull richness oI the visible world, a radar operator sees a

194
For the history and technology oI radar, I mainly relied on two sources: Echoes oI War
(Boston: WGBH Boston, n.d.), videotape; McGraw-Hill Encyclopedia oI Science & Technology:
an International ReIerence Work in Twenty Volumes Including Index (New York: McGraw-Hill,
1992.)
195
Echoes oI War (Boston: WGBH Boston, n.d.), videotape.
196
Numerous variations oI basic radar technology exist. For instance, in addition to active
radars which send a signal and detect energy reIlected by objects there are also passive radars
which do not send a signal themselves. However, all radars have in common the use oI
electromagnetic radiation (radio waves) to detect and measure objects in their vicinity.

screen, a dark Iield with a Iew bright spots. Here, the Iunction oI visual nominalism, which
perspectival image perIormed along with many other Iunctions (representation oI light, color,
shade, texture, etc.), is isolated and abstracted.
Radar serves a single Iunction -- but it perIorms it more eIIiciently than any previous
perspectival technique or technology. First, the detection oI objects' positions in space is no
longer limited by conditions oI visibility. Vision is no longer conIined by the distance resolvable
by the human or camera eye, since radar can detect objects hundreds oI miles away, and at any
time oI day or night. Second, this recording takes place instantly. Previously, military
commanders had to wait until pilots come back Irom surveillance missions and the Iilm was
developed. The inevitable delay between the time oI the observation and the delivery oI the
Iinished image limited its useIulness: since by the time the image was produced, enemy positions
could have changed. Now, the imaging is instantaneous.
With radar, its vision not limited by visibility, its image literally tracking the reIerent,
rationalization oI visual nominalism reaches a new stage. The useIulness oI vision and oI
imaging are greatly extended.
This expansion is achieved by Iundamentally redeIining the very Ioundations oI vision
and imaging technologies. While radar came to supplement perspectival representation and
photography in terms oI its Iunction, it also represents a Iundamental break with older
technologies. Like the photographic camera, radar captures the positions oI objects and displays
them on a Ilat surIace. But how it sees has little resemblance to what was commonly meant by
vision as deIined by older optical apparatuses -- Irom human eyes to photography.
Instead oI "looking" in a particular direction, radar sees all around, its antenna scanning
the area in a circular sweep. Instead oI passively receiving light reIlected by objects, radar sends
energy into the environment. Its vision is active; it can be compared to exploration oI the

environment by a blind person.
197
Instead oI relying, like photography, on the small region oI
the electromagnetic spectrum to which our eyes are sensitive, it uses other regions, sending and
receiving waves oI diIIerent lengths. Vision is no longer limited by the spectral capacity oI the
human or camera eye; it is extended to include the whole oI electromagnetic spectrum. The
visible becomes a small part oI a larger Iield oI sensory exploration oI the environment.
Images produced by radar are also Iundamentally diIIerent than those oI previous
imaging technologies, Irom drawing paper to photographic Iilm. The photographic image is an
imprint corresponding to a single reIerent or to a limited time oI observation. With the invention
oI the radar screen, the image surIace is no longer static. For the Iirst time it becomes constantly
updatable in real time. Here, with radar, we see Ior the Iirst time the mass employment oI a
Iundamentally new kind oI display, which soon comes to dominate modern visual culture, Iound
today in video or computer screen. What is new about such a display is that its image can change
in real time reIlecting changes in the reIerent, be it the position oI an object in space (radar), any
change in visible reality (live video) or changing data in computer memory (computer screen).
The radar screen changes, tracking the reIerent. But while it appears that the element oI
time delay always previously present in imaging is eliminated, in Iact time enters radar image in
a new way. With older photographic technologies, all parts oI an image are exposed
simultaneously. Now the image is produced through sequential scanning, which means that its
contiguous parts in Iact correspond to diIIerent moments in time. Like in an audio record,
moments in time become circular tracks on a surIace.

Along with radar, many other technologies oI visual nominalism came into existence Iollowing
the advances in electronics and computers during World War II: ultrasonic imaging,

197
There also exist passive radars which have no transmitters but are equipped to measure
signals Irom targets themselves. The active radar can be thought as a historical continuation oI
projectors previously used by the military to artiIicially light up the scene oI a battle to increase
visibility.

multispectral photography, multispectral imaging, inIrared, sonar, magnetic resonance imaging,
and so on.
198
As radar, these technologies are eIIectively used to record distance, position,
layout, shape, and volume. Sonar, Ior instance, detects objects in the water by using sound
waves. Ultrasonic computer tomography uses sound waves and computer graphics to construct
images oI body tissues. Multispectral photography isolates energy reIlected Irom surIaces in a
number oI given wavelength bands.
Coming to supplement perspectival techniques and technologies, the new technologies oI
visual nominalism are more eIIicient but also more specialized. II older optical technologies
indiscriminately capture a number oI dimensions oI visual reality (color, shape, texture, tonality),
now a single dimension can be isolated. More precisely, a single dimension, in this case 3-D
inIormation, is broken into distinct components recorded by diIIerent imaging technologies. The
technique oI perspective allows one to record both the shape oI objects and their relative
positions in relation to each other. Radar only records positions, it is blind to shape. On the other
hand, a laser or ultrasound range Iinder produces a depth map, treating as shapes both the actual
objects and the empty space between them.
This specialization can be seen as a side eIIect oI the automation oI visual nominalism.
This automation requires a priori assumptions about the nature oI reality which become
embedded in diIIerent technologies. In the case oI radar, reality is reduced to a set oI geometric
points which have no extension. In the case oI a range Iinder, reality is treated as iI a skin, a
membrane, a continuous 3-D Iorm. We may recall here the Iar-seeing prediction oI the eIIects oI
photography published in 1859 by Oliver Wendell Holmes: "Every conceivable object oI Nature
and Art will soon scale oII its surIace Ior us. Men will hunt all curious, beautiIul, grand objects,

198
See McGraw-Hill Encyclopedia oI Science & Technology.

as they hunt cattle in South America, Ior their skins and leave the carcasses as oI little worth."
199

Holmes' words seem to come true not only metaphorically (modern culture which values only
images) but also literally, as range Iinders methodically skin reality.
Along with the separation between technologies that record positions and technologies
that record shapes, another specialization emerges. Depending upon whether the position oI the
observer in relation to objects in space is important or not, the technologies oI visual nominalism
specialize in creating two diIIerent types oI representations which can be called viewer-oriented
and viewer-independent.
200
The Iirst kind oI representation is exempliIied by the radar screen, a
record oI distances between enemy targets and the subject -- the point in the center oI the screen.
Here, even more so than in the traditional perspective, the world is represented in relation to the
observer who becomes the center oI the coordinate system. In the second type oI representation,
the distance between the observer and the object is unimportant and in Iact, the position oI the
observer cannot even be deduced Irom the image. These are images which employ types oI
projection other than perspective: maps, engineering and architectural plans, and aerial and
satellite photography, whether conventional, inIrared or multispectral.
A third specialization, a third division oI labor brought about by new technologies is that
between vision and imaging, which now become distinct processes. Engineering textbooks and
encyclopedias group many new technologies oI visual nominalism under the term "remote
sensing," deIined as the gathering and imaging oI inIormation without actual physical contact
with the object or area being investigated.
201
This deIinition clearly separates the two

199
Oliver Wendel Holmes, "The Stereoscope and the Stereograph," in Photography, Essays and
Images: Illustrated Readings in the History oI Photography, ed. Beaumont Newhall (New York:
Museum oI Modern Art, 1980), 60.
200
This distinction is related to but not synonymous with the distinction between "viewer-
centered" and "object-centered" representations, common since David Marr's Vision (New York:
W.H. Freeman and Company, 1982). It is also related, but again, not equivalent to, the diIIerence
between perspectival and orthographic projections. The Iirst records reality Irom a single point oI
view, the second does not have a single point oI view.
201
McGraw-Hill Encyclopedia oI Science & Technology, 15: 311.

operations: the gathering oI inIormation (vision) and its presentation (imaging). The Iirst
operation may have nothing to do with what is visible to the human eye, but in the second
operation the eye eventually comes into play since the gathered inIormation has to be presented
to the human observer in visual Iorm in order to be useIul.
Technologies oI remote sensing convert the data obtained by technologically augmented
biological senses (Ior instance, we can think oI radar, which sends waves in order to detect
objects, as an augmented sense oI touch) into inIormation perceivable by the human eye. They
map other parts oI the electromagnetic spectrum into a small visible part. The invisible (sound
waves, microwaves, ultraviolet waves) is converted into the visible. This is why these
technologies are also oIten called imaging technologies -- multispectral imaging, ultrasound
imaging, inIrared imaging, and so on.

I have emphasized the continuity oI Iunction between perspectival drawing, photography, and
newer technologies such as radar and ultrasound imaging. Yet, not only these newer technologies
oI visual nominalism serve the same Iunction as the old -- to capture the identity oI individual
objects and spaces by recording distances and shapes -- but they rely on the same principle oI
perspective. We can justiIiably reIer to them as perspectival technologies iI we understand
perspective as extending beyond the domain oI the visible.
In his seminar "OI the Gaze as Object Petit a" Jacques Lacan argued that this is how
perspective should be understood.
202
He starts by reminding us that an image is anything
deIined "by a point-by-point correspondence oI two unities in space." To obtain an image oI
something we do not have to rely on light or to operate in the domain oI the visible. Nor do we
have to limit images to 2-D representations oI 3-D reality. We can represent an object by another

202
Jacques Lacan, "On the Gaze as Objet Petit a," in The Four Fundamental Concepts oI
Psycho-Analysis, ed. Jacques-Alain Miller (New York: W.W. Norton & Company, 1981), 67-
122.

object or represent a 2-D Iorm by another Iorm. All that is required is a rule to establish the
correspondence between the points oI the object being imaged and the points on the image.
Similarly, says Lacan, "what is an issue in geometric perspective is simply the mapping
oI space, not sight."
203
Perspective is one such rule, a particular method to establish a
correspondence between the object and its image. The method oI perspective consists oI
connecting a single point in space (usually reIerred to as subject's point oI view) with a number
oI points on the object by straight lines; the intersection oI these lines with a plane creates an
image. It is coincidental that perspective, whether as a part oI the human sight apparatus or as a
part oI the photographic apparatus, works through light. Light travels in straight lines, thereIore
it can be used to create perspectival images. But one can construct such images without light: "In
Descartes, dioptrics, the action oI the eyes, is represented as the conjugated action oI two
sticks."
204
As Lacan points out Iurther on in the seminar, this idea that perspective is not limited
to sight alone but Iunctions in other senses as well deIines the classical discourse on perception:
"The whole trick, the key presto!, oI the classic dialectic around perception, derives Irom the Iact
that it deals with geometric vision, that is to say, with vision in so Iar as it is situated in a space
that is not in its essence the visual."
205

Lacan's clariIication that the principle oI perspective is not limited to the visible helps us
understand that the technologies oI remote sensing Iunction on the principle oI perspective.
Regardless oI their lengths, all waves travel in straight lines, and thereIore points in space are
connected by straight lines to a point oI reception (such as radar antenna) or recording (such as a
photographic camera). Radar, inIrared imaging, sonar, and ultrasound are all part oI what Lacan
called "geometric vision," perspectival vision extending beyond the visible.

203
Ibid., 86.
204
Ibid., 87.
205
Ibid., 94.

The technologies oI remote sensing made it possible to map space without the limitations
oI visibility. In the next section I will discuss the technology oI 3-D computer graphics, which
Iurther extended the useIulness oI perspectival representations by making them interactive.




From the moment oI adaptation oI perspective, artists and draItsmen have attempted to aid the
laborious manual process oI creating perspectival images.
206
Between the sixteenth and the
nineteenth century various "perspectival machines" were constructed. They were used to
construct particularly challenging perspectival images, to illustrate the principles oI perspective,
to help students learn how to draw in perspective, to impress artists' clients, or to serve as
intellectual toys. Already in the Iirst decades oI the sixteenth century, DYrer described a number
oI such machines.
207
One device is a net in the Iorm oI a rectangular grid, stretched between the
artist and the subject. Another uses a string representing a line oI sight (Iig. 12). The string is
Iixed on one end, while the other end is moved successively to key points on the subject. The
point where the string crosses the projection plane, deIined by a wooden Irame, is recorded by
two crossed strings. For each position, a hinged board attached to the Irame is moved and the
point oI intersection is marked on its surIace. It is hard to claim that such a device, which gave
rise to many variations, made the creation oI perspectival images more eIIicient, however the
images it helped to produce had reassuring mechanical precision. Other major types oI
perspectival machines that appeared subsequently included the perspectograph, pantograph,
physionotrace, and optigraph.

206
For a survey oI perspectival instruments, see Martin Kemp, The Science oI Art (New Haven:
Yale University Press, 1990), 167-220.
207
Ibid., 171-172. 4. 3-D Computer Graphics: Interactive PerspectivaIism

Why manually move the string imitating the ray oI light Irom point to point? Along with
perspectival machines a whole range oI optical apparatuses was in use, particularly Ior depicting
landscapes and in conducting topographic surveys. They included versions oI camera obscura
Irom large tents to smaller, easily transportable boxes. AIter 1800, the artist's ammunition was
strengthened by camera lucida, patented in 1806.
208
Camera lucida utilized a prism with two
reIlecting surIaces at 135. The draItsman careIully positioned his eye to see both the image and
the drawing surIace below and traced the outline oI the image with a pencil.
Optical apparatuses came closer than previous perspectival devices to the automation oI
perspectival imaging. However, the images produced by camera obscura or camera lucida were
only ephemeral and considerable eIIort was still required to Iix these images. A draItsman had to
meticulously trace the image to transIorm it into the permanent Iorm oI a drawing.
With photography, this time-consuming process was Iinally eliminated. The process oI
imaging physical reality, the creation oI perspectival representations oI real objects was now
mechanized. However, this mechanization did not aIIect other uses oI perspectival
representation. According to Latour, the greatest advantage oI perspective over other kinds oI
representations is that it establishes a "Iour-lane Ireeway" between physical reality and its
representation. We can combine real and imagined objects in a single geometric model and go
back and Iorth between reality and the model. By the twentieth century, the creation oI a
geometric model oI both existing and imagined reality still remained a time consuming manual
process, requiring the techniques oI perspectival and analytical geometry, pencil, ruler, and
eraser. Similarly, iI one wanted to visualize the model in perspective, hours oI draIting were
required. And to view the model Irom another angle, one had to start all over again. The
mechanization and automation oI geometrical modeling and display were yet to come.


208
Ibid., 200.

Nothing perhaps symbolizes mechanization as dramatically as the Iirst assembly lines installed
by Henry Ford in U.S. Iactories in 1913. The assembly line relied on two crucial principles. The
Iirst was the standardization oI parts, already employed in the production oI military uniIorms in
the nineteenth century. The second, newer principle, was the separation oI the production process
into a set oI repetitive, sequential, and simple activities that could be executed by workers who
did not have to master the entire process and could be easily replaced.
It seemed that mechanical modernity was at its peak. Yet, in the same year the Spanish
inventor Leonardo Torres y Quevedo had already advocated the industrial use oI programmed
machines.
209
He pointed out that although automatons existed beIore, they were never used to
perIorm useIul work:

The ancient automatons...imitate the appearance and movement oI living beings, but this has not
much practical interest, and what is wanted is a class oI apparatus which leaves out the mere
visible gestures oI man and attempts to accomplish the results which a living person obtains, thus
replacing a man by a machine.
210


With mechanization, work is perIormed by a human but his or her physical labor is augmented
by a machine. Automation takes mechanization one step Iurther: the machine is programmed to
replace the Iunctions oI human organs oI observation, eIIort, and decision.
The term "automation" was coined in 1947; and in 1949 Ford began the construction oI
the Iirst automated Iactories. Mass automation was made possible by the development oI digital
computers during World War II and thus became synonymous with computerization. A decade
later, the automation oI the process oI constructing perspectival images oI both existent and non-
existent objects and scenes was well underway.
211
By the early 1960s Boeing designers already

209
Charles Eames and Ray Eames, A Computer Perspective: Background to the Computer Age
(Cambridge: Harvard University Press, 1990), 65-67.
210
Qtd. in ibid., 67.
211
I am not aiming here by any means to provide a Iull account oI the history oI 3-D computer
graphics or its various uses. I am concerned with computer graphics as one development, among
others, in the general move toward the rationalization oI perspectival imaging. For a more

relied on 3-D computer graphics Ior the simulation oI landings on the runway and oI pilot
movement in the cockpit (Iig 13).
212

By automating perspectival imaging, digital computers completed the process which
began in the Renaissance. This automation became possible because perspectival drawing has
always been a step-by-step procedure, an algorithm involving a series oI steps required to project
coordinates oI points in 3-D space onto a plane. BeIore computers the steps oI the algorithm
were executed by human draItsmen and artists. With a computer, these steps can be executed
automatically and, thereIore, much more eIIiciently.
The details oI the actual perspective-generating algorithm which could be executed by a
computer were published in 1963 by Lawrence G. Roberts, then a graduate student at MIT.
213

The perspective-generating algorithm constructs perspectival images in a manner quite similar to
traditional perspectival techniques. In Iact, Roberts had to reIer to German textbooks on
perspectival geometry Irom the early 1800s to get the mathematics oI perspective.
214
The
algorithm reduces reality to solid objects, and the objects are Iurther reduced to planes deIined by
straight lines. The coordinates oI the endpoint oI each line are stored in a computer. Also stored
are the parameters oI a virtual camera -- the coordinates oI a point oI view, the direction oI sight,
and the position oI a projection plane. Given this inIormation, the algorithm generates a
perspectival image oI an object, point by point.

comprehensive account oI 3-D computer graphics techniques, see J. William Mitchell, The
ReconIigured Eye: Visual Truth in the Post-Photographic Era (Cambridge, The MIT Press,
1992), 117-162.
212
Jasia Reichardt, The Computer in Art (London and New York: Studio Vista and Van
Nostrand Reinhold Company, 1971), 15.
213
L.G. Roberts, Machine Perception oI Three-Dimensional Solids, MIT Lincoln Laboratory
TR 315, 1963; L.G. Roberts, Homogeneous Matrix Representations and Manipulation oI N-
Dimensional Constructs, MIT Lincoln Laboratory MS 1405, 1965.
214
"Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and
Harvard," in SIGGRAPH '89 Panel Proceedings (New York: The Association Ior Computing
Machinery, 1989), 72.

The subsequent development oI computer graphics can be seen as the struggle to
automate other operations involved in producing perspectival stills and moving images. The
computerization oI perspectival construction made possible the automatic generation oI a
perspectival image oI a geometric model as seen Irom an arbitrary point oI view -- a picture oI a
virtual world recorded by a virtual camera. But, just like with the early perspectival machines
described by DYrer, early computer graphics systems did not really save much time over
traditional methods. To produce a Iilm oI a simulated landing, Boeing had to supplement
computer technology with manual labor. As in traditional animation, twenty-Iour plots were
required Ior each second oI Iilm. These plots were computer-generated and consisted oI simple
lines. Each plot was then hand-colored by an artist. Finished plots were Iilmed, again manually,
on an animation stand.
215
Gradually, throughout the 1970s and the 1980s, the coloring stage
was automated as well. Many algorithms were developed to add the Iull set oI depth cues to a
synthetic image -- hidden line and hidden surIace removal, shading, texture, atmospheric
perspective, shadows, reIlections, and so on.
216

Today, these algorithms make possible the simulation oI almost any object in such a way
that its computer image is indistinguishable Irom the photograph. Yet, this by itselI does not
represent a radical break with older techniques oI visual nominalism. A painter can paint the

215
This mixture oI automated and pre-industrial labor is characteristic oI the early uses oI
computers Ior the production oI images. In 1955 the psychologist Attneave was the Iirst to
construct an image which was to become one oI the icons oI the age oI digital visuality --
random squares pattern. A pattern consisted oI a grid made Irom small squares colored black or
white. A computer generated table oI random numbers has been used to determine the colors oI
the square -- odd number Ior one color, even number Ior another. Using this procedure, two
research assistants manually Iilled in 19,600 squares oI the pattern. Paul Vitz and Arnold B.
Glimcher, Modern Art and Modern Science (New York: Praeger Publishers, 1984), 234. Later,
many artists, such as Harold Cohen, used computers to generate line drawings which they then
colored by hand, transIerred to canvas to serve as a Ioundation Ior painting, etc.
216
For Iurther discussion oI the problem oI realism in computer graphics, see Lev Manovich,
"'Real' Wars: Esthetics and ProIessionalism in Computer Animation," Design Issues 6, no. 1
(Fall 1991): 18-25; Lev Manovich, "Assembling Reality: Myths oI Computer Graphics,"
AIterimage 20, no. 2 (September 1992): 12-14.

same image, although it would take much longer. What does represent a radical break is
interactive computer graphics.
In 1962 Ivan Sutherland designed his now legendary Sketchpad program. With
Sketchpad, a human operator could create graphics directly on the computer screen by touching
the screen with a light pen. In the same year, ITEK began marketing its Electronic DraIting
Machine, an apparatus similar to Sketchpad.
217
Although both programs dealt only with 2-D
graphics, they introduced a new paradigm oI interactive graphics: by changing something on the
screen, the operator changed the data in the computer's memory.
218

When this paradigm oI interactive editing was combined with the algorithms oI 3-D
graphics, a radically new way oI using perspectival images emerged. This development was
more revolutionary than the automation oI perspective construction per se. Indeed, traditional
draItsman could have accomplished what the computer at Boeing was doing -- generating plots
in perspective given 3-D database -- only more slowly. But now it became possible to change the
point oI view oI a virtual camera and see the corresponding changes in the perspectival image
aIter a short delay. It also became possible to build and modiIy 3-D models interactively and
observe the changes on the screen.
The emergence oI interactive 3-D computer graphics started the race to eliminate the time
delay between the action oI an operator and the displayed results. To be able to move through
virtual world in real time a computer has to generate 30 Irames a second. The amount oI

217
"Retrospectives II: The Early Years in Computer Graphics at MIT, Lincoln Lab, and
Harvard," 51.
218
In Iact, interactive computer graphics technology appeared earlier, although it was not
publicized. Already in the 1950s the Air Force used interactive CRT displays and light pens in
order to more eIIiciently process inIormation obtained by radar. Both CRT displays and light
pens were designed at Lincoln Laboratory as part oI the SAGE project. Using this technology,
Lincoln researchers created a number oI computer graphics programs. They included programs
which made posible the display oI brain waves (1957), oI simulated planet and gravitational
activity (1960), as well as the creation oI 2-D drawings (1958). "Retrospectives II: The Early
Years in Computer Graphics at MIT, Lincoln Lab, and Harvard," 42-54.

calculations involved increases proportionally as the model becomes more complex and as more
depth cues are added such as shading and shadows.
In this race Ior speed, which accelerated in the 1970s as synthetic images began to be
utilized in Ilight simulators, the algorithms oI 3-D graphics were gradually transported Irom
soItware into hardware, each algorithm becoming a special computer chip. In 1966 Ivan
Sutherland and his colleagues started research on the Iirst head-mounted display, a prototype oI
Virtual Reality -- the ultimate interIace Ior 3-D computer graphics. The research was
cosponsored by ARPA (Advanced Research Projects Agency) and the OIIice oI Naval
Research.
219
Describing the project, Sutherland wrote in 1968:

The Iundamental idea behind the three-dimensional display is to present the user with a
perspective image which changes as he moves. The retinal image oI the real objects which we see
is, aIter all, only two-dimensional. Thus iI we place suitable two-dimensional images on the
observer's retinas, we can create the illusion that he is seeing a three-dimensional object...The
image presented by the three-dimensional display must change in exactly the way that the image
oI a real object would change Ior similar motions oI the user's head.
220


The images seen in 1970 by the Iirst users oI Sutherland's system, such as a wireIrame
cube or a square room, looked remarkably similar to those utilized by Renaissance artists in their
demonstration oI perspective. Demonstrations oI perspective always Iavored such simple
geometrical objects because they most clearly showed the achievement oI perspective -- the
creation oI a 2-D image oI a 3-D object with Ioreshortening that matches our visual
experience.
221
The diIIerence was that now the perspectival image oI a cube was not Iixed on a
sheet oI paper; it appeared to an observer as Iloating in the air in Iront oI her, changing its view

219
Howard Rheingold, Virtual Reality (New York: Simon & Schuster, Inc., 1991), 105.
220
Qtd. in ibid., 104.
221
Later, Renaissance artists would deliberately insert checkerboard Iloors in perIect
perspective in their paintings to impress their clients -- a move repeated in computer graphics,
where such checkerboard background receding towards inIinity became a Iavorite motiI.

in correspondence with the slightest movement oI the observer's head. Perspectival imaging
became interactive.
In order to achieve this interactivity, special hardware was built. To speed up the
calculations necessary to update the synthetic image 30 times a second, Sutherland's group broke
down the process oI 3-D image synthesis into a series oI discrete steps. Each step was delegated
to a special electronic circuit: a clipping divider, a matrix multiplier, a vector generator. Later on,
such circuits became specialized computer chips, connected together to achieve real-time, high
resolution, photorealistic 3-D graphics. Silicon Graphics, one oI the major manuIacturers oI
computer graphics hardware, labeled such a system "geometry engine."
The term appropriately symbolizes the second stage oI the automation oI perspectival
imaging. At the Iirst stage, the photographic camera, with perspective physically built into its
lens, mechanized the process oI creating perspectival images oI existing objects. Now, with the
perspectival algorithm and other necessary geometric operations embedded in silicon, it becomes
possible to display and interactively manipulate models oI non-existent objects as well.
In the next section, I will discuss the Iinal stage in the automation oI perspectivalism --
the development oI computer vision.



In his papers, published between 1963 and 1965, Roberts Iormalized the mathematics necessary
Ior generating and modiIying perspective views oI geometric models on the computer. This,
writes William J. Mitchell, was "an event as momentous, in its way, as Brunelleschi's perspective
5. Computer Vision: Automation of Sight

demonstration."
222
However, Roberts developed techniques oI 3-D computer graphics having in
mind not the automation oI perspectival imaging but another, much more daring goal -- "to have

222
Mitchell, The ReconIigured Eye , 118.

the machine recognize and understand photographs oI three dimensional objects."
223
Thus, the
two Iields were born simultaneously: 3-D computer graphics and computer vision, automation oI
imaging and oI sight.
The Iield oI computer vision can be seen as the culmination oI at least centuries-long
histories. The Iirst is the history oI mechanical devices designed to aid human perception, such
as Renaissance perspectival machines. This history reaches its Iinal stage with computer vision,

223
"Retrospectives II: The Early Years in
Computer Graphics at MIT, Lincoln Lab, and
Harvard," 57.

which aims to replace human sight altogether. The second is the history oI automata, whose
construction was especially popular in the seventeenth and eighteenth centuries. Yet, despite
similarity in appearance, there is a Iundamental diIIerence between Enlightenment automata
which imitated human's or animal's bodily Iunctions and the modern day robots equipped with
computer vision systems, artiIicial legs, arms, etc. As noted by Leonardo Torres, old automata,
while successIully copying the appearance and movement oI living beings, had no economic
value. Indeed, such voice synthesis machines as Wolgang von Kempelen's 1778 device which
directly imitated the Iunctioning oI the oral cavity or AbbZ Mical's T tes Parlantes (1783)
operated by a technician hiding oIIstage and pressing a key on a keyboard were used only Ior
entertainment.
224
When in 1913 Torres called Ior automata that would "accomplish the results
which a living person obtains, thus replacing a man by a machine" he was expressing a
Iundamentally new idea oI using automata Ior productive labor. A Iew years later, the brother oI
the Czech writer Karel Capek coined the word robot Irom the Czech word robota, which means
"Iorced labor."
225
Capek's play R.U.R. (1921) and Fritz Lang's Metropolis (1927) clearly
demonstrate this new association oI automata with physical industrial labor.
ThereIore, it would be erroneous to conclude that, with computer vision, twentieth
century technology simply added the sense oI sight to eighteenth century mechanical statues. But
even to see computer vision as the continuation oI Torres', Capek's or Lang's ideas about
industrial automation which replaces manual labor would not be Iully accurate. The idea oI

224
Remko Scha, "Virtual Voices," MediaMatic 7, no. 1 (1992): 33. Scha describes two
Iundamental approaches taken by the developers oI voice imitating machines: the genetic method
which imitates the physiological processes that generate speech sounds in the human body and
the gennematic method which is based on the analysis and reconstruction oI speech sounds
themselves without considering the way in which the human body produces them. While the
Iield oI computer vision, and other Iields oI artiIicial intelligence, Iirst clearly Iollowed
gennematic method, in the 1980s, with the growing popularity oI neural networks, there was a
shiIt towards the genetic method -- direct imitation oI the physiology oI the visual system. In a
number oI laboratories, scientists begin to build artiIicial eyes which move, Iocus, and analyze
inIormation exactly like human eyes.
225
Eames and Eames, A Computer Perspective, 100.

computer vision became possible and the economic means to realize this idea became available
only when automation entered its second stage aIter World War II. The attention turned Irom the
automation oI the body to the automation oI the mind, Irom physical to mental labor. This shiIt
to the automation oI mental Iunctions such as vision, hearing, reasoning, problem solving is
exempliIied by the very names oI the two new Iields that emerged during the 1950s and 1960s --
artiIicial intelligence and cognitive psychology. The latter gradually replacing behaviorism, the
dominant psychology oI the "Fordism" era. The emergence oI the Iield oI computer vision is a
part oI this cognitive revolution, a revolution which was Iinanced by the military escalation oI
the Cold War.
226
This connection is solidiIied in the very term "artiIicial intelligence" which
may reIer simultaneously to two meanings oI "intelligence": reason, the ability to learn or
understand, and inIormation concerning an enemy or a possible enemy or an area. ArtiIicial
intelligence: artiIicial reason to analyze collected inIormation, collected intelligence.
In the 1950s, Iaced with the enormous task oI gathering and analyzing written,
photographic, and radar inIormation about the enemy, the CIA and the NSA (National Security
Agency) began to Iund the Iirst artiIicial intelligence projects. One oI the earliest projects was a
Program Ior Mechanical Translation, initiated in the early 1950s in the attempt to automate the
monitoring oI Soviet communications and media.
227
The work on mechanical translation was
probably the major cause oI many subsequent developments in modern linguistics, its move
towards Iormalization; it can be discerned in Noam Chomsky's early theory which, by
postulating the existence oI language universals in the domain oI grammar, implied that
translation between arbitrary human languages could be automated. The same work on
mechanical translation was also one oI the catalysts in the development oI the Iield oI pattern
recognition, the precursor to computer vision. Pattern recognition is concerned with

226
De Landa, "Policing the Spectrum," 194-203.
227
Ibid., 214.

automatically detecting and identiIying predetermined patterns in the Ilow oI inIormation. A
typical example is character recognition, the Iirst stage in the process oI automating translation.
Pattern recognition was also used in the U.S. Ior the monitoring oI Soviet radio and telephone
communication. Instead oI listening to every transmission, an operator would be alerted iI
computer picked up certain words in the conversation.
As a "logistics oI perception" came to dominate modern warIare and surveillance and as
the space race began, image processing became another major new Iield oI research. Image
processing comprises techniques to improve images Ior human or computer interpretation.
228
In
1964, the space program Ior the Iirst time used image processing to correct distortions in the

228
The Iirst paper on image processing was published in 1955.
L.S.G. Kovasznay, and H.M. Joseph,
"Image Processing," Proceedings oI IRE 43 (1955): 560-570.

pictures oI the Moon introduced by a on-board television camera oI Ranger 7.
229
In 1961, the
National Photographic Interpretation Center (NPIC) was created to produce photoanalysis Ior the
rest oI the U.S. intelligence community and, as Manual De Landa points out, by the end oI the
next decade computers "were routinely used to correct Ior distortions made by satellite's imaging
sensors and by atmospheric eIIects, sharpen out-oI-Iocus images, bring multicolored single
images out oI several pictures taken in diIIerent spectral bands, extract particular Ieatures while
diminishing or eliminating their backgrounds altogether..." De Landa also notes that computer
analysis oI photographic imagery also became the only way to deal with the pure volume oI
intelligence being gathered: "It became apparent during the 1970s that there is no hope oI
keeping up with the millions oI images that poured into NPIC...by simply looking at them the
way they had been looked at in World War II. The computers thereIore also had to be taught to
compare new imagery oI a given scene with old imagery, ignoring what had not changed and
calling the interpreter's attention to what had."
230


229
RaIael C. Gonzalez, and Paul Wintz,
Digital Image Processing (Reading, Mass.: Addison-Wesley Publishing Company, 1977), 2.
230
Qtd. in De Landa, "Policing the Spectrum," 200.

The techniques oI image processing, which can automatically increase an image's
contrast, remove the eIIects oI blur, extract edges, record diIIerences between two images, and so
on, greatly eased the job oI human photoanalysts. And the combining oI image processing with
pattern recognition made it possible in some cases to delegate the analysis oI photographs to a
computer. For instance, the technique oI pattern matching used to recognize printed characters
can also be used to recognize objects in a satellite photograph. In both cases, the image is treated
as consisting oI two-dimensional Iorms. The contours oI the Iorms are extracted Irom the image
are then compared to templates stored in computer memory. II a contour Iound in the image
matches a particular template, the computer signals that a corresponding object is present in a
photograph.
A general computer vision program has to be able to recognize not just two-dimensional
but three-dimensional objects in a scene taken Irom an arbitrary angle.
231
Only then it can be
used to recognize an enemy's tank, to guide an automatic missile towards its target or to control a
robotic arm on the Iactory Iloor. The problem with using simple template matching is that "a
two-dimensional representation oI a two-dimensional object is substantially like the object, but a
two-dimensional representation oI a three-dimensional object introduces a perspective projection
that makes the representation ambiguous with respect to the object."
232
While pattern
recognition was working Ior images oI two-dimensional objects, such as letters or chromosomes,
a diIIerent approach was required to "see" in 3-D.
Roberts' 1965 paper "Machine Recognition oI Three-dimensional Solids" is considered to
be the Iirst attempt at solving the general task oI automatically recognizing three-dimensional

231
Within the Iield oI computer vision, a scene is deIined as a collection oI three-dimensional
objects depicted in an input picture. David McArthur, "Computer Vision and Perceptual
Psychology," Psychological Bulletin 92, no. 2 (1982): 284.
232
Paul R. Cohen and Edward A. Feigenbaum, eds., The Handbook oI ArtiIicial Intelligence
(Los Atlos, CA: William KauImann, Inc., 1982), 3: 139.

objects.
233
His program was designed to understand the artiIicial world composed solely oI
polyhedral blocks -- a reduction oI reality to geometry that would have pleased CZzanne. Using
image processing techniques, a photograph oI a scene was Iirst converted into a line drawing.
Next, the techniques oI 3-D computer graphics were used:

Roberts' program had access to three-dimensional models oI objects: a cube, a rectangular solid, a
wedge, and a hexagonal prism. They were represented by coordinates (x, y, z) oI their vertices. A

233
L.G. Roberts, "Machine perception oI
three-dimensional solids," Optical and Electo Optical InIormation Processing, ed. J.T. Tippett
(Cambridge: The MIT Press, 1965) .

program recognized these objects in a line drawing oI the scene. A candidate model was selected
on the basis oI simple Ieatures such as a number oI vertices. Then the selected model was rotated,
scaled, projected, and matched with the input line drawing. II the match was good, the object was
recognized, as were its position and size. Roberts' program could handle even a composite object
made oI multiple primitive shapes; it subtracted parts oI a line drawing Irom the drawing as they
were recognized, and the remaining portions were analyzed Iurther.
234


Was this enough to completely automate human vision? This depends upon how we
deIine vision. The chapter on computer vision in The Handbook oI ArtiIicial Intelligence (1982)
opens with the Iollowing deIinition: "Vision is the inIormation-processing task oI understanding
a scene Irom its projected images."
235
But what does "understanding a scene" mean? With
computer vision research Iinanced by the military-industrial complex, the deIinition oI
understanding becomes highly pragmatic. In the best tradition oI the pragmatism oI James and
Pierce, cognition is equated with action. The computer can be said to "understand" a scene iI it
can act on it -- move objects, assemble details, destroy targets. Thus, in the Iield oI computer
vision "understanding a scene" implies two goals. First, it means the identiIication oI various
objects represented in an image. Second, it means reconstruction oI three-dimensional space
Irom the image. A robot, Ior instance, need not only recognize particular objects, but it has to
construct a representation oI the surrounding environment to plan its movements. Similarly, a
missile not only has to identiIy a target but also to determine the position oI this target in three-
dimensional space.
It can be seen that Roberts' program simultaneously IulIilled both goals. His program
exempliIied the approach taken by most computer vision researchers in the Iollowing two
decades. A typical program Iirst reconstructs the three-dimensional scene Irom the input image
and then matches the reconstructed objects to the models stored in memory. II the match is good,
the program can be said to recognize the object, while simultaneously recording its position (Iig.
14).

234
Cohen and Feigenbaum, The Handbook oI ArtiIicial Intelligence, 3: 129.
235
Ibid., 127.

A computer vision program thus acts like a blind person who "sees" objects (i.e.,
identiIies them) by reading their shapes through touch. As Ior a blind person, understanding the
world and the recognition oI shapes are locked together; they cannot be accomplished
independently oI one another. With computer vision, visual nominalism, which, aIter all, is just a
particular use oI vision, attains unprecedented importance. Now, to see means to see shapes and
distances. Visual nominalism and vision itselI have become equated.

I have presented the new twentieth century technologies oI visual nominalism as building upon
perspective, extending its powers in space and time. Technologies oI remote sensing, such as
radar, extended perspective beyond the realm oI the visible. 3-D computer graphics speeded up
and automated design and perspectival display oI the models oI both real and imagined objects.
With the emergence oI the Iield oI computer vision, perspectival sight reaches its
apotheosis and at the same time begins its retreat. At Iirst computer vision researchers believed
that they could invert the perspective and reconstruct the represented scene Irom a single
perspectival image. Eventually, they realized that it is oIten easier to bypass perspectival images
altogether and use other means as a source oI three-dimensional inIormation.
Latour points out that with the invention oI perspective it became possible to represent
absent things and plan our movement through space by working on representations. To quote
him again, "one cannot smell or hear or touch Sakhalin island, but you can look at the map and
determine at which bearing you will see the land when you send the next Ileet."
236
Roberts'
program extended these abilities even Iurther. Now the computer could acquire Iull knowledge
oI the three-dimensional world Irom a single perspectival image! And because the program
determined the exact position and orientation oI objects in a scene, it became possible to see the
reconstructed scene Irom another viewpoint. It also became possible to predict how the scene

236
Latour, "Visualisation and Cognition," 8.

would look Irom an arbitrary viewpoint.
237
Finally, it also became possible to guide
automatically the movement oI a robot through the scene.
Roberts' program worked using only a single photograph -- but only because it was
presented with a highly artiIicial scene and because it "knew" what it could expect to see.
Roberts limited the world which his program could recognize to simple polyhedral blocks. The
shapes oI possible blocks were stored in a computer. Others simpliIied the task even Iurther by
painting all objects in a scene the same color.
However, given an arbitrary scene, composed Irom arbitrary surIaces oI arbitrary color
and lighted in an arbitrary way, it is very diIIicult to reconstruct the scene correctly Irom a single
perspectival image. The image is "underdetermined." First, numerous spatial layouts can give
rise to the same two-dimensional image. Second, "the appearance oI an object is inIluenced by
its surIace material, the atmospheric conditions, the angle oI the light source, the ambient light,
the camera angle and characteristics, and so on," and all oI these diIIerent Iactors are collapsed
together in the image.
238
Third, perspective, as any other type oI projection, does not preserve
many geometric properties oI a scene. Parallel lines turn into convergent lines; all angles change;
equal lines appear unequal. All this makes it very diIIicult Ior a computer to determine which
lines belong to a single object.
Thus, perspective, which until now served as a Ioundation oI new technologies oI visual
nominalism, becomes the drawback which needs to be overcome. Perspective, this Iirst step
towards the rationalization oI sight (Ivins) has eventually become a limit to its total
rationalization -- the development oI computer vision.
The realization oI the ambiguities inherent in a perspectival image itselI came aIter years
oI vision research. In trying to compensate Ior these ambiguities, laboratories began to scrutinize

237
Cohen and and Feigenbaum, The Handbook oI ArtiIicial Intelligence, 3: 141.
238
Ibid., 128.

the Iormal structure oI a perspectival image with a degree oI attention unprecedented in the
history oI perspective. In 1968 Adolpho Guzman classiIied the types oI junctions that appear in
line representations aIter he realized that a junction type can be used to deduce whether regions
oI either side oI a junction line were part oI the same object.
239
In 1986 David Lowe presented
a method to calculate the probability that a particular regularity in an image (Ior instance, parallel
lines) reIlects the physical layout oI the scene rather than being an accident due to a particular
viewpoint.
240
All other sources oI depth inIormation such as shading, shadows or texture
gradients were also systematically studied and described mathematically.
Despite these advances, a single perspectival image remained too ambiguous a source oI
inIormation Ior practical computer vision systems. An alternative has been to use more than one
image at a time. Computer stereo systems employ two cameras which, like human eyes, are
positioned a distance apart. II the common Ieature can be identiIied in both images, then the
position oI an object can be simply determined through geometric calculations. Other systems
use a series oI continuous images recorded by a video camera.
But why struggle with the ambiguity oI perspectival images at all? Instead oI inIerring
three-dimensional structure Irom a two-dimensional representation, it is possible to measure
depth directly by employing various remote sensing technologies. In addition to video cameras,
modern vision systems also utilize a whole range oI diIIerent range Iinders such as lasers or
ultrasound.
241
Range Iinders are devices which can directly produce a three-dimensional map oI
an object. The same basic principle employed in radar is used: the time required Ior an electro-
magnetic wave to reach an object and reIlect back is proportional to the distance to the object.
But while radar reduces an object to a single point and in Iact is blind to close-by objects, a range

239
Ibid., 131.
240
David Lowe, Three-Dimensional Object Recognition Irom Single Two-Dimensional Images,
Robotics Report 62 (New York: Courant Institute oI Mathematical Sciences, New York
University, 1986).
241
Cohen and Feigenbaum, The Handbook oI ArtiIicial Intelligence, 3: 254-259.

Iinder operates at small distances. By systematically scanning the surIace oI an object, it directly
produces a "depth map," a record oI an object's shape which can be then matched to geometric
models stored in computer memory thus bypassing the perspectival image altogether (Iig. 15).

What began as separate developments -- to automate the recording oI three-dimensional
inIormation, the generation oI perspectival views, and the identiIication oI objects -- eventually
converged. Remote sensing, 3-D computer graphics, and computer vision Iorm a closed circle.
As we have seen, algorithms oI 3-D computer graphics were Iirst developed by Roberts in order
to solve the general problem oI computer vision. From that point on, the two Iields developed in
parallel. The Iield oI computer graphics was gradually learning to simulate more and more
"realistic" images oI reality, adding shading, shadows and texture to initial wireIrame drawings.
Meanwhile, the Iield oI computer vision was learning to deal with shading, shadows, and texture
as sources oI depth inIormation.
In eIIect, the goal oI computer vision is to undo what 3-D computer graphics aims to
achieve. The Iormer tries to reconstruct a scene Irom its photographic image, to deduce viewer
independent inIormation about the objects: their shapes and dimensions. The goal is to "produce
an object centered representation that is independent oI the particular details oI the viewing angle
or photographic process."
242
The aim oI 3-D computer graphics is exactly the opposite: given
the objective inIormation about the scene (shapes and positions oI objects, their materials,
direction oI light) to produce its image which is virtually indistinguishable Irom a photograph.
At the Ioundation oI this mutual dependence is the shared geometrical model oI reality as
a set oI surIaces having a particular orientation, position, and curvature. The circle is completed
by the technologies oI remote sensing that, by producing depth maps oI real scenes, can

242
David Peat, ArtiIicial Intelligence: How Machines Think (New York: Baen Enterprises,
1985), 230.

automatically reduce them to geometric models. Portable shape acquisition cameras are already
being developed that, instead oI taking snapshots oI a scene, will automatically generate
geometric models which then can be used Ior computer vision or 3-D computer graphics.
243

Technologies oI remote sensing can automatically create a depth map oI reality. Using
such depth map as input, 3-D computer graphics can be used to generate an image which, in its
turn, can act as input to a computer vision system. Computer vision then can reconstruct
"objective" inIormation Irom an image, a model which can act as a source Ior Iurther 2-D
computer graphics synthesis. A series oI mirrors, constructions and reconstructions are triggered
by each other, with speed only limited by the processing power oI "geometry engines."



In this chapter I discussed the twentieth century automation oI the Iunction oI vision which I
have called visual nominalism -- capturing the inIormation about shapes and distances and
representing this inIormation in two-dimensional displays. Radar and other technologies oI
remote sensing extend the capacities oI human vision beyond the limits oI visibility, making it
possible to track positions oI objects in real time. Techniques oI 3-D computer graphics allow the
interactive display and construction oI geometric models oI both real and non-existent objects.
Finally, computer vision automates the recognition oI objects and the reconstruction oI three-
dimensional scenes Irom their images.
In this discussion, the concept oI perspective played a special role Ior a number oI
reasons. I relied on the Iamiliarity oI this concept to explain the principles oI operations oI newer
twentieth century technologies oI visual nominalism. I have also relied on Ivins' account oI
perspective and its extension by Latour to suggest that the Renaissance's adaptation oI

243
Rheingold, Virtual Reality, 229, 252. 6. ConcIusion

perspective represented the Iirst step in the automation oI visual nominalism. While other
cultures used sophisticated methods oI space mapping, the importance oI perspective lies not in
its representational superiority but in its algorithmic character. This algorithmic character
enabled the gradual development oI visual languages oI perspective and descriptive geometry
and, in parallel, oI perspectival machines and technologies, Irom a simple net described by
DYrer to photography and radar. And when digital computers made possible mass automation in
general, automation oI perspectival vision and imaging Iollowed soon. It is this automation oI
vision and imaging which gave the West superiority in the "logistics oI perception," and
contributed to its victory in the GulI War.
A two-dimensional representation containing inIormation about geometry and
topography allows one to control reality across space and time. The advantage oI perspective is
that it makes it possible to present this inIormation in an easily readable Iorm within a single
image.
In the West, until this century, perspective Iormed the Ioundation oI techniques and
technologies oI visual nominalism. New twentieth century technologies oI visual nominalism
extend perspective, utilizing to the extreme its inherent qualities such as the algorithmic
character and the reciprocal relationship it establishes between reality and representation. For
instance, the perspective algorithm, a Ioundation oI both computer graphics and computer vision,
is used to generate perspectival views given a geometric model and to deduce the model given a
perspectival view. In another example, technologies oI remote sensing, such as radar, extend
perspectival vision beyond the visible, relying on the Iact that "what is an issue in geometric
perspective is simply the mapping oI space, not sight" (Lacan). Yet, while giving rise to new
technologies oI "geometric vision," perspective also becomes a limit to the Iinal automation oI
sight -- recognition oI objects by a computer. Finally, it is displaced Irom its privileged role,
becoming just one among other techniques oI space mapping and visualization.

In order to automate sight, to replace human vision with machine vision, a new
understanding oI vision became necessary. David Marr's Vision (1980) has been the most
inIluential account oI this paradigm, shared by computer scientists and psychologists. The book
opens with this statement:

What does it mean, to see? The plain man's answer (and Aristotle's, too) would be, to know what
is where by looking. In other words, vision is the process oI discovering Irom images what is
present in the world, and where it is.
244


There is nothing "plain" about this deIinition oI vision. Marr projects the goals oI computer
vision onto human vision: the identiIication oI objects represented in an image and the
reconstruction oI their positions in three-dimensional space. The larger part oI the book is
devoted to the discussion oI algorithms by which the human nervous system may accomplish this
reconstruction.
In this way, nature was redeIined as the most useIul instrument in its own replacement.
What David Marr, and many others have assumed, is that the key to successIul computer vision
systems lies in emulating the algorithms hidden in human nervous system. II only these
algorithms, created by nature during millions oI years oI evolution, could be understood, the way
to replace human vision by computer vision will be open.
The automation oI vision is a part oI the overall process oI industrial automation aIter
World War II. Automation aIIects the modern understanding oI vision in two crucial ways. On
the one hand, automation entails the replacement oI human cognitive Iunctions by a computer,
such as the substitution oI vision by computer vision. Here, visual nominalism becomes the
dominant paradigm Ior the study oI vision. On the other hand, the automation also involves the
integration oI human and machine in new man-machine systems, such as radar installations or
pilot cockpits. The need to describe the perIormance oI human and machine components in the

244
Marr, Vision, 3.

same terms leads to the emergence oI another paradigm -- vision as inIormation processing --
which will be discussed in the Iollowing chapter.




Writing in 1927, Lszl Moholy-Nagy makes a comparison between engineering and design:

A modern engineer, iI his goal and the Iunctional purpose oI his work are clear, can without any
great eIIort make a product that is Iormally adequate and perIect in its economic construction. But
the photographic advertisements oI our time are not so easy to deIine. They don't come with
"user's instructions." Research into the physiological and psychological laws oI visual
eIIectiveness is still Iar behind the times, compared to the study oI the physical laws.
245


In the opening chapter, where this quotation already appeared, I have traced one line oI research
into "the physiological and psychological laws oI visual eIIectiveness," suggesting that
investigations oI the psychological eIIects oI basic colors and elemental Iorms, conducted in
experimental psychology since the second halI oI the nineteenth century made possible the idea
oI a rational visual language composed Irom simple elements. In theory, this model called Ior the
enumeration oI visual elements and the description oI their eIIects in order to compose
dictionaries oI "visual eIIectiveness." In practice, it translated into visual works which at Iirst
only contained and then gradually became Iully composed oI simple abstract Iorms. Such theory
and practice were already pursued in the nineteenth and early twentieth centuries by a number oI
artists (Seurat, Signac, Kandinsky, Klee, Mondrian) and critics interested in "scientiIic
aesthetics" (Blanc, Henry). However, in the 1920s, when artists Iound themselves in the roles oI
the designers oI mass communication, the idea oI a scientiIically grounded visual language
acquired new urgency and importance. It was no longer a question oI scientiIic aesthetics, oI a
work oI art producing an aesthetic eIIect in an individual spectator. Now it became a question oI
rationalizing mass communication -- a question oI economic and political importance. The
design oI mass communication was too important to be leIt to an artist's intuition. ThereIore,

245
Lszl Moholy-Nagy, "Photography in Advertising," in Photography in the Modern Era,
ed. Christopher Phillips (New York: Aperture, 1989), 87. Chapter 4: The Engineering of Vision from INKhUK to MIT 1. Not Artists but Engineers

Moholy-Nagy wrote about the need to precisely "engineer" photographic advertisements, and the
Bauhaus welcomed experimental psychologists. The trend reached its extreme in Soviet Russia,
where artists thought that they could control not just the consumer habits oI segments oI society
(as in the West), but the consciousness oI the whole country --"aIIecting mass psyche, organizing
the will oI the class" (Tretyakov). Here the discourse oI engineering visual communication
became most systematic, translating into the establishment oI a number oI psychological
laboratories at various art institutions, such as VKhUTEMAS, GAKhN, and the State Institute oI
Artistic Culture. Their goal was to create a Iully rationalized visual language oI mass
communication where each visual element is capable oI communicating a meaning, producing an
emotion or causing a behavioral response. In the words oI a paper entitled "Engineerism"
presented at INKhUK: "visual sensations as such will concertedly shape the human being as an
organized unit."
246

This is one development, one trajectory -- Irom Blanc's theories and Fechner's
psychophysical experiments to INKhUK and VKhUTEMAS -- where the dream oI controlling
the masses through perception was systematically pursued. This trajectory leads Irom Wundt's
psychological laboratory to the "laboratory" explorations oI constructivists and rationalists; and
Irom the laboratory to the sites oI mass communication -- a movie hall, a printed page, an
architectural space, a stadium, a city street with posters, advertisements, and placards.
In this concluding chapter I will Iollow the next part oI this trajectory, the next stage in
the rationalization oI visual communication: Irom the avant-garde's Iantasies oI psychophysical
control oI the masses to inIormation theory, engineering psychology, and human-machine
interIaces. The visual language advocated by Rodchenko or Lissitsky -- elemental abstract Iorms,
each having a predictable eIIect -- seemed the most eIIective solution to the problem oI

246
Nicholas H. Allison, ed., Art Into LiIe: Russian Constructivism 1914-1932 (New York:
Rizzoli International Publications, 1990), 81.

rationalizing vision. But aIter World War II, a new kind oI rationalization became necessary. A
new paradigm was now needed to encompass not only human communication but also
communication between a human and a machine, the two linked in a human-machine system. It
is the emergence oI this paradigm which is the subject oI this chapter.
The new paradigm, called inIormation theory, emerged in the 1920s in response to the
growth oI modern telecommunications. The theory was originally developed by engineers in
order to measure the eIIiciency oI communication systems, such as telephone, radio, and
television. However, in the late 1940s, the theory leIt its engineering context and was embraced
by aestheticians, linguists, social scientists, and semioticians. Most importantly, it was taken up
by psychologists Iaced with the practical task oI designing human-machine systems which
increasingly came to dominate both the battleIield and the workplace aIter World War II (radar
screen, aircraIt controls, computer terminals oI the automated Iactory). As human vision became
the main channel oI communication between human and machine, it also came to be understood
in terms oI inIormation theory which now provided a new quantitative model oI a human
observer. Thus the dreams oI the avant-garde Ior an engineering approach to visual
communication were IulIilled.

The avant-garde was concern with rationalizing vision, with creating visual communication
according to scientiIic principles. In this avant-garde artists likened themselves to modern
engineers, as can be seen Irom Moholy-Nagy's statement. But what is a modern engineer?
The engineering proIession emerged in the 1870s as the liaison between science and
industry. The job oI an engineer was to put any industrial or corporate process on a scientiIic
basis in order to achieve the maximum output with a minimum investment oI time, materials,
and energy. In other words, the job oI an engineer was to make a process eIIicient, regardless oI
what industry s/he worked in. A 1933 book on engineering deIines it in this way: "Engineering is

the science and art oI eIIicient dealing with materials and Iorces...it involves the most
economical design and execution oI a vast number oI important undertakings, assuring, when
properly perIormed, the most advantageous combination oI accuracy, saIety, durability, speed,
simplicity, eIIiciency and economy possible Ior the design and service."
247
The avant-garde
artists aimed to apply these principles oI engineering to the Iield oI mass visual communication.
In 1921, during a heated discussion at INKhUK (Institute oI Artistic Culture) over how to
turn their Iellow artists into "constructivists," who would utilize modern engineering methods Ior
construction -- be it the construction oI consumer goods or mass psyche, industrial design or
mass propaganda -- participants expressed doubts about whether traditional artists could
accomplish this transition Irom "art into liIe." Boris Arvatov suggested that accomplished artists
should become political activists, and the young ones would have to go to the polytechnic.
248

More categorical was the opinion oI Sternberg, who, aIter asking the same question, "What are
artists to do?," answered it in these words, "They're good Ior nothing, they should be dealt with
the way the Cheka deals with counterrevolutionaries."
249
A Iew minutes later in the discussion
he suggested a more humane solution, the solution which was repeatedly put into practice in
regard to Soviet non-conIormist artists decades later -- their conIinement to a mental institution.
"All the artists who conduct youth groups, and who are going to continue their work and perhaps
teaching the way they have in the past, we should really stuII a hospital with these artists."
250

Arvatov's request that artists should get their training in an engineering school, and
Sternberg's warning that those who reIuse to comply would be shot, illustrate the grave
seriousness with which many Soviet avant-garde artists oI the 1920s took the ideal oI
engineering as a model Ior their own practice. Consider the term constructivism itselI. In

247
Qtd. in Theodore Hoover and Charles Fish, The Engineering ProIession (StanIord: StanIord
University Press, 1941), 416. Emphasis mine -- L.M.
248
Allison, Art Into LiIe, 74.
249
Ibid.
250
Ibid., 77.

Russian, konstruktor (the one who constructs) is a synonym Ior engineer. This allowed a number
oI artists to call themselves konstruktors and to reIer to their movement as constructivism. The
term conveniently emphasized that, like engineers, they would construct utilitarian objects or
propagandistic works on the basis oI scientiIic, rational principles, in contrast to traditional
artists who were said to rely on inspiration. Translated into English, the word constructivism
loses its close association with engineering, leading to a somewhat diIIerent interpretation oI the
entire movement.
In the early 1920s, constructivist artists competed to be the best at embodying the ideal oI
an engineer. Vertov announces his desire to become "kinok-engineer, controlling cameras over
distance."
251
Eisenstein, who, like Sternberg, graduated Irom an engineering school beIore the
1917 Revolution, compares the theatrical apparatus (actors' perIormances, sound eIIects,
lighting) to the machinery (orudie obrabotki) which a director should use to Iorm the viewer -- as
a milling machine Iorms machinery parts.
252
Later, he likens the eIIect oI a Iilm to the work oI
an engine -- a signiIicant analogy since engineering was historically identiIied with the design oI
engines.
253
Tatlin photographs himselI next to his already legendary design Ior the monument to
the Third International. The monument, with its two intertwining lattice spirals 400 meters high,
looks like an EiIIel Tower put at an angle, eternally moving into the sky, a symbol appropriate to
signiIy the idea oI permanent revolution, the (Iuture) industrial might oI the new state and, at the
same time, the engineering skills oI its designer. Finally, Lissitsky and his students design
another triumph to engineerism -- an inclined lattice girder entitled "Lenin Tower."

251
Dziga Vertov, "Kinoki. Perevorot" (Kinoki. A revolution), LEF 3 (1923): 143.
252
Sergei Eisenstein, "Montazh atraktsionov (Montage oI attractions)," LEF 3 (1923): 71.
253
A book on the engineering proIession published in 1941 quotes these dictionary deIinitions
oI engineering: "engineering: originally, the art oI making engines; in its modern and extended
sense, the art and science by which mechanical properties oI matter are made useIul to man in
structures and machines;" "engineering: the art and science oI making, building, or using engines
and machines, or oI designing and constructing public works or the like requiring special
knowledge oI materials, machinery, and the laws oI mechanics." Hoover and Fish, The
Engineering ProIession, 416.

Here is another device: similar to traditional portraits where the identity oI a person was
established by depicting the instruments oI his or her craIt, constructivists' portraits and selI-
portraits prominently Ieature engineering instruments. Lissitsky's selI portrait: on the background
oI graphed paper -- his Iace and through it -- a hand holding a compass, its leg touching a
perIectly drawn square. The title oI this photomontage is Konstruktor. Even more dramatic is
another Lissitsky's montage, a portrait oI Tatlin (Iig. 16). Tatlin is standing on a chair, holding a
long pole -- a ruler. By his Ieet are mathematical symbols (an integral, a square root), a curve
crossing axis lines. Behind him is a blackboard with more symbols. And, a monstrosity: a huge
compass, widely open, growing straight Irom Tatlin's eye. A man-compass -- measuring,
calculating, constructing.
Constructing what? "My reaction to the participation oI the artist in production is
positive," declared architect Vesnin during one INKhUK discussion, "but an artist must look
aIter his own aIIairs, and his aIIair is the |psycho-physiological| eIIect oI Iorm on |human|
consciousness." In other words, Vesnin is saying: let us leave the engineering oI machines and
mass-produced objects to real engineers. Our business, the business oI designers is the
engineering oI perception, the engineering oI visual communication -- the creation oI
architecture, mass spectacles, posters, designs, photomontages, which would produce calculated
bodily responses, emotions, and meanings in the viewers. "The modern engineer has created
brilliant objects: the bridge, the steam engine, the airplane, the crane,...the modern artist must
create objects that are equal to them in power, intensity, and potential in the context oI their
psycho-physiological impact as an organizing element in man's consciousness."
254
The job oI an
artist is to engineer vision, and to engineer vision means to aIIect the viewer with engineering
precision, predictability, and eIIectiveness.

254
Qtd. in Anatole Senkevitch, Jr., "The Sources and Ideals oI Constructivism in Soviet
Architecture," in Art Into LiIe: Russian Constructivism 1914-1932, ed. Nicholas H. Allison
(New York: Rizzoli International Publications, 1990), 173-174.

In this statement Irom the early 1920s, Vesnin envisions an engineer as the designer oI
physical objects -- the bridge, the steam engine, the airplane, the crane. But already in the same
decade a new kind oI engineer appears, whose job is to design not objects but communication
systems -- telephone, radio, television. A new paradigm appears as well -- inIormation theory
which becomes the scientiIic basis oI communication engineering. In the next section, I will
Iollow its emergence.



In 1922 Moholy-Nagy created his Iamous "Telephone Pictures." Here is his account:

In 1922 I ordered by telephone Irom a sign painter Iive paintings in porcelain enamel. I had the
Iactory's color chart beIore me and I sketched my paintings on graph paper. At the other end oI the
telephone the Iactory supervisor had the same kind oI paper, divided into squares. He took the
dictated shapes in correct position. (It was like playing chess by correspondence.)
255


There is no conclusive prooI that this event actually took place. At any rate, the story helped
Moholy-Nagy get a teaching job at the Bauhaus the next year, where he could Iully pursue his
vision oI using the latest communication systems as artistic tools. His colleague wrote oI the
Bauhaus atmosphere: "There is incessant talk oI cinema, optics, mechanics, projection and
continuous motion...Is this the atmosphere in which painters like Klee and some others oI us can
go on developing? Klee was quite depressed yesterday when talking about Moholy."
256

The story oI "Telephone Pictures" became one oI the myths oI modern art, or rather, oI
the modern struggle against the Romantic notion oI art: to reIuse subjectivity, the unique touch

255
Lszl Moholy-Nagy, The New Vision (New York: Wittenborn, 1947), 79. The three
existing accounts oI this episode are discussed in Eduardo Kac, "Aspects oI the Aesthetics oI
Telecommunications," in SIGGRAPH '92 Visual Proceedings, ed. John Grimes and Gray Lorig
(New York: The Association Ior Computing Machinery, 1992), 52-53.
256
Qtd. in Otto Stelzer, "Moholy-Nagy and His Vision," in Lszl Moholy-Nagy, Painting,
Photography, Film (1925; reprint, Cambridge: The MIT Press, 1973), 146. 2. Information Theory: an Engineer AnaIyzes Communication

oI the artist's hand and the artist's eye (exempliIied in Bauhaus by Klee) and to embrace the
mass-produced, the industrial, the mechanical, electrical, electronic, and digital -- Irom
Duchamp's ready-made to Op Art, Kinetic Art, copy art, telecommunication art, Iax art,
computer art -- in short, the tradition oI industrial art, throughout the twentieth century Ieeding
on the latest technical developments in communication and imaging. And Moholy-Nagy
intended the story this way, as a myth, a beginning more important than Duchamp's ready-
mades. Indeed, not just art as chess, as a logical game (as with Duchamp), but -- art as "playing
chess by correspondence," a communication over distance between a sender and a receiver who
do not occupy the same physical space.
Let us analyze Moholy-Nagy's story Irom the point oI view oI telecommunication -- the
communication over distance. It mentions three diIIerent communication systems. First, the most
ancient: communication with the help oI signs, such as traIIic or street signs (Moholy-Nagy
orders his pictures at the sign Iactory). Second, more modern: the postal service. Third, still a
recent development Ior the 1920s: the telephone. As communication systems developed, they
made possible the transmission oI more inIormation over longer distances in a shorter period oI
time. The eIIectiveness oI road signs was limited by the distance at which they were visible, and
the amount oI inIormation they carried was minimal. With the postal service, this distance limit
was overcome, but a signiIicant time delay remained. This delay was Iinally eliminated by the
telephone.
In the story, Moholy-Nagy communicated his design to the Iactory supervisor over the
telephone, dictating coordinates and colors oI squares. He was able to do this only by Iitting his
design to a pre-existing Iactory grid, treating the grid's squares as minimal units oI his
composition. (Moholy-Nagy, in other words, Iully embraced digital visuality and computer art
long beIore digital computers.) But could he, in 1922, transmit this design as a picture -- and in

real time? And what iI the picture was an arbitrary image, Ior instance a photograph oI a real
scene, rather than a by-product oI the communication code?
The rapid transmission oI photographic pictures was just becoming possible, utilizing the
same method employed by Moholy-Nagy -- digital representation. In 1921, digitized newspaper
photographs began to be regularly sent between London and New York over a submarine cable
in less than three hours. On one end, a picture was segmented using a grid, and the tone oI each
cell was coded; on another end, the received code would drive a special printer Iitted with
typeIaces simulating diIIerent tones.
257

A Iew years later, Moholy-Nagy included the examples oI this "wireless telegraphed
photography" in his Painting, Photography, Film, proudly putting them at the very end oI the
photographic section oI the book to represent the latest achievement in mechanized tele-seeing
(Iig. 17). Yet, this system was not Iast enough Ior the age oI "the Iilm; the electric sign,
simultaneity oI sensorily perceptible events."
258
The speed oI wireless telegraphed photography
was limited by the channel capacity oI an older communication system -- telegraphy. Images
contained much more inIormation than telegraphic cable was originally designed to carry, so
speed had to be sacriIiced. Throughout the 1920s, engineers in Russia, the U.S., England, and
Germany were working on a new system to transmit images in real time called television --
vision over distance.
The explosive development oI modern communication systems -- telegraphy, telephone,
radio, and television, along with their growing economic importance -- urged the creation oI a
general communication theory.
259
With telephone, radio, and television becoming the
Ioundations oI whole new industries in the earlier decades oI the century, the question oI

257
M. D. McFarlane, "Digital Pictures FiIty Years Ago," Proc. IEEE 60, no. 7 (1972).
258
Moholy-Nagy, Painting, Photography, Film, 39.
259
For a history oI the developments leading to the creation oI mathematical theory oI
communication, see Colin Cherry, On Human Communication, 2nd ed. (Cambridge: The MIT
Press, 1968), 41-52.

eIIiciency came to the Ioreground. Engineers needed a quantitative theory which could deIine
and measure the commodity that was communicated over telegraph, telephone, and radio
networks. Quantitative measures were also required in order to compare the capacities oI various
communication systems (Ior instance, wire versus radio transmission) and to evaluate their
perIormance.
The distant precursor to modern communication theory can be Iound in the centuries-old
interest in inventing secret codes Ior military and diplomatic communication. But with the
appearance oI modern telecommunications, the problem oI designing codes to make the
deciphering oI messages more diIIicult Ior the enemy was joined by a new problem, that oI
designing codes which could make everyday communication more eIIicient. Already by 1825,
telegraphists designated certain commonly used messages, such as commercial expressions and
greetings, by single numbers, thus making coded messages shorter. The Iamous dot-dash code,
introduced by S.F.B. Morse in 1832, associated the most commonly used letters with the shortest
dot-dash symbols, thus making coded messages use even Iewer symbols.
260

The problem oI how to devise the most eIIicient methods oI encoding, transmitting, and
decoding signals was greatly intensiIied by the development oI television in the 1920s.
Television, the "Far Seer," required the simultaneous transmission oI a much greater amount oI
inIormation than in previous communication technologies: "masses oI inIormation had to be read
oII at high speed at the camera end, transmitted, and reassembled in the receiver."
261
The great
channel capacity required Ior television spurred new theoretical studies, and by the end oI the
1920s, the Iundamental ideas oI the modern mathematical theory oI communication (which also
came to be called inIormation theory or statistical communication theory) were Iormulated.
(SigniIicantly, it was the great promise and also the great diIIiculty in the development oI a

260
Cherry, On Human Communication, 36-37.
261
Ibid., 42.

system Ior visual telecommunication -- television -- which was decisive in the rise oI the new
paradigm oI a quantitative approach to communication.)
The essential ideas oI inIormation theory were presented in 1927 at the International
Congress oI Telegraphy and Telephony by R.V.L. Hartley oI the Bell Telephone Laboratories in
a paper entitled "Transmission oI InIormation." Hartley proposed "to set up a quantitative
measure whereby the capacities oI various systems to transmit inIormation can be compared"
and to "discuss its application to systems oI telegraphy, telephony, picture transmission and
television over both wire and radio paths."
262
Hartley's theory conceptualized the transmission
oI a message as the successive selection oI signs Irom a set, known to both the sender and the
receiver. II the receiver knows beIorehand which oI the symbols will be transmitted, no new
knowledge (no inIormation) is communicated. The amount oI inIormation is proportional to the
uncertainty about which sign can be received. II a particular message is made up Irom N signs
which can be chosen Irom a total set oI S signs, then there are S
N
distinct possibilities. Hartley
proposed that the "quantity oI inIormation" (H) oI such a message is HN log
2
S. Hartley also
established one oI the basic laws oI inIormation theory: the total amount oI inIormation a system
can transmit "is proportional to the product oI the Irequency range which it transmits by the time
available Ior transmission."
263
In other words, the more signals a system transmits
simultaneously (Irequency range, or bandwidth) and the longer the time available Ior
transmission, the greater its inIormation capacity.
With Hartley's paper the two Iundamental tools oI inIormation theory were now in place:
a quantitative measure oI inIormation and an equation speciIying the inIormation capacity oI a
communication system. It may seem that the theory deIined inIormation in an abstract and
counter-intuitive way by associating it with the Ireedom to make a choice (on the part oI the

262
R.V.L. Hartley, "Transmission oI InIormation," Bell System Tech. Journal 7 (1928): 535.
263
Ibid., 554.

sender) and with uncertainty (on the part oI the receiver). Yet, the deIinition oI inIormation is
quite pragmatic Irom the point oI view oI an engineer concerned with making a communication
system eIIicient. What the measure oI inIormation (H) deIines is the minimum number oI binary
choices which enables the receiver to reconstruct the message, or conversely, which is required
to transmit it.
264
In other words, "it is a measure oI the minimal eIIort by which message can be
transmitted."
265

The development oI inIormation theory was Iurther motivated by the work oI scientists
and engineers on communication and electronic systems during World War II, the most
important oI which was radar. In its modern Iorm, the theory was Iormulated by Claude
Shannon; in 1949, his papers, together with a non-technical overview by Warren Weaver, were
published in what became the bible oI the coming inIormation age -- Mathematical Theory oI
Communication.
266

In his contribution, Weaver extended Shannon's model oI inIormation transmission,
created to describe physical systems oI telecommunications, such as telegraphy or radio, to a
more general model oI any communication situation (Iig. 18). To emphasize this generality, his
description oI the model deliberately juxtaposes examples oI telecommunication systems and
human communication:

The inIormation source selects a desired message out oI a set oI possible messages...The
selected message may consist oI written or spoken words, or oI pictures, music, etc.
The transmitter changes this message into the signal which is actually sent over a
communication channel Irom the transmitter to the receiver. In the case oI telephony, the channel

264
Each binary choice constitutes one bit oI inIormation. Weaver writes: "iI one has available
say 16 alternative messages among which he is equally Iree to choose, then since 162
4
so that
log
2
164, one says that this situation is characterized by 4 bits oI inIormation." Claude E.
Shannon and Warren Weaver, The Mathematical Theory oI Communication (Urbana: The
University oI Illinois Press, 1949), 100-101.
265
D. Gabor, "A Summary oI Communication Theory," in Proceedings oI a Symposium on
Applications oI Communication Theory, London, 1952, ed. Willis Jackson (London: Butterworth
ScientiIic Publications, 1953), 2.
266
Shannon and Weaver, The Mathematical Theory oI Communication.

is a wire, the signal a varying electrical current on this wire; the transmitter is the set oI devices
(telephone transmitter, etc.) which change the sound pressure oI the voice into the varying
electrical current. In telegraphy, the transmitter codes written words into sequences oI interrupted
currents oI varying lengths (dots, dashes, spaces). In oral speech, the inIormation source is the
brain, the transmitter is the voice mechanism producing the varying sound pressure (the signal)
which is transmitted through the air (the channel). In radio, the channel is simply space (or the
aether, iI any one still preIers that antiquated and misleading word), and the signal is the
electromagnetic wave which is transmitted.
The receiver is a sort oI inverse transmitter, changing the transmitted signal back into a
message, and handing this message on to the destination. When I talk to you, my brain is the
inIormation source, yours the destination; my vocal system is the transmitter, and your ear and the
associated eighth nerve is the receiver.
In the process oI being transmitted, it is unIortunately characteristic that certain things are
added to the signal which were not intended by the inIormation source. These unwanted additions
may be distortions oI sound (in telephony, Ior example), or static (in radio), or distortions in shape
or shading oI picture (television), or errors in transmission (telegraphy or Iacsimile) etc. All oI
these unwanted changes in the transmitted signal are called noise.
267


It is not accidental that the principal examples in this description -- telegraphy, telephony, and
oral speech -- all have to do with written or verbal communication. By the middle oI the century,
both the engineer and the public still associated electronic communication with telephone and
radio, and this is why the example oI human communication which naturally comes to Weaver is
speech. It would be Iew decades beIore electronic communication would become synonymous
with the communication oI images (television, teleconIerencing, multimedia, teleoperators,
ISDN, cyberspace); beIore the terms oI inIormation theory, such as channel, bit, bandwidth will
enter the speech oI journalists -- telling us everyday about yet another development in visual
telecommunication -- as well as the speech oI policymakers, endlessly debating who will control
these channels and how wide their bandwidth will be.



The Mathematical Theory oI Communication appeared in 1949 and within years the theory was
taken up by linguists, psychologists, social scientists, and even art historians. Thus, Roman

267
Ibid., 98-99. Emphasis in the original. 3. The InfIuence of Information Theory or the IdeoIogy of the Code

Jakobson, who as a proIessor at MIT was close to the source oI the paradigm, adopted the model
Ior linguistics and then semiotics.
268
Even quicker to embrace the paradigm were experimental
psychologists.
269

As inIormation theory was transIormed Irom a tool oI the communication engineer into a
broad intellectual paradigm, three crucial developments took place. First, the theory was
extended to understand not just communication within an electronic system, but also
communication between humans as well as between humans and machines. Second, the theory
was adopted to describe the meaning and eIIect oI communication -- its semantic and pragmatic
aspects. Third, the basic assumption oI the theory, that communication is a one-way process, was
extended to theorize social communication. In this section, I will Iirst consider these three
developments in turn.
The Iirst development is connected with the Iigure oI Norbert Wiener, a proIessor oI
mathematics at MIT who supervised Shannon's research. In 1948 Wiener, published Cybernetics
or Control and Communication in the Animal and the Machine which proposed that seemingly
diIIerent entities -- a machine, an animal, a human, an animal collective, a human collective --
can be conceived oI as communication systems. Wiener also equated communication and
control, thus opening a way to think oI social, political, and economic problems in inIormation-
cybernetic terms. He justiIied this equation in this way:

When I communicate with another person, I impart a message to him, and when he communicates
back with me he returns a related message which contains inIormation primarily accessible to him

268
Roman Jakobson, (1956), "Metalanguage as a Linguistic Problem," Presidential address
delivered at the Annual Meeting oI the Linguistic Society oI America, 1956, published in The
Framework oI Language (Ann Arbour: Michigan Studies in the Humanities, 1980): 81-92;
Roman Jakobson, (1960), "Closing Statement: Linguistics and Poetics," in Semiotics. An
Introductory Anthology, ed. Robert Innis (Bloomington: Indiana University Press, 1985).
269
G.A. Miller, Language and Communication (New York: McGraw-Hill Book Co., Inc.,
1951); D.E. Broadbent, Perception and Communication (OxIord: Pergamon Press, 1958); H.
Quastler, ed., InIormation Theory in Psychology: Problems and Methods (Glencoe, IL.: Free
Press, 1955); F. Attneave, Applications oI InIormation Theory to Psychology (New York: Holt,
1959).

and not to me. When I control the actions oI another person, I communicate a message to him, and
although this message is in the imperative mood, the technique oI communication does not diIIer
Irom that oI a message oI Iact. Furthermore, iI my control is to be eIIective I must take cognizance
oI any messages Irom him which may indicate that the order is understood and has been
obeyed.
270


In this explication Wiener talks about human communication, but his realization that the
"problems oI control engineering and oI communication engineering were inseparable" emerged
during the war when he worked on anti-aircraIt artillery systems. Wiener realized that the key to
successIul control lies in Ieedback -- the modiIication oI behavior oI a system (be it human or a
machine) by taking into account the results oI the actual (rather than predicted) perIormance. The
results oI the perIormance are communicated back to the control mechanism, making it adjust
the system's behavior. A prototypical example oI a Ieedback mechanism is a selI-guided shell:
"When we desire a motion to Iollow a given pattern, the diIIerence between this pattern and the
actually perIormed motion is used as a new input to cause the part regulated to move in such a
way as to bring its motion closer to that given by the pattern."
271

The authors oI inIormation theory always insisted that the theory was only concerned
with the conditions Ior the correct and eIIicient transmission oI signals constituting the message,
and not with its meaning, interpretation, or eIIect. Hartley points out that "in estimating the
capacity oI the physical system to transmit we should ignore the question oI interpretation."
272

Shannon also begins by insisting that "semantic aspects oI communication are irrelevant to the
engineering problem."
273
Finally, Weaver starts his essay included in The Mathematical Theory
oI Communication by distinguishing between the three levels oI a communication problem:
technical (how accurately the signs can be transmitted), semantic (how accurately signs convey

270
Norbert Wiener, (1950), The Human Use oI Human Beings. Cybernetics and Society (New
York: Avon Books, 1967), 24-25.
271
Norbert Wiener, Cybernetics. OI Control and Communication in the Animal and the
Machine (New York: John Wiley & Sons, Inc., 1948), 13.
272
Hartley, "Transmission oI InIormation," 538.
273
Shannon and Weaver, The Mathematical Theory oI Communication, 3.

the desired meaning), and pragmatic (how eIIective the message is in aIIecting the behavior oI
the receiver). Weaver reIers to these levels as A, B, and C.
274
He next emphasizes that the
mathematical theory oI communication as developed by Shannon applies only to the Iirst level,
yet also immediately suggests that it is deeply relevant Ior the second and third. Impressed by the
Iact that the technical aspect oI a communication process is Iinally controlled by a precise
mathematical theory, Weaver clearly hopes that the theory can somehow be extended to handle
the elusive questions oI semantics and pragmatics with the same quantitative precision.
Throughout the essay he promises a detailed discussion oI how this can be done. Weaver never
keeps his promise; instead, in a number oI places he suddenly announces that perhaps no
modiIications will be necessary. The announcements appear abruptly at the end oI sentences,
explosions in what is otherwise the dry rhetoric oI a scientiIic discourse. "It is the purpose oI this
concluding section to review the situation, and to see to what extent and in what terms the
original section was justiIied in indicating that the progress made at Level A is capable oI
contributing to levels B and C, was justiIied in indicating that the interrelation oI the three levels
is so considerable that one's Iinal conclusion may be that the separation into the three levels is
really artiIicial and undesirable." "It is almost certainly true that a consideration oI
communication on levels B and C will require additions to the schematic diagram on page 97
|Iig. 18|, but it seems equally likely that what is required are minor additions, and no real
revision."
275

The temptation to extend inIormation theory to handle the meaning and eIIect oI
communication proved too great to resist. This was understandable given the historical context oI
the Cold War during which the theory matured and attracted public attention. This context is not
hard to see in Weaver's descriptions oI diIIerent levels oI communication: "The semantic

274
Ibid., 96.
275
Ibid., 114, 115. Emphasis mine -- L.M.

problem has wide ramiIications iI one thinks oI communication in general. Consider, Ior
example, the meaning to a Russian oI a U.S. newsreel picture." "The problem oI eIIectiveness
involves...all the psychological and emotional aspects oI propaganda theory."
276
Even more
revealing is the opening paragraph oI the essay, where within a Iew lines we are transported Irom
artistic communication, painting, theater, and ballet, to the theater oI the new cybernetic warIare,
which emerged during World War II and now protected the U.S. against a Soviet attack:

The word communication will be used in a very broad sense to include all oI the procedures by
which one mind may aIIect another. This, oI course, involves not only written and oral speech, but
also music, the pictorial arts, the theatre, the ballet, and in Iact all human behavior. In some
connections it may be desirable to use a still broader deIinition oI communication, namely, one
which would include the procedures by means oI which one mechanism (say automatic equipment
to track an airplane and to compute its probable Iuture positions) aIIects another mechanism (say a
guided missile chasing this airplane).
277


In 1952 Y. Bar-Hillel, who was working in the Iield oI automatic language translation (which,
as I mentioned in the previous chapter, also developed during Cold War with the promise to
automate the surveillance and analysis oI Soviet military communication and media), extended
inIormation theory to deIine and measure "semantic inIormation" oI the propositions in a
language system. In the same years, the Shannon-Weaver model (Iig. 18) was adopted in the
U.S. by the growing Iield oI mass communication studies as the theoretical Ioundation oI the
Iield. Weaver's hope that the model, originally designed to deal solely with the technical aspects
oI telecommunication, could be applied to describe semantic and pragmatic levels as well, was
Iully realized: the model was adopted to describe the process oI mass communication: sender
(Iilm studio, TV station, publisher) -- message (a Iilm, a television program, a newspaper story) -
- receiver (Iilm viewer, television viewer, newspaper reader).
278


276
Ibid., 97.
277
Ibid., 95.
278
Wilbur Schramm, "How Communication Works," in The Process and EIIects oI Mass
Communication (Urbana: University oI Illinois Press: 1954): 3-26.

In this transIer, the basic postulate oI the inIormation theory became an ideology.
Shannon begins his exposition oI inIormation theory by stating that "the Iundamental problem oI
communication is that oI reproducing at one point either exactly or approximately a message
selected at another point."
279
American scholars oI mass communication applied the same
postulate to the semantic level. Even a textbook on mass communication published in the 1970s,
still states: "iI the meaning oI the destination is isomorphic with the meaning oI the source which
originated the act, then communication can be said to have taken place."
280
With such a
deIinition, any discrepancy between the codes oI a sender and a receiver becomes undesirable
"noise." As Stuart Hall pointed out, the ideology lies in assuming that encoding and decoding
codes are or should be the same: "What are called 'distortions' or 'misunderstandings' arise
precisely Irom the lack oI equivalence between the two sides in the communication
exchange."
281
Thus, while American social scientists contrasted American democracy with
Soviet totalitarianism, they simultaneously adopted the theoretical model according to which
mass communication was synonymous with Iollowing the prescribed meaning. In short,
communication was deIined as control.

Despite the Iact that modern mass communication crucially depends on the visual channel,
American scholars oI mass communication never seriously conIronted its visual aspects. Others,
however, have tried to use inIormation theory to understand visual communication. I will next

279
Shannon and Weaver, The Mathematical Theory oI Communication, 3. Shannon includes
"approximately" because one oI the main concerns oI a communication engineer is to increase
the eIIiciency oI a communication system by making it transmit only the inIormation which is
absolutely essential Ior the correct identiIication oI a message by the receiver. For instance, the
telephone does not carry the whole range oI Irequencies oI the human voice but only those
suIIicient Ior understanding what is said.
280
Melvin L. De Fleur and Sandra Ball-Rokeach, Theories oI Mass Communication (New
York: Longman, 1975), 127.
281
Stuart Hall, "Encoding/Decoding," in Culture, Media, Language, ed. Stuart Hall et al.
(Hatchinson, 1980), 131.

discuss these attempts which took place in the late 1950s and in the 1960s in semiotics,
experimental aesthetics, and art history.
In the early 1960s Roland Barthes published The Photographic Message and Rhetoric oI
the Image where he applied a semiotic apparatus to the analysis oI mass media images, in his
case, publicity photographs.
282
However, it may be surprising to discover that The Photographic
Message opens by Iormulating the situation oI mass communication in terms oI inIormation
theory, exactly in the same way as it was done in communication studies:

The press photograph is a message. Considered overall this message is Iormed by a source oI
emission, a channel oI transmission and a point oI reception. The source oI emission is the staII oI
the newspaper, the group oI technicians certain oI whom take the photo, some oI whom choose,
compose and treat it, while others, Iinally, give it a title, a caption and a commentary. The point oI
reception is the public which reads the paper. As Ior the channel oI transmission, this is the
newspaper itselI...

Barthes, however, was not going to propose, as communication researchers did, that the
system oI mass communication also contains a Ieedback mechanism -- the right oI readers to
write letters to newspapers with complaints or suggestions -- thus assuring that mass
communication truly serves the purposes oI Democracy and Free Speech, rather than
unquestioned ideology. In Iact, Barthes' whole purpose in his articles on the semiotics oI mass
media images was to reveal the mechanisms by which images communicate or subvert an
ideological meaning. Communicate or subvert: Barthes' position oscillates between these two
mutually exclusive positions. On the one hand, Barthes proposes in The Rhetoric oI the Image
that the speciIicity oI a photograph's codes is what allows it to naturalize the ideological message
in a particularly transparent manner; on the other hand, in the same article he also claims that an
image is essentially polysemous and thereIore a caption is required to anchor a single meaning.
In a later article The Third Meaning Barthes makes an even stronger claim Ior the subversive

282
Roland Barthes, (1961), "The Photographic Message," in Image, Music, Text, ed. Stephen
Heath (New York: Hill and Wang, 1977); "Rhetoric oI the Image," (1964), in Image, Music,
Text.

nature oI the visual code in relation to both ideology and interpretation by suggesting that a Iilm
still contains an excess, an "obtuse meaning" not reducible to its "inIormational" or "symbolic"
meanings.
283

The code. More than any other, it was this particular concept oI inIormation theory that
entered semiotics in the 1960s. And Ior semioticians concerned with visual signs, it became the
main preoccupation oI their research: is there a unique visual code, an autonomous "language oI
pictures," and iI so, what is its speciIicity? Now, aIter we have Iollowed the development oI
inIormation theory, we can see the historical speciIicity oI approaching vision as a code by
Barthes, J.M. Floch, Felix ThYrlemann, the members oI the Groupe Mu, Fernande Saint-Martin,
and other semioticians.
284
The notion oI the code, as inIormation theory itselI, has emerged in
response to the practical problem Iaced by engineers: how to make telecommunication systems
more eIIicient and how to measure this eIIiciency. Encoding is one oI the principal ways to
achieve this eIIiciency. Morse code condensed transmitted messages by associating the shortest
combination oI dots and dashes with the most Irequently used letters. Later, communication
engineers were able to achieve even greater economy by encoding only as much inIormation as
necessary Ior the receiver to decode the message with suIIicient accuracy.
285
When carried over
to visual semiotics, the notion oI the code retained its original engineering meaning, its
association with the issue oI eIIiciency. Indeed, to ask what kind oI inIormation can be obtained
through seeing and/or represented as images; how visual codes diIIer Irom other kinds oI codes;
whether visual representations have any unique qualities in terms oI their eIIect on the viewer --
in short, to propose the uniqueness oI the visual code or to deny this uniqueness is already to

283
Roland Barthes, (1970), "The Third Meaning," in Heath, Image, Music, Text.
284
For a critical discussion oI visual semiotics, see Gsran Sonesson, Pictorial Concepts.
Inquiries into the Semiotic Heritage and its Relevance Ior the Analysis oI the Visual World
(Lund, Sweden: Lund University Press, 1989).
285
A principal contribution oI Shannon was his Iamous Sampling Theorem which speciIied the
absolute minimum oI the signal which needs to be retained beIore inIormation is irreversibly
lost.

conceive oI vision as productive, as potentially a special way oI transmitting inIormation, to
conceive oI it as a communication engineer would.
286

The semioticians did not go as Iar as to apply the mathematical apparatus oI inIormation
theory to the analysis oI visual signs. However, the Iield oI experimental aesthetics, which, Irom
its origins in the nineteenth century always tried to reduce the eIIect oI an image to a
mathematical Iormula, welcomed the quantitative aspect oI the theory. But how to apply a theory
developed Ior the purpose oI engineering communication to a very diIIerent question oI aesthetic
pleasure? Experimental psychologists interested in aesthetics assumed that a measure oI
inIormation in an image could represent its aesthetic eIIect. In the 1960s the German aesthetician
Max Bense developed a theory oI "inIormation aesthetics" and tried to measure the
"inIormation" contained in such classical images as Rembrandt's etchings. It was not at all
obvious, however, how to measure the "inIormation" in an etching or a painting, so other
researchers went on to construct simple abstract patterns where "inIormation" was easier to
quantiIy. They reasoned that since the amount oI inIormation in a message is directly
proportional to its unpredictability and inversely proportional to its redundancy, a pattern made
Irom small squares, where the tone oI each square is determined randomly, would be the most
unpredictable and thereIore would contain the most inIormation. On the contrary, in a pattern
like Mondrian's or Malevich's painting there is much more redundancy (more predictability) and
consequently, the amount oI inIormation is low. In a number oI experiments, subjects rated their
"aesthetic pleasure" when looking at diIIerent patterns (Iig. 19); psychologists then tried to

286
This carrying over oI assumptions oI inIormation theory into semiotics reaches its extreme in
Jurij Lotman's Structure oI the Artistic Text (1970). Lotman was strongly inIluenced by
cybernetics which in the 1960s was more popular in the Soviet Union than it ever was in the
West and which became as important Ior Soviet semioticians as Saussure or Peirce. (In part, it
was used to justiIy semiotics in the eyes oI the authorities by presenting it as a part oI
cybernetics, concerned with the study oI diIIerent artiIicial languages, Irom computer languages
to the languages oI art.) Lotman claimed that the artistic text is a message coded in a particular
way and that art is the most economical way to transmit inIormation because oI the unique
Ieatures oI the artistic code. Jurij Lotman, (1970), The Structure oI the Artistic Text (Ann
Arbour: Department oI Slavic Languages and Literatures, The University oI Michigan, 1977).

determine the level oI visual organization that was biologically most pleasurable.
287

Psychologists, turned aestheticians. Aestheticians, turned communication engineers.
Finally, one oI the most inIluential texts oI post World War II art history is also heavily
indebted to inIormation theory.
288
In Art and Illusion (1960) Ernst Gombrich provides a
revealing analysis oI Greek vases and mosaics as precursors to modern digital codes; he
compares the reading oI pictures to the decipherment oI codes; and he uses various concepts oI
inIormation theory throughout the book.
289
However, Gombrich does more than simply pay
tribute to a Iashionable theory. During the war, Gombrich was employed Ior six years by the
British Broadcasting Corporation to listen Ior radio transmissions.
290
He was conIronted daily
with the central problem oI modern inIormation theory -- the identiIication oI signal in the
presence oI noise. As he himselI admits, it was this experience which led to his deep interest in
inIormation theory as a paradigm Ior understanding perception.
Rather than being accidental, the paradigm oI inIormation theory plays a central role in
Art and Illusion. One oI Gombrich's key concepts is "the beholder's share": the importance oI
inIerence in interpreting the image. According to Gombrich, a representational image is
essentially incomplete.
291
As a representation oI three-dimensional space, a two-dimensional

287
See, Ior instance, D.E. Berlyne, "Novelty, Complexity, and Hedonic Value," Perception and
Psychophysics 8 (1970): 279-86; D. DorIman and H. McKenna, "Pattern PreIerence as a
Function oI Pattern Uncertainty," Canadian Journal oI Psychology 20 (1966): 143-53; P.C. Vitz,
"PreIerence Ior DiIIerent Amounts oI Visual Complexity," Behavioral Science 2 (1966): 105-14.
It is interesting to compare this work in experimental aesthetics with Lotman's view oI art as
communication (see Iootnote 278). Lotman proposed that a unique Ieature oI art is that it carries
more inIormation than any other kind oI communication, while experimental aestheticians
proceeded under the assumption that there is some optimal level between the complete lack oI
inIormation and the complete saturation by it which characterizes the most pleasurable works oI
art.
288
I am grateIul to Michael Ann Holly and Norman Bryson who brought to my attention the
relevance oI Art and Illusion to the problematic oI this chapter.
289
E.H. Gombrich, Art and Illusion (Princeton: Princeton University Press, 1960), 39-40, 88,
205.
290
Ibid., 204.
291
Ibid., 208, 211.

image is always ambiguous, potentially corresponding to numerous spatial conIigurations. It is
equally ambiguous as a representation oI other aspects oI visual reality, substituting a Iew brush
strokes Ior a Iield oI grass, a Iew spots oI color Ior a human Iorm. The job oI a beholder is to
supplement the partial inIormation in the image with her or his own projection. In other words,
Gombrich understands visual communication in terms oI inIormation theory: as the decoding oI
a signal in the presence oI noise. According to inIormation theory, this decoding is always
probabilistic: there can never be the absolute certainty than an original signal is correctly
reconstructed. Similarly, Gombrich describes interpretation as the process oI Iorming and
rejecting hypotheses, a process which is never Iinished.
For an engineer, this uncertainty oI communication is a problem to be solved. Gombrich,
however, interprets it as the source oI our aesthetic pleasure. We derive pleasure Irom art
precisely because in trying to "identiIy a signal" we have to use our imagination. There is an
unexpected agreement here between Gombrich and experimental aestheticians oI the 1960s, such
as Bense. Both start Irom the assumption that the aesthetic pleasure is proportional to the amount
oI inIormation in a message, i.e. its unpredictability. As Gombrich explains it, "the greater the
probability oI a symbol's occurrence in any given situation, the smaller will be its inIormation
content."
292
Because oI their ambiguity, representational images in Western art carry signiIicant
inIormation -- and, correspondingly, aesthetic pleasure.

Mass communications, visual semiotics, and aesthetics are just some oI the Iields that adopted
concepts oI inIormation theory in order to understand visual communication. The avant-garde oI
the 1920s, which dreamed oI engineering visual communication, would welcome all these
applications oI inIormation theory. The theory, which was developed by a communication
engineer in response to the growth oI television, was now used to rationalize the idea that mass

292
Ibid., 205.

communication is synonymous with control, and that to understand is to decode an intended
meaning (mass communication studies). The theory, designed to measure only the eIIiciency oI
telecommunications networks, was now adopted to describe all other levels oI communication as
well. Visual semiotics and experimental aesthetics extended inIormation theory to the levels oI
meaning and eIIect, the Iormer addressing the speciIicity oI visual signiIication, the latter
attempting to quantiIy the eIIect oI an image by measuring its "inIormation" content.
However, it was the Iield oI experimental psychology that was most transIormed by
inIormation theory. AIter World War II, two crucial developments took place. The Iirst is that
psychologists, by adopting the approach oI inIormation theory, essentially became engineers
who approach human vision as a communication system, studying its channel capacity,
bandwidth, and other engineering characteristics. The second is that work, more and more,
became a matter oI the mental processing oI visually presented inIormation. And this made
experimental psychology, Ior the Iirst time since the discipline emerged, practically important; in
Iact, as important as communication engineering. In the next three sections we will see how
these developments came about and what their impact was on modern understanding oI vision.



Why was it that even though the essential ideas oI inIormation theory were articulated in the
1920s, it was only in the late 1940s, aIter the publication oI The Mathematical Theory oI
Communication, that the theory was suddenly widely accepted in so many Iields? Is there a
relationship between this acceptance and the social and economic changes which were taking
place in modern society aIter World War II? The best way to answer these questions is to
compare the role oI inIormation theory in the second halI oI the twentieth century to the role
4. From "Human Motor" to "Human Information Processing"

previously occupied by thermodynamics.
First, a historical parallel. Hermann von Helmholtz Iormulated the universal law oI the
conservation oI energy in 1847. Exactly a century later, the theory oI inIormation in its modern
Iorm was developed by Shannon and Wiener. Helmholtz was instrumental in turning the theory
oI thermodynamics into a more general paradigm which treated the work oI nature, the work oI
machines, and human labor on the same terms. Similarly, Wiener extrapolated the concepts oI
inIormation theory and the techniques oI control engineering into the science oI cybernetics
which described the organization and behavior oI human, machine, and society using a single set
oI terms as well -- inIormation, Ieedback, and entropy.
The concept oI entropy, in Iact, directly connects thermodynamics and inIormation
theory. In 1929 the physicist Leo Szilard identiIied entropy with inIormation, and the measure oI
inIormation with the negative oI the measure oI entropy.
293
Entropy is the central notion oI
thermodynamics, reIerring to the tendency oI a closed system to change Irom an organized,
diIIerentiated, and less probable state to a chaotic, undiIIerentiated, and more probable state.
Since entropy is the measure oI randomness, i.e. the unpredictability oI a thermodynamic system,
and since inIormation similarly measures the unpredictability oI a message, the equations Ior
entropy and inIormation are the same. As Weaver explains, "Ior a communication source one can
say, just as he would also say it oI a thermodynamic ensemble, 'This situation is highly
organized, it is not characterized by a large degree oI randomness or oI choice -- that is to say,
the inIormation (or the entropy) is low.'"
294

But beyond these connections, there is a more crucial parallel. Just as thermodynamics
provided the model Ior understanding human work in the latter nineteenth and Iirst halI oI the

293
Howard Resnikov, The Illusion oI Reality (New York: Springer-Verlag New York Inc.,
1989), 5.
294
Shannon and Weaver, The Mathematical Theory oI Communication, 103.

twentieth century, inIormation theory has been crucial Ior a new post-industrial model oI work,
which gradually emerged aIter 1940.

In The Human Motor: Energy, Fatigue and the Origins oI Modernity Anson Rabinbach
demonstrated how the scientiIic ideas oI thermodynamics, Iormulated in the middle oI the
nineteenth century, became central Ior the conception oI work in modernity. Helmholtz, who
discovered the law oI the conservation oI energy, promoted this law as the universal principle
which equally applies to nature, machines, and humans. Helmholtz "portrayed the movements oI
the planets, the Iorces oI nature, the productive Iorce oI machines, and oI course, human labor
power as examples oI the principle oI conservation oI energy."
295
All work was understood as
the expenditure oI energy, with a crucial consequence oI redeIining human labor as labor power,
the expenditure oI the energy oI a body. Thus a worker was redeIined as a "human motor." This,
in turn, lead to the emergence, towards the end oI the century, oI the movement which
Rabinbach calls the European science oI work, "the search Ior the precise laws oI muscles,
nerves, and the eIIicient expenditure oI energy centered on the physiology oI labor."
296
In
manual labor, the energy stored in the body where it was accumulated through the intake oI Iood,
sleep, and rest is transIerred into muscular Iorce -- hammerer striking a blow, Iiler Iiling a
machine part, and so on. ThereIore, psychologists, physiologists and industrial experts searched
Ior methods to maximize both the accumulation oI a worker's energy (through proper nutrition,
shorter working hours, appropriate breaks) and its expenditure in labor. Just as an engineer
designing an engine was concerned with the most eIIicient transIer oI Iuel energy into
movement, European work experts aimed to maximize worker eIIiciency and to eliminate
possible waste. Central to the quest Ior the eIIiciency oI the human motor was the struggle

295
Anson Rabinbach, The Human Motor: Energy, Fatigue, and the Origins oI Modernity (Basic
Books, Inc., 1990), 3.
296
Ibid., 10.

against Iatigue, understood as the equivalent oI entropy. "As entropy revealed the loss oI energy
involved in any transIer oI Iorce, so Iatigue revealed the loss oI energy in the conservation oI
KraIt to socially useIul production. As energy was the transcendental, 'objective' Iorce in nature,
Iatigue became the objective nemesis oI a society Iounded on labor power."
297

The European science oI work may appear to be very similar to the American scientiIic
management movement pioneered by Frederick Winslow Taylor, a Iormer engineer turned
management consultant. As a part oI his program, Taylor aimed to minimize and standardize the
time required by a worker to perIorm each operation. He employed the method oI time studies
whereby the best workers were timed and the results became the norm to be Iollowed by the
rest.
298
Later, Frank and Lilian Gilberts (he -- an engineer, she -- a psychologist) popularized
another method oI motion study.
299
They argued that maximizing worker productivity is best
achieved by the elimination oI unnecessary movements and making the necessary more eIIicient.
Although both time and motion studies and the European science oI work were concerned with
the eIIiciency oI manual work, there was a Iundamental diIIerence between the two
approaches.
300
Taylorism aimed Ior maximum productivity, and had no concern Ior the
exhaustion and deterioration oI the human motor. In contrast, European scientists aimed Ior
optimum productivity, and thereIore were concerned not only with the rationalization oI the
workplace, but also with the workers' health, nutrition, saIety, and the optimal length oI a
workday. In short, Taylorism had no reservations about replacing one exhausted human motor
with another -- the philosophy which in the U.S. seems to go hand in hand with the emerging
ethics oI the consumer society and with immigration policies which assured the constant supply

297
Ibid., 68.
298
Frederick Winslow Taylor, The Principles oI ScientiIic Managment (New York, 1967).
299
William R. Spriegel and Clark E. Myers, eds., The Writings oI the Gilbreths (Homewood,
IL., 1953). I am grateIul to Lisa Cartwright Ior introducing me to the work oI the Gilbreths and
its relevance Ior the history oI modern vision.
300
Rabinbach, The Human Motor, 117, 277.

oI a cheap labor Iorce. Europeans, on the other hand, were committed to caring Ior and servicing
the human motor. The two paradigms converged aIter World War I, when European
industrialists partly adopted the more brutal, but ultimately more eIIective Taylorist methods,
while U.S. managment experts became more sensitive to workers' physiology and psychology.
Taylorism reduced the worker's body to a mechanical machine and had no concern Ior
her or his mind. Indeed, as Marta Braun points out, Taylorism aimed to systematically rob the
worker oI any degree oI independence or even understanding oI the overall work process by
"separating responsibility Ior the execution oI work Irom its planning or conception."
301
This
disdain Ior the mind was shared by behaviorism, which matured at the same time as the
European science oI work and Taylorism, and which equally well characterizes the imaginary oI
hard-edged social engineering oI the Iirst halI oI the twentieth century. In 1913, J.B. Watson, the
Iounder oI behaviorism, explicitly deIined it as the science oI social control: "Psychology as the
behaviorist views it is a purely objective experimental branch oI natural science. Its theoretical
goal is the prediction and control oI behavior."
302
Behaviorism approached the human subject as
an input-output system oI stimulus and response to be controlled through conditioning.
Concerned with controlling the body, it almost completely suppressed any studies oI perceptual
or mental processes between 1920 and 1950 in the U.S. It was a psychology well suited Ior
controlling the subject already reduced to the brainless human motor.




301
Marta Braun, Picturing Time: the Work oI Etienne-Jules Marey (1830-1904) (Chicago: The
University oI Chicago Press, 1992), 337.
302
Qtd. in Eliot Hearst, "One Hundred Years: Themes and Perspectives," in The First Century
oI Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979), 27. 5. Communication Engineer AnaIyzes Human Vision

AIter World War II, the reIerences to time and motion studies disappear. And by 1960, the
dominance oI behaviorism is over as well. Industrial society became post-industrial society.
In this shiIt, the concepts oI manual labor, production oI goods, and Iatigue were replaced
by new concepts oI cognitive labor, inIormation processing, and noise. Taylorism, Gilberts'
motion studies, and behaviorism gave way to engineering psychology, "human inIormation
processing," and cognitive science. In short, with the transIormation oI industrial society into
post-industrial society, the disciplines oI the eIIiciency oI the body were replaced by the
disciplines concerned with the eIIiciency oI the new instrument oI labor -- the mind.
One obvious and perhaps the earliest sign oI this transition was the speed with which
inIormation theory was disseminated into so many Iields by the end oI the 1940s and early
1950s. But, along with being a symptom oI post-industrial society which was yet to Iully
maniIest itselI, inIormation theory also provided actual techniques Ior deIining and studying the
human subject as an "inIormation processing unit," and its visual apparatus as a crucial part oI
"human inIormation processing." InIormation theory thereIore is central Ior understanding the
new ideas and techniques oI vision that emerged aIter the World War II.
The signiIicance oI the adaptation oI inIormation theory as a new model oI the human
observer, which took place in the 1950s, can be best seen through the Iollowing analogy:
Industrial society: manual labor, energy, Iatigue.
Post-industrial society: mental labor, inIormation processing, noise.

1. Manual versus mental labor. The point is not whether corporeal labor was indeed
universally displaced by mental labor: this is diIIerent Irom country to country, Irom industry to
industry. What is important is that the obsession with the rationalization oI corporeal work
(Taylorism, European science oI work, psychotechnics) disappeared, displaced by new obsession
with the rationalization oI the mind (cognitive psychology, artiIicial intelligence, cognitive

engineering). Regardless oI the percentage oI the work Iorce that still may be engaged in manual
labor, society is no longer concerned with spending more intellectual resources to perIect
workers' movements. What comes under scrutiny since the 1950s, when cognitive psychology
begins to displace then dominant behaviorism, are mental Iunctions: perception, attention, text
comprehension, memory, and problem solving.
This replacement oI manual work by cognitive work is directly related to automation.
Already in 1961, in an inIluential study oI automation in French industry, Pierre Naville and his
Iellow sociologists had described the transition Irom the "work oI the laborer to the work oI
communication," work which became primarily "cognitive or semiotic."
303
In his summary oI
this study Rabinbach writes, "The appearance oI the cerebral worker whose material and product
is 'inIormation' is emblematic oI the vast distance traversed between the worker who surveys
complex technologies oI communication and the 'man-beeI' oI Taylor."
304
It is important to note
that automation does not lead to the replacement oI human by machine. Rather, the worker's role
becomes one oI monitoring and regulation: watching displays, analyzing incoming inIormation,
making decisions, and operating controls. And it is the corresponding human Iunctions oI
perception, attention, memory, and problem solving which become the subject oI research by
new cognitive sciences.
What Taylor's scientiIic management was Ior the age oI industrialization, cognitive
sciences are Ior the age oI automation. In the 1940s, Herbert Simon worked on theories oI
management, the Iield oI research originated by Taylor. Having recognized the increasing
importance oI mental skills in the corporate workplace, Simon became one oI the pioneers oI
cognitive science with his work on automatic reasoning by computer. In 1964 he wrote that "the

303
Qtd. in Rabinbach, The Human Motor, 298.
304
Ibid., 298.

bulk oI productive wealth consists oI programs...stored in human minds."
305
Another pioneer oI
cognitive science was Jerome Bruner. ReIlecting back on his work in the 1950s, he noted in
1983: "It seems plain to me now that the 'cognitive revolution'...was a response to the
technological demands oI the 'post-industrial revolution.' You cannot properly conceive oI
managing a complex world oI inIormation without a workable concept oI mind."
306


2. Just as thermodynamics in the nineteenth century provided a model Ior the laboring
body, inIormation theory provided a new model Ior the mental processes, becoming particularly
inIluential in the research on human senses, including vision.
In 1958 the English psychologist D.E. Broadbent published Perception and
Communication, which was the Iirst attempt to understand human sensory-motor perIormance as
inIormation processing, or, as he put it, to describe it "in terms originally developed Ior
telephone engineering."
307
This statement may at Iirst remind us oI the comparison between
nerves and telegraphy, Irequently evoked by nineteenth century physiologists and psychologists,
including Helmholtz.
308
Yet, the approach oI Broadbent and other psychologists who Iollowed
him was Iundamentally new. It was less concerned with human physiology than with the limits
oI human perIormance understood as a communication system. In other words, just as a modern
telephone engineer is concerned with the capacity oI the telephone system, a psychologist is
concerned with the amount oI visual or auditory inIormation that can be adequately processed by
a human system; sensed, recognized, stored in memory, recalled, acted upon. What Ior
Helmholtz was a metaphor became a systematic program oI research a century later.

305
Qtd. in Douglas Noble, "Mental Materiel: The Militarization oI Learning and Intelligence in
U.S. Education," in Cyborg Worlds: the Military InIormation Society, ed. Les Levidov and
Kevin Robins (London: Free Association Books, 1989), 34.
306
Qtd. in Ibid., 34-35.
307
Broadbent, Perception and Communication, 36.
308
Jonathan Crary, Techniques oI the Observer: on Vision and Modernity in the Nineteenth
Century (Cambridge: The MIT Press, 1990), 93-94.

For a communication engineer, inIormation theory provides both a standard set oI terms
to speciIy quantitatively the perIormance oI a communication system and a set oI principles
which speciIy the limits oI this perIormance. For instance, the capacity oI a system depends on
the code used. Morse code makes transmitted messages shorter by associating the most
Irequently used letters with the shortest dot--dash symbols. The time saved in transmitting,
however, is oIIset by the additional time now required to encode and decode the message.
Another example is the principle, already established by Hartley, that the maximum amount oI
inIormation a system can transmit is the product oI its bandwidth and the time available Ior
transmission. In the 1920s, it was possible to transmit images over telegraph wires -- a small
bandwidth system -- by sacriIicing time.
In addition to the adoption oI the terms oI inIormation theory, such as bandwidth,
inIormation capacity, and Iilter, psychologists also applied its principles to understanding the
limits oI human perIormance. One oI Broadbent's contributions was the idea that the nervous
system cannot process all oI the sensory stimulation which may be available and that there is "a
Iilter at the entrance to the nervous system which will pass some classes oI stimuli but not
others"
309
(Iig. 20). In other words, because oI its limited bandwidth, a nervous system can only
process a certain amount oI inIormation in a given unit oI time. How much can be processed
depends on the nature oI this inIormation since diIIerent human senses such as vision and
hearing, now understood as separate communication channels, have diIIerent inIormation
capacities.
Once Iiltered through the senses, the inIormation is Iurther processed through a series oI
stages. Each stage is understood as a speciIic communication system with its own capacities
determined by its codes and bandwidth. Figure 21, taken Irom 1986 Handbook oI Perception and

309
Broadbent, Perception and Communication, 42.

PerIormance, summarizes the current knowledge oI the temporal capacities oI diIIerent stages oI
sensory and mental processing.
So just as the productivity oI a laboring body was limited by the available energy, it is
now assumed that the productivity oI the mind is limited by its inIormation-processing capacity.
In Iact, the notion oI a limited reservoir oI the energy in the body has Iound a direct correlatation
in the notion oI there being a limited amount oI human inIormation-processing resources which
can be used in diIIerent ways to deal with the incoming inIormation. For instance, the subject
may concentrate all his/her resources on processing visual inIormation, thus sacriIicing his or her
attending to auditory stimuli.

3. Fatigue versus noise. Just as Iatigue is the opposite oI energy, the limit to the body's
productivity, noise, always present in a communication system, whether natural or man-made, is
the limit to perIect communication and, consequently, the limit to Iaultless human inIormation
processing. Noise is the source oI error. And error is the worst enemy oI the human-machine
system in post-industrial cybernetic society.
The late nineteenth century science oI work came about as a struggle against Iatigue, this
"subversive element in the human motor, the body's IiIth column...Iatigue was the physiological
limit oI even the most perIectly executed work, the horizon oI the metaphor oI the human
motor."
310
As physiologists and psychologists were preoccupied with Iighting the body's
Iatigue, cognitive psychologists became preoccupied with preventing human cognitive and
perceptual errors. A textbook on human Iactors begins a chapter on error by optimistically
asking: "Are errors a basic part oI human nature that cannot be changed, or is there some way to
eliminate or reduce error and the damage errors cause?" The experts on human Iactors classiIied

310
Rabinbach, The Human Motor, 118.

various types oI human errors, came up with mathematical Iormulas to calculate human
reliability, and compiled tables oI Human Error Probability (HEP) Ior diIIerent tasks.
311

What are the sources oI human error? In contrast to a manual worker oI the industrial
age, an operator in a human-machine system is primarily engaged in the observation oI displays
which present inIormation in real time about the changing status oI a system or an environment,
real or virtual: a radar screen tracking a surrounding space (see chapter 3); a computer screen
updating the prices oI stocks; a video screen oI a computer game presenting an imaginary
battleIield; a control panel oI an automobile showing its speed, etc.
312
From time to time, some
inIormation causes an operator to make a decision and to intervene in the system's operation: tell
the computer to track an enemy bomber noticed on the radar screen; buy or sell a stock; press a
joystick; change gears. It is not essential that in some situations these interventions may be
required every second (a pilot engaged with an enemy, a computer game player, a Iinancial
analysis monitoring stock prices), while in others they are needed very rarely (a technician
monitoring an automated plant, power station, a nuclear reactor; a radar operator monitoring a
radar screen, waiting Ior potential enemy planes). This is why a human-machine system is
deIined as "an equipment system, in which at least one oI the components is a human being who
interacts with or intervenes in the operation oI the machine components oI the system Irom time
to time."
313

In a Iamous passage Walter Benjamin characterized modern experience as a constant
periodic rhythm oI perceptual shocks; the experience shared by an assembly line worker, by a

311
Barry H. Kantowitz and Robert D. Sorkin, Human Factors: Understanding People-System
Relationships (John Wiley & Sons: 1983), 30-57.
312
A 1965 textbook on human-machine systems calls an automobile "a Iirst rate example oI a
true man-machine system...a highly complex system in which the operator plays a commanding
role or actively intervenes in the system Irom time to time." Alphonse Chapanis, Man-Machine
Engineering (Bemont, CA: Wadsworth Publishing Company, Inc., 1965), 16.
313
Chapanis, Man-Machine Engineering, 16. Emphasis mine -- L.M.

pedestrian, and by a Iilm viewer.
314
This experience, characteristic oI modernization, Iinds a
direct continuation in one type oI cybernetic workplace: the constant overwhelming amount oI
inIormation; the constant cascade oI cognitive shocks which require immediate interventions (a
pilot engaged with an enemy, a player oI a computer game). Now, however, these shocks arrive
exclusively through the visual channel (dials, computer screen, head-mounted display). Thus oI
the roles mentioned by Benjamin, it is the Iilm viewer rather than the assembly line worker who
directly anticipates the experience oI an operator in this type oI human-machine situation.
However, there is also a second type oI work experience, new to post-industrial society: work as
waiting Ior something to happen. A radar operator waiting Ior a tiny dot to appear on the screen;
a technician monitoring an automated plant, power station, or nuclear reactor, knowing that a
soItware bug will eventually maniIest itselI, making a pointer on one oI numerous dials shoot
into the red...
II a manual worker is eventually overcome by Iatigue, an operator monitoring displays in
situations where nothing may happen Ior hours eventually loses vigilance. In terms oI the
inIormation processing approach, when a signal occurs inIrequently and randomly over time, an
operator is more likely to miss the signal.
315
This is one source oI human error. Another source

314
"Whereas Poe's passers-by cast glances in all directions which still appeared to be aimless,
today's pedestrians are obliged to do so in order to keep abreast oI traIIic signals. Thus
technology has subjected the human sensorium to a complex kind oI training. There came a day
when a new and urgent need Ior stimuli was met by the Iilm. In Iilm, perception in the Iorm oI
shocks was established as a Iormal principle. That which determines the rhythm oI production on
a conveyer belt is the basis oI the rhythm oI reception oI Iilm." Walter Benjamin, "On Some
Motives in Baudelaire," in Illuminations, ed. Hannah Arendt (New York: Schochen Books,
1969), 175.
315
In the extreme case oI sensory deprivation the subject becomes totally disoriented,
experiences illusions and even Iaints. It is interesting that in the 1950s, when automation was
making this new work-as-waiting-Ior-signal commonplace, experimental psychologists began to
conduct experiments in artiIicial sensory deprivation. In 1953 L.A. Riggs and his colleagues
discovered that iI the image is stabilized on the retina, thus eliminating miniature eye movements
normally accompanying vision, the image vanishes. In 1954-57, W. Heron constructed an
experimental cubicle to study the eIIects oI total perceptual isolation. Subjects, deprived oI all
visual, auditory, and tactile stimuli, experienced extreme boredom, restlessness and, eventually,
visual hallucinations. L.A. Riggs, et al., "The Disappearance oI Steadily Fixated Visual Tests

is visual illusions, limits to the reliability oI human vision, which preoccupied experimental
psychologists Irom the early days oI the discipline. Still another source is the inherent limitations
oI a human as a decision maker. When cognitive psychologists began to study human reasoning,
they made a grave discovery: human judgments do not Iollow statistical probabilities and ignore
inIormation about the prior probability oI an event.
316
So even iI the operator detects a signal in
time, s/he may still make a wrong decision.

Along with providing a model that allowed psychologists to conceive oI human vision as a
communication system where perIormance may be described in quantitative terms, inIormation
theory also provided a general concept to speciIy an inherent limit to this perIormance -- noise.
The incorporation oI the concept oI noise is what distinguishes modern inIormation theory, as
developed by Shannon, Wiener, KolmogoroII, and others, Irom the original Iormulations by
Hartley. In any real telecommunication system noise is always present due to the random motion
oI electrons in all electrical conductors. ThereIore, in practice, the signal at the receiving end is
always to some degree diIIerent Irom the signal originally sent. One oI Shannon's principal
contributions was an equation Ior determining the maximum capacity oI a communication
channel to transIer inIormation in the presence oI noise. As Colin Cherry notes, "This channel
capacity concept is somewhat analogous to the notion oI conservation oI energy; it is a deIinite
limit which no practical system can exceed, and its principal value is that it provides a standard
against which eIIiciencies may be assessed."
317

Just as entropy limits the perIormance oI any thermodynamic system (such as a motor),
leading to some oI the energy being lost unproductively, noise sets the limit on the eIIiciency oI

Objects," J. Optical Soc. America 43 (1953): 495; W. Heron, "The Pathology oI Boredom,"
ScientiIic American 196 (1957): 52-56.
316
Gillian Cohen, The Psychology oI Cognition, 2nd ed. (London and New York: Academic
Press, 1983), 190.
317
Cherry, On Human Communication, 52.

communication. And just as attempts to rationalize manual perIormance stumbled across the
problem oI Iatigue, the "physiological limit" to the eIIiciency oI work, psychologists recognized
noise as the inherent limit in the perIormance oI a human inIormation processing system. II the
source oI noise in a telecommunication apparatus is the random motion oI electrons, the source
oI noise in the nervous system is the random Iiring oI neurons. Always present, these random
Iirings are assumed to generate a small but constantly present level oI neural activity which
interIeres with human inIormation processing. The observer may miss a signal (Ior instance, a
Ilash oI light in a psychophysical experiment or a dot on a radar screen in a real situation) or, on
the contrary, detect a signal where none was present; or s/he may make a mistake in estimating
the signal's strength or its position, and so on.
In Iact, the noise in man-made components and biological noise become combined. Due
to the noise in its electronic parts, radar may display a dot when nothing is present or the location
oI the dot may be inaccurate. Additional noise is added by the imperIections oI a display, such as
glare, inadequate contrast, or placement. Finally, neural noise may cause the radar operator to
miss the dot, to misjudge its size and, consequently, either to miss an enemy plane or to send an
intercepting weapon towards a Iriendly aircraIt. In this way, the overall perIormance oI a human-
machine system, such as a radar installation, is equally limited by the perIormance oI its
electronic and human parts.



A radar operator? A human-machine system? Was not I talking about inIormation theory as a
paradigm Ior experimental psychology, a purely academic discipline? A discipline whose
practitioners, according to William Estes, "developed the tradition and the reputation oI being
6. Human Engineering

among the purest oI pure scientists" during its Iirst decades?
318
Practitioners who Ior decades
never leIt the seclusion oI the university laboratory while their colleagues -- physiologists,
Iatigue experts, behaviorists -- seemed to hold the knowledge necessary to maximize the
productivity oI industrial society?
The Iact that aIter 1950 experimental psychologists displaced their colleagues in
importance, oI course, Iits in with the larger shiIt Irom industrial to post-industrial society and
the new image oI work: visual and mental processing oI inIormation rather than corporeal labor.
The problem with such global explanations is that they explain things too well. It is easy to
connect the interest oI experimental psychologists in human inIormation processing to the shiIt
to post-industrial society. In Iact, it is as easy as to connect the very emergence oI experimental
psychology in the nineteenth century to the processes oI modernization, industrialization, or
rationalization. For instance, discussing Fechner's psychophysics, developed in the 1850s,
Jonathan Crary writes that Fechner's equations were "a means oI rendering a perceiver
manageable, predictable, productive and above all consonant with other areas oI
rationalization."
319
Similarly, in his discussion oI reaction time experiments, which became
standard in experimental psychology by the 1890s, Didier Deleule points out that "the measured
time is only the sign oI the living machine's capacity Ior increasing its productivity."
320

Both Crary and Deleule are right in seeing the social utility oI experimental psychology.
It should be also noted, however, that this social utility remained unIulIilled Ior many decades. In
Iact, the enormous amount oI knowledge accumulated by psychologists did not Iind its
systematic application until World War II.

318
William Estes, "Experimental Psychology: an Overview," in The First Century oI
Experimental Psychology, ed. Eliot Hearst (Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979), 629.
319
Crary, Techniques oI the Observer, 147.
320
Didier Deleule, "The Living Machine: Psychology as Organology," in Incorporations, ed.
Jonathan Crary and SanIord Kwinter (New York: Zone Books, 1992), 215.

The gradual expansion oI the practical applications oI experimental psychology provides
a precise map oI the new occupations and new conditions oI modern experience which call Ior
perceptual skills. During World War I, England, Germany, and France utilized experimental
psychologists to design and administer tests Ior aviation pilots, aeronautical, and airplane
observers, hydrophone operators, and submarine "listeners-in."
321
During peacetime, a number
oI psychologists published papers on the readability oI written text and oI highway signs and on
the visibility oI lights at sea.
322
However, in the industrial world which conceived oI the worker
as a human motor and was largely concerned with the productivity oI manual rather than
perceptual labor, these studies were an exception rather than the mainstream rule.
It was World War II which Iinally put to use the expertise oI experimental psychologists.
Why did this happen? The Iirst textbook on applied experimental psychology (1949) opens by
describing the recent origins oI the Iield:

For years experimental psychologists have worked diligently in academic laboratories studying
man's capacities to perceive, to work, and to learn. Only very slowly, however, have the Iacts and
methods which they have assembled been put to use in everyday liIe. A particularly glaring gap in
modern technology, both industrial and military, is the lack oI human engineering -- engineering
oI machines Ior human use and engineering oI human tasks Ior operating machines. Motion-and-
time engineers have been at work on many oI these problems, but the experimental psychologist is
also needed Ior his Iundamental knowledge oI human capacities and his methods oI measuring
human perIormance.
The recent war put the spotlight on this gap. The war needed, and produced, many
complex machines, and it taxed the resources oI both the designer and operator in making them
practical Ior human use. The war also brought together psychologists, physiologists, physicists,
design engineers, and motion-and-time engineers to solve some oI these problems. Though much
oI their work began too late to do any real good, it has continued on a rather large scale into the
piece.
Today, there are many groups busy with research on man-machine problems. They use
diIIerent names to describe the work in its various aspects: biotechnology, biomechanics,
psychoacoustics, human engineering, and systems research. Other names may be appropriate and
may appear in the Iuture. In casting about Ior a title Ior this book, we tried to select one that would
describe the subject matter without the restrictive connotations attaching to some oI the names

321
Morris Viteles, Industrial Psychology (New York: W.W. Norton & Company, Inc., 1932),
43.
322
Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook oI Experimental
Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1287-
1340.

mentioned above. Applied Experimental Psychology seems best to Iill these requirements, because
the traditional data and subject oI experimental psychology are Iundamental to this Iield.
323


Already beIore the war, experimental psychologists assisted in selecting military personnel Ior
such jobs as pilot or airplane observer by administering special aptitude tests. During the war, a
much greater number oI pilots, radar operators and other similar personnel became needed. The
emphasis was shiIted, thereIore, Irom selecting personnel with particularly good perceptual and
motor skills to designing the equipment (controls, radar screens, dials, warning lights) to match
the sensory capacities oI an average person.
324
And it was the Iield oI experimental psychology
that possessed the knowledge about the sensory capacities oI an average, statistical person: how
visibility and acuity vary between day and night; how the ability to distinguish colors and
brightness vary with illumination or distance; what the smallest amount oI light is which can be
reliably noticed; and so on.
325
All this data was now utilized Ior designing better displays and
controls oI the Iirst modern human-machine systems such as high-speed aircraIts or radar
installations.
The development oI these new human-machine systems during the war pushed human
perceptual and mental perIormance to the limit and this was the second reason why experimental
psychologists were called in. The perIormance oI a human-machine system was limited by
human inIormation capacity to process inIormation. In the words oI the authors oI Applied
Experimental Psychology,

We can make a machine that will do almost anything, given enough time and enough engineers.
But man has limits to his developments, at least as Iar as we can see it. When we think how much
a single radar can do in a small Iraction oI a second, and then realize by comparison that even the
simplest Iorm oI reaction Ior a human being requires about a IiIth oI a second, we realize what we
are up against... The Iull potential oI radar, Ior example, lagged Iar behind physical developments

323
Alphonse Chapanis, Wendell R. Garner, and CliIIord T. Morgan, Applied Experimental
Psychology (New York: John Wiley & Sons, Inc., 1949), v.
324
Ibid., 8.
325
Estes, "Experimental Psychology: an Overview," 630.

because human operators could not master the complex operation oI this machine system. We had
to worry about such things as a new kind oI visual signal -- very small and not very bright.
326


Considering that the authors described the work oI time-and-motion engineers as directly leading
to applied experimental psychology, this rhetoric can be expected. Taylor was impatient with the
limitations oI the body; now there was a similar impatience with the limitations oI human
inIormation processing. With Taylor, it was the question oI the speed oI muscular movements;
now, it became the question oI reaction time: the minimum time in milliseconds required Ior an
operator to detect a signal, to identiIy it, to press a control.
In order to measure normal human sensory capacities, experimental psychologists have
always put subjects in, so to speak, boundary conditions. They measured sensory thresholds,
such as the least amount oI light which can be detected. They also measured just noticeable
diIIerences (j.n.d.), the smallest diIIerence between two stimuli which can be detected. Finally,
they measured reaction times, the measure which became the main tool to deduce the time taken
by diIIerent mental processes. In order to measure these characteristics, a number oI standard
experiments were designed, and they remained largely unchanged Irom the times oI Weber,
Fechner, and Wundt. In a detection experiment, the task oI an observer is to detect the presence
oI barely visible stimuli, Ior instance a tiny light brieIly Ilashed in the dark (did I see
something?). In an identiIication experiment, the task is to identiIy which oI possible stimuli was
presented, Ior instance which oI two colors (which one did I see?). In a recognition experiment,
the task is to not only detect something, but to recognize what it is, Ior instance: what was the
shape that brieIly appeared (what did I see?)
During World War II, the radar operator, the anti-aircraIt gunner, the aircraIt pilot Iound
themselves in the same situations in which nineteenth century psychologists put their
experimental subjects. The setups oI psychophysical experiments became, in all details, the

326
Chapanis, Applied Experimental Psychology, 7-8.

conditions oI military work; the tasks devised by psychologists to study human vision became
the actual tasks Iaced by the operators oI human-machine systems. Like the subject oI a detection
experiment, a radar operator scans the radar screen Ior a barely noticeable dot oI light.
327
Like
the subject oI an identiIication experiment, he has to try to guess whether this dot is the same or
diIIerent Irom another dot which Irom his previous experience he knows to correspond to a
Iriendly airplane. An anti-aircraIt gunner is subjected to a recognition experiment, trying to
identiIy a plane by its shape. And all oI them, especially the pilot, are engaged in a sort oI
reaction time experiment.
Thus, nineteenth century psychophysical setups became the military, and soon, civilian
workplaces oI post-industrial society; Irom there, they traveled back into laboratories, leading to
such close interrelations between basic research in experimental psychology and its practical
applications that they were no longer separable.
328
The terms "applied experimental
psychology," "human engineering" and "man-machine engineering" were replaced by another
term standard today -- "human Iactors." The radar operator who in the 1940s and 1950s was the
prototypical example oI a human-machine system, was replaced by the 1980s by a new
prototypical Iigure, the computer user. Thus, reIerences to "human-machine systems" became
reIerences to "human-computer systems." The same amount oI intellectual energy and research
which in the middle oI the century went into theorizing the perIormance oI a radar operator and
adapting him and radar display to each other, today goes into the work on computer interIaces. In

327
As Paul Fitts notes in his 1951 overview oI engineering psychology, "radar operators are
oIten Iorced to search Ior weak signals at near-threshold levels." Fitts, "Engineering Psychology
and Equipment Design," 1290.
328
For example, a 1947 article in American Psychologist describes the work oI Naval Research
Laboratory as Iollowing these three directions: "the design oI gun Iire control and missile control
instruments Irom the point oI view oI ease and eIIiciency oI operation; the design and evaluation
oI synthetic gunnery and missile control trainers; and basic psychological research." But what is
meant here by "basic research"? We read that "at present, all basic research studies are aimed at
the eye-hand coordination problem involved in target tracking." "Target tracking" is just one
example oI a military task which traveled into a psychological laboratory, and gradually become
a standard psychophysical experiment. Franklin Taylor, "Psychology at the Naval Research
Laboratory," American Psychologist 2, no. 3 (1947): 87, 91.

retrospect then, we should recognize the radar operator as the central Iigure standing at the
origins oI post-industrial society, the Iigure which put directly into motion the new disciplines oI
the eIIiciency oI the mind: engineering psychology, human inIormation processing, and
cognitive science.
We are now in a position to see that the ready acceptance oI inIormation theory by
psychologists in the 1950s was not simply a part oI a "Zeitgeist" oI the coming inIormation age.
Rather it was a practical necessity. The 1986 Handbook oI Perception and Human PerIormance
describes the rationale Ior adopting the inIormation processing approach in psychology in this
way:

Because the language oI inIormation processing provides an objective and quantitative way oI
describing the basis oI human perIormance, it has proven useIul in applications. Indeed, much oI
the impetus Ior the development oI this kind oI empirical study stemmed Irom the desire to
integrate description oI the human within the overall systems. In the design oI cockpits, control
systems Ior weapons, and other military applications, it has been useIul to have descriptions oI the
speed, accuracy, and reliability oI the human in terms comparable to those used Ior other
mechanical parts oI the system.
329


As psychologists became involved in designing human-machine systems, they needed a language
to characterize human perIormance quantitatively. But not just any language. II the human is
conceived oI as part oI a human-machine system (Ior instance, in the role oI a radar operator), it
is convenient to use the same language to describe the perIormance, capacities, and errors oI
both the electronic and human components. Put diIIerently, iI the human-machine system is
conceived oI as a machine, and its perIormance is to be characterized in engineering terms, the
same terms need to be applied to its human components iI the perIormance oI the overall
machine is to be characterized at all. Since inIormation theory already provided a language to

329
Michael I. Posner, "Section V: InIormation Processing. Overview," in Handbook oI
Perception and Human PerIormance, ed. Kenneth BoII, Lloyd KauIman, and James P. Thomas
(New York: John Wiley and Sons, 1986), II: V-6.

characterize the perIormance oI electronic components, it was logical to apply it to the human
component as well.



Modernization brought with it a special discipline concerned with eIIiciency -- engineering. The
job oI an engineer was to ensure maximum perIormance with a minimum investment oI energy,
materials, and time, be it the perIormance oI machines (mechanical engineering), communication
systems (communication engineering) or human bodies (scientiIic management, time and motion
studies). Inspired by modern engineering, the avant-garde oI the 1920s tried to systematically
apply its principles to vision.
To engineer vision meant to eliminate waste, to use minimal material resources. Thus,
constructivist graphic design streamlined typography, eliminating complicated typeIaces in Iavor
oI block letters consisting oI straight lines; it also eliminated illustrations and "wasteIul"
decorations by making type itselI the main element oI design. The goal: maximum impact with
minimum use oI ink.
To engineer vision also meant to minimize the psycho-physical resources required oI the viewer.
Vertov writes in his Iamous 1923 maniIesto: "The least advantageous, the least economical
communication oI a scene is theatrical communication."
330
In contrast, montage Iorces the eye to see the
right thing at the right time, thus eliminating the visual waste oI theater, ballet, painting, and other
traditional Iorms. In montage, "camera drags the eyes oI a Iilm viewer Irom hands to legs, Irom legs to
eyes and the rest in the most advantageous order..."
331


330
Vertov, "Kinoki," 139.
331
Ibid., 139. Emphasis in the original -- L.M. 7. ConcIusion: the Labor of Perception

To engineer vision also meant to ensure perception in the shortest possible time. Here as
well, the avant-garde promoted montage as an example oI possible economy, in this case
economy oI time. Maud Lavin describes the 1930 maniIesto oI the group oI leading German
designers headed by Kurt Schwitters: "Walter Dexel writes that modern man has the right to
expect communications in the shortest possible time. Willi Baumeister points out that
photomontage is eIIicient, allowing Ior the quick grasp oI several images at once...Similarities
between photomontage and Iilm are oIten emphasized, with photomontage being considered a
quicker, more eIIicient medium."
332

Finally, to engineer vision also meant to be able to measure its eIIiciency, or, to use the language
oI a communication engineer, to measure "system perIormance." Eisenstein, Iresh Irom engineering
school, invented his Iirst theory oI artistic communication, the Iamous "montage oI attractions":
"...Laboratory analyses and diagrams...Mendeleev's Periodic Table and the laws oI Gay-Lussac and Boyle
Mariotte in the realm oI art!...Let us thereIore search Ior the unit which will measure the inIluence exerted
by art! Science has its "ions," its "electrons," its neutrons." Art will have -- attractions!"
333


In its desire to engineer vision, the avant-garde was ahead oI its time. The systematic engineering
oI vision took place only aIter World War II when vision become the major instrument oI labor,
the most productive organ oI a worker in a human-machine system. To ensure the maximum
perIormance oI such a system, it became necessary to engineer it around the capacities and the
limitations oI human vision. It also became necessary to understand vision in a new way: as
inIormation processing.

332
Maud Lavin, "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle oI
New Advertising Designers," in Montage and Modern LiIe: 1919-1942, ed. Matthew Teitelbaum
(Cambridge: The MIT Press, 1992), 54.
333
Qtd. in Jacques Aumont, Montage Eisenstein (London and Bloomington: BFI Publishing and
Indiana University Press, 1987), 41. Emphasis mine -- L.M.

This is the history oI the engineering oI vision Irom INKhUK to MIT: Irom the Soviet art
institute oI the 1920s which gathered the avant-garde artists united by the idea oI engineering
vision -- to the premier engineering school where much oI this engineering later occurred
through the work on inIormation theory, on radar and computer graphics, on new human-
machine interIaces, on algorithms oI human vision.


This dissertation begins at the scene oI heated debates at the Soviet art institutes in the
early 1920s. It ends with the recent research by cognitive scientists and engineers at MIT. What
justiIies this temporal continuity, on the one hand, and disciplinary shiIt, on the other hand --
Irom the Soviet artistic avant-garde to one oI the premier science institutions in the world, which
today is in the IoreIront oI research on robotics, artiIicial intelligence, and human-computer
interIaces? What can be in common between Soviet constructivism and American cognitive
science?
Both are the product oI the age oI engineering. Engineering, the quintessentially modern
proIession, appeared in the 1870s. The job oI an engineer was to ensure maximum perIormance -
- oI machines, human bodies or communication systems -- with minimum investment oI energy,
materials, and time. The engineer was the new specialist in the eIIiciency oI any productive
process, regardless oI its nature. And this ideology oI "engineerism" became the modern myth,
the religion oI a society aspiring to total rationalization.
334
In order to employ vision eIIiciently
in its new roles, whether as a medium oI propaganda and advertising or as the channel oI
communication between human and machine, artists, designers, and scientists adopted the
engineering approach as well. It is not surprising that Soviet constructivists, the pioneers oI
modern propaganda design, insisted on calling themselves engineers rather than artists.
It is also not surprising that the archeology oI twentieth century vision would uncover
numerous similar solutions to the problem oI its rationalization in such seemingly distant Iields
as Iilm and robotics, graphic design and experimental psychology. For instance, let us compare
two statements: one by a designer, the other by a psychologist. El Lissitsky wrote in the 1920s
regarding his new geometric style oI poster design: "The most unambiguous and immediately

334
Max Weber proposed rationalization as an umbrella concept to describe the social processes
oI modernity. See, Ior instance, From Max Weber: Essays in Sociology (New York: OxIord
University Press, 1958).
Max Weber, General Economic History (New York, 1927). ConcIusion

recognizable Iorms are geometric Iorms. No one will conIuse a rectangle with a circle, or a circle
with a triangle."
335
In 1951, the already mentioned experimental psychologist Paul Fitts echoed
practically the same words in his summary oI research on the use oI graphic symbols Ior human-
machine interIaces: "Geometrical Iigures diIIer considerably in legibility. Straight lines have
been Iound to be more legible than curved ones. The triangle, the rectangle, and the square have
been reported to be more easily recognized under conditions oI low illumination and in
peripheral vision than circular and hexagonal Iorms."
336
Both Lissitsky and Fitts were concerned
with the same problem -- reliability and speed in the recognition oI visual symbols -- and,
thereIore, it is logical that they recommended the same solution.
Whether Iilmmakers or psychologists, engineers or interIace designers, twentieth century
proIessionals have understood vision as a medium oI communication and have attempted to put
its use on a scientiIic basis. II in the early part oI the century this research Iocused on human
communication (the human as the subject oI mass propaganda and mass entertainment), aIter
World War II, the Iocus shiIted to human-machine interIace. Human vision became the key
instrument oI post-industrial labor as the channel oI communication between human and
machine.
This history can be summarized by three images. The Iirst image: a portrait oI Tatlin by
Lissitsky (Iig. 16). A compass, extending straight Irom Tatlin's eye, a metaphor oI vision Ior
work.
The second image, a popular icon in the late 1980s and early 1990s, promoted virtual
reality interIace designed at NASA/Ames Human Factors Research Center (Iig. 22).
337
Instead

335
Qtd. in Igor Golomstock, Totalitarian Art (New York: HarperCollins Publishers, 1990), 24.
336
Paul Fitts, "Engineering Psychology and Equipment Design," in Handbook oI Experimental
Psychology, ed. S.S. Stevens (New York and London: John Wiley & Sons, Inc., 1951), 1298.
337
On NASA/Ames virtual reality research in the 1980s, see Scott S. Fisher, "Virtual InterIace
Environments," in The Art oI Human-Computer InterIace Design, ed. Brenda Laurel (Reading,
Mass.: Addison-Wesley Publishing Company, 1990): 423-438.

oI the metaphor oI the eye-compass, a reality: video monitors strapped to the eyes. The notion oI
vision as work is now Iully realized: the operator wearing the gear works by mentaly processing
visually presented inIormation. The gear is designed using all the available knowledge
accumulated by experimental psychology about human vision. In the photograph we see the last
leItover Irom the age oI manual labor -- an arm in a DataGlove. It will soon disappear since
through gaze tracking the operator can control the system by merely looking at diIIerent points in
virtual space.
What is the third image which stands between these two? Let me quote Irom the
description oI the history oI engineering psychology Iound in an 1965 overview oI the Iield:

The primary emphasis in time-and-motion engineering has been on man as a worker; that is, as a
source oI mechanical power. It was not until World War II that a new category oI machines
appeared -- machines that made demands not upon the operator's muscular power, but upon his
sensory, perceptual, judgmental, and decision-making abilities. The job oI a radar operator, Ior
example, requires virtually no muscular eIIort, but makes severe demands on sensory capacity,
vigilance, and decision-making ability. This new class oI machines raised some intricate and
unusual questions about human abilities: How much inIormation can a man absorb Irom a radar
screen?
338


The World War II radar operator is this third image, a historical intermediary between Lissitsky's
man-compass (1920s) and NASA's virtual reality user (1980s). The Iigure oI a radar operator
stands at the gates to the post-industrial society oI perceptual labor, at the origins oI research on
human-machine interIace, and the understanding oI human vision as inIormation processing (Iig.
23).
"I am a mechanical eye," wrote Dziga Vertov in 1923.
339


338
Alphonse Chapanis, Man-Machine Engineering (Bemont, CA: Wadsworth Publishing
Company, Inc., 1965), 9-10.
339
Dziga Vertov, "Kinoki. Perevorot" (Kinoki. A revolution), LEF 3 (1923): 141.



Allison, Nicholas H., ed. Art Into LiIe: Russian Constructivism 1914-1932. New York: Rizzoli
International Publications, 1990.

Anderson, John Robert. Cognitive Psychology and Its Implications. W.H. Freeman and
Company, 1980.

ArgYelles, JosZ. Charles Henry and the Formation oI a Psychophysical Aesthetics. Chicago:
University oI Chicago Press, 1972.

Arnheim, RudolI. "A Plea Ior Visual Thinking." In New Essays on the Psychology oI Art. Los
Angeles and London: University oI CaliIornia Press, 1986.

Arnheim, RudolI. Art and Visual Perception. The New Version. Berkeley: University oI
CaliIornia Press, 1974.

Attneave, F. Applications oI InIormation Theory to Psychology. New York: Holt, 1959.

Aumont, Jacques. Montage Eisenstein. London and Bloomington: BFI Publishing and Indiana
University Press, 1987.

Barthes, Roland. "Rhetoric oI the Image." In Image, Music, Text, edited by Stephen Heath. New
York: Hill and Wang, 1977.

Barthes, Roland. "The Photographic Message." In Image, Music, Text, edited by Stephen Heath.
New York: Hill and Wang, 1977.

Barthes, Roland. "The Third Meaning." In Image, Music, Text, edited by Stephen Heath. New
York: Hill and Wang, 1977.

Baxandall, Michael. Painting and Experience in FiIteenth Century Italy. OxIord: OxIord
University Press, 1972.

Beliajeva, E.L. Arkhitekturno-prostranstvennaya sreda soroda kak obyekt zritelnogo vospriyatiya
(Architectural and spacial city environment as an object oI visual perception). Moscow:
Stroyizdat, 1977.

Benjamin, Walter. "On Some Motives in Baudelaire." In Illuminations, edited by Hannah
Arendt. New York: Schochen Books, 1969.

Bergin, Thomas Goddard and Max Harold Fisch. The New Science oI Giambattista Vico. Ithaca:
Cornell University Press, 1968.

Berlyne, D.E. "Novelty, Complexity, and Hedonic Value." Perception and Psychophysics 8
(1970): 279-86.

Borch-Jakobson, M. The Absolute Master. StanIord: StanIord University Press, 1991.

Braun, Marta. Picturing Time: the Work oI Etienne-Jules Marey (1830-1904). Chicago: The
University oI Chicago Press, 1992.

BibIiography

Brik, Osip. "T.n. 'Iormalnyi metod'" (So called 'Iormal method'). LEF 1 (1923): 214-215.

Broadbent, D.E. Perception and Communication. OxIord: Pergamon Press, 1958.

Bryson, Norman. Looking at the Overlooked: Four Essays on Still LiIe Painting. Cambridge:
Harvard University Press, 1990.

Buck-Morss, Susan. The Dialectics oI Seeing: Walter Benjamin and the Arcades Project.
Cambridge: The MIT Press, 1989.

Bulgakova, Olga. "Sergei Eisenshtein i ego 'psikhologicheskiy Berlin' -- mezhdu psikhoanalizom
i strukturnoy psikhologiey" (Sergei Eisenstein and his 'psychological Berlin': between
psychoanalysis and structural psychology). Kinovedcheskie Zapiski 2 (1988): 174-191.

Chapanis, Alphonse. Man-Machine Engineering. Bemont, CA: Wadsworth Publishing Company,
Inc., 1965.

Chapanis, Alphonse, Wendell R. Garner, and CliIIord T. Morgan. Applied Experimental
Psychology. New York: John Wiley & Sons, Inc., 1949.

Chen, Jack. Soviet Art and Artists. London: The Pilot Press, Ltd., 1945.

Cherry, Colin. On Human Communication. 2nd ed. Cambridge: The MIT Press, 1968.

Cohen, Gillian. The Psychology oI Cognition. 2nd ed. (London and New York: Academic Press,
1983).

Cohen, Paul R. and Edward A. Feigenbaum, eds. The Handbook oI ArtiIicial Intelligence. Los
Atlos, CA: William KauImann, Inc., 1982.

Cook, Richard. "Serious Entertainment." Computer Graphics World (May 1992): 40-48.

Crary, Jonathan. Techniques oI the Observer: on Vision and Modernity in the Nineteenth
Century. Cambridge: The MIT Press, 1990.

Damisch, Hubert. L'origine de la perspective. Paris: Flammarion, 1987.

Danziger, Kurt. Constructing the Subject: Historical Origins oI Psychological Research.
Cambridge: Cambridge University Press, 1990.

De Fleur, Melvin L. and Sandra Ball-Rokeach. Theories oI Mass Communication. New York:
Longman, 1975.

De Landa, Manuel. War in the Age oI Intelligent Machines. New York: Zone Books, 1991.

Deleule, Didier. "The Living Machine: Psychology as Organology." In Incorporations, edited by
Jonathan Crary and SanIord Kwinter. New York: Zone Books, 1992.

Descartes, RenZ. "Meditations on the First Philosophy," In The Rationalists. Garden City, NY:
Doubleday & Company, 1960.

DorIman, D. and H. McKenna. "Pattern PreIerence as a Function oI Pattern Uncertainty."
Canadian Journal oI Psychology 20 (1966): 143-53.


Druckrey, Timothy. "Revenge oI the Nerds. An Interview with Jaron Lanier." AIterimage (May
1991): 5-9.

Dudley, Andrew. Concepts in Film Theory. OxIord: OxIord University Press, 1984.

Dulac, Germaine. "Aesthetics, Obstacles, Integral CinZgraphie." In vol. 1, French Film Theory
and Criticism, edited by Richard Abel. Princeton: University oI Princeton Press, 1988.

Eames, Charles and Ray Eames. A Computer Perspective: Background to the Computer Age.
Cambridge: Harvard University Press, 1990.

Echoes oI War. Boston: WGBH Boston, n.d. Videotape.

Edgerton, Samuel. The Renaissance Rediscovery oI Linear Perspective. New York: Basic Books,
1975.

Eduardo, Kac. "Aspects oI the Aesthetics oI Telecommunications." In SIGGRAPH '92 Visual
Proceedings, edited by John Grimes and Gray Lorig. New York: The Association Ior Computing
Machinery, 1992.

Edwards, Paul, ed. Encyclopedia oI Philosophy. New York: Macmillan, 1967.

Eisenstein, Sergei. "A Dialectical Approach to Film Form." In Film Form: Essays in Film
Theory, edited by Jay Leyda. New York: Harcourt Brace and World, 1949.

Eisenstein, Sergei. "Montazh atraktsionov" (Montage oI attractions). LEF 3 (1923): 70-75.

Epstein, Jean. "On Certain Characteristics oI PhotogZnie." In vol. 1, French Film Theory and
Criticism, edited by Richard Abel. Princeton: University oI Princeton Press, 1988).

Estes, William. "Experimental Psychology: an Overview." In The First Century oI Experimental
Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers,
1979.

Farah, Martha. "Is Visual Imagery Really Visual? Overlooked Evidence Irom
Neuropsychology." Psychological Review 95, no. 3 (1988): 307-317.

Farocki, Harun. "Reality Would Have to Begin." Documents 1/2 (1992): 136-146.

Finke, Ronald A. Principles oI Mental Imagery. Cambridge: The MIT Press, 1989.

Fisher, Scott S. "Virtual InterIace Environments." In The Art oI Human-Computer InterIace
Design, edited by Brenda Laurel. Reading, Mass.: Addison-Wesley Publishing Company, 1990.

Fitts, Paul. "Engineering Psychology and Equipment Design." In Handbook oI Experimental
Psychology, edited by S.S. Stevens. New York and London: John Wiley & Sons, Inc., 1951.

Flew, Antony, ed. A Dictionary oI Philosophy. London: Pan Books Ltd., 1984.

Florensky, Pavel. "Obratnaya Perspektiva" (The Inverted Perspective). In vol. 1, Sobranie
Sochineniy, ed. N.A. Struve. Paris: YMCA-PRESS, 1985.


Florensky, P.A. "Symbolarium (Slovar simvolov). Predislovie" |Symbolarium (The dictionary oI
symbols). Introduction|. Trudy po znakovym sistemam V. Tartu: Uchebnye zapiski Tarturskogo
Universiteta 284, 1971.

Foster, Hal, ed., Vision and Visuality. Seattle: Bay Press, 1988.

Freud, Sigmund. The Interpretation oI Dreams. New York: Avon Books, 1965.

Freud, Sigmund. Standard Edition oI the Complete Psychological Works. London: Hogarth
Press, 1953.

FriedhoII, Richard Mark and William Benson. The Second Computer Revolution: Visualization.
W.H. Freeman and Company, 1991.

Gabor, D. "A Summary oI Communication Theory." In Proceedings oI a Symposium on
Applications oI Communication Theory, London, 1952, edited by Willis Jackson. London:
Butterworth ScientiIic Publications, 1953.

Gardner, Martin. Logical Machines and Diagrams. 2nd ed. Chicago: The University oI Chicago
Press, 1982.

Golomstock, Igor. Totalitarian Art. New York: HarperCollins Publishers, 1990.

Gombrich, E.H. "Pictorial Instructions." In Images and Understanding, ed. Barlow Horace, Colin
Blakemore and Miranda Weston-Smith. Cambridge: Cambridge University Press, 1990.

Gombrich, E.H. Art and Illusion. Princeton: Princeton University Press, 1960.

Goodman, Nelson. Languages oI Art. Indianapolis: Hackett, 1976.

Hall, Stuart. "Encoding/Decoding." In Culture, Media, Language, ed. Stuart Hall et al.
Hatchinson, 1980.

Hansen, Miriam. "The Hieroglyph and the Whore: D.W. GriIIith's Intolerance." The South
Atlantic Quarterly 88, no. 2 (1989): 361-392.

Hartley, R.V.L. "Transmission oI InIormation." Bell System Tech. Journal 7 (1928): 535-563.

Hearst, Eliot. "One Hundred Years: Themes and Perspectives." In The First Century oI
Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates,
1979.

Heidegger, Martin. What is Called Thinking? New York: Harper & Row, Publishers, 1968.

Heron, W. "The Pathology oI Boredom." ScientiIic American 196 (1957): 52-56.

Hochberg, Julian. "Sensation and Perception." In The First Century oI Experimental Psychology,
edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates, 1979.

Holly, Michael Ann. PanoIsky and the Foundations oI Art History. Ithaca: Cornell University
Press, 1984.


Holmes, Oliver Wendel. "The Stereoscope and the Stereograph." In Photography, Essays and
Images: Illustrated Readings in the History oI Photography, edited by Beaumont Newhall. New
York: Museum oI Modern Art, 1980.

Hoover, Theodore and Charles Fish. The Engineering ProIession. StanIord: StanIord University
Press, 1941.

"Institut Khudozestvennoy Kultury" (The Institute oI Artistic Culture). Russkoe Iskusstvo 2-3
(1923): 85-88.

Ivins, William M. On the Rationalization oI Sight. New York: Da Capo Press, Inc., 1975.

Jakobson, Roman. "Closing Statement: Linguistics and Poetics." In Semiotics. An Introductory
Anthology, edited by Robert Innis. Bloomington: Indiana University Press, 1985.

Jakobson, Roman. "Metalanguage as a Linguistic Problem," Presidential address delivered at the
Annual Meeting oI the Linguistic Society oI America, 1956. In The Framework oI Language.
Ann Arbour: Michigan Studies in the Humanities, 1980.

Jay, Martin. Downcast Eyes: the Denigration oI Vision in Twentieth-century French Thought.
Berkeley: The University oI CaliIornia Press, 1993.

Jay, Martin. "Scopic Regimes oI Modernity." In Vision and Visuality, ed. Hal Foster. Seattle:
Bay Press, 1988.

Johnson-Laird, Philip. Mental Models: Towards a Cognitive Science oI Language, InIerence,
and Consciousness. Cambridge: Cambridge University Press, 1983.

Joravsky, David. Russian Psychology: A Critical History. Cambridge, Mass.: Basil Blackwell,
1989.

Kandinsky, Wassily. Point and Line to Plane. New York: Solomon R. Guggenheim Foundation,
1947.

Kandinsky, Wassily. "Programma raboty Instituta khudozhestvennoy kultury" (The program oI
the Institute oI Artistic Culture). In Sovetskoe iskusstvo za 15 let. Materialy i dokumentatsiia
(FiIteen years oI Soviet art: materials and documentation), ed I. Matsa (Moscow, IZOGIZ:
1933).

Kantowitz, Barry H. and Robert D. Sorkin. Human Factors: Understanding People-System
Relationships. John Wiley & Sons: 1983.

Keller, Evelyn Fox and Christine R. Grontkowski. "The Mind's Eye." In Discovering Reality:
Feminist Perspectives on Epistemology, Metaphysics, Methodology, and Philosopohy oI
Science, edited by Sandra Harding and Merrill B. Hintikka. London: D. Reidel Publishing
Company.

Kemp, Martin. The Science oI Art. New Haven: Yale University Press, 1990.

Kenez, Peter. The Birth oI the Propaganda State. Soviet Methods oI Mass Mobilization, 1917-
1929. Cambridge: Cambridge University Press, 1985.


Khan-Magomedov, S.O. "INKhUK: vozniknovenie, Iormirovanie i pervyy period raboty. 1920"
(INKhUK: appearance, Iormation and Iirst period oI its work, 1920). Sovetskoe Iskusstvoznanie
(1981): 332-368.

Koerner, Joseph Leo. "The Shock oI the View." The New Republic (April 26, 1993): 32-38.

Kogan, P.C. "O zadachakh akademii i yeyo zhurnala" (About the goals oI the academy and its
journal). Iskusstvo 1, no. 1 (1923).

Kovasznay, L.S.G. and H.M. Joseph. "Image Processing." Proceedings oI IRE 43 (1955): 560-
570.

Kozulin, Alex. Psychology in Utopia: Toward a Social History oI Soviet Psychology.
Cambridge: The MIT Press, 1984.

Krauss, Rosalind. The Optical Unconscious. Cambridge: The MIT Press, 1993.

Lacan, Jacques. "On the Gaze as Objet Petit a." In The Four Fundamental Concepts oI Psycho-
Analysis, edited by Jacques-Alain Miller. New York: W.W. Norton & Company, 1981.

LakoII, George. "The Invariance Hypothesis: Is Abstract Reason Based on Image-Schemas?"
Cognitive Linguistics 1, no. 1 (1990): 39-74.

LakoII, George. "Cognitive Linguistics." Versus 44/45 (1986): 119-154.

Latour, Bruno. "Visualization and Cognition: Thinking with Eyes and Hands." Knowledge and
Society: Studies in the Sociology oI Culture Past and Present 6 (1986): 1-40.

Lavin, Maud. "Photomontage, Mass Culture, and Modernity. Utopianism in the Circle oI New
Advertising Designers." In Montage and Modern LiIe: 1919-1942, edited by Matthew
Teitelbaum. Cambridge: The MIT Press, 1992.

Lavrent'ev, A.H. "Propedevticheskaya distsiplina 'GraIika.' Vkhutemas. 1920-22 gody" (The
discipline 'Graphics.' VKhUTEMAS. 1920-22). Tekhnicheskaya Estetika 7 (1984): 16-21.

LeIebvre, Henri. The Production oI Space. OxIord: Blackwell Publishers, 1991.

LZvy-Bruhl, Lucien. Les Iunctions mentales dans les sociZtZs primitives. Paris: 1910.

Locke, John. An Essay Concerning Human Understanding. Edited by A.S. Pringle-Pattison.
OxIord: Clarendon Press, 1924.

Lotman, Jurij. The Structure oI the Artistic Text. Ann Arbour: Department oI Slavic Languages
and Literatures, The University oI Michigan, 1977.

Lowe, David. Three-Dimensional Object Recognition Irom Single Two-Dimensional Images.
Robotics Report 62. New York: Courant Institute oI Mathematical Sciences, New York
University, 1986.

Mandler, Jean Matter and George Mandler, eds. Thinking: From Association to Gestalt. New
York: John Wiley & Sons, Inc., 1964.


Manovich, Lev. "Assembling Reality: Myths oI Computer Graphics." AIterimage 20, no. 2
(September 1992): 12-14.

Manovich, Lev. "'Real' Wars: Esthetics and ProIessionalism in Computer Animation." Design
Issues 6, no. 1 (Fall 1991): 18-25.

Marr, David. Vision. New York: W.H. Freeman and Company, 1982.

Matsa, I., ed. Sovetskoe iskusstvo za 15 let. Materialy i dokumentatsiia (FiIteen years oI Soviet
art: materials and documentation). Moscow, IZOGIZ: 1933.

McArthur, David. "Computer Vision and Perceptual Psychology." Psychological Bulletin 92, no.
2 (1982): 283-309.

McFarlane, M. D. "Digital Pictures FiIty Years Ago." Proc. IEEE 60, no. 7 (1972).

McGraw-Hill Encyclopedia oI Science & Technology: an International ReIerence Work in
Twenty Volumes Including Index. New York: McGraw-Hill, 1992.

Michelson, Annette. "Reading Eisenstein Reading 'Capital'." October 2 (1976): 27-38; October 3
(1977): 82-88.

Miller, G.A. Language and Communication. New York: McGraw-Hill Book Co., Inc., 1951.

Mitchell, J. William. The ReconIigured Eye: Visual Truth in the Post-Photographic Era.
Cambridge, The MIT Press, 1992.

Mitchell, W.J.T. Iconology. Image, Text, Ideology. Chicago: The University oI Chicago Press,
1985.

Moholy-Nagy, Lszl. "Photography in Advertising." In Photography in the Modern Era,
edited by Christopher Phillips. New York: Aperture, 1989.

Moholy-Nagy, Lszl. The New Vision. New York: Wittenborn, 1947.

Lszl Moholy-Nagy. Painting, Photography, Film. 1925. Reprint. Cambridge: The MIT Press,
1973.

Molok, Yu. A. "'Slovar simvolov' Pavla Florenskogo. Nekotorye margonalii" (Pavel Florensky's
'dictionary oI symbols.' A Iew margins). Sovetskoe Iskusstvoznanie 26 (1990): 322-343.

MYnsterberg, Hugo. The Photoplay: A Psychological Study. New York: D. Aplleton & Co.,
1916.

Newhall, Beaumont. Airborne Camera. New York: Hastings House, Publishers, 1969.

Nietzsche, Friedrich. "On Truth and Falsity in their Ultramoral Sense." In Early Greek
Philosophy and Other Essays. New York: The Macmillan Company, 1924.

Noble, Douglas. "Mental Materiel: The Militarization oI Learning and Intelligence in U.S.
Education." In Cyborg Worlds: the Military InIormation Society, edited by Les Levidov and
Kevin Robins. London: Free Association Books, 1989.


Nsth, WinIried. Handbook oI Semiotics. Bloomington and Indianapolis: Indiana University
Press, 1990.

PanoIsky, Erwin. Perspective as Symbolic Form. New York: Zone Books, 1991.

Peat, David. ArtiIicial Intelligence: How Machines Think. New York: Baen Enterprises, 1985.

Pinker, Steven. "Visual Cognition: An Introduction." Cognition 18 (1984): 1-63.

Posner, Michael I. and Gordon L. Shulman. "Cognitive Science." In The First Century oI
Experimental Psychology, edited by Eliot Hearst. Hillsdale, NJ: Lawrence Erlbaum Associates,
Publishers, 1979.

Posner, Michael I. "Section V: InIormation Processing. Overview." In vol. 2, Handbook oI
Perception and Human PerIormance, edited by Kenneth BoII, Lloyd KauIman, and James P.
Thomas. New York: John Wiley and Sons, 1986.

Quastler, H., ed. InIormation Theory in Psychology: Problems and Methods. Glencoe, IL.: Free
Press, 1955.

Rabinbach, Anson. The Human Motor: Energy, Fatigue, and the Origins oI Modernity. Basic
Books, Inc., 1990.

Rappoport, A.G. "El Lissitsky i ego 'Pun-geometiya'" (El Lissitsky and his 'Pan-geometry').
Sovetskoe Iskusstvoznanie 25 (1989): 113-130.

Reichardt, Jasia. The Computer in Art. London and New York: Studio Vista and Van Nostrand
Reinhold Company, 1971.

Resnikov, Howard. The Illusion oI Reality. New York: Springer-Verlag New York Inc., 1989.

Reveaux, Tony. "Virtual Reality Gets Real." New Media (January 1993): 36-41.

Rheingold, Howard. Virtual Reality. New York: Simon & Schuster, Inc., 1991.

Riggs, L.A. et al. "The Disappearance oI Steadily Fixated Visual Tests Objects." J. Optical Soc.
America 43 (1953): 495.

Roberts, L.G. "Machine perception oI three-dimensional solids." In Optical and Electo Optical
InIormation Processing, edited by J.T. Tippett. Cambridge: The MIT Press, 1965.

Roberts, L.G. Homogeneous Matrix Representations and Manipulation oI N-Dimensional
Constructs. MIT Lincoln Laboratory MS 1405, 1965.

Roberts, L.G. Machine Perception oI Three-Dimensional Solids. MIT Lincoln Laboratory TR
315, 1963.

Rose, Jacqueline. Sexuality in the Field oI Vision. London: Verso, 1986.

Saint-Martin, Fernande. Semiotics oI Visual Language. Bloomington and Indianapolis: 1990.

Sampson, GeoIIrey. Schools oI Linguistics. StanIord, CA: StanIord University Press, 1980.


Sardarov, A.S. "5000 let evolutsii doroznogo znaka" (5000 years oI the evolution oI a traIIic
sign). Tekhnicheskaya Estetika 9 (1984): 14-19.

Scha, Remko. "Virtual Voices." MediaMatic 7, no. 1 (1992): 27-43.

Schramm, Wilbur. "How Communication Works." In The Process and EIIects oI Mass
Communication. Urbana: University oI Illinois Press: 1954.

Sekula, Allan. "The Instrumental Image: Steichen at War." In Photography against the Grain:
Essays and Photo Works, 1973-1983. HaliIax: The Press oI the Nova Scotia College oI Art and
Design, 1984.

Sekula, Allan. "The Body and the Archive." October 39 (1987): 3-64.

Shannon, Claude E. and Warren Weaver. The Mathematical Theory oI Communication. Urbana:
The University oI Illinois Press, 1949.

Shepard, S. and J. Metzler. "Mental Rotations oI Three-dimensional Objects." Science 171
(1971): 701-703.

SIGGRAPH '89 Panel Proceedings. New York: The Association Ior Computing Machinery,
1989.

SIGGRAPH '92 Final Program. New York: The Association Ior Computing Machinery, 1992.

Sonesson, Gsran. Pictorial Concepts. Inquiries into the Semiotic Heritage and its Relevance Ior
the Analysis oI the Visual World. Lund, Sweden: Lund University Press, 1989.

Spriegel, William R. and Clark E. Myers, eds. The Writings oI the Gilbreths. Homewood, IL.,
1953.

Stelzer, Otto. "Moholy-Nagy and His Vision." In Lszl Moholy-Nagy, Painting, Photography,
Film. 1925. Reprint, Cambridge: The MIT Press, 1973.

Taylor, Brandon. Art and Literature under the Bolsheviks. London: Pluto Press, 1991.

Taylor, Franklin. "Psychology at the Naval Research Laboratory." American Psychologist 2, no.
3 (1947): 87-92.

Taylor, Frederick Winslow. The Principles oI ScientiIic Managment. New York, 1967.

Titchener, Edward BradIord. A Beginner's Psychology. New York: The Macmillan Company,
1915.

Todorov, Tzvetan. Theories oI the Symbol. Ithaca: Cornell University Press, 1984.

Tretyakov, Sergei. "Otkuda i kuda? (Perspektivu Iuturizma)" |Where Irom and where to?
(Perspectives oI Iuturism)|. LEF 1 (1923): 192-203.

Vasilieva, Yu. A. "Eisenstein: Kontseptsija 'Agressivnogo Iskusstva'" (Eisenstein: the concept oI
'aggressive art'). Kinovedcheskije Zapiski 3 (1989), 205-209.

Vertov, Dziga. "Kinoki. Perevorot" (Kinoki. A revolution). LEF 3 (1923): 135-143


Virilio, Paul. Lost Dimension. New York: Semiotext(e), 1991.

Virilio, Paul. War and Cinema: the Logistics oI Perception. London: Verso, 1989.

Viteles, Morris. Industrial Psychology. New York: W.W. Norton & Company, Inc., 1932.

Vitz, P.C. "PreIerence Ior DiIIerent Amounts oI Visual Complexity." Behavioral Science 2
(1966): 105-14.

Vitz, Paul C. and Arnold B. Glimcher. Modern Art and Modern Science. The Parallel Analysis
oI Vision. New York: Praeger, 1984.

Vygotsky, Lev. Psikhologija Iskusstva (Psychology oI art). Moscow, 1968.

Weber, Max. From Max Weber: Essays in Sociology. New York: OxIord University Press, 1958.

Weber, Samuel. The Legend oI Freud. Minneapolis: University oI Minnesota Press, 1982.

Wiener, Norbert. The Human Use oI Human Beings. Cybernetics and Society. New York: Avon
Books, 1967.

Wiener, Norbert. Cybernetics. OI Control and Communication in the Animal and the Machine.
New York: John Wiley & Sons, Inc., 1948.


FIGURES

Figure 1. Humbert de Duperville's "Synoptic Table," 1827-32.
Figure 2. Charles Henry's aesthetic protractor, 1988.
Figure 3. Diagram illustrating Georges Seurat's theory oI expression.
Figure 4. The exercises developed by Rodchenko Ior his course on graphics at VKhUTEMAS,
1921-22. Top: examples oI basic Iorms given by Rodchenko. Bottom: example oI a student
work.
Figure 5. Example oI computer generated visualization.
Figure 6. Representations oI propositions as Euler's diagrams, an earlier Iorm oI Venn's
diagrams. Max MYller, The Science oI Thought (London: Longmans, Green, and Co., 1887),
543.
Figure 7. Composite photography. Frontispiece Irom Francis Galton, Inquiries into Human
Faculty (London: Macmillan, 1883).
Figure 8. Freud's diagram illustrating the relation between two triads oI concepts: id, ego, and
superego, and unconscious, preconscious, and conscious.
Figure 9. Top: Sigmund Freud's schematic picture oI sexuality. Bottom: example oI a modern
diagram.
Figure 10. Example oI diagrammatic graphic language.
Figure 11. Figure Irom Monge's GZomZtrie descriptive, 1799.
Figure 12. Two draughtsmen plotting points Ior the drawing oI a lute in Ioreshortening, Irom
Durer's Underweysung, 1525.
Figure 13. WireIrame computer graphics by Boeing, early 1960s.
Figure 14. ACRONYM computer vision system processes an aerial photograph, late 1970s.
Figure 15. Laser range Iinder, early 1980s.
Figure 16. El Lissitsky. Tatlin at Work. Photomontage. Illustration Ior Ilia Erenburg's Six Tales
with Easy Endings. 1922.
Figure 17. Wireless telegraphed photography.
Figure 18. Schematic diagram oI a general communication system.
Figure 19. Example oI a stimulus used in experiments on preIerence as a Iunction oI the amount
oI inIormation.
Figure 20. One oI the earliest models oI a human as inIormation processing system.
Figure 21. The model oI a human as inIormation processor which summarizes the available data
(by 1986) about human sensory, cognitive and motor perIormance.
Figure 22. NASA Ames Virtual Reality Environment Workstation, late 1980s.
Figure 23. SAGE (the "Semi-Automatic Ground Environment") -- the Iirst human-machine
interactive display system, mid 1950s.