the articulation of

visual experiences
through algorithm
Carlos Lunetta
The articulation of visual experiences through algorithm
by Carlos Lunetta
Thesis submitted in partial fulllment of the requirements
for the degree of Master of Fine Arts and is approved by the
MFA Design Review Board of the Massachusetts College of
Art in Boston.
May 2005
Jan Kubasiewicz
Thesis Advisor
Joe Quackenbush
Thesis Document Advisor
Brian Lucid
DMI Faculty
Toby Bottorf
External Reviewer
Abstract
The Articulation of Visual Experiences Through Algorithm explores the
concepts and possibilities computation presents as a creative medium for
design.
This thesis investigates the relationship between code and visual language.
From this relationship, a new vocabulary of new media will emerge,
driven by ideas that stretch their existence simultaneously through all
means visual, aural, verbal, haptical, temporal and logical.
My methodology was based on discrete experiments native to the compu-
tational medium which investigated a range of topics including: musical
intervals; generative and interactive form systems; interactive op art;
database visualization; and the plastic components of time.
7
9
Aknowledgements
I traveled 4817 miles from So Paulo to Boston to improve myself as a
designer. This would have been impossible without the support of my
family, but I own absolutely everything to my wife Margarida, and her
love, support and patience.
Id like to express my gratitude to Jan Kubasiewicz, not only my mentor,
but also a good friend.
To Joe, Gunta and Brian, for all the worthy hard work.
To my fellow classmates, Stephen, Keiko, Kate, Karolina and specially
Lynn (and Ronaldo) for all the support and friendship.
To my friends Colin Booth and BJ Jones, for trusting and receiving a
Brazilian weird student at the US.
To Dave and everyone at Beam interactive, for the opportunity of
practicing the thesis preach.
To Ben Fry, Mario Klingemann and all Processing community.
And nally to Google, for returning 264,000 thesis acknowledgments to
my amusement. There are surely a lot of grateful people out there!
11
Table of Contents
Denitions 17
Roots 27
Concepts 39
Praxis 53
Experience 67
Case Studies 71
Sound in Space 73
Form Systems 85
Riley Patterns 99
Logic Molecules 109
Patterns of Time 115
Findings 121
Bibliography 125
13
Preface
Every work of art is the child of its age [...] Each period of culture
produces an art of its own which can never be repeated.
Wassily Kandinsky
15
Disclaimer
To dene is to limit.
Oscar Wilde, in The Picture of Dorian Gray
The limitation serves a purpose - better understanding.
This thesis document denes a series of concepts and ideas, many of
them vast, open and entangled. By any means those denitions intend
to limit concepts they mean to raise understanding by disclosing
relevant issues.
Ideas are not and shouldnt be - limited to their verbal counterparts.
17
Denitions
Visual experience, dened.
Design is an act of assembly of form through composition.
1
A composition that has a purpose: to communicate.
The idea of communication implies that a message will eventually be
received by someone; there is an audience, a receptor. Also, communi-
cation implies the existence of a medium, a dimension where composition
becomes reality.
Composition, audience and medium are linked together: each one inu-
ences and is inuenced by the other, in a mutual, constant change.
The term visual experience is dened by the connection of these three
concepts; a composition acknowledged of its audience and aware to the
nature of its medium.

1
Design is the method of putting form and content together. Design, just as art, has multiple denitions,
there is no single denition. Design can be art. Design can be aesthetics. Design is so simple, thats why it is
so complicated. Paul Rand so complicated. so complicated.
a model for visual experience
18
Computer: a new medium
The personal computer has worked as a tool for design production and
artistic experimentation since its tender age. During the mid-nineties,
though, a shift in the role of computing occurred: the advent of the
internet, that allied with the proliferation of household computers gave
birth to a new mass communication medium.
Computational devices
2
became a medium, with an important difference
from other known media they allow direct feedback and participation
from the audience. The concepts of broadcaster and viewer got blurred,
since one can become the other after few keystrokes.
As a medium, computation affects the audience. Consequently, graphic
design is inuenced by the change; the new medium opens a myriad of
possibilities for creation in time, space, shape and sound. The partici-
pation of the audience raises the concept of interactivity.
Computers have became no longer just a production tool, but also the
living space for the design creation.
New media creativity
Once a new medium has been established and the gates for new dimen-
sions in design have been opened, a question arises: how is the conception
of visual experiences changed? How can new media affect creativity?
The initial transformation in design was an adaptation; new media design
was a collection of items from different media bonded together. A print
design piece would receive navigation buttons, a small animation intro
and be published as a new media piece.
The adaptation evolved according to the features that mainstream
graphic software packages would offer; software companies like Adobe
and Macromedia ended up dictating the aesthetics of interactive com-
munications by offering canned solutions, like templates and plug-ins,
2
Although this document aims the exploration of personal computers, like desktop or portable systems, any
micro processed object is considered a computational medium: cell phones, palm tops, text messengers, DVRs
and others
19
and generating design elements according to their own proprietary algo-
rithms, like vector antialiasing and motion tween functions.
Up to this point in time, the creative process per se changed very little
more dimensions were added, but the visual experience conception
remained the same. Under the navigation buttons and animated items,
conceptually print media pieces live online.
Birth
The computer is both a medium and a tool for design but the creative
process seems to still be far from the machine. Why is it the case that the
principles of computation are not present in the creative framework, in
the conception of a visual experience?
They are present, but for a very small number of designers, still far from
the mainstream. Theres a gigantic gap between the number of designers
that use computers as a tool and the number of designers that use the
computer as a part of the concept in the design work. There are a small
number of design pieces that can only exist and be understood inside the
digital medium.
The language of design is in constant evolution, and now it faces the
opportunity to embrace the concepts and processes of its new hosting
medium, becoming native, being reborn.
What are the concepts of computation?
Computation is about change.
The personal computer is a highly evolved calculator, performing billions
of arithmetical and logical operations per second. The processor works by
getting certain inputs, performing calculations and outputting them; data
is constantly being chopped to small instructions, calculated and rebuilt.
Lets use a keyboard input, for example; the physical keystroke is
converted to a binary signal; the signal is processed and converted to a
visual output, the graphical representation of the key that is displayed
20
on screen. The core of the computational process is about translating,
converting a certain input to a certain output.
The creative use of computation takes advantage of the nature of data pro-
cessing in favor of design. There are situations where traditional design
meets the limits of human labor; a computer can address a different scale
than human hands in artistic creation, dealing with the very large, the
very small, the very precise, the very exhausting, so on and so forth.
Some of the computation characteristics are:
Transformation: as stated above, the main concept of compu-
tation is to transform data.
Repetition: a processing instruction can repeat itself eternally, as
long as its physical part works. An element can repeat itself over time
or over space the replication is lossless.
Randomness: true randomness comes from nature, in events
like radioactive decay. Computer systems feature pseudo-random
numbers, a formula that produces no repetition, smooth numeric
distribution, and more important: the lack of predictability.
Precision: computer processing elements can be extremely exact
and precise when necessary.
Predictability: once a set of rules is created, the code behaves
accordingly. Just like the physical world is attached to the laws of
physics, a program can be attached to a set of unbreakable laws that
will be always obeyed.
a model for computation

21
Discretion: program elements can be discrete; events can be
triggered by any kind of input rather than linear time. This is the key
for interaction.
Complexity: computation can achieve and display unbearable
amounts of visual or logical information.
Simplicity: the opposite of complexity; the ability to complete a
task with a clear, austere and minimal output.
Remembrance: the ability to store data, and recall it later, with no
logical capping of time or quantity the only limit is physical.
The concepts of computation are implemented in a computer code as
formulas, or sets of instructions known as algorithms.
The origins of the gap
There is a resistance in incorporating computer processing characteristics
in the concepts of design. The resistance has mainly two reasons;
The rst reason is the difculty in communication between humans and
computers; the computer language is different than common verbal
language. Some specic training is necessary so one can be fully able to
read and write computer code. At the same time, the amount of training
today is a small fraction from the amount of training twenty years
ago. Computer language is evolving at a similar speed of the computer
hardware evolution. Coding is more accessible now than ever, and theres
a prognostic for future improvements. The challenge is to learn what to
tell the computer and not simply how to tell it. Language syntax depends
on the available technology, but the idea to be coded is timeless.
The second reason is cultural and social; western education branches arts
and humanities from math and science. From high school to postgraduate
education, theres a strong sense of separation between different areas
of knowledge. Computational design merges art, math, physics and even
psychology, when dealing with the study of interactive design reaction.
The union of different branches of knowledge is required accompanied
by a mindset prepared to create blissfully in any necessary eld of study.
22
Verbal, visual and computational
How different is the computational language from visual language?
The human skill of communication is granted by sensorial channels;
ideas, facts and feelings are understood and expressed whether in verbal,
visual or any other sense.
The compounding elements of a channel have similar counterparts on
another channel; correlations can be addressed between elements with
similar conceptual functions
The correlation of elements has a clear role in the understanding of
language; think about the word red. This word has strong visual in-
formation attached to it, representing the visual hue known as red. The
experience of red can be expressed verbally, through the word, or visually,
through the hue. It also can be learned both ways.
We can understand a concept by contextualizing it in a communication
channel; words can describe visual objects, phrases can describe visual
actions - and vice versa.





correlations between verbal and visual domains
23
The computational language acts as an extra channel - it has similar
structural properties to other communication means, thus, correlations
of similar elements in the structure can be established.
The computer language, in essence a binary series of logical and arith-
metical operations, can be understood by contextualizing other channels
especially, but not limited to, verbal and visual realms. We can commu-
nicate to a computer by typing verbal commands; the verbal will become
a digital process, returned and converted to any output verbal, visual,
haptical, sonorous or any other available.
Language is a system
4
, where elements are composed to form meaning;
The structure of the computational language is similar to the structure of
visual, verbal or any other language; a structure of elements interacting
in a composition. Elements can be arranged to work together in a logical
environment just as in the visual or verbal form.

4
A language is a system of pure values Ferdinand Saussure A language is a system of pure values A language is a system of pure values
domain correlation examples
24

correlations between verbal, visual and logical domains

domain correlation examples
25
Module and System
Module is an independent part, a single element that exists by itself,
having built-in functionalities. System is the working interrelation of
modules, comprised as a whole.
Scale is a determining factor to dene something as a module or a system;
an atom can be taxonomized as a module a unique, independent part -
or a system, formed by the particles and sub particles that interact within
its apparent unity.
The building blocks that dene a language are considered modules; each
module is unique, and represents a concept, an idea. Modules interrelate,
creating a system - the composition. Words interact in a sentence, shapes
interact in a canvas, and programming objects interact in a code.
Articulation
The articulation of ideas in the visual, verbal or computational dimen-
sions is based on the composition of elements.
As initially stated, design is composition - acknowledged by its audience
and medium. The articulation of visual experiences can grow along the
creative potentials of the new computational medium;
Its time to articulate visual experiences through algorithm.





the structure of model/system,
with an example of application.
27
Roots
Art, Science and Religion
The underlining concepts of computation and design have met many
times through history; accessible computation may be a new format, but
mathematical and scientic concepts aligned with art are not.
The branches of knowledge have been treated as one through time; the
knowledge of understanding nature, the knowledge of understanding
the spirit and the knowledge of expressing both. Science, religion and art
are linked together in the history of civilization. Pyramids, temples and
statues that remain still prove how engineering and art were combined in
service of belief.
Notable thinkers in history also show that the path of science and art can
be one; Greek philosophers like Plato and Aristotle, for example, inves-
tigated math and physics along with aesthetics and poetry all inquiries
about the real world.
Jumping in time and space, one of the best examples is Leonardo da Vinci.
Da Vinci had a strong grasp of mathematics and engineering, showed
in his fabulous inventions, and strong artistic ability, proofed with his
invaluable paintings and sculptures. His creativity had no borders.
a solid articulation of geometry.
28
The renaissance era of da Vinci gives us an important demonstration on
the incorporation of mathematical and scientic concepts in the artistic
creation. Before this movement, medieval paintings were visually at.
Space and volume were represented, but did not reect reality. The artistic
composition, as a concept, was focused in the symbolical components;
the concept of composition was centered in the meaning.
The renaissance marks the advent of perspective techniques. Perspective
drawing was initially a work of architects only, used in construction
projects. But the new aesthetic of space spread and many talented painters
embraced it when creating their work.
Art as opposite of science? Masaccio, who is often hailed as
the rst painter of modern times, was a member of the guild of
apothecaries and grocers
Karl Gerstner
In a non-digital way, Renaissance paintings are visual experiences ar-
ticulated through algorithm; the formulas of perspective were carefully
implemented when picturing space, scientic data was used to create
visual compositions and geometry became part of the creative process.
Sassettas The Meeting of Saint Anthony
and Saint Paul:the vanished vanishing point
29
Advancing more in time, we arrive at the modern model of knowledge
categorization; each eld of study is highly dened and specialized. As
civilization evolves, science grows in complexity its intangible to grasp
all elds of knowledge. Art became a distant branch, and even the lower
stages of education impose a learning path choice.
After a certain high level of technical skill is achieved, science
and art tend to coalesce in esthetics, plasticity, and form. The
greatest scientists are always artists as well.
All religions, arts and sciences are branches of the same tree
Albert Einstein
Masaccios Trinity. Masaccio
employed complex mathematical
detailing when producing his work.
30
Physicists like Albert Einstein and Werner Heisenberg believed that
knowledge has similarities in different specializations and theres a
mutual benet of connecting those similarities.
The Massachusetts Institute of Technology (MIT) plays an interesting role
in this relationship. Renowned for the development of groundbreaking
technologies in different elds of science, MIT maintains the CAVS
Center for Advanced Visual Studies, founded by Gyorgy Kepes. Kepes
was a designer that did not believe in the separation of art and science;
his explorations on visual language and his books reect this idea, and
for almost thirty years (1946-74) his presence at the MIT maintained this
ideal alive.
An experimental effort to encourage such interthinking between
different disciplines in the visual arts and scientic and technical
elds is more than overdue. As the twentieth century has grown
older, such intercommunication has become seemingly more
improbable
The dynamic interdependence of the wisdom of the senses and
wisdom of reason is fundamental to a balanced growth
Yet, history shows us that art and science, two basic human
activities springing from a common love of nature are interde-
pendent. Each achieves stronger growth when cross-pollinated
by the other.
Gyorgy Kepes
Computation
The domains of visual and computational languages met right after the
advent of processing machines, during the fties. The early explorations
involved tweaking the graphical output of huge computers to record lm
frames and create animations, or by simply photographing an oscillo-
scope display that could be programmed. John Whitney, Sr. was one of
the most important founding fathers of computer graphics.
31
The union of color and tone is a very special gift of computer
technologies
Time has become visual
My computer program is like a piano. I could continue to use it
creatively all my life.
John Whitney, Sr.
These quotes regard Whitneys explorations on computational abstract
animation and music
1
.
In 1962, A. Michael Noll at New Jerseys Bell Labs created a computer
program that featured an algorithm whose function was to build visual
compositions; the reference for the algorithm rules was the composition
style of Mondrian.
The work of Noll has extreme importance; besides Mondrians mimic, his
programs investigated composition from a probability stand point visual
forms were derived from a computational concept. Lines would be drawn
(or not) according to the visual density of the composition. Other Noll
Oscilloscope recording by Whitman
1 Two of his short movies can be accessed at
http://www.siggraph.org/artdesign/prole/whitney/nobio_intro.html
32
works dealt with geometric randomness or repetition; he clearly took
advantage of the newborn computation process in a creative way. The
machinery evolved a lot since then, but Nolls ideas are still fresh there
is no conceptual obsolescence.
At the same time, Douglas Engelbart was creating the foundations of
electronic communications and computational ofce work. Engelbart
developed the mouse, the hyperlink, video conference, le structure
navigation, word processing and more. Although he wasnt pursuing
artistic objectives, his work supports the employment of creativity in the
computer medium, proposing to use the machine to augment the human
intellect.
The MIT made strong contributions in the graphic computing eld. In
1963 the Sketchpad was created the rst CAD (Computer Aided Design)
interface. Sketchpad was extremely advanced for its time; the user would
interact with the application directly on screen, with a pen, through a
graphic interface. All the interaction concepts of this system are the
basically the same we use today.
In the early seventies, Xerox Corporation founded the Palo Alto Research
Center, also known as PARC. It was inside the PARC that most of the
modern concepts used in computation were invented, like the icon,
object oriented programming, laser printer, WYSIWYG (what you see
Nolls Mondrian experiment
33
is what you get), computer notebooks, paint and animation applications
(before that, everything had to be coded or use dedicated hardware), midi
composition and many other ideas that shaped the terms multimedia
and graphical interface. The PARC computers operating system was
called SmallTalk and it featured something that is still lacking in todays
operating systems: seamless integration between the graphic interface
and the code behind it. The user was able to customize, change and create
new functions whenever it was necessary, since there was no barrier to
explore the operating source code. PARC successfully conducted experi-
ments where teenagers were taught to code in SmallTalk language, and
soon they would use their creativity to generate graphic applications.
SmallTalk can be considered the golden age of computational design,
when the visual and the logical sides were always connected and evident.
The contemporary contribution from the MIT comes from a new gen-
eration of scientists/artists/designers developing projects regarding the
The SketchPad
PARC SmallTalk - 1976
34
interaction of humans and technology. This happened at the Visual
Language Workshop, in 1973, and later, in 1982 when the Media Lab was
founded by Nicholas Negroponte. The specic topic of visual language
mixed with computation was more intensely researched years later by
Muriel Cooper, a pioneer in digital typography who worked to develop the
language and structure of electronic design, and to expand the graphic
design culture to four dimensions. John Maeda, considered Coopers
successor, founded the Aesthetics + Computation
2
group, where many
researchers were involved in visual issues through programming, and how
the visual language could be expressed and interacted with an audience
through computation.
The Aesthetics and Computation group started its operations at the
same period that the internet blossomed and desktop computers became
present in common households. Designers started to create for the
computational medium; the graphic software packages would keep the
programming language underneath a friendly interface, protecting
designers from the necessary internet coding. The edge of programming
for artists was consolidated with Macromedia Flash and Macromedia
Director, commercial software packages where advanced users could
script their own visual behaviors code.
The term Aesthetics + Computation wasnt just university lab concept
anymore it was present in the society. One of the projects from Maeda
dealt with accessible computer coding focused in visual elements, called
Design by Numbers a coding interface whose strength was in its
minimalism, where the communication between designer and machine
could be clear. Maedas project appears to have strong inuence from
the previous Turtle project from MIT, a software application where an
element on screen called turtle could be moved through code in order to
build images. The turtle project later was know as Logo, and was largely
used in schools during late seventies/early eighties. Logo helped to teach
the basic concepts of computing the same idea that Design by Numbers
explore. John Maedas work also helped to spread the idea that design
could be achieved through computational means, that creativity could be
expressed through code with his interesting explorations of algorithmic
visuality.
2 http://acg.media.mit.edu/
35
The second project from MIT towards a visual creation code was
created by Ben Fry and Casey Reas, and is called Processing. It adds
a great number of features and extend Design by Numbers, providing
more creative freedom and keeping the accessibility for non-computer
scientists. Processing has today around 1000 users a small number
compared to the whole interactive design community, but a big number
of participants in a non-commercial, non-advertised university project,
that is still entering its beta stage of development. The most important
fact: the massive majority of the users are interactive designers, who were
initiated to scripting languages when developing for the internet, and
sought ways for doing more. Processing has no timeline, no visual thrills,
and its interface is a simple text editor. But the concept of the application
is clearly targeted to the artistic creation, hence its success among the
design community.
Logo interface
Maedas Design by
Numbers interface Numbers Numbers
36
Processing Beta interface
39
Concepts
Creative Framework
The idea of a creative framework can be explained through examples; lets
use painting and photography. Two forms of expression that share similar
characteristics theyre visual, compositive, bi-dimensional, have color
hues or tones that form shapes and they communicate through visual
information.
Although similar, theyre also very different. Its easy to discern one from
another; they have different aesthetic values, and generate completely
different reactions.
Photography strongly reects reality, augmenting or diminishing an
aspect of it by light, texture and positioning; photography is subtractive
the artist takes the physical world and frames a theme out of it, a part
out of the whole. The kinds of camera, lenses, time of the day or season of
year are manipulable elements that the photographer can use to achieve
his intended visual result.
Painting has its strength on expressing the authors internal world; even
a photo-realist painting translates the artists view of light, color, form.
A painting is built from a clear canvas; it is additive, colors and shapes
are added to build a visual piece. Pigments, solvents, brushes and stroke
techniques are elements that the painter can manipulate on creating his
work.
Both examples - painting and photography - are visual experiences; com-
positions that communicate in the visual realm. But theyre built through
different creative concepts, using different creative tools.
Creative concepts are the mechanics of expression in a medium the ways
that an expression can become reality. Light is a concept in photography;
space is also a concept. Both determine how the visual experience of a
shot will be. In the same way, shapes, colors and dimensions are concepts
40
in painting; each medium has its own set of procedures. The instruments
that allow those concepts to materialize - lenses, brushes, pigments, lm
are creative tools.
Computational Concepts, revisited
As stated initially, computation is about change - and its mechanics feature
concepts like transformation, repetition, randomness and others.
Pure computation, just like pure mathematics, is an abstraction; it does
not necessarily have a component in the visual domain in order to exist.
If a piece of code is written on screen, the written letters are just a visual
representation. The computational language can be reduced to a series
of electrical signals traveling trough logic circuits. The elemental state of
computation is a traveling electron.
It might appear difcult to correlate such abstract medium to the creation
of visual experiences, but we can compare it to cinema; the nucleus of the
moving picture is time the articulation of visual experiences through
time. What is the visual component of pure time? Can time be touched,
seen? Time is an abstraction that applied to the photographic medium
produces montage composition in time, a movie.
We are able to understand pure abstraction by contextualizing it in a
tangible medium. Film, for example, contextualizes time through image
montage - ashbacks, slow motions, time lapses - the lm strip is able
to demonstrate properties of time visually. The pure abstraction of
computation can be contextualized - visually, through pixels, or verbally,
through code. The intangible traveling electron can, nally, generate a
visual experience.
41
Modules
Scale determines if a concept should be understood as a module or as a
system the category depends on the point of view. In a standard user
level point of view, the general computational module is called data.
Data is a computational fact, the declared digital information. The
category of data is called data type. There are two kinds of data types:
primitive and composite.
Primitive data is single information, attached to a value. Numbers are
primitive data. There several categories of numbers, according to the
amount of bytes they take into the computer memory. Logical states (true
or false) and single characters (a, b, c) are also primitive data. The
value of a declared data can be xed, or vary according to a function; data
with mutating values is simply called variable.

1
1.0
1e20
1.77e-3
true
false
hello
{1, 2, 3, 4}
class example
{
var example1;
var example2;
}

42
Composite data is the name given to primitive data combined; a word,
for example, is a combination of characters, and is called string. Arrays
are another kind of composite data: theyre groups. They can be groups
of other groups (nested arrays) number, characters, objects or any other
datatype.
Classes and Objects are composite data. Theyre an important concept
to be grasped, since almost all modern computation is based on Object
Oriented Programming. Object is a philosophical term: it denes
something, a particular item that has properties and relations. A software
object can be compared to a physical world object, or a visual object.
Every object belongs to a class: the class is the template, the category that
denes the possible properties that an object can have.
> this.teaPot
a beautiful physical object a reference for a logical object
things the make a real teapot
be a teapot:
1.ability to hold liquids
2. handle
3. lid
4. spout
class Teapot {
oat size;
oat color;
String lid
String spout
}
class Teapot: things the make
a teapot object be a teapot.
The initial dened properties
are size, color, the kind of lid
and the kind of spout.
43
The main feature of programming through objects is the modularity:
objects can be multiplied, reused, changed, and are independent beings
inside a system.
Another important computational concept is the function. A function
describes an action to be taken, the relation between elements to be
realized; it drives the change on any kind of computational module.
Modules in action: system
Computation is about change; through functions or other methods,
computational elements can be put to work, perform a concept act as
a system.
Lets review some of the systematical concepts of computation:
Repetition
A system is never truly static except when data is stored in a physical
media, its existence lies within electrons. A static module is an illusion: it
is a module that is repeating itself over time, reafrming its existence on
each circuit cycle.
The invisible repetition can be visual, when a graphic element stands
in time - or repeats itself over space. The simultaneous repeated copy is
called instance.
In the following example called repetition.pde, coded using Processing,
a rectangle object is repeated over space - and innitely, over time. The
code also performs two iteration loops to proper position the instances
along a grid.
void setup() {
size(100,100);
ll(0);
}
void draw() {
background(255);
for(int i=0; i<3; i++){
for(int j=0; j<2; j++){
rect(10+i*30, 25+j*30, 20, 20);
}
}
}
44
Transformation
When a property of a module is being changed over space, or time, or
any kind of input, a transformation is occurring; almost everything in
computation is an example of transformation.
The following simple code called transformation.pde increases the value
of the x coordinate of a square over time. When the edge of the screen is
reached, the value is reset, and the transformation starts again.
Randomness
As initially written, true randomness comes from uncertain events in
nature. A known source of uncertainty is the radioactive decay, or the
movement of gas atoms in vacuum. Randomness is a controversial topic
in physics and mathematics, but no technical background is required to
use the idea of unpredictability in while designing. Random algorithms
produce a number inside a certain range; the range can be a space
coordinate, size, or a moment in time. Any property can be tweaked in
random ways.
The following example (randomness.pde) generates a random coordinate
for the square each frame:
int x = 0; // coordinate in x axis
void setup() {
size(100,100);
ll(0);
}
void draw() {
background(255);
x = x+1;
if (x>=width) { x = 0; }
rect(x, 40, 20, 20);
}
oat x; // coordinate in x axis
oat y; // coordinate in x axis
void setup() {
size(100,100);
ll(0); }
void draw() {
background(255);
x = random(width);
y = random(height);
rect(x, y, 20, 20); }
45
Precision
The generation of images through algorithm sometimes requires a great
deal of mathematical processing; certain values go through hundreds,
thousands of calculations and the visual output depends on how exact
those calculations are perfomed.
The case study of the harmonograph (page 78), for example, shows
musical intervals converted to images; the purity of a interval determines
if the image will be symmetrical and stable or not. The visuality of the
whole piece changes with any small deviation of the interval value, even a
0.0000001 unit. The concept of precision thus is valuable when obtaining
certain visual pieces.
Predictability
Predictability comes from nature; for example, its absolutely predictable
that all physical objects on Earth will obey the laws of gravity. Physical,
chemical and biological processes occur with condence that they were
supposed to happen. In a computation system this concept can be
recreated; a function can set behaviors to any module, and this behavior
will happen, always, while the system is active. Creatively, it allows the
designer to create his own rules: like that all objects should vibrate all
the time. Or everything should become red when touched and so forth.
Computer bugs are the outlaws of predictability.
harmonograph: any change in the
interval value creates a different visual
representation
46
Discretion
Being modular and being discrete are almost synonyms; events in com-
putation dont need to be linear; events can happen in modules they can
be handled through any kind of input. A computer code can have a set of
functions, and these functions be called to act any time. A user interacting
with a mouse is an example; when the mouse clicks an item, an event is
called.
The mouse was the event that determined the moment in time where
the action took place; that action could happen again and again or not.
Theres no need for linear time in logical modules.
Complexity
The concept of complexity is to use computation to achieve a degree of
element profusion and element interaction that is intangible to human
hands; based on the principles of repetition and transformation, com-
plexity can become more an aesthetical statement, searching beauty in
the intricate.
Simplicity
Simplicity is the holy grail of computation, since the complexity is a
natural tendency; bulk processing powers opens extensive creative
options. A visual experience can be successful either from complexity or
simplicity.
Peter de Jong
algorithm shown with
2,000,000 iterations.
47
Memory
There are two kinds of memory in a computer system; the volatile, logical
memory where the necessary data for realizing a code is stored, and the
physical storage.
The logical memory has a temporary nature; to perform operations, the
computer stores data of any datatype in its functional memory banks.
A computer can retrieve with precision any item from its electronic
memory.
The physical memory is permanent
1
; the role of time is very clear on the
concepts of computation, and the physical memory is the place where
time stands still. Like a printed book, binary data is printed onto an
optical or magnetic surface, where it can be physically stored.
The creative value of storage is in its power: a database can hold an un-
imaginable amount of information, in any kind of format. The internet
itself is a gigantic database, that can be retrievable through search engines.
The Human Genome Project and the CERN nuclear collision database are
examples of huge unique data storage projects, with billions of bytes that
can be retrieved, sorted and shaped in any way.
Malevichs Black Square and Red Square :
simple and powerful.
1
Subject to physical decay through time; under controlled storage conditions, the decay is minimal.
48
The concepts of computation can be visualized, or verbalized, in countless
forms; the key to achieve visual experiences through them is to employ
artistic creativity in all imaginable or unimaginable - ways.
The issue of generative design
Generative design is a term vastly employed to describe visual pieces
generated through computational processes; but its true concept is not
clear. Between its diverse denitions, it can be used as design generated
by a computer or design generated through a computer very different
ideas.
Can a computer design?
Algorithms can generate visual compositions from ether; visual elements
can be built and tweaked in any amount, following all the concepts of
computation. The illusion that a computer is designing comes from the
capacity of variation and randomness on the construction of new com-
positions.
Sadly, random and abstract are not synonyms. Abstraction is a thought
process. Computers dont think (at least yet).
Ben Frys Haplotype Lines: a creative visualization of the massive genomic database.
49
The computer intelligence is a widely discussed topic; one of the most
famous writings on it comes from Alan Turing, a mathematician that
published in 1950 a test where machines would be considered intelligent
if they could engage a non discernable conversation with a human being.
The validity of the Turing test is questioned today, but no machine could
achieve the victory so far. Despite the advancements on articial intel-
ligence, it remains articial - a mimic, or a source of algorithmic varia- articial articial
tions.
The conception of visual experiences is lead by original ideas; algorithms
can be programmed to follow the principles of visual language pro-
portion, distribution, color harmony; they also can be programmed to
follow style guidelines use certain color harmonies, certain line styles,
and achieve an overall aesthetic sense. But the leading idea remains in the
hands of the designer.
The term generative design shouldnt be used to dene design created by
the computer it should be used to describe through the computer
instead. In a broad sense, any design that is visualized through a computer
can be considered generative, but the usage of the term becomes more
appropriate when describing automated processes - the computer is
programmed to build automatically certain visuals according to a set of
variables and pre dened laws with no need of human interference.

50
the language of the computer is the language of technology,
not the language of design. Without a knowledge of design, the
computer (like the pencil) is more than useless, for it is capable
of producing enough superuous material to create the illusion,
that one is inventing when in fact, one is merely producing varia-
tions of a theme, often of nothingness.
Paul Rand, Design Form and Chaos, 1993
53
Praxis
The concepts of computational design need to be applied somehow; In
order to transform abstract concepts into real projects, tools need to be
used.
The computer is obviously the most necessary tool for the task. It needs
to feature a screen a device capable of displaying an array of pixels,
light elements that are lit or not to build shapes and colors. It also needs
to feature input devices, like keyboard, mouse or joystick. Finally, it
needs the software, a logical layer between the computer circuit and
the designer. The choice of software is a controversial issue: different
packages offer completely different experiences for the designer and
for the audience. Also, every software title imprints different aesthetical
details to the pieces generated by them.
A software title that helps the creation of visual experiences will act as an
interpreter, a translator between the designer and the machine. Different
interpreters will build the translated content in different ways; the overall
meaning might be the same, but the lexical construction will differ. Also,
software packages use libraries of functions pre written formulas that
help the development of a visual piece.
When the functionality of pre-written libraries grow, an effect called
pasteurization might happen: little individual differences in the code
between one author to another are suppressed; the code, thus the visual
(since one generates the other) gain a very similar aesthetic aspect. They
look the same. Commercial packages often break the barrier of canned
aesthetic subtleties and offer completely industrialized, homogenous
visual solutions - templates. Less, easier work but evident less visual
value in design.
54
Forms of Code
Canned or artisan, the software tools provide a clean slate for creativity.
There are many options to design through algorithm and these options
have mainly three categories: compiled, scripted (aka interpreted) or
markup languages.
Their denition is a little fuzzy; the main idea is that a code written in
a compiled language is converted to machine language compiled -
before it is distributed and executed. The nal product does not contain
the original written code, but a translated machine code. For example,
C and Pascal are purely compiled languages. C++ and Java are semi-
compiled, they stand in an intermediate structure between compiling and
scripting modes.
Scripting languages are interpreted at run time; the code is converted
to machine instructions at the same time the program is running. The
distributed/executed code contains the original language written by the
programmer, and it needs the support of a compiled structure to receive.
The performance of scripting languages is worse than purely compiled
ones, but its exibility is greater; Postscript, JavaScript and Flashs
ActionScript are popular examples of scripting languages.
Markup language is the name for content that is tagged to be correctly
displayed and interpreted. HTML and XML are markup languages texts,
pictures or any data with the correct tags for the future interpretation.
Forms of Image
Another important distinction to be noted in graphic software is the de-
nition of bitmap and vector images.
Vector graphics are shapes dened in the computer memory by the
formulas of its form, the formulas of how lines and curves are posi-
tioned and connected. They became very popular on the web thanks to
Macromedia Flash, and very popular in print design thanks to Adobe
Postscript. Vectors can be written as a map of coordinates in two or
three dimensions, creating 3d models. The vector formulas need to be
55
converted to pixels before being shown on the screen; depending on
the software that is interpreting them, different rendering results are
noted, different visual styles are achieved. Pixels are square elements,
and theyre required to represent points in curved shapes; The rendering
engine needs to decide which pixels should t in the curved lines, and also
decide how the jagged edges can be smoothed (also know as antialiasing)
to complement the illusion of curviness
Raster (or bitmapped) graphics are visual les where an array of pixel color
values is stored; all the pixel values together form an image. Photographs
are examples of bitmapped images; bitmaps can be compressed through
algorithms that perform pixel statistic operations, discarding information
that it judges not relevant to the image and making them smaller. Bitmaps
are already made out of pixel information, so unlike vectors they dont
need to be converted to be displayed on screen.
Software
The most common software tools available today for the different code
form categories are:
Adobe Photoshop/Illustrator/After Effects
the Adobe software titles are used as graphic production tools, not
coding environments; but the reality is that algorithms can be written
a vector image, and a blown-upof its pixel representation.
56
and implemented on then. After Effects, the broadcast motion graphics
standard has support to the common JavaScript. Illustrator can
import postscript les that can be generated through code; Photoshop
also can have plug-ins developed, automate functions through scripts
or deal with postscript les. All these coding doors can be taken as
advantage in the creative framework.
Macromedia Director
One of the oldest multimedia authoring platforms; Director is rst of
all, an aggregator: it can unite almost any kind of media asset inside a
project. Director has a powerful, but very extravagant code language
called Lingo - extravagant because its syntax is very different from
the coding standards used by other popular multimedia applications.
Director has full support for real time, hardware accelerated 3D
graphics and is largely used in complex networked projects, where
ash is not powerful enough.
Macromedia Flash
Flash is one of the most successful pieces of software in history; the
initial plug-in was designed to visualize vector graphics in a web
browser, and animate them through tiny le sizes. The original
l996 software called CelAnimator evolved, having more and more
features added every year. Flash is a program that can be used in in-
numerous ways as a linear animator, as an illustrator or as a creator
of applications and web sites. Flash has a scripting language called
ActionScript, in part responsible by the actual movement of designers
working through code. It is necessary to learn ActionScript in order
to build interactive movies; the basic scripting of a site navigation
is an easy task for non programmers which mean a smooth tran-
sition from traditional design to the possibilities of design through
code. ActionScript can be very simple, but it is also capable of doing
complex tasks as the user learns more coding techniques. There are
limits in ash though; the package still does not offer control of the
actual pixels being drawn on screen; the user can only code through
vectors, and the closed ash engine process the conversion to screen
pixels.
57
Java
Java was created in 1995; its initial concept was to be an extremely
portable language that could be executed in any computational device
from desktop computers to kitchen fridges. The Java-enabled
consumer electronics were never fully employed, with the exception
of cell phones, but the technology turned out to be very useful for
network applications including the web. Java is a programming
language modeled after the Object Oriented concept, and is con-
sidered a semi compiled language, designed to be easily ported to
different computer systems.
Processing
Processing a programming language created aiming artistic and
visual applications developed through code. Programs generated
through Processing are usually called sketches, and can be run in any
computer platform, or over the internet. Processing is built over Java,
having added its own graphic library and functions. Java language
has a steep learning curve, and requires lots of technical background.
Processing does not have a graphical input interface; the sketch is
designed in a textual way, but the logic within it is very visual, making
sense for visual oriented users, as artists and designers. Processing
grants pixel-level control; the screen array can be manipulated
directly. Also, it has great extensibility through additional libraries
like sound, midi, HTML or physical interfaces. Almost anything can
be integrated into a artistic sketch project.
Every software package demands time from its user to fully learn all the
concepts, functions and possibilities. The technology of the future is almost
unpredictable, making hard to decide which platform will have more ac-
ceptance and more capabilities in few years. Macromedia Director, for
example, was really popular for the CD-ROM production in the nineties;
its usage has been decreasing, being now restricted to some specic 3D
projects and its likely to become less popular in few years. Other factor
to consider when learning a language is the commercial versus free
software factor; commercial packages usually have, at least in theory,
58
more technical support, because theyre products being sold to customers;
the designer is paying the developers. In fact, free software communities
end up having a good network of users who support each other, even in
a better way that paid application service. The real advantage on com-
mercial titles is the marketing that convinces users to adopt means that
will allow projects to be seen. Recently, Microsoft decided to cut the free
Java as a pre-installed plug-in for the Explorer browser; meanwhile, the
Macromedia Flash player still comes pre-installed, and is present in 95%
of the user browsers.
Using Java/Processing as a creative tool also features the advantage of
controlling the visual rendering process to a low level depth. As said
before, pre-programmed functions end up having an aesthetic inuence
to the visual product; programs like Processing allow the designer
to customize the aesthetic imprint by writing their own set of visual
rendering functions.
still from Bzier Ballet, by Mario Klingemann
(http://www.quasimondo.com).
Unhappy with the Processing default
antialiasing algorithm, Klingerman wrote his
own, capable of displaying the smoothness
and delicacy of thin bzier lines.
59
Mathematical Tools
Each software package has its own features and its own coding syntax to
be learned. But one thing is common among them all: Math.
It is necessary to know some math concepts in order to create visual
experiences through algorithm; and this is one of the reasons that people
are driven away from this kind of creative process. Art education has been
far from Math, and mathematical ideas are seen as impossible problems
by most of designers.
Happily, the truth is that the necessary math for visual creations is very
tangible, and all it requires is the high school math remembrance. 95%
of the creative visual work produced through algorithm today employs
high school level math only, including algebra, trigonometry, physics and
geometry.
The key is not to know complex math; the key is to use creatively the math
you already know. Math is a tool, and the complexity of the tool never
meant the quality of the creation neither in arts or sciences.
The Leica camera used by
Cartier Bresson had only basic
photographic functions. It is
a quality simple instrument,
the tool of Bressons beautiful
photographic expression.
60
Remembering high school
What is the basic math a designer is supposed to know?
A good general review at the high school books is recommended;
sometimes, easy formulas or concepts that may be forgotten can help
to transform an idea into a visual. There are some topics that deserve
special attention; a good foundation on trigonometry, for example, will
solve a great number of issues related to circular motion and visual oscil-
lation. The Pythagorean Theorem, that solves the right triangle lengths
(c
2
= a
2
+b
2
) , will also solve questions regarding positioning, distance and
linear motion along the space of a visual piece. Basic operators (addition,
subtraction, division, multiplication) can also have creative potentials
- since a block of code will loop indenitely, and a number can go through
an operation and change its value on each cycle. The modulo (the operator
that returns the remainder of the division between two numbers) is one
of the forgotten high school arithmetic topics that can have a great use on
computation modulo values are cyclic, they repeat themselves when the
operand is constantly incremented in a loop. Colors, forms and position
can be cycled along the modulo result. The author of this document knows
very little math beyond the topics related above and even so, produced a
series of intriguing mathematical visuals; the best advice to overcome the
Math-phobia is to roll up the sleeves and work on visual projects learn
the math that will solve each specic visual inquiry, one at a time.
Following, a code example built in Processing that shows the application
of modulo and trigonometry; a box rotates in circular motion achieved
by calculating the x and y coordinates according to trigonometric rules;
its color switches to black and white according to the modulo value of the
loop cycle. The sketch can be found on the attached DVD under the name
Trigonometry.pde.
Art and Mathematics are complementary different; that is to
say: they differ each other just as much as they complement each
other.
Karl Gerstner
61
The Code Language
Theres not a universal language for computer coding; each coding
platform has its own syntax. Different languages can share similarities,
but they have different ways to handle the code structure, making them
unique - and requiring unique training.
Someone whos aspiring to write code should choose which language ts
best to his needs; for commercial internet creation, Flash is surely the
best choice today. For artistic experimentation, Java and C++ are better
options, since they offer less limitations and more performance than
Flash. The Processing language has a lot of advantages for the artistic
eld, including strong community support and a balanced learning curve.
There are many other options Python, C#, Objective C all computer
languages can be a source for creative work.
Theres also no single method of learning the code syntax; each author
proposes a different path through the functions and commands of each
platform. A recommendation for the learning process of a visual-oriented
user can be something like:
oat counter;
int radius;
oat x, y;
void setup() {
framerate(20);
size(200,200);
counter = 0;
radius = 50;
}
void draw() {
background(122);
if(counter%2==0) { ll(0); }
else { ll(255); }
x = sin(counter/10)*radius;
y = cos(counter/10)*radius;
rect (90+ x, 90+ y, 20, 20);
counter = counter+1;
}
create a loop counter variable:
create a circle radius variable:
create the x and y coordinates variables:
congure the piece:
set the velocity of the animation:
set the dimensions of the sketch:
set the initial value of the counter:
set the value of the circle radius:
end of the conguration block:
loop starting point:
set background color:
modulo operation:
if the value is even, ll the square with black:
if odd, ll it white:
x axis value according to trigonometry circle:
y axis value according to trigonometry circle:
set the rectangle coordinates and size;
increase the counter value:
end of the loop block:
62
learn to compose a basic code structure setup and loop blocks.
create and congure the visual space for a piece (the stage)
learn the methods of drawing basic shapes;
learn the methods of tweaking the attributes color, size, rotation
of those shapes;
learn how to move things around.
learn how to make the user interact with elements.
This sequence would start out of graphic functions, instead, for example,
the common foundation on Object Oriented Programming that involves
more abstract and less visual thinking.
The Processing platform has a lot of support for learning from the artistic
point of view; the language reference includes simple graphic examples
on the syntax of each topic at http://www.processing.org/learning and
http://www.processing.org/reference.
63
The Praxis of Coding
The process of writing code is similar to the process of writing a narrative
text.
A narration tells a story; there are characters, each character has a role
and things happen as the story develops, coming to an end. The same
narrative story can be written in different styles, different languages;
it can be written in a very erudite form, or it can have a teen literature
lexicon.
A computer code with a determined objective also can have different
styles, and different structures. The style and structure of coding can
be very personal; the way variables and objects are named, and how an
algorithm is written or a problem is solved. But there are principles that
should be followed.
The basic principles of coding are simplicity, clarity, generality and au-
tomation.
By denition, computation is a problem solving task. And there is usually
more than one way to solve a problem; the principle of simplicity is
that the simpler way to solve a problem is the best less computational
resources are allocated, leaving more room for other operations.
Clarity is an issue for both man and machine; code should be clear for
other people rather than the author to read it, and for the machine to un-
derstand it. The clarity is achieved through good naming practices, using
parentheses to clear ambiguities (sometimes its use is not mandatory,
but helps the human reading), commenting the chunks of code, and
describing any relevant information in the program body. Another item
for clarity is the avoidance of contradictions; for example, the usage of
lower and uppercase. A variable named Test and a variable named
test are completely different in a case sensitive environment. If a piece
of code is using a variable in lowercase, all references to it should also be
in lowercase.
Generality is a principle that follows the spirit of object oriented pro-
gramming. A program can have very specic functions for each task;
64
sometimes, these functions are similar, and could have been written as
one and reused. General functions allocate less memory in a computer
system and allows better performance of the code.
Automation is a principle where the computer should handle repetitive
tasks, not the designer. It is an idea related to the principle of generality;
Instead of writing a commands or functions that perform a similar task
hundreds of times, it is better to write one function that operate the same
way with different parameters, saving time on the development and
making the debugging process easier.
A good practice for someone who is not comfortable with the task of
coding (or for someone with a very complex task in hands) is to rst write
the program statements in plain English; list all the actions, variables
and describe all possible functions. With a cohesive and well structured
English text in hands, just translate it to the programming language
syntax and the code is done.
67
Experience
Graphic design which evokes the symmetria of Vitruvius, the
dynamic symmetry of Hambidge, the asymmetry of Mondrian;
which is a good gestalt, generated by intuition or by computer,
by invention or by a system of coordinates is not good design if it
does not communicate
Paul Rand
A Visual Experience involves the audience; designing a visual experience
does not contain only the what to do issue and the how to do issue; it
also contains the whom to do, the concern on how a visual piece will be
experienced by the audience.
The experience is an individual process; each person responds to a com-
munication stimulus in a particular form. The creation of visual expe-
riences deal with probabilities; dening the most probable response to
each presented stimulus, and planning how the visual piece will develop
according to the most probable situations.
Theres no real passive audience; if a visual piece is not interactive, theres
always a choice for someone to see it or not; to ignore it or enjoy it. The
act of ip a page or not, close a browser window or not is a basic level of
interactivity.
Interaction is any reaction to the design stimulus.
A positive reaction is the successful communication. The designer should
compose a piece in visual terms but also in interactive terms; how a piece
will be originally seen? Is there a waiting time to see it? How much infor-
mation is given at each step, each section, each second?
Dosage: the concentration of information on every design node, part,
section, screen considering information every visual, aural or any media
output. The design of interaction deals with the dosage of how and when
things will be experienced along a piece.
68
Regarding the visual terms, the shape of information should raise identity.
Identity means that the each person in the audience will recognize aes-
thetical ideas in common between him and the presented experience. The
recognition can be simple as a I like it response and it does not mean
that the user identies the output as something already seen before,
but as something in his personal taste, a match to his preferences. The
aesthetic values should be shared between both ends of communication,
the designer and the audience.
The development of visual experiences through algorithm opens doors
of limitless customization, interactivity and the generation of customized
information channels: each piece can be designed and react according
to the viewer. Despite of the unpredictability of a human response, the
audience should be acknowledged during the creation process - the
designer should not lock himself in an ivory tower, design is about public
experience design is about communication.
69
70
71
Case Studies
The following projects were developed during the MFA program at
MassArt; they relate, identify and explore issues on the task of articu-
lating visual experiences, and they are the product of a designer whose
education was centered in humanities dealing with computation as a
tool for the visual domain.
72
73
Sound in Space
Music is geometry in time
Arthur Honegger (1892-1955, Swiss composer)
Whats the relation between the aural and visual domains?
Is there an image for sound?
Sound is powerful enough to survive without our eyes. Music is a legitimate
expressway from and to our souls; music strikes the heart with no need
of visual alliances. At the same time, once we add a visual component to
sound, a relationship is formed - visual and aural domains stick together,
building an undeniable bond.
The bond can be formed in two ways; the rst is by converting the
character of the musical signal into a visual representation; Apple Itunes
does this very well. Its visualizations uses the pitch and volume to feed
graphical algorithms. The second way evokes the meaning of music along
with the meanings of the visual domain; this idea is seen (and heard) in
cinema. A dramatic scene has a dramatic soundtrack; dramatic is an
adjective for the musical meaning; the pitches and volumes alone are not
dramatic but the musical movement altogether can be.
The relationship of visual and aural has been explored throughout history
in many different ways; in eastern civilization, one of the oldest scientic
approaches to it comes from Isaac Newton. In 1672, he studied the color
spectrum and related 7 colors to 7 musical notes. Later, Louis Bertrand
Castel built the rst visual keyboard the Clavecin Oculaire that
displayed a different color to each played pitch. The idea of a keyboard
that would display colors for each note reappeared in different places and
moments in time; there was, though, never a rm proof that a certain
74
color is a certain pitch. The relationship always had an arbitrary element
in its formation. Different color/pitch tables were developed, and many
authors agree on the color distributions but there was never a scientic
standard. Intuition and feeling, personal factors, remain present.
Besides color, sound always had a strong relation to math and geometry.
The notable study of this relationship is harmony. Harmony is the study
of the simultaneity of sounds; Pythagoras, more than 2,500 years ago
started to study why some sounds had a pleasant character when played
together. Pythagoras studies had a strong mathematical sense, and he
established the concept of intervals.
Each musical note is a vibration; each vibration has a frequency. Our
perception of pitches, or musical notes, has a cyclic nature every time
a frequency doubles, we perceive it as the same pitch, on a higher tone.
Its called an octave the space between two frequencies with the same
note. Numerically, the octave interval is notated as 2:1 proportion the
frequency of the interval note is 2 times bigger than the whole funda-
mental note. A fundamental note can be divided in any other proportion,
and some of them are used as standards in music, like 3:2 (1.5 times bigger
frequency than the fundamental - fth interval), 4:3 (fourth interval), 5:4
(major third), 6:5 (minor third), so on and so forth.
the proportional lenght between some musical intervals.






75
A mathematician called Jules Lissajous developed an experiment in 1857
that projected a vibration movement through a light dot in a screen. He
managed to reect the light beam through 2 mirrors, each one vibrating
in an axis, vertical and horizontal. The resulting projected shapes became
known as Lissajous Figures
Different vibrations could be applied to the mirrors, producing different
shapes; musical notes are vibrations, so different pitches could be
reproduced and visualized - and when different pitches were projected
together, musical intervals could be seen.
The Lissajous gure might be the most scientic approach to a visual
representation of sound; it is based on the physical properties of musical
notes that cannot be easily contested like the previous color/sound ex-
periments.
original Lissajous experiment
76
The Chordinator
The Chordinator re-creates and expands Lissajous experiment as an
interactive computer application. Lissajous projection was limited to one
single interval; two notes, because his physical projection system could
only be bi-dimensional. The starting point of the Chordinator is to take
advantage that a computer system can create the illusion of depth easily.
There are 3 spatial dimensions that the human perception can discern:
height, width and depth. Musical chords are 3 (or more) notes played
together; two intervals in the same period of time, in a chord structure (
root, II degree, III degree, the button labels) The idea is to assign a spatial
dimension to a sound frequency, and explore the plasticity of sound as a
sculpture.
Chordinator interface screen
77
The sketch allows the independent control of the intensity of each chord
component, as well as its phase. Phase can be understood as the coordi-
nation point between sound waves
To help the sense of 3 dimensional space, a cube was added marking the
capping of the volume; rotation was added so the point of view can change
over time, allowing a full visualization of sound in 3D.
Despite of the strong sound concept, the Chordinator experience aims to
sculpt a visual structure over time: the keyboard is controlling a form in
space, not music.
Production notes
The Chordinator employs a very simple programming concept: parametric
equations, where independent equations drive different coordinate axis.
The musical note visual component is a graph plotted through time; each
coordinate is being fed by an algorithm that determines the amplitude,
pitch and phase of sound, independently. Theres an independent
module that creates the cube and another that controls the interface. The
algorithm follows trigonometric principles, where the sound vibration can
be expressed as a sine wave, with dened amplitude, length and period.
The 3D visualization was built in Processing, and the keyboard interface
in Flash. they both work together in a browser window, allowing this
visual experience to be shared online.

phase example: three sound

waves with same amplitude
and wavelenght with different
alignments in time.
78
The Harmonographer
The harmonograph was an invention built and sold in the nineteen
century and it consisted in pendulums attached to a pen; the pendulum
movement set the pen coordinates, drawing the resultant of two (or
three) harmonic motions. The nal drawing is a frequency interval, like
the Lissajous experiment; the pendulums also had the nice feature of
simulate the damping of a musical note, the slow decay to silence, thanks
to the mechanic energy that is dissipated within the motion. Another
feature of the harmonograph was the use of rotary motion; it added a
new dimension and perspective to the visualization. The frequencies were
more dimensional, more representative of their physical nature in space
han the Lissajous projection.
Harmonographer interface screen
79
There was a strong relation between musical harmony and visual
harmony in the interval drawings of the Harmonograph; perfect intervals
were symmetrical and visually stable; numerical approximations in the
frequencies produced more unbalanced visuals, with a more intricate
nature.
The Harmonographer project is the translation of the physical device to
the computer; it has the feature of dialing in the musical notes, obtaining
the correct intervals. It also simulates the musical damping, and the
phase oscillation.
The images produced by the Harmonographer should be understood as
drawn from the outer edges (maximum volume) to the center (silence)
- the path of decay. The constant rotation of the drawing has a purpose:
show the interval at all possible sound coordination points, and its
circular motion comes from the phase value. The phase is a cyclic event (it
repeats according to the length of a wave), so theres a constant rotation
impression to the coordinates.
The sketch also features acceleration from one interval to another
and variable interval change speed; the function of both is to induce a
smooth transition between different visuals, letting the user know what
is happening from one note to another or just y through all possi-
bilities.
an original Harmonograph
80
A Portamento button is available to turn on the innite sliding between
notes; in the portamento mode, the interval grows eternally, and the user
can keep on watching and observing how the intervals grow and their
visual elements change with the growth.
The main objective of the Harmonographer is let the user explore the
interval relations through their visual representation; or simply explore
the visual patterns and their mandala-like formations, tweaking the
possible graphical formations that the algorithm can build.
A special version of the Harmonographer was also built, that allows the
user to export the drawing as a vector le, that can be used in any graphic
application or printed. The user can also save the drawings as bitmaps,
and create video animations.
Production Notes
The coding behind the harmonographer is based on parametric equations,
independent plot instructions for x and y values. The code is slightly
more complicated than the Chordinator, since there are formulas for the
damping - that use the harmonic damping formula from physics books.
The damping was implemented as an algorithm, and the phase rotary
motion was added as well - using again the trigonometric circle concept.
The code has separate sets of acceleration and portamento functions,
and all the interface commands function, along with the Flash virtual-
keyboard element.
81
The Near Dorian Experience
How can a musical piece be visualized through the harmonograph?
To answer this question, a video called The Near Dorian Experience was
developed, visualizing an original music composition.
The musical soundtrack intentions for the Near Dorian were to work
with few and strong musical intervals, and create a hypnotic trance of
repetition - but at the same time calm and meditative. Theres one guitar
with a mellow, woody tone and another with a twang slide sound. The
percussion is nothing more than a triangle being hit twice in the music
and the base sound is one single chord being played in a Hammond
organ, oscillating through rotating speakers. The intervals explored in
the song were pre selected through their visuals the interesting visuals
had a granted place. The video has layers for the different instruments
and their visuals; color was added, in the form of blue tones denoting
the cold electric resonance, and yellow/orange denoting the warmness
of the guitar strings. The white denotes the purity of the bell sound, the
Near Dorian Experience frame still
82
moment of illumination. Of course, this description of the soundtrack is
from a author standpoint, and the actual experience of musical meanings
may vary.
The piece shows the coordination between aural and visual intervals. The
title, Dorian, is a musical mode of the natural musical scale, known by its
simplicity. Since the song escapes the scale in a part, and the visual escapes
the simplicity due to the overlaying, its a near Dorian experience.
This project has the conceptual importance of being a completely
authorial project in new media the author developed the expression
in all domains: visual, temporal, musical, and logical. All the pieces were
original and neither contained parts from other sources or projects.
Beyond that, it has the proof of possibility to develop a concept in the all
domains at the same time.
A software application was created to achieve the visual of the video,
and it couldnt be done in another way; even if a physical harmonograph
was constructed, for example, it wouldnt animate the phase variations
of rotating speakers - there are features that can only be achieved in a
logical medium.
The sound in space project was essentially pseudo-scientic; it had sci-
entic fundaments, but embraced creative freedom to show a personal
view of the spatial information of music. The personal view was expressed
through a computer program that helped the development of a song and a
video - the freedom of articulation came through algorithms.
83
85
Form Systems
A visual composition is formed by individual elements; these elements
interact. The study of how and why the elements in a composition interact
is the subject of visual language.
Many artists and writers published their research on the nature and prop-
erties of visual language, and one of these authors is Karl Gerstner, Swiss
painter and graphic designer. Gerstner has a remarkable point of view
regarding form and color according to him, theyre body and spirit.
Gerstner considered math an aid to ultimate master of ones medium.
His most notable characteristic is the usage of structure as a mean of
expression; many of Gerstner art pieces were not xed compositions,
they were xed formulas where the components could be interchanged.
Although Gerstner didnt employed computers, he created artistic algo-
rithms; his pictures are variation machines. This computational spirit is
the inspiration for the series of form systems experiments in which the
nature of variation is explored.
Gerstner work is the inspiration for a series of experiments, where
concepts of the computational medium that connect with the idea of form
as a system are explored.
The Monkey+Gerstner and No_More_Mondrian
A good example of Gerstners art is the Carro 64 piece. It consists in
64 aluminum blocks, in 16 colors (a 16 step color gradient) that form a
do-it- yourself composition: the blocks can be arranged in any way. The
arrangements can follow symmetrical operations or be random; there are
trillions of possible combinations in the random mode, but the number
of options is smaller when compositing laws are inserted, forming deter-
86
mined color patterns.
As an experiment, I rebuilt Carro 64 as a java sketch where the compos-
iting possibilities would be alternated but never repeated. When trans-
lated to the electronic medium, the blocks location exchange is effortless.
The canvas becomes an almost innite art generator, rebuilding the color
blocks in a different conguration on each determined time interval.
Are the color compositions interesting? Eventually. Being random, theres
a random possibility that an interesting or visually attractive composition
Carro 64 advertisement
87
will emerge. An interesting experiment was built on the web by the
software company AardAsNails, called Monkey Shakespeare Simulator
1
.
Launched in 2003, it proposes to prove the mathematical innite monkey
theorem, by the use of bulk grid computing and random letter string gen-
eration a monkey simulator, that would run on each user computer.
If you have enough monkeys
banging randomly on typewriters,
they will eventually type the works
of William Shakespeare.
The record so far is 24 letters from King Henry II and it took only
2,737,850 million billion billion billion monkey-years to be completed.
Eventually is a tricky word that tends to the impossible. Maybe thats
why Gerstner, even though creating an innite art form, exposed congu-
rations with carefully planned symmetric harmonies that required plenty
of artistic sensibility to be achieved.
The point of this experiment is to explore the role of authorship in the
so called generative design; the absence of an authors guidance creates
a monkey situation for the machine. Theres value on the concept of an
eternal change art canvas, but the isolated result of the variation is not
guaranteed. In a form system, the author can create rules, directions
and behaviors that will channel the creation interesting levels, instead of
abandoning the visual dimension to its own random generative luck.
the original Carro 64, and one of its random computer recreations
1 http://user.tninet.se/~ecf599g/aardasnails/java/Monkey/webpages/index.html#results
An experiment with real monkeys was also conducted with hilarious results at
http://www.vivaria.net/experiments/notes/publication/NOTES_EN.pdf
abandoning the visual dimension to its own random generative luck.
88
The other side of a generative form system occurs when a complete, vast
set of rules is laid to control the unpredictable randomness. The uncertain
composition is tamed, but the absence of the monkey factor can also
create banal pieces that always look the same.
The predictability usually occurs on systems designed to mimic an
abstract process. During the early days of graphic computing, Michael
Noll developed experiments where a computer was programmed to
mimic Mondrians Composition with lines. Nolls conclusion? Viewers of
his experiment could not tell whether the painting was created by the
human Mondrian or the electronic Mondrian.
Recreating Nolls experiment in Java has brought me some interesting
conclusions; rst, its complex to mimic a human artistic behavior.
Second, its really easy to create a sketch that draws geometric forms in
random patterns, similar to many abstract paintings but no abstract
painting is like that. Even simple geometric paintings deal with a complex
mental process of decision making: should a line be on the left or right?
Should a square be big, small or medium? What will be the horizontal to
vertical lines proportion?
To answer these questions, rules need to be written in the code. Rules that
determine average size, average proportion, placement limits, so on, so
forth. Even a minimalist painting mimic needs a great amount of rules;
the more rules, the closer the generated image will be to the original but
it will also be more predictable. The image wont bring new, interesting
formations that could exist outside the boundaries of a certain rule.
Any painting can have an innite set of rules they can grow in com-
plexity, more the rules, less the randomness, more faithful to the original
generative design processor
89
artwork. For the No_More_Mondrian sketch only a limited set of rules
were written, a number enough to resemble the original experiment, and
the sketch runs forever always trying to add a new layer of ruled shapes.
Dull, as expected.
The Squircle
The key to creating generating form systems is striking a balance between
rigid rules and anarchic randomness.
The Squircle is a small experiment toward nding this balance. Based on
Gerstners Color-Form Continuum, in which he explored transitory se-
quential states between different shapes and colors, the Squircle explores
all the shapes that lie between a circle and a square and the visual compo-
sitions that derive from them.
The Squircle was built using the formula of Bzier curves - a common
tool for Postscript, CAD and vector illustration applications. Bzier curve
is a parametric cubic equation that plots a path with initial and nal
points inuenced by control points along the way. The control points can
be tweaked to change the overall shape of the curve. By coding a closed
shape with symmetrical control point changes, the result is a regular
polygon whose sides bend and morph in all possible positions - the
clash of generations: the original Noll experiment and the actual Java recreation.
90
screen captures
from The Squircle
and Gerstners
Color Form
structure model
91
square and the circle are the two extremes of the possible forms. Instead
of copying a Gerstner piece or simply creating all possible shapes in a
random fashion, the Squircle uses an oscillatory pulse that changes the
control points position, and the amplitude of the change can be set by the
mouse movement. The sole purpose of the piece is for the user to enjoy
creating forms with symmetrical aspect and experiencing the possibilities
between two basic geometric shapes.
Interactivity is a useful tool when theres a quest to create an endless form
system that is not random; user interactivity determines a composition in
the Squircle, that is a unique and endless, but not random.
The Anaconda Paint
In another experiment, Anaconda Paint, the user builds a volumed form
according to mouse coordinates. The visual composition is obtained by
varying the color and position of a circle, with either a dented or plain
stroke texture. The illusion of a dimensional painting is created in a at
surface by overlapping circles through time. To enhance the organic
aspect of the composition, the shape moves with acceleration toward the
cursor, avoiding abrupt movements.
It looks better in motion.
92
The trick that makes the Anaconda Paint interesting is the color shifting;
all the colors in the screen are constantly changing, in a cyclic way. Each
circle is assigned with a different initial color and when all the screen
colors cycle, the illusion of volume is reinforced.
The Anaconda visual experience is supposed to be fun to play with, letting
the user create endless snake gures. This piece is completely different
from the Monkey+Gerstner, but both have the same concept: endless
compositions, based on a xed shape, with multiple instances and varied
color and location. There are innumerous ways to translate a single
concept.
Roto-Champ
The idea of keeping a shape xed, but to vary its color, size, and position
over time is full of possibilities; the Roto-Champ experiment explored
another point of view on this theme. The project is based on Marcel
Duchamp Roto-reliefs, a series of gramophone discs painted with patterns
that would rotate and create unexpected kinetic results. The Roto-Champ
is an exploration of Duchamps concept translated into the computational
medium.
The shapes have xed form and color, and rotate at a constant speed.
Illusions of volume and form distortion are created by the visual inter-
action of the concentric rings, just like the original piece.
The Outline Anaconda
93
The experiment has two different approaches; the rst regards the
geometric nature of the circle. The circle is a pure mathematical form,
and its presence in a plain color background has a strong abstract and
ethereal sense. I questioned how to add more physical, concrete value
to this form - and the answer was to tweak the relative size and rotation
of the rings. The values inserted were transformed according to a value
very common to Botanic: golden section . The sacred mathematical pro-
portion is found on the growth of plants, on the patterns of seeds and on
nearly every natural organic form. Both size and rotation would respond
to organic values, linking the ethereal form to a more physical dynamic.
Marcel Duchamp invites you for the Roto-relief experience Roto-relief Roto-relief
Roto-Champ: motion blur version
the proportion resulting from the division of a straight line into two parts so that the ratio of the whole to
the larger part is the same as the ratio of the larger part to the smaller Oxford English Dictionary
94
The second approach deals with interactivity; with one mouse coordinate,
the user can change the perspective of the piece something that would
happen in the original Duchamp piece, since the perspective would change
according to the position of the viewer. By visually cropping parts of the
concentric gure, the illusion is changed and different visual impressions
occur. The other mouse coordinate was used to displace the center of each
ring in relation of its inner sibling, generating then different frequency
patterns, and different aesthetic experiences.
Roto-Champ: golden proportion and standard versions in perspective.
95
My Friend Jong
Form systems can be taken to extremes; as in the case of strange
attractor visuals. This kind of visual has been explored by a number of
designers and scientists, and it is known by algorithms that plot points
with an apparent random, chaotic, order but that gravitate around
certain xed values (hence the name attractor), generating interesting
visuals. My own recreation of a strange attractor used the Peter de Jong
algorithm, written in 1991 by Paul Bourke. It was chosen because of its
simplicity - a small mathematical abstraction that can lead to complex,
visually rich results. The Jong algorithm simply subtracts a cosine from a
sine value, in both screen axis (x and y).
The trick is: given a starting point, the sine and cosine values are calcu-
lated based on the previous operation result, in a loop. This loop is called
iteration, and the number of iterations can be predened ranging from
1 pair of coordinates to billions. Organic patterns emerge from the plotted
dots the cold mathematical abstraction is transformed in a shape that
looks and behaves like organic dynamic forces, as the smoke of an incense
stick. Given a smaller (few thousand) iteration cycles, the attractor can
be animated in real time, or respond to user interaction, like mouse
movement. Attractors are systems with little visual control (they have an
unpredictable nature) but are a good playground where small changes can
be multiplied and re applied millions of times. The concept of iteration
can be used as a creative tool theres not much control over the form
itself, but on how the form will emerge and unfold through iteration. This
is an important abstraction on the realm of generating images through
code: sometimes, a visual cant be grasped; it has to be tamed through the
numbers that feed its creation.
Peter de Jong - algorithm by Paul Bourke.
x
n+1
= sin(a*y
n
) - cos(b*x
n
)
y
n+1
= sin(c*x
n
) - cos(d*y
n
)
96
My friend Jong: 150,000,000,000 iterations.
(I counted them myself to double check if there was any missing.)
99
Riley Patterns
Op art is an artistic movement that blossomed in the middle sixties,
through the work of artists like Victor Vasarely and Bridget Riley. The
main characteristic of op art is a sensorial overload; the perception is
challenged with tricky impressions of volume, color and motion. Op art is
extremely kinetic - but in a perceptual, and not physical level.
Another trademark of op art is the creative use of patterns; visual
rhythms with distortions and modulations that conduct the composition
experience. The use of patterns and modulations usually follow a dened
set of rules a strong sense of algorithm.
Riley and Vasarely works were not computational their creations were
set in a time when computers where scientic secrets. My intention is
to investigate how optical visual patterns can be created and modulated
through code, and explore the computers advantage as a variation
machine: visual ideas can be developed in endless ways, especially when
the idea has an algorithmic nature. What are the changes in op art when
it is reclaimed by the computational medium?
Baba O Riley
The art of tomorrow will be a common collective treasure or it
will not be art at all.
-Victor Vasarely, 1960
Vasarely professed the mass media possibilities at an early date. In which
ways can art be collective?
Baba O Riley is an attempt to answer this question, following Gerstners
spirit of having structure as art. Baba O Riley contains no composition: it
contains the structure for endless compositions. The authorship is shared:
100
the designer authors the aesthetic framework; the audience experiences
the creation of a composition from that framework. Its a true form of
collective art everyone can do it.
The user has the ability of drawing lines and circles, with different color
frequencies - always alternating black and white. Even being just a
computer application experience, the decision process on how and where
the drawing elements should be placed is similar to the physical painting
creative process. The medium is different, but the abstract thought
process is alike
The exibility on instantaneous composition is interesting for the artist/
designer; it allows painless experimentation. The successful tryouts
can be saved, the unsuccessful discarded and the saved ones can even
become sketches for physical paintings. One of the major difculties of
creating physical real world op art creations is the technical production
of visual patterns; they require a lot of effort and planning. A computer
application can help a lot the process of planning, easing the technical
barriers and amplifying the creative freedom.
Baba oRiley
interface screen
101
Development
Baba O Riley was developed from scratch, programmed into the
Processing environment. It involved an interesting aspect of computer
graphics: drawing a circle. Processing was chosen because it permits
great control of the screen; pixels can be drawn and manipulated directly,
feature absent in programs like Macromedia Flash.
Designers take the computational circle drawing for granted; in Photoshop,
Illustrator, Flash or even on the simple Windows Paint, theres a tool that
allows the cursor to draw the basic gure. But how does the tool achieve
that work?
That was the starting point of the investigation. Pixels are square; the
screen is a big group of squares. How can such a round gure exist in this
harsh environment?
The rst answer: it cant. A completely perfect abstract curve cannot be
drawn to the square based screen. Its necessary to build a function that
returns the coordinates of the pixels that better match the path of a curve.
Curve lines on a screen are approximations; thanks to the resolution, they
look acceptably good.
The principles of trigonometry grant the knowledge of discovering such
coordinates; once the radius of the circle is known, the Pythagorean
Theorem can be applied to every pixel in the x coordinate it will
return a matching y, creating coordinate pairs the rst step for a circle
algorithm.
Unfortunately, the resulting curve line is not continuous; the frequency of
y coordinate steps wears thin toward the edges the pixels imprecision
appears. The second step of the algorithm is to ll those gaps with
square pixels failing to follow
a curved path
102
straight lines. The result is a perfect circle built through inconstant
coordinates and straight lines, in a computational illusion. This is the
trick that Photoshop does all the time for its users.
The third aspect of the algorithm is to build the curve with the right color.
It rst evaluates the present color in the x/y coordinate, and redraws
it with the opposite value the law of high contrast is followed all the
time.
The remainer of the code deals with mouse/keyboard response and
interface parameters. The program is very simple, and although there are
much better techniques of implementing the idea, it uses the simpler ones
in an extensively commented code. Only one basic mathematical concept
was employed proong that just a little math is already very useful to
express ideas through code.
The Baba ORiley project was a success in terms of audience the public
enjoyed the online experience (2,000 unique visitors so far) and it shows
how an interactive art project can be technically simple, but visually in-
teresting at the same time.
Current; Flow
One of the rst computer graphic images in history, along with Mondrian
experiments, was a recreation of Rileys Current piece, built by A.
Michael Noll, called Ninety Parallel Sinusoids with Linearly Increasing
Period
2
. I decided to explore the Nolls experiment, due to its historical
value and also due to its visual value; Current one of the most interesting
op art paintings.
Current consists in a repetition (for Riley, repetition is a mean for ampli-
cation) of waves with increasing periods (the length of each vibration),
and constant amplitude (the height of the wave). Its a very well math-
ematically dened piece, and maybe thats the reason with Noll chose it
for his early computer experiments.
The objective of the rst experiment was to explore the multiplicity of
means to an end; Nolls experiment was rebuilt through different com-
2
Nolls version was actually a recreation of Plate 3, another painting from the Current series.
103
original Noll experiment
line polygons Catmull-Rom curves
Bzier curves pixels
Current: many means to one form.
104
putational techniques different processes for a single end. The visual
sketches were built using line polygons, Catmull-Rom splines, Bzier
Curves, and nally, direct pixel manipulation. Also, the mouse coordi-
nates were mapped to tweak amplitude and wavelength settings, helping
to mark visually appealing values.
Each technique has strong and weak features. The rst sketch used line
polygons, which are fast and easy to manipulate, but are not real curves
theyre small line segments that resemble a curve. Working with
straight lines requires less processing power, but the fake nature of the
curves might become visible. The second implementation used a curve
called Catmull-Rom; it is a curve formula, where the path goes through
the control points. Also, a control point in the curve line affects only the
segment where it belongs, and not the whole curve. Catmull-Rom curves
also have easy implementation, and are fast. Closed sinusoidal shapes
designed with Catmull-Rom are difcult to be properly lled, at least
using Java ll algorithms; the ll functions have a hard time determining
which portions of the sine wave should be convex or concave to apply the
correct lling.
Catmull-Rom is an appropriate solution if the target code is not supposed
to become a print piece. This kind of curve is not native to PostScript, as
the Bzier curves are. For a truly cross-media piece, the solution would be
to use the famous Bzier lines.
Its harder to achieve the perfect variable period sine wave using Bzier;
while the Catmull implementation was effortless, this required a lot of
planning and calculation on the control points positioning. It also required
many more control points to achieve the right smooth curve visual; on the
other hand, the sketch could be the size of the screen or the size of an
ocean liner, since the vector is size and resolution independent
Finally, the pixel implementation: the best way to avoid the difcult lling
of sinusoidal shapes is to draw the pixels directly. Instead of drawing a
sine shape and lling it, I can set the screen pixels with different colors
according to the form of a sine wave. The pixel version is not suitable for
printing it is limited to the 72 dpi screen dimension but it works ne
on screen and despite not being optimized for speed, it runs the lled sine
transformations much faster than the vector versions. Another difference
105
is that while vector drawings can have anti-aliasing functions to work on
top of them, pixel manipulation cant the antialiasing function has to be
written specically to the sketch.
The Current process exploration helps to demonstrate not only that a
visual idea can be unfolded in different ways, but also that the chosen
technique inuences the nal aesthetic of the piece: the speed of motion,
the antialiasing algorithm, the curve rendering are unique in each
version. The concept is drawn on screen in a similar way, but the style of
the drawing is inherited from the coding techniques. In a computational
visual experience, the process matters.
The second part of the experiment dealt with the translation of a concept
from a gallery wall to the screen. Riley had very interesting ideas about
her painting she says that her paintings arent about optical illusion,
but about optical reality. Riley explored the relationships between nature
and vision, and how the dynamic forms of nature could be represented
in their essence. Current has a very organic motion feeling into it, and I
wondered how to translate this feeling taking advantage of the computa-
tional medium.
The Flow sketch was the result of this exploration. It has two versions;
an interactive, where a user input whether the mouse or any physical
device triggers different wave settings and an automatic version, where
the waves oscillate by themselves. The sketch intends to capture the tran-
sition from a geometric state the straight lines to an organic behavior
resembling water currents. Dots were used instead of lines for more
delicacy, and also to imprint the idea that a current too strong becomes
noise; the dots mix themselves and the line pattern disappears.
Flow is a very simple sketch; it has few lines of code and uses line vertex
no curves and is even suitable for printing in any size. A simple
concept, simple implementation, interesting results.
Flow was also converted to an openGL version; openGL is a graphics
library that allows geometry to be calculated by the computer graphics
card, with increased performance in relation to the computer main
processor screen rendering. The Vanishing Flow mixes the renaissance
perspective with the sixties op art, creating a different aesthetic expe-
rience from the at version.
106

The op art explorations are product of my personal pursuit on exploring
and expanding this traditional art form to new possibilities; visual patterns
are great form systems that t naturally into the creative concepts of
computation, and my role was to make the natural inclination become
true.
Flow: dot and perspective versions.
109
Logic Molecules
Database: a collection of information, stored in a retrievable system.
The database concept is the foundation of the internet today; the web is
formed by uncountable servers that store all possible categories of data,
constantly retrieved by browsers worldwide.
What does a database look like?
A database has no face; physically, it is stored in a binary mode, like any
digital information - theres no difference between a music CD and a
complex relational database source in its physical register. Logically, it
has no face either. The interface that manages the relation between user
and database has a face but it is a middle man, an agent, and it can be
simple as a blinking command prompt.
The database is intangible. The nearest a human can get to it is manipu-
lating a query program, that will dive into the structured information and
return with a completed request.
Amazon, Internet Movie Database, eBay, Craigslist, Google: the major
websites serving content and services are a shell for database querying
engines all their power lie on the amount of information they contain,
and how the information can be retrieved. How big is Amazon? How big
is eBay? Where does it begin, and where does it end? How can a database
system be understood in a graspable form?
The Particle Database project investigated this issue. The intent was to
connect the idea of database to a visual materiality, and raise the identity
between databases and the physical world.
The database is a mass with different categories of information in different
amounts. It has a similar structure to physical matter: a mass built by
different molecules, from different categories in also different amounts.
When a database is queried, the relevant bits of information are retrieved
by a query agent; information is added and removed of the system as well.
110
screenshots from the Particle Database querying system.
111
Building a parallel to physical matter, a database would be a exible, uid
material where the molecules could react and be attracted by different
agents.
Another feature of physical matter is density: the amount of molecules
per spatial area. A database representation would have the screen as its
total spatial area, and the density would be a key idea to represent the
amount of information populated in a system.
A potential viewer of the physical database would be able to directly
manipulate it, molding the visual data according to particular queries.
Two concepts were developed for this task; rst, is the compliance to the
mechanical laws of physics. Data molecules would need to be inuenced
by gravity, and their mass generate gravitational elds. Acceleration and
inertia would need to be present by reecting the real world, a sense
of comfort is generated in the user experience, the comfort of being ac-
quainted to physical rules like the known real world. Second, the idea of
chemical agents; database queries would be formulated and assume a
physical form; like a magnet or a chemical compound that would react
attracting certain types of data and repelling others. Information could
be attracted and repelled manually, allowing a sense of scale between the
amount of attracted molecules and the whole molecule universe.
The molecule/universe relationship has a clear metaphor in the compu-
tational medium: object/system. A molecule, with its components and
properties, is similar to the logical object - an element with a series of
functions and properties bound together. The concept of universe, the
multi dimensional space where molecules exist, is the same concept of a
computer system, the space where logical objects exist.
Objects are abstract data concepts; they can be modeled to contain any
visual presence on screen, and respond to any physics algorithm. Besides
an arbitrary visual representation, an object can contain actual entries
from a database.
There are scientic applications where a large number of objects with
properties need to be displayed and manipulated on screen; wind and
aerodynamics simulations, atomic reaction research, astronomical
models and many others. The name particle system is given to the style
112
of representing a large number of individual objects that have dened
properties, and that interact according to common rules.
The Particle Database project followed the visual style of a particle
system, having each node of information represented as a glowing pixel.
The pixels would stand grouped in a screen sized pixel sphere the
universe, with variable density. The attraction/reaction agents would
make the particles ow around; other tools would allow certain particles
to be hidden, labeled or reveal their data content on screen. The project
was built as animated demo, non functional, where an example query is
done to envision the advantages and exibility of direct manipulation of
data.
Coded Particles
Particle systems are the most explicit example of object oriented pro-
gramming; a logical object does not need to be visual to exist but in a
particle eld, each pixel is an object instance. The task of coding a particle
system consists in coding the behavior of a particle kind. This kind is
called class, the group of rules, characteristics and behaviors of an object.
The class describes all topics in the life of a particle its appearance,
motion and dynamic responses for different stimulus caused by the user
or other particles.
The Database Field sketch is a study of how the Particle Database would Database Field Database Field
be coded; for experimental purposes, attracting and repelling behaviors
were written, and the particles behave according to a faux Brownian
motion random in space. The sketch adds the idea of the real interac-
tivity, absent on the application demo.
interactive particle attraction/repulsion tests
115
Patterns of Time
Edwin Abbott, English writer and clergyman, published in 1884 a short
book called Flatland. The book is an interesting and very early exercise
into the perception of space; the human being lives in 3+1 dimensions
3 spatial planes and a punctum of time. The characters of Flatland live
in a 2+1 universe; thus the atness of the title, the completely eliminated
perception of volume.
It is an interesting exercise to alter the relation between spatial and
temporal dimensions; the Patterns of Time project explored an inversion
on the perception of time and space; instead of 3+1 dimensions, 1+3. How
can we expand time in more dimensions, while compressing space to a
punctum? What is the spatial component of time?
The computational medium is a good support for this kind of experiment;
it allows great exibility for experimentation, and helps the imagination
in such a entangled topic. The project proposal is an interface that allows
the compression/expansion of dimensions to occur; it does not intent to
be scientic, but poetic.
It is easy to imagine all the special dimensions compressed to a point;
timelines do that. A timeline transforms all the physical events to a
printed point; time becomes a line, dot after dot representing day after
day, year after year. But is time a line? The characters of Abbots Flatland
had the rm belief that a third dimension didnt exist at all things were
at by nature. The visualization of time as a line of events might be stuck
in a Flatland mentality.
Pattern is a concept of many denitions, but they all agree in the idea that
a form may exist and be recurrent according to a rule a conguration.
Patterns are everywhere in nature the conguration of clouds, waves,
the growth of plants, structure of minerals. Common laws sew different
realms together, binding them by mathematical concepts like golden
ration and prime numbers.
opposite page: Patterns of Time Studio openGL version
116
The idea of pattern is usually employed to describe spatial or logical
events; the project idea was to extend this application to time as well
escaping temporal Flatland.
What are the components of time?
According to a personal point of view, corroborated by the quantum
physics and approved by poetic license, time is light. Light is a periodic
wave vibration, thus time probably shares the same properties of periodic
waves - known as amplitude, wavelength and phase.
What is the amplitude, wavelength and phase of time?
Phase is the point of the amplitude at which a wave begin. Applied to
time, phase should be neutral time starts at the beginning of time (a
recursive tautology). Amplitude and wavelength are related to the notion
of facts.
What marks the passage of time? Facts
1
. The happening of facts in the
physical world is a beacon for proong the time ow.
Each fact in time is unique
2
; but facts share similar congurations. Facts
form patterns; theres a summer in every year its a predictable event.
Each summer is unique but events that characterize the fact summer
happen every year.
In different scales, all facts have patterns of occurrence some of them
are obvious, other subtle.
The facts in time are points in the amplitude wave. Each fact follows a
pattern of occurrence, so each fact has its own frequency, wavelength.

1
The world is the totality of facts Wittgenstein, in Tractatus
2
Upon those who step into the same rivers different and ever different waters ow down. - Heraclites
117
How to represent the concept of patterns in time visually?
The developed model of time used the line as a starting metaphor, added
with the pattern idea. The facts have a frequency of occurrence, so they
can have a cyclic nature. The end of each cycle does not mark the return
to the beginning; the line of time continues to the future. So the cycle is
not a closed circle it is a line with a coil appearance.
Different patterns have different frequencies; bigger patterns encap-
sulate smaller ones like an year encapsulate 365 days. So the coil is not
absolute it is formed by another coil; which is formed by another and
other in an innite progression.
The computer is a helper to achieve such concept in a visual way; the
initial project was done after research on coil algorithms, and how could
they be applied. A recursive coil model was developed in 3D, and a Flash
demonstration of the concepts, was built around it. The Location of
Time is a demonstration of the concepts of time and space, and it was
projected to be the initial point of a functional interface.
Patterns of Time Studio: interactive online version
118
Patterns of Time: concept demo screens
119
The interactive version of the Location of Time was partially coded at
the cost of much sweat since the concept is very challenging mathemati-
cally. Innite levels of recursiveness in a three dimensional structure is
not exactly the everyday problem solving task of a designer. The rst levels
of time recursion can be explored through different points of view, and
different time line compression. Technically, the code describes a big three
dimensional polygon with variable sides, and variable vertex frequency.
The user can tweak and experiment on every possible parameter of the
timeline and the code is suitable for screen or print, in any given size.
Patterns of Time Studio: openGL version
120
121
Findings
As a result of creative process, the product of design has its structure
formed by concepts, form and medium. Concepts are ideas, consolidation
of the human imagination. The concepts become real through form (the
aesthetic faculties). And form needs the support of a medium, wherein it
can be expressed.
Noteworthy design work contains a sense of equilibrium in those com-
ponents. Such work can usually be characterized as based on strong
conceptual thinking, executed by a pleasing aesthetical form within full
employment of the medium.
We in live in a moment in time marked by the proliferation of the personal
computer and the fast growth of the internet - a computer network as a
medium. The computer, as a vast new territory for design, is offering an
interactive and dynamic space for creation.
medium
form
concepts
medium
form
concepts
time
equilibrium of the creative components
122
At this moment, the focus on computer as media seems to have outrun
the concepts in design creativity.
The focus on media innovation may have eclipsed the role of concepts
and form in creation. Instead of being based on ideas, design solutions
became based on the medium; if something can move, move it; if
something can blink, let it blink. There are innumerous examples of
senseless employment of computational strengths that conrm Henry D.
Thoreau Arcadian quote Our inventions are wont to be pretty toys, []
they are but improved means to an unimproved end.
A hundred years ago, a similar event occurred: the advent of cinema.
A new medium was introduced; a new technology was available for
expression. But there were neither an aesthetic nor language of cinema;
a new vocabulary needed to be built, where time was a plastic element
that could be manipulated through montage. The cinematic medium
component leaped ahead of available creative forms and concepts, and
only after years of worldwide experimentation a proper language of the
motion pictures emerged.
The concepts of computation should be taken and used to build a new
vocabulary of graphic design, where the computing logical plane shall be
seamlessly welded to the visual, aural, temporal and haptical dimensions.
The incorporation of computation is the key for conceptual thinking
to reclaim its leadership in design creativity, bringing reasoning and
meaning to interactive experiences.
medium
form
concepts
time
media seems to have outrun the concepts in design creativity
123
The scientic, mathematical approach of computation and the whimsical
nature of art are not truly opposites, and have already met numerous
times through history. We are facing a new instance of this integration,
where algorithms are agents of translation from concept to form, from
form to medium.
Designers confront the challenge of learning the praxis of computation;
creative freedom in new media can only be achieved by knowing how to
deal with the transformation machine. This knowledge does not imply
that designers should become computer scientists, but that designers
should become wiser.
The visual expression through code is not and should not be hype,
a trendy toy of ephemeral design. Neither should it be an obligation, a
mandatory solution to all communication questions. The visual expression
through code should be considered as a current stage of the evolution in
design which is a process where different media can coexist.
The articulation of visual experiences through algorithm is the compre-
hension of new media design as an integrated whole: when conceptual
and technical barriers are tore down, creativity can be expressed simulta-
neously in all dimensions.
Creativity is borderless.
125
Bibliography
Arnheim, Rudolph. Art And Visual Perception Art And Visual Perception. University of California
Press, 1974.
Ashton, Anthony. Harmonograph Harmonograph. Walker, 2003.
Barthes, Roland. Camera Lucida. Hill and Wang, 1982.
Bense, Max. Estetica. Perspectiva, 1975.
CAVS-MIT. Gyorgy Kepes The MIT years Gyorgy Kepes The MIT years. MIT Press, 1978.
Chediak, Almir. Harmonia & Improvisao Harmonia & Improvisao. Lumiar, 1986.
Dondis, Donis. A primer of visual literacy A primer of visual literacy. MIT Press, 1973.
Eisenstein, Sergei. The Film Form. Harcourt, 1969.
Eisenstein, Sergei. The Film Sense. Harcourt, 1969.
Gerstner, Karl. The Spirit of Colors Spirit of Colors. MIT Press, 1981.
Gerstner, Karl. The Forms of Color. MIT Press, 1986.
Kandinsky, Wassily. Concerning the Spiritual in Art Concerning the Spiritual in Art. Dover, 1977.
Kandinsky, Wassily. Point and Line to Plane. Dover, 1979.
Kepes, Gyorgy. Language of Vision Language of Vision. Paul Theobald, 1944.
Kepes, Gyorgy et al. Language + Vision Language + Vision. George Braziller, 1966
Kudielka, Robert. Bridget Riley Bridget Riley. British Council, 1978.
Maeda, John. Maeda and Media. Thames & Hudson, 2000.
Munari, Bruno. Design Design e Comunicao Visual o Visual. Martins Fontes, 1968.
Nachmanovitch, Stephen. Free Play Free Play. Tarcher, 1991.
Toben, Bob. Space Time and Beyond Space Time and Beyond. Bantam, 1983.
Wardrip-Fruin, Noah and Nick Norton. The New Media Reader. MIT
Press, 2003.
Wurman, Richard Saul. Information Anxiety I Information Anxiety I. Bantam, 1990.
127
The following is my rst interactive object, developed at age of 5.
The procedure is simple:
1) turn off all the lights
2) close your eyes and position a camera ash right in front
of them
3) spin your boy, a lot
4) when its getting almost impossible to spin, re the ash as
many times as possible.
I can describe the effects: colored circles, patterns, oating around.
Theres a scientic explanation, but it doesnt really matter.