Zrich

CSE Seminar in Robotics

Presentation by: Maria Veiga
Zrich
Introduction
Motivation
What is out there
In depth projects
Painting fool
eDavid
Conclusion
Zrich
Artists perspective & questions:
Generative art introduces complexity to an art piece;
Often used as a tool, thus:
What techniques can be used?
How to introduce randomness to art?

Scientific perspective & questions:
Interest in setting up an autonomous creative agent
How to formalize a creative process?
How to create an autonomous creative agent?
How to teach a robot to paint?
Zrich
Timeline
Mozart
18th century
Alfred Jensen:
The Apex is Nothing
(1960)
Ellsworth Kelly:
Four Black and Whites
(1957)
John Cage:
Music of Changes
(1951)
The Painting Fool
(2006)
eDavid
(2009)
BNJMN
(2013)
AARON
(1973)
Zanelle
(2012)
Vangobot
(2012)
Frieder Nake
(1963)
Zrich
Timeline
Mozart
18th century
Alfred Jensen:
The Apex is Nothing
(1960)
Ellsworth Kelly:
Four Black and Whites
(1957)
John Cage:
Music of Changes
(1951)
The Painting Fool
(2006)
eDavid
(2009)
BNJMN
(2013)
AARON
(1973)
Zanelle
(2012)
Vangobot
(2012)
Frieder Nake
(1963)
Zrich
Started in the Computational Creativity Research
group at Imperial College

In 2006, the software was given the name
Painting Fool

Objective is to build a software system that is
one day taken seriously as a creative artist
Zrich




The question is: how to build software that
exhibits behaviours deemed creative in people?
Zrich
As an AI
project
Simulation of paint strokes on a canvas
Aware of emotions of the subject
Cultural context
Critical appraisal of its own work
Zrich
As an AI
project
Simulation of paint strokes on a canvas
Aware of emotions of the subject
Cultural context
Critical appraisal of its own work
Non-Photorealistic Rendering (NPR):
By producing software that simulates
natural media
Allow artists to teach the software:
Allow artists, designers, and anyone else to
train the sofware in aspects of its
processing (from analysing digital
photographs to constructing and painting
scenes)

Zrich
As an AI
project
Simulation of paint strokes on a canvas
Aware of emotions of the subject
Cultural context
Critical appraisal of its own work
Mapping of painting style to emotions:
Mapping composed of researchs group
choices and also randomly generated
Detect emotion in images and videos:
Emotion detection software that identifies:
1) The frame that shows more emotion
2) Locations of the facial features
3) The emotion expressed by the subject
(Using methods by Valstar and Pantic)

Zrich
As an AI
project
Simulation of paint strokes on a canvas
Aware of emotions of the subject
Cultural context
Critical appraisal of its own work
Show intention in the paintings:
Scan the internet for text and image
resources for production of artwork.
1) Software downloads news headlines
2) Use a text rank algorithm to find most
important nouns
3) Use the keywords to scan image
repositories (Google, Flickr)
4) Create a piece by collage of selected
pictures and using non-photorealistic
rendering software


Zrich
As an AI
project
Simulation of paint strokes on a canvas
Aware of emotions of the subject
Cultural context
Critical appraisal of its own work
Still an active area of research:
How to formalise the process of appraisal of
artwork?
Implemented abilities for the software to
calculate objective measures of abstract
art, e.g: location of symmetries
BUT hard to train a software to appreciate
works of art without being an extension of
its creators subjective views
Zrich
Emotionally aware painter
Demonstrating intent Travelling salesman dancers
Zrich
Zrich
Objective: Build a robot able to paint with real
color on a real canvas.

How?
Commercial robotic arm + Visual System + Smart Algorithms
Zrich
Objective: Build a robot able to paint with real
color on a real canvas.

How?
Commercial robotic arm + Visual System + Smart Algorithms
-one-arm industry robot used in car manufacturing
modified for painting

- several adapters for different mediums
Zrich
Objective: Build a robot able to paint with real
color on a real canvas.

How?
Commercial robotic arm + Visual System + Smart Algorithms
Zrich
Objective: Build a robot able to paint with real
color on a real canvas.

How?
Commercial robotic arm + Visual System + Smart Algorithms
Painting as an optimization process consider spatial division, color and
content composition.
Iterative operation scheme to produce color distributions.

2 ways to compute the brush stroke:
- Pre-define a set of stroke candidates
- Dynamically generated strokes Compute image gradient using a Sobel
operator. Randomly generate initial positions. For each position, a path using
Line Integral Convolution of the gradient vector field is generated and then
Runge Kutta numerical integration is performed.


Zrich
Objective: Build a robot able to paint with real
color on a real canvas.

How?
Commercial robotic arm + Visual System + Smart Algorithms


Semantic Hints:
- Use different techniques for different parts of a picture
- Automatically segment the picture
- Choose which method dynamic or pre-defined brush strokes best
minimizes the distance between canvas image C and target image T.



Zrich
Zrich
Pre-defined brush stroke vs dynamic brush strokes
Zrich
Traditionally, artists use robotics & generative
methods as way to enhance their work
Whereas scientists seem to also be looking into a
formalisation of the creative process (Computational
Creative Theory)
However, collaborations between artists and
scientists are increasing boundaries arent so clear
anymore (Vangobot, Zanelle, Computational
Aesthetical Evaluation)

The intention is not to replace artists but software
can perform tasks that people cannot perform e.g:
the calculation of where to place thousands of
strokes at thousands of time steps



Zrich
Rise of highly costumizable designs and art
pieces
Exploration of new ways of aesthetics

Other projects that were influenced by this
research field

Generative code
Procedural gaming
Formalised criteria to judge Goodness of an art piece