Embodied Design

Cognition:Action-based
formalizations in
Architectural Design
Daniel Smithwick and Larry Sass

international journal of architectural computing issue 4, volume 12 399

 Embodied Design Cognition:Action-based
formalizations in Architectural Design
Daniel Smithwick and Larry Sass

This paper frames design knowledge as formalizable

physical actions that can more fully exploit future
design tools and production methods. As computational
design tools become more physically interactive and
integrated into our environment we need new research
frameworks to develop theories of physical design
action as a form of knowledge. Symbolic theories of
design knowledge traditionally frame design activity as a
mental process and as a result researchers have not
fully explored the potential for bodily-based
computational design knowledge.We present an action-
based design notation drawing inspiration from music
performance theory to illustrate how this may impact
design research.We discuss findings from situated
cognition in cognitive science as an alternative
framework for exploring and expanding design
knowledge.We conclude with suggestions for future
work in robotic-aided design cognition.

400
1. INTRODUCTION
In a world largely defined by digital processes that render knowledge to that
which can be programmed on a computer, we must question what design
knowledge is, where it exists, and how to expand upon and improve it. Is it
in our minds? Is it in our bodies? Is it in our artifacts? Can it be embedded
in our tools? Such questions lie at the intersection of design cognition and
design computation, in other words—in how designers think about and do
design in a formalized way.
Early theories of design cognition were not only based on metaphors of
the mind as a symbol processor, but were constructed as and tested with
symbol processors [1].Theories of design thinking became defined by the
very tools used to simulate design activity.This led to a design science [2]
and research methods [3], [4] that framed design as a symbol manipulation
process. As a result we now have computational design tools that neglect
and limit development of the embodied knowledge involved in designing.
Embodied design knowledge may include the physical actions designers
perform when sketching or model-making, the tacit feelings surrounding
new ideas, or the social interactions in design environments.The mentalist
perspective found in traditional design cognition excludes such experiences
and is best summed up by Cross: “Asking ‘Can a machine design?’ is similar
to asking ‘Can a machine think?’” [5].We propose that design knowledge is
more than thinking and symbol manipulation—it is doing, feeling, and
interacting.
Alternative theories of human cognition such as Embodied or Situated
Cognition are not limited by the symbolic representation required when
building information-theoretic models. Instead, “Knowledge depends on
being in a world that is inseparable from our bodies, our language, and our
social history” [6].This work explores what design knowledge can be if we
think through our bodies instead of through the mental aid of our symbol
processing tools.

1.1. Skill or knowledge? Cognitive offloading in digital design

Digital tools provide a clear starting point for finding evidence of a
designer’s embodied knowledge because we can directly compare the
symbolic description of a design activity with a non-symbolic description.
For example, while the process of drawing a simple shape with a CAD tool
can be symbolically described as text commands or algorithms, it can also
be richly formalized through the physical motions of the body as it interacts
with the tool (Figure 1).
These motions are bodily-based knowledge in action and can be
considered separate from skill. Noë calls this “sensorimotor knowledge”
through which we give structure to our bodily experience [8]. Sensorimotor
knowledge was given its name by first framing the phenomenon as ‘cognitive

Embodied Design Cognition: Action-based formalizations in Architectural Design 401


Figure 1: Motion tracking reveals
the movements involved in drawing a
circle with a CAD tool.

offloading’ (see [7] for in-depth discussion). An anecdotal example of

cognitive offloading is found in memory retrieval.When attempting to
remember a phone number, a person might raise up her hand and type out
the number in space on an imaginary keypad. By doing this the
memorization task has been ‘offloaded’ to the physical actions of typing the
numbers, which may help in its recall. But in order to offload to the
environment, the person first know that it exists at their disposal.
Returning to our example above in Figure 1, let us consider what is
cognitively offloaded when creating a circle with a CAD tool.When first
learning how to draw a circle with a digital tool, the novice designer
depends heavily on its symbolic descriptions such as the text commands,
their sequential ordering, and the location of the keys on the keyboard.
Over time, as the novice draws more and more circles, she relies less upon
the symbolic descriptions for knowing how to create a shape and more
upon her actions for simply ‘doing’ so. Framed this way, the knowledge
needed for drawing a circle can be attuned to becoming aware of the
sensations structuring movement, feeling, and seeing.
Imagine the embodied knowledge an experienced designer exhibits
when designing an entire building. Our question becomes: is the expert one
who memorizes more symbolic descriptions or one who exhibits more
awareness of his movements and feelings? Described in these terms, the
embodied knowledge of architectural designers can be likened to that of
exceptional physical performers.We imagine design activity that is as
generative as an improvisational jazz pianist, as exhilarating as a gymnast, and
as precise yet graceful as a ballet dancer.

1.2. Schon’s limitations of digital computation

The traditional conception of design knowledge exemplified through current
digital computation technologies continues to frame knowledge through
symbolic representations, whether these are the CAD commands for
drawing 2D shapes, or algorithms for parametrically arranging facades, or

402 Daniel Smithwick and Larry Sass


 Figure 2:What is the Shape
composed of? Is it one triangle and
three V’s (A); is it nine separate lines
(B); or is it three brackets (C); or 3
X’s? (Drawings adapted from [12])
BIM systems for project life-cycle analysis, e.g., see [9]. Schon presents this
symbolic framing as a limitation of digital computation when he makes his
observations of design as ‘local move experiments’. He writes, “Computers
are unable- at least presently unable - to reproduce: the perception of
figures or gestalts, the appreciation of qualities, the recognition of
unintended consequences of moves” [10]. Schon is saying that our digital
computation tools cannot account for, support, or enhance a large part of
what goes on in design activity: how we are able to see something in a new
way through a succession of seeing-moving-seeing.
1.3.Visual and physical computation
We suggest that the traditional focus on symbolic formalization has blinded
researchers to the tacit dimensions [11] of design knowledge that Schon
identified over 20 years ago. Stiny is one of the few who has seen this
dimension and formalized it, although he discusses a limited example. He
writes about Schon’s kind of seeing through his terms of “visual calculating”
[12]. Shape grammars are his example of a non-symbolic formal system that
engages the designer’s ability to see anew—to perform a visual
interpretation on non-discrete elements such as lines and shapes.That the
lines and shapes in Figure 2 can merge and break apart in an infinite yet
solvable number of ways is what makes shape grammars a compelling
example of a non-symbolic visual calculating system.
What makes shape grammars formal is that they exploit a designer’s ability
to not only see and recognize Schon’s ‘unintended consequences of
moves’—in this case, new shapes—but more importantly they allow a
designer to “actively use them as input for further computations” [13].This
requires some kind of memorization.Within their Classic and Non-Classical
Computation Framework, Knight and Stiny categorize this kind of visual
computation as one that utilizes ‘non-classical representation’ [13]. Building
from their framework and through theories found in embodied cognition
we explore the idea that physical actions can be formalized in non-symbolic
terms. If shape grammars are an example of a visual computation system
that harnesses how designers see, can there be a physical computation
system that harnesses how designers move?
Embodied Design Cognition: Action-based formalizations in Architectural Design 403
 1.4. Paper structure
In Section 2 we present the background to our work.This section will serve
readers who are unfamiliar with the symbolic vs non-symbolic debate in
design computation and cognition research.We first review theories and
methods in design cognition and protocol analysis; we outline arguments
from cognitive science that suggest knowledge is not limited to our ‘minds’;
and we present recent applications found in Human-Computer Interaction
that demonstrate potential advancements for design tools. In Section 3 we
show an action-based notation from music performance theory as an
example of a non-symbolic system that formalizes bodily movement. In
Section 4 we introduce our action notation and speculate its potential
impact on design cognition and computation.We then discuss how research
in embodied cognition may provide a foundation for further developing
bodily-based design knowledge. In Section 5 we outline future work
including our early exploration into robotic tools for enhancing
sensorimotor knowledge. Lastly, we provide a summary of contributions and
offer our conclusions in Section 6.

2. BACKGROUND
2.1. Symbol processing in design protocol analysis
Design protocol studies have led to novel formalizations of the design process
and have expanded computational models of design knowledge [14]. Protocol
studies commonly represent the design process through coding schemas and
segmentation, e.g., [15]; or as descriptive narratives of the design process made
through video analysis, e.g., [16]; or from thoughts voiced by the designer
known as the voice-out-loud or concurrent protocol method [17]. Some
studies do account for the physical actions designers make, e.g., sketching,
looking, and gesturing [18], however these actions are neither hypothesized to
impact the design process nor constitute a kind of design knowledge.
These symbolic approaches to formalizing design activity can be traced
back to theories of cognition and computation put forth by Herbert Simon
and others. Particularly illuminating is a series of articles published in a
special issue of Cognitive Science on Situated Activity theory, an early
articulation of embodied cognition [19- 23]. Beginning with an article by
Vera and Simon, the authors present cases for whether or not knowledge is
fundamentally symbolic in nature or whether it is contextually situated.
Greeno and Moore sum up the central question underlying both sides:
“The question, then, seems to be something like this: whether (1)
to treat cognition that involves symbols as a special case of
cognitive activity, with the assumption that situativity is
fundamental in all cognitive activity, or (2) to treat situated activity
as a special case of cognitive activity, with the assumption that
symbolic processing is fundamental in all cognitive activity.” [20]

404 Daniel Smithwick and Larry Sass

 Vera and Simon clearly take the symbol-processing side arguing that
perception, thinking, and action can be described definitively in terms of
abstract symbols, or information:
“Sequences of actions can be executed with constant interchange
among (a) receipt of information about the current state of the
environment (perception), (b) internal processing of information
(thinking), and (c) response to the environment (motor activity).”
[19]
In Vera and Simon’s theory a designer can be modeled as an input/output
device with a mind in between that operates like the central processor of a
digital computer. Based on this hypothesis, one of the key challenges for
design protocol analysis has become an algorithm development problem as
a method for formalizing the cognitive steps involved in the ‘internal
processing of information’.
More recent work in design protocol analysis has expanded symbolic
formalization to include the external, context-specific conditions of design
activity. Smith and Gero discuss this in terms of ‘situatedness’ whereby
design is framed as an interaction between constantly changing variables in a
dynamic solution space [24]. An example formalization from design protocol
analysis based on this symbolically situated approach is the Situated
Function-Behavior-Structure framework [25].

2.2. Situating embodied cognition in design activity

Theories of embodied cognition can further expand our conception of
design knowledge by questioning the symbol-processing model of cognition.
However, there is still much debate surrounding what exactly embodied
cognition means and there are many competing names it goes by: situated
cognition [26], enactive cognition [8], and extended and distributed
cognition [27], to name a few. Nemirovsky and Ferrara write:
“Embodied cognition rejects the notion that behind perceptual-
motor activity there is a ‘mind’…Whatever we can recognize as
rational, rule-based, or inferential, is fully embedded in our bodily
actions; perception and motor activity do not function as input and
output for the ‘mental’ realm; what we usually recognize as mental
are inhibited and condensed perceptual-motor activities that do
not reach the periphery of our nervous system.” [28]
Anderson states that the central focus of work in embodied cognition is the
symbol-grounding problem which questions where symbols acquire meaning
for humans. Anderson writes that symbols “must ultimately ground out in
[terms of] the agent’s embodied experience and physical characteristics”
[29]. In other words, while we can and do use symbols to represent and
communicate knowledge, these symbols depend on our bodies and the
physical environment for us to make sense of them.
From the embodied cognition perspective, it is relevant to ask what

Embodied Design Cognition: Action-based formalizations in Architectural Design 405

 designers do with their bodies in their environment during design activity.
They are sitting at desks, typing on computers, standing around laser
cutters, handling tools, manipulating materials, working alone, gathering in
small groups, and so forth. According to theories of embodied cognition
these seemingly ordinary everyday activities are ripe with generative
potential for shaping a design outcome.
The impact of embodied cognition is just beginning to be felt in design
research. Although many have speculated about integrating new theories of
cognition to reframe the design process [24], [30-31], little work has been
done to empirically test the theories, let alone to implement formal
systems. Most work simplifies the design task in order to formalize the
embodied activities. Knight and Stiny for example develop a formalization of
the physical actions involved in craft-making activities such as knot-tying. In
their view, design, even in computationally aided processes, is not an
intellectual activity but rather “a kind of making itself, an activity that
demands perceptual, bodily engagements with the materials of the world
[32]. Maher et al. present how physical interaction with programmable
toys— Sifteo Cubes—leads to more creative solutions in simple word-play
tasks. Using protocol analysis they investigate how gestures affect creative
interaction by increasing the affordances of the programmed cubes [33].
Smithwick and Sass categorize three different types of physical actions
involved in assembling digitally fabricated structures: interfacing—the
configuration state of the hand in relation to material components;
interacting—the progression of interface states over time; and
interchanging—transformations between the hand, the components, and the
work environment [34].

2.3. Human-computer interaction

Although most work in embodied cognition is theoretical or empirical there
are project-based research examples demonstrating practical tools that
exploit the physical interactions between users and tools [35]. Recent work
from the field of Human-Computer Interaction (HCI) demonstrates
embodied cognition theories applied in prototyping tools. For an overview
of HCI from an architectural computation perspective, see [36]. Some tool
developers situate the control and design interface of digital fabrication
tools such as CNC routers directly on the machine and allow for real-time
control of the cutting bit through motion tracking technologies [37-38].
Zoran and Paradiso constructed a handheld Dremel-like digital sculpting
tool called FreeD enabling a user to more directly engage with the sculpting
material through the aid of computational feedback to correct movement
errors based on a digital design [39]. Braumann and Cokcan have developed
gestural interfaces for controlling robotic arms using the motion of the
body [40].They explore physical computing technologies such as Arduino
microcontrollers and Kinect cameras to develop programming platforms for

406 Daniel Smithwick and Larry Sass

novice designers to more intuitively and directly control robotic motion.
Such projects demonstrate how digitally integrated physical tools can
engage the designer’s body by means of haptic, motion, and infrared sensors.
As robotics and other environmental computing technologies are subsumed
into the architectural design process, designers and design researchers will
need theories of embodied interaction to frame the technical development
in meaningful and productive ways.

3. A NON-SYMBOLIC NOTATION FOR MUSIC

PERFORMANCE
Before we present our action-based notation for design, we would like to
present a related piece of work from the field of music performance
research.We find this comparison inspirational and useful upon many levels.
For the reader who is also a musician, we think this comparison will provide
novel insight into the creative undercurrents found in both a musician’s
performance and a designer’s actions. Many researchers have proposed
metaphors relating music to architecture; however, we propose a more
literal comparison: Designers and musicians alike interact with a wide range
of tools to develop and express conceptual ideas which result in physical
phenomena. It is also the case that formal symbol systems are used
extensively in music composition and design computation to represent and
communicate ideas. By considering how musicians use their bodies to
interpret formal systems, designers may learn embodied computational
strategies for performing design activity.
Below is a study from the Music Performance Research journal of German
composer Helmut Lachenmann’s action-based notation developed for his
1969 piece for solo cello, Pression (Figure 3) [41]. Orning describes
Lachenmann’s action notation as:
“A reversal of traditional hierarchies, prioritizing the performance
over the musical text. Lachenmann shifts the focus from the score
as musical text to the action embodied in performance.This shift in
compositional focus calls for a complementary shift in analytical
focus.” [41]

Embodied Design Cognition: Action-based formalizations in Architectural Design 407

 3.1. A hidden dimension between the notes  Figure 3: Action-based music
notation by Lachenmann for ‘Pression’,
In Lachenmann’s notation, the musical notes as traditionally represented 1969. (Modified from original [41])
(half notes, quarter notes, etc.) still exist and function accordingly; however, Red dots highlight the traditional
the notation reveals and prioritizes a ‘hidden dimension’ between the notes music notes, blue lines highlight the
that the musician can exploit. At an intuitive level to the experienced suggested physical actions for the
musician this hidden dimension is very real: music happens ‘off the page’ and musician, and green highlights the
the notes as a formal system are there only as music theory’s best physical locations on the cello.
approximation for delivering a desired aural effect.What Lachenmann’s
notation does is formalize how the musician might physically interpret and
perform this musical dimension through different types of lines (zig-zags,
dotted lines, sloping lines, horizontal arrows, etc.) and graphics (figurative
drawings of hands and cello parts) (Figure 3). For example, the zig-zag could
be performed as the shape of the motion a musician makes with his hand,
and the schematic drawing of the fret board identifies where actions take
place on the instrument.
What makes Lachenmann’s system of lines and graphics non-symbolic is
that the musician’s interpretation is defined only by his body in action
rather than with a predetermined set of motion paths. In other words,
every interpretation will be equally unique and correct.With Lachenmann’s

408 Daniel Smithwick and Larry Sass

 action-based notation we have an example of a formalization that expands
the world of pre-determined symbolic representation without sacrificing the
structure of a framework that makes it possible to share, preserve, and
improve an active form of knowledge. Is this possible in the designer’s
world?

4. A NON-SYMBOLIC FORMALIZATION FOR

DESIGN
Our formalization shifts the traditional hierarchies found in design cognition
and computation which place priority on symbolic steps and processes as
the foundation of creative work in much the same way that traditional
hierarchies in music place priority on the musical composition as the
creative work.To illustrate our action-based formalization we return to the
example of drawing a shape with a CAD tool.
To more clearly convey our formalization we contrast it with a symbolic
formalization of the same exercise of drawing a circle in AutoCAD (Table
1).The primary purpose in developing this example is to reveal that even
within the simple act of drawing a circle with a CAD tool the traditional
conception of design knowledge is blind to how a designer uses his body
interactively to express and develop design ideas (Figure 4). As such, this
formalization should be read as an illustrated thought experiment to ground
the discussion.

 Figure 4: Detail of articulated hand movements when typing the word ‘circle’ in AutoCAD.

Embodied Design Cognition: Action-based formalizations in Architectural Design 409

 Table 1: Comparison between a symbolic formalization with an action-based notation for drawing a circle with a CAD tool.

410 Daniel Smithwick and Larry Sass

 The formalization in Table 1 to be read from left to right as one might
read musical notation. In Step 1, the horizontal dashed line represents the
line made by the fingers as they rest on the ‘home row’ of a standard
computer keyboard.The up and down arrows indicate the order, direction,
and quality in which the designer can move his fingers as they strike the
keys to spell out the word ‘circle’. For example, in Step 1 the left-most black
down arrow indicates to the designer that using his left index finger (labeled
1L) he should strike the ‘C’ key on the keyboard. Next, the designer should
use his middle finger on his right hand (labeled 2R) to strike the ‘I’ key on
the computer keyboard.The remainder of Step 1 in Figure 3 follows
accordingly. Steps 2-6 in Table 1 complete the circle drawing exercise with
notation for using a computer mouse.
Most importantly, our action notation suggests potential for harnessing
the physicality of a designer’s movements. This is the ‘hidden dimension’ we
discussed in Section 3. For example, the timing between movements (Figure
4A), the amount of force used (Figure 4B), and duration of striking (Figure

 Figure 5:The physical qualities of a

designer’s movements such as the A)
timing; B) amount of force used; and
C) the length of duration one can hold
the key can be prioritized through
physical design notation.

Embodied Design Cognition: Action-based formalizations in Architectural Design 411

 4C) can now be considered as meaningful engagement between the
designer and the tool.What these physical actions mean and how they
support and structure the design process is an open question that needs
exploring.

4.1. Discussion:What can designers learn from the physical

actions of Tetris players, mathematicians, and dancers?
As a thought experiment, our formalization opens up an avenue of
investigation overlooked in design research: how designers use their bodies
to explore novel solutions. However, where and how do we draw the limits
of design as an interactive activity? Design is certainly more than the motions
of typing AutoCAD commands on a keyboard, but should we also consider a
student’s posture as part of his design thinking? What about the gestures he
makes while presenting his project to critics or the way he constructs
physical models with his hands? Are such superfluous actions connected with
design knowledge? Research from psychology suggests a relationship
between spatial thinking and gesture production [42, 43].As we identified
above, designers perform many different types of physical actions for many
different reasons in many different settings. How can we make sense of these
intuitive and tacit dimensions of designerly interaction with the world?
Theories developed from work in embodied cognition may help
structure the study of physical action in the context of design research.
Kirsh and Maglio developed the theory of Epistemic Action which says that
not all actions are for pragmatic goal-oriented purposes.They witnessed
expert Tetris players utilize “physical actions [to] make mental computation
easier, faster, or more reliable” [44].The players would perform seemingly
extraneous physical actions during game play, such as overly rotating zoids
or translating zoids across the entire screen, taking them further away from
achieving the immediate goal at hand—dropping the zoids into rows. But
ultimately, claim Kirsh and Maglio, such actions “make it easier for [Tetris
players] to attend, recognize, generate and test candidates, and improve
execution” [44]. Do designers perform similarly extraneous actions that
help them make design decisions?
Much of design knowledge, however, is conceptual rather than practical.
How does bodily action support conceptual thinking? Recent research in
the learning sciences investigates the connection between physical action
and learning mathematical concepts. Goldin-Meadow and Beilock
demonstrate how specific types of gestural actions enhance the ability to
learn abstract concepts [45]. Martin and Schwartz demonstrate how
interaction with physical objects like Montessori Blocks help children to
learn mathematical concepts that they were unable to learn through mental
representation only [46].These examples show how gesturing and physical
interaction aid in communicating abstract ideas and in helping students think
through conceptual problems.

412 Daniel Smithwick and Larry Sass

 Although Cross is concerned with equating the bodily intelligence of
dancers with that of the creativity of inventors [5], new research is
expanding the significance of kinesthetic intelligence in relation our
imaginative powers. In recent work, Kirsh studies the use of abstract bodily
motions called ‘marking’ in professional dancer’s practice routines [47].
When learning new choreographies, instead of practicing the steps and
phrases ‘full out’, more successful dancers devise limited bodily actions to
simulate the full maneuvers.What this suggests is that contrary to the
traditionally defined role of the body as an output device of internal
thought, the body serves as a physical extension of thought capable of
generating structure upon which new shapes, orientations, and sequences of
thought are possible.

5. FUTURE WORK: EXPLORING EMBODIED DESIGN

KNOWLEDGE FOR FORMALIZING ROBOTIC
INTERACTION
What these examples show is the opportunity design researchers have to
explore and expand conceptions of design knowledge by considering the
body as an active and multi-functioning participant in the design process.
What are the epistemic actions designers perform? Are such physical
actions related to the reflective actions Schon describes in ‘seeing-moving-
seeing’ [48]? What are the conceptual problems designers communicate
through gesture or explore through interaction with physical materials? Do
designers construct prototype models similar to how dancers perform
marking as a strategy for scaffolding creative thought? What new tools can
we devise to harness this action-based embodied knowledge?
One of the most exciting and potentially fruitful applications of these
questions in design research is in developing robotic-aided design cognition.
We have begun investigating this area by conducting empirical studies on
fundamental physical actions designers perform in support of creative
problem solving tasks.We compare and contrast these actions with how a
robotic arm may be programmed to conduct similar tasks.These are basic
grasping and displacement actions used to manipulate physical objects
including: pushing and pulling blocks on 2D surface (Figure 6A); picking,
placing and stacking blocks in 3D patterns (Figure 6B); assembling
interlocking components to form simple planar assemblies (Figure 6C); and
assembling interlocking components to form complex curved structures
(Figure 6D).
By studying these basic physical actions we can develop a fundamental
framework of the embodied knowledge designers deploy in design activity.
Although much research has been conducted on the automation of
translating digital models into physical constructions with the aid of digital
fabrication tools [49] and recently with robotic tools [50], little is still
known about the designer’s sensorimotor knowledge to interact with

Embodied Design Cognition: Action-based formalizations in Architectural Design 413

 material objects when developing design ideas. Questions still remain: Do  Figure 6:We use protocol studies to
expert designers interact differently than novices? Can robotic tools compare the physical actions of
enhance our embodied cognitive knowledge through demonstration and designers with programming of a
interaction? Can we formalize student-teacher interactions to integrate robotic arm. 2D manipulation in
pushing and pulling tasks (A); 3D
robotics into the design studio?
manipulation in picking and placing
tasks (B); simple assembly tasks (C);
6. CONCLUSION and complex assembly tasks (D).
The central question we have explored in this paper asks:Where does
design knowledge exist? Is it an intellectual activity within the mind or is
design knowledge embodied in our finger tips or does it extend out into
the world as we interact with it? Such questions have been asked in many
fields of research including cognitive science, artificial intelligence, and
philosophy of mind with many different answers. As we have shown,
traditional theories of design cognition have held the narrow view that
design is an information or symbol-processing activity. One contribution of
our work has been to question this view by exploring theories of embodied
cognition and discussing examples of bodily-based knowledge in other
creative human activities.
We have also contributed an illustrated thought experiment of how
embodied theories of cognition may provide insight into new forms of
design knowledge.We use a novel example from music performance
research to highlight a tacit dimension which exists in between pre-defined
symbols, whether they are musical notes or automated CAD commands.
At the same time, we acknowledge that design researchers cannot exist
in the tacit dimension alone. New technologies open the door for further
research and expansion of design knowledge. Similar to how digital
computation technologies defined the early research frameworks of design
cognition, physical computation technologies such as integrated electronics
and robotic tools are poised to challenge traditional notions of design

414 Daniel Smithwick and Larry Sass

cognition.We propose that theories of embodied cognition can give design
researchers a structure for the endeavor ahead of us.
Stiny asks, “How would calculating be different if Turing was an artist or
designer and not a logician?” Instead of calculating by counting,Turing would
have calculated by seeing, says Stiny [51].We ask: what if Stiny was musician
or dancer? Would he calculate by performing physical actions and
interacting with the world?

ACKNOWLEDGEMENTS
Funding for this research was provided by the MIT/SUTD International
Design Center. Much of this work was conceived during the lead author’s
experience of playing drums in the MIT Festival Jazz Ensemble under the
direction of Fred Harris and attending lectures in George Stiny’s Research
Seminar in Design Computation.

REFERENCES
1. Newell, A. and Simon, H., Computer Simulation of Human Thinking, Science, 1961,
134, 2011-2017.
2. Simon, H., The Sciences of the Artificial, MIT Press, Cambridge, 1969.
3. Eastman, C., On the Analysis of Intuitive Design Processes, in: Moore, G., ed.,
Emerging Methods in Environmental Design and Planning, MIT Press, Cambridge,
1970, 21-37.
4. Gero, J., Computational Models of Creative Design Processes, in: Dartnall, ed.,
Artificial Intelligence and Creativity, Kluwer, Dordrecht, 1994, 9-21.
5. Cross, N., Natural Intelligence in Design, in: Cross, N., ed., Designerly Ways of
Knowing, Birkhauser, Basel, 2007, 49-62.
6. Varela, F.,Thompson, E. and Rosch, R., The Embodied Mind: Cognitive Science and the
Human Experience, MIT Press, Cambridge, 1993.
7. Wilson, M., Six Views of Embodied Cognition, Psychonomic Bulletin and Review,
2002, 9(4), 626-636.
8. Noë, A., Action in Perception, MIT Press, Cambridge, MA, 2004.
9. Aksamija, A., and Iordanova, I., Computational Environments with Multimodal
Representations of Architectural Design Knowledge, International Journal of
Architectural Computing, 2010, 08(04), 439-460.
10. Schon, D. and Wiggins, G., Kinds of Seeing and Their Functions in Designing,
Design Studies, 1992, 13(2), 135-156.
11. Polanyi, M., The Tacit Dimension, Doubleday Publishing, Garden City, NY, 1966.
12. Stiny, G., Shape:Talking about Seeing and Doing, MIT Press, Cambridge, 2006.
13. Knight,T. and Stiny, G., Classical and Non-Classical Computation, Information
Technology, 2001, 5(4), 355-372.
14. Dorst, K. and Dijkhuis, J., Comparing Paradigms for Describing Design Activity,
Design Studies, 1995, 16(2), 261-274.
15. Kavakli, M. and Gero, J.,The Structure of Concurrent Cognitive Actions: A Case
Study on Novice and Expert Designers, Design Studies, 2002, 23, 25-40.
16. Suwa, M., and Tversky, B.,What do Architects and Students Perceive in their
Design Sketches? A Protocol Analysis, Design Studies, 1997, 18, 385-403.

Embodied Design Cognition: Action-based formalizations in Architectural Design 415

 17. Lloyd, P., Lawson, B., Scott, P., Can Concurrent Verbalization Reveal Design
Cognition?, Design Studies, 1995, 16, 237-259.
18. Suwa, M., Purcell,T. and Gero, J., Macroscopic Analysis of Design Processes Based
on a Scheme for Coding Designers’ Cognitive Actions, Design Studies, 1998, 19,
455-483.
19. Vera, H., and Simon, H., Situated Action: A Symbolic Interpretation, Cognitive
Science, 1993, 17(1), 7-48.
20. Greeno, J., and Moore, J., Situativity and Symbols: Response to Vera and Simon,
Cognitive Science, 1993, 17(1), 49-59.
21. Agre, P.,The Symbolic Worldview: Reply to Vera and Simon, Cognitive Science,
1993, 17(1), 61-69.
22. Suchman, L., Response to Vera and Simon’s Situated Action: A Symbolic
Interpretation, Cognitive Science, 1993, 17(1), 71-75.
23. Clancey,W., Situated Action: A Neuropsychological Interpretation Response to
Vera and Simon, Cognitive Science, 1993, 17(1), 87-116.
24. Smith, J., and Gero, J.,What Does an Artificial Design Agent Mean by Being
‘Situated’?, Design Studies, 2003, 26, 535-561.
25. Gero, J., and Kannengiesser, U.,The Situated Function-Behavior-Structure
Framework, Design Studies, 2004, 25, 373-391.
26. Robbins, P., and Aydede, M., A Short Primer on Situated Cognition, in Robbins, P.
and Aydede, M., eds., The Cambridge Handbook of Situated Cognition, Cambridge
University Press, Cambridge, 2009, 3-10.
27. Clark, A., Supersizing the Mind: Embodiment, Action, and Cognitive Extension, Oxford
University Press, Oxford, 2011.
28. Nemirovsky, R., and Ferrara, F., Mathematical Imagination and Embodied
Cognition. Educational Studies in Mathematics, 2009, 70, 159-174.
29. Anderson, M., Embodied Cognition: A Field Guide, Artificial Intelligence, 2003,
149(1), 91-130.
30. Lyon, E., Autopoiesis and the Digital Design Theory: CAD Systems as Cognitive
Instruments, International Journal of Architectural Computing, 2005, 03(03), 317-333.
31. Visser,W., Design: One, but in Different Forms, Design Studies, 2009, 30, 187-223.
32. Knight,T., and Stiny, G., Making Grammars: From Computing with Shapes to
Computing with Things,Workshop paper presented at:The Sixth International
Conference on Design Computing and Cognition, 2014 June 23-25, London, UK.
URL:[http://descomp.scripts.mit.edu/computationalmaking/wp-
content/uploads/2014/05/MakingGrammars_KnightStiny.pdf]
33. Maher, M., Gonzalez, A., Grace, K., and Clausner,T.,Tangible Interaction Design:
Can we Design Tangibles to Enhance Creative Cognition?, Poster session
presented at:The Sixth International Conference on Design Computing and
Cognition, 2014 June 23-25, London, UK. URL: [http://cs.gmu.edu/~jgero/DCC14-
PosterAbstracts.pdf]
34. Smithwick, D., and Sass, L., Physical Design Cognition: Embodied Intelligence for
Taming the Digitally Fabricated Singapore Horses, Poster session presented at:
The Sixth International Conference on Design Computing and Cognition, 2014
June 23-25, London, UK. URL: [http://cs.gmu.edu/~jgero/DCC14-
PosterAbstracts.pdf]
35. Dourish, P., Seeking a Foundation for Context-Aware Computing, Human
Computer Interaction, 2001, 16, 229-241.
36. Dade-Robertson, M., Architectural User Interfaces:Themes,Trends and
Directions in the Evolution of Architectural Design and Human Computer
Interaction, International Journal of Architectural Computing, 2013, 01(11), 1-19.

416 Daniel Smithwick and Larry Sass

37. Willis, K., Lin, J., Mitani, J. and Igarashi,T., Spatial Sketch: Bridging Between
Movement and Fabrication, at: TEI’10 ACM, Cambridge, USA, 24-27 January 2010.
38. Willis, K., Xu, C.,Wu, K., Levin, G. and Gross, M., Interactive Fabrication: New
Interfaces for Digital Fabrication, at: Tangible, Embedded, and Embodied Interaction
Conference, Portugal, 22-26 January 2011.
39. Zoran, A. and Paradiso, J., FreeD – A Freehand Digital Sculpting Tool, in: Baudisch,
P., Beaudouin-Lafon, M. and Mackay,W.E., eds., CHI’13:The 31st International
Conference Extended Abstracts on Human Factors in Computing Systems, ACM, Paris,
2013, 2613-2616.
40. Braumann, J., and Cokcan, S., Digital and Physical Tools for Industrial Robots in
Architecture: Robotic Interaction and Interfaces, International Journal of
Architectural Computing, 2012, 10(04), 541-554.
41. Orning,T., Pression – A Performance Study, Music Performance Research, 2012, 5,
12-31.
42. Morsella, E. and Krauss, R,The role of gestures in spatial working memory and
speech. American Journal of Psychology, 2004, 117, 411-424.
43. Wesp, R., Hess, J., Gestures maintain spatial imagery. American Journal of
Psychology, 2001, 114, 591-600.
44. Kirsh, D. and Maglio, P., On Distinguishing Epistemic from Pragmatic Action,
Cognitive Science, 1995, 18, 513-549.
45. Goldin-Meadow, S., and Beilock, S., Action’s influence on thought: the case of
gesture. Perspectives on Psychological Sciences, 2010, 5(6), 664-674.
46. Martin,T., and Schwartz, D., Physical Distributed Learning: Adapting and
Reinterpreting Physical Environments in the Development of Fraction Concepts.
Cognitive Science, 2005, 29, 587-625.
47. Kirsh, D., Embodied Cognition and the Magical Future of Interaction Design. ACM
Transactions of Computer-Human Interaction, 2013, 20(1), xx-xx.
48. Schon, D, Designing as reflective conversation with the materials of a design
situation. Knowledge-Based Systems, 1992, 5(1), 3–14.
49. Sass, L., Oxman, R., Materializing design:The implications of rapid prototyping in
digital design, Design Studies, 2005, 27(3), 325–355.
50. Kohler, F., Gramazio, M., and Willmann, J., The Robotic Touch: How Robots Change
Architecture, Park Books, Chicago, 2014.
51. Stiny, G.,When is Generative Design Design?, Keynote speech at Computation:
The New Realm of Architectural Design [27th eCAADe Conference] Istanbul
(Turkey), 16-19 September, 2009.

Daniel Smithwick1 and Larry Sass2

1MassachusettsInstitute of Technology
Department of Architecture
MIT-SUTD International Design Center
265 Massachusetts Ave
djs2@mit.edu

2MassachusettsInstitute of Technology
Department of Architecture
77 Massachusetts Ave
lsass@mit.edu

Embodied Design Cognition: Action-based formalizations in Architectural Design 417