Thoughts on embodiment, tangibility and situated interaction in Digital Live Art

Vincent Olislagers
K3, Malm Hgskola Interaction Design master programme +46-765774843 M10e0352@student.mah.se

ABSTRACT
In this essay I examine a selection of interactive artworks, installations and electronically enhanced performances in pursuit of a deeper understanding of the potential value of tangible interfaces and situated interaction for digital live art (Sheridan, 2006). I investigate in which way embodiment (Dourish, 2001) can offer new interaction affordances and how these can be employed as novel tools for creative expression. I rely on Fels model of embodiment (2000) as basis for the description of relationships between art and audience and continue to build on it with practical examples from contemporary art. In addition, I touch upon various forms of digital live art and the ways in which they embody interactivity, tangibility and place specific computing. I conclude my work by evaluating the immersive qualities of tangible and situated interaction and by reflecting on their significance to digital live art in the public space.

new, hybrid forms of art. Digital live art (DLA) (Sheridan, 2006) is such a hybrid form of art, culminating live art, public performance and humancomputer interaction (HCI). My interest in this essay is to investigate how tangible interfaces and place specific computing can contribute to Digital Live Art and in which ways they are already made manifest in contemporary interactive artworks. Is tangibility and location awareness at all relevant for Digital Live Art and if the case, in which way do they contribute to the experience? I begin by defining what is digital and live about digital live art and proceed by reviewing the concept of embodiment as a convergence of art & technology and artwork & audience.

2.

DIGITAL LIVE ART

Categories and Subject Descriptors

H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems; H.5.2 [Information Interfaces and Presentation]: User Interfaces; J.5 [Arts and Humanities]: Performing arts.

General Terms
Performance, Design, Experimentation, Human Factors

Keywords
Tangible interaction, place specific computing, digital live art, installation art, performance art, embodiment

1.

INTRODUCTION

Contemporary HCI literature has documented both the emergence of tangible interfaces and place specific computing as their potential for creative expression in

Dixon states that all art is an interaction between the viewer and the artwork, and thus all artworks are interactive in the sense that a negotiation or confrontation takes place between the beholder and the beheld. He argues that digital interactive art differs from traditional art in the ability of the user or audience to activate, affect, play with, input into, build or entirely change it. (2007, pp. 559) A good example can be found in the Gogh Bike project (Cohen, et al. 2010) in which ordinary bicycles have been repurposed as collaborative creative tools. Gogh bicyclists can paint on a digital canvas by driving, steering and braking. The paint bucket on the back of the bicycle can be used by other participants to mix and refill colors. The affordances of the bicycle have been redefined to allow its user to do more than merely travel or exercise. Moreover they embody a new quality (collaborative painting) which was not present in the original object. The notion of embodiment is an important aspect of many interactive artworks. Dourish (2001, pp.19) argues that designers of interactive systems have increasingly come to understand that interaction is

intimately connected with the settings in which it occurs. Embodiment denotes the participative status of the artwork itself. Dourish states: Embodiment is about the fact that things are embedded in the world, and the ways in which their reality depends on being embedded. (Dourish, 2001, pp.18-19) The interaction with the artwork, and in some sense its meaning, is defined both by the context it resides in (it is place specific) and the materials it chooses to employ for interaction (its measure of tangibility). So how exactly does the notion of embodiment apply to digital art?

2.1

Embodiment in digital art

Embodied phenomena occur in real time and in real space, are concrete and particular, and gain their meaning through participative status as objects in felt experience. (McCarthy and Wright, 2004 pp. 17) Embodiment can be further defined by denoting four types of relationships that can emerge between people and objects: (Fels, 2000, pp. 13) 1. The person communicates with the object in a dialogue. 2. The person embodies the object. 3. The object communicates with the person. 4. The object embodies the person. The first kind of relationship is a mostly causal one. The work of art derives most of its meaning from the result that follows from the interaction with the user. Fels observes: it is cause and effect that is aesthetic. That is, the person exercises control on the device and the result is communicated back. The result is critical for evoking any emotional response in the person. (2000, pp. 13) Life Writer (Sommerer and Mignonneau, 2006) is an installation which is the subject of this kind of relationship. It employs user input, a string of ascii characters, as catalyst for meaning. By typing on an old typewriter the user generates small digital insects on a projected surface. The shape and size of these insects, as their behavior towards each other is hard coded into them depending on the message the user wrote. In the second kind of relationship the person takes ownership of the object with the aspiration of

mastering it. The object becomes an extension of its user, aesthetic value being derived from the act of using. In this type of interaction the emotional response comes from the control of the instrument rather than the result itself. The pleasure is in the doing, not the achieving. (Fels, 2000 pp. 14) This situation can be illustrated through discussion of the iPoi project (Sheridan, et al. 2006). iPoi are small sphere like devices which can be can be swung around the body as part of an expressive dance form. When used, iPoi trigger projected visualizations and electronic sound schemes depending on the users movements. As the poi user becomes more skilled manipulating the devices, the animations and melodies would become more intricate and coherent. Another type of embodiment exists in the communication from object to person. In this exchange the interaction is one-way; the object is not in any way affected by the presence of its beholder. Fels explains this in his description of how traditional art communicates with its audience: A painting on a wall may evoke a response in a person looking at the painting; however, the response is a function of introspection by the person. The painting does not alter its signals in any way depending upon the person. (Fels, 2000, pp. 14) Holthuis Bitquid (2009) (discussed in more detail later in this essay) can be classified into this category, while still interacting with the environment; it runs autonomously from its audience. The installation does not necessitate in-the-moment interaction in order to a produce meaningful, art-like result. The last form of embodiment denotes the complete surrendering of the user to the object. Meaning evolves from being submitted to the rules and limitations the object imposes onto the user. In this situation, the person derives an aesthetic feeling through relinquishing control of himself so that the object can manipulate him. [] For this type of relationship, the object must be able to control the person and the person must be in a state to allow the control. (Fels, 2000, pp. 14) The transmedia performance group CREW engages their audience in such a relationship in their performances. For instance, their performance Eux

(2008) is a complete sensory experience, in which the audience not only participates but is fully enveloped in the performance. They call their participants immersants, as they are immersed in a world of sensory illusion. Using an omnidirectional video camera on a trolley and an array of supporting electronic devices the illusion is created that the immersant encounters himself in an abandoned French monastery. In Eux performance blends with being, the immersants live the experience and meaning evolves from full embodiment in the experience.

such a complex way that its states could never be fully comprehended. (Holthuis, 2009) Holthuiss installation embodied both the virtual and the real and combined them into a hybrid representation of both in the shape of a series of warped, physically altered re-digitized imagery.

Figure 1: Eux introduces immersive technology in its performance. The audience is no longer a passive spectator but becomes the center piece of the performance. Participation means being fully immersed in an alternate reality with all senses.

Figure 2: Bitquid examines the convergence between the analogue and the digital and the real and the virtual. One image from Holthuis series was a analoguely reproduced representation of a digital photograph of a chicken egg.

2.2

Embodiment as subject to art

The notion of embodiment has been an inspiration on a conceptual level as well. Jeroen Holthuis installation Bitquid which featured at the 2009 edition of the STRP Arts and Electronics festival in Eindhoven extracted digital information into the physical world by transcoding the bits that made up digital imagery into a constant jet of bubbles in a translucent liquid which was pumped through a tubing system. The bubbles were then transcoded into bits again and a copy of the original digital image was rebuilt. In the conversion process the properties of the digital file were altered by influences from the physical world; the shape and speed of the bubbles, the lighting conditions of the room and the length of the tubes. The installation visualised how the physical and digital world can converge and directly influence one another. Holthuis also mentions the qualities and limitations of the differences between these worlds: In contradiction to digital information, which consists out of ones and zeros (which in fact are analogies for electrical or magnetic loads), and thus only exists in two states, a fluid behaves in

Another work of art to which the suggestion of embodiment can be related is Peter Ablinger, Winfried Ritsch and Thomas Musil's installation Deus Cantando (God Singing). (Ars Electronica, 2011) Deus Cantando consists of a computercontrolled piano which is capable of playing up to sixteen keystrokes per second using eighty eight mechanical fingers. Instead of playing music, the piano 'speaks' to its audience by transcoding the frequency spectrum of a spoken text into a sequence of notes and chords. The interpretive capacities of the human brain then transform what were initially abstract musical structures into a sequence of words in a human language. It is as if the piano literally speaks out. The piano recites various texts about the topic of sustainability and environmental friendliness. In a way the installation comments on technology as a tool for both creating and destroying. In producing its voiceless solo, the piano is no longer a musical instrument but becomes a hybrid object; it embodies properties of both man and machine.

Figure 3: Deus Cantando explores the borders between human and machine, a computer controlled piano mimics the human voice by playing sequences of musical notes.

Figure 4: In the intermedia dance performance Mortal Engine by Chunky Move responsive projections are overlaid on top of a live performance.

2.3

Tangibility and its expressive potential

2.4

Situated interaction as prerequisite

Equally important is the notion of tangibility which plays a key role in many digital artworks. Tangible interaction, through tangible interfaces has been described as coupling digital information to everyday physical objects and environments (Ishii, Ullmer 1997, pp. 2). As means of illustration I draw on the example of Fields and Thresholds (Dunne & Raby, 1995) wherein two steel benches in different cities were augmented electronically to allow the people that used them to interact with one another in an unforeseen way. When one person sat down on one bench the corresponding spot on the other bench would warm up. When this spot was occupied by another person, a bi-directional sound channel was introduced. The resulting transition became more audible and free of static the longer the participants remained seated at their respective locations. This allowed people to communicate their feeling of connectedness over distance or prematurely cancel their ephemeral relationship. The concept of tangibility also transfers to the domain of performance art. For example, the intermedia dance performance Mortal Engine by dance company Chunky Move (2010) features responsive projections overlaid on top of a live theatre performance. The movements of the actors are recorded and tracked using a computer vision algorithm which consequently triggers the relevant projections. The projections adopt the physical properties of the actors, their movements, size and location when in turn the actors are augmented with colors and animations by the projections that overlay them. The actors and projections embody and holistically become the performance.

Dunne and Rabys concept in a way also dealt with location specific, situated interaction. The experience of remotely feeling another persons body heat while being seated on the bench, was exclusive to the place and context wherein it had taken place. The artwork derived its meaning from its context of use (being seated, which can be viewed as a kind of tangible interaction) and its locus, the public space. Another work, entirely dependent on the environment in which it is deployed is Gunnar Greens and Frederic Eyls Parasite (2005). Parasite is an interactive installation specifically designed for use in the subway. It features an independent projection-system which can be attached to the outside of a metro train. The installation projects surrealist imagery on the walls of the subway tunnels, augmenting the normally dark and gloomy environment with beautiful visualizations. Parasite necessitates the tunnel network in order to display its contents, reinterpreting the walls as digital canvas.

3. DIGITALLY ENHANCING PERFORMANCE AND INSTALLATION ART

Perhaps one of the most well-known artists fusing arts and technology is the Cypriot-Australian performance artist Stelarc. Stelarcs thesis that the human body is obsolete is central to his work. In Third Hand (1980) he wrote words on a transparent surface using an electronically controlled prosthetic hand mounted onto his real hand. Using his real hands and the prosthetic hand he was able to simultaneously write different letters of the same word. In his performances Stelarc augments not only his shows but also his body with electronics and robotics.

Stelarcs work illustrates one vision of cyborgs. His work explores the interplay between selfcontrol of the body and its control by the technological logic embodied in prosthetic devices. (Dunne, 1999, pp. 31)

chapter I will further examine which part interactivity can in different types of digital live art in the setting of the public space.

Figure 7: In Composition on a Table the user can make generative music by manipulating the directional pegs which control the direction of sound generating bouncing orbs.

Figure 5: Stelarc writing the word evolution using both his real hands and electronically enhanced prosthetic hand.

4. DIGITAL LIVE ART IN THE PUBLIC SPACE 4.1 Creative expression through public performance
The online performance group Improv Everywhere organizes periodic flash mobs and events of predetermined public performance. They carry out improvisational pranks which they call missions, taking their motto we cause scenes quite literal. Their public installation Say Something Nice (2011) was placed in public areas around New York. The installation consisted of a wooden lectern and a megaphone holster. Adorned with a sign that read Say something nice, the installation called out to passersby to make a cheerful public announcement. Say Something nice was produced as part of the Guggenheim Museum exhibition stillspotting nyc which focus was to highlight the need for mental and physical space in a city that never sleeps. While being low-tech, the project still caused a lot of commotion. Perhaps more driven from a technological stand point, Mann et al.s (2007) cyborg street performances consisted of a series of interventions in the public space. While wearing a specially designed electronics-enhanced suit, the wearer would be able to play musical notes by touching jets of water in public fountains and by striking the surface of urban materials like sidewalk bricks and pool tiles. The recorded sound would then be digitally altered using computer vision software to gain musical qualities by making the resulting sound resemble the sound of a bell.

The interlacing of the physical and the virtual takes center stage in the work of interactive media artist Toshio Iwai. In Morphovision (2005) solid 3D objects are projected on with a specially designed projection system, creating a digitally distorted optical illusion.

Figure 6: In Morphovision an optical illusion is achieved by projecting on objects which are rapidly spun around on their x axis.

Iwais Composition on a Table (1998) features a series of projected interfaces allowing the user to create generative music. By manipulating the directional pegs the user can change the direction of a set of moving orbs. Each orb represents a different octave. Each time an orb passes through one of the pegs a note of the concurrent octave is sounded, which note is played depends on the direction the corresponding peg is facing. Composition on a Table is. Over time the notion of interactivity has come to play a more prominent role in Iwais works which came to include gestural interfaces and generative, aleatoric music creation. I the following

4.2

Interactive cinema

The earliest example of what can be classified as interactive cinema is Radz ineras Kinoautomat (1967). At nine points during the film a moderator would enter the stage and ask his audience to choose between two scenes. After a voting session the appropriate scene was played, producing a non-linear storyline. ConFIGURING the CAVE (Shaw, et al. 1996) is an interactive video installation which has a wooden puppet as user interface. The computer generated imagery projected on the surfaces around the user and the accompanying sound composition can be influenced by manipulating the puppet. The artwork goes beyond a mere cinematic experience by exploring space, performance and a new bodily interaction language. MSNBCs Newsbreaker Live (2007) was a place specific game which converted cinemagoers into human joysticks. By waving their arms the cinemagoers could collaboratively control an onscreen paddle which was used to bounce a ball on the screen to destroy a virtual brick wall. Tangible and gesture based interaction is not exclusive context of the cinema. The next part of this text examines how these fields can be of value for performance and installation art.

Figure 8: Jeroen Hofs known under the name Eboman, uses a sensor suit to remix recorded video and audio samples in his live performances using a SenSorSuit.

4.3 Tangible and situated interaction in performance and installation art

Tangible interfaces and situated interaction have proven to be contributory to public performance. The work of Jeroen Hofs illustrates this development. Jeroen Hofs known under the name Eboman, uses a sensor suit to remix recorded video and audio samples in his live performances using a SenSorSuit; a haptic gestural interface worn on the body and controlled by movement. In one of his performances in Utrecht in 2008 he uses the SenSorSuit to make improvisational electronic music by resampling and granulating prerecorded and live audio and video material which is displayed on a video wall. In a more recent project (Wikivideo, 2011) he also enables people to upload video samples to his website which he uses in his live VJ performances at clubs and festivals.

On another account, tangible and situated interaction have also proven to be relevant to installation art. Golan Levin exemplifies this in his installations. Levin makes art that literally looks back at you. His installation Double-Taker (2008) was stationed over the entrance to the Pittsburgh Center for the Arts. Double-Taker is was a two and a half meter long robotic inchworm which responded to the presence of people around it. A computer vision algorithm makes the installation react in unforeseeable ways. The robotic worm had a character of its own; it would express its curiosity by bending towards the onlooker but would cower away once the person came closer. Levin plays with size and scale and the notion of gestural choreography. His installation internalizes aspects of human behavior and has an identity of its own. It not only embodies tangibility and situatedness, in a way it reflects on how the intellections of tangibility and situatedness apply to the public domain in a broader sense.

Figure 9: Double-Taker is an two and a half meter long robotic inchworm which responds to the presence of people around it. A computer vision algorithm makes the installation react in unforeseeable ways.

5. CONSIDERATIONS & FUTURE PERSPECTIVES

In this essay I have attempted to highlight the relevance and applicability of Dourishs notion of embodiment (2001) to interactive art. Building on Fels (2000) descriptive framework of embodiment in the relationships between art and its audience, I have further illustrated these relationships with contemporary examples from the art world. Moreover evaluation of the work of Holthuis, Ritsch et al., Sommerer and Mignonneau has shown that the phenomenon of embodiment is, in many instances, at the core of the theme and concept of the artwork, by making visible connections between the analogue and the digital, the virtual and the real and the organic and the robotic. Investigation of the work of Mann et al., Hofman and Ikais work has evinced that wearable, tangible and gestural interfaces allow for new interaction affordances which enable previously impossible creative expression. CREW and Shaw et al. offer valuable perspectives on performance art by highlighting the value of responsive and immersive environments. This feeds into the notion of situated interaction, art deriving its meaning from the context it resides in. This is exhibited in the Improv Everywheres performances and Dunne et al.s and Green et al.s place specific installations. Levins work goes as far as to offer a critique on the way we move in and take possession of publics spaces through the way the audience interacts with it. It is hard to offer concrete future perspectives, but it is my opinion that tangible and situated interaction will become ever more relevant to digital live art in an increasingly globalized and technocratic world.

[3]

6.
[1]

REFERENCES
Cohen, B. Greenberg, R. Joyce Tsai, K. (2010) Gogh Bike: Bicycles as a tangible interface for creative expression. CHI 2010, April 1015, 2010, Atlanta, Georgia, USA. ConFIGURING the CAVE. 1996. Center for Interactive Cinema Research. [ONLINE] Available at:http://www.icinema.unsw.edu.au/projects/confi guring-the-cave/project-overview/. [Accessed 04 October 2011].

Peter Ablinger. 2009. DEUS CANTANDO (God, singing) for computer-controlled piano and screened text . [ONLINE] Available at: http://ablinger.mur.at/txt_qu3god.html. [Accessed 08 October 11]. Dourish, Paul (2001). Where the action is: the foundations of embodied interaction. Cambridge, Mass.: MIT Press Dunne, A. (1999). Hertzian Tales: Electronic Products, Aesthetic Experience and Critical Design. London: RCA CRD Research. Dixon, Steve (2007). Digital performance: a history of new media in theater, dance, performance art, and installation. Cambridge, Mass.: MIT Press FACT | What's On | Roy Stringer Lecture Stelarc. 2011. FACT | What's On | Roy Stringer Lecture - Stelarc. [ONLINE] Available at:

[4]

[5]

[2]

[6]

[7]

http://www.fact.co.uk/whats-on/roy-stringerlecture---stelarc?listing_id=1457. [Accessed 08 October 2011].

[8]

[18]

Fels, S (2000). Intimacy and embodiment: implications for art and technology. In Proceedings of the 2000 ACM workshops on Multimedia (MULTIMEDIA '00). ACM, New York, NY, USA, 13-16. Gunnar Green and Frederic Eyl. 2005. Parasite. [ONLINE] Available at: http://ve.swxrrytq.vesrv.com/gunnargreen.de/html /?q=projects/parasite. [Accessed 08 October 11]. Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '97). ACM, New York, NY, USA, 234-241. Improv Everywhere. 2011. Improv Everywhere. [ONLINE] Available at: http://improveverywhere.com/2011/08/22/saysomething-nice/. [Accessed 08 October 11]. Jeroen Holthuis. 2009. Bits & Atoms: Designing the void in between. [ONLINE] Available at: http://www.jeroenholthuis.nl/wordpress/category/ bitquid/. [Accessed 03 October 11]. Kinoautomat, 1967. [Movie] Radz inera, Czech Republic: Czechoslovak Pavilion at Expo '67 in Montreal. Mann, S Janzen, R Lo, R Fung, J. 2007. Nonelectrophonic cyborg instruments: playing on everyday things as if the whole world were one giant musical instrument. In Proceedings of the 15th international conference on Multimedia (MULTIMEDIA '07). ACM, New York, NY, USA, 932-941. McCarthy, John & Wright, Peter (2004). Technology as experience. Cambridge, Mass.: MIT Press MSNBC. 2007. NewsBreaker Live. [ONLINE] Available at: http://www.msnbc.msn.com/id/24114408/. [Accessed 08 October 11]. Sheridan, J., Bayliss, A., and Bryan-Kinns, N. (2006) iPoi: Acceleration as a medium for digital live art. Demonstration. In Proceedings of UbiComp 2006.

Sheridan, J.G. (2006) Digital Live Art: Mediating Wittingness in Playful Arenas. PhD Thesis. Lancaster University, Lancaster, United Kingdom. Sommerer C. and Mignonneau, L. Life Writer, in Enter Action Digital Art Now, Aarhus Kunstmuseum catalog, ed. P.T. Dinesen et al. (AROS Kunstmuseum, 2009), 72-75.

[19]

[9]

7.
[1]

IMAGES
CREW, (2008), Still the immersive performance Eux [ONLINE]. Available at: http://www.crewonline.org/art/project/11 [Accessed 08 October 11]. Jeroen Holthuis, (2009), Stills from Bitquid [ONLINE]. Available at: http://www.jeroenholthuis.nl/ [Accessed 08 October 11]. Peter Ablinger, (2009), Still from Deus Cantando [ONLINE]. Available at: http://ablinger.mur.at/txt_qu3god.html [Accessed 08 October 11]. Chunky Move, (2010), Still from Mortal Engine [ONLINE]. Available at: http://www.youtube.com/watch?v=sbjOMualLVs [Accessed 08 October 11]. Stelarc, (1980), Still from Third Hand [ONLINE]. Available at: http://www.youtube.com/watch?v=sbjOMualLVs [Accessed 08 October 11]. Toshio Iwai, (2005), Still from Morphovision [ONLINE]. Available at: http://www.nhk.or.jp/strl/morphovision/index_e.h tml [Accessed 08 October 11]. Toshio Iwai, (1998), Still from Composition on a Table [ONLINE]. Available at: http://www2.kahbonn.de/1/34/0e.htm [Accessed 08 October 11]. Eboman, (2008), Still from Eboman performance [ONLINE]. Available at: http://www.youtube.com/watch?v=Zppy_57-hmE [Accessed 08 October 11]. Golan Levin, (2008), Still from Double-Taker [ONLINE]. Available at: http://www.flong.com/projects/snout/ [Accessed 08 October 11].

[10]

[2]

[11]

[3]

[12]

[4]

[13]

[5]

[14]

[6]

[7]

[15]

[8]

[16]

[9]

[17]