This paper proposes a set of ideas behind physical interfaces. It considers the types of communication that may arise as a result of the linking of body and mind. The main idea is that wearable technology can give people the option to simultaneously express ideas and thoughts through an ensemble type of dialogue.
This paper proposes a set of ideas behind physical interfaces. It considers the types of communication that may arise as a result of the linking of body and mind. The main idea is that wearable technology can give people the option to simultaneously express ideas and thoughts through an ensemble type of dialogue.
This paper proposes a set of ideas behind physical interfaces. It considers the types of communication that may arise as a result of the linking of body and mind. The main idea is that wearable technology can give people the option to simultaneously express ideas and thoughts through an ensemble type of dialogue.
hybrids of physical and virtual worlds Anne-Marie Skriver Hansen Abstract avatars arise from 3D miniature projected worlds gains This paper proposes a set of ideas behind physical popularity in Second Life, EVE and other 3D computer interfaces that provide us with the ability to express games. Since this scene of fantastic imagery can not abstract concepts in the hybrid of virtual and physical be experienced physically, it has been important to worlds. It considers the types of communication that simulate real natural forces to reach the feeling of may arise as a result from the linking of body and gravity and momentum that we use to orient ourselves mind, and it debates the use of stimulus in the com- to and navigate in the real world. The virtual reality munication with other people and our surroundings. By scene represents an inherent paradox: First it claims referring to the habituation process that happened, that it immerses people in phenomena that are not when people adapted to technology in the industrial- possible in the real world, but at the same time, it is ized countries, this paper demands that technology necessary to simulate the real world till such a degree now needs to adapt to us, so that we can expand the that the virtual world can provide the immersive experi- idea of how technology is used as a tool of expression ence that it promises. The other scene that has devel- and mediation. Here experience from other cultures, oped in parallel with the virtual reality scene was the where physical expression is part of the everyday dia- physical computing scene. A part of the physical com- logue, is seen as a valuable source of inspiration. The puting inventions are used as tools in virtual reality main idea presented, is that wearable technology can worlds to obtain a sense of visceral connection with give people the option to simultaneously express ideas the elements of the virtual world. Physical computing and thoughts through an ensemble type of dialogue. It technology has provided us with innovative tools of also suggests that it is possible to communicate with navigation and expression. By integrating all possible each other simultaneously in the co-creation of an body movements and signals, sensor devices merge experience. Finally, the ideas presented are linked to a the body with the medium - mind and thought does no specific attempt to address the topics discussed: The longer have to be separate. We can reach the bodily Vacuole Robes interface is described as an example experience of an imagined and abstract world. of a wearable interface that offers the opportunity for Enactive environments consisting of multi-variable simultaneous physical speech and co-creation of an real-time graphics, sounds and perhaps even physical experience. materials are mapped to multi-modal interface struc- tures that we can physically inhabit. The body has Introduction become a premise for experiencing non-substantial Upon having entered the 21st century, it is clear that content as if it was substantial. The idea of body-as- we have a choice to integrate our body in a close rela- interface has finally become an extensive research tionship with technology in our everyday communica- area. tion forms. This reverses the idea that our body is obsolete in the use of technology. Wearable technolo- This changes the our percepion of interacting with gy fosters new forms of physical and non-verbal com- each other, because now, we are able to ascribe new munication that challenge the dominating verbal lan- meaning to physical movement. The dismissal of the guage. New levels of communication with each other body as-interface would be to regress to single point and our surroundings are possible, because the con- and click navigation. We are no longer consumers of a tent-based digital media can be linked to postures and medium that offers us a narrow selection of possible gestures that express stored-, projected- and meta-lay- routes to follow. We now have the opportunity to use ers of meaning and thinking. So far, two parallel interfaces that allow us to manipulate content as if it scenes inside digital media have developed alongside was a piece of clay. We can choose to challenge our with each other. They have also benefited from each role as a passive audience and actively participate in a other while learning from mutual experiences. The first non-verbal conversation - become virtuosos within the scene, which was prominent in the 90's is the scene of ranges of expression that media artists, designers virtual reality, where people are "immersed" in the architects and engineers challenge us to explore medium. The realization of a "platonic" world scenario, through interfaces that address our physical engage- where story telling, imaginative landscapes and ment. We have the opportunity to let our body become a resonating organ of our surroundings and other peo- Intelligent Agent 8.1 social fabrics.skriver-h hansen.01 ple's inner worlds. We have the tools to establish a Our bodies cannot and will not adapt to the visceral connection to our mind, and if we manage, we current array of electronic communication tools that can share this amongst each other at the speed of are designed to reach the lowest denominator and movement and touch. often homogenize our ways of expressing ourselves. We need to be aware of the limitations of these inter- The "Slush-C Condition" - a Risky Moment of faces in terms of their conceptual design, resolution Transformation and modality. Furthermore, the commercial require- A "risky" moment of adaptation between technology ments of mainstream product quality need to be and body is currently introduced and debated by the addressed and we need to impress upon the product media art world. The adaptation is "risky", because industry that wearable technology can be integrated limitations of resolution and speed and the size and with individual and personal needs to a much higher amount of sensor equipment can "slow down" the way degree. If the interface was able to grow with a per- we express ourselves. It may limit us and seem primi- son's newly obtained bodily knowledge and skills, it tive compared to the speed with which we communi- could possibly diverge from the stasis condition that cate with words. But there are already limitations most technology products encounter, when designed inside the traditional media based communication to satisfy a limited amount of functions. If body inter- forms, where static graphical icons and the written and faces were physical platforms for software that spoken word cannot be experienced in any visceral address complex human conditions, the same physical sense. Furthermore, the word is single-threaded and interface would become multi-functional and open to temporal, while physical communication is multi- newly discovered ways of using it. This will support the threaded. [1] development of a completely new vocabulary for phys- ical speech. By positioning ourselves in a concentrated test area that will provide a cocoon-like shelter for our bodily Physical versus Mental? transformation into an ultra sensory multi-modal inter- There are several connotations related to the dichoto- connected and expressive unit, we can address my between "physical" and "mental". The most domi- aspects of how technology becomes symbiont with our nant connotation is that physical action is "primitive", needs to physically express and experience the and mental activity is "sophisticated". Humans have abstract process of thinking. The way we perceive the developed through the activity of building tools that different types of media, when they are closely con- connect our body to our surroundings in a functional, nected to physical activity needs to be studied along complex and sophisticated relationship. The tool mak- with the speed and intensity of gestures and postures! ing process is based on layers upon layers of physical A new discourse inside physical computing design and experience, where we have learned through a combi- new media art can for example involve several thou- nation of conscious and unconscious, mental and sub- sand years of erudite human experience with the link- liminal organization of physical actions and reactions. ing of body and mind. This profound knowledge can Even though we have removed ourselves from the provide the theoretical background for physical inter- bare physical action of "labor", we are still in contact face experimentation. This will include physical exer- with the surrounding world, however, through several cises in the shape of ritual behaviors that are perhaps levels of abstraction. The problem with the current not symbolic, but used as tools to obtain mental expe- abstraction level is that it appears to be removed from riences through specific physical performance. It will the premises of the body. Instead of controlling things include processes of habituation with the various types while being physically active in the situation, we "play of interfaces, so that mental activity can be transferred back" a set of selective actions that have been pre- into precise and meaningful physical activity. This composed and presented in front of us. As an observ- experimentation extends beyond the creation of physi- er, we activate a set of subtasks through switches cal exercise programs, where the goal is to entertain labeled with icons. In this position, we are so far the user during the time of an exercise, and where the removed from the action situation, that we have diffi- physical movement by itself does not provide any culties inventing new tools that address the situation meaning. The goal of the current experimentation with as such. The labeling of actions also extends to the physical interfaces could be to stimulate the physical labeling of what we call users into target groups, communication of the user as a speaking person. where interaction strategies are developed according Currently, experiments inside contemporary dance per- to the user's demographical relationships, education, formance have established an interesting explorative access to technology, life style etc. These are artificial connection between dancer and medium, where mod- labels that perpetuate inequities. Instead the user's sit- ules of artificial intelligence notice, and in some cases uation, when encountering the physical interface, respond to the dancer's movements. (Here I will men- could be analyzed according to actor-defined purposes tion research groups like Soniacillari, The Open Ended and action strategies. [2] If today's wearable technolo- Group, Proyecto Bipopus, Troika Ranch and gy is positioned in a synergetic relationship with our Palindrome). body, it will adapt to the way we experience a situa-
Intelligent Agent 8.1 social fabrics.skriver-h
hansen.02 tion, as it follows the bodily perception and adjustment select and share pre-composed material that we can of the surrounding world. Here, the two seemingly dis- collect and "modify" according to our own individual parate processes of being physical and mental con- needs. The mass-produced spectacle of pre-com- verge. posed expressions helps us relate to the part of the mainstream community that we find pleasing. This Does technology disconnect bodily requires some degree of engagement, but these col- features? lect-and-combine-interfaces quickly become dusty Throughout the industrial age, machines have been libraries of media material, as collections get bigger. It developed to efficiently replace labor. In order to reach is difficult to navigate in the huge amount of static the freedom of communication over a distance, and material that is available. As an action situation passes the release of time consuming action, technology has by, it may be cumbersome to select and activate the become a surrogate for actions that we find "unneces- right kind of material for the right kind of action. But sary" to perform. The only physical act that we have when technology merges with our body, transforma- not labeled "unnecessary" is the sexual performance. tional media types can evolve naturally with the action This brings us to the point, where any physical activity flow. Because we start to carry mobile pieces of intelli- can possibly be related to the sexual act. The gence that can be communicated through physical restrained body language in many industrialized activity, we include the features of our body and are (Western) cultures shows that the body is no longer now able to not just play back an action, but to used as a tool of communication. Unless we are express it with a flexible and dynamic medium. The trained sportsmen, musicians, actors or dancers, our D.I.Y. community explores the possibilities of "hand- body's ability to make complex physical movements made electronics' that challenge physical skills: First in has become unfamiliar to us. What is left is the deco- relation to musical performance tools, where gesture is ration of our body, where fashion styles carry refer- essential, but also in relation to people's sphere of ences to other cultures, where physical communication social connections, where the physical signaling of is still part of the everyday dialogue. This type of deco- desires and needs has become an innovative, some- ration carries static symbols of how physical action times provocative form of communication. The genre used to provide a special level of communication. of design noir takes into account that the body has an Static symbols are as 'compressed' layers of meaning infinite amount of expression features that electronic difficult to unwrap and explore and sometimes we devices can help "legalize" the use if in social situa- have even lost contact with the original meaning of the tions. [3] A concrete example of this is Maywa Denki's symbols that we refer to. tools and gadgets that are quirky, imperfect pieces of technology that have been developed from the desire When information is compressed in static symbols and of acting physically upon desires and needs. The cus- icons (like the ones we navigate with on the computer tom made electronic devices behave with their own screens), we need to be professional in order to "per- personality, when operated by a human. [4] form", in order to be able to unpack and reconstruct icons of action, so that we can communicate impul- The Body-A As-IInterface - a Tool of Sensation sively, intuitively and creatively. Personal trainers, spin- and Expression doctors and technology help us with that. In talent The body-as-interface consists of three key concepts shows people obtain their own individual professional- that are digitally advanced mutations of features that ism, and they are finally 'allowed' to act according to our body already contains: The body as an enhanced their desires. But their performance is still limited by a multi-sensorial organ, the body as a physical converter strict set of theatrical guidelines that do not change of abstracted meaning, and the body as an intercon- their iconic appearance. Their individual performance nected unit that immediately transfers sensed and in front of an audience does not give them the oppor- experienced material to other bodies. These enhanced tunity to share their professionalism with anyone features encourage active contribution to a physically directly. A physical interface that goes beyond point- mediated community of people, who make their ideas click navigation of pre-selected professionalized and thoughts manifest through visceral and physio- actions presented as icons makes it possible for us to aesthetic experiences. When digital information is learn and express ourselves simultaneously, as we deployed through physical gestures, there are possibil- practice towards becoming virtuosos. If the medium is ities of a multi-level transfer of information. This adds provided as a material that can be shaped, instead of another dimension to the understanding of the mes- being a collection of static symbols, we can construct sage sent, because meta-levels of meaning exist our own dramatic and meaningful language through between the bands of expression that are made possi- physical behavior. An entire new diversity of languages ble with a multimodal interface. New movement- and can enter the sphere of communication, and these lan- action-based memes will start to populate the sphere guages can be learned through experiencing process- of language, as we currently know it. The concepts of es rather than memorizing icons and functions. transmission and reception will be focused on a Currently, the majority of electronic gadgets help us to process, where information permeates our body before Intelligent Agent 8.1 social fabrics.skriver-h hansen.03 it reaches our brain, and the idea of the body as a res- discussions need to follow along with technology that onating organ will be further elaborated. Interfacing is developed to either invade or enhance the body and possibilities will be far more holistic in their nature. Co- discuss the fact that people can tinker with each culturing with plant and animal life will be prominent, other's bodies and hack features that their body has when we can learn physically from their ecological been given. We need to discuss how sub-cultures can structures and transfer our experiences to robotic sys- design their own custom made devices and modify the tems. The idea of the person as a user and consumer body, and in the most radical sense form sub-species of a product will disappear, when technology merge that differ by life style, social connections and physical with body-related features and is adapted by us as languages. This is not unlike what we see in local organs that interface us with other people. We will communities, where cohorts of people develop special become participators of a social and physical situation skills, language and a common sphere of understand- and (re-)discover the possibilities of being able to co- ing. Like we customize our clothes and electronic invent a story, an environment, or space of information devices, we customize the way we communicate with as it unfolds through the ensemble of actions. our body. By mastering customized body tools, various groups will diverge towards the specific kinds of com- Interconnected Body Tools - Sensations as munication that they share in public and private Transfer of Knowledge spheres. A person, who wants to communicate with Tools of communication that are attached to our body different sub-communities, must then develop a versa- and perhaps even embedded in it make us able to tile array of modes and skills that can be attached and communicate through actual actions. Here an action deployed as needed. does not need to be re-formulated and interpreted through words and texts, before another person under- Communities of Performers stands and can respond. Body interfaces hold people As groupings of physical communication methods in the experience of the situation. The physically arise, a person may be forced to acquire skills in order expressed medium that contains thoughts and ideas is to transgress private, perhaps even esoteric language immediately transferred. This means that instead of systems that have arisen within a community. A domi- collaborating about reaching a goal that is first set nating language like English homogenizes communi- through a verbal conversation, people can co-con- cation and makes it easier to communicate across bor- struct the goal with a multi-threaded action-based con- ders of understanding. However, it does require that versation. In addition to this, the body related inter- you learn how to speak the language. How difficult faces could even provide visceral stimulus, so that would it be to learn non-verbal languages that are information can be immediately transferred from one expressed through physical gestures and correspon- person to the other. It does however require a person ding media forms? In a community of performers that to adapt to the interface so that s/he can learn what act together in an ensemble type of constellation, the the sensations mean. But when learned, the stimulus person, who is foreign to the media situation generat- based information transfer can be multi-threaded, ver- ed, would need to understand the physio-aesthetic lan- sus the language based information transfer that is guage that is shared among the people, who partici- single-threaded. People can acquire knowledge by pate in a non-verbal conversation. The question is, if harvesting sensations from others in a space. When these language forms share some universal character- directly interfaced with other people, animals, plants istics? Would there for example be a basic platform, a and materials, bits of information can freely traverse scene, or an interface that is embedded in the archi- the space of a situation and be manipulated by the tecture of a "conversational space" that gives people actions performed in that space. the possibility to tune in to each other? Or what if the body-related interfaces share some common features Body Interface Speciation and differ on others, so that the medium will be trans- The question is: How many physical information trans- formed along its trajectory through people's actions? In fers can our body handle? Is there a risk that "mind contemporary dance performance, where media is part space" will overwhelm and replace the physical inputs of the choreography, there has been an interesting we get from the unmediated world? Imagine an inter- integration process between body and media. The per- face that helps you "switch off" basic human sensa- formance aspect can make it easier for a person to tions such as pain. What if you cannot differ between decode what is being said. Movement is in general an the pain that comes from the physical environment and overt activity. As soon as a newcomer starts to partici- the pain that a physical information transfer is pro- pate in the physical dialogue, the elements of improvi- grammed to provide you? A group of people can agree sation and performance may require practice, but on developing an immunity system that operates since the newcomer's movements are immediately vis- against the basic sensations that help us navigate in ible to other people, a mutual adjustment process the world. If thoughts are mediated, so that they affect could be a natural and intuitive result of the newcom- our body radically, wearable technology can be used er's attempt to engage. Furthermore, physical move- as tools with limitation and control purposes. Ethical ment can be done simultaneously and still be compre-
Intelligent Agent 8.1 social fabrics.skriver-h
hansen.04 hensible. To lighten the burden of having to practice world through medium generation. The robe blinds the several physical languages, the sensor devices and person, who is inside it in order to draw attention to the media that is generated as a result of physical the co-invention of the sound medium that the two movement can for example be designed in a modular people generate. Through the delimiting character that fashion, so that there is room for an integration the interface has, it is the idea that the feeling of being process among different conversation groups. enclosed in a concentrated space can evolve a sense Common key nodes in possible forms of expressions of connectedness. could be embedded in the physical interfaces that peo- ple use as conversation tools. The idea of a utility fog The sculptural aspect of the Vacuole Robes proposes envisioned by John Storrs Hall could in this case be an interface that connects with other species and it realized as media units that contain introductory suggests speciation through special interface features. modes of expression that automatically connect to When veiled by the robe, a person's anonymous other people's media units, so that a common lan- appearance puts his/her movements in a different light, guage is possible. [5] In general, I think there may be where person changes identity and becomes a plant a peculiar concurrence between sensation and expres- or an animal. Several people that wear this interface sion when the expressed medium resonates in a new- would form a sub-culture, a forest of sculptures that comer's body. both communicate and perform in an ensemble type of unit. The Vacuole Robes is one interface among many The Vacuole Robes - a Body Interface that others to come, where people start to interface them- allows Simultaneous Contribution to a Non- selves physically with each other and their surround- ings. It is an attempt to demonstrate a set of ideas verbal Conversation regarding the linking of body and mind. These ideas The Vacuole Robe interface is an attempt to reach a can be explored even further, so that new interfaces stage, where two people can transfer ideas and help people to act in the situation they are presented thoughts with simultaneous physical movements. The to in the real and the virtual world. I hope that the situation that this type of interface aspires to reach is invention of new tools will address people's life situa- the moment, where ideas and thoughts are transferred tions and life conditions in hybrid spaces and reach from one person to another through the establishment much more profound levels of communication! of a common condition - a type of condition that requires simultaneous contribution of all the participa- tors. This form of conversation differs from the verbal References conversation, where an idea can only be transferred in [1] Buxton, Bill, The Natural Language of Interaction, small portions over time and cannot be felt simultane- INFOR vol. 27, May 1989 ously in a commonly experienced condition. The [2] Dervin, Brenda and Louis Foreman-Wernet with cocoon-like design of the Vacuole Robes puts the peo- Eric Lauterbach. Sense-Making Methodology Reader. ple, who use them into the "slush-condition" of being Hampton Press Inc. 2003 physically engaged with technology till a degree, [3] Dunne, Anthony and Fiona Raby. Design Noir - The where their movements can form meaning in the Secret Life of Electronic Objects. Birkhäuser 2001 shape of a medium. However simple the technology in [4] Denki, Maywa. The Three Lines of Maywa Denki the robe interface may be, it proposes a form of physi- Products. 2008. Maywa Denki Official Website. 14 cal interface design that involves the entire body seen January 2008 as a speaking unit. The condition that the interface <http://www.maywadenki.com/english/00main_e_con- immerses people into, establishes a common ground tent.html> for a specific type of mutually generated speech. Each [5] Hall, John Storrs. Utilitu Fog: The Stuff that Dreams Vacuole Robe surrounds a person till such a degree are Made Of. Autogeny, 11 January 2008 that a person is forced to experience the surrounding http://autogeny.org/Ufog.html