Title page

MINDtouch - Ephemeral Transference: Liveness in Networked Performance with Mobile Devices Camille BAKER

A thesis submitted in partial fulfilment of the requirements of the University of East London for the degree of Doctor of Philosophy

October 2010

i/ thesis abstract
This practice-based thesis investigates the four key qualities of 'liveness, feltness, embodiment and 'presence' in mobile media performance, in order to shed light on the use qualities and sensations that emerge when mobile technologies are used in tandem with wearable devices in performance contexts. The research explores mobile media as a non-verbal and visual communication tool that functions by repurposing the mobile phone device and its connection to a wireless network, not only for communication but explicitly for the expression of emotion in the form of a video file representing an interpersonal connection shared over distance. The research aims to identify and supplement existing scholarly discourse on the nature of these four key strands of kinaesthetic philosophy made live in the online network, applying knowledge gained through the practice of enhancing participant experience of the use of simple ubiquitous mobile tools with bespoke biofeedback sensors and an online repository for the playback of users visual expressions. This enhanced toolkit enables participants to share personal relationships and social interactions in an immediate way, with collaborators at a distance. The selected methodology of active research using kinaesthetic tools in live performance seeks to identify and clarify new ways of simulating or emulating a non-verbal, visual exchange within a social participatory context, with particular attention paid to a sense of feltness as an element of presence or liveness, and with attention to the experience of a sense of co-presence arising in real-time collaborative mobile performances at a distance. To best explore these concepts as well as the bodily sensations involved for participants, the thesis analyses original data gleaned from a larger R&D project (conducted in tandem with this thesis project, sponsored by the BBC) as its major case study. The project, called MINDtouch, created a series of unique practice-based new media performance events played out in real-time networked contexts. The MINDtouch events were framed as a means for participants to simulate dream exchange or telepathic thought transfer using mobile phones and biofeedback devices, linked to a bespoke video file protocol for archiving and sharing visual results. The corporeal, non-verbal forms of communication and visual interaction observed when participants use such devices within participatory performance events is examined by way of demonstrating the impact of specific live encounters and experiences of users in this emerging playing field between real-time and asynchronous, live and technologised forms expressing liveness/presence/distance. The thesis benefits from access to the larger MINDtouch project and its original data, providing this research with a set of process-based evidence files both in video and transcript form (contained in the thesis appendices). By analysing this unique data set and applying the theoretical contexts of kinaesthetic philosophies where appropriate, the thesis demonstrates both the practical and the critical/contextual effectiveness of the media facilitation process for the participants, and shares their senses of liveness and presence (of themselves and of others) when using technology to externalise visual expressions of internalised experiences. This thesis makes an original contribution to scholarship in the fields of Performance and New Media, with additional contributions to the cognate fields of Philosophy and Technology, and locates its arguments at the locus of the fields of Performance Art, Mobile Performance/Locative Media, Philosophies of the Body and Communications. The thesis uses methods, practices and tools from Phenomenology, Ethnography, Practice-As-Research, and Experience Design, bringing together the relevant aspects of these diverging areas of new media research and media art/performance practices. The research demonstrates that there is a need for new technological tools to express viscerally felt emotion and to communicate more directly. It is hoped that this study will be of use to future scholars in the arts and technology, and also that it may help to demonstrate a way of communicating rich emotion through felt and embodied interactions shared with others across vast distances (thus supporting political movements aimed at reducing global travel in the age of global warming).

ii/ acknowledgements
I wish to acknowledge all the people who have made both this academic thesis and the larger research project possible. In such a large project, there are many to thank. Primarily, I would like to thank Professor Lizbeth Goodman for all her hard work both professionally and personally in terms of coaching, encouragement and provision of opportunities, for being an amazing friend and confidante, for hunting down and finding funding and resources for me and all the SMARTlab PhD cohort; for all her love and support, and all the tireless work she does for all of us and for the community and the domain of education. I must of course thank the invaluable team of SMARTlab supervisors who joined me on this doctoral project in the last year of the process: an A team recruited specially to assist me in my last dash to the finish line: Dr Ryya Bread, Dr Sher Doruff and Dr Allan Parsons. I would also like to thank my friends from the adhoc online PhD support group provided both within and beyond the SMARTlab cohort, which began before I left Canada in the spring of 2007: Dr Lone Koefoed Hansen, Cynthia Poremba, and Valerie LaMontagne friends and valuable peers, reviewers and supporters. Although we had different topics, our work was related enough to allow us to share reading, theory and links to ideas and networks. The group provided a vital source of encouragement and peer support, not only on the research but also on the personal issues that often overlapped with our work. This group also provided consistent and intriguing intellectual engagement and encouragement, as did the cohort of PhD students and friends from SMARTlab, including the high level peer support developed in the later stages of thesis writing from the SMARTlab senior citizens (e.g. those of us who had spent longest in the programme by year 4 and were thus nearest to completion), for their amazing support and insightful inputs. Their positive influence on both my life and my studies has been considerable. Thanks belong with all the SMARTgang and particular thanks go to: Kasia Molga, Anita Mckeown, Denise Doyle, Taey Kim, Kate Sicchio and Turlif Vilbrant. A project of this technological complexity could not have been achieved without the help of numerous engineers and experts in online systems and tools. In this regard, I want to thank Michael Markert for his diligent work in technical development and innovative contributions to the larger BBC project that informs this thesis: particularly to the invention of unique ways to link the electronic and software tools required for visualising body data and video on mobile phones. Michael contributed considerably to the technical production of the BBC MINDtouch Project, despite his physical distance from the team. Thanks go to Michael for his patience and diligence as he worked through the projects challenges with the state of play in current technology. While he was not able to provide exactly what I had envisioned, his efforts paid off greatly in helping to make as much of the vision a reality as was technically possible in the timeframe of the project. Similar thanks are due to Manjit Bedi for his significant technical contributions in the late project stage to mixing visuals using software, with additional work from Evan

Raskob. I would like additionally to thank Dr Marc Price, Huw Williams and the BBC for their financial support, and for providing a framework within which an art project and a scholarly dissertation could both take shape, within the larger remit of an R&D project for the BBC. Marc Prices intellectual support, insight and advice as co-supervisor were also appreciated throughout the process of envisioning and then writing a thesis linked to, but separate from, the BBC project. In addition to the direction provided by SMARTlab itself, I would like to thank Dr Susan Kozel for her continual involvement in my graduate studies and conceptual development, since my Masters studies in Interactive Arts at Simon Fraser University (Vancouver, Canada: 20022004) through to the rooting of this new work in the UK at SMARTlab. Her rigour of thought and consistent thoroughness in feedback has helped and taught me immensely as a thinker, writer and artist/curator. Her attention to my work was in particular a support and source of comfort to me in the early stages of the research, as I not only went from one level of study to the next, but also from my home country to another country/home/life and career. Susans ongoing friendship and personal support, and her role as an effective professional artistic/curatorial collaborator between my graduate degrees, has been greatly appreciated and acknowledged throughout the thesis. I would also like to thank Dr Thecla Schiphorst for her positive input in my early work; collaborating with her and Susan Kozel on the whisper[s] project (discussed herein) had a significant influence on my current direction and career.

ii.1/ collaborative research notes and professional credits

While the thesis submitted for examination is a solo work that puts forth my own arguments, this work was achieved in the larger context of a practice-based R&D project and performance experiment using mobile technologies, sponsored by the BBC. While the ideas and arguments of the thesis are entirely mine, and while the conceptualisation, direction and performance of the practical components of the major case study were also mine, still a group of collaborators contributed to the larger project in specific ways that bear a mention here. I want to acknowledge and thank profusely all the people who worked on MINDtouch, and assisted me usually gratis with the technology, workshops and performances, especially: Perparim Rama and the 4MGroup, Tara Mooney, Rachel Lashebikan, Pawel Borkowski, Evan Raskob, Fred Brown and Steve Lauder of the Matrix East Lab at UEL (under the direction of Professor Haim Bresheeth), Toby Borland, Clilly Castiglia, Stanislava Mislanova, Vanessa Wiegand and Mike Murphy, along with engineers/programmers: Dr Li Zhang and Jeremi Sudol. The first performance experiment in July 2009 was guided by Cathy OKennedy, Kate Sicchio and Anita Mckeown, to whom thanks are also due. Other performance experiments involved too many people to mention, as did the archiving and analysis of the many hours of footage provided by the groups generous participants, who gave me permission to use their video expressions. There is not space here to extend my thanks further in print. There are many more to be made, but they must go without saying here.

iii/ statement of originality

I Camille Baker hereby declare that all the intellectual ideas and research materials included for examination in this thesis are entirely mine, and that the collaborations on the larger BBC research project (not included for examination) have been duly credited. Signed: research candidate, Camille Baker _____________________________ Date: ________________________ Countersigned: Director of Studies and PI of the BBC Project for UEL: Professor Lizbeth Goodman __________________________________ BBC R&D Principal Investigator/sponsor: Dr Marc Price __________________________________________________

table of contents
title page .......................................................................................................... 1 i/ thesis abstract ................................................................................................ 2 ii/ acknowledgements ......................................................................................... 3 ii.1/ collaborative research notes and professional credits .................................................................4 iii/ statement of originality ................................................................................... 5 table of illustrations (images and figures) in text ..................................................... 9 iv/ statement of the artist as researcher ............................................................... 11 v/ style notes .................................................................................................... 12 v.i/ fonts.........................................................................................................................................12 v.ii/ tense/first and third person voices ............................................................................................12 v.iii/ italics .......................................................................................................................................12 1/ introduction: ephemeral transference backgrounds and contexts ...................... 13 1.1/ setting the stage (and a note on placement of the distributed literature survey) ......................13 1.2/ MINDtouch summary of the major case study and its aims/outcomes ..................................17 1.3/ motivations, roles and voice.....................................................................................................20 1.3.1/ motivations .......................................................................................................................20 1.3.2/ key terms defined: 'liveness', 'feltness', 'embodiment', 'presence', 'co-presence', 'ephemerality', 'network', 'transference' .....................................................................................22 1.3.3/ voice ................................................................................................................................25 1.3.4/ roles .................................................................................................................................26 1.4/ methodologies.........................................................................................................................27 1.5/ performance context (Literature Review Part 1)........................................................................29 1.6/ mobiles, databases and network freedom ...............................................................................31 1.6.1/ mobile and database media..............................................................................................31 1.6.2/ dreams of networked freedom..........................................................................................33 1.7/ philosophical landscape (Literature Review Part 2) ...................................................................36 1.8/ connective tissue ...................................................................................................................38 1.8.1/ exegesis/thesis structure ..................................................................................................40 Part I: mobile media practice in critical context ..................................................... 42 2/ hybrid methodologies of practice .................................................................... 43 2.1/ art-as-research ........................................................................................................................43 2.2/ phenomenology.......................................................................................................................51 2.2.1/ journaling phenomenologically ..........................................................................................58 2.3/ ethnography ............................................................................................................................59 2.3.1/ interview process..............................................................................................................60 2.3.2/ performance ethnography ................................................................................................61 2.4/ experience & sensorial design..................................................................................................62 2.5/ embodied design / workshopping..........................................................................................64 2.6/ structured improvisation ..........................................................................................................67 2.7/ visual methodologies ...............................................................................................................68 2.8/ technology and software design methods................................................................................70 3/ participatory performance .............................................................................. 71 3.1/ participation as performance ...................................................................................................72 3.2/ politics of participation .............................................................................................................75

3.3/ presence in performance ........................................................................................................ 81 4/ video art + live cinema performance ................................................................ 87 4.1/ a video art journey................................................................................................................... 87 4.2/ live cinema .............................................................................................................................. 88 Part II: theory into praxis .................................................................................... 95 5/ mobile video aesthetics, non-verbal / non-linguistic ........................................... 96 5.1/ the emergent aesthetics of mobile video ................................................................................. 96 5.2/ non-linguistic mobile video communication ........................................................................... 110 5.3/ generative and database video ............................................................................................. 117 6/ networks, telepresence + connectivity ........................................................... 122 6.1/ telepresence and telematic performance............................................................................... 122 6.2/ electronic threads between us .............................................................................................. 124 6.3/ access / protocols / privileges............................................................................................... 128 7/ presence: permutations ................................................................................ 130 7.1/ presence: feltness sensing .................................................................................................... 131 7.2/ emotional presence............................................................................................................... 133 7.3/ distance + intimacy: mobile digital presence + connection ................................................... 136 7.4/ presence and telepathy......................................................................................................... 139 7.5/ collective consciousness, spiritual presence and out-of-body-experiences ........................... 142 8/ embodiment and affect ................................................................................ 149 8.1/ embodiment towards a working definition .......................................................................... 149 8.2/ embodiment of space / absence and the space between ..................................................... 153 8.3/ experience senses, perception and affect ............................................................................. 161 8.4/ liveness when is now?........................................................................................................ 170 Part III: conclusions & references for further study .............................................. 177 9/ thesis conclusion: ....................................................................................... 178 9.1/ implications and new directions ............................................................................................ 178 9.2/ in conclusion......................................................................................................................... 182 bibliography ................................................................................................... 183 Appendix I/ the MINDtouch project (BBC R&D sponsored) .................................... 193 A.1/ - technical appendix and background source material on mobile performance, practice and process ....................................................................................... 194 A.1.1/ mobile video collection workshops .................................................................................... 194 A.1.1a/ biosensors and electronics .......................................................................................... 196 A.1.1b/ garment design ........................................................................................................... 200 A.1.1c/ performance events ..................................................................................................... 202 A.1.1d/ participant expectations .............................................................................................. 205 A.1.2/ outcomes, analysis + interpretation: performances events ................................................. 206 A.1.2a/ workshop outcomes and observations ........................................................................ 207 A.1.2b/ performance discoveries, results + analysis ................................................................. 211 A.1.2c/ demographics ............................................................................................................. 212 A.1.2d / event survey feedback................................................................................................ 213 A.1.2e/ event analysis: production report ................................................................................. 215 A.1.2f/ participant response analysis ........................................................................................ 223 A.1.2g/ overall performance assessment ................................................................................. 227

A.1.3/ visual mixing software + streaming development process...................................................230 A.1.4/ video content categories + analysis ....................................................................................236 A.1.5/ mobile/ networks and connectivity......................................................................................238 Appendix II/ workshop materials + technical elements ......................................... 241 A.2.1/ technical timeline for technical development .......................................................................242 A.2.2 / workshop participant consent form ..................................................................................243 A.2.3 / workshop participation and non-exclusive licensing agreement .........................................246 A.2.4/ Participant Profile Information .............................................................................................248 A.2.5/ step-by-step videophone expressions workshop procedure...............................................249 A.2.6/ workshop mind quieting script............................................................................................250 A.2.7/ post-workshop questionnaire .............................................................................................252 A.2.8/ post-workshop interview questions ....................................................................................255 A.2.9/ workshop video file content database.................................................................................256 A.2.10/ technical updates and software guides.............................................................................266 Appendix III/ performance event materials .......................................................... 293 A.3.1/ streaming server set-up and operational notes ..................................................................294 A.3.2/ technical workplan..............................................................................................................296 A.3.3/ performance preparation checklist......................................................................................297 A.3.4/ performance preparation outline .........................................................................................298 A.3.5/ event technical preparation.................................................................................................299 A.3.6/ additional logs of facebook, website and online interaction .................................................300 A.3.7/ first performance: introductions script.................................................................................311 A.3.8/ performance mind-quieting script .......................................................................................312 A.3.9/ group guiding information...................................................................................................314 A.3.10/ performance event participant consent form.....................................................................320 A.3.11/ post performance audience survey...................................................................................321 A.3.12/ event participant experience feedback themes .................................................................323 A.3.13/ performance survey feedback ..........................................................................................326 .................................................................................................................................................326 Appendix IV/ collaborator and technical feedback ............................................... 334 A.4.1/ testimonials on network issues ........................................................................ 335 A.4.2/ screenshots of forum discussions on Nokia Realplayer issues ............................................336 Appendix V/ DVD Support Materials: Table of Contents ....................................... 338 A.5/ Notes ........................................................................................................ 339 A.5.1/ Disc 1 - participant workshop video clips ...........................................................................339 A.5.2/ Disc 2 - Workshop documentation videos and stills............................................................340 A.5.3/ Disc 3 - Performance videos and final VJ mixes................................................................341 A.5.4/ Disc 4 - Promotional materials ............................................................................................342 A.5.5/ Disc 5 Technical materials ...............................................................................................343 Appendix VI/ published papers relevant to the thesis ........................................... 344 A.6.1/ Liveness and presence in bio-networked mobile media performance practices: emerging perspectives (Dec 2008) ...............................................................................................................345 A.6.2/ aesthetics of mobile media art (July 2009) ..........................................................................365

table of illustrations (im ages and figures) in text Figure 1 2007 C. Baker - early technical diagram of technical development phases....................... 19 Figure 2 2007 C. Baker chart of project roles............................................................................... 26 Figure 3 2010 C. Baker Diagram of thesis knowledge domains.................................................... 39 Image 1 2007 C. Baker still from Dublin workshop, October 2007 ............................................... 66 Image 2 2007 C. Baker participants in video collection workshops .............................................. 73 Images 3 and 4 2003 Blast Theory Uncle Roy All Around You ..................................................... 76 Images 5 and 6 2007 Blast Theory, Riders Spoke ......................................................................... 77 Image 7 2004 - still from PDX_01 by NomIg in live performance .................................................... 90 Image 8 20042008, still from A/V performance [SILENT ROOM] film-pome, skoltz_kolgen .......... 91 Images 9 and 10 2007 Sue Costabile still from audiovisual performance ........................................ 92 Images 11 and 12 2005-2006 stills from A/V performance - Sara Kolster and Derek Holzer .......... 93 Image 13 2004 Steve Hawley still from Speech Marks ............................................................... 103 Image 14 2008 Mark Amerika still from Mobile Phone Video Art Classics................................... 103 Image 15 2009 Mark Amerika Immobilit ...................................................................................... 104 Image 16 2005 Dean Terry ........................................................................................................... 105 Images 17 and 18 2006 Max Schleser Max With a Kaitei........................................................... 106 Image 19 2005 Giselle Beiguelman, screen shot from sometimes never ....................................... 107 Images 20 and 21 2005 Giselle Beiguelman, screen shot from sometimes never ......................... 107 Image 22 2006 Tina Gonsalves FEEL.TRACE ............................................................................ 108 Image 23 2004 Tina Gonsalves with Tom Donaldson MEDULLA INTIMATA .............................. 109 Image 24 2007 C. Baker still from participant video during Dublin workshop, October 2007 ...... 112 Image 25 2005 whisper[s] project screenshot from a video I shot at SIGGRAPH conference ..... 113 Image 26 2005 whisper[s] project, showing the electronics, conductive fabric and designs ....... 113 Image 27 Phillips Design, Skin Probes .......................................................................................... 114 Image 28 Philips Design Skintile................................................................................................. 115 Image 29 1995 Char Davies, Ephemere ....................................................................................... 116 Image 30 1995 Kathleen Rogers, Psi Net ..................................................................................... 117 Image 31 2002 Lev Manovich, Soft Cinema ................................................................................. 119 Figure 4 2009 Camille Baker......................................................................................................... 130 Figure 5 2010 Camille Baker key ideas and theorists on embodiment ........................................ 169 Figure 6 2009 Camille Baker......................................................................................................... 170 Images 33, 34, and 35 2007 C. Baker participants video stills from workshops .......................... 195 Image 36 2007 Mind Media Nexus 4 Bluetooth Sensor transmitter........................................... 197 Images 37, 38, 39, 40 and 41 2007 Mind Media: electrode cable and sensors ............................... 197 Images 42, 43, and 44 2008 C. Baker: Michael Markert developing the custom electronics. ........ 197 Image 45 2008 C. Baker: custom biofeedback visualisation software. .......................................... 198 Image 46 2007 collaborator Brady Marks: vision of the mobile interface for social VJing............... 199 Image 47 2009 C. Baker: Quartz Composer early software patch view......................................... 199 Image 48 2009 C. Baker: Alpha / Garment 1 with embedded sensors.......................................... 201 Images 49, 50, and 51 2009 C. Baker: Beta / Garment 2 with embedded sensors....................... 201 Figure 5 2009 C. Baker - early technical diagram of the networked performance project .............. 202 Figure 6 2009 C. Baker staging and organisation performing groups live context........................ 203 Images 52 and 53 2009 C. Baker stills from July 2009 first performance ................................... 204 Images 54 and 55 2009 C. Baker................................................................................................. 204 Images 56, 57 and 58 2009 C. Baker........................................................................................... 205 Image 59 and 60 2007 C. Baker still images of participants in video collection workshops ........ 207 Images 61 and 62 2007 C. Baker - from Vancouver workshop June 2007 ................................... 208 Image 63 and 64 2007 C. Baker - stills from Vancouver workshop June 2007 ............................. 208 Images 65 and 66 2007 C. Baker workshop still Dublin July 2007 ................................................ 209

Images 68 and 69 2007 C. Baker - stills from participants videos Vancouver, August 2007 .........210 Images 70, 71 and 72 2007 C. Baker - stills from participants videos Dublin, October 2007 ........210 Image 73 2010 C. Baker screen shot of the Quartz Composer patch for video mixing................230 Figure 7 2010 C. Baker physiological sensors mapped to life-force qualities & visual effects.......231 Image 73 2010 C. Baker zoomed in screen shot of patch for video mixing from sensor data......232 Figure 8 2009 C. Baker demonstrating the technology connections ...........................................233 Figure 9 2010 C. Baker demonstrating the server-side processing .............................................234 Figure 10 2009 Evan Raskob (collaborator) sketch of networked interactions .............................235 Figure 11 C. Baker chart of the activities, content and connection, database and sensors .........237 Image 74 2010 C. Baker spreadsheet of database of participant videoclips ...............................238

10

iv/ statement of the artist as researcher

I am an artist-performer/researcher/curator within various art forms: interactive and performance installation, music composition and performance, video art, web animation, and experience design. I have been immersed in many forms of media creation and performance since 1994, including roles as: new media installation and performance curator; videographer, animator and documentary producer; video artist; photographer; media art instructor; web designer/developer, CD ROM and multimedia designer; visual arts curator; sculptor and modern dancer/performer. I was in Masters in Applied Science in Interactive Arts completed in April 2004, within the School of Interactive Arts and Technology at Simon Fraser University, Vancouver, Canada. This research and thesis project was an immersive media installation that explored the concept of initiating a mind/body to computer communication aided by biosensors to trigger media to calm the mind to enable telepathy in participants. Prior to moving to the UK between 2004 and 2007, I was involved in: starting and developing a performance media society, the Escape Artists Society (www.escapeartists.ca), which put on several performance events in 2005/2006. From 20022005, I was researcher for the wearable art installation and performance project whispers/between bodies project (http://whisper.iat.sfu.ca) with Dr Thecla Schiphorst and Dr Susan Kozel. This involved research in performance and wearable design, biosensor testing and development, prototype garment construction, movement development with wearable computing, as well as website development and video editing. I was also the primary Event Producer/ Interactive and Performance Curator (2004), the Lead Curator/Conference Director (2003), and the Visuals Coordinator (2002) for the New Forms Festival (www.newformsfestival.com), which involved extensive research in media art and performance, practices and history; exhibition, event production and staging coordination; arts funding administration; and curation. I was editor-in-chief of an online pop-culture/relationship magazine (Tales of Slacker Bonding) in 20002003; I was a singer/songwriter, performer, musician and producer with three primary musical projects from 1992 to 2007: Spiritual Heroine, EPX and ultrapuss, as well as singing for many side projects, and student dance and film projects. After my undergraduate degree in Sociology/Anthropology I began as an Employment and Life Skills instructor for at-risk youths and adults, from 19922000. In high school and into university I studied contemporary dance, and from 2001 to the present I have been an active yoga practitioner. Both these and other body practices have factored heavily in and influenced my art practice. I decided to undertake both the PhD research and MINDtouch practice as a set of related but separate work projects, in order to combine my particular background, artistic history and interests into a unique practice. I was led to this work, and this combined focus, by years of work experience as a performer, musician, media artist/designer, curator and employment facilitator. I have been enthused and motivated to work with mobile phones and video in a networked performance context since the idea occurred and began to develop in 2000. This PhD research and the related BBC project thus

11

provided a valuable opportunity to further explore and develop my own voice within the context of performance, entwining video art and facilitation skills, and in the process creating a new form for others to share and experience.

v/ style notes
The style of the thesis complies with the universitys regulations regarding house style in all regards. A number of specific stylistic devices have been used to increase the level of aesthetic expression in line with the content and domain media of the thesis.

v.i/ fonts
The thesis uses new iterations of both the Arial and Helvetica fonts. The fonts employed are, specifically, Arial Rounded MT Bold for headings in various point sizes to reflect the hierarchy of content categories throughout the text. Helvetica Neue Light is also used at 10.5pt, with a 1.25 paragraph spacing, for the body of the text, in lowercase for the Table of Contents and lowercase italic for the Table of Figures.

v.ii/ tense/first and third person voices

While practice-based research in the Arts and Humanities has, in recent years, made it more acceptable to use the first person voice for parts of any academic project that require an explicit inclusion of the personal experience of the author and her/his practice, still it remains the case that the extent of use of the first person voice sits rather uneasily within academic discourse. This thesis attempts to strike a balance of voices. It uses the first person where appropriate to describe the aspects of the practice that draw on the phenomenological and personal. This usage is, however, limited to those aspects of the writing where the personal is of political, practical or philosophical relevance. A more detached third person voice is otherwise employed.

v.iii/ italics
Text appears in italics in sections where a more personal voice is required: this technique is intended to help the reader to move seamlessly and without confusion between parts of the academic argument that stand on their own as objective argument, in standard font, and those sections where a phenomenological reading imbued with a personal voice adds another layer to the argument.

12

1/ introduction: ephemeral transference backgrounds and contexts

If you could exchange your sleeping dream imagery, feelings and sensations, with your friends and loved ones, what would it be like? If you could not only share and exchange, but remix and collage them, what would it look like? The mobile media performance project MINDtouch is based on a vision of the exchange of dream imagery and embodied sensations through mobile media, wearable biofeedback sensors and participatory, collaborative, networked activities as live media performance events. This telematic, physicalvirtual event-system is intended as a simulation of synaesthetic and embodied dream experiences or to mimic a telepathic affective exchange, creating a paradox in the notions of liveness/presence and of feeling/being-there. The use of live body data as a medium to mix concurrent streams of mobile video in a multi-thread, non-linguistic conversation that creates a collaborative visual dialogue, could enable a more embodied, meaningful and personalised exchange between remote groups of people.

1.1/ setting the stage (and a note on placement of the distributed literature survey)

MINDtouch explores embodied, non-linguistic communication and interaction, using wearable biosensing devices and mobile phones as the interfaces to remotely connected media experiences, within social and performance events. In addition, the MINDtouch research has attempted selfreflexivity by being cognisant of the effectiveness of the facilitation of enabling participants to experience (or not) liveness and presence within these new mobile media social contexts. MINDtouch is a performative research project that explores the experience of liveness and presence within a series of live, iterative, in-person, staged and virtual scratch participatory mobile media social events. These scratch or performance experiments involved improvisation and experimentation in generating live, collaborative visualisations using mobile mixed video triggered by biosensor data from participants bodies. The aim was to investigate how people connect to each other with technology, in alternate, embodied ways, to create a simulation of telepathic [1] dream exchange. Within the project, I sought to understand how bodily sensations, perceptions, interactions and responses might be meaningfully utilised to find unique ways to visualise the body/mind [2] activity, and to experience that activity in a collaborative performance environment.

1 Here I use the term telepathic to mean thought or experience transfer through the mind. It will be defined further on in this thesis in Part II in the exploration of Presence. I want to be clear at the onset that I am not trying to prove telepathy in this research, but rather I am trying to emulate it, and that requires a suspension of disbelief on the concept of telepathy for the reader. 2 The term body/mind is chosen to acknowledge Massumis studies and theory, neurobiologist Antonio Damasio, Benjamin Libet and other philosophers, showing that the mind not only exists in the brain but also in the parts of the body, especially the senses, and is discussed further in Chapter 7.

13

To identify how or whether liveness and presence can be sensed during mobile performance activities, MINDtouch used video as the mode of expression to explore levels of engagement, embodiment, affect and sensation within live social and mediated environments using mobile and wearable devices. I have attempted, through iterative or experimental performances [3], to examine the sensing of liveness and presence engagement, using biofeedback and mobile devices, during a few social events. The aim of this study has been to understand what the affect, reception, and experience(s) of liveness, presence of the performer and the audience might be, at the same time revealing their intrinsic embodiment within the network, during the structured events and activities devised, accessible only phenomenologically. As it has turned out, key to getting to this part of the research has been tackling the technological challenges to enable this mobile participatory interaction to happen smoothly for participants. Critical to the investigation have been the active attempts to enable individual and collective perceptions and embodied sensations of liveness and presence, within the context of the virtual, invisible-space of mobile networks. Thus, the project has explored new ways to simulate, emulate, or even facilitate telepathic connections or at least the sensing of feltness, presence and/or liveness, copresence and collaboration within a mobile performance. I have created events to enable presence and liveness using physiological sensors to intensify the embodied interaction and engagement. During the performance events the physical presence of participants was transformed, transduced or translated into a digital video form, to allow them to touch and play with others, remotely and non-locally [4]. This was visually expressed and represented through their videophone expressions of their presence. The various project threads were unified through the coding to enable the phones and sensors to communicate with one another in as close to real-time as was possible. Mediating between disciplines of performance and mobile video creation has been important in this work, because the project has primarily been about devising and implementing methods to embody the technological and communication networks and in a feedback loop. The first is from the body directly with the sensors, and the second is through the interpretations of the body in the videos by participants with the mobile phones. These impressions were then sent out to others to intermingle the expression of physicality across distance. In this way, the two modalities of sending expressions of physicality and embodiment through the network, the resulting visual mixes, as well as the audience experience of the outcome, each required a specialised treatment and method of facilitation, not to mention discipline for manifesting each, as will explained much more herein. Spanning these disciplines has been also important for me personally and creatively, as I have been trying to find and implement ways to address and integrate my many passions and artistic endeavours.

3 As will be explained in this document. I will show how I organised these events to examine these qualities in practice and within social contexts. 4 Radin and other Quantum physicists define non-local as physical objects (or people) that may seem unrelated or separate, but are connected, transcending the limitation of space and time; or events that take place simultaneously either through space and time, in space or in time (Radin,1999: 277). This will also be discussed briefly in Part II.

14

Telepathy, for purposes of this project, is considered and further argued as the desirable and longsought-after ability to develop, as Jeffery Sconce discusses in his book Haunted Media: Electronic Presence from Telepathy to Television (2000), in order to communicate directly to others without words over distance. For this work, telepathy is considered desirable as a way to avoid the miscommunication that can often occur through language. I assert herein that, if we could express our emotions, thoughts, desires and sensations more directly to others by essentially duplicating or sending our embodied experience to others minds and bodies, this might facilitate more empathy and understanding between people. For this work however, I am using telepathy primarily as a metaphor, and am using the media art project to simulate telepathy through a weave of technologies and activities. I also suggest that this simulation of telepathic dream experience necessitates the understanding and facilitation of liveness and presence of others to occur before the exchange is possible. During this research process, a phenomenological perspective was taken to engage with the key concepts of presence, liveness, embodiment, and telepathy in the practical context of performance, as a means to assess my own assumptions about these more abstract notions. I also incorporated discoveries made along the way. My underlying questions were:
1. Can a mobile, social interface simulate a telepathic exchange or a sense of presence at a distance? 2. How can artists understand previous notions of performance in relation to ever-new technologies and mediated activities, using both existing and developing approaches? 3. How is performance changing in response to networked technologies and wearable devices (mobile, biofeedback and other sensors, satellite/GPS, Internet)? 4. What is the relationship of 'real-time' computing or streaming to liveness in mobile performance and games? 5. Is performer another term for audience/viewer/visitor/participant of interactive mobile media performance work?

Discussed within this exegesis is the journey of developing this expressive, collaborative, participatory telematic event series as a means of enacting mediated performance experiences over a mobile network. It also explores the constructed virtual space of a mobile, networked interaction and how, or if, mobile participatory performance might be differently perceived from other forms of live performance. I have also interpreted participants feedback on experiences during the mediated participatory events. Another approach to participatory practice has been explored that facilitates embodied experience of liveness during a mobile, networked event. This research also hopes to contribute an understanding of remote or distance perception and interaction, using mobile creativity by enabling collaboration over the network. The theoretical research examines whether liveness can be sensed within these mixed modalities of streamed, real-time, virtual, within in-person, live activities, using mobile and wearable devices to interface the interaction between modalities.

15

I posit that mobile media phones act [5]: 1) as a conduit for non-literal or abstracted non-verbal expression of experience and as an extension of the body/mind; and 2) as a vehicle to express inner sensations between participants or with oneself. The unique contribution also includes my own phenomenological perspective on the process of creating this ambitious project and the discovery of the complexities involved with mobile networks. The goal has been to combine technologies not otherwise used together to connect people remotely, allowing them to re-engage with each other affectively and expressively in new non-verbal/textual ways, unlike current practice. Thus, the research brings together divergent areas of new media research and digital performance practices, incorporating physiological sensors that interface with mobile phone technologies, in unique ways. For the BBC-funded dimension of the thesis project, the result is a greater, well documented understanding, achieved through practice, of the possibilities and limits of technology as it can be employed today to support a social networking application which enables participant interaction, creativity and collaboration, in a more playful, visual, personal mode of non-linguistic communication. The technical aspects of this project act as a prototype for such a tool, which would facilitate alternative, direct, social and intimate communication and modes of expression between the body and the digital device, with the user as the technology conduit in a mediated yet embodied dialogue. This research aims to make a real-world impact as well as a theoretical contribution to the field of new media with emerging mobile art, film, games design, locative media artists and designers, as well as to the performing arts. It creates new modes and tools for performance expression and exploration. This work may provide new parameters or a framework analysing this offshoot of the performance milieu and media currently called pervasive media or mixed reality [6]. It could impact the Human Computer Interaction (HCI) community, in terms of body-machine interfaces and modes of interaction, contributing to new understandings of interface and platforms for intimate communication [7].

a note on the distributed literature survey While it would be traditional to include a section containing analysis of all the key texts and authors influencing this thesis in one section titled The Literature Survey, it is more appropriate to the structure and subject matter of this thesis that such materials be distributed throughout the thesis, where they can best demonstrate their own importance to the overall argument in relation to the diverse specific theories and practices with which this thesis engages. For this reason, there is a set of analytical summaries of key literature in relevant chapter sections in both parts of the thesis.
5 In conjunction with the appropriate software. 6 Defined and discussed further in Chapters 3 and 5. 7 Currently, many of the mixed reality and pervasive media projects come from the video-game and HCI communities, and very few from the performing arts. However, in 2010 both Blast Theory and a group called Theatre Sandbox are funding residencies and projects by theatre and performance professionals, in an effort to educate and encourage them to work more with technology in their performance practices.

16

1.2/ MINDtouch summary of the major case study and its aims/outcomes
This thesis engages with and analyses data gleaned from a larger R&D project (conducted by the researcher in tandem with this thesis project, sponsored by the BBC) as its major case study. The project, called MINDtouch, created a series of unique practice-based new media performance events played out in real-time networked contexts. The MINDtouch events were framed as a means for participants to simulate dream exchange or telepathic thought transfer using mobile phones and biofeedback devices, linked to a bespoke video file protocol for archiving and sharing visual results. While the issues (theoretical and technical) arising from MINDtouch as a project extend far beyond the scope of this thesis, the first results as analysed here suggest that the mobile videophone could in future become a means to make greater use of sensual semiosis and non-linguistic forms when communicating from different physical locations. The corporeal, non-verbal forms of communication and visual interaction observed when participants use such devices within performance events are examined in this thesis. The thesis benefits from access to the larger MINDtouch project and its original data, providing this research with a set of process-based evidence files in both video and transcript form (contained in the appendices, along with a full account of the project, performance events, user experience reports, and biofeedback sensor design briefs). MINDtouch designed, developed and tested a new set of biofeedback sensors and mobile phones, used together with a mobile network for video file sharing. This set of tools was envisioned in the MINDtouch system as a possible emerging technology toolkit to provide an alternative bodycomputer interface to trigger actions on mobile phones, in order to discover what might be meaningful about technology-supported human interactions using ubiquitous mobile devices. By analysing this unique data set and applying the theoretical contexts of kinaesthetic philosophies where appropriate, the thesis demonstrates both the practical and the critical/contextual effectiveness of the media facilitation process for the participants, and shares their senses of liveness and presence (of themselves and of others). The thesis engages with the major MINDtouch case study and applies theories from fields of kinaesthetics and phenomenology, including current theories of embodiment, in considering notions of presence, liveness, feltness and telepathic exchange (all terms defined at length in this chapter). The research brings together new thinking in the fields of media art theory, telecommunications, neuroscience, performance and new media practices. The practice-based approach to exploring theory has involved designing and applying bespoke biofeedback sensors to interface with mobile phones, in the context of performance workshops incorporating theatre games and other creative approaches to being in both live and remote spaces. The thesis has also combined methodologies, methods, practices and tools from Phenomenology, Ethnography and Practice-As-Research, and Experience Design, bringing together the relevant aspects of these diverging areas of new media research and media art/performance practices.

17

Thus, the MINDtouch project positions the physical body as the primary site of investigation from which to embody technology. As such, the phrase body as interface, or body interface, can be used in the context of this work. Body data and bodily processes, such as breath, blood pressure, heartbeat, muscle electricity and sweat conductivity (as an indicator of stress), resulting from impulses and responses, were used in tandem with custom mobile software to activate other technology in this research. This data becomes the conduit for creative play and communication. The wireless physiological sensors assist by connecting the embodied interface and translating the data through the mobile devices in a new form of non-linguistic expression. MINDtouch harnesses the primary exteroceptive and proprioceptive senses through the sensing devices, translating sensations and perceptions from participants bodies into a moving visual form, from the minds eye [8]. Instead of verbal or gestural language, the immediate surroundings of the body, the architecture and the landscape within the frame, are employed with the visual or cinematic language of the mobile video camera. Aided by technology, MINDtouch practically explores a form of interpretation or conscious and synaesthetic translation of ones internal world through the external environment. The project developed a deliberate, strategic methodology that served as the mediation between the various disciplines and practices involved and discussed here. This hybrid methodology and overall mode of working helped to achieve the goal of testing the qualities of liveness and presence in the context of a performative technology-led project. This in turn lead into the specifics of the three project phases below: Briefly, the phases of MINDtouch project development were (see Appendix I for details): 1) phase one involved conducting mobile video workshops to collect the media. Participants in the first phase video collection workshops were asked to explore and visually represent their physical sensations, perceptions, emotions, thoughts and impressions in a non-verbal, visual way, using the mobile video recording phones to later share them with others; 2) phase two involved the technical development of the interface software for receiving the physiological data on mobile phones, creating the video mixing software, as well as assembling custom electronics and wearable sensors systems, then embedding these electronics and sensors into suitable custom garments; 3) phase three was the staging of the live, participatory visualisation events, using the sensor-embedded garments on participants, to trigger both the workshop video clips and live streamed mobile video, from the event space. When these were mixed, a new video collage streamed back to local and remote participants phones.

8 This is further developed in Chapter 5 on mobile video aesthetics, and in Appendix I on the workshops.

18

Phase 1

Phase 2

Phase 3

Figure 1 2007 C. Baker - early technical diagram of technical performance development phases and interactions

For MINDtouch, events were organised for groups to engage in live improvisation and experimentation with mobile video, collaborative and generative VJing activities using the technology. Within these events my goal was to see which activities and techniques worked with participants and which did not, in terms of sensing others in the network. The data of participants experiences in this primary series of mobile media performance events is collected and analysed in Appendix I. My approach was to interview participants in order to gather their first-person impressions of their performative experiences, in their own words. This data was then analysed to determine whether they felt or experienced liveness and presence differently from during traditional performances. This analysis would then inform the direction my research would take. The primary media performance series for MINDtouch involved developing a database of archived video clips from the phase 1 workshops alongside live-streamed video from participants mobile phones during the events. These videos had visual effects added based on the body data of other participants. A live, collaborative, visual performance montage of the effected participant videos was simultaneously created. The remixed live cinema montage or installation was then restreamed back to the remote and live participants phones and over the Internet.

19

Participants wearing the physiological sensors would be performing the visuals through their body data, creating the unique mixes, engaging in a conversation with the audience. The body data thus controlled the video effects application and mixing, which were displayed in the live setting, while accessed and contributed to both by live and remote participants. This video collage thus became a collaborative, narrative and global mobile-cast, converging previously distinct technologies and practices, bringing all the different virtual presences together in the visual mix, simulating collective consciousness. During workshops and live events, participants become collaborators in the interactive, generative video performances, streaming directly from their phones, using qik (proprietary third-party software), intended to be in real-time [9] and from the media archive. This facilitated a type of collective consciousness [10]. Each mobile media social event started with a short guided-improvisation activity, to inspire participants to explore creatively. Some participants also used their bodies in a range of expressive, creative, nonverbal/non-textual ways, which were then translated into the video mix.

1.3/ motivations, roles and voice

1.3.1/ motivations I began creative practice with a relationship to words (as a poet/singer/songwriter/musician), then to video and dance, moving to combinatorial practices of new media, performance video and mobile media. Now, using all of these, my interest has moved to making the digital flesh, the electronic visceral and the static alive through performance and social interaction. Throughout my creative and academic life, I have been fascinated with the concept and experience of telepathy. Definitions of telepathy often describe it as presence at a distance; others call it thought communication (Radin, 1999, and Sheldrake, 2003). My definition is that it is the experience of sending or receiving a feeling, sensation, experience, thought or idea from someone else (generally someone close to you), non-verbally or through ones mind, across distance, and most often during dreams (Ullman, Krippner and Vaughan, 2002; Baker, 2004) [11].

9 Issues regarding and defining real-time will be elaborated on at the end of Chapter 8. 10 This is referring to the well-known Jungian sense of the term used in popular culture, defined here to mean minds coming together to a form of unity over distance. This will be discussed further in Part II theory and praxis, in Chapter 8. 11 In the past, I have reviewed many prominent scientists (biologists, physicists and neuroscientists, such as Fred Alan Wolf, Arnold Mindell, Dean Radin, Robert Jahn and Rupert Sheldrake) and their research on the existence and experience of telepathy, and have had the experience myself.

20

This background is relevant because MINDtouch focuses on simulating or emulating a telepathic connection and a co-located sense of feltness, presence, liveness, co-presence and collaboration within a mobile biosensing performance project. I have explored these experiences more deeply and conceptually through the meta-theory level of the research. The premise has been that: in order to discover whether liveness and presence is perceived by live and remote mobile participants during a
networked performance event, a visceral, sensory, emotional and philosophical grounding is key. This

project initially set out to explore the body as interface, but extended to create a visual, non-verbal communication mode, possibly relevant to those who cannot, or who choose not to, speak. In this and previous media research projects, I have worked with biometric medical devices as a means to explore the use of the body as a conduit for media interaction, enabling users to explore and alter their consciousness and experience a form of telepathy within an enclosed, mediated space. Using the body essentially as a mouse, users engaged in a visceral way which, as intended, affected them very deeply and personally. Finding ways to create in the body/mind of the participant directly from an internal narrative, this past workpaired media supplied fragments intended to trigger mental imagery, body experiences and sensations, memories, associations, fantasies and dreamscapes from the users past or current reality. This in turn triggered the media in an endless feedback loop meant to instil or induce a meditative or pre-dream-like state. For this previous work, the participatory media installation was an immersive, experiential media space, incorporating biofeedback sensors, a variety of media, yoga and meditation to facilitate telepathic exchanges. Ideally, this exploration was to lay the groundwork for more sophisticated telepathic portals or telephone directory, utilising mediated environments and systems. The impetus then was also inspirational dream-like imagery, meant to induce a receptive state of creative consciousness in participants, helping them to connect with others non-locally. My premise was that the body is the integral site of the mind, so the experience had to take place within the body. I saw this as a return to the body for interpersonal exchange, wisdom, transcendence, and exploration through body practices and technology. This embodied, immersive experience was intended to attune people to their own bodily sensations; a means to harness the virtuality of sensation, imagination and a dreamlike imaginative state. This work indirectly led me to the current project.

21

1.3.2/ key terms defined: 'liveness', 'feltness', 'embodiment', 'presence', 'co-presence', 'ephemerality', 'network', 'transference'
liveness

Liveness is the sense or experience of an event or person in-person, physically present, in the hereand-now, being or taking place at this moment, in front of an audience in a traditional performance context, or remotely (as with a live TV broadcast or online event). Liveness is intertwined with time and/or space and physicality simultaneously, and entails an event or experience that is happening concurrently with someone witnessing it. Yet it is not a priority that an audience is experiencing it inperson or even in the same physical location as the event itself; or in the flesh at the event location, but the event must be experienced as taking place now and must be witnessed now, in real-time or within seconds of the event. In a digital context, now-ness is critical to live events, but physical presence is not: a computer avatar can stand in to substitute for actual bodily presence, so representing a physical player is acceptable, and using voice-over-IP (Skype, etc) or video conferencing to replace the corporeal presence can also be considered live.
feltness

Feltness is the sensation or perception of touch or the experience of being touched, either physically or emotionally. It is a synonym for presence, yet has a more tactile quality and is concerned with the feeling of others, viscerally, physically, or on an intuitive or emotional level. It is akin to how one feels when speaking to a friend or loved one who is not actually there with one in person, for example on the telephone, or similar to the feeling of someone across the table from one, registered through the qualities of the other persons voice or their gestures. It is a physical, lived sensibility, like a vibration or physical agitation. It is an exchange that is not one-sided, but a heightened perceptual sensation of otherness and reciprocity, expressed between people or entities engaged in an embodied connection (Massumi, 2002: 25) [12]. It is the feeling of others in localised real space or distant virtual space.
embodiment

Embodiment is the entanglement of mind and body as one, encased within the skin, enveloping the bones and organs, blood, the brain and other organs, fluids and tissue. Yet it incorporates a presence of a being, a spirit, of consciousness, or a soul that seems to be separate from the corporeal or fleshy encasement, but living within it (Varela et al., 1991: 27) [13]. Embodiment includes the elastic ability of the mind to move beyond the boundaries of the skin and for one to send ones presence across distance (Idhe, 2002: 6). It is both inside and out, crossing boundaries of the senses, so that they function separately, entirely outside of the mind (and vice versa), in an elastic sense, but not detached entirely from the body:

12 This is argued and defended in Chapter 7 on presence. 13 This is discussed at length in Chapter 8.

22

Brain and skin form a resonating vessel. Stimulation turns inward, is folded into the body, except that there is no inside to be in, because the body is radically open, absorbing impulses quicker than they can be perceived, and because the entire vibratory event is unconscious, out of mind. (Massumi, 2002: 29)

Technological embodiment is a form of non-corporeality within which virtual connection can be expressed through invisible exchanges of sentiment (Richardson, 2005: 6) or through a digital mode of extended mind (Beauregard and OLeary, 2007: 157), through projection or by sending the consciousness through technology as a conduit. So embodiment can be 1) being in the body, and the mindbody integration; and 2) being in the body, but also sensing and extending the mind beyond the skin during dreams and in altered states, virtual technological spaces, and in other modes of consciousness [14].
presence

A sense of presence is a physical, lived experience akin to a vibration or agitation, felt both viscerally and intuitively. Presence as defined here is a sense of otherness, a feltness that can be sensed in a room, just as one feels a friend or loved one who is not actually there with one in person. It is the sense of feltness of the other. Presence can be experienced as meeting or sending ones extended body/consciousness or awareness of self or other [15] via invisible, non-corporeal embodiment. While the flesh is not directly stretched over the distance, the mind and body are engaged by both parties across distance, through extended embodiment. Presence is a togetherness without necessarily being in the same physical space or place, but with an awareness of another.
co-presence

See presence above; co-presence is when one is aware that there are others feeling ones own presence. It is the simultaneous intuition of another person and their involvement in some mutual activity or space (virtual or in person). It is felt when one is engaged emotionally, intellectually, playfully, or otherwise taking part in an experience with others, and having the feeling of them also engaged in it. It is the visceral knowledge that there is an other out there somewhere, one who is alive and aware of you, who is thinking of you, and you of them, either consciously or unconsciously; who senses you and is reaching out to you consciously or unconsciously, emotionally or cognitively, physically or virtually, who is concerned for you and who you are concerned for [16].

14 Here I am referring to virtual in the digital and online context. 15 See Rupert Sheldrakes theory of the Extended Mind in A Sense of Being Stared At (2003). 16 This knowledge is based on prior relationship with that person as discussed in Chapter 7.

23

ephemerality

Ephemerality is the quality in something of being temporary or transient and intangible, which goes beyond language, leaving no visible trace (Phalen, 1993: 148, when describing Performance; McKean, 20052010: online)
network

A network is a form like a spiderweb or rhizomatic structure, like a root system. Mitchell describes it this way:
[an] accumulation and habitation [of] sites, links, dynamic flow patterns, interdependencies, and control points (2003: 9).

Or Massumi says,
The network distributes. Interlinks. Relates. The network is the relationality [...] (2002: 86).

The New American Oxford Dictionary 2nd ed. (2005) defines it this way:
network |net-wrk| noun 2 a group or system of interconnected people or things [] a number of interconnected computers, machines, or operations. (McKean, online)

These are only two definitions, but the key term is interconnected. My work is about interconnection between people, and between people and things (i.e. mobile and computer networks, systems and operations). My own conception of network connections is that they resemble: threads, sewing, netting, weaving, knitting, fabric; enmeshed, entangled or tethered forms; knots or rigging; branches and roots; cellular forms, veins; and tunnels or path systems. MINDtouch involves a network of bodies, connected to sensors, connected to phones, connected to a database of video clips, manipulated by the body data, connected to the world of networked possible interactors/participant performers online globally.

transference

For the purposes of this thesis, transference is the act of moving ideas and/or sensations from one location to another: a thought, feeling or sensation moves from one person or body to another, over air and distance. Sometimes transference is the act of extending embodiment through technology (see above).

24

1.3.3/ voice Throughout this document, and particularly in Part II (which explains the relationship between theory and practice), a first-person voice periodically enters the text. At times, the voice takes on a deliberately abstract or poetic register (primarily in the section on theory, Chapters 7 and 8). The use of italics is intended to help the reader see the shift in register from the practical and academic viewpoint to the personal and phenomenological. This is a stylistic and personal choice, evident when discussing my conceptual and phenomenological experience framing my position on theories of presence and embodiment. As mentioned above, I have different roles in this research, as performer/artist, as well as the director planning it, and the curator/researcher discussing the works of others. As such, in this exploration I have included some reflections and epiphanies (I often call them musings) or personal insights on concepts discovered through the work or inspired by others. These reflections are contextualised through others experiences, which aided me in finding other approaches and like minds to demonstrate shared thinking. Anita Ponton says in her thesis Vertigo: The Technophenomenological Body in Performance (2005):
While I do not address voice as a specific issue in my thesis, throughout the text there are different registers of voice. Given reign to speak, they are symptomatic of my desire to write through the practice, to follow the (il)logic of my practice, in order to take the text into the same zone (at the very least) as my practice. (Ponton, 2005: 23, my emphasis)

In a similar way, there are sections here that have a more first-person, poetic and introspective position than the more theoretical sections. In these sections I have sometimes mused upon conceptual explorations from my own experience and practice of presence in mobile and digital interactions. Other sections are written in a more conventional academic, third-person voice. This is maintained for academic rigour when discussing the ideas, theories, practice and works of others. As this is the work of an artist, singer/songwriter, performer, poet, videographer and former dancer, it is important that the theories and philosophy be read through the practice and in the language, metaphors and musings from my journey and the experience of it, explored and uncovered during the research and framed at times personally. Taking a phenomenological approach has meant that in practice and in writing, the work and process is described from a first-person, lived perspective, to allow the reader to follow my journey within and during the project. I discuss this choice more deeply in the methodology section in Chapter 2, and I defend the Art-as-Research and phenomenological approaches as a critical means to explore and analyse the experiences of others more empathetically, since they demand a sensitive understanding. As an artist, I can only read through my own experience of the practice and its staging, rather than pretending that any research is truly objective, especially as a personal art project.

25

1.3.4/ roles As the author of this thesis and also the creative director of the larger MINDtouch research project on which the findings are based, I have played several roles in the research process, including: 1) media artist and performer; 2) action researcher using the phenomenology of experience and experimental mobile performance in order to explore abstract conceptual notions and qualities of experience; and 3) performance director/facilitator of the case-study events. This last role is critical to my personal aim (and my reason for choosing to undertake the PhD with the socially committed team at SMARTlab) of practising my theory and applying all knowledge gained to the act of facilitating social experience, both socially and artistically. My background leading to this point is critical, as it explains how I arrived at this project and not any other. It demonstrates that my unique embedded, embodied and tacit knowledge as a curator, performer, media arts researcher, and even employment counsellor/facilitator, from my past lives, makes me uniquely qualified to carry out this research, in this way, as discussed further in the methodology section in Chapter 2. Below is a chart that expresses the various roles more clearly: Researcher/Curator role
Exploring the concepts (liveness, presence, embodiment, telepathy, etc) Finding recognised theory to ground the concepts Finding relevant artists to ground the practice Surveys and feedback analysis of

Artist/Director role
Devising performance activities Developing database categories from video content Working with technical collaborators on video effects, sensors/electronics and mobile software

Facilitator role
Facilitating workshop activities Facilitating performance activities Ensuring the technical collaborators were engaged and feeling valued for their work Guiding the guides for the performance events Setting up a virtual space for online participants

Devising the interfacing design of the technology Directing the staging of the events

Following-up with participants after workshops and events.

Figure 2 2007 C. Baker chart of project roles

As a curator [17], I have been concerned with new media performance and all its permutations (live, online, etc), as well as net art, interactive installation and technology-based performance. What excites me is how current developments in media-based and new media performance practices are addressing or investigating the presence or aura of the performer, over time and through or across space, and how this in turn affects audience members and their reception of the performance. I am also concerned with the performance experience for the media artist and her/his audience, as well as how the content of the work affects, or is experienced by, the audience. The performance staging and
17 I was Co-Executive of The Escape Artists Society, based in Vancouver, BC, Canada, from 2005 to 2007, which was an electronic and new media performance event organisation focusing on presenting site-specific works from interventionist artists.

26

nature of the actual experience for the performer are also worthy of investigation. Curatorial issues and practices have been challenged by new modes of presentation and performance in the last decade, especially with technology-based and participatory works, so this area warrants review as well [18]. In my practice as a video and digital artist, performer and curator, lecturer in film and video history, media aesthetics and practices, various curiosities have surfaced, such as VJing and Live Cinema. It has also long been a fascination of mine to use mobile devices to marry performance video mixing with telematics (also known as networked performance, discussed in Chapter 3). Upon starting this research, I had a vision of the work I wanted to create, based on my knowledge of the Internet, of other digital art projects that used telematics, mobile and wearable devices, and on past art research, such as with whisper[s] (Schiphorst and Kozel, 2003-2005). Experience as curator with the Escape Artists Society (2005-2007) and the New Forms Festival (2002-2004) in Vancouver also aided. However, I quickly became aware of the ever-changing landscape of mobile technology, as well as the latest in medical, as well as DIY, physiological sensing technologies on the market. Here the curator is transformed into more of a theatre director role, devising and rehearsing the performance elements to get the best possible performance and experience from and for the participants, or actors. I bring embedded experience as a life skills and employment counsellor from another part of my life into my practice as a facilitator. When I returned to post-graduate studies in media art I left six years of working in social services, aiding and facilitating people to find work or new career paths, better living conditions or to at least get off the street. I have tried to carry the learning I gained from this former career into my media art and performance practice, to find ways to continue to enrich peoples lives through enabling them to have deeply felt experiences. This background makes my approach to practice somewhat different from others. Facilitation has a whole rich history and philosophy not covered in this thesis. However, I carry my own interpretation of it as the enabling and shepherding of others through a process or through learning. It necessitates checking regularly with participants that they are safe and comfortable in the experience and that their needs are being met as well as possible, to reach their own goals or expectations in the process, whatever process that might be.

1.4/ methodologies
The MINDtouch research project has combined methodologies, methods, practices and tools from Phenomenology, Ethnography, Performance, Art-as-Research, Experience Design, bringing together diverse areas of new media research and media art/performance practices. The project involved the use of biofeedback sensors interfacing with mobile phone technologies in a unique way.

18 Yet sadly these are beyond the scope of this research.

27

The MINDtouch research project has been a means to innovate performatively and technologically, within a relatively new performance and technology domain. However, this research is developed as a media artwork within the context of an academic environment, and realised through a mode of practice not possible through written scholarship alone. Art-as-Research methods, more commonly used in recent art and performance research, are primarily employed herein. Other methodologies and their methods, practices and tools used in the research are complementary and layered together as part of this Art-as-Research (AAR) methodology. In this research project, the mobile media performance series became a case study, from which data on the experiences of the participants have been collected. The hybrid practice-based art research methodology devised and composed for the project includes: 1) performance-based structured improvisation techniques from theatre and dance used in the workshop devising activities, and as a means of helping to engage participants in the activities during the events; 2) ethnography for collecting and analysing the interview data; 3) phenomenology for the overall creative design approach and process analysis; 4) experience design for devising and implementing the experience both of the activities and of the digital interfaces; 5) visual methods for categorising and analysing the video imagery; and 6) technology design (empirical) methods used in the technology development. These methodologies and their methods, tools and practices were implemented to create this hybrid methodology, as a way to thoroughly address the work. I have also borrowed an approach called scratch performance, used in London theatre and live performance, which means work-in-progress or iterative theatre, performance that asks for feedback from the audience to develop a piece. This is similar to iterative design and other creative approaches that use repetition, discussed more in Chapter 2 on methodologies. Important to note here is that the work would not be realised by one of these methodologies alone, as each addresses different aspects of project development for performance staging and technological development:
Phenomenology for observation of self as artist/director/researcher in planning, performance staging and experience analysis; to work from a first-person, lived-experience perspective in the various roles I play in the research, and as a way to be a participant in the research as well as a researcher, artist/director and facilitator; and as a way to understand my own inspiration and choices as they surface in order to (in a way almost akin to Action Research) devise the research and be inside the research to better do so;

Structured Improvisation for devising and actively engaging participants in the performance activities; to explore ways for the participants to interact with the technology and each other and to discover how best to devise the activities;
Ethnography for observation of participants, note-taking and analysis of the participant experience within the activities; to understand the experience from the perspective of the participant but also to observe their behaviour from the outside, in order to analyse performances and feedback from them;

28

Experience Design interwoven with and interdependent on the previous two; focused on creating engaging activities with the technological devices, garments and software, as well as the collaborative video collage they create together; taken from the graphics and web design field as a means to craft the interaction design of the experience with the technology effectively for participants; Visual Methods for analysing the content of the visual imagery in order to devise appropriate database categories for the software to trigger the images from body data received by the sensors; as a way to understand the intended meaning and content of the visual material created by participants, from a visual aesthetics and symbolic position; Technological Design M ethods (empirical) employed mainly by and with the technical collaborators and software developers, in order to create, assemble, code, test, debug, iterate and finally work with the technologies used; used to iterate and perfect the technology for use in performances.

These are all discussed in much more detail in terms of why they were chosen and used in Part I, Chapter 2 and their use in practice and the outcomes of their use in Appendix I.

1.5/ performance context (Literature Review Part 1)

I am not a scholar of the recent currents in the debates on Art or Practice-as-Research, especially as it applies to artist-researchers doing PhDs in the UK versus Europe, North America or elsewhere. The discussion here is based on research on artists experience and the use of their practice to create knowledge. I am interested in what they have to say regarding their processes and methods and how they have gleaned knowledge from these to contribute to the art world. The authors below were selected in terms of how their ideas resonated with my own experience. I engage well with the artists and voices presented here because they contextualised my use of practice in research. Therefore, nuanced debates around the differences between Practice-as-Research, Art-as-Research or Practicebased Research are not addressed within this discussion. There were no differences found during searches. I am aware that Practice-led Research research based on the work of others or ones previous practice, rather than a current project is different from the above categories within academia. I also know many practicing artists personally who were not aware of these distinctions until entering academia, so they may be distinctions primarily imposed by the academy and not by practicing artists. Further discussion on these debates is not within the scope of the thesis. Key thinkers in performance theory have contextualised old and new directions in performance practices, and were essential in identifying where new insights might be found. Sociologist Erving Goffman first brought common behaviour of individuals to the fore (1959). He focused on the

29

performativity of everyday life how we all play a role in each social or personal interaction and defined ways in which we do so. Richard Schechner (1988 [2003]) has become a strong voice in Performance Studies in the last 30 years. He broadened thinking and previously held views on performance definitions within Theatre and Performance Studies. He did so by demonstrating that the performativity in everyday life and practices, such as in play, sports, legal proceedings, rituals and pop culture, was valid beyond the arts realm of theatre, dance and performance art. On the other hand, Philip Auslander (1999) focused on how liveness is in opposition to what he calls mediatisation in performance, claiming that liveness did not exist outside the context of the mediatisation of the 20th Century and that this aspect is key in understanding performance in the modern era. Meanwhile, pragmatist philosopher Richard Shusterman (2000) explored the aesthetics of the experience of bodies and performance for the audience, suggesting that conventional audiences and art viewers feel that the art establishment has lost interest in satisfying their needs for affective, emotional, transformational or aesthetic experiences [19]. He claims that people have gravitated to popular art forms and entertainment to meet these needs. Boals (1992) participatory theatre work was highly influential, challenging and working with actors and non-actors in order to engage (or reengage) them socially and politically with the world. As discussed in Chapter 5, performative video/live cinema/VJing is a form that uses my combined interests in video, music and performance and thus the practice becomes relevant and exciting to explore further in the project. Although there is still little written on it theoretically, this is changing. Jaeger (2005) attempts a critical discourse on VJ practice and culture through a history of it, describing what it is and is not artistically and practically. He then contextualises it in the larger art and cinematic context, placing VJing within live and improvised performance practices. He discusses the practice of jockeying and its behaviour and labour, as well as the role VJs play as artists, software designers and performers, closely tied to film and digital art. Youngbloods (1970) Expanded Cinema is a forerunner of wider contemporary performance video practice and is a surprisingly current and astute look at networked technologies of that day and the new role that cinema could play with them. Youngblood looks at video-based artworks that used the networked technology of the time (telephones, television, satellites, etc), comparing them to the human brain, suggesting that they simulated a collective consciousness. This utopian work envisions how cinematic and new technologies might extend human communication capabilities beyond imagination. It also explores the possibilities of computer-based and holographic cinema, with visionary hopes for cinematic development eventually leading to performative and live video creation. This work leads well into the project examined here and its efforts in mobile social VJing [20].

19 And, I would argue, some in the art establishment are too elitist for the populace. 20 This is what I was calling the project to draw participants to the events when advertising it as part of the social festivities of the parties it was a part of.

30

1.6/ mobiles, databases and network freedom

1.6.1/ mobile and database media The discoveries made during the MINDtouch research on mobile screen aesthetics (Chapter 5) have come primarily from observing mobile video capturing behaviour within the custom workshops and participatory performances. Some of these include: the ease of expression of video capture using the mobile; the intimacy [21] and affective [22] nature of mobile devices, the inherent gestural qualities implied by the size and portability of the phones, as well as the ubiquity and accessibility of them. The landscape I entered when I began the PhD study was varied and the research works in my areas of interest all seemed unrelated to each other. Similarly, when I initially reviewed telematic and other relevant art works using telepresent performance or mobile phones, databases and networks, they were all varied and differed greatly from each other and from my focus. None exactly addressed my interests. However, a few key voices stood out early on in the research, and they are described below: Glesner (2002) presents telematic and distributed performances as emerging performance forms, challenging the spatial relations between performer and viewer, by describing telepresence and liveness within Internet mediated environments as site-specific performances. This is shown in the light of these performances interactivity, the audiences sense of presence of performers, and the overall qualities of liveness. This is an intriguing concept if one thinks of the Internet as a place or site where performances can occur, which then assumes an embodiment of or within the network (discussed more in Part II). Dealing with presence slightly differently, Hauber et al (2005), come from a Social Psychology position, and tackle the quality of presence within the contexts of two-dimensional and three-dimensional interfaces for videoconferencing. They define social presence in terms of the sense of being together or being there with others, and base their claims on research of participants' reported experiences of interacting with each other using different conferencing interfaces. Again this assumes the embodiment of virtual space and of the network as a place or site. This in turn assumes an extension of non-corporeal consciousness, discussed further in the embodiment theory in Chapter 8 in Part II. Among the few intriguing voices on locative media is Gianni Corino in his thesis Spatial issues and performative media in digital mobility: a network perspective (2006). He focused on the social, urban and media aspects of performance in the use of mobile communication devices and how they enhance peoples sense of place and connection. He examined the history of digital mobility, the development of
21 In the context of this project and my work overall, the term intimacy is used in a generic, common, dictionary use of it, meaning: closeness, private, personal, familiar, in relationship to or on the body, or between two bodies/ people in relationship. 22 Affect and my use of it are discussed at length in the Literature Review sections, particularly looking at Massumi, MerleauPonty and Groszs interpretations of it. Briefly, I use Brian Massumis definition: Affect [...] as couched in its perceptions and cognitions [...] implying a participation of the senses in each other: the measure of a living thing's potential interactions is its ability to transform the effects of one sensory mode into those of another (tactility and vision being the most obvious but by no means the only examples; interoceptive senses, especially proprioception, are crucial) [...] Formed, qualified, situated perceptions and cognitions fulfilling functions of actual connection or blockage are the capture and closure of affect. Emotion is the most intense [...] expression of that capture [...] (Massumi, 2002: 35)

31

the cellular phone and social network theory. Corino also discussed recent activities in performative media and artworks to expose the directions that future mobile communications may take based on current explorations and applications. In Flash! Mobs in the Age of Mobile Connectivity (2005), Nicholson focused on how mobile phones have changed the social practices of public performance and activism. He looked at how the performative phenomena of Flash Mob events in 20032004 did this. He demonstrated how mobile phones have transformed the way in which political and social activist events are organised, then moved to how they paved the way for anti-political Flash Mob events to take off. Nicholson details the incremental changes and precedents set that enabled these activities to come about, and how they altered the way artists, and people in general, began to use mobile phones to instigate distributed social activity and performative events [23]. As a precursor to mobile phones and mobile art, there were also artists who explored communication and distance through other forms. Neumarks (2005) At A Distance: Precursors to Art and Activism on the Internet addresses art that uses distance as the main constraint, influenced by geography, technology, time, presence, space, connection between two points and the gaps in between. In particular the chapter called Introduction: Relays, Delays, and Distance Art/Activism focuses on text in postal art and the precedent it set for current Internet art, specifically the concepts of relays, delays and the anticipation of waiting. Here Neumark cites prominent theorists and their interpretations of time and space, and how distance artists addressed these theories, both before and after Internet art developed, exploring the past to understand current practices in bridging distance through art. With respect to networks and network art, the article Theses on Distributed Aesthetics (2005) by Munster and Lovink develops a theory reaching beyond the Internet and television to now include multiplayer games and mobile media. The authors theory of distributed aesthetics which a network as a theatre black box of possibility for various media, activities, practices, users, artists and experiences. They see this black box as an amorphous, unstructured and unpredictable space, like an emergent entity or complex ecology, not easily mapped or navigated. They also include a history of when computer network mapping started in the early 1990s, alongside changes in definitions of aesthetics. Several artists working with art and databases were featured in the book Making Art of Databases (Brouwer, 2003). The artists included: Sher Doruff, Raphael Lozano-Hemmer with Brian Massumi, Lev Manovich and Joel Ryan. In this series of essays based on four master classes, artists helped to develop various systems to work with archives and databases to make interactive, dynamic and responsive media works. Each artist wrote an essay on their own master class, as well as their practical interpretation of database in the context of their own work, and their conceptual view of database structures and the complexity arising from their experiences with them. Lev Manovichs work, as well as some others here, will be explored in more detail in terms of database and generative video aesthetics in Chapter 5. Artists that cover all the domains discussed above are delved into more deeply in Part I.
23 Later I look at Boals work and how he could be considered a forerunner in instigating this type of social or theatrical activism.

32

1.6.2/ dreams of networked freedom The MINDtouch work collates many different popular visions or daydreams and science fiction desires together, such as the sleeping dream recorder in Until the End of the World (1991) by filmmaker Wim Wenders. These visions are introduced here to sketch and introduce the territory covered within this exegesis. The work combines the vision of the video-capable mobile phone from so many sci-fi films, including real-time mobile video interaction (which may well be developed by the end of 2010). As life imitates art, art and science also imitate art. These films have not directly ignited my passion, but the vision to make performances involving videophones has been my desire for almost a decade. I have been waiting, and am still waiting, for the technology to catch up. Now I wish to find ways to push it closer. A seemingly simple goal of simulating embodied dream telepathy, researched and demonstrated as possible for some during sleep (Ullman and Krippner, 2002) [24], has become a more complicated and constrained activity in many ways. This is due to the complexity of current mobile networked systems, not to mention private or proprietary systems that involve boundaries, passwords, firewalls, etc. Oneto-one connections in these networked contexts are generally reliable, however one-to-many is still problematic [25]. So many-to-many, in collaborative, real-time video interaction is still complex, difficult and resource-intensive, as has been discovered during this research, especially as a lone researcher/artist with a limited budget. The history and expansion of mobile services has been more extensive and quicker in Europe (Greene and Haddon, 2009: 22), and mobile cinema, art and participatory projects have sprung up and expanded as a result. In addition, mobile phone use has been more ubiquitous and has grown quicker in Europe (compared to Canada, my country of origin). Other mobile advancements are also superior to what I was used to [26]. An exciting development occurred when Nokia and other mobile manufacturers rolled out their phones with video recording capabilities to the market in North America and Europe in the mid-2000s. As big international corporations, it would be expected that Nokia and their competitors would have test marketed these products and negotiated with the partner service providers to enable support for video and video streaming, anticipating its adoption and varied use [27]. Thus, one would expect that they would have negotiated more extensive and accessible network use

24 Radin (1997) has done metaresearch on all telepathy studies over the last couple of hundred years; whereas Ullman and Krippner (2002) have done much research on dream telepathy, especially how and when it is experienced, studies that inform my assertions herein, especially in Chapter 7 in Part II. 25 iChat claims it can support many video-chat windows simultaneously, yet in practice this is not successful with video fidelity and smooth, uninterrupted connection, unless one is on an extremely robust, fibre-optic broadband Internet connection (see this report online for more at http://www.dslreports.com/forum/remark,13288787 [accessed May 25th, 2010). Many of the multithreaded chat sessions I have participated in were not able to support a medium to high-resolution audio-visual quality involving 23 participants, with only audio coming through effectively; 1-to-1 Skype video is now theoretically possible on a mobile phone, but no one I have spoken to who has tried it is able to get it to work effectively yet. 26 See news story on this issue online at http://www.cbc.ca/news/background/tech/cellphones/economy.html [Accessed May 25th, 2010]. 27 They had, but in a limited way. This is slowly improving.

33

with these telecom service providers, to ensure wide adoption and ease of use of these services, for all types of possible activities that the phone models enable. However, this research has found that this was not exactly the case. Security has become an obsession in society, in the digital as well as the 'real' world, since the September 11th, 2001, World Trade Centre terrorist attacks. Many other world incidents of high-level hacking (such as the hacking of Google by the Chinese in 2010) have also underlined its importance. The media add to this fear. And yet, we are never truly safe in the real, everyday world. We are always vulnerable (physically or otherwise) and that is, for me, what makes living interesting. Yes, people die everyday, every minute they are attacked, raped, bombed, etc, and experience loss, pain and torment. This is life. The obsession with security, privacy, pass-codes, 'secure connections', firewalls, virus checkers, censorship, etc, takes the individual responsibility for good behaviour and citizenship away from individuals and communities, placing it in the hands of governments, institutions and corporations. By buying their products, our ability to govern our neighbourhoods and ourselves is removed, which is more frightening. This makes open-source and online creative commons in the coding and digital creative worlds an inspiring but subversive act, or at least a political decision. However, openness and the commons have not fully infiltrated the more controlled mobile world, similar to the Internet. This might be due to the nature of the market-driven mobile system, the proprietary, dependant and closed nature of mobile operating systems to date, which are managed by corporate stake-holders. Some may say that open-source is just a corporate strategy to keep innovation high while reducing research-and-development costs, by outsourcing to a wider network of coders. This may be true, but if so it still allows more open access for users and developers to explore software design, giving more options overall [28]. Many of my peers who are artists and technologists are beginning to develop useful open-source tools and projects. These projects will require more open networks to stream media more easily, without continually encountering economic and security barriers [29] as I have encountered when trying to stream video from university campuses. This is discussed further in Appendix I (see also the testimonials from technical collaborators on these issues in the appendices). This project was initiated from experience in working with mobile technology and wearable sensing devices [30]. My expectation was that the intervening few years of development would have improved these devices interoperability, especially in light of the last decades exponential speed of technological
28 An open-access summit was discovered while writing this section, taking place at the same time, discussing these very issues, and a new mobile ecosystem. It can be referred to online at http://www.openmobilesummit.com/aboutus.aspx (accessed on May 25, 2010). 29 More on markets and content delivery models will be discussed later on. Feijo et al. (2007), in their paper Why YouTube Cannot Exist on a European Mobile: The European Regulatory Strategy on Mobile Content, discuss the different models that Europe and the world have taken on and why open access is not attractive to them from a monetary standpoint, even if it is what users and content developers might want. I was able to get the proceedings to the conference late 2008, and only ready in the project timeline, to address this in July 2009, but only encountered severe difficulty in after that. 30 I started working on MINDtouch having come from working on my Masters project and on the whisper[s] project for 2 years (see project online at http://whisper.iat.sfu.ca/ and discussed later in this Part I), which lacked the level of funding and engineering expertise of MINDtouch.

34

advancement. As an artist, I had to find a technical collaborator who knew the intricacies of medical sensor devices and electronics, mobile phone programming and network systems, someone who was interested in the challenges of working on a performance art project. This was a difficult task. After several false starts and a year into my research, a few different people were found to work on the project. In all the planning discussions with these technical experts, and others throughout the duration of the project, nobody expressed much concern about any substantial mobile network barriers. As the only constant person on the project and not a network systems expert, I became alarmed when these barriers were encountered near the end of the project, during the performance events when network connectivity was most needed. At the onset of this research project, the indications were that the level of mobile networking that we required would be possible online using mobile phones. Using advanced smartphone models, such as the video-enabled Nokia N-Series phones, the systems and services initially appeared to be in place to support complete mobility, including the ability to stream live video, but in fact only now, at the conclusion of the research in 2010, is this becoming realistic. However, it is still not easy to do, or reliable, without an accommodating service plan, phone, stable connectivity or company/service provider, access codes, etc. Hopefully this will change in the very near future to enable a next version of this project. From the beginning of the project, 'unlocked' phones were chosen (meaning not locked to a contract or service plan). This would not have been easily done in North America in 2007; even though one could purchase the phones, they would not work in North America. In early 2007, one could rarely find pay-as-you-go SIM cards in North America, as a different system was in use (although this changed soon after). However, by the end of 2007 in the UK, one could buy 'throw away' SIM cards, which were purchased for the research phones for workshops and performances. My programmer collaborators aided in 'hacking' into the medical sensing systems for the project initially with the help, knowledge and blessing of the manufacturer to interface with the chosen phones. This began to backfire as there were difficulties eliciting assistance from the sensor manufacturer in order to create the custom software for our phones, [31] and their source code was too complex (and too poorly documented) for my programmers to work with or for use with the phones operating system. It was then decided that I continue down the DIY path, as artists often do (partly because it is less costly). We began to work instead with more off-the-shelf sensors and electronics, which Michael Market, my eventual collaborator, assembled and programmed. We also employed free, open-source coding options. This meant sacrificing some quality and fidelity in the sensor data (less critical to artists than medical doctors), as we had to daisy-chain the technologies and find workaround options (at least to the extent of the original vision), but it lent us more freedom. Michael and other technical experts advised taking this route since the technology was not quite ready at the time for how I had
31 This was partly because they also produced their own sensing Software Development Kit (SDK), and perhaps did not want or have time to support our endeavour.

35

envisioned the project to work. I was optimistic that we could still create a version of my original vision. What was not openly acknowledged was that the artistic decisions we made (partly due to limited resources) would cause the project to collide head-on with the normal, condoned practice when using mobile phones. While streaming video from a mobile phone is completely feasible using mobile software applications such as qik, livecast and ustream (qik now ships with all Nokia N-series smartphones since N-97), in practice delays and network barriers have consistently been encountered at every organised event. This was especially true when trying to stream to and from university campuses in the UK, as well as other firewalled networks. The reason given generally is that this activity, no matter how low the numbers of streams, opens their secure ports to malicious hackers and is thus a security risk. So we had to avoid standard phone systems, instead repurposing the technology for our own uses. This created added difficulties and challenges to achieving the original goals. Although none of what was envisioned is impossible according to most technical experts I have spoken to it is harder with limited resources and an ad hoc technical team.

1.7/ philosophical landscape (Literature Review Part 2)

Theories and critical texts relevant to the MINDtouch project are expanded upon in Part II. Theories were investigated through practice directly or indirectly within the series of iterative events featuring collaboratively performed visuals meant to simulate dream and embodied mixing and exchange. Herein is a review of relevant reading that was engaged in during this practice-based research. A few key theoretical concepts that ground the practical work include: presence and perception, consciousness, embodiment, the senses, telepathy, dream state, participatory performance practices, mobile media aesthetics, and live cinema. Various theorists are examined herein, though these by no means cover all the territory explored in the research process. However, below are discussed those who were most relevant to the work and/or were key influences on my thinking, development and practice. For thousands of years, philosophers have asked questions and postulated answers regarding the nature of life, the universe, the nature of human experience, etc. Prominent thinkers who are still studied include the Greeks such as Plato, Socrates and Aristotle. Building on their work in the last several hundred years, prominent philosophers have included Kant, Kierkegaard, Hegel, Heidegger, William James, Sartre, Bergson, Derrida, Foucault and Wittgenstein. Early in my academic studies, stimulated by a personal tragedy, I was drawn to Existentialism and its focus on individual and social daily existence and its expression, [] which attempts to man in his immediate, his original, relationships to the universe, in all his concrete plentitude and problematic ambiguity (Spanos, 1976: 2).

36

In masters-level study with Dr Susan Kozel, the field of Phenomenology was first brought to my attention as a relevant framing methodology and critical context for my work. According to Spanos, Phenomenology is critically related to Existentialism. I paid close scrutiny to the relationship between those concepts, and application of those ideas to the work of other phenomenologists including Edmund Husserl and his student

Martin Heidegger, followed by Merleau-Ponty and many others. The Stanford Dictionary of Philosophy defines Phenomenology as a philosophy of existence and states that it is:
[] the study of structures of consciousness as experienced from the first-person point of view. The central structure of an experience is its intentionality, its being directed toward something, as it is an experience of or about some object (2003: online).

It also states that,

For Heidegger, we and our activities are always in the world, our being is being-in-the-world, so we do not study our activities by bracketing the world, rather we interpret our activities and the meaning things have for us by looking to our contextual relations to things in the world [] We must distinguish beings from their being, and we begin our investigation of the meaning of being in our own case, examining our own existence in the activity of Dasein (2003: online) [32].

As a prominent phenomenologist who is used extensively herein, Merleau-Ponty describes Phenomenology as:
[] the study of essences; and according to it, all problems amount to finding definitions of essences: the essence of perception, or the essence of consciousness, for example [ it is] also a philosophy which puts essences back into existence, and does not expect to arrive at an understanding of man and the world from any starting point other than that of their facticity (Merleau-Ponty, 2002: vii).

Merleau-Pontys Phenomenology of Perception (2002), has provided seminal theories for many within the media arts. Not an exception, I too will look at his ideas. However, I will not reference the other usual suspects Deleuze, Bergson, Benjamin and McLuhan. Instead, I delve into Kozel, Varela, Grosz, Massumi, Idhe and others who explore presence, embodiment, the senses and technology. I also have an interest in the ideas of a few pragmatist philosophers, such as Richard Shusterman, John Dewey and Mikhail Bakhtin, who influenced McCarthy and Wright. In their Technology as Experience (2004), McCarthy and Wright deal with the nature of human experience, the body and technologys effect on people in various working scenarios. They argue that designers need to consider the emotional, intellectual and sensory dimensions of a users interactions with technology, arguing that people do not just use technology, they live it. Shusterman is compelling as he deals with aesthetics, performance and liveness in his book Performing Live: Aesthetic Alternatives for the Ends of Art (2000). He positions performing live more in terms of the reception, experience and aesthetics for the
32 The Stanford Dictionary of Philosophy (First published Sun Nov 16, 2003; substantive revision Mon Jul 28, 2008), available at: http://plato.stanford.edu/entries/phenomenology/#4 (Accessed May 4th, 2010).

37

audience, rather than against the mediated (Auslander, 1999) or the supremacy of the ephemeral (Phelan, 1993). He also tackles the body and somatics in his recent Body Consciousness: A Philosophy of Mindfulness and Somaesthetics (2008), from the point of view of body image or body consciousness in
modern society. To do so, he looks at the philosophies of Dewey, Merleau-Ponty, de Beauvoir, Wittgenstein and Foucault, and critiques each, making a new case for his own soma-aesthetics approach. He posits that we need a more reflective sense of body-awareness, which can result from self-cultivation, focused on experiential embodiment, as opposed to the current societal body beautiful obsession.

Other philosophers reviewed include Keith Ansell Pearson and Pierre Lvy on virtuality and the virtual. Ansell Pearson, in Philosophy and the Adventures of the Virtual (2002), focuses on Bergsons theories contrasted with Deleuzes interpretations of them, and his own philosophy on the virtual, time, perception, memory and being in modern contexts, in which he argues for the importance of rethinking these concepts in light of modern concerns. In Becoming Virtual: Reality in the Digital Age (1998), Pierre Lvy makes a philosophical reinterpretation of virtuality, and addresses concepts of the real versus the virtual, the actual versus the possible, and how the body, text, economics and culture are virtualised. Lvy also explains what his version of the virtual means philosophically and practically to modern society in terms of the Internet and networked technology. He puts emphasis on this collective intelligence and its positive potential for freeing social activity, by enhancing knowledge production and dissemination. Idhe was a fantastic revelation for rethinking many of these aforementioned philosophers, while contextualising them in terms of theorising the body and embodiment within current technology use. He focuses on the effects of cyberspace exploration on our lived experience, on what it means to be embodied when using technology, and on the role of our bodies in virtual reality contexts. This will be discussed further in Chapter 8 on the topic of technological embodiment.

1.8/ connective tissue

In placing my professional work, I have always been torn between three domains: the intellectual and conceptual, the social, and the field of art/performance. To reconcile these, I have tended to create work that incorporates all three; MINDtouch is no exception. The glue that adheres these three areas in MINDtouch is the exploration of intellectual concepts through a practical, collaborative performance video project. To connect the different domains covered in this research, there need to be connective threads. MINDtouch research has been about connection: connecting ideas, experiences, the body and mind, technology and art, contrasting concepts, databases with digital and mobile networks, video with performance, performance with mobile phones, body data with streaming video, dreams with telepathy, telepathy with networks, etc, but most of all, people with each other in new and unique ways.

38

Presence, liveness and telepathy are connected and interwoven concepts (see mind-maps of the concepts in Part II in the theory section), as they are each about sensing others: in the room or across distance, in near-to-real-time. The concern is with sensing at a distance and how it can be done differently through our technologies to help us exchange more familiar experiences like vivid dreams and the preconscious, emotive or visceral sensations. Such experiences are hard to otherwise express or share in their true form, except by some who experience genuinely telepathic exchanges. In Part II, I unpack these conceptually and philosophically at more length and show the connections I see. The diagram below shows the relationships between the topic areas covered in this exegesis and how I see them interrelated. presence, liveness and telepathy communication over distance

mobile and wearable technologies (+networks)

participatory performance activities

Figure 3 2010 C. Baker Diagram of thesis knowledge domains

Participatory performance is one approach used by artists as a method of engaging the public in exploring both technology and the real world simultaneously, in a practice called mixed reality, locative media or pervasive gaming [33]. Participatory performance is used here as a way to enable people to explore embodiment in technology, and to assist participants to sense presence over distance differently, through the technology. So this is a means to explore high-level meta-concepts through practice. The connection is: embodying specific instances of mobile and wearable technology. Why mobile technology? Partly due to its powerful new image and video capabilities, which inspire more creative and playful potential, enabling participation and mobility in a way that a computer cannot. But primarily its potential is in its closeness to the bodies of users, and its ability to go anywhere with them. So, the effort of learning of the new technology is diminished, unlike that of other software or hardware options for social networked interaction, such as Second Life. As a performer and video artist, I am fully sold on technologys power and have been anxiously waiting to work with mobile videophones in performance since the mid-1990s. I have worked with people for years, in numerous capacities, and been engaged in trying to empower people in various ways. This project seemed a natural approach to engage people in performative and creative activities, using their most closely held technological objects to play, collaborate and sense each other over distance.

33 These practices will be explained more in Chapter 6.

39

Wearable biosensing devices were incorporated, partly due to the funding imperative for the research, based on prior experience with using them (my masters project and whisper[s] in 20022005). Therefore, it was an intriguing opportunity to connect participants in additional embodied ways, turning the live connection into a more physicalised connection. It was also an imperative to connect remote audience/participants and attempt to enable them each to sense or feel the other live participants in a more tangible way. The concept was that they would sense this by sending their breath data, etc, through the network to touch and breathe life into the mobile video clips, expressing the feeling of breathing through the visual. This would then create a full feedback loop of body-to-body connection sent through digital translation, with networks as invisible threads connecting people a digitally embodied touch and exchange. The video is connected as a way to translate bodily and emotional sensations visually, but it also acts as a simulation of abstract, embodied dreams, sent and intermingled with others dreams. This simulation is meant as a mechanism to express and explore what it would be like to share and play with your own and anothers dream sensations communicating this emotional and visual state [34] over distance. The three domains of the digital, the bodily and ephemeral sub-consciousness (dream state) are connected by threads of philosophical concepts, manifested through the practical exploration and activities of the project. This is based on some initial assumptions, as well as discoveries made along the way. These discoveries neither prove nor disprove my assumptions, but imply new and different possibilities and outcomes that could not have been known without this exploratory process.

1.8.1/ exegesis/thesis structure This introductory chapter sets the stage for the remainder of this dissertation, giving an overview of the project, the methods, the theory, the issues and concerns encountered, and the approach taken to achieving the original contributions to knowledge that the thesis as a whole seeks to make. Part I: Practice, provides an overview of methodological, philosophical and actual approaches to the practice. This is followed by an overview of the artistic modalities used in the major MINDtouch case study, including modalities of: participatory performance, live cinema, mobile media, telematics, generative and database cinema and network art. This section discusses key inspirational projects and the work of exceptional artists from the domain of media practice, who engage in art works and performances that involve digital media or emerging technology, including mobile media, wearable technology, interactive video installation, VJing/ live cinema, generative cinema, participatory media performance, etc. The works and artists selected and referenced here can obviously only provide a
34 The dream state as defined here refers to a state of consciousness: [] that judging by the overall electrical activity of the brain, dreams are a kind of conscious state [] (Baars, 1997: 108). In an extended search of relevant sources in reference material and psychology literature review sections, there emerged very little of value in the way of a set of specific definitions of dreams except a semi-aware state of consciousness featuring a series of images, thoughts and sensations in ones mind (New Oxford American Dictionary, 2005: online www.oup.com/us/noad) and rapid eye movement.

40

sample of the larger field, since there is not space within the scope of one thesis to analyse the work of the full range of artists in these large and overlapping fields. Part II: Theory into Praxis, dives into a deeper discussion of the meta-level conceptual thinking behind the research, focusing on theories on presence, embodiment, liveness, and telepathy over distance. The theories and definitions of these key terms are extended beyond those outlined in Part I. Their relationship to each other and to the practices of grounding experiences, from and within the practical work, are also discussed with reference to the MINDtouch case study. This part makes extensive reference to Appendix I, where the MINDtouch project is included in summary form. This section is also where the case-study data shows an active relationship between the technical and performative processes in the development, planning and performance, as well as detailing the outcomes based on participant feedback about their experiences of the workshops and performance events in which they were involved. Part III: Conclusions & references for further study, offers a summary of findings and a list of suggested areas for further research beyond the scope of the PhD.

Part IV: Appendices (not for examination, but for elucidation and background information for the examiners)

41

Part I: mobile media practice in critical context

42

2/ hybrid methodologies of practice

As a hybrid performance project, MINDtouch has drawn heavily from the full range of media disciplines and practices in which I have engaged as an action researcher: video, media art, modern dance, music and media curation/production. Personal experience, creative practices, professional work, studies and training, combined with current activities and research, have been integrated for this PhD project. MINDtouch project development and research has straddled and combined methodologies such as phenomenological techniques using lived experience observations and the writing of journals (or journaling) with more objective ethnographic tools, like audience surveys and participant interviews. However, the overarching approach throughout the research has been an intuitive, performance-based strategy seen as a form of Art-as-Research. This chapter describes the philosophical and methodological approach taken within the MINDtouch research project as the main exemplar of the praxis for this thesis, as manifested in a mobile media performance event. It combines methods to create the practical participatory activities, in order to study presence and liveness that is embodied in the technology through the network.

2.1/ art-as-research
Throughout the process of writing this thesis, the methodological engagement has evolved and shifted focus to some extent, steering away from HCI (HumanComputer Interfaces) or Design Theory approaches [35] over time, and beginning to draw towards dance and performance methods and techniques. Art-as-Research (AAR) is the most encompassing, flexible, and therefore appropriate research approach for this work and is thus one of the primary methodological frameworks employed, despite being controversial [36] in that its status as a philosophically consistent approach has yet to gain full acceptance within the academy (in part because it combines methods from so many different disciplines). AAR [37] is used and considered by artists as a methodology, although some more staunch academic methodology purists might disagree. The AAR method as it has been understood and applied to this thesis is outlined in more detail below. Briefly here, it can best be defined as an iterative approach that defines itself as it evolves in relation to a specific transdisciplinary practice, and that can be applied as a methodology for research in the sense that it provides a philosophy of itself as a research method within its application. In other words, AAR can be seen as the methodology of the artistic practice of philosophy (and the philosophy of practice), which incorporates, or works as an
35 Which tend to use approaches like empirical methods for computing design or use-case scenarios for products in product design, discussed more later in the chapter. 36 Supervisors have suggested this. 37 The debates around the different terms for artists using practice as the source of their research is addressed in the introduction chapter and will not be further addressed here. I focus my attention here more on the reasons for using it instead.

43

umbrella, borrowing and employing multiple methods, tools and techniques from other methodologies. In terms of AAR, practice can be defined as the act of making art, performance or other practical, hands-on, embodied creative work (coding, dance, architecture, music, design, etc), i.e. making things (digital or physical). The research aspect involves reading, studying, exploring, examining, observing and analysing others artistic approaches and then synthesising writing about them while applying the knowledge to ones own practice and/or thinking while developing ones own manifestations of practice. In the book Practice As Research: Approaches to Creative Enquiry, editors Barrett and Bolt:
[] propose that artistic practice be viewed as the production of knowledge or philosophy in action [...] exploration of artistic research demonstrates that knowledge is derived from doing and the senses [... and] generative enquiry that draws on subjective, interdisciplinary and emergent methodologies. (Barrett and Bolt, 2007: 1) Common to art and performance practitioners and researchers, the AAR process can be summarised as

follows: search for inspiration, brainstorm, create a prototype, improvise, repeat, improve, practise, gather more skills, collaborate, practise, demonstrate or perform. AAR is often a personal, implicit, inferred, experiential and largely unconscious process, deriving from training, practice and a natural creative sensitivity combined. This step-by-step approach often also includes collaboration, feedback, presentation, review, and discussion similar to a standard design methodology. (Burgess, 2002: 4) The processes of art and performance practices often involve trial and error. Barrett and Bolt would add to this by saying:
As a result of this reflexive process, methodologies in artistic research are necessarily emergent and subject to repeated adjustment, rather than remaining fixed throughout the process of enquiry. (Barrett and Bolt, 2007: 6)

The diverse creative practices and explorations employed over the years in my artistic work has increasingly pushed me to be inquisitive, develop more skills and do research into new ways of working within new media. Barrett and Bolt have deeply influenced my thinking, with their notion that:
[] innovation is derived from methods that cannot always be pre-determined, and outcomes of artistic research are necessarily unpredictable. (Barrett and Bolt, 2007: 3, my emphasis)

Estelle Barretts chapter Foucault's 'What is an Author': Towards a Critical Discourse of Practice as Research, goes further in defining the artistic process for knowledge production:
In the creative arts, the outcomes that emerge from an alternative logic of practice are not always easy to articulate and it can be difficult to discuss the work objectively given the intrinsically emotional and subjective dimensions of the artistic process. (Barrett and Bolt, 2007: 135)

44

Barrett and Bolt suggest that the innovation and critical potential of the research is within the artist, in both the personally situated knowledge they bring to the work, and the techniques chosen to externalise that knowledge in their own unique practice, within philosophical, social and cultural contexts for wider application and engagement. Carter adds to this in the chapter Material Thinking:
[] a dialogic relationship between studio practice and the writing of the creative arts exegesis is crucial to articulating and harnessing studio methodologies for further application beyond the field of creative arts [] is validated alongside other more traditional forms for research derived essentially from the scientific method [] Embodied vision [] links experience, practice and theory to produce situated knowledge, knowledge that operates in relation to established knowledge and thus has the capacity to extend or alter what is known. (Carter in Barrett, 2004: 143145, my emphasis)

This last point highlights this process of engaging the theory through the practice. My process in linking the embodied vision with experience, practice and theory to produce situated knowledge, has involved taking the experience of facilitation and organising people in previous work and my social life, in tandem with my training in yoga and meditation, into the devising of the performative activities. Intuitive and embedded knowledge won from experience was accessed in this project while designing activities that would ensure participants would have a meaningful experience, deeply connecting with their inner world, all the while feeling safe to do so. Carters point and many other artists herein have resonance for this research and their
ideas and experiences bulwark much of my thought processes and aspects of my practice. To punctuate

this, Dianne Reid also adds from her chapter Cutting Choreography: Back and Forth Between 12 Stages and 27 Seconds:
[] practice makes tangible the theoretical [; ] practice as a response to lived experience, the temporal, the personal, and the collaborative reveal[s] how new subject matter requires new forms of expression and representation. (Reid, in Barrett and Bolt, 2007: 10)

Carters focus on artistic research being conducted like scientific experimentation, especially in media art or technologically manifested art, directly relates to my experience. I found that working with the technical aspects in the work, as well as the performances themselves, which involved making or planning an element, then testing it and iterating it, can be said to be borrowed from the scientific process of experimentation. Marilyn Burgess, formerly of the Media Arts division of the Canada Council for the Arts, which promotes media artists in Canada, stated that media artists usually work beyond traditional art practice methods, and incorporate lateral thinking, re-purposing and invention, borrowing from sciences to measure their outcomes:
[] such as collaborativeness, algorithmic thinking, interface innovations, philosophical commentary, adaptability or robustness, humour, critical commentary and scalability, which point to the specificity and radically different artistic paradigm of new media arts The artist's role is to challenge, critique and investigate existing conventions. (Burgess, 2002: 2)

It would appear from Burgess and others here that a hybrid practice-based approach is standard in media arts and performance practices. Thus, the AAR approach was intuitively developed in

45

MINDtouch: not always conscious, but working according to the needs and explorations emanating from the work-in-progress, and acted upon as the need arises, rather than necessarily according to a step-by-step path. When working with materials, Barrett suggests:
Materials methods and theoretical ideas and paradigms may be viewed as the apparatuses, or procedures of production from which the research design emerges [...] forged in relation to established or antecedent methods and ideas. (2007: 137138)

Barrett additionally asserts that this approach,

[] explores the complex interrelationship that exists between artistic research and other research and scholarly paradigms [...] Acknowledging the emergent and subjective dimensions of artistic research, Stewart describes this method as a process of continuous discovery, correspondence contradictions, intuition, surprise and serendipity. (Barrett on Stewart, 2007: 12)

AAR theorists and artists have been studying and developing more standardised and rigorous methods as more undertake PhD studies (in the UK, Europe and North America). Many do this to validate their work within an academic arena, and to complement their work in the art world. One researcher, Anne Douglas, in a collaborative article with Karen Scopa and Carole Gray, outlines three different Art-asResearch methods she has encountered within the Fine Arts: 1) personal research, 2) research as critical practice, and 3) formal research (Douglas et al., 2000: 4). They say that personal research is generally intimate, private, and often unpublished, work that an individual artist develops in a piece or project. It is research that involves professional practices of sketching, storyboarding, creating, exhibiting and publishing interviews. Research as critical practice, however, involves artists finding innovative ways of working, attracting new audiences, and challenging conventions of exhibiting and approaches to creativity. It [] is both embodied in individual examples of practice and carried across professional platforms for debate around practice, such as artists talks, discussion platforms and publications [] (2000: 4). Finally, formal research is conducted within academic environments, helping artists to gain research skills, contributing to knowledge in the field through standard academic processes that are formalised, taught and defended, as well as promoting an understanding of how and why things are done in art practice (2000: 5). MINDtouchs creative approach lies somewhere between research as critical practice and formal research. The project aims to innovate within a relatively new performance domain, using interdisciplinary methods to understand and negotiate a relationship with the audience, as participant performers, within and outside the conventions of exhibiting and performing. So, within Douglas definition of formal research, this work is a relative, heuristic research. The media performance contributes to a shared body of knowledge, through a recognised process towards a degree, which provides additional academic skills and credentials, but also toward further practice, and for the benefit of other artist researchers. Sullivan adds, in Art Practice As Research: Inquiry in the Visual Arts, that

46

visual arts knowingly situates the imaginative and intellectual processes as artists make use of an [] ongoing dialogue between, within, and around the self, artworks, viewers, and settings, where each is used to help create new understandings. (Sullivan, 2005: 191) Evaluating the approaches of many of these artists and practitioners has helped to clarify my own practice, approach to the work, thinking and research, by validating my use of an embodied and embedded knowledge to guide the process as opposed to following a formulaic methodology for the sake of it and letting the work and experimentation with it guide next steps. This has meant being sensitive and open to taking new directions from the work and improvising to the next level. As such, even though it started with phenomenological and ethnographic guidance, tools from other methodologies have been additionally applied to help balance it out. The participatory and improvisational nature of activities within the events echoes the process of the project evolving organically to its outcome, and this openness was necessary to deal with all the different interdependent pieces of the work. If one aspect did not work I had to improvise and trial the next solution to make sure all the other interlocking aspects would continue to work as well. Therefore, one rigid, predetermined method would not have been effective. The process has been a balance of working with technology, people and performances, each with their own systems, ecologies and needs. In his chapter Correspondences Between Practices, Stephen Goddard describes how to approach research, writing: Rather than relying only on the written component of an exegesis to demonstrate a reflective process, it can also be reflexively performed with the practice itself [] (2007: 117). While it could have been intriguing to write some of the MINDtouch analysis using the mobile phone video itself, as a self-reflexive mode of further exploring theory, there was already enough going on in this practice, and this idea can be saved for a future project. Goddard further discusses the writing process in art-based research:
As a methodological strategy, it was useful to integrate the narrative of the research process into both the practice and the exegesis [...] The overall narrative of the research process includes the story of the practiceexegesis relationship, and the ways in which both the practice and the exegesis reflect upon the chronology of the research process. (Goddard in Barrett and Bolt, 2007: 118, my emphasis) I have thought of the narrative as the phenomenology of process. This narrative is critical to the work as it unfolds; it is the research as much as the more practical, hands-on aspects. I have been writing throughout the research and the writing has responded to the practice and been the practice as much as the work itself. The writing, especially through journaling (discussed more in the next phenomenology section), has helped to clarify the practice throughout this research. Intriguingly, Goddard explores the creative practice as part of the methodology (2007: 119). He muses on

his process of finding his method within the practice and the exegesis itself as it evolved:
[] both the video production and the written exegesis were two parts of the same practice. I

47

was attempting to trace the ways in which writing and video technologies mediated and recorded my memories, stories, annotations and analysis. (Goddard in Barrett, 2007: 118)

Goddards point that video and writing worked to mediate his memories, annotation and analysis made
me realise that this had been my experience as well. The approach taken for MINDtouch has likewise

been very subjective and has allowed for a change in approach in cases where the process appears unintuitive or untrue to the work, or a mere exercise in objectivity for the sake of it. Yet the work is not only in the writing about it, but exists in the experience, and is embodied in the artefacts from it by participants. These video artefacts hold the experience of not only what is within the frame but also for me they hold some essence of the participant and my memory of them. They tell me a little about each of them and help me to understand their experience from my own phenomenological perspective. So, unlike Goddard, the video in my research did not record my memories or stories, but instead the experiences and sensations of the participants. Since they conjure up memories and stories from the workshops and events and what is not in the frame through the viewing and analysis of them, I am able to write and annotate in a similar fashion to Goddard. The viewpoint taken here is that one method an artist has is to create their own philosophical or conceptual expression of methodology, directly informed and/or led by her/his practice. In this way, the more an artist uses their work as their research, with the exegesis to support it, the more the artwork, artefact or performance, in its intrinsic artistic form alone, is the contribution to knowledge. Thus, the methods and modes of artistic enquiry used to make the work become accepted in the academy. Since my programme promotes practice-based research for the artists and performers, as do many art and design programmes in the UK, this is happening more and more. However, my programme as a media institute also sits within a computing school, as do many, so a tension might be perceived. As such, convincing the academy that within interdisciplinary programmes such as this, practice is a valid and critical mode of knowledge creation can still be complex. Yet new media art has been one catalyst among other interdisciplinary art-science research modes, from which artists have wished to pursue doctoral research and contribute their artistic knowledge, as research, to the scientific community and the world at large in meaningful ways. However, there continues to be a question as to why it is not yet accepted in the same way as the discipline of music, or computing for that matter, which allows for the examination of students through the work itself. Within other forms of art and performance, this form is not always an option, and theory alone is the mode of enquiry. What this issue indicates is that the artist-researcher may wish to create their own version of a musical score, code or map of the work, available for their peers, examiners and the academic community to experience and to recreate the work as it is intended [38]. Also, artists must still have the written exegesis available to accompany the work to further tease out their thinking, the conceptual research, as well as the project development processes, to ground the work as knowledge creation. Yet, it should also be acknowledged that conducting research is a practice itself; as such, artists need to contribute to the foundations of the examination design and process for their higher academic designation. The more we do so (even if

38 I hope to have the opportunity to stage a performance for my examiners, to practice this.

48

logistically painful to organise at times), the more artistic knowledge can become integrated into the academy, on its own merits, in the same way that a practical engineering, business, law or medical practice is. There is a slow shift in this direction in western countries, and potentially elsewhere. A first-person voice is often used in art-based research. The immediate, intuitive approach to writing by artists commonly rejects the use of a removed, third-person voice characteristic of traditional academic contexts, since the work studied is lived by the artist in the first person. In the same way, artists often choose a phenomenological approach in their academic art research. The entrenched academic traditions and constraints of disassociation and so-called objectivity imposed on research practice, to lend scientific credibility and reproducibility of experiments, feel foreign and antithetical to art practice, where ideas originate from the artist and there is little desire for reproducibility of their art. There is an evident paradox or tension here between scientific and artistic cultures of knowledge. Resistance can also surface when artists are asked to contextualise their practice, ideas and experience of process within those of other thinkers in the discourse within the research community, especially when asked to make an original contribution to knowledge. No matter how similar or widely discussed a more recognised and knowledgeable artists or theorists ideas are, an artist needs to feel that they are true to the practice and their own experience first. To be clear, this is not a resistance to explore others ideas or learn from them on the contrary, since these theorists do inspire, influence and motivate personal notions and approaches to the work. However, it can feel disingenuous when, upon having a thought, feeling, sensation or experience in life or in practice which can often become the motivation or inspiration for the work or theory one has to credit someone else for that experience, thought or sensation, as if they have copyright on the experience. I try to experience my own encounters with ideas through lived experience first, as pure sensation and expression as it surfaces, and then express this in the writing and in the work. It is then possible to contextualise with others who have had a similar sensation, thought or experience. I argue that an individual experience in ones own life and art practice still has its own dimensions, lived insight and nuance differing to that of others work and ideas, which can be added to and contextualised amongst those other, more prominent ideas. Regarding this practice of crediting others, Barrett makes a similar argument:
An innovative dimension of this subjective approach to research lies in its capacity to bring into view, particularities of lived experience that reflect alternative realities that are either marginalised or not yet recognised in established theory and practice. (2007: 143)

Individual and contextualised academic and artistic ideas are explored in this research process through reflective musings, as mentioned in the introduction. Sometimes these are based on new connections or realisations from the practice itself, and sometimes they are inspired by others. The process of contextualising my own ideas, creative visions and unique thoughts through others does indeed help to find other approaches or like minds to demonstrate that my thinking is shared. We cannot help but be influenced; yet artistic work is often about expressing ones own unique experience or interpretation of the world, to reflect back to others and to contribute and share with others. Finding the right mix is a

49

fine balance. Artists and performers undertake research in order to explore their own work and specific concerns, while adding to their own process, understanding and skills, and finding their place in the field and discourse. Learning is primarily gained through the experience of other creatives and thinkers in their work and thought, and synthesised, written about and applied as relevant to ones own practice and/or thinking. Much of my research has been inspired by the insights of other artists and theorists, found by reading, studying, exploring, examining, witnessing and analysing in order to find a path of my own. During this research process, my personal life and experiences have also greatly influenced my thinking, experience and understanding of researching mobile presence, influencing the choices and approaches to making the work. In this vein, Reid confessed that her approach was even more personal in her dance work, resulting in a blur between personal and academic. She reveals:
I have drawn on autobiographical incidents in which the inner self and the outer body collide, where a point on or movement of my body references and reveals a specific emotional experience or intellectual realisation. [...] extrapolated from stories connected to specific scars or points on my body associated with accidents or trauma. (Reid in Barrett and Bolt, 2007: 58)

Through my journaling I was reflecting and revealing specific emotional experiences as Reid did, triggered from my own mobile communication exchanges and play with the device, as intellectual realisations bubbled through, connecting the personal to the conceptual, as can be seen in Part II. This was similar to Goddards experience, as he has noted on his own work:
My research was situated within and across the overlapping fields of autobiographical writing and subjective video practices. (Goddard in Barrett, 2007: 117)

This briefly inspired me to consider writing sections of this exegesis as a diary. However, this has been abandoned in favour of including the personal/poetic observations of lived experience and musings of the mobile phone, virtual and actual presence, embodiment and telepathy. Many entries in my research journal and online wiki were called MUSINGS and have been included in this writing. These are integrated as an example of my phenomenological practice, within the theory exposition in Chapters 8 and 9.
Within her book Barrett charts many artistic approaches as a guide for artist researchers. One section states: The researcher traces the genesis of ideas in his/her own works as well as the works/ideas of others; compares them and maps the way they inter-relate; examines how earlier work has influenced development of current work; identifies gap/contribution to knowledge/discourse made in the works. (Barrett, 2007: 139, my emphasis) [39]
39 I have been advised not to do this but I agree here that this is a critical aspect to understanding or contextualising the genesis of current work.

50

As we can see, there are a great number of instances where artists allow personal experience to filter into the research process, to inform and shape the work (Barrett, 2007: 68). This approach appears to be essentially a phenomenological methodology, since one might argue that all research is influenced by the researchers personal perspective, experience and embodied skills and knowledge; this is where artists start. It could be argued that it is more honest to admit that this personal embedded experience is key to the works authenticity. Barrett adds in her Guide to Art as Research Methods chart:
The researcher discuss[es] the work in relation to: lived experience, other works; application of results obtained; contribution to discourse; new/transgressive possibilities; obstacles encountered and the remaining problems to be addressed in future research. (2007: 139)

I have tried to follow this guide directly and indirectly and found it quite helpful in this process. Barretts final point has informed how I have approached the obstacles and remaining pieces of the work: to be addressed in future research. To conclude this section, Kolb suggests that AAR is a problem-based learning approach, relevant to participatory learning as it focuses on knowledge generated through action and reflection. Thus, the experiential approach put forth by Kolb (2004) [] starts from ones own lived experiences and personal experiences [] (Barrett, 2007: 5) through action and then reflection on the action. What is relevant here is that AAR can also be seen as a form of Action-based Research (ABR), used in education and other forms of community research. In ABR activities and approaches to teaching or making changes and improvements in, for and with communities are planned, experimented with in the classroom or in the intended community setting to be studied. The observations and outcomes are then analysed and evaluated, and the activities are modified, refined and reworked for trial again. This participatory approach works toward finding the best method to inspire learning by students or for the betterment of the group or community (Reason and Bradbury, 2002). In some ways this is not dissimilar to programming or HCI design, or even a scientific approach in its experimental trial-and-error approach. This is the approach I have taken in MINDtouch. While each of the voices here have slightly different nuances to their interpretations of Art-as-Research as a methodological approach, they all essentially state that valuable knowledge can be derived from practice. Ranging from Australian, Canadian and UK academic contexts, all have similar approaches and are influential to my own experience of using practice as a source of knowledge creation.

2.2/ phenomenology
This research process has been explored through a phenomenological lens, not only methodologically, but as a practice and process of living and thinking; putting theory into daily practice. As such, phenomenology as a method and approach has been explored as both an initial guiding approach and a grounding philosophy for practice. This approach is appropriate since media performance can be very intimate in its creation processes. The body-centred approach used in this project, specifically the

51

use of biofeedback devices and the facilitation of self-attention with participants, has required a more intimate, personal, first-person treatment (as discussed in Appendix I). It is also a good fit with a Merleau-Pontian [40] perspective on perception and lived experience. Thus phenomenology, as a firstperson methodology, is encompassed by the Art-as-Research approach discussed above. In addition to the definitions provided in the introduction, Francisco Varela and Jonathon Shear state that phenomenology is the study of, and the method of studying, lived, first-person experience that is not about things-out-there, but rather ones mental contents-in-here. They state that:
[] the level of the user of ones own cognitions, of intentions and doings, in everyday practices the realm of experience is essential for human activity and life involving the use of ones own mind [] the experiential domain can be explored [] (1999: 3)

This has been a good rationale for using a first-person perspective in my conceptual explorations during the research. Varela and Shears phenomenological approach and tools of phenomenology include: introspection; attention during defined tasks using verbal accounts; meditation or sustained attention, uncontrived awareness and suspension of mental activity (1999: 5). I have employed these tools and approaches when thinking through the design and application of conceptual notions of presence and technological embodiment within the participatory activities in the workshops and events. Many of the concepts and realisations employed in the project come from the lived experience of using the mobile and digital communication technology with friends and family, as mentioned above. So I was introspective and paid attention during these activities, meditating on or musing on these experiences to draw further from them for the work. Martha Ladly has another interpretation of Heideggers phenomenological methods in [] the self showing in itself [] (Heidegger, 1996 in Ladly, 2007:142) and she states that,
[] it is the active process we undertake when we assign meaning to our experiences [which] cannot be separated from reality; it is an active process and as such, a creative act. Communication one to self; one to one; one to many; and many to many. (2007: 142)

This notion of assigning meaning to our experiences is one employed in my research on multiple levels: in understanding the overall creative processes, but also in the specific activities when working with the devices, the technical collaborators, and with participants. These positions manifested in my work in terms of: one to self, in my phenomenological process of creating and reflecting the work throughout it; one to one, in how I communicated with distance family and friends through mobile and digital means daily, as well as in the more formal communication with each of the technical collaborators; one to many, in how I related to the participants in the workshops; and many to many, in how the work was made so that many live and remote participants could communicate.

40 I will work from Merleau-Ponty more directly in upcoming writing and the final methodology chapter to be able to refer to his theories more directly.

52

Ladly

also

reinterprets

Herbert

Speigelbergs

elaborate

seven-step

method

from

The

Phenomenological, stating that his steps are to:

[] formulate a phenomenological description using phenomenological intuition, [] make a phenomenological reduction [] determine [] which parts of the description are essential. [] isolate the object of consciousness, the thing, situation, emotion or person that constitutes the experience [] based directly on the experience, rather than on a conception of what the experience may be like [] produce a phenomenological interpretation, an attempt to signify meaning, using hermeneutic analysis. (2007: 142)

Some of these, and Susan Kozels step-by-step methods, were employed intuitively in MINDtouch. Kozels methods of phenomenology were later discovered through her book Closer, which is a rethinking (reconsideration) of Merleau-Pontys phenomenology. In it she provides a set of phenomenological methods for body-based practices, performance, and for technology art practices, relevant to my practice. This set of Merleau-Ponty-influenced tools is constructed and interpreted through her performance and technology practice, and her practice as a philosopher. In it she reveals that one of the objectives in her reinterpretation or framing of phenomenology is to devise a means for creative practitioners and performers to use phenomenology to reflect and write on their practices. Kozels list of processes or tools [41] to guide practitioners include that we:
[...] loosen our rationalist structures of meaning sufficiently to permit qualities associated with the pre-rational, such as ambiguity of meaning, fluidity of existential and conceptual structures, scope for entirely new thought, perceptions, including contradictions, reversals of meaning, or paradoxes. (2007: 1819)

This perspective gives the artistic researcher more freedom to allow uncensored perceptions and encounters with the work, and intuitive responses to it, express themselves, either directly or after further reflection. Kozels guidance as a supervisor during much of this research must be acknowledged: her influence is very palpable. Closer exposed and provided a deeper examination of her poetic exploration of her phenomenological, practice-based approach. While I have my own approach from personal and professional experience and background, Kozels perspective has guided the refinement of my thinking and approach to the processes of practice. Directly working with Kozel on other projects also influenced my practice, even before reading Closer. This work with Kozel has unearthed an intuitive phenomenological approach or sense in establishing a methodology from the felt experience and process of making this work. Thus, phenomenology became a foundational methodology [42]. Working through the practice and with Kozels encouragement, a hybrid approach was developed and realised in the work. Therefore, her Merleau-Pontian-infused phenomenology flavours the thinking, intuition and writing in my own process, as can be seen in this chapter and in Part II on embodiment.
41 I have quoted the bulk of these in my methods essay and chapter, so I will not repeat them here. 42 Closer was not yet published when I started the PhD, but was purchased much later in the process and has been used to help me articulate my process through Kozels philosophical clarity and breadth of experience.

53

Mindfulness meditation techniques are indirectly incorporated into the work from prior studies of them. Dance, yoga practice and meditation training have also inspired my approach to facilitating participants in the workshops and performances. Kozels methodology and methods are akin to Buddhist Mindfulness practices. [43] An example of her steps is quoted here:
Take your attention into this very moment. Suspend the main flow of thought. Call your attention to your body and what it is experiencing (are you short of breath, is your back hurting, are you hungry?). Witness what you see, hear, and touch, how space feels, and temperature, and how the inside of your body feels in relation to the outside. What thoughts enter your mind once you suspend the main rational thrust? Register any seemingly trivial anxieties or thoughts but do not try to delve into their deeper significance at this moment. Let your mind wander and notice lateral associations. You sense-data retrieval depends on your context. Do what seems appropriate. Spend time getting in touch with your senses. (Kozel, 2007: 5355) [44]

A similar guiding script was used in the mind-quieting exercises conducted with participants in the workshops and shorter performance events for MINDtouch. Participants were asked to become aware of and focus on their sensations and translate them into visual low-focus [45] note-taking (see Appendices for the script). To create a record of their sense retrieval and exploration, the activities for participants were designed phenomenologically by revisiting, reiterating, revising and repeating each activity for each participant. This process of reiterating, revising, and repeating each activity also included observation of participations, an Action Research technique [46]. To uncover the most effective way to shape and facilitate the activities and experiences for others, this iterative approach was used. Next, the observations and experiences were compared with participants actual accounts. However, participants' accounts only partly influenced how changes were made to the activities. Unlike in Participatory Design [47], what is key here is the experience of observing participant responses to the activities and process, listening to them, seeing them interact with the devices and interpret the activities given to them, sensing the flow of the process, and seeing how well participants grasped the instructions. These all determined how activities would be changed for each iteration. Each performance staging was also based on the experience of the technological preparation, the set up and tear down of previous versions, making a significant difference in successive stagings or iterations of the performance experience. In this way, my role as director/facilitator was responsive, first-person and reflexive, but also leading development. So while each workshop and the overall process was designed, the complete synthesis of how the methods and strategies framing the phenomenological
43 Pema Chodron, a prominent Buddhist teacher and writer, as well as others, has many books and teachings on Mindfulness that contain practical methods similar to Kozels that she uses with her students. Start Where You Are: A Guide to Compassionate Living (1994) is one Buddhist book that teaches Mindfulness meditation. 44 This very truncated version of the method gives the reader the thrust of it as is relevant to my work and perspective. 45 David Gelernters phrase, discovered through Kozel (2007: 52). 46 Action Research is not explored more deeply here, as it was not used as primary research methodology, but aspects of it resonate in this study and it is referenced, since there are many parallels to it in the MINDtouch methodological approach. As a teacher / lecturer I perhaps also intuitively use Action Research in all my participant-based work. 47 Participatory Design approaches were explored earlier in the process and found to be greatly different from mine. The project does not use PD, because in the design sense I am not designing an object or artefact. This is discussed further in the workshopping section in Appendix I.

54

perspective were evolving was only clear after reflection and analysis of a few workshops. The impact of Kozel is also evident in the MINDtouch methodology through the performance development process gleaned from lived experience; since collective bodies, networked art and other embodied practices are used in both our work. Kozel begins by clarifying that she seeks:
[] more than an integration of the body-machine duality, more than a creation of a continuum or spectrum with the body at one end and mind at the other, but [] an immanentist approach to each of the dualities [...] as enfoldings or entwinements. (2007: xvii)

This exegesis similarly argues for mindbody extension through embodiment in technology and explores this through the phenomenological lens both of myself as artist and of the participants in the project. The hybrid methodology of MINDtouch was first experienced intuitively, and first surfaced through the development of the goals, expression and possible activities for the project. The intuitive approach developed through a vision of how others might experience the interaction, as well as how they would be encouraged to record their experiences during the events. Following this vision, a sketch of the experience design for participants was created, and then enacted as a design process for walking through the activities. The first step in this process was to note down these ideas like a script. They were then tested, both in workshops and on my own. It was an AAR process of trial and error (as Burgess suggested above) that was subsequently enacted by the participants, each with individual and different responses to the work. I did not attempt to predetermine or influence their outcomes or experiences, since each person comes with their own history, perspective and relationship with their own and others' bodies, as well as expectations and interpretations of what might happen. They would also explore the activities and use the device to interpret their mind's vision or visualisation of their embodied experience of their sensations in varied and unique ways. This discussion frames the intuitive process in the context of these methods and steps, rather than trying to suggest what they were adhered to closely from the start. Regarding an intuitive approach to writing from a phenomenological perspective of practice, Kozel muses:
Writing from lived experience often amounts to writing without a clear methodological mandate, or demands the courage to assert that the methods are fluid and subjective. Paradigms are scraped together (defiantly, guilefully, playfully, intuitively) from philosophy, literature, the social sciences, physics. This bricolage or hybridisation is done in part to find a voice in the academy, but more important, to help the writer herself understand what it is that she is experiencing and to communicate these experiences. It is done to give voice to the shifting terrain of the writer's experience and to find others with whom this might resonate, and to help her understand her own experiences and express them in such a way that others might find it meaningful. This sometimes precarious process is sustained by the realization that reflection is not only a secondary process or a commentary on experience, but also the process of thinking that transforms the doing. (Kozel, 2007: 9, my emphasis)

55

This perspective gets to the heart of how this research has evolved, validating the hybrid approach to the development of the practice. Kozel addresses phenomenological voice in writing of lived experience and practice (research), by pointing out that Varela and Shear compare first-, second- and third-person methodologies, stating that the content revealed is not different, but rather [] how the researcher is 'inserted into the network of social exchanges [] (2007: 57) and is validated makes the secondperson researcher an empathic resonator. This is because the researcher becomes sensitive to the other persons phrasing, body-language and expression. This is the role of a facilitator as well as of a director and experience designer (discussed further in this chapter). In these roles, I became an empathic resonator in responding to the participants. In Closer, Kozel emphasises Merleau-Pontys insistence that a pre-reflective state is the starting point to accessing and evaluating experience. In a phenomenological approach, prior to reflection (especially when writing on lived experience as it is encountered), pre-reflection is necessary. In reference to the pre-reflective state, Kozel clarifies Merleau-Pontys terms, useful in this research:
This [pre-reflective] belongingness with the world that preexists our conceptual engagement with it, but does not exist apart from it [] not just primordial but as mysterious: the body is 'a mysterious and expressive mode of belonging to the world through our perceptions, gestures, sexuality, and speech. It is through our bodies as living centres of intentionality ... that we choose the world and the world chooses us.' (Merleau-Ponty, 1986: 74, quoted by Kozel, 2007: 17, her emphasis)

This phenomenological intentionality of connecting to and engaging with the world through the body and through reflection, shapes most encounters, whether understood and experienced as primordial, primitive or mysterious. Nevertheless, Kozel, through her performance lens, reconfigures MerleauPontys position in such a way that repositions my experiences in life, art and performance more concisely and how I too try to engage through the body with intentionality and through reflection, cultivating pre-reflection though meditation and in art practice. This discussion illuminates another question to make a reflective practitioner pause and think: perhaps it is not possible not to violate or alter the pre-reflective state in the process of translation. One could argue that culture, gender and language, along with other factors, alter and colour or filter pre-reflection to enable us to work with and understand it in lived, daily contexts that we find ourselves in, as Kozel and others suggest. As artists, we can find ways through the practice to manifest nearer to the original pre-reflective state. When Kozel speaks of opening to aesthetic experience in her introduction, she is speaking of pre-reflection and hyper-reflection within phenomenological art practice. She maintains that this is important as a means of working with bodies and technology listening to the body prereflectively and:
As we assess the artistic, social, and corporeal implications of developments in computational devices and ambient systems, we need to have a method and an intent to receive information from our own bodies and from the bodies of others. In giving a voice to diverse corporeal states we can overcome naive distinctions between matter and spirit, between

56

body and mind. The second reason for crafting a role for the pre-reflective is that it can open a way for understanding the deep entanglement between reflection and experience, between thinking and making, which is so important to many arts and sciences. (2007: 22 23, my emphasis)

This perspective clarifies and reconciles my own conceptual conflicts encountered regarding the dualities of consciousness and corporeality that previously confounded my thinking and pre-reflection on daily, lived experience. It also clarifies the concerns and philosophical notions explored in the work as well as how to facilitate others in their experiences within technologically enabled performance activities. Kozel makes a case for exploring the elastic nature of embodiment and consciousness, to enable this pre-reflection, and to understand its paradoxical relationship with reflection:
[...] this philosophical impulse is like reaching beyond ourselves conceptually, perceptually, and existentially, but then returning to ourselves. (2007: 19) She also proposes that a way to present an interpretation of anothers experience within the context of ones own practice is as a heterophenomenologist: [] based on two realizations: the first is that phenomenological philosophy provides the conceptual framework and methodological sketch for interpreting the experiences of others; the second is that sensibility and interpretive power come from the physical experience of the phenomenologist [] or heterophenomenologist [... which] precisely relies upon shared experience, or some degree of empathy. (2007: 58, my emphasis)

This concept of the heterophenomenologist interpretation helped frame the task of reviewing participants post-workshop questionnaires and video interviews, and the video documentation of the workshops. This focus was particularly important when reviewing anothers experiences, recognising my role as researcher and my perspective with my observations. Thus what emerges is that MINDtouch is semi-phenomenological. When determining how to analyse and interpret participant data and observations to an ethnographic or heterophenomenological approach, Kozel suggests it:
[...] amounts to a slice of a shared knowledge, a morph of the lived experience of two people because the originator of the experience is trans-substantiated through the one who produces the phenomenological document [ and] that in an attempt to try to understand the experience of others, the fabric of one's perceptible, intelligent world comes through. (2007: 5859, Kozels emphasis and my emphasis)

To understand the experience of others has been an important consideration in how to process and interpret participants feedback and how to iterate the performance staging and activities, to understand the effectiveness of the technological system, and to determine whether it enables the experience of feltness, presence and liveness in the mobile network. Yet, as the researcher crossing roles over to the director mode in the process, the approach to this work is then not entirely phenomenological or even hetero-phenomenological, as I do not pretend to interpret or transsubstantiate (as Kozel calls it, 2007: 59) participants' experiences. Instead, my approach is more analogous to how Kozel describes Rouhainen:

57

[] she confesses that she never completely grasped what her interviewees meant. She writes of understanding the contents of the interview material only in a manner that is meaningful to her [...] (2007: 59)

This is a more authentic perspective for MINDtouch. The improvisational style that is used originates from dance studies and is also commonly used in theatre. It involves creating and producing or staging the events and workshops with a trial-and-error or experimental approach, involving the participants in creating the work. Ladly contributes a contrasting point about phenomenologically interpreting others

experiences:
[] one must temporarily remove oneself from an ongoing synthesis of experience, in order to reflect, to create definition, and interpretation the interpreter goes back and forth between experiencing an event or situation, and assigning meaning to the event [] (Ladly, 2007: 142)

This synthesis of experience by the interpreter as phenomenological art researcher was also the approach taken in the analysis of participant feedback. Effort was made to be reflective using a firstperson process to evaluate participants feedback appropriately. The phenomenological tools for the MINDtouch research have been: the first-person accounts of the experience of the research process and project development, including the workshops, the technical aspects and the mobile performance events; hetero-phenomenological experiences and observations of the participants and their videophone expressions/recordings. First-person accounts and theorisations of my own mobile and other personal experiences of virtual presence and liveness in daily life were recorded, as well as the performance design process.

2.2.1/ journaling phenomenologically

One method common to practice-based research, phenomenology and ethnography is keeping a journal and taking copious notes. For MINDtouch an online wiki was used, based on a paper-bound journal, in addition to video recordings of and by the participants for the project. Journaling also involved notes of the conceptual and design processes and outcomes. Typically, journaling in my practice of it involves noting any ideas that surface, usually during mundane activities or while reading related works, especially when trapped on a bus or train or sitting in outdoor cafes. These musings have always been a rich source for planning, philosophising and structuring writing and research activities. During the preparation, construction and experimentation of this project, much has been documented through journaling, taking photos, as well as videophone recording. All documentation of the video workshops, events and performances, VJ mixing and any other work-in-process is to be

58

sorted, processed, compiled and then designed into a multimedia format such as a Web/DVD. What is sometimes blurry is the distinction between a phenomenological account and documentation, as the latter implies a third-person removal, or objectivity, which is not my experience of journaling usually a more non-linear, first-person, lived account or brain-dump of the work. The journaling of research processes and thinking helps to clarify experiences, concepts, design planning, tests and activities. Sketching ideas out, keeping track of advisory discussions and other reference names/directions, helps to concretise the next steps, and create an overall repository [48]. Journaling every day is not necessary, but then again may occur twice a day: mainly whenever critical ideas surface, meetings take place, or activities need to be explored and developed. Sometimes these notes are not reviewed again for months. Writing in a physical, portable, bound, paper book is part of the process, as it is the most comfortable and usable format for my process, since it can be done anywhere, at any time. Salient notes are later transferred and reformulated, or ideas cemented in the online wiki. The journal notes are often used as fodder for the final written work, as well as for journal or conference papers. Two large, bound, paper journals were used for MINDtouch, starting from October 2006. Thus, journaling is a key phenomenological approach used here.

2.3/ ethnography
Ethnographic methodologies are generally used in social sciences to study human populations (Marshall, 1998). They often include several parallel techniques to maintain objectivity, while performing qualitative analysis of group behaviour. Ethnographic qualitative research techniques are sometimes employed individually, but frequently they are performed in combination, to acquire the most accurate and efficient accounts of group activities and structure. The ethnographic models employed here derive from my Sociology/Anthropology degree and interdisciplinary postgraduate background [49]. Ethnographic techniques can include: interview-based studies, participant observation, textual analysis, and conversation analysis. The ethnographic techniques used in this research include: workshops, questionnaire analysis, observation of video clips created by participants in the workshops (also part of Visual Methodologies discussed further on in this chapter), as well as: the analysis of interviews with participants after the performance events; and questionnaires, observation and interviews conducted with the workshop and event participants.

48 The research wiki was kept online but locked from the public (due to this wiki being spammed, it is here http://cami.chni.ca/wiki/index.php?title=Main_Page&chni_camiwiki_session=a7c81204fcb41f9dbfe3eec2ba5f947e but will be moved and made available elsewhere for examination purposes). 49 This understanding of Ethnography comes from extensive undergraduate studies in qualitative and quantitative methods, particularly ethnographic techniques that were used for embedded study of a Wiccan group. These skills and practices were updated through masters research and this present research.

59

More specifically, after each video collection workshop, interviews on the participants experiences were collected to develop a sense of their perception and engagement with the activities. Techniques also include: pre- and post-reflective experience questions, observational notes, videophone interviews, and observational/documentary video recordings. Also, each iterative performance or scratch event involved post-performance surveys and a few video interviews with participants, in addition to observational/documentary video recording. For the workshops, it was more disarming for participants to interview each other. Customised structured improvisation techniques were developed for the videocollecting workshops and participatory performance events, adapted from previous performance training [50] in order to most effectively explore and develop the participatory aspects. An ethnographic approach in my research involved observing both the participants in video-collection workshops and performance events, and those involved remotely through the network [51]. Multiple manifestations of sensing presence were facilitated during workshops and performance events, and observed as possible evidence or forms of the experience of presence in the real and virtual interaction: engrossed attention during mobile phone use; physical presence of participants and their body data; in-between invisible space of the data being sent to the phone; followed by in-between of the phone accessing the database of video clips; in-between space of the network connection; captured emotional, aesthetic presences of the participants within their video clips; physical co-presences of the participants in the space of the events; and virtual multi- or co-presences of the remote participants and global collaborators.

Some of these are presented conceptually in Chapter 8, which will discuss presence, and the results in Part III.

2.3.1/ interview process For the video-collection workshops (and also for performances), participants were recruited via an email call, or through word-of-mouth. Video interviews were conducted with almost all of the workshop participants after each workshop. The interviews were semi-structured, with the same questions asked of each participant (see appendices for the questions). Interviews were usually of short duration, depending on how much the interviewee remembered, cared to divulge, or wished to speak of their experience. For privacy reasons, interviewees were not required to reveal their faces on camera in these interviews. Participants provided personal accounts of their experiences of documenting their videophone expressions/recordings during these exit questionnaires and interviews. Certain topics were explored further, depending on an
50 Similar to Augusto Boals techniques, discussed more in the performance section in Chapter 3. 51 Technical delays in the development of the networked connectivity made some of this difficult to study, see more on this in Appendix I.

60

interviewees responses. Some questions were omitted if interviewees had already answered said question elsewhere in the interview. Each interview was reviewed and notes were made on each. Analysis details are discussed in the Outcomes chapter in Appendix I. I also looked for any patterns that emerged, any common experiences of interviewees, from questionnaires and video interviews, as well as the overall analysis of workshops and performances.

2.3.2/ performance ethnography

After several recommendations, it became clear that Denizens approach was relevant as a possible methodological guide to contrast with and contextualise my own performance methodology. Differing from the ethnography described above, in his book Performance Ethnography: A Critical Pedagogy and the Politics of Culture, Denizen discusses his approach to methods and methodology using an ethnographic approach for performance works:
Performance is an act of intervention, a method of resistance, a form of criticism, and a way of revealing agency [...] As fluid on-going events, performances mark and bend identities, remake time and adorn and reshape the body, tell stories and allow people to play with behavior that is restored [...] The way a performance is enacted describes performance behavior [...] (Denizen, 2003: 9)

What is key here is that Denizen, as with Boal, Phelan, McKenzie and others (discussed further in the next chapter), confronts not only methods of conducting performance research, but the political dimensions of performance work and its role in society. When carving a space for performance ethnography in contrast to autoethnography, he states:
The performance paradigm privileges an "experiential, participatory epistemology" (Conquergood, 1998: 27). It values intimacy and involvement as forms of understanding. This stance allows the self to be vulnerable to its own experiences as we are to the experiences of the other (Behar, 1996: 3). In this interactionist epistemology, context replaces text, verbs replace nouns, structures become process. The emphasis is on change, contingency, locality, motion, improvisation, struggle, situationally specific practices and articulations, the performance of con/texts (2003: 16, quoting Pollack, 1998: 38, my emphasis)

This is a critical point when examining my own approach, which is about action, process, improvisation and interactions, and allowing the self to be vulnerable to its own experiences. In a direct sense this is employed in the making of the performative activities for participants but also within the workshops and events themselves in MINDtouch. In the director role, I guided the whole performative process, from the devising to the enacting features change, contingency, locality, motion, improvisation, struggle, situationally specific practices and articulations that Denizen puts forth as performance ethnography. This role might also be considered reflexive performance or art research, or considered Kozels heterophenomenology, or all of those rolled into one with myself in different roles in the research as

61

set out here. This is also where the scratch performance concept or iterative theatre can work, as it is responsive to the audience or, in this case, the participant co-creating the work. Denizen changes the focus to state:
The performance text-as-an-event is situated in a complex system of discourse where traditional and avant-garde meanings of the real, the hyperreal, mimesis and transgression, audiences, performances, performance video, and performance art pedagogy all circulate and inform each other. (2003: 36)

This perspective contextualises the mode of performance I use as differing from the traditional: it may even be considered post avant-garde in its use of technology and removal of the audienceperformer dichotomy. I will discuss and unpack this, and concepts of the real, liveness, performance, and performance video, further in coming chapters.

2.4/ experience & sensorial design

With a background in performance, media art curation and event production, I have developed production sense in creating experiences for others. This involves reviewing what has worked (for me and the performers, technicians and the audience one and the same in MINDtouch) and what has not worked in the past, what could be finessed and what needs to be cut. The London-style scratch approach of staging performances was adopted the style of audience interaction that involves iterating the performance based on audience feedback which was compelling after exposure to it at several alternative, small, London theatre and technology-based productions [52]. A focus-group-style method of receiving direct feedback on what was experienced from the audience and how to improve the production, it appeared an effective way to include the audience in the development, staging and iteration of the work. Previous art-based research was revisited, in order to repurpose and reuse methods and philosophy of Experience Design. Experience Design is often used by Interaction Designers in web design, game design and in HCI for software design. Research on sensorial design [53] exposed that a framework informed by performing arts was needed to properly assess the effectiveness of sensorial media art design with users. Although focused on online education, Meredith Davis article, A Curriculum Statement: Designing Experiences, Not Objects, is relevant to Experience Design researchers. I modified her list for use regarding sensorial media design such as art and performance installation. She

52 One of those shows, called Robot Show was by robots, at the Battersea Arts Centre, February 2007. The actors gave out surveys at the end of the show and then had subsequent shows that implemented changes suggested by the audience. There were others but I do not have their information. 53 Sensorial Design is a term coined by Nathan Shedroff (1994), mostly used when discussing interactive web advertisement design.

62

suggested that the Experience Designer should: 1) observe and understand the difference between users responses to objects versus experiences; 2) analyse and synthesise what is meaningful and enjoyable in human experience and interactions with the world and other people in it, including: the physical, cognitive/intellectual, emotional, spiritual, social and cultural dimensions of our daily lives; 3) evaluate current and past forms of technological mediation on sensorial experience and the impact on people (positive and negative) in their daily lives; 4) assess users patterns of social interaction (ex. human conversation, interpersonal exchange) and use these patterns when designing technologically mediated experiences and environments; 5) understand how technologically mediated experience alters experience expectations and in turn changes human behaviour; 6) evaluate the mechanisms and tools used to create the experiences, by attempting to represent/simulate/visualise/transform perceptions, events and ideas in an artificial environment, eliminating all aspects of the non-artificial and potentially distracting or disengaging elements; 7) evaluate the appropriateness of the way in which the experience is received by particular users/audiences, while understanding how it shapes the understanding of intended meaning of the messages within the environment or an appropriate fit of form versus context, the very nature of sensorial experience design, and how it will change over time; 8) evaluate how these experiences impact upon users perceptions of the world, potentially changing them permanently; 9) evaluate how these experiences compare to other successful interactive, sensorial experiences, as well as to other sensual experiences, understanding the conventions and expectations created by users previous experiences, keeping in mind potential impact on both the user and society in the larger context such as altering human relationships, values and decision-making abilities; 10) evaluate the social characteristics of these technologically mediated sensorial experiences in terms of how they meet users emotional, physical and cognitive needs, such as how they respond, clarify, or provide feedback to users; 11) evaluate the way in which human attitudes toward, critical assessments and perceptions of each other in terms of an experience designers credibility, reliability, and authority, as well as their own role in self-determination and choice or agency in navigating an experience or the structure of persuasive arguments or puzzles, which then stimulates critical thinking and utilises common skill sets and values. (paraphrased and modified from Davis, 2000) This framework is relevant and has indirectly influenced MINDtouch. Interactive and media artists can call themselves sensorial designers by contributing to the field of design, although it is differently motivated. The current research follows this framework in some ways, including:

63

observation of participants needs in a variety of experience contexts; analysis of participants needs, goals and behaviour in general, but particularly within the context of the experience provided; and, delving deeply into phenomenological and psychological domains of the participants involved (from surveys, interviews and observations) so as to understand what motivates them within or during these experiences and performances, in order to then ask why they engage in these experiences.

For MINDtouch events I constructed a questionnaire, based on Davis and a similar framework I developed for a previous interactive project (also based on the workshop participants feedback, and curatorial audience surveys see Appendices). The purpose of these questions is to assess the experience of each participant, as in Davis framework above. From each workshop and performance event the purpose is to understand what they experienced and then try to iterate each performance or workshop from there. An example would be that if participants felt that one activity using the mobile phone was too complex or they did not enjoy it, I would then evaluate that and decide to modify or cut that activity if it was not crucial to their overall experience. In the facilitator role this is a critical function: making sure the participant is having a good experience and then, based on the overall goals of the project, modifying it for each iteration to ensure that an enriching experience is being had. See further analysis of the feedback in Appendix I for specifics on the implications of the questions and what they were trying to elicit, and the implications of the answers.

2.5/ embodied design / workshopping

Due to the complexity of the research, several methodological approaches were layered together to make a hybrid methodology borrowing tools and methods from those discussed in this chapter, as mentioned in the introduction. Here we look at what I am calling workshopping (it could also be called the workshop method, bodystorming or embodied design), used in HCI and other forms of product design. Experience as a research assistant for the whisper[s] project [54] exposed me to the embodied-design approach through its workshops. The intention of these workshops was to test user interactions with the technological solutions, without the technology. This solution, also called bodystorming [55], is a form of usability testing, from a performance and body practices perspective. Thecla Schiphorsts PhD practice was formalised in her thesis, The Varieties of User Experience: Bridging Embodied Methodologies from Somatics and Performance to Human Computer Interaction, which used the whisper[s] project as its main case study. Schiphorsts approach and thinking influenced my own artistic practice, although we each have our own research workshop design approaches and contrasting methods. Schiphorsts methodological approach, in her own words, is
54 See more on the project itself further on in this section. 55 Bodystorming is discussed by Oulasvirta, A. et al., Understanding Contexts by Being There (2003).

64

using:
[] the notion of paying attention to ones self, and using this sense of self or body-state to connect and exchange with another. This requires an experience of inner space or intimacy with oneself and an ability to recognize and transfer this sense of self to another person [ so] we turned to performance and Somatics methodologies [ and] techniques for extending our bodily awareness through attention to breath and movement [] common to performance methodologies found in theatre, dance and body-based disciplines. The techniques within these domains build both intra-body and inter-body experience and knowledge through technical exercises that focus on our perception of our own physical data. (2007: 111, my emphasis)

Schiphorst explains how workshop techniques used in her work were borrowed from performance practices as means to act out or act through aspects of the project design and planning process, and that this was especially key when using or working with people and their bodies. She explains:
Within performance processes, workshops have a specific set of functions, which include methods of exploration and discovery that are directed toward the development of new material, or bringing new life to repertory. Richard Schechner, a Performance Studies practitioner and scholar has written: A workshop is the active research phase of the performance process [...] Probably the most prevalent kind of workshop is used to open people up to new experiences, helping them recognize and develop their own possibilities. (quoting Schechner, 2002: 199, Schiphorst, 2007: page 117)

She points out that the model in whisper[s] of using intention to test, develop and iterate is a theatrical model, utilised as a user interaction model to explore [] experience concepts such as intention, gesture, direction of attention, relationship, rhythm, body-state, and attitude to space []. She explains that this process can inform the physical and technological design and assist in the evaluation and analysis of the relational elements in constructing experience (2007: 117) Her thesis outlines her project process and her discoveries in embodied interaction design to create and develop the whisper[s] wearable installation art project.
What was learned from whisper[s] was the process of developing and conducting performance research workshops, enabling people to engage with themselves, as a means to understand how to use technology in this context. But it also aided in learning through someone elses approach to the design of body-based technology systems how to allow people to listen to their bodies while engaging with others. Although whisper[s] and MINDtouch share some similarities, their approaches differ in intent, outcome and process. whisper[s] workshops engaged in experience modeling, as Schiphorst calls it, which were short exercises to mimic technology use without the technologies themselves, using prototype or stand-in tools. The intent was to design new devices and technology systems based on participant responses and behaviour with placebo versions of them. In contrast, the intentions of the workshop approach used in my research was instead to test new, embodied, performative and artistic uses for current technology. The making of new tools or software was only implemented where needed to bridge technologies in new ways or in new uses. My workshops were also intended to be a means of generating a media database for a

65

future performance installation. MINDtouch workshops and performance events were planned as experiences to assist people in exploring body states and sensations, similar to whisper[s], but also to assist participants in capturing and transforming those experiences into new, synaesthetically enhanced and ever-changing experiences for the performance collage/social VJ mixes. My goal was not to further design technology with the end artefact. Although technology and system design did take place, this was not as a result of the workshops, but to facilitate those workshops and performances. Technology was merely a facilitator and means to simulate and stimulate specific and unique forms of technological embodiment. Yet, the approach of whisper[s] has been hugely influential in clarifying the differentiation of the two approaches to workshopping. Each project uses the technique as a way to try out something with participants to be used later, in the context of technological experience. Here Schiphorst elucidates her use of workshopping as a technique: The whisper experience workshops were born from the desire to explore how people engage in the act of paying attention to themselves: their senses, their inner state and their world. The initial intention was to explore whether such an activity could be meaningful: could it have instrumental value in an interactive technologically mediated experience? (Schiphorst, 2007: 115)

Image 1 2007 C. Baker still from Dublin workshop, October 2007

For the MINDtouch project, the workshops have been essential in helping to distil methods in the creation of the participatory experience design. They were initially used as a means toward designing the collaborative performative events and live video artefact. Other intentions included: to help people explore the sensations of their bodies and their internal perceptions; to facilitate mental visualisations of these internal perceptions and sensations; and to assist participants to explore the act of externalising and representing these mental images (Image 1 from one of the workshops, shows participants during the guided mind-quieting exercise). They were also an experiment in facilitating non-verbal and gestural communication that could have other non-artistic, social or assistive outcomes after the project ended. Another goal was to create a media archive for the end performative video collage. So, these

66

workshops were intended to: 1) facilitate internal focus, attention and mental visualisation (i.e. help to capture visual imagery) in participants; 2) facilitate participants exploration in representing their mental imagery and impressions of their bodies sensations, using colours, shapes, patterns, objects, etc., from the external world; 3) facilitate the simulation of thought transfer and/or non-linguistic communication with others in the workshop; 4) enable participants to use the mobile video clips cameras to capture with the intention of sending these visual expressions of body perceptions to someone else (not in the room); 5) have participants collaborate in creating a database of video clips, to be used in the future performance events. The details and results of the workshops, and how they informed and shaped the performances, are discussed at length in Chapters 10 and 11.

2.6/ structured improvisation

Structured improvisation, borrowed from theatre, dance and telematic performance, has been most influential in the design and methodological approaches of MINDtouch. Training in improvisation techniques from choreography and other Modern Dance techniques have assisted in the design development of the participant activities. In dance and theatre experience, especially in Contact Improvisation (a dance form that involves physical contact and communication between dancers), dancers are warmed up in their bodies and then given small movement activities, either alone or with a partner [56]. These activities may be based on a visual or metaphorical element used to inspire the dancer to move in particular ways, or it may be based on sound or music played or used to explore certain forms of physical trust, weight, gravity and balancing activities with a partner. For the whisper[s] project, in the design of their experiential workshops, Schiphorst and Kozel used a specific approach to create the structured activities for participants to explore within, during the workshops discussed above. In a similar way, for MINDtouch participants were given tasks or activities [57] and were then observed as to how they encountered these. Participants were then asked how they phenomenologically experienced performing the tasks, as well as what they perceived or sensed in their bodies when using the technology during the activities.
56 For more on Contact Improvisation see the online Contact Newsletter available at http://www.contactquarterly.com/cq/webtext/resource.html (accessed May 27th, 2010). I have participated in these warm-up activities and contact classes. 57 More on the techniques and their use is discussed in Appendix I. See the Appendices also for the scripts used to guide participants through the exercises, using these techniques.

67

Madeleine Lord (2001) states from her research that there are five basic teaching strategies used to engage students to generate movement spontaneously: 1) setting up the situation; 2) presenting the exercise; 3) giving students a transition to execution of movement; 4) guiding the task execution; and 5) revisiting the situation. In her paper, Fostering the Growth of Beginners' Improvisational Skills: a study of dance teaching practices in the high school setting, Lord studied teaching strategies in secondaryschool dance programs, and these forms of improvisation techniques used in dance to inspire students
to move creatively emerged from her research. Similarly, in my workshops I would start sessions with a

mind quieting exercise (set up); present participants with the activity; assist them if them had trouble trying to engage with the activity and start using the video (transition to execution); guide them in the task execution (if needed); and then when they completed that activity I would bring them together to start again and revisit what had just been done. Improvisation techniques for theatre were also developed my Augusto Boal. His work and these techniques (as they apply, and were referred to in this project research) will be discussed in the next chapter on Performance.

2.7/ visual methodologies

As mentioned in the introduction and discussed further in Chapter 5, visual methods were employed in this project in order to evaluate and work with the video clips generated by participants in the workshops and events. These were mainly for analysing the content of the visual imagery in order to devise appropriate database categories. These categories were necessary for the custom software to appropriately use the body data received from the sensors, as a way to understand the intended content of the visual material created by participants. The methods used for the selecting, metatagging, categorising and analysing the video clips for the database and video mixing for the performances were borrowed from Gillian Roses book Visual Methodologies: An Introduction to the Interpretation of Visual Materials (2007). Roses book influenced how to analyse each videos visual content as intended by participants. Her work on art and design research considers different ways to look at images and visual materials from compositional analysis [58], including: 1) formal analysis, which focuses on the form of the object in question elements of form analysed include: line, shape, colour, composition, rhythm, etc. At its simplest, such an analysis relies heavily on the researchers ability to think critically and visually; and 2) stylistic analysis, which focuses on the particular combination of formal elements into a coherent style (Rose, 2007: 57). This stylistic analysis makes reference to

58 I have focused on these as the most relevant to this project since other visual methods she discusses are for analysing museums and their collections, film, TV and other media for analysis not related to this work. She also discusses Content Analysis, which I chose not to employ with all the other methods involved.

68

movements or trends in art and design as a means of drawing out the impact and import of a particular object. Rose suggests that the way to go about compositional interpretation of image content involves looking at and reading images by using the contextual information about the artist, their style and the imagery they were depicting and inspired by: 1) looking at the image itself for what it is or says, rather than what it does or how it is used; 2) looking at the content and form of images, rather than ways in which they were produced or social contexts/practices; 3) focusing on the technological modality of making the image how it was made technically and the materials used to make it, including: a. colour: i.e. hue, saturation, value;
b. spatial organization: atmospheric perspective, geometrical perspective, eye-level, logic of figuration (inside and outside the image), focalisers; c. e. light; mise-en-scene/camera for moving images: screen ratio, screen frame, screen planes, multiple images, superimpositions, shot distance, focus, angle, point of view, pan, tilt, roll, tracking, crane, zoom; montage for moving images: editing, continuity editing, cut; and d. expressive content;

f.

g. sound for moving images: environmental, speech and music diegetic, non-diagetic, narration. (Rose, 2007: 3558)

A modified version of compositional analysis was used here, focusing on participants (as artists in Roses work) videos from the workshops, since those were the most collected and plentiful. The content primarily featured the environment and spatial context of where the workshops were held, and included elements inspired by the activity participants were engaged in, and the available visual imagery they were depicting or capturing around them. This content is what Rose would call materials in #3 above, which were used to make the videos. These materials mostly contained imagery of architectural (interior and exterior) or structural elements that featured patterns. The materials also contained movement and other expressive content, such as extreme close-ups of a persons eye or the translucent piece of skin between the thumb and index finger. I chose to do a fairly cursory analysis, as the goal was not to judge or interpret the intentions or symbolism of these expressions by participants. Rather, this review and compositional analysis was performed to determine how to organise, categorise and meta-tag the clips for the visual mixing archive. There was also no intention of judging the quality of materials, as the participants were recruited from the general public, with no necessity for visual or performative background, so any expertise or talent in creating pleasing material was incidental. The aesthetics of the material is discussed in more detail in Chapter 5. Details of the content patterns found in the participant videos

69

can be found in Appendix I. The selection criteria for the video database is discussed further in Chapter 6.

2.8/ technology and software design methods

As I am a media artist and not a software developer or coder, I am not able to fully comment on the details of the methodology ascribed to by the technical collaborators working on the project. However, I can give an overview of how they developed the biosensing technology for the phones, the computer VJ mixing software to work with the video archive and the assembly of the electronics. In our meetings and interaction, it was reported that the main methods for working with these systems came from an empirical methodology, a process that moved from concept development and discussion to materials or coding exploration, technology development, testing, debugging, retesting and finally system implementation. There were many working sessions where we would start from a diagram or concept I had prepared and then work through and develop an action plan; these technical experts would then go away and implement this action plan, while I would follow up from time to time to make sure all was on track in the timeline. When we worked with the electronics, research on the appropriate sensor systems was done together, and then decisions would be made as to which to purchase. These were ordered through the university and then my collaborator Michael Markert would assemble them, try the mobile software out on his own phones in his home workshop in Germany, then send the sensors and the software to me to test on the research phones. We would retest until it worked with all phones, which were all different N-Series models (since there would be different models of participants phones). The software did not easily work on all systems right away and modifications had to be made so that it could. Each technical collaborator would train me in what they had created, and, each in a different style, made notes and created a user manual for their software after the creation as well, in line with good practice. This chapter has attempted to show how the differing methods and methodological frameworks have been integrated and implemented into this research to create a new hybrid Art-as-Research methodological form to shape the research. One can see that these mixed methods were used to complement each other for the sake of thoroughness, to address specific aspects of each phase of the project design and implementation, and to create an engaging and personal participatory technologybased experience, with the ultimate goal of implementing systems to enable the sensing of liveness and presence of others over the mobile network. However, each methodological dimension also has had a distinct role to play in uncovering different aspects of the work.

70

3/ participatory performance
This chapter focuses on the performance aspect of the project, on prominent voices in the performance studies, and on how different approaches have been implemented in the research. It discusses participatory performance practices and their use within the project. The form of performance applied in MINDtouch borrows from the participatory or public performance art of the Fluxus or Happenings or street theatre experimentation of the 1960s. This form often involves a structured improvisation with available, untrained participants, guided (or coaxed) by trained performers in semi-structured activities (Boal, 1992). For MINDtouch, participatory performance is used as a way to explore the mobile video and wearable devices as the means to re-engage people in performance and new media art. It is more aligned with new media and fine and performing arts than it is with mobile or locative games. It also serves as a way to (re-)engage people with the physical world through the media tool. This decision came from experience in performing arts and video, as well as a discovery that many current locative media theorists and creators come from a computing HCI (HumanComputer Interaction) or video-game background. In this way, this work contributes to interactive arts, video art, performance and live art more than to the video gaming, mobile entertainment, interface design, HCI or computing worlds, even though I use emerging technologies and computing to achieve similar ends. For modern dance influences and borrowings in my practice, key figures include Isadora Duncan, Martha Graham, Doris Humphrey and Rudolf von Laban. From music, as a consumer and musician, experiences are drawn from popular music and rock from the 1970s onward, which are many and varied. The non-musical performative experience and history is highly influenced by yoga, Tai Chi, and other body practices I have studied. As a performer, my notion and experience of modalities of performance have evolved and shifted: starting with a traditional understanding of it within music and dance, as well as an audience member, to also now include media and performance curation. It now includes a more interventionist, site-specific, live art [59] and networked performance approach, with some mixed-reality games (another name for locative media and psycho-geographic works, discussed in the next chapter) being introduced more recently. Gregory Sholette, as editor of the book Interventionist: Users Manual for Creative Disruptions of Everyday Life, defines Interventionist Art as:
[] the liquidation of artistic detachment by staging a fresh assault upon the tenuous boundary between art and life [] understands this conflict as a site for critical, artistic engagement within the public sphere [] the contemporary, so-called interventionist reveals a definite congruence with the historic avant-garde program [] (Sholette, 2004: 133134)
59 This was the mandate of The Escape Artists Society, of which I was an executive director. Tate Glossary is available online for interventionist art at http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=276 (accessed March 20th, 2010); Performance and Live Art online at http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=155 (accessed March 20th, 2010) and http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=218 (Accessed March 20th, 2010).

71

Sholette explains how interventionists make socially useful art in the vein of the Soviet avant-garde of the 1920s. Of course, socially useful might be a debatable term, but the form has definitely been seen to confront and find practical solutions for social issues, as can be seen from the work profiles in Sholettes book. The Tate Glossary has defined the term site-specific as: [] a work of art designed specifically for a particular location and that has an interrelationship with the location. If removed from the location it would lose all or a substantial part of its meaning (2010, online). This project has not been site-specific in this sense, but if we are to return to the earlier mentioned authors in the Introduction, in terms of telematics and virtual performance, some practitioners such as Glesner (2003) definitely consider the network to be a site. The Tate defines Performance and Live Art as referring to: [] Performance art and Action art and their immediate precursor Happenings, together with the developments of Performance since the 1960s [ and being] Art in which the medium is the artist's own body and the artwork takes the form of actions performed by the artist (2010: online). From these definitions, we can see how this project can be correlated with these performance forms, borrowing from the interventionist approach by repurposing the everyday mobile phone in order to socially engage and challenge people to communicate differently, while using the site of the network and the artist in this case the participant as artist or co-artist/collaborator, as part of the artwork. It is important to define the space this project occupies and why it is classified as performance rather than a mobile video or pervasive game (defined below). While MINDtouch might also find itself categorised with decidedly technological projects, it can be distinguished in many ways from these, because: a) location and geography is not the focus of the work; b) it borrows from performance and media practices, aesthetics and philosophy; c) many of the locative media and psycho-geography projects apart from the locative sound work of Janet Cardiff have originated from a video gaming tradition [60]; and d) this project has a more introspective, emotive, touchy-feely focus, rather than being an action-based, competitive treasure hunt, history walk or game, which many mobile projects are at present.

3.1/ participation as performance

To understand the design of the performance activities in MINDtouch, one should understand how the project began working with participants. In the first phase of the project (as described in the Introduction), video collection workshops were organised and participants were asked to share their internal images, feelings, thoughts and impressions, using the phone in a non-verbal, non-textual visual way, with the video recording features sharing these videos with others. To help participants to engage in the experience and activities, they were informed that their videos were part of the larger, collaborative video, and this collection of their personal clips has since been used in the live, networked

60 Which is traditionally male-dominated and many locative projects also demonstrate this fact see the name listing of locative media creators at the Mixed Reality Lab in Nottingham, for example: http://www.mrl.nott.ac.uk/people.html.

72

performance experiments/events. As you can see from the image below, the performative engagement of the participants made them express themselves with the video camera to others within the workshops in a very engaged and gestural way.

Image 2 2007 C. Baker participants in video collection workshops

On the surface, MINDtouch might fall into the locative media or mixed reality category [61], related to psycho-geography [62], connecting place and location to emotional or personal experience through sound and media. However, this work is deliberately situated within the avant-garde, live art, theatre games, or live cinema and video art contexts from which I come. To understand performance and participatory performance more deeply we can start with David Saltzs stance on performance, audience and interactive arts, as explained by Kozel:
[ i]n his essay 'The Art of Interaction' (1997) [...] hinges on performance being constituted by actions performed before an audience: If we accept performing for an audience as the distinguishing characteristic of performance, it follows that all staged interactions are performances, and all participatory interactions are not. [] if people see their actions as aesthetically significant, they are performing. (Kozel, 2007: 68, quoting Saltz, 1997, my emphasis)

On this point, one could argue that aesthetic significance or even an awareness of aesthetic significance is not necessary. An audience member or participant can have an engaged experience that contributes to the aesthetic significance of the work, without realising this significance. Rather, just having any emotional, existential, corporeal or thoughtful transformation or response to the activities engaged in within a participatory performance is in itself of significance. It can be argued that most art is about shifting perceptions and trying to communicate with an audience or spectator.

61 i.e. using mobile devices in the real world, alongside computer game contexts and interfaces, as Blast Theory and Steven Benford use the term. 62 Janet Cardiff and other sound artists and musicians.

73

Awareness of ones role in the event or piece, or some sense of contribution to the work, is indeed performance, as it is a sense of dialogue that the participant/audience/spectator has with the artist, yet it is more than that. Erving Goffman (1959) [63] might say that we all play a role or perform at any given time, that just being part of a participatory performance places one in a different role in ones life with or without awareness of its aesthetic significance, but with an aesthetic experience. Thus, it automatically has aesthetic significance since it is transformative. This does not mean participants become artists/performers themselves, in the same way as the artist who created it, but they become actors within the work and the artist/creator of the work becomes the choreographer or director (or facilitator): each is important for the work and each is aesthetic in their role. In a search of other discussions on the current discourse in Participatory Performance, I came across two practitioners in dialogue on its nature. David Goldenberg and Patricia Reed define what constitutes participatory practice and how it manifests practically through this exchange:
Reed: [...] the project is initiated, which means there is a condition of response inherent to it you play within, around, against that initiation, so in that sense its not infinitely open [ so] partial unpredictability [....] the maintenance of joint attention in a conversation the perpetuation of partial novelty. Basically there are enough unwritten rules in a conversation that we know how to perform it [...] Goldenberg: By setup I mean whatever location, site, or context where a participatory practice takes shape without acknowledging the restricted notion of place where art occurs, whether in a gallery, museum, or public space [] so the possibility exists for participants to actively take over the running of a project [] There is a sense if you are involved in a participatory practice experiment that a level of success is achieved if the participants take control of the project on an equal footing. (Goldenberg and Reed, 2007, online)

This is a definition of Structured Improvisation: providing guiding elements for participants to play within, with a setup and gravitational force, or activity containment factors, so that the activity is not anything that goes and doesnt become totally unclear to participants. It also has partial unpredictability or a wide range of possibilities of expression, exploration and experimentation. To reiterate in another way, Steve Dixon says that the:
[] four types of interactive art and performance we discern are ranked in ascending order in relation to the openness of the system and the consequent level and depth of user interaction: 1. Navigation, 2. Participation, 3. Conversation and 4. Collaboration. (2007: 563)

and that,
[] there are more genuinely collaborative models in interactive arts where the users input is freer, more open and more significantly changes what happens. (2007: 565)

63 Goffman has been studied during this research, but his work is not central to the work. His discussion on selfhood as an everyday series of roles we play, rather than performance itself, is intriguing, especially when working with non-actors. But in this regard, Boal is more relevant.

74

This project would fall into the collaborative category in this scheme, although it uses a form of interaction that is more of a conversation: between participants, participants bodies, and their own interpretations of their bodily sensations mediated through the video imagery and technology. Saltz view is relevant however, as it contrasts with my approach of challenging the conventional notion of performance that is staged in front of an audience and instead allows the audience to be anywhere in space and time, and to become performers with access to the final live cinema performance event, in front of them (via their devices).

3.2/ politics of participation

During the research planning and initial video workshops, a realisation surfaced that in order to draw people to the events, an important step would be to make every event a non-performance in other words, to drop the word performance from all calls for participation. The word carries baggage and connotations of power relations between the audience and performer, and could be intimidating for potential participants. A sense of fear or inadequacy had initially been witnessed in a few participants, some admitting that they felt they may not perform properly, as if there was a correct way to experience their own sensations or to do the activities. The word 'performance' is loaded and can intimidate, suggesting the participants are performers and not that they simply perform. Some may feel that they must try to be 'good' or 'trained', which could have an immobilising effect. So the words 'performance' and 'performer' were removed when recruiting participants [64], and in later events I chose to remove the performance language when guiding participants as well. Instead, it was framed as a more inclusive social event, more like a party or participatory entertainment event, with the hope that this would decrease pressure on participants to think they needed to be individual centres of attention, performing for others [65]. Removing the term had the added advantage that people with performance backgrounds would be as likely (as opposed to more likely) participants as those less interested in such things. So, in choosing these more casual social contexts, the performance activities could become more personalised for participants, yet playful within the larger social context where participants would feel they had full control over when and what to share. Through iterations of the work, it was discovered that when the workshops or events were called activities rather than performance, more participation came from the non-creatives and non-performers. Thus, if the goal is to reach and engage the average Joe or Jane on the street non-traditional, untrained performers it is key to find a new language or means to make them feel comfortable to engage, thus less intimidated and less inhibited. It is also important to make the activity meaningful to the participant as a
64 Participant recruitment methods will be covered in Appendix I on the workshop details. Briefly, they were found through wordof-mouth, email-outs, posters and Facebook. 65 Keep in mind that in this work there is no audience per se all participants are meant to be active and even those remotely engaging on their mobile phones or online watching the live mixed collaboratively performed video collage are considered participants. Those in the party setting who are not part of the work might be the closest to an audience. However, if they are not aware of whats going on around them they are not an audience either.

75

part of the project and not to create a sense of right or wrong in their expressivity [66]. So this was also an effort to change the views of the participants, particularly on how they conceptualise what constitutes a performance, as well as to change their relationship and involvement with technology, as well as to encourage them to use technology in unique ways to express themselves, to empower them to communicate creatively. So again [...] if people see their actions as aesthetically significant, they are performing (Kozel, 2007: 68, in response to Saltz) [67]. Since the Fluxus movement and the age of Happenings, participatory performance has developed considerably; Bishop as well as Bourriaud suggest that this is a growing area of art practice. Combining participatory performance with interactive mobile media and video games, the group Blast Theory has pioneered the creation of compelling participatory projects. Blast Theory has a talent for making their projects intimate and personal, yet playful and collaborative for participants at the same time. They combine theatre, video game play and technology with live interaction, in-the-streets engagement and exploration. Thus, they have become the signature and poster-child group for locative mobile media arts performance. Although there are others now, Blast Theory remain ahead of the pack and have thus influenced my project a great deal. Kate Adams explains the political aspects of Blast Theorys project Uncle Roy All Around You [68] in the context of participatory theatre:
Participatory theatre is associated with the political because of the possibility of creating a sense of community and empowerment as the audience interacts directly with the performers [] they are active in constructing meaning and active in relations with others [] which in turn requires the negotiation of boundaries within the relationship between performers and participants. (Adams, 2003: 7)

Images 3 and 4 2003 Blast Theory Uncle Roy All Around You

What was most intriguing about Uncle Roy All Around You was that it was a participatory performance as well as a mobile and Internet game. It engaged participants in unique, theatrical activities, involving the participants/gamers in the city using portable devices and phones rather than
66 This is based on observations made and interactions with participants in the workshops and events. 67 See page 97 in Chapter 5 for a definition of aesthetics for this research. 68 This project can be seen on their website, available online at http://www.blasttheory.co.uk/ (accessed March 20th, 2010),

76

only avatars on computers. This is inspiring since MINDtouch is also about creating unique embodied, media experiences that move participants away from the desk indoors and engage them more in the world and their own bodies. It is political in the sense indicated by Adams, because it creates an active engagement with both others and self by negotiating the performer/participant boundaries and relationship. Blast Theory have had several other exciting projects, such as Riders Spoke, which involved individuals exploring London city centre on a bicycle and with a PDA. The PDA had a customised, playful digital application, which both guided participants to certain spots, and enabled them to add their own experiences and stories about other spots. A friend participated in Riders Spoke and showed me the interface, which had a very comic-book-style aesthetic, untypical for a digital interface (see Image 6 below), which had the effect of instilling a certain trust and intimacy in the project for this friend. Participants were guided on their own journeys through the city through the device attached to the handlebars of the bicycle (see Image 5 below), by previous riders. This created a collaborative, interactive narrative. In a similar way, participants in my project have been contributing to a larger database; in my case a database of visual experiences that add to the collective work.

Images 5 and 6 2007 Blast Theory, Riders Spoke

A prominent and original voice in political participatory theatre is Augusto Boal and his Games for Actors and Non-Actors. What is crucial about Boals guide is that it is filled with practical advice and exercises to prepare participants for theatre games, and actors or guides for participatory performance and intervention. It also contains many examples of these activities in practice. The term theatre game was coined by Boal and is used carefully here since my project is definitely not a game. When devising the improvisation activities in my workshops and preparing the first version of the performance staging, Boal became a guide to theatre games. His ideas helped to clarify how to develop ways to motivate participants and facilitate engagement in the process and the theatrical activities of the performance production. It also helped to clarify how to create a dialogue between the participants as performers and the trained performance guides assisting them. Boal is extremely relevant in this approach and in contextualising the directorial or choreographic/activity design

77

process, because I consulted his games and techniques when devising my activities. His philosophy was most deeply considered when working with the bodies of actors and non-actors, firstly regarding specific exercises that I developed and was already using, which were similar to his techniques, in my mind-quieting and body-focusing warm ups [69]. His methodologies, while discovered after I created my activities, validated my approach. He explains his performance philosophy:
A bodily movement is a thought and a thought expresses itself in a corporeal form [] the idea of eating can induce salivation, the idea of making love can produce erection [] The phenomenon is less obvious when it relates to a particular way of walking, sitting, eating, drinking, speaking. And yet all ideas, all mental images, all emotions reveal themselves physically [] Bodily activities are activities of the whole body. We breathe with our whole body, with our arms, our legs, our feet, etc. (Boal, 1992: 61)

This is critical to consider when working with people to connect with their bodies, especially when asking them to tune in and interpret their own perceptions and sensations as creative and performative acts. When reviewing the exercises and games he developed from his Theatre of the Oppressed, Boal says,
The games [] deal with the expressivity of the body as an emitter and receiver of messages. The games are a dialogue, they require an interlocutor. They are an extroversion. (1992: 60)

Some participants revealed in the video workshops that they never think about their bodies or listen closely to them, and that the activities they were asked to do helped them to do so. This was a revelation and exciting because: a) it was one of the intents of the project; but also b) it revealed that there may be a need to help reconnect people to themselves and engage or listen to their bodies in new and engaging ways, other that calisthenics and sports. Technology, specifically mobile phones, may be able to help in this way. Boal informs this when he says:
In the bodys battle with the world, the senses suffer. And we start to feel very little of what we touch, to listen to very little of what we hear, and to see very little of what we look at. We feel, listen and see according to our specialty; the body adapts itself to the job it has to do. This adaptation is at one and the same time atrophy and hyperatrophy. (Boal, 1992: 61)

Boal is often credited as being responsible for developing the political approach to public intervention, empowerment and change, as are other performance artists of the 1950s and 1960s, such as Marina Abramovic. They were instigators of happenings and public interventions [70] and responsible for sowing the seeds of modern participatory theatre and live art. My work is not apolitical in general, and it must be acknowledged that most art and performance has a role to play, politically and culturally, in

69 However, since I worked with and was heavily influenced by Thecla Schiphorst who studied Boals work closely, one could say I was indirectly influenced by Boal. 70 Public interventionists are the topic of Claire Bishops Participation (2006), not discussed here, but this use here is primarily in reference to the books excerpt from Nicolas Bourriauds Relational Aesthetics (2001) and his discussion on the change in direction of current art practices, leaning more toward socially engaged, relational works, some also calling it the performative turn.

78

confronting society with itself, especially in historically critical moments. However, this work is more about challenging society to take a look at how it uses its communication tools and technologies, from a more philosophical than political perspective. It can be seen as interventionist in that it confronts people, but it has a more emotional and expressive approach, focusing on a different mode of empowerment; this mode of empowerment involves an understanding of self, and uses alternate forms of communication to do so. As mentioned earlier, the word game or the game rhetoric of other locative media or 'mixed reality' projects has been avoided in the MINDtouch project, since these projects have roots in video games [71] (apart from Blast Theory, who are a theatre group), computing, video games, interaction design or HCI. This research is a direct attempt to steer away and separate from other mobile and mixed reality games, opting more for this theatre game or participatory theatre approach and perspective. This choice stems from personal training, but also from my feeling that performance is not properly acknowledged or understood within the field of mixed reality and pervasive games [72], apart from the work of Blast Theory [73] and the theories of Brenda Laurel. Thus, aligning with Boals work and approach to using participatory theatre to confront people with themselves is decisive. In a broader sense, Richard Schechners key text Performance Theory provides a more cultural and historical perspective on performance, in its many varieties and permutations. As a definitive voice in performance studies since 1988, challenging traditionally held views in the field, Schechner demonstrated the existence of performance in everyday life, across cultures and history, in play, sports, legal proceedings, rituals, and in pop culture, in addition to the more traditionally accepted theatre, music and dance. He has provided the foundation for performance studies by examining cultures and history, covering all contexts in which humans perform. He borrows much from Sociological, Anthropological and Archaeological studies, to create an overarching view, still referred to widely today. While Schechners ideas might be obvious to many today, they were not always. Schechners work grounds current participatory practices by expanding the parameters of performance, which tends not to be acknowledged in most participatory and locative performance projects. Yet performance practitioners are now crossing boundaries or definitions of performance through participatory performance and street theatre, not to mention video games or mixed reality and pervasive media [74]. All of these forms are perhaps not new, but have been reconfigured, using new and emerging technology.
71 The term mixed reality meaning a mix of locative media game in the streets like a mobile treasure hunt with video or computer elements played by different participants interacting with the street counterparts. 72 Plenty within the gaming and interaction design fields use Laurel as their only performance or theatre reference or expert, ignoring much of performance studies, as noticed in numerous articles on locative media (i.e. Steve Benford et al. at the Mixed Reality Lab in Nottingham). 73 Again see more specifically on the project Uncle Roy All Around You by Blast Theory at http://www.blasttheory.co.uk/bt/work_uncleroy.html or read Kate Adams piece on the project in the Body, Space Technology. 74 See also the article The Players' Realm: Studies on the Culture of Video Games and Gaming (J. Patrick Williams and Jonas Heide Smith, 2008).

79

Schechner has his own categories of the different types of performance that exist: 1. aesthetic = the audience is transformed by the performer; 2. ritual = a subject (or person) is transformed by an official performing a ceremony; 3. social drama = dramatic life events occur right in front of people, transforming them; they are real but out of reach (for example, watching the planes hitting the World Trade Centre in New York on television), so the audience is transformed but helpless, and thus becomes indifferent. Then he suggests that participatory performance has another form of transformation:
Some people react by making and/or enjoying art thats more real, introducing into aesthetics the interventions and feedback eliminated from ordinary life. Thus, it is no longer strange in theater or performance art to involve the audience directly in the story, to stage actual encounters among people [] (Schechner, 1988, 2003: 194, my emphasis)

Schechner defines what he means by real (in performance theory), as,

The actualization of art the existence of theater combining the social with the aesthetic. (1988, 2003: 195)

Thus, in these two passages, he is suggesting that participatory performance can give participants the opportunity to give feedback or somehow intervene in art more directly, feeling they have more of an impact and perhaps even empower them when other aspects of their lives might be disempowering or leaving them feel helpless. The real here is then bringing art actually into the social lives of the audience, rather than the power dynamics of us and them in the traditional staged context. Then, Schechner goes further on how the audience experiences and receives or is transformed by a performance:
The function of aesthetic drama is to do for the consciousness of the audience what social drama does for its participants: providing a place for, and means of, transformation [] Aesthetic drama compels a transformation of the spectators view of the world by rubbing their senses against enactments of extreme events, much more extreme than they would usually witness. (1988, 2003: 193, his emphasis)

This is a compelling statement on the function of the audience within a fictional drama. He suggests it is like a test run for the real thing, in their minds but not bodies. What I find fascinating about this idea is that it puts the participant into the action it becomes real for them, more deeply than participatory performance. It suggests the fictional dramatisation but puts the social drama into the consciousness in a much more powerful way, so that the real and the performance are blurred. In recent participatory theatre, groups like Blast Theory have gone as far as staging fake social dramas, such as kidnappings [75] to further blur the lines and push the boundaries between the real and the fictional. The

75 People actually paid to be kidnapped, see Blast Theory on the 1998 project Kidnap (online), available at:

80

performance was so real that the actor no longer even knew what the difference was between this kidnapping and a real one, since they were going through all the same actions, except that the victim volunteered for it. This is a more extreme version of Big Brother and other reality television shows, which create social dramas and turn them into a blurred staging between real and fictional. We will continue on the real, the actual and presence next.

3.3/ presence in performance

As a performer, the experience of performing comes in several forms. The two most prominent are: 1) the common, informal sense of storytelling and engaging with friends, such as describing a setting, feeling, experience or event to the fullest, most visual and embodied extent, using vocal tone, inflection and flourishes, gesturing with eye contact; 2) the more formal aesthetic form performing music, dance, etc., or teaching or demonstrating in front of a group or audience. This second form usually takes a great deal of mental and creative preparation, planning, and nervous energy. Understanding performance in these contexts, and in preparation for investigation of how the sense of presence is experienced for non-performers, I have noted this from my experience:
Getting up in front of people sometimes causes great panic at first, seeing blank, expectant or judgmental faces it is a scary thing, like waiting to be beaten or insulted. Having had experiences of this terror of negative response, one becomes braver to do it again and again, but the fear returns each time. This stage fright often vanishes after a few minutes or lessens once the performance commences, when a new sensation often takes over difficult to describe a sensation like an out-of-body experience. I lose focus on the specifics of what I am doing or presenting, or stop thinking and space and time slow down. I just allow the performance to take over my body and actions. I am in the performance, but I am floating in the mix of watching the audience, feeling the music or words, or my body (in dance), or whatever I am doing, takes over. It is almost as if I have also become an audience member, witnessing the overall experience and not fully in my body. It is like a split consciousness not thinking or dwelling on specifics, just being within it. I occasionally even disconnect from the activity and place, filled with thoughts or observations that appear unrelated. It is like I am two beings/people: one in the performance actively (like a puppet), on beat, message, place, engaged with the performance, and another watching the audience watching me, studying their faces, clothing, drinks, interaction with each other, the room, etc, with an inner voice making comments and notes about the moment and what is happening. My body feels split also, as though I am there in front of them, but also in a trance-like state, subtly removed from it as well. My body takes over and knows what to do, taking over while my mind is multi-tasking. When this happens the audience can see or shares it too it is magic, or a spell, or a charisma that takes over the performer, like my consciousness is actually touching the audience emotionally, triggering the intensity.

http://www.blasttheory.co.uk/bt/work_kidnap.html (accessed March 31, 2010). See also this 2005 Guardian article on another artist, Brock Enright, who charges clients $1,500 to abduct them (online), available at: http://www.guardian.co.uk/artanddesign/2005/sep/20/art (accessed March 31, 2010).

81

So am I present in that state, place or time? It feels like a heightened state of presence that involves intermittent absences to reengage and maintain constant presence, with the double-awareness: one in the body acting or performing and responding to the other musicians, dancers, the music, or the computer, or students, or questions, or whatever else is happening in the performance/presentation space. I am also at some meta-level of consciousness, observing it all and feeling a sensation of removal or separation from my body. Strangely, often the more removed I am the better the performance is. So it begs the question of what it means to be in your body or out of it during performance? And what is presence here?

This subjective perspective amplifies the argument that the extended body/mind is a key instance of presence, or of non-corporeal embodiment. Schechner addresses slightly different aspects of presence and liveness in performance, considering the gap between live experience of there and then in live theatre on the one hand, and in the non-live event or asynchronous event on the other hand (whether or not it is recorded and thus preserved as a replayable event):
Theatre, to be effective, must maintain its double or incomplete presence, as a here-and-now performance of there-and-then events. The gap between here and now and there and then allows an audience to contemplate the action, and to entertain alternatives. (Schechner, 1988, 2003: 190, his emphasis)

In this way, he is referring to differing timeframes. The audience engages and understands the disconnect between live and non-live events. Not totally convinced by the argument in Liveness, I went back to Auslanders earlier work, Presence and Resistance (1992), to investigate his views on performance and presence, and found it more relevant. On presence and the audience, he explains,
[...] presence usually refers either to the relationship between actor and audience [] the actor's psychophysical attractiveness to the audience, a concept related to that of charisma. Concepts of presence are grounded in notions of actorly representation; the actor's presence is often thought to derive from her embodiment of, or even possession by, the character defined in a play text [... in] the experimental theater of the 1960s [...] the presence of the actor [was] as one living human being before others [] spiritually and psychologically liberating in itself [] (1992: 37)

Auslanders definition of presence relates to intensity or the spell cast between the audience and the performer, mentioned in my experience, and perhaps is what captivates audiences or temporarily transforms them in the way Schechner describes. To focus on this relationship between the audience and performer and the implicit power dynamics, Auslander adds, quoting Vera Frenkel, that it could be defined in a psychoanalytical sense:
[...] charisma as projection ... by which we attribute to others, especially a leader, entertainer or artist the secret images within ourselves. (1992: 43)

He adds that we are moving away from:

82

[...] postmodernist performance [which] seeks to escape presence by working against mutual projection between audience and performer and toward 'a non-charismatic understanding [] (Auslander, 1992: 43)

Auslander continues that performers no longer wish to be seen as other or put in the implied power relation with the audience (1992: 43). Following this, the goal of MINDtouch was to break down barriers and the power dynamics between the trained actor or expert with her charismatic otherness and the participants; to empower participants as creative collaborators, as co-creators, and not intimidate them. One might also argue that all people are potential performers and creative beings so these distinctions need to be blurred. To segue into a discussion on presence in performance, Auslander made an early attempt to contextualise technology in performance art. He states that by incorporating technology into her performance, Laurie Anderson was also incorporating the epistemology of film. In this way, the performance escaped presence through itself in a paradoxical relation to mediation, because the mediation was both a phenomenon of and a reaction to the performance. (1992: 49) Compelling as it suggests that the presence of the performer was then missing or removed by escaping. However, it has become a very common feature in modern performance to have film or video projection in performance and I do not think the audience would say that the performers presence is missing or has escaped through the projection. Auslander developed this position further although less tentatively, in Liveness, where he added liveness to the mix, although the concept of presence is escaping now through his contentious position on mediation. On this he states:
I have also suggested that performance art's most defining characteristic, its presence or 'liveness' [] is on the verge of being subsumed within a cultural flow that renders the distinction between live and mediated performance virtually irrelevant [ and] the relation of any individual to the medium itself is one of alterity, regardless of whether or not she or he agrees with the message being conveyed. (1992: 79)

Here, Auslander highlights that the disconnect is between the audience and the medium/performance itself, whereas in later discussions he argues that the liveness aspect of presence versus mediation is the issue. So, as with the complexity and intertwining of the perceived duality of mind and body, there is a complexity of duality or distinction, as Auslander states, between liveness and the medium of performance. MINDtouch has tried to integrate the two and addresses this distinction by integrating the medium/performance with the participant as audience. From a feminist perspective, Peggy Phelans critical work Unmarked has continually been used as a contrast to Auslanders view on the ephemerality of the live versus the mediated in performance. Phelan emphatically states that [] performance cannot be saved, recorded, documented, or otherwise participate in the circulation of representations of representations: once it does so it becomes something other than performance (1993: 146). And yet, she cheapens the experience of viewing recorded versions of performances, negating the emotional, sensory reception felt by viewers

83

who identify with the performance emotionally, whether live or recorded. If performance has the power to trigger emotional intensity, as Massumi posits in Parables for the Virtual: Movement, Affect, Sensation, then does it matter if it is stimulated by a live or recorded version (consider how a voicemail message can equally trigger an emotional response)? It can be said that a live conversation on the phone or the viewing of a film, can trigger emotional intensity as easily as a live conversation/play/dance or a live art piece. However, in-person experience allows for a more multi-level exchange that is not possible in the mediated form (i.e. no eye contact or in a film, no exchange at all). This exchange is the missing ephemeral piece that Phelan does not even mention, yet this intensity and emotional experience is key for most interactive and performance art, as well as for effective communication. Phelans intent is to politically empower the visible and real performance, and in her opening chapter she states:
By locating a subject in what cannot be reproduced within the ideology of the visible, I am attempting to revalue a belief in subjectivity and identity when it is not visibly representable. (1993: 1)

Or put another way, since she sees performance as ephemeral, it is living only in the memory of the audience, because it can never be truly reproduced and nor can the experience of it be completely captured or reembodied by media. So she wants us to revalue the real of that moment. Phelan affirms Judith Butlers point that it is the before and after that define the real, and that,
[] representation becomes a moment of the reproduction and consolidation of the real. ([Butler]: 106) The real is read through the representation, and representation is read through the real [] (Phelan, 1993: 2)

So, in opposition to more common associations of the real as being something tangible, to Phelan the real is only this exact moment. She also speaks of the impossibility of materiality and the act of vanishing, loss and even corporeal death. She tries to convince us, by interpreting science and Lacan, that there is a disconnect between the seer and the seen, insisting that the real is immaterial or only in our consciousness or memory. She is adamant that representation in any form is not real, nor can it ever be. The real is invisible (though this appears to be an oxymoron). Phelan adds that, Performance is the art form which most fully understands the generative possibilities of disappearance, poised forever on the threshold of the present, performance enacts the productive appeal of the nonreproductive (1993: 27). She says that we are never complete or here, so perhaps that is why the real is invisible or non-existent. The real can only be experienced in the moment and is elusive to capture or record. Thus, the performer and the viewer experience differently, and there is no such quality as real. This brings to the discussion the concepts of becoming and potential, delved into more deeply in Part II. The heart of Phelans argument is the supremacy of performance and the impossibility of recording or documenting it, since it lives only in the experience and memory of the audience:
[] the real through the presence of living bodies [] there are no left-overs, the gazing

84

spectator must try to take everything in. Without a copy, live performance plunges into visibility in a maniacally charged present and disappears into memory, into the realm of invisibility and the unconscious where it eludes regulation and control. (1993: 148)

In relation to performance liveness and becoming, Phelan says:

Performances only life is in the present [] Performances only being [] becomes itself through disappearance [] Performance occurs over a time which will not be repeated. It can be performed again, but this repetition itself marks it as different. The document of a performance is only a spur to memory, an encouragement of memory to become present. (1993: 146)

So these points speak to the performance imperative, You had to be there. A realisation from my research [76], as with other aspects of performance, relationships and communication, is that the unique exchange of the physicality and temporality of humanness can only take place once, in its own particular way. All of the unexpected, improvised nuances between two (or more) individuals, each with their distinctive identities, life history, gender and culturally inscribed characteristics, can only be experienced in the time and manner that they are delivered. So ideally you have to be there, in-body, in-person, to experience all these unique, ephemeral and fleeting qualities [77]. However, one could argue that if the performance is telematic or networked, then being-there changes to being online. As such, this state of being is the liveness. If the time and modality of presence requirements are there, in-body or embodying the technology (i.e. using the technology with the body), then one is extending the body through the network, and this is being there and therefore being present. From my experience as a performer and as a viewer, it can be personally affirmed that the subjectivity of the viewer their experience, moods, expectations, physical comfort (or lack of same) at the moment of the performance, not to mention that of the performer(s), all play into the irreproducibility of the performance. This performance research has focused on creating a more personal and private performance, without an actual spectator only with interactors. Yet each performance is still subject to so many factors that it can never be duplicated. This ephemerality, now-ness, also segues into the discussion of liveness and presence, with renewed focus. I would still argue that this effect or quality of disappearance can be transmitted virtually, if not reproduced through any form or media. And this is what is compelling about trying to embody technology the effort of trying to communicate this essence of performance or presence through a network. If this is possible, perhaps it becomes a third modality, between live and mediated. Phelans overriding point is that performance eludes mass production, technological mediation and economic consumption, and that in its ephemerality goes beyond language, designed only for a defined audience within a confined space and time, leaving no visible trace (1993: 148). In a related sense, Schechner would concur that aesthetic performance (and other performance types) has a transformative effect. Phelan would argue that this effect is invisible, but I would suggest it is not. I
76 As will be discussed further in Part II on presence. 77 This is discussed in more detail in the theory section in Part II.

85

would suggest that performance, if truly transformative (effective and affective), changes a person inside, and potentially externally (visibly) in terms of their direction or choices in life and views perhaps as only a transformative event can, even if infinitesimally. As this discussion proposes, the superiority of the live, here-body [78], in-person corporeality for performance, as well as in other human relationships or communication contexts, is convincing and is supported by neurological studies (discussed herein) and personal experience. This art research can be seen as a means to simulate, stand in, temporarily substitute, or half fulfil this ultimate fleshy interaction. It can be seen as a hybrid interface for people when apart, over distance, during separation, to mind the gap or space between. It has become increasingly necessary to investigate alternatives to face-toface interaction, in the late capitalist world, due to Climate Change, volcanic ash and travel limitations or regulations that force us to stay home and travel less. To maintain global business interactions, as well as community and personal relations, we need better substitutes and nearly real facsimiles. Alternatively, we should radically restructure the way we live and work to travel less, to live closer to where we work, to be closer to our families and friends and/or communities of interest. Both of these solutions are sure to emerge in coming years. A political stance might be to state that this and future work are efforts to provide more real distance connections. To do this, techniques and tools become part of the solution participatory art can act as a means both to encourage a reduction in travel and to facilitate more felt interaction across distance. This chapter has been an overview of participatory performance and the issues around performing presence and liveness in performance, as well as key voices in the discussion, and how they are relevant to the research. This can be seen as the foundation practice for the other media art practices layered into this project.

78 A term Don Idhe and Merleau-Ponty use, discussed in Part II.

86

4/ video art + live cinema performance

This chapter discusses the background and rationale that has led to combining the different media practices and technologies in this project. It covers my video art background that led to this research and the definition and relationship of live cinema to the work. It demonstrates the connections in my mind, and the relationship of each practice and technology to each other in the external world, stemming from earlier practice and heading into this research. It is important to contextualise the journey that led to this point, especially in video art and performance video practice, and particularly within the current performance research project.

4.1/ a video art journey

I have been an informal student of cinema from pre-teenage years, watching late-night art-house films on TV, and going to the last drive-in cinemas in the 1970s/early 1980s (with the speaker box on a blanket on the ground in a car space). Later, not able to afford to attend film school, I sat in on film history classes and took summer video production classes, attending every film festival during and after undergraduate studies in Sociology and Dance. I had further video production and new media training, and dreamed of making films on video. Still later, during Masters studies, an opportunity to be a teaching assistant for a film history class came up, so I zealously seized on it and studied for it. Video production and video art appealed to me, as a cheaper and more accessible option than film, and the video art philosophy seeped into my thinking on media production, politically and artistically. More akin to and aligned with performance art [79], video art in Vancouver started as a means of documenting performance art and became firmly embedded in the gay and lesbian scene, especially through the works of Vancouver video artist Paul Wong. Wong was a founder of the video movement in Vancouver and the artist-run Video In Studios production centre (now called VIVO [80]). Eventually, video art evolved into its own performative form. There has always been a deeply political association between video production and poverty in Vancouver, which has been the politic of Video In since the 1970s the mandate of Video In was to enable artists to make video productions comparatively inexpensively, in exchange for volunteer hours. Video In had a decidedly political position of empowering those experiencing poverty and a sense of otherness (particularly feminists [81], the gay/lesbian/transgender community, and various ethnic minorities) by involving them in alternative media art. Video In became a Mecca for those who wanted their stories made but could not break into
79 UK Video Art History is documented online at http://ukvideoart.tripod.com/ (Accessed on May 28th, 2010) and a complete video art history in the western world is discussed in Rees A History of Experimental Film and Video (1999). 80 More on VIVO can be found on their website available online at http://videoinstudios.com/aboutus.php (accessed on May 28th, 2010). 81 More on feminism, film and video in Vancouver in the 1960s and 1970s is available online at http://front.bc.ca/research/texts/8 (accessed on May 28th, 2010).

87

the film industry. This is where my media practice started [82] and where I explored and reflected these politics in my work, either directly or indirectly, as a way to address that marginalisation. Film was for white, well off, mostly male creatives, and video for everyone else who wanted to make their work. Eventually I had the desire to be a documentary filmmaker and tackle the competitive world of film and TV. However, after some effort, numerous marketing sessions and the 1999 Banff Television Festival, I discovered that searching for major financing through the pitching process could go on fruitlessly for years. This was not how I wanted to explore the medium, or media and my creativity in general. Just as I abandoned dancing after I envisioned a life of poverty in low-paid jobs, getting nowhere except exhausted, I felt this too was not my path. I longed for more immediate access to the resources to enable production and creative process, with a direct connection to audiences. Yet, without the finances (and investing in a music career simultaneously), I instead explored the new frontier of the Internet. This was a time (circa 2000) when online video streaming was just beginning. Producing videos online and learning more about the online creative world looked like a path that could eliminate waiting for financing, and losing control of the creative process. The potential of the web, over and above video, for communication and creativity appeared fantastic. However, this was a bit too soon (or late depending on ones perspective), as neither web audience nor bandwidth were as robust for video streaming as they are today. Just as I was diving in, the dotcom bomb hit North America at the end of 2000 and all of the intelligent, unique, creative Internet projects and entertainment initiatives [83] almost entirely died within the year (2000). All the risky investment into inventive ideas and creative work went elsewhere, since many could not find adequate business models to support their creative production. Thus, online creativity died down for a time, and so I returned to study digital media production, and continued on to postgraduate studies to dive more deeply into digital media and interactive art. This enabled me to use my video skills in new ways to communicate and explore innovative ideas with media and moving images.

4.2/ live cinema

One of my many fascinations, since learning to produce video in the mid-1990s, has been exploring live performance video both as an artist/producer and as a curator. This interest was piqued when I held the position of lead curator for the New Forms Festival (2002 to 2004) [84], featuring electronic and interactive art and performance. This was an exciting opportunity to research the field first-hand and
82 What is written here is from my own experience. However, a history of Video Art in Canada and Vancouver to support this experience is available online at http://www.thecanadianencyclopedia.com/index.cfm?PgNm=TCE&Params=A1ARTA0008366 (Accessed March 20, 2010). 83 See more on the 2000 dotcom bomb in Casey Kait and Stephen Weiss 2001 book Digital Hustlers: Living Large and Falling Hard in Silicon Alley, NYC, New York: HarperBusiness; and the movie Startup.com, available online at http://www.imdb.com/title/tt0256408/ (Accessed March 20, 2010). 84 The New Forms Festival is also in Vancouver, http://www.newformsfestival.com (Accessed March 20, 2010). BC, Canada. More information is available online at

88

become exposed to, among other media arts, the developing art practice of the live performance video. Since then, while not becoming a VJ or Live Cinema practitioner, I have explored the software and hardware tools of the practice, while learning about the form from other artists and curators. There is now a great range and variety of technology options, such as Module8, ArKaos, Resolume Avenue, FlowMotion, Union, CoGe, C[]NTR[]L, Isadora. Some artists make their own and sell them as is the case with Module8 in visual programming environments like Max/MSP/Jitter or PureData, Quartz Composer(CoGe) and OpenFrameworks [85]. Live cinema or VJing as a form of video art, has its roots in the rave party culture of the late 1980s and early 1990s. It integrates video, computer programming and generative, new media tools for sound, images and performance. VJing, also called Audio/visual and Live Cinema performance bridges: electronic music, media installation, animation, motion graphics, experimental film and video art in a live context. It is now slowly becoming recognised as a media art form and performance practice, outside of the party context, and is moving into the streets, galleries and performance venues, adding sitespecificity to the live component of audio/video performance. RES magazine [86] featured VJ artists in the January/February 2006 issue, focusing on how they were moving further into the art sphere, gaining better exposure, recognition and respect, with art institutions now seeing that their work is an extension of installation and video art. So the art establishment is noticing them more as they expand their range of works, tools and practices beyond performance and into other digital art manifestations. This article also discussed how Live Cinema artists are reinventing themselves, exploring new ways to present themselves to the world beyond the late-night scene (although some still make the most money and get the most exposure there). The performance video duo NomIg Collective, or Stephanie McKay and Ed Jordon, curated an event at the New Forms Festival in 2003, and it really inspired my own deeper exploration into Live Cinema as a refined visual form (their excellent selection of performing artists was eye-opening). They state their definition and philosophy of live cinema:
[... Audio/visual performance is] the creation of works where the audio and video components are composed and performed together with an awareness of each other's inherent compositional characteristics. [... Live cinema] requires that neither the audio nor the video serve the role of accompaniment, but that they work together to form a synergy of audio and video [] (McKay and Jordan, 2006, online)

85 I now have an extensive collection of video clips and visual material for my Live Cinema practice. 86 In the Jan/Feb 2006 issue of RES magazine www.res.com no longer available online (accessed July 2007)

89

Image 7 2004 - still from PDX_01 by NomIg (Stephanie McKay and Ed Jordan) live performance [87]

The acronym VJ or term VJing has become a less attractive label for practitioners, as it carries the baggage of associations with drug-infused parties, distracting from the art form itself. Thus, many artists now call it live cinema, audio-visual performance or performance video. There have been some stunning works from the Montreal artist team skoltz_kolgen (Herman Kolgen and Dominique Skoltz) [88], Vancouver filmmaker Velcro Ripper, the NomIg Collective and French Canadian film artist Jean Pich, as well as US artist Sue Costible, not to mention a great many fantastic British artists such as Hexstatic, and other global artists/performers, increasing in numbers and artistry yearly. The works I have seen by these artists were primarily while in my role with the New Forms Festival, although I have seen several other similar events in Vancouver, Montreal and London before and since.

87 PDX_01, NomIg Collective (2004). Find their work online at www.nomig.net and at http://www.freewaves.org/artists/nomig/ their site says of their work: NomIg.s artistic interests lie in applying compositional techniques, usually reserved for music, to digital video. [] Together the duo are committed to approaching audiovisual art in terms of what can be learned from the intersections between musical and cinematic points of view on both theoretical and practical levels. (NomIg, 2004, online) 88 Find skoltz_kolgens work online at http://www.skoltzkolgen.com/ and http://www.youtube.com/watch?v=isX7IvevM4Y&feature=related. They say of their work, skoltz_kolgen is a plurimedia work cell based in montreal, comprising dominique [t] skoltz and herman w kolgen. Rigorous and raucous creators, their artistic pursuits plumb the integral linkages between sound and image. liberated by digital media they simultaneously assume numerous positions, inhabiting a space between film, photography, audio art, and installation. Architects of worlds, skoltz_kolgen penetrates the ephemeral skin between solid matter and the unsubstantiated, the intimate and the objective, their work conjures bewitched worlds that gestate betwixt accident and intent. (Skoltz and Kolgen, 2008: online)

90

Image 8 20042008, still from audiovisual performance [SILENT ROOM] film-pome, skoltz_kolgen

According to the Transmediale Festival press release and website in 2005:

[...] the term [...] Live Cinema today stands for the simultaneous creation of sound and image in real time by sonic and visual artists who collaborate on equal terms and with elaborate concepts [... presenting] a much broader conception of cinematographic space [...] The term 'Cinema' is [...] embracing all forms of configuring moving images, beginning with the animation of painted or synthetic images. (Makela, 2006: 22)

This definition clearly carves out Live Cinema as an artform and not merely party eye-candy, as some still see it. While the audio aspect of VJing is decidedly not used in MINDtouch, I nonetheless recognise how intrinsic music and sound are in the live performance context. There are still very few academic voices on this art practice. The few that there are, such as Jaeger (2005), DFUSE (Faulkner as M/D-Fuse, 2006) and America (2005/7), have not been academically published. Jaeger attempts a critical discourse about VJ practice and culture, providing a history of VJing and video performance, describing what the practice involves and what it is not, while placing it in the larger art and cinematic context. In this way, he creates an initial framework to critique video performance art. Yet, from discussions with live cinema artists, it has been suggested that they are still not respected or considered a legitimate artistic force within interactive and media arts, not to mention the mainstream art world. Attempting to change this are a British academic artist group called VJ Theory, who generated a collection of articles from contributors defining and discussing Live Cinema practice and aesthetics [89]. The founders of VJ Theory have collected these primarily through their website, editing and self-publishing a book that features interviews with prominent global Live Cinema artists and VJs, as well as recent research on visuals performance in the field. These articles have begun to shed light on the art process and practice, as well as the dynamic nature of this visual performance form, both for artists and for audiences [90]. Also acting as an entry point to expose and

89 VJ Theory is based in Falmouth, England. More information is available online at www.vjtheory.org (Accessed January, 2009). 90 It should be noted that D-Fuse and ondotzero representatives were recently speakers and performers at the prominent digital design/interactive art exhibition and conference DECODE, at the Victoria and Albert Museum in London. This exhibition ran from December 2009 to April 2010 (conference in February 2010). This inclusion of live cinema artists proved to legitimise their practice and the work of other such artists, as part of digital art (a relatively new form to be legitimised in the art world itself), and within the high art context, for nearly the first time.

91

focus on this form more seriously are the arts organisations Rhizome [91] (US), the British Film Institute [92] (UK), and the Canadian online journal Vague Terrain. Well-known Finnish artist Mia Makela is more precise in distinguishing VJing from Live Cinema, as do others in Vague Terrains issue 9, Rise of the VJ (2008). She points out that VJing is the act of mixing material pre-created by others, though sometimes by the VJ themselves. Live Cinema, on the other hand, is the act of creating visual material and experiences live, in real-time, somewhat spontaneously, and using custom software or specialised tools, materials and hardware to do so (such as Sue Costible [93], who uses film projectors, film strips and other materials to create something on-the-fly).

Images 9 and 10 2007 Sue Costabile still from audiovisual performance

In her thesis on Live Cinema, Makela explains:

LIVE CINEMA describes work which is in essence artistic, to make a separation from VJing, which is basically visual DJing. DJs dont produce their own material, they mix music, the same way as VJs mix already existing material. This does not mean that VJs would not also create their video-clips, but there are many who consider that producing material is not necessary for a VJ [] content is not as important as its usability in a mix. The act of mixing and selecting becomes the work of a VJ. (2006: 23)

So in this context and while it may be obvious to some, the term artist is meant to define a person who creates his/her own visual and performance content and constructs new works from them, in a unique way to express his/her own creative voice. This contrasts with the VJ, who presents other peoples material or creates light, background entertainment. However, artists borrow and use other peoples material at times, and in the age of remixing and appropriation it is a borrowers unique
91 From the editorial on the Montreal digital music festival Mutek 2009, where a great number of VJs and Live Cinema artists perform each year. More information is available online at http://rhizome.org/editorial/2668#more (Accessed January, 2009). 92 The BFI in 2009 and 2010 has been in collaboration with the showcasing organisation onedotzero to host Live Cinema performance events called Dark Fibre with performance group Hexstatic http://www.onedotzero.com/ and https://www.bfi.org.uk/whatson/bfi_southbank/events/dark_fibre_0 (Accessed January, 2009). 93 Sue Costabiles work available online at http://www.sue-c.net/video.html and http://www.23five.org/archives/suecostabile.html (Accessed March 20, 2010). Her site says. Employing a variety of digital tools to create an experimental animation "instrument," Costabile synthesizes cinema from photographs, drawings, watercolors, hand-made papers, fabrics and miniature interactive lighting effects. Dark, moody, textural, and physical, her live films inherit equally from the kinetic languages of Stan Brakhage's abstract cinema and Nicolas Schffer's lumodynamic sculptures. (Costabile, 2007)

92

approach and style that often defines a work as art. This can open up a whole other debate on what is art, who can be defined as an artist, etc., which will not be discussed here, as it is outside of the scope of this thesis.

Images 11 and 12 2005-2006 stills from audiovisual performance by Sara Kolster and Derek Holzer [94]

Makela frames a Live Cinema performance staging and set up by stating:

What is needed in order to perform visuals in realtime? ... SPACE, TIME, PERFORMANCE, PUBLIC and PROJECTION, as these elements are present in all live cinema performances. (2006: 6, her emphasis)

She further discusses each of these elements of SPACE, TIME, PERFORMANCE, PUBLIC and PROJECTION in detail, but she also adds to the discussion on the process of the performance design, by stating that the construction uses such building blocks as video clips or:
[] in the case of abstract imagery, the structure can be thought of as variations of rhythms, movements, colors and shapes in order to arouse (emotional) responses in the spectators. Live cinema has more "open architecture" than cinema, so there are more possibilities to build the performance. (Makela, 2006: 54)

There is a relationship between Live Cinema and MINDtouch in terms of liveness and presence in performance and this is discussed further in Part II. However, important here is addressing the performance in terms of liveness and presence and involves the realness of the experience for the audience, Makela says:
When we see "Live from New York" flashing on the TV screen, we know that the image is 'real', this is what is really happening. Normally the effect of 'realness' or 'liveness' is enforced by certain "reality effects", like a hand-held camera or even technical problems which makes us recognize the output as more "real" than the carefully chosen, edited and manipulated image
94 ResonanCity 2005-2006 was a Sara Kolster and Derek Holzer performance. Their work is available online at http://www.sarako.net/performance.html (Accessed January, 2009). Their site says of their work: Both artists find inspiration in the history of experimental cinema and electroacoustic music, as well as in contemporary video and microsound practices, and a variety of live sources such as photographic film and found objects are used to generate the visions and sounds [] (Kolster and Holzer, 2006: online).

93

material normally transmitted [] Anything can happen in a live situation and that is what keeps it interesting. Live is connected to real life, it is something happening in front of us in natural time. (2006: 36, my emphasis)

This is crucial to put into perspective my experiences and frustrations in setting up technology in preparation for a live networked or telematic performance for MINDtouch. Similarly, from my performance experience, much of the sense and expectation from the audience of realness or liveness is because the live performance features the unexpected. The liveness, as Phelan might suggest, when a performer forgets their move or line or musical notes, or decides spontaneously to vary the piece, makes it a unique event. Makela points out that in live cinema or telematics, the technology can fail or aspects of it might not deliver as desired by the artist/performer. However, audiences want to see the process or mechanics of performance, with all its flaws, and this adds to their experience. This foreshadows a discussion in the next chapter on the imperfections of mobile video, in terms of resolution and pixilation, which suggests it is often the very imperfections which give the performance (or image) its life, a humanness, a realness and liveness that audiences find special and seek out. Overall, what is compelling is the accessibility and DIY nature of the Live Cinema community: developing their own software tools along, not regulated by the same tastemakers and gatekeepers of other forms of visual art (unless on the gallery or academic circuit). However, what has been most of value in exploring the Live Cinema form, is witnessing its way of speaking visually. It is how performers use images, often spontaneously like a conversation, responding to the audience, unlike traditional cinema. Live Cinema is like a play of images that dance in your mind when in a semi-awake, hypnogogic state, mixing with the imagery from half-remembered dreams, while surfacing conscious thoughts poke through to the waking mind. This notion is what motivates the MINDtouch performance project. I have very vivid dreams, full of colour and passion, very visceral and full of design and creativity, often featuring wildly designed shoes, clothes or architecture. So these dreams can be seen as embodied emotional expressions that I often wish could be shared directly with the world or recorded, as in Until the End of the World (Wenders, 1990), mentioned earlier. This is how this participatory performance project was born and inspired. To wrap up, this chapter has looked at my path to video art and live cinema, and how both came into my practice to help shape this research process. This is important, as it shows how this work could not be created in the same way by others, due to my embedded or tacit knowledge, making me uniquely qualified to develop the approach put forth here, as my unique contribution to knowledge.

94

Part II: theory into praxis

95

5/ mobile video aesthetics, non-verbal / non-linguistic

As discussed in the last chapter, the main concept that MINDtouch is based around is a vision of sharing dream imagery and embodied sensations telepathically, and it attempts to simulate that experience through mobile media, wearable devices and participatory, collaborative, networked activities [95]. The aim of this chapter is to discuss: 1) a developing theory for aesthetics of mobile media; 2) how video can be used as a new form of mobile communication; and 3) how database and generative video informs and is used in this project. These are grouped together as they deal both with the making and using of mobile video in this performance work. Within the context of this chapter and thesis, the term aesthetics refers to a visual or performative expression or artwork that seeks to instill an affective, emotional, sensory, transformational or pleasurable experience, or creates some form of meaning for the participant and/or viewer. It does not necessarily subscribe to a specific sense, use or direct history of beauty in its use, but rather suggests that an audience, participant or viewer has a meaningful (to them) response to the work. This use of aesthetics is focused especially within the context of video art and performance discussed here, and influenced by theorists discussed within the introduction and elsewhere, such as Schechner, Shusterman, Christiansen and Bourriaud [96].

5.1/ the emergent aesthetics of mobile video

Theoretically, and in practice, works and research in the field of mobile performance media have been few but growing, many being locative and mixed reality [97] media games. The approach taken by many of these locative media projects is to focus on urban space, on the environmental and ecological, as well as on geographical activities using GPS or RFID in tag games. As mentioned in the introduction and in Chapter 3, most of these projects do not reference or ground their work in traditional performance practices, studies or history, but mainly come from a gaming or HCI perspective. The focus for this research has been the emotive, affective perception or experience, and individualised, visceral experiences of participants, rather than the psycho-geographical or situationist approach focused on the urban condition or social geography, so popular in recent years.

95 MINDtouch workshop video edits are available online at www.swampgirl67.net in Appendix I in the workshop section. 96 Schechner (1988, 2003: 194); Shusterman (2000: 15-34); Christiansen (2006: 203); Bourriaud (1998, 2002: 84-102); and Benjamin (1929). There are hundreds of books and articles on art and aesthetics, so the definition, philosophy or history of aesthetics will not be discussed here, and while video and film aesthetics are most relevant here, this is also not the thrust of this argument, so it has only been explored on a cursory level and comes more from training in video and new media. Shusterman in particular has been referenced from his theory of the Philosophy of Aesthetics, Performance and Somatics. 97 Locative game projects now call themselves mixed reality, mixing digital gaming interfaces with live performative or treasure-hunting type tasks and game play, using mobile devices.

96

Over the years of electronic, digital and networked art development and practice, new media performance practices using them have evolved into four areas, as identified by various media artists and theorists [98], some in combination [99]. They are as follows, with the first three addressed in some way in this performance project (as can be read throughout this thesis): (1) Telematic performance and events: telematics connect people to people or people to objects through a network, such as live video steams of performances or collaborations (or even performance tele-conferencing) across geographic distances (the focus of Turbulence arts organisations Networked Performance blog); (2) Locative media or psycho-geography: locative media provides a location-aware connection through GPS or mobile phones (such as Blast Theorys work, discussed earlier); (3) Wearable projects: wearables extend the body's senses through technological sensor devices (such as the whisper[s] and Philips projects mentioned herein); (4) Active objects and responsive environments: smart environments enable spaces and objects to respond to environmental changes of state, generated by people within them [100]. The current notion of mobile media involves images, sound or videos created by, existing on, or received through mobile phones or other portable devices [101]. In MINDtouch, low quality and low-resolution video imagery on mobile phones has been valued for its immediacy, poor image quality, and pixilated imperfection its own unique aesthetic. The mobile phone is also valued in this project for its innate encouragement of spontaneity, like the speed of thought in the mind, that enables the rewriting, superimposing and remixing of ideas, flashes and clashes of images and emotion, layering of meaning and stream of consciousness, as well as a simulation of telepathy and collective, if chaotic, intelligence. For those of us using this new movingimage medium, the aesthetic of mobile video, as with the early video cameras, is one of imperfection of image quality. However, in exchange for imperfection we are given an immediacy and a form of personal empowerment through the simplicity of the device the pixilated resolution holds its own beauty and power. Due to its portability and close relationship to the body, the mobile videophone has an inherent embodied expressivity. The PhD dissertation As We May Feel: Interpreting The Cultural of Emerging Personal Affective Mobile Media (2006) by Martin Sonderlev Christensen, is one of the few, if not the only, theoretical discussion

98 See the listserve topic: networked performance by various authors (2005a), from soft_skinned_space -empyre- (Online). 99 See the turbulance.org blog topic: network-enabled performance by various authors (2005b), networked performance blog (Online). 100 An example of this can be found at the website of the company Experia, who claim to be: [] designers, manufacturers and installers of sensory equipment, sensory rooms and multi-sensory rooms. Available online at http://www.experia-innovations.co.uk/ (Accessed May30, 2010). 101 This includes mp3 players, portable digital assistants (Blackberry, Palm Pilot), iPods, and other such devices, sometimes using GPS or RFID components as well.

97

thoroughly exploring the affective qualities of mobile phones through their multimedia capabilities. He examines the intimate, social uses that have been applied to them and the embodied, personal expression possible when using mobile devices. For Christensen, affective expression has been integrated into the device design because of mobile media devices and how users initially used them affectively, influencing their design, but they:
[] need in greater extent [sic] to encompass the notion of affect as a design motif. This 'culture of affect' has arrived not because technologies have catered for this in a user-friendly or hassle-free manner, but because users actively and critically have engaged with these technologies, while carrying these into the very centre of their personal everyday life activities. (2006: 10, his emphasis)

Christensen traces the transition of computing devices from static scientific, academic and business machines to personal, affective, dynamic accessories. He claims that with the emergence of portable devices, initially in the form of the Sony Walkman, the culture of affect began. His position is unique because he collates many ideas and phenomena from various, often disconnected domains, into one place. He specifically focuses on the relevance of the multimedia capabilities of recent phones and their significance in changing from being merely a telephone, to becoming a new media creation and affective sharing device. He says that in the emergence and adoption of the mobile phone from a business device to a personal device used in everyday life, it has become part of our lived experience and personal identity. He adds that,
[] mobile image technologies are productive in sustaining an embodied relation to the world through capturing in local contexts and sharing online experiences of mundane everyday life relations. (2006: 10)

Christensen ties many elements together to bolster the argument that mobile devices are affective and social, and one of the more important new media tools and technological developments today, claiming:
[...] the mobile image device, which allows me to take pictures and send them to others elicits a particular way of being in the world [] to experience the mundane world differently through the device and the activities [...] People take pictures of their everyday encounters [] (2006: 6-7) and further: [] mobile device technologies are reinserting the real world experiences into new media context [...] offer[ing] locative and corporeal embodiment of technology [] that promotes at the same time upholds different relations such as an embodied being-in-the-world and hermeneutical reading-the-world in one [] (2006: 11)

In these passages, Christensen highlights the view that people are becoming more engaged or reengaged with the world and others affectively; they are looking at the world around them more closely, with the intent of sharing their embodied experience of it. We are seeing the mundane more closely and richly, with or through the lens of our mobile phones (pun intended). I too have found myself engaging

98

with the phone, asking myself: what can I capture (looking more carefully), what can I do with this moment/image? Who shall I send it to or where shall I put it, on Flickr or Facebook? Who am I sharing it with? Am I doing this for me or to find a way to connect with others? Christensen articulates my own experiences, investigations, and observations of participants using these behaviours. Thus, mobile images are often experienced as personal, intimate and private; once sent, the immediacy of them feels like a giving or blowing a kiss to another through the network or sharing pieces of yourself (depending on the content). This is also my experience. So if a text message is like thought transfer, then an image or video is like sending your sight, your visual experiences, your thoughts and feelings, as well as your unique expression and perspective. The final section of Christensens thesis was inspiring in conceptualising the mobile phone as a window or microscope on life around us - helping us to see ourselves in the world more clearly and communicate/connect this insight outward to others: to find the world and ourselves in it.
Affect then was seen as a way to understand what transpires as technologies become a more active part of the life-world of humans, the central concept being movement, bodily and culturally. (Christensen, 2006: 204, his emphasis)

I mused on the back page of the print copy of his thesis: mobile phone as window or more like microscope to life around us, helps us to see ourselves in the world better and communicate/connect this insight outward to others: to find ourselves in the world. Another mobile video experimenter with similar objectives, Dean Terry, says about his work, a mobile video project from 2005 called mo.vid.1:
[ It] is centered on the idea of remapping private spaces into public ones, of reversing scale, of inverting and rejecting the consumerist idea of quality and its technological expression in ever higher resolutions by exploiting the limits of the devices. (The videos are really pretty poor with all the compression artifacts []) (2005).

This reflection echoes my experience of working with the participants in my video collection workshops, enabling ordinary, non-artists and non-performers to become artists [102] by inverting what is considered quality image construction or filmmaking, using the limitations of the tool and pixilated resolution as an asset, rather than as a hindrance to image-making. It is the very messiness or glitchy quality of the medium that makes it visceral: the tension of imperfection created by the compression and image artefacts lends rawness to the medium and gives it a feeling of realness or liveness, making it special. This echoes the way that a live music concert becomes special when the musicians play the wrong key or have audio issues in front of the audience it is not edited and perfected but

102 This whole project is about bringing art and performance to the non-artist and engaging the public in collaboration and creative interaction, as is discussed throughout the thesis.

99

heard as it comes out. This can add to a concerts uniqueness and ephemeral quality, distinguishing it from any other and making it feel special to the observer who has co-experienced that moment. This again refers to the argument in Chapter 3, to Phelans points on the real and to Makelas arguments on imperfection lending a sense of liveness. I would extend this to mobile video, especially factoring in its affordability and innate performativity, discussed next. Key aspects of mobile video making observed within this research process [103], both with participants and in my personal explorations include these four features: 1. innate performativity, or movement or gestural qualities encouraged by the device size; 2. the portability factor (the ability to watch or shoot anywhere), with the mundane becoming fascinating and easy to capture; 3. the phenomenon that users view the world through the camera vision of the phone screen, rather than their own eyes alone, adding novelty, but also a re-engagement with the world; 4. the innate intimacy of expressivity fostered by the device and predilection for doing close-ups framing only that which is intended to be seen. To exemplify the performativity and intimacy aspects of the mobile recording practice, Terry (see his work later in the chapter) states that the phone encourages one:
[] to project [the] very private space immediately surrounding the body into metaspace. Many of the videos show objects little more than a few centimetres beyond the tiny lens, often some body part, like hands or forearms that obscure an unknown, overexposed background space. Other pieces are gestural performances, recording the movements required when following a line, or when trying to create shapes by moving the camera in certain ways (2005, my emphasis).

Exactly in this way, the participants in my workshops tended to capture their experiences and explore the meta-space of the body as the main focus of the mobile medium. It appears to be a common and instinctive approach one takes to mobile video capture. The device inherently encourages movement, resulting in blurry, abstracted patterns, and a splattering effect through gesture. This intimate approach to mobile video production is facilitated or afforded by the device size itself. Unfortunately, the manufacturers will inevitably be adding more image stabilisation and anti-shake technology to compensate for this, removing much of what makes these media creations special. When participants in the workshops used and worked with the phones, it appeared that their immediate impulse was to wave the device around, as if it was part of their hand, blurring the images intentionally to see the patterns that resulted. Thus, it inspires a playful, gestural or performative exploration, not suggested, but observed. This is intriguing since it is to be expected when using bigger, more professional video cameras (even consumer home-movie camcorders), that shake stabilisation is used and effort made NOT to blur the image. Yet, a new shooting aesthetic may be unfolding, resulting from the very size of the device and how it lends itself to movement and gesture. The performance aspect of this project combining video and mobile phone and working with untrained participants benefits from this
103 See observations in Appendix I in the workshop section for more on this.

100

inherent aspect of the device. From the perspective of aesthetics and intimacy with the device, during the FILMOBILE conference [104] in April 2008, video artist Steve Hawley presented his mobile cinema artwork and discussed how, with the advent of HD High Resolution (and now 3D) video cameras and televisions, the world is now shown to us in a larger than life reality and quality, creating the illusion that we are somehow getting closer to the world, more intimately close to places or things we could never otherwise get close to or see clearly. However, he pointed out that in this way, HD and 3D present more of a spectacle of the world, a world which most of us cannot get close enough to except through these professionals. In this way, it keeps the medium and its presentation firmly in the hands of the expert filmmaker or videographer who have more access and training in the medium. The images and work of the high-resolution medium are usually in the National Geographic style of close-up, telephoto shots of creatures in the Amazon forest or of huge vistas in a faraway, inaccessible landscape. Meanwhile, Hawley suggests that the mobile phone, with its poor, imperfect resolution, pixilated/compressed quality and small aspect ratio, is in some ways, more real [105], in that everyone has access to the medium and can learn it easily, resulting in more imperfect, but more personally meaningful, images. More importantly, it captures what I am doing now, in this moment, and has a personal immediacy that the imperfection of the image actually authenticates, through its lack of production values, or high resolution, clean editing, and large aspect ratios. It is a more intimate and everyday life document, which the ordinary person can relate to, and hence, feels more real to them as it is imbued with memory and emotion. Through its messiness, mobile video at once encourages a more personal, nonexpert, every-person, frank expression, not possible with the HD or 3D video image. What has become evident in this research using mobile phone media features, is that the portability factor enables users to explore the world around them differently as Christensen suggests, with new eyes or to see differently and observe the details of the world more closely. With the ability to watch and shoot anywhere, the mundane becomes compelling: a way of viewing the world through the camera vision or through the lens of the phone. Thus, we are becoming more engaged or re-engaged with the environment, looking at the world around us more closely, and sharing our experience of it. I have witnessed people recording video or taking images of the world spontaneously, as they encounter it, almost randomly, and perhaps seeing differently. The phone becomes a window on the world that has users returning to truly observe the world in order to capture it, and then finding details that they might not have otherwise noticed along the way (Christensen, 2006: 6-7). In this way, not only are they learning how to be amateur digital videographers, but they are also re-engaging and interacting with their surroundings more deeply, in a way they may not otherwise do if merely experiencing a mediated,
104 FILMOBILE is a network project developed at the University of Westminster, bringing together the mobile phone industry, filmmakers and artists working with mobile devices. In April and May 2008, FILMOBILE organised a major international event consisting of a gallery exhibition, cinema screenings and an international conference. This event explored the cultural and economic impact of new mobile technologies and initiated debate between artists, the media and the new mobile industry. The emphasis of the exhibition was on mobile cinema, but there were audio and photographic pieces, as well as MMS and SMS, and animated projects in the exhibition, of which I was a part. Information is available online at http://www.filmobile.net (Accessed March 20, 2010). 105 This sense of real refers to a feeling of being close or true to everyday lived experience.

101

2D sense of the world. The portability factor also encourages a high degree of intimacy one can take ones phone to bed, or to the toilet, or to places where cameras often are not allowed, communicating much more personal, private or hidden imagery than standard video cameras enable, due to the size and ease of concealment of a phone [106]. This encourages complexity and a freedom and uninhibited quality in what one can represent to others, with an immediacy and accessibility that enables participation and intimacy unlike other media devices. This can be seen in the video that participants made, with closeups of areas in the rooms into which other cameras might not fit, or footage of the insides of mouths and such. So here, when intimacy is used it is with reference to a closeness to the body, and a sense of personal and private engagement or exchange [107]. In the MINDtouch workshops, participants were asked to try to find imagery that represented, in an abstract or literal way, how they were feeling, perceiving or sensing their own bodies, in a way that could be communicated to others. This enabled more freedom, and perhaps gave permission to people to explore and express themselves in less censored or guarded ways, and their visuals became highly intimate, interpretive and subjective. What resulted, since most had already been exposed to a vast amount of media daily, was that even without a cinematic language or training, they inherently knew how to express themselves and create imagery, with instinctive framing and composition, using the device [108]. Below are some examples of other projects. While not all come from the same motivation, all have embraced the low-resolution, small-screen aesthetic fully. The work below by Steve Hawley was striking because he both explores the small frame and plays with the frame in new forms of composition. He also examines the aspect of being here now and the possibilities of surreptitious use of the phone. He also takes advantage of the low-resolution pixilation as a painterly nature of the medium. He says the work is:
A 3 minute video, shot on a mobile phone. The stuttering low-fi images, collaged together, look like the very earliest television. (2004:online)

106 To demonstrate this point, at the FILMOBILE Exhibition in 2008 in London (see footnote 104), one of the artists had a set of mobile images and videos that she had recorded in prisons, which would not have been possible to obtain with a regular video or photo camera. 107 This is discussed more deeply in relation to Schiphorsts performance methods and the whisper[s] project (in Chapter 2 of this thesis). 108 With the exception of course of those videos that were just pure movement, but even those more often than not were capturing patterns.

102

Image 13 2004 Steve Hawley still from Speech Marks [109]

The work of video artist Mark Amerika surfaced numerous times during the research. His approach to media art creation can be seen to parallel that employed in this research project, as he has translated the mode of his practice from performance to video to mobile video. Amerikas multi-form digital art piece Mobile Phone Video Art Classics (MPVAC) [110] mixes and reconstitutes various art personas and artworks to create a new narrative, using mobile phone video scenes in an attempt to bring forth spirits of the past as well as hauntological actors of the present. This piece engages in an effort to re-edit art history using mobile phone low-resolution video as its primary mode, and edited using the iMovie tool (Amerika, 2008: online).

Image 14 2008 Mark Amerika still from Mobile Phone Video Art Classics

109 Steve Hawley, Speech Marks (2004), available online at http://www.artdes.mmu.ac.uk/profile/shawley/image/10349 (Accessed March 20, 2010). 110 Mark Amerika, Mobile Phone Video Art Classics (2008), available online at http://www.eaf.asn.au/2008/amerika.html. Artist's website and blog available at www.markamerika.com/ (Accessed May 6, 2010).

103

Mark Amerikas work is important to the medium and genre of video art because of his approach as a prominent VJ. His methods of creating performance video on mobile phones is relevant because he takes advantage of and uses the low-resolution, pixilated quality of mobile video and explores how it can expose and push the painterly qualities of the video image aesthetic to the forefront. He recently produced a mobile phone movie project callled Immobilit. He promotes the project, proclaiming it is,
The world's first feature-length mobile phone art film [] Amerika's Immobilit mashes up the language of foreign films with landscape painting and literary metafiction. The work was composed using an unscripted, improvisational method of acting and the mobile phone images are intentionally shot in an amateurish or DIY [do-it-yourself] style. (2009: online)

Image 15 2009 Mark Amerika Immobilit

An arrogant claim, since I saw a feature-length mobile art film at FILMOBILE in 2008. However, it is exciting that Amerika uses improvisational approaches with mobile video, using a similarly low-tech approach with the remix [111] to my own. His marketing text adds:
Mobile Remixes [] Immobilit director Mark Amerika introduces the concept of remixology. In Amerika's version of art-life, we are all born to remix. What are dreams and active memories if not personally rendered remixes of multi-media source material? (2009: online)

Dean Terrys database approach [112] is one that was adopted by my project initially as a means to assist in storage, retrieval and mixing of mobile videos. It is curious that many of us working with mobile

111 Although I thought of this in 2006, before starting this project. 112 Dean Terry, mo.vid.1 (2005) video paintings, from a camera phone, are available online at www.100lies.com and http://www.deanterry.com/blog/index.php/2005/10/ (Accessed May 6, 2010).

104

art have similar approaches to working with this medium in a mobile video creation/display form. He says of the work:
mo.vid.1 is a mobile video project that updates every time a video is sent from my mobile phone to the gallery, often several times a day. The display changes as new videos are sent from the phone, through wireless networks, and to a computer in the gallery. (Terry, 2005: online)

Image 16 2005 Dean Terry

and: The project is centered on the idea of remapping private spaces into public ones, of reversing scale, of inverting and rejecting the consumerist idea of quality and its technological expression in ever higher resolutions by exploiting the limits of the devices. (2005: online, my emphasis)

This point is key when working with video with participants, as I learned in the video collection workshops, as it allows ordinary people to become artists by inverting what is considered quality and using the limitations and pixilation as an asset or tool, rather than a hindrance. What is fascinating about Terrys project is that even though it is a few years old, he was observing the performative aspects mentioned earlier in this chapter. Very few have written about this gestural element or the innate performative features that were either an afterthought or accidentally afforded by the very nature of the mobile device, but are perfect features for creating mobile video art. He adds on his work:
[] This project is atomistic and performative. The bits (individual 15 second videos) pile up over time, and the composite, the themes, are the paintings collections of idea streams, are captured in development, in process. This is a real time art piece, and so the themes are being worked out over the course of two months. (2005: online)

105

Max Schleser uses mobile filmmaking [113] as an alternative approach to the film and documentary practice. He claims to provide a new definition of documentary while creating a new aesthetic approach to feature-length, non-narrative documentary filmmaking.

Images 17 and 18 2006 Max Schleser Max With a Kaitei

In his own words Schleser describes one of his projects: The city film Max With a Kaitei, which was produced entirely on a mobile phone in Japan in 2006, depicts the new emerging pixel aesthetics [] The experimental feature documentary is juxtaposing the advanced Japanese mobile video format, which is based on the mpeg 4 codec, with the former 3gp compression format. (2009: 2)

His new form of documentary is not only shot on mobile phone, it is more abstract than most common documentary and specifically uses the digital aesthetic and pixilated resolution as a feature of the medium. Giselle Beiguelmans project sometimes always, sometimes never [114] was the first piece of the mobile art form I encountered. It made a huge impression on me and inspired me to make MINDtouch in some ways, especially the collaging/remix aspect (see images below), even though it is totally different in outcomes. She says on her website on the project:
Sometimes Always, where the audience also sh[o]ots images with mobile phones to be deconstructed by the same algorithmic process [ and] consists of an interactive projection, based on generative systems, which allow the audience to sh[o]ots images with cell phones with video cameras and send them via Bluetooth to big screens. On mouse over, the videos fragments in frames can be reorganized by the interactors, following the movements and draws they do with the mouse. (Beiguelman, 2005: online)

113 Max Schleser, Max With a Kaitei (2006), available online at http://www.filmobile.co.uk/ (Accessed May 6, 2010). 114 Giselle Beiguelmans sometimes always, sometimes never (2005), So Paulo, is available online at http://www.desvirtual.com/sometimes/never/index.htm and sometimes always video http://www.desvirtual.com/sometimes/always/video.htm and http://www.desvirtual.com/sometimes/never/video.htm (Accessed May 6, 2010).

106

Image 19 2005 Giselle Beiguelman, screen shot from sometimes never

And:
Sometimes Never is a (de)generative video which is decomposed through the inputs of its interactors. Videographic images, shot with mobile phones in the exhibition space, can be manipulated by keyboards and mouse and the audience edits, in real time, the order of its original frames, their position on the screen as well as introducing colored filters on the new images. When someone leaves the mouse, the original film restarts over the layers built by the interactors. (2005: online)

These companion pieces were a revelation to me and helped me to visualise the output of the mobile video collage/installation in the live collaborative video performance space, on a larger screen. Sometimes Never helped to conceptualise and explain to others the potential visual performative installation in a live setting.

Images 20 and 21 2005 Giselle Beiguelman, screen shot from sometimes never

107

When I started the MINDtouch research, using body knowledge and impulses to interface with mobile video technology as a means of alternative communication, I was aware of others concurrently working in similar domains. Tina Gonsalves is an artist whose work is similar in intention, nature and outcomes to mine. Her site says:
[ Gonsalves] has used the fluid and malleable medium of video and interactivity to explore complex emotional landscapes. Rich, painterly video abstractions evoke intimate associations with personal space. [] she attempts to enrich the public understanding of the hidden emotional language of the body. Using embodied interactive experience, she attempts to explore new ways of experiencing the internal body and the external environments. (2006: online, my emphasis) [115]

She infers that video abstractions are capable of evoking intimate associations in viewers in a way that is akin to dream visual material and experiences. Video art is a perfect medium to explore the hard-todefine-and-describe emotional content of mindbody sensations and perceptual experiences. It is important to find ways to explore the hidden emotional language of the body and continue to investigate new ways of experiencing the internal body through external environments. The mobile phone, in its affective, immediate, intimate and portable nature, has the built-in capacity to externalise the internal in very personal and individualised ways for each user.
[ FEEL.TRACE] is a psychophysiologically responsive video installation synthesizing art [ It] explores new, more embodied languages of interactive and emotional communication, investigating the inter-relationship of the internal body and the external world [] Sensors begin to diagnose patterns of internal arousal states of the body, picking up how the body is responding to the imagery. (2006: online)

Image 22 2006 Tina Gonsalves FEEL.TRACE

115 Her work can be seen online at http://www.tinagonsalves.com/ (Accessed March 20, 2010).

108

In FEEL.TRACE, Gonsalves focus is on exploring the mind/body as a much ignored means of communication and expression, and the idea of our embodied and emotional responses speaking via visual imagery:
Bodily reactions of the viewer continue to trigger changes in the projected video and audio content. The video footage constantly adjusts in response to the viewers internal state (creating a bio feedback dialogue between the image and the participant). The software reads the responses of the body, making a choice to either calm the viewer or further stress them out. (2006: online, my emphasis)

Gonsalves approach of having the video content adapt to the bodies of viewers and their internal states is inspiring.
MEDULLA INTIMATA is a necklace that contains a video screen and biometric sensors. The sensors [] monitor the wearer's emotions to guide real-time video-generation that evokes a sense of seeing beneath the surface of the skin, exposing the emotional and physical inner body. (2004: online)

Image 23 2004 Tina Gonsalves with Tom Donaldson MEDULLA INTIMATA

The imagery displayed is an emotional portrait, an exploration into the secret life of the emotional, physical, spiritual and psychic body of the wearer. (2004: online)

It is curious how the jewelry managed to expose internal thoughts and feelings of the wearer. There are compelling parallels between Gonsalves philosophies and approach to her work and my own. She uses body data to explore video imagery and trigger emotionally charged content, to affect the visitor/participant in particular ways. She is equally concerned with the connection between visual images and how the body responds, and vice versa in a feedback loop. She also employs wearables and biosensing technologies to tap into this relationship between video and the emotional or

109

physiological. She also uses wireless, portable devices to unearth new methods for artistic discovery in participant interactors. She integrates the spiritual and consciousness dimensions, exploring various approaches to interweaving these through her projects. While I discovered her work well after starting my own and was not influenced by it, it was rewarding to find a parallel researcher with similar concerns.

5.2/ non-linguistic mobile video communication

My interest in non-verbal, visual communication for this research was increased when I realised, during mind-quieting activities in the video collection workshops, that participants were repurposing the phones. This was evident when conducting the body sensation visualisation and telepathy-simulation activities with participants. When they were asked to express their perceptions with videophones, instead of using the dominant modes of use of text and voice, what resulted were ingenious ways of communicating through the imagery. Thus, the focus shifted from just making embodied video expressions for the performance video to helping people to communicate non-verbally in order to express body perceptions visually, and using their senses rather than relying solely on words. Video is the chosen form, but now the mobile phone video camera allows one to repurpose the phone to visually express emotions and perceptions experienced internally, rather than it being merely a device for documentation or an entertainment gadget. Murtagh noticed non-verbal behaviour in his research on mobile phone use, and saw that:
[...] the focus on bodily gesture and eye contact emerged from the data when it was noticed that activities with and responses to mobile phone use were almost invariably non-vocal in nature [] Yet, so much of mobile phone use in public is organised through non-verbal action and interaction. It is suggested that these non-verbal aspects of phone use display the 'unwritten rules' of usage behavior in public. (2001: 82)

Non-verbal communication normally means using eye-contact, physical gestures, body language, tone of voice and other physical indicators to express certain information to another person, to underline a conversational exchange and make a point more clear [116]. Sometimes it is a mode of communicating the unspeakable, before verbal communication and introductions have been established, or as a means of initiating contact, or to express something intimately without the need to verbalise the sentiment. We all use non-verbal modes of expression and learn them at a very young age (possibly before we are able to speak). In this work, since participants were asked to use videophones and imagery to speak for them, they had to draw upon the visual material in their environment, as well as from their own sensations, perceptions, thoughts and emotions, as their vocabulary. It also aided them in using their prior knowledge or experience of visual language to find ways to create a language or
116 Based on 'non-verbal communication research and techniques (Argyle, 1975: 243260).

110

visual vocabulary from the raw visual and sensory materials, and to construct video sentences or utterances into something communicable. This falls outside physical or body language and is more akin to a cinematic language [117] of video art, abstract cinema or Live Cinema (previously discussed). Thus, participants desirable for the workshops would be people untrained as artists or performers (some participants did have creative lives, but usually in a different form). The only training participants were expected to come with was that of the regular exposure we all have to a daily stream of advertising, television, Internet, films, games and entertainment media. This exposure is considerable for most, enough to establish an equal starting point for engagement in the project activities. Even if some came with a more intuitive talent for visual composition, or a natural (or trained) eye for aesthetics, all were able to create something exceptionally poetic. My principle has always been that all expressions are valid, since each is someones personal expression, from their own unique perceptions, represented in their own unique voice, just as language allows. This exploration, however, was not about teaching or developing a new or specific vocabulary for participants to express within, except: within the limitations of the device; their comfort and knowledge in manipulating the device; the limitations of the available environmental visual material; the participants ability to create or use the material to represent their emotions, perceptions and sensations or experiences. AND, within the structured activities they were guided through to find ways to express or record patterns and other visual elements. In this way, certain vocabularies or syntax of representations of internal experience could emerge. If more workshops were conducted, with more variety of structured improvisation activities for different types of expression, a symbolic language of nonlinguistic video expression could be developed, but that was not the original aim of the project, although it could become the aim of future projects. Thus, non-verbal communication emerged here, through gestures and the physicality of participants and in their exploration of representing their internal experiences through the external world visually, using movement afforded by the size and portability of the mobile phone. This is not the definition of non-verbal communication generally used, and does not require in-person presence or liveness in other ways. In fact, that is the point: this is a new, digital and remote, visual mode of non-verbal communication (I posit) and through the absence of the other (in-person), the concept of presence comes back to the foreground. This mode of non-verbal expression is like a mobile, democratised form of live cinema, more of an exchange or conversation than a performance by one artist or expert for a passive audience: it is an active, emotive interchange. What is unclear, and also an area for future exploration, is how the receiver experiences and interprets what is communicated. Does direct, literal understanding matter? Yes it might, if we are to make a new semantic language. Nevertheless, it can be argued that, like in a dream or an artwork, the receiver/viewer/dreamer has the freedom to interpret a message any way they like. In some cases, the
117 It is worth mentioning non-verbal films, like Baraka or Koyaanisqatsi, which are visual essays communicating their messages purely through imagery or cinematography.

111

imagery is clear enough that the communication is evident and obvious, as with any form (see Image 24 below [118]): the way the communicator constructs the message will either help or hinder its interpretation and how it is decoded. Future research projects could look at making a more specific visual alphabet and grammar, but that is not what was attempted here. The aim was merely to give people the tools and a loose structure, to help them to create a language of their own. This could be developed further into a more structured language (for future uses with the device), and could prove useful for people with real physical, verbal or linguistic limitations. This study, however, is an exploration of the possibilities of expression of personal experiences using the mobile phone, allowing the average, non-artistic user to be playful, creative and expressive in an intimate and novel way [119].

Image 24 2007 C. Baker still from participant video during Dublin workshop, October 2007

The next few pages feature technological art works that employ non-verbal, wearable sensing and/or mobile or other virtual connection technologies to communicate across distance. As mentioned in the methodologies section in Chapter 2, I was involved in the whisper[s] project [120] from 20032005 as a researcher/coordinator, working with artists/researchers Thecla Schiphorst and Susan Kozel. My role involved research in performance, wearables, experience design, biosensor testing, prototype garment construction, movement development with wearable devices, participant

118 This image is an example in which one of the participants videos was very clearly communicating her frustration of being stuck in her wheelchair after previously being an able-bodied dancer of course only I know this (or those in the workshop), since I know her and knew she couldnt walk down the stairs. And so, perhaps this was/is only obvious to me as the researcher, and anyone else I may have shown it to (maybe it was a message to me). 119 Johansson explains Serres theory of The Five Senses: Philosophy of Mingled Bodies, which says that tactility is the most fundamental sense of the human soul and its experience of itself and its environment (2003: 143, paraphrasing Serres), and that this can be extended to how we experience images. Images are tactile if taken in ones hands in a literal sense. However, Serres continues that tactility encompasses all senses and that photography and cinematography are physical impressions of the referential world or a true image of reality presented on a membrane or skin (2003: 144). Thus, [] a tactile image being a medium of (physical) impressions and emotions, of impressions of the physical world and their implications for sensation. (2003: 144145) 120 This project is available online at http://whisper.iat.sfu.ca/ (Accessed March 20, 2010).

112

workshop development and assistance, as well as website design and maintenance, videography and video editing. The projects main aim was to explore how embedded, wireless devices in clothing might create a body area network, in which bodies and people could exchange their own body data that data could then be seen or felt by others within the clothing, through various actuators or outputs, or vice versa. Below is a better definition by Thecla Schiphorst:
whisper[s]: wearable, handheld, intimate, sensory, personal, expressive, responsive system. whisper[s] is a collaborative project involving artists (dance, sculpture, music), designers (of visuals, objects & textiles), computer scientists and hardware/software engineers. We are developing technology and communications metaphors that enable networked wearable devices to communicate affective states in a continuous manner. (Schiphorst, 2005: online)

Image 25 2005 whisper[s] project screenshot from a video I shot at SIGGRAPH conference

whisper[s] is about finding new modes of embodied communication and exchange and a playful

means of incorporating fashion, gesture and movement, media installation, expressive devices and art in new and compelling combinations.

Image 26 2005 whisper[s] project, showing the electronics, conductive fabric and designs

113

The whisper[s] project had significant support through Canadian arts and technology funders due to the innovative nature of the work, and it presented at prestigious technology exhibitions (Siggraph 2005), and conferences (several CHI conferences), and iterations of clothing and technology. Beautiful garments were created for it and many participants played with it. When I left the project in 2005, the technology had just begun to work in terms of body data consistently being exchanged between participants through rfid tags, and represented both on led lights on the clothing and through triggering mobile phone sounds, as well as other visual representations and installations.

Like whisper[s], Skin Probes focus on finding new, embodied, playful or sensual means to communicate non-verbally emotions to others. and An sensations

interesting aspect of this project is that it has been created by a large corporation. The website states:
As part of SKIN, we have developed two Soft Technology outfits to identify the future for high tech materials and Electronic Textile Development in the areas of skin and emotional sensing. The dresses show emotive technology and how the body and the near environment can use pattern and color change to interact and predict the emotional state. (Philips, 2008: online) The various manifestations of the projects are
Image 27 Phillips Design, Skin Probes

very intriguing:

Skintile: Electronic Sensing Jewelry has been conceived alongside a European project, STELLA [121], developing stretchable, flexible electronic substrates that integrate energy supply, sensors, actuators and display. [122] Skintile the Electronic Sensing Jewelry further explores emotional and physiological sensing [ using] wireless, stick-on body sensors that re-define traditional body adornment [] It explores a range of functionalities in new product forms that are playful, sensual, mood affected, bio activity stimulated, and arousal enhancing. It is a semi disposable, bio compatible, nonallergenic, breathable, mass customizable, self-contained body worn accessory. (Philips, 2008: online)

121 More on STELLA is available at www.stella-project.de. 122 See the Design Probes website, available at http://www.design.philips.com/probes/projects/electronic_sensing_jewelry/index.page (Accessed May 6, 2010).

114

Image 28 Philips Design Skintile

What is exciting in this research is the cable-free and aesthetic way in which these devices or jewelry pieces sit on the body. This project is merely a conceptual (proof of concept) prototype. Yet, if they were functional, that would be an exciting achievement.

Bare Conductive [123] ink was recently awarded a prize at the prestigious Ars Electronica electronic art festival for 2010; it uses another exciting form of non-verbal communication technology, drawing people back to the body as the primary mode of exchange and interaction.

Char Davies (1995) Ephemere To use biometric medical devices with mobile phones is not new, as can be seen by Gonsalves and whisper[s] above. However, the intention is to work with the body as interface from an embodied paradigm for media interaction, as well as communication exchange in a non-linguistic and visual form. Char Davies is one of the most interesting artists to use biometric devices and new media in her famous VR piece Ephemere (1995) [124], in which participants could control their movement with their breathing within the virtual space. Davies had participants explore within a beautiful, abstract immersive 3D virtual world, using the body as a mouse. To experience the artwork she painstakingly crafted, participants navigated through it engaging the whole body in a fully visceral way. The work affected participants very deeply, personally and intimately. MINDtouch similarly facilitates the participant to create and share their own perceptual narrative, triggering media created by them, through their body experiences, sensations, memories, associations and imagination.

123 More on Bare Conductive is available online at http://www.bareconductive.com/ (Accessed May 30, 2010). 124 See more on Char Davies work online at http://www.immersence.com/ephemere/index.php (Accessed May 30, 2010).

115

Image 29 1995 Char Davies, Ephemere

Kathleen Rogers (1995) Psi Net Terms such as virtuality and wireless communication can be applied to telepathic communication, and incorporated into new practices and theories of communication and culture. This was a project I discovered and I found unique and compelling during previous research. Kathleen Rogers piece Psi Net (1995) [125] piqued my attention because her work was the first, and one of few, that addressed concepts of telepresence and cyberception. These notions were coupled with medical and military technologies within the context of a parapsychology experiment in her performance piece. Psi Net compared telepathy and electronic virtuality, mediumship and electromagnetic technology with wireless networking, by studying and recording participant responses. Rogers created a pseudo-science experiment, from concepts and discoveries in physics, psychology and parapsychology, delving into the unknown and controversial for artistic and theoretical ends. Rogers tapped into similar issues of embodied, virtual abilities or innate personal energy (as she calls it), by exploring ways in which technology can enhance or facilitate channelling of this energy, in a clearer manner.

125 Kathleen Rogers, THE PSI NET PROJECT: Psi-Phenomena - Telepresence Virtual (1995). Available online at http://web.ukonline.co.uk/Members/kathleen.rogers/index.html (Accessed May 30, 2010).

116

Image 30 1995 Kathleen Rogers, Psi Net

My work continues to be focused on aspects of telepathic communication and its nature as a model for other types of communication. Rogers had these same pursuits and explored new ways to use emerging technologies to assist in the exchange between humans, in embodied (or, in Rogers words, telepresent communication) and natural networks. In her project Psi Net, she recorded the personal energies of the participants, by recording three days of three-hour-long, live, remote viewing and mediumship experiments, using highly sensitive thermographic optical scanning technology, biofeedback heart monitors, and voice recordings. She then transmitted them live in the nearby area, via a microwave link. Rogers was less concerned about the participants experience and more concerned with the telepresent images that were produced from this experiment a fundamental difference in our concerns. For my practice it is vital to have both in tandem, informing each other. Her work, however, speaks to my interest in dream studies, parapsychology, neuroscience, studies of brain functioning and consciousness.

5.3/ generative and database video

In Making Art of Databases (2003), mentioned in the introduction, Anne Nigten introduces the artists and the master classes discussed in it. She explains that there are two forms of database and archive art project: 1) projects with predefined and domain-specific applications, with content familiar to the maker, and outcomes representing ways to configure and select the content; and 2) dynamic environments and applications that are created through participant interaction, based on rule sets or learning-based decision-making, using a wide range of parameters (2003: 7). Nigten indicates that a combined approach, connecting the two different types, could be interesting to consider for projects where the interpretation of a wide range of parameters and non-linguistic inputs needs to lead to some

117

sort of coherent environment (2003: 78). MINDtouch partially has this combinatorial approach, although this is not in order to produce a coherent environment. On the contrary, abstraction is very desirable in this project as an emulation of dream experience. Lev Manovichs early experiments at generative narrative using a non-linear video structure have been intriguing, particularly conceptually. His work has informed the creation of a database of mobile videos from my workshops, which are the basis of the performance collage, where the videos are remixed live over the network. His generative film project Soft Cinema (20022004) is relevant to live video performance, with its focus on the non-linear, non-narrative, ambient or generative video constructions, as well as software and database systems developed for them. Many artists and filmmakers are finding new ways to generate narrative or non-narrative cinema online, using computer databases and programming to display random film clips (as in video paintings) and audio. His ideas are conceptually (if not artistically) relevant here in terms of the live and generative cinema aesthetic and database-driven aspect. Similarly, a database was used in MINDtouch to store archived footage from the workshops. This database was also necessary for the temporary caching (temporary storage) of the live-streamed media clips, as they made their way to the server during the live events. This generative aspect of the custom software is similar to processes used by many Live Cinema/VJ performing artists, who make specialised VJ software in programming environments like MAX, MSP, JITTER or Pure Data. In this way, the live performative, the programmed and the semi-generative elements are intertwined. The generative aspect for my project comes from the body data sent to the server, which accessed and mixed archived and live footage, with specific visual effects added. Originally, it was not meant to be in any way generative. Everything was to be controlled through participant interaction [126]. Conceptual aspects of the database, along with visual methods [127] for tagging and categorising for network use have become part of the work as well. For Manovich, generative, database-driven video is always creating something new and live. He states:
Soft Cinema consists from [a] large media database and custom software. The software edits movies in real time by choosing the elements from the database using the systems of rules. The software decides what appears on the screen, where, and in which sequence; it also chooses music tracks. In short, Soft Cinema can be thought of as a semi-automatic VJ (Video Jockey) or more precisely, an FJ (Film Jockey). (Manovich, 2004: online)

126 Being developed for future iterations. 127 I cover visual methodologies in the Gillian Rose sense in the methods chapter.

118

Image 31 2002 Lev Manovich, Soft Cinema [128]

In MINDtouch, narrative is not a key element. Any narrative connections are either those of the participants making the video clips or in the minds of the viewers of the final layered collage of the different videos. One can say, just as a dream may be meaningful only to the dreamer, here the narrative and meaning is made in the mind of the viewer. There are connections between the video content and the sensor and visual effects applied to the videos, but these are more emotional and sensory connections I have applied, not narrative ones [129]. In contrast, Manovich propounds his view on the relationship between narrative and the database, stating that a,
[] database supports both the database form and its opposite narrative, a composite organization of an image on the material level supports two opposing visual languages. One is modernist-MTV montage two-dimensional juxtaposition of visual elements designed to shock due to its impossibility in reality. (2001: 228)

Watching the Soft Cinema video combinations, the argument for narrative as he explains it seems a bit tenuous. I would argue that it is not a narrative that is constructed from these algorithmic combinations, but more a painting or photographic journey. Definitively cinematic, montage-like meaning constructions can be made from the different combinations. Shock is a dramatic term, however, for the visual content he presents. Manovich might consider MINDtouch also to have a modernist approach [130], but it is not for shock value. He also claims he is exploring four ideas [131]:

128 See this project online at http://www.softcinema.net/ (Accessed May 29, 2010). 129 Unless we called personal interpretive visuals narratives of experience. 130 I imagine he means in contrast to a more traditional Russian early cinema, Eisenstein definition of Montage, intended for intellectual and political confrontation and agitation. 131 More on Manovich is available online at www.manovich.net (Accessed on March 20, 2010).

119

First is "Algorithmic Cinema": which is the writing and implementing of a script and a system of rules, that are defined by the creators or authors of the cinema clips, the software then controls what the screen layout is, including the number of windows and their content. Then the author/videographer can choose to have minimal control letting most of choices to be made by the software, or they can determine which clips the viewer will see at specific points in time. However, the actual editing is done in real time by the system and the video clips and sound can run continuously without repeating the same edits. (2004: online)

During the development and design of the MINDtouch mixing software, it was not my intention to make it an algorithmic process between the data coming from the biofeedback sensors on the body, the mobile phones and the computer database/streaming server. I wanted to give control over the mixing to other live or remotely connected participants. The algorithmic approach has, however, been employed in the current version of the system, due to the complexity of the technical work needed to make the project work given the current state of network technology development and the resources available [132]. Yet the aim is to move away from an algorithmically driven, generative system to a more multi-participant, live and participant-controlled system. Manovich continues:
Then theres "Macro-cinema": where the computer user uses the various windows sizes and dimensions within the larger frame. Next is "Multimedia cinema": where the video is meant as only one type of representation, with others being 2D animation, motion graphics, 3D scenes, diagrams, maps, etc. (2004: online)

Finally, he explores the concept of value here is that of Database Cinema:

[] where the media elements are randomly selected from a database of clips and sound, to create virtually limitless (limited only by the number of clips and media elements themselves) combination number of video elements or different versions of the same film. (2004: online)

This is an approach to using a database as a new representational form in and of itself, in order to find new ways to display the videos in the database. Thus, it is an exploration of database versus narrative structure or recombinations for various narrative effects, and this is what is also attempted in the MINDtouch project. However, I wanted the recombinations and choices to be made by the participants as performers, more in the manner of Live Cinema performances, rather than by generative means. Perhaps I will explore this further in future versions. What is compelling about Manovichs thoughts on the narrative potential of the database is his suggestion that the narrative exists in the minds of the interactors or viewers in a cinematic database context:
The elements on a syntagmatic dimension are related in praesentia, while the elements on a paradigmatic dimension are related in absentia [] while the paradigmatic sets [] the elements which make it [] are present in reality, while pieces [] which could have been present instead different [] only exist in the viewer's imagination. Thus, syntagm is explicit
132 This is discussed more in the outcomes and technical development chapter.

120

and paradigm is implicit; one is real and the other is imagined. Literary and cinematic narratives work in the same way. Particular words, sentences, shots, scenes which make up a narrative have a material existence; other elements which form an imaginary world of an author or a particular literary or cinematic style and which could have appeared instead exist only virtually. Put differently, the database of choices from which narrative is constructed (the paradigm) is implicit; while the actual narrative (the syntagm) is explicit. (Manovich, 2001: 30 31)

I have adopted this approach to the narrativity of the collaborative, database-supported, live cinema performance enabled in MINDtouch: the narrativity is being made with the abstraction in the participants minds, as body mixers, as video creators and as viewers. This is akin to making dream visuals, with implicit meaning, while embodied through the biofeedback sensors, mobile video capturing and streaming. The various types of nonlinear, live, generated and performed video Manovich discusses in his books and projects are compelling in terms of ways to perform with video, and important to this research since they incorporate elements of live cinema or performance video and database triggering. Combining VJing and live cinema or video performance mixing for my project is a means to emulate the sharing of embodied dream imagery. This is a key aspect of my practice using currently available technology to simulate a telepathic exchange between two or more bodies, lucid dreaming visually together. While not a Manovich follower, I find his ideas conceptually compelling and artistically inspiring. Even though the outcomes of the Soft Cinema project are not that exciting in my view, they have been a partial inspiration to aspects of the MINDtouch database, inasmuch as an archive of mobile videos created in the workshops constitutes the building blocks of the work. Fundamentally, the goal of this project has been to enable completely live, mobile video VJing, without a central computer (even a server) intervening, but rather with a distributed system, across all users mobile phones this is the ultimate mobile, social, expressive collaboration tool. However, that is for a future version, MINDtouch2. In summary, this chapter has proposed a new form of video aesthetic using the mobile phone. It has also provided a rationale for using generative systems for database cinema in this work, and compared it to other similar works in video art.

121

6/ networks, telepresence + connectivity

This chapter reflects on telepresence and telematic performance, the connective threads of digital networks in practice, and the access issues that can challenge mobile telematic performance. It contextualises the network dimensions both performatively and in terms of the technical aspects and constraints encountered in the project, alluded to earlier at the end of Chapter 1.

6.1/ telepresence and telematic performance

Telepresence is that sense of presence of another person that can be felt using technology and/or telecommunications devices and systems, the presence felt in a telephone call to a friend. The MINDtouch project explores this sense of presence in performance and uses telepresence to explore ideas on telepathy and extended mind (discussed at length in Part II). Steve Dixon quotes Inke Arns definition of telepresence:
Telepresence allows the viewer parallel experiences in three spaces at once: 1. The real space in which the viewers body is physically located; 2. per tele-perception in the virtual, simulated visual space reproducing a fictional or real, remote visual sphere; and 3. per teleaction at the physical location of the data work or even of a robot controllable over ones movements or equipped with a sensory apparatus over which one can find ones bearings. (Dixon, 2007: 419, quoting Arns, 2004: 336)

Edwina Bartlem describes telepresent art and its development history through theorists and artists, such as Roy Ascott and Paul Sermon, in her article Reshaping Spectatorship: Immersive and Distributed Aesthetics in the 2005 Distributed Aesthetics issue of the online journal Fibreculture:
Telepresent, networked installations share some concerns with telematic art in terms of linking participants from distinct locations, foregrounding the concept of a networked community, stressing process-orientated art practice and enticing multiple users into participatory relationships with art [] The operator extends their body through hardware and software technologies. A technological device becomes an extension of the operators body, continuing human presence beyond the corporeal body through information networks and into a mechanical form. (Bartlem, 2005)

Yet it seems that telematics involves more than a connection, but a performance in one or each location. Dixon succinctly defines it in his chapter heading:
Telematics: Conjoining Remote Performance Spaces (Dixon, 2007: 419)

I was first introduced to telematics and network performance during my masters studies and through the New Forms Festival. In July 2004 the US arts organisation Turbulence started the

122

Networked_Performance blog [133], which I began following. The works that were listed on the blog were (and still are) varied and there are some truly amazing projects. Susan Kozel led a course on telematic performance, at an undergraduate and graduate level, where I was doing my masters studies. After completing the course, I later became her teaching assistant for the undergraduate course. She says:
Telematics permits for a play across absence and presence, as well as a range of dynamic impulses, both human and nonhuman. The suffix matics [] is infused with enough ambiguity for it to be open to diverse unspecified interactions [] performances and installations [] revealing how the human senses of intimacy and physical connection are dependent on playing across what is revealed and what is concealed, rather than simply on what is visually displayed. (Kozel, 2007: 86)

These experiences aided in the curatorial process for the New Forms Festival, and ultimately led me to create the MINDtouch project, a mobile video telematic work. I witnessed a striking performance by cyborg artist Stelarc at the Western Front Artist-Run centre in Vancouver in February 2002, of his famous Ping Body, developed in 1996 [134]. This performance involved online participants sending commands to a remote computer to trigger electrode sensors attached to the skin of Stelarcs arm, sending electrical pulses to shock him. This was a very visceral interaction and the audience present empathetically experienced the shocks with him, watching the responses on his face. This performance demonstrated the power of the remote networked participant and the effect on the audience of such a performance. Ladly addresses this sense of empathy and the blurring of the boundaries between the real and the virtual, exemplified so well in this Stelarc performance:
It would seem that the telematic performance of reality has become reality indeed, and that the boundaries between performance of the real and the virtual have dissolved to the extent that they have become one act, at least in the minds of most young users of the technology. (Ladly, 2007: 143)

I also had the pleasure of participating in a seminar/workshop with Paul Sermon, at Goldsmiths Universitys Intimacy conference in London, in December 2007. Sermon is a pioneer in telematic performance. His performance installation Telematic Dreaming, from the early 1990s, featured Kozel as the performer, connected to the audience in another room in the exhibition space through video cameras only [135]. While this piece was a local network only, it represented the performative issues and the qualities and experiences of digital presence and absence, as well as public interaction and responses within such a construction. It demonstrated the power of networked interrelations and the affective experience of those interrelations, bringing questions around the real, actual and virtual, as
133 This blog is moderated and entered by Jo-Anne Green and Helen Thorington, of Turbulence.org, and Michelle Riel, and was started as a way for them to record [] network-enabled practice, to obtain a wide-range of perspectives on issues and to uncover commonalities in the work (2005). Available online at http://turbulence.org/blog/about/ (Accessed May 30, 2010). 134 See a review of his work online at http://stage.itp.nyu.edu/history/timeline/pingbody.html (Accessed May 30, 2010). 135 There has been a great deal written on this work. Bartlem (2005) discusses it at length, as does Kozel, Sermon himself and many others.

123

well as liveness, into the foreground. All of these issues, and current ones that Sermon delves into in his recent work in Second Life, were discussed and explored in the seminar. This workshop with Sermon was another part of the journey to MINDtouch, inspiring me to explore physiological sensing as another way to embody the experience of presence in participants during the performance.

6.2/ electronic threads between us

Connecting people through the invisible threads [136] or ephemeral connections of mobile telecommunications networks has always been the crux of the MINDtouch telepresence aspect (defined further in Chapter 7). Complexity theory [137] is beyond the scope of this research, but it should be acknowledged that involving network systems brings in elements of complexity, conceptually and practically. It has been manifested through the network of bodies, connected to sensors, connected to phones, connected to a database of video clips, manipulated by the body data, through the output to the world of networked and possible interactors/participant performers online globally. We could say that this is a complex set of body/mind interactions, connected through the ether to other people/bodies in a real intuitive (telepathic) and simulated virtual (mobile phones) form. What is a network? It has been described as a spiderweb-like or rhizomatic structure, or like a root system. However, Mitchell describes it this way:
The archetypal structure of the network, with its accumulation and habitation sites, links, dynamic flow patterns, interdependencies, and control points, is now repeated at every scale from that of neural networks (neurons, axons, synapses) and digital circuitry (registers, electron pathways, switches) to that of global transportation networks (warehouses, shipping and air routes, ports of entry). (Mitchell, 2003: 9)

This represents just a couple of forms of network [138], but the key word is interconnected and this is what my work is about: interconnection between people, and between people and things (i.e. mobile and computer networks, systems and operations).

136 This is discussed conceptually in terms of notions of presence in Part II. 137 Sher Doruffs thesis does a fantastic job in looking at complexity, network and databases and the wider related philosophical concepts. Her PhD dissertation, The Translocal Event and the Polyrhythmic Diagram (2006), is based on her own experience working on projects related to my own, developing software and systems for collaborative networked performance. 138 See also the New American Oxford Dictionary definition in Chapter 1.

124

My own play on the notions of network connections arrived like this: links = chain old definitions web = net threads / sewing / netting / weaving/ interwoven/ knitting / fabric connective tissue (Susan's) networking = social connection mesh / enmeshed / entangled / tethered / tied / knots / rigging rhizome / nodes branches / roots cellular / veins / tunnels / paths inter- / intraAs mentioned earlier, the initial concept was for the project to be completely free of computers, free of wires, and non-nuclear. The intention was to move away from the immobile, computer-based videoconferencing setup to a completely mobile system, a more untethered, true sense of telematics. Over time, through much frustration and many conversations about the vision for the project with different collaborators, who tried to steer me toward pragmatism, I discovered: a) that the smartphones used (there are more sophisticated ones as I write, iPhones [139], etc.) were not yet sophisticated enough to replace a computer server system; b) that video is still too computationally complex for mobile phones to mix, process and stream at an acceptable frame rate and resolution for satisfying viewing; and, c) that researching and developing garments with wireless, cable-free biofeedback sensors, conductive threads, etc, would constitute a separate project, and a resource-intensive one. For MINDtouch, we had to work with DIY, cumbersome sensors and a lot of cabling, which had to be hidden within the garments. At the start of the project, I was unaware of the corporate and political, behind-the-scenes issues at play regarding mobile networks. From my North American perspective, where mobile use is less
139 At the time of writing this, WIRED magazine has announced news of Skype for the iPhone. There is no mention of video, but claims of 3G capability suggest video would be possible. See news available online at http://www.wired.com/epicenter/2010/05/skype-over-3g-comes-to-the-iphone-its-not-all-goodnews/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+wired%2Findex+%28Wired%3A+Index+3+%28T op+Stories+2%29%29 (Accessed May 30, 2010).

125

ubiquitous than in Europe, the European networks had the appearance of being similar to the Internet: more freely networked and more interconnected. I discovered that economic issues, as well as political and corporate agendas, require mobile manufacturers and services providers to create network security systems, firewalls and one-to-one-only Bluetooth connectivity on mobile devices (although Bluetooth can be enabled for one-to-many broadcast). I became aware (late in the project, however) of how extensively controlled mobile networks are when I ran into several barriers challenging the ideal of truly untethered, unblocked or unrestricted mobility. New questions arose, such as: why can one make a phone call from anywhere, but not stream video from anywhere, using the same phone in the same place? How are the different negotiations between big corporate providers made? Why are such negotiations regarding coverage or what is included in ones service plan not transparent or readily divulged to customers/users until they are prevented from doing something. It was discovered that, due to the differing historical, technological and market development of the mobile phone versus the Internet, there are different levels of accessibility. The Internet, developed by the US military in the 1980s and called the ARPANET, was initially to be used to provide free, open and easy accessibility between researchers at universities [140]. Services in the corporate telecom universe were provided based on competition for territory and customers. The carving up of territories has resulted in more controlled and restricted phone access, tethered by where you live, which provider you subscribe with, which plan you have, and which model of phone you have on your contract. This economic model of mobile phone services is similar to that of fixed phone lines severely limiting mobility and what can be done using a mobile phone [141]. The ideal of untethered and unrestricted mobile connections appears increasingly distant in this highly controlled, monitored and restricted mobile world (Mitchell, 2003: 59-60). This artistic exploration stumbled into and uncovered a quagmire of contradictions about networked connectivity. By contrast, telepathy is (so far) still free. The lack of unanimous acceptance of its existence may be a positive. One could say that telepathy has its own levels of security, in that it is said only to normally occur between people who have established a level of intimacy or trust, such as between mother and child, siblings, lovers, and close friends [142]. However, the connection is not always obvious or solid and signals can be obscure. Perhaps if it were accepted, it too would be harnessed, controlled and infiltrated by government, corporations and marketers. In fact, mobile communication networks were more free in 2007 at the start of the project, but through ongoing development technically, politically and economically they have become more controlled and restrictive. This increasing control is manifested in the ever-increasing implementation of locked systems, passwords and firewalls (Greene and Haddon, 2009: 2324). Initial experiments, made using

140 This is not to be confused with the development of the World Wide Web that Tim Berners-Lee is credited for through the European Organization for Nuclear Research (CERN) in 1989. Sourced on the Computer History Museum web archive, available online at http://www.computerhistory.org/internet_history/internet_history_80s.html (Accessed May 30, 2010). 141 More can be read on the development and history of the mobile phone economically, politically and socially, as well as mobile communications evolution, in Greene and Haddon (2009) Mobile Communications: An Introduction to New Media. 142 Ullman, Krippner and Vaughan did numerous studies on this and it will be discussed in Part II on forms of presence.

126

the service Fring [143] in London early in the research, demonstrated that many wireless hubs were open to piggyback on and join without a password. This changed quickly as people and businesses became more security aware. My interest in using an open, roaming WLAN (wireless internet) system was not in order to be a freeloader, but to minimise mobile costs for participants and experimentation costs for the project when streaming video [144]. Other barriers began to crop up when conducting workshops and performance experiments, trying to send sensor data over the Internet, trying to connect remotely to the streaming server, or from any university campus I went to (even though IT staff always assured me it would be no problem). Overall, there was a general lack of network accessibility, interoperability of technology systems, cooperation from providers and institutional IT support. Frustration developed as I exposed the myth of open networks, free roaming and ease of connection sold to the public by providers, and my optimism soon faded. Mitchell calls this [] fluctuating conditions of freedom and constraint [] (2003: 9) and addresses it in this way:
With the proliferation of networks and our increasing dependence upon them, there has been a gradual inversion of the relationship between barriers and links [] Extension and entanglement trump enclosure and autonomy. (Mitchell 2003: 10)

However, this state of affairs was not enough to deter me. I gradually learned that the open connectivity I envisioned is possible, but only with the right access codes, political power, money and influence. This does nothing for the common person, researcher or artist trying to use the system to reach out in nonstandard ways. Mitchell equates the vascular system of the human body with the larger social/political body of twentyfirst-century connectivity. In essence this equates to the emotional, affective, consciousness connectivity of people, animals and the planet (Mitchell, 2003: 22). However, this particular body has some major artery clogs, in my view. Perhaps my vision is utopian, but as has been discussed by others here and elsewhere, when given the tools people want to reach out and connect and will find creative ways to do so. In recent history this has manifested through technology in the telegraph, Morse code, the landline telephone, the fax machine, the Internet, mobile phones and, recently, social networking tools. These all demonstrate our desire to make connections. The somewhat sinister side of this is uncovering the extent to which governments and corporations wish to intervene, control and

143 A service and application for the mobile that searches for WiFi or WiLan hubs nearby while you move about. This can access the Internet for free and much faster and easier than the mobile network. This was quite easy in Vancouver in 2007, when very few people locked their wireless hubs with WEP passwords since it was only becoming widespread. The city is also small and a business hub compared to London. As I write this, Fring have just announced an application for iPhones that supports Skype voice-over IP and suggests on its webpage that this should include video, available online at http://www.fring.com/ (Accessed May 30, 2010). This is free for now. See also a story on this in WIRED magazines affiliated site Webmonkey, available online at http://www.webmonkey.com/2008/10/fring_turns_your_iphone_into_a_free_skype_phone/ (Accessed May 30, 2010). This development does not change the fact that most WiFi hubs are locked or have to be paid for, so easy roaming is not possible, but perhaps that will change; these things are in flux. 144 There have been movements toward free WiFi zones in city centres like London, but these have been slow or inconsistent (see this story online from 2007: http://www.zdnet.co.uk/news/networking/2007/07/13/london-gets-free-wi-fi-39288023/. I will discuss the European open model conflict. There is free WiFi in the downtown core of Toronto since 2007, see info online at http://wirelesstoronto.ca/ (Accessed May 30, 2010) and other cities and places in the world are moving toward this. Organisations like Wi-Fi-FreeSpot (see online at http://www.wififreespot.com) are mapping free hotspots elsewhere, so things could change for the better (Accessed May 30, 2010).

127

monitor citizens in how we connect and interact with each other. In some limited cases, this is with good reason, but often it is simply in order to capitalise on human desires and loneliness. I may have a nave perspective. However, I see myself as an optimistic artist who wishes to use these systems to enrich peoples lives for free. Through this project, I am now more aware of the structures of these complex systems and am armed with more tools and strategies to achieve my aims within and around them.

6.3/ access / protocols / privileges

Due to the issues mentioned above, the full network connection envisioned (see Figure 10, page 234) would have required more knowledge, time and resources and collaborators [145] to access the right Internet protocols, gain permissions and privileges to cross borders (digital/analogue or Hertzian), or gain entry into independent networks. Big corporations have access through business negotiations and contracts, but independent artists and coders require other means and ingenuity to circumvent or negotiate access. This process can be complex. As Mitchell states, [...] access capabilities and privileges are more important than traditional forms of ownership and control of property (2003: 59). I discovered this the hard way. The simple idea of connecting people via their phones using video becomes a complex set of workarounds and bandaid solutions to enable linkages and connections that should, to my mind, be straightforward. To enable the data and video to move freely across invisible borders, negotiating invisible transactions and exchanges to connect, requires a daisy-chain of connections. This delays the movement of data tremendously as it needs to be translated through various programming languages, codes and protocols on its way to delivery. In a digital world, the possibilities of losing original content and meaning are increasingly diminished compared to older, analogue systems. But the inevitability of delay, or loss of real-time (and sometimes quality of image), is increased in the daisy-chain system that had to be employed for my performance events. This then becomes entirely about coding languages, translation and negotiation. What was envisioned for this project was the same idealist vision Mitchell had: of the 'wireless nomad' (2003: 60). Yet the dream of untethered, freely collaborating minds and bodies creating together, as they might telepathically over space, is not yet as easy as it could be. In fact, it may be getting harder. Networking issues are shifting continually in several directions, so we will see what changes unfold in the coming years. All programmers and network specialists who were briefed on my vision said that it can be done, and that it is a nice vision, but that it is tricky [146] and complicated. Why is it tricky? Why does it need to be? My view is that the reasons are fear, profiteering, control, security, and a desire for privacy and law. This idea of ubiquitous mobile network

145 Such collaborators have been found for future work. 146 See testimonials by my collaborators and other mobile or network experts in the Appendices.

128

access is described by Mitchell (2003: 60) as an idealistic vision. Access privileges are becoming more tightly controlled all the time, at least in Europe and North America [147]. Barriers to free interconnection of devices began manifesting partway through the project when a collaborator, Michael Markert, informed me that we could not use the mobile operating system alone for all the computing we would have to use the Internet and a computer server to enable the project. This was my first concession or compromise in the project. Because the mobile phone operating system was not capable of performing the intensive processing required (perhaps iPhone is able to now), my laptop had to be added to the chain, receiving the sensor data and sending it over the Internet to the server to visually mix the mobile video clips. Also, during the events, we had to choose whether to connect the sensors to the phones or to the computer in order to mix the video: Bluetooth allows only a limited one-to-one connection, which would not allow both. Finally, we had to use a streaming server to store the video clips, mix them and stream the processed video back out. The result was that the original vision had to be modified continually to accommodate what was, in fact, possible. Related to the access issues, I became curious as to whether I was alone or if there was an opensource/open-access mobile phone community, as there is with the Creative Commons online community. On first search what was discovered was a consortium of corporations trying to tackle the issue of freer roaming, interoperable services, mobile operating systems, applications and network connectivity [148]. Discussions on the lack of options are also quite active now, between mobile users, artists and a small number of service and product developers, with communities having arisen around specific phone models [149]. In light of this, there is certainly hope for future projects. This concludes this chapter on practice. This chapter contextualised the networked aspect of this performance research in terms of the medium, and the issues and constraints experienced for this work. The next two chapters focus more deeply on the meta-theory behind and investigated in the work, both conceptually and practically.

147 This situation is totally different in Africa and third-world countries, as Sadie Plant showed in her closing keynote presentation at the International Symposium of Electronica Art (ISEA), 2009. She expounded on how the mobile phone is transforming Africa and India positively, empowering many lives and enabling small business. There is also a more open model in Asia, as Feijoo, Gomez-Barroso and Marin show in their article Why Youtube Cannot Exist on a European Mobile: The European Regulatory Strategy on Mobile Content Access (2007), pp. 263275. 148 Original articles on this were lost, but I did find an active group at the end of this research. Created in October 2008 at the end of the MobileActive.org conference in Johannesburg, the Open Mobile Consortium was investigating the issues in South Africa. See http://www.open-mobile.org/technologies/technologies (Accessed May 30, 2010). Their mission is similar to my desires, found online here http://www.open-mobile.org/about-omc/why-open-mobile-consortium-needed (Accessed May 30, 2010). 149 The relatively new (2009) Nokia N900 has a Linux-based operating system Maemo 5 and so it has opened the doors to open-source application development, which has flooded in too late for this research, but perfect for exploration in future versions. This, of course, needs to grow beyond one phone. See information online at http://danlynch.org/blog/2010/01/n900-review/ (Accessed May 30, 2010). Nokia also began looking for projects to fund on this model (see http://blogs.nokia.com/pushn900/), almost as a way to outsource their research and development (a positive but perhaps a calculating approach as well).

129

7/ presence: permutations
This chapter focuses on the philosophical and conceptual notions of presence, liveness, embodiment and telepathy, the meta-level concepts that underpin the MINDtouch practice-based media art research. These are the crux of the high-level thinking in this thesis, primarily the intertwining of presence, embodiment, mind/body interconnection, extended mind-body and telepathy. There will also be a discussion on the connected concepts and perceptions of consciousness, love, intimacy, absence, the space between and other related experiences. A predominant question emerging from this practice is: how can or do embodiment and consciousness exist within and outside the flesh of the body, simultaneously? While this dualistic conundrum is not solved here, it is teased out in an effort to reconcile the issue for the research, to show how both can be the case in an intermingled fashion, rather than in conflict. Through this investigation, I outline how I have managed to balance the dualities as they inform and wrap around the practice-based performance developments. The mind map below was created to help visualise the connecting concepts and how they all relate back to presence:

Figure 4 2009 Camille Baker

This diagram demonstrates the various non-linear facets, permutations and expressions of presence and how I am working with them. Most of these threads are unpacked and brought together hereafter, but starting with presence at the centre of the investigation, then followed by explorations of embodiment, liveness and affect as they relate back to presence and how presence is essential to each. It will be shown that many of these concept threads are interconnected and overlapping, and

130

that the linear discussion here is not a true reflection of how they are woven together.

7.1/ presence: feltness sensing

The starting position on presence is that a sense of presence is a physical, phenomenological sensibility, almost akin to a vibration or physical agitation, felt viscerally and intuitively. Presence is the feeling or sensation of another person and their engagement in some activity, alone or with another. A sense of presence is experienced when one is engaged actively, emotionally, intellectually, playfully or otherwise in an activity with others and can feel their presence within the experience. Philip Auslander in his book Liveness states that presence has an emotional impact, is ephemeral and evanescent, [... an] intangible expression [that] only exists in the transitory present moment (1999: 132). Conversely, presence for philosopher Shusterman (2000) requires one to be physically present in person, in bodily attendance and in a physical space. Hans Gumbrecht states: What is present to us (very much in the sense of the Latin from prae-esse) is in front of us, in reach of and tangible for our bodies (2004: 17). It is an embodied engagement, a felt or intuitive sense of otherness, emotional, physical or virtual, elsewhere in time and/or space. The term Mobile Presence, coined by mobile phone manufacturers and software developers, means that the phone literally has the capability for the user to see when and how their friends are available to be contacted, based on the Internet-based SMS feature of Skype and other online chat software (Borden: 2007). These companies consider presence to be ones availability-to-connect status, either set by them or the phone system. Such availability statuses include: available to talk; available for SMS only; or not available. Therefore, the various modes of being reachable via the mobile include text, voice, photo and video messages. This is in order that friends can remotely monitor (or keep tabs on) each other. This already happens with SMS/chat software, but can now be done on mobile phones anywhere, at any time (providing more potential for stalking). This is an attempt to fill in the perceived gap between liveness and virtuality through mobile presence, what Ingrid Richardson calls the mobile technosoma, or the incorporation of mobile devices into our evolving corporeality (Richardson 2000: 6).
With so much emphasis in digital media circles over the last decade on digital presence within 3D immersive worlds and virtual-reality environments, my contrasting experience of presence has been more of a non-technological one. I often experience a form of non-digital virtuality [150], a very embodied, immersive, almost liminal, space in the pre-waking/sleep state [151]. My understanding and experience of
150 This is in reference to the ontology of virtuality and is a pre-computer definition of virtual, meaning in contrast to actual, yet still real. This will be discussed further in Chapter 9 on Liveness and also references earlier discussion in Chapter 3 on performance and the real. Pierre Levy is introduced on this topic later and has another definition of virtual that connects this older version with a new sense that incorporates technology, networks and interconnections between people. 151 Farthing (1991) says this is the state of hypnogogic (pre-sleep) or hypnapagogia (pre-waking) in his well-known Psychology of Consciousness, pp. 269270.

131

presence informing my practice and daily life is a sense of otherness, a feltness that can be physically sensed in a room. It is concerned with the feeling of others viscerally, physically and on an intuitive or emotional level, just as one feels when speaking to a friend or loved one who is not actually there with one in person. It is akin to feeling someone while speaking to them on the telephone or across the table, through the qualities of their voice or their gestures [152]. This notion of presence also encompasses a physical, conceptual, lived sensibility, like a vibration or physical agitation. It is a feltness that is not one-sided but is instead a heightened physical sensation of otherness and reciprocity, expressed through an exchange between people or entities in an embodied connection [153]. It is the simultaneous intuition of another person and their involvement in some mutual activity or space (virtual or in-person). It is felt when one is engaged emotionally, intellectually, playfully or otherwise taking part in an experience with others, having the feeling of others also engaged in it with one. This connection is key and it is experienced as an emotional one even if those in the exchange are not in a close relationship (i.e. with an uninvolved audience member and an actor on stage). This engagement and felt or intuitive sense of others in time and/or space (physical or virtual) transcends physical attendance, including time but not necessarily space [154]. This assumes that individuals have an awareness or sense of themselves in their bodies in space and time and what Merleau-Ponty calls body-image (2002/1962: 100102) as a reference to gauge otherness outside themselves such as seeing the outline of ones body next to someone elses or catching a glimpse of oneself in the mirror next to others. When a memory or idea surfaces in our minds, our emotions can be physically felt as a lump in the throat, a flushed face or other bodily sensations, which might be thought of as inward sensing. In this context, presence can be experienced as extended body/consciousness or awareness [155], as an ephemeral non-corporeal embodiment; while the flesh is not directly stretched over the distance; the mind and body are engaged by both parties. It is like the concept of telepathy, but when transmitted by technology, such as through a telephone line or digital network, it is as if someones consciousness reaches through a camera or videoconferencing apparatus. Or it can be expressed by one and felt by another, with the aid of actuators or mechanisms in a head-mounted apparatus, or visually expressed by the actions of an avatar within a virtual environment [156]. These technological

152 This is the connection between emotions and intensity, which Massumi (2002: 25) describes, and emotions in terms of the neurological necessity (and physical) connection Lewis et al. focus on (Lewis et al., 2000: 7475), also discussed in Chapter 8 on embodiment. 153 Others study this from the perspective of VR and CAVE environments, such as the presence project out of Stanford University, the University of Exeter and University College, London. See http://presence.stanford.edu/ for more on that project. 154 Such as a text message or video Skype exchange, taking place in different time zones but at the same moment if the interaction or conversation is very personal in content, presence is felt through the vehicle of emotion. 155 In reference to Rupert Sheldrakes theory of the Extended Mind in his book A Sense of Being Stared At. 156 I attended the Performing Presence Conference at the University of Exeter in March 2009, a critical conference for my work on presence and performance using technology. I realised others were coming to similar conclusions on these concepts as I have, and that in-person still is the gold standard of presence for the best reception of performance, and most other communication. For subsequent conference proceedings, information is available online at http://projects.exeter.ac.uk/performingpresence/con.ann.php.

132

versions are the most recently studied instantiations of the qualities of presence [157] as they are commonly manifested within digital performance, virtual reality or other socially networked contexts and visually represented technological experiences [158]. This presence represents a togetherness without necessarily actually being in the same physical space or place. It is merely the awareness of another. But what is this awareness? How do we become aware? How is this awareness related to the extended mind-body, and how might they be connected to virtual reality? The proposal here is that there is an emotional investment [159], even in a performance context (through attended expectation and anticipation), that binds the interactors together, if only temporarily. This enables the exchange to occur and facilitates the sense of feltness of the other. Perhaps this is why it is difficult to describe presence and liveness with language. We can say that this presence is the visceral sensation and knowledge of an other, out there somewhere, one who is alive and aware of you, who is thinking of you and you them, either consciously or unconsciously, and who senses you and is reaching out to you, consciously or unconsciously, emotionally or cognitively, physically or virtually, and who is concerned for you and you for them [160]. This experience is also akin to the expression of, and experience of, love. It is the invisible connective tissue that Kozel speaks of, or the threads in my analogy.

7.2/ emotional presence

In A General Theory of Love, psychiatrists Thomas Lewis, Fari Amini and Richard Lannon add to and support the original behavioural psychology theory of Attachment, made famous by Bowlby [161] and others since the 1950s. Attachment theory (Bowlby, 1988: 135) states that throughout our lives we need physical affection and emotional bonding (Lewis et al., 2000: 6970). Attachment theory is important to this discourse on presence and emotional connection, as it asserts that the human brain is neurologically formed by physical interaction, affection and attachment (6970), and that mammals evolved attachment as a survival mechanism. Lewis et al. focus on the evolutionary and neuroscientific
157 Ralph Schroeder did research on presence experienced in shared virtual environments and discusses it in his paper Being There Together and the Future of Connected Presence (2006). Friedman, Brogni, Guger, Antly, Steed and Slater did a metaanalysis of other research on presence in virtual environments and state that: We regard presence as a successful substitution of real sensory data by artificially generated sensory data (2006). I see this as a limited and narrow definition, completely computercentric, and hence not part of my argument. 158 See more on this through the Performing Presence: From the Live to the Simulated research project (completed in 2009), managed by Nick Kaye (Exeter, UK), Gabriella Giannachi (Exeter, UK), Mel Slater (University College, London) and Michael Shanks (Stanford, USA), (Online) available at http://presence.stanford.edu/. 159 Some might be uncomfortable with this word due to its financial connotations, yet I cannot find a more appropriate word to emphasise the commitment involved the time and energy devoted to the activity. 160 This knowledge is based on prior relationship with that person and what they have expressed of course see reference to Lewis et al. and Sheldrake later in the chapter. This experience would be mere projection if this prior knowledge was not established in advance. 161 See his books A Secure Base (1988) and Attachment and Loss, Volume 1: Attachment (1986), in which he discusses the theory, how he developed it and its initial poor reception in the Freudian-focussed era of the 1950s.

133

development of the limbic system and its role in attachment. They provide in-depth analysis of the neuro-cognitive make-up of the brain in mammals, including humans, explaining the functioning of the limbic system in the brain and how in humans, unlike other mammals, the limbic system, central to the control of emotional responses, is not self-regulating, but open. This fundamental developmental survival process starts with the intimate bond with our mothers (7475) and subsequently transfers onto romantic partners (206208), enabling our continued neurological development throughout life. They also show that improperly nurtured children, and later adults, may have limbic impairments, disabling them from healthily engaging in relationships. This indicates that we need continual intimate interaction with others through physical contact in order for the brain to be properly and continuously regulated. To emphasise this they say:
The science of our day is allowing us to understand what interdependence is for [] We are attached to keep our brains on track, in a process that begins before birth and sustains until it end [] attachment changes a young mammal forever, as limbic regulation carves enduring patterns of knowledge in the developing circuits of the mind. (2000: 9899)

The argument is that babies are born with open limbic systems that are not regulated entirely on their own. They need their mothers physically close to them to slowly regulate their limbic system (2007: 74). This is explained as a two-way exchange, as the mother also needs the limbic connection and is in turn regulated by the baby:
As the nervous system matures, a baby reclaims some regulatory processes and performs them autonomously. Even after a peak parenting experience, children never transition to a fully self-tuning physiology. Adults remain social animals: they continue to require a source of stabilization outside themselves. That open loop design means that in some important ways, people cannot be stable on their own not should or shouldn't be, but can't be [...] Stability means finding people who regulate you well and staying near them [...] We recognize instinctively that healthy humans are not loners. (2000: 86)

According to Attachment Theory, not only is in-person, physical presence essential for clear communication and interaction, but essential for survival and the ongoing physical health of individuals [162]. The authors point out that this basic form of presence is primary to the brains ongoing development:
Ongoing exposure to one persons Attractors does not merely activate neural patterns in another it strengthens it In a relationship, one mind revises another [ they] remodel the emotional parts of the people we love, as our Attractors activate certain limbic pathways, and the brains inexorable memory mechanism reinforces them. (2000: 144)

Following on from this line of thinking, Lewis et al. might suggest that there is no point in trying to connect remotely if there is no solid relationship in person as well. Friends have claimed that they can
162 Lewis et al. cite numerous medical studies and reports on how social contact and relationships affect health, such as Dean Ornishs Love and Survival: 8 Pathways to Intimacy and Health (1999: 80), which surveyed medical literature on the relationships between isolation and human mortality. There are many others cited throughout the book.

134

feel a sensation of transmitting or receiving warmth from each other through email, texts messages and Skype calls. Yet only a fraction of this experience can be sent or portrayed; just as a photo cannot capture the experience of a natural vista or just as text message content or a voice conversation can only convey a fraction of the totality of the communication exchange. Lewis et al. address this:
Relatedness is a physiological process that, like digestion or bone growth, admits no plausible acceleration [] Advances in communication technology foster a false fantasy of togetherness by transmitting the impression of contact phone calls, faxes, email without its substance. (2000: 205)

Thus, the authors suggest that to maintain relationships and communicate effectively (not only in conflict, but in any type of communicative process), distance must be reduced as much as possible, since technological substitutions are unable to fill the gap of physically felt absence (in-person, corporeal presence). With this evidence, why would one attempt to replace a physical presence? The answer is that we have set up our society to be more and more globalised, with people living further and further from loved ones or friends for the purposes of work, school or pleasure, and technology offers options that did not exist before. As a result, we have become more dependent on digital forms of connection. Lewis et al.s interpretation of Bowlbys Attachment Theory is crucial to my argument, as it bolsters claims that emotional and physical attachment, and even anticipation (which might be the case for a performance) of interaction are required for presence to be experienced. Lewis et al. are not merely saying that in-person interaction is an imperative because it is more immediate or enjoyable, but also that:
Because loving is a reciprocal physiologic influence [] Limbic regulation affords lovers the ability to modulate each others emotions, neurophysiology, hormonal status, immune function, sleep rhythms, and stability. If one leaves on a trip, the other may suffer insomnia, a delayed menstrual cycle, a cold that would have been fought off in the fortified state of togetherness. (Lewis et al., 2000: 207208)

They point out that this basic form of presence is key to the brains ongoing development, and:
Because limbic resonance and regulation join human minds together in a continuous exchange of influential signals, every brain is part of a local network that shares information [] Limbic Attractors thus exert a distorting force not only within the brain that produces them but also on the limbic networks of others calling forth compatible memories, emotional states, and styles of relatedness to them [] Each relationship is a binary star, a burning flux of exchanged force fields, the deep and ancient influences emanating and felt, felt and emanating. (Lewis et al., 2000: 142, my emphasis)

This expressive notion of emanating and felt, felt and emanating feeds well into Sheldrakes theory of the extended mind and the common experience of feeling the presence of a distant loved one through

135

an awareness that they exist and breathe somewhere, even if they have not been in recent contact [163]. There is clearly a case for an interweaving of the concepts, theory and phenomena of Attachment Theory (Lewis et al., 2000) with that of the Extended Mind (Sheldrake, 2002), with respect to understanding how and why telepathy may (or may not) be experienced more widely by people who are emotionally intimate (children with parents, lovers, siblings, best friends, etc) and have established a physical connection. It could be argued, based on these theorists work, that we evolved telepathically to stay attached to each other for survival. Perhaps, through Morphic Resonance (2002) [164], as Sheldrake argues, we have evolved to be able to sustain connection and interaction through this embodied presence due to our past physical interactions (combining these two evolutionary theories) between our loved ones and ourselves when apart.

7.3/ distance + intimacy: mobile digital presence + connection

A key question for this research has been, what does it mean to FEEL someone through the phone, the Internet, artwork, or digital media? How does that translate into a mobile art/performance project? This research has intertwined and overlapped with emotional and philosophical experiences of daily life, yet the art investigations are also intertwined with my personal life and vice versa [165], so through this project attempts have been made to make sense of both. This section is a reflection on that as it relates to presence in real and digital interactions. Musings on remote and virtual presence from past experiences with a distant loved one revealed that the sense of presence exists due to, and is interconnected with, intimacy, emotional attachment and love. With these notions in mind, some poetic threads surfaced along the way:
presence vibrates in a sensation, an intuition, an indescribable touch or caress of another, a knowledge of you, a connection, the invisible thread in the ether, an emotional bond that links me to you it is now, but anywhere,

163 This speaks to the notion that mothers whose children have gone missing can feel that their child is still alive somewhere. This is the topic of the recent Hollywood film Changeling, based upon a true case in the US in the 1920s, called the Wineville Chicken Coop Murders. More is available online at http://www.riversideca.gov/library/history_aids_northcutt.asp (Accessed May 6, 2010). 164 According to Sheldrake, Morphic Resonance is simply an inherent memory or a form of DNA learning that crosses generations. It [] occurs between patterns of activity in self-organizing systems on the basis of similarity, irrespective of their distance apart. Morphic Resonance works across space and across time, from the past to the present. Through morphic resonance, each member of a species both draws upon and contributes to a collective memory of the species (2003: 278). 165 As other art researchers quoted in Chapter 2 also have found. In a research methods class early on in my PhD studies, we were told that ones personal life inevitably shapes and influences ones research, and to take note of that.

136

it has no material quality or manifestation, no shape, no specific language, it is part of the non-quantifiable realm, it is only felt and known to the intended receiver/sender. presence is you, here, beside me and the heat of your body, your touch, your breath on my face, your voice in low tones telling me all your thoughts, opinions, feelings. it is holding my hand as we walk down the street and pulling me close when we stop. it is you caressing my arm absent-mindedly as we ride the train, just to stay connected. it is electricity between us when we are together, and the regular emails and texts exuding love and affection when we are not. presence is the emotional thread, that invisible, sticky, spider-like web that holds us to each other when we are both together and apart. it is the way I feel when I hear your voice on the phone, instantly feeling a warm overwhelming joy, love, and longing to see and touch you. it is the chemical explosion when our lips touch and my heart beats faster and all my body tingles with love and desire. [166]

An awareness of the sense of desperate isolation and disconnection that many people experience [167], along with a deep empathy to help others, is behind the drive to find ways to communicate in more felt ways, to connect more deeply. Facebook [168] and other social networking tools are poignant examples of a collective urge to connect, share, express and experience collectively to have some sort of feeling of connection to others in the vacuum we may feel in our lives, work and experiences. A friend once told me that she was very unhappy with the 'poke' or 'x-me' applications in Facebook. I sympathised, since Facebook seems to me to be a fun but superficial means to experience a digital connection or show affection to distant friends and family it is not a touch or a physical hug, but only our outer self being shared. Our fears and trials are not shared, merely our 'best face' is posted. How can we find a way to share what we feel physically, viscerally, emotionally and deeply within these social digital realms when we are distant from our friends and loved ones? While we all sit alone in front of our computers in our offices at home or at work? To whom, when and how can we
166 This may appear to imply a connection with a person already known. This model of telepathy can rarely be experienced in the same way with someone new or a stranger since, as Sheldrake and Ullman, Krippner and Vaughan show, most telepathy occurs between people who are intimate, like family. 167 While writing this, a new report on loneliness from the UK Mental Health Foundation was released to show that 1 in 10 people feel lonely today and half are getting lonelier. This report is available online at http://www.mentalhealth.org.uk/media/newsreleases/news-releases-2010/25-may-2010/ (Accessed June 1st, 2010). Also the popularity of people flocking to social networking sites and tool should be an indicator of people feeling the need to connect. A piece of research on the effects of social networking on depression has already been done, see the article Potential Benefits and Harms of a Peer Support Social Network Service on the Internet for People with Depressive Tendencies: Qualitative content analysis and social network analysis (2009), by Takahashi Y, Uchida C, Miyaki K, Sakai M, Shimbo T, Nakayama T. Available at http://www.ncbi.nlm.nih.gov/pubmed/19632979 (Accessed May 31, 2010). 168 Statistics for social network popularity and penetration in the UK from December 2007 to Dec 2009 can be found online at http://www.clickymedia.co.uk/2010/02/social-media-statistics-february-2010/ (Accessed May 31, 2010). Sherry Turkle studied online behaviour in depth, in her book Life on screen: Identity in the Age of the Internet (1997), dealing with online identity and interaction, pre-social networking and Internet dating tools.

137

span the vast expanses? How do we truly share our presence, liveness, touch? These questions have driven me to continue to explore and express, and seek to enable others to share through digital technologies. Will these technologies solve the question of who out there will share? No, not likely. Will the feeling of isolation diminish if we have more tools with which to share the deeper parts of ourselves more viscerally? Perhaps. What does it mean to be connected emotionally, socially, or even in work or business contexts? There must be a more personal way to connect than telephone, Internet, and other media. Such a connection has to be more directly felt emotionally and physically, in the cells of our corporeal being or flesh. It should be experienced though gut sensation, from real interaction and the experience of being 'touched', either by an emotional closeness to another or an actual physical embrace, or both. The connection must be experienced as real. As discussed previously, we know that babies become severely developmentally delayed, and can die, if not touched and cuddled regularly. Premature babies still need regular touching when in their isolated incubators in order to regulate their bodies. As adults, we need this too, but as we live more densely together in growing metropolises, we are actually becoming more isolated, losing 'touch' more and more with family, friends and tangible relationships that are present in the physical sense. We work more and are reverting to or reaching for our devices and technologies more often in order to find a connection, a network, or as Kozel calls it, the 'connective tissue' of friends and family (2007: 2831). A friend once told me that his best friend was a guy in New York, whom he had only met online and played video games with daily until he went in person to the friends wedding. The friend had also found his wife through online dating. One of my closest friends lives in Denmark and we Skype regularly. I have a video Skype call with my parents once or twice a week, so that they can see how I am from halfway across the world, even if they cannot actually hug me. I find myself not accepting friends on Facebook if I have not met them in person. Yet I still want in-person, live 'face-time or experience, and to know someone in the flesh. Most people still want in-person interaction, not only in personal relationships, but also in business, to assess people instinctively. So, is it possible to create technologies that can exude this fleshiness, this touch? The porn industry has tried to exploit this type of virtual tactility not with emotional intent, merely for superficial sexual entertainment value and commercial purposes. What I am focused on here is how we can share our thoughts and emotions and/or experience each other more deeply and viscerally, when we are apart or at a distance for work, school or other reasons both for everyday life and in art and performance.

138

7.4/ presence and telepathy

As a controversial biologist, Rupert Sheldrakes theory of telepathy posited in The Sense of Being Stared At (2003) is based on his own and others multitude of double-blind human, biological and animal studies on extrasensory experiences. In his book, Sheldrake discusses how telepathy might be a sense evolved to assist with survival in the wild and over distance [169]. He puts forward his theory of Morphic Resonance also called Morphic Fields which says that cellular memory exists over generations, allowing animals to perform better at tasks with each generation. The former theory is more relevant to my work (2003: 17111), suggesting that telepathy is a form of embodied and virtual presence, influenced by emotional interconnection, which I try to emulate (not create) through digital, networked interaction. Sheldrakes work, and that of other scientists studies (Radin, 1997; and Wolf, 1995), have been covered extensively in previous art research (Baker, 2004), so will not be revisited here in great detail. Sheldrakes theories are echoed in my own experience and conception of how telepathy works and can be understood. My fascination with telepathic experience here, and desire to engage or enable others to exchange telepathically, is predicated on actual exchanges of feelings of love with distant friends and family. The term telepathy [170] is used here to mean sending empathically, and in an embodied way, thoughts, feelings, experiences and sensations to an intended other. Within close relationships, I have experienced a knowing, an intuition or precognition about loved ones in the form of an uncanny intuition or foresight of something about to happen. This would manifest as something like: X is calling me now, even though my ringer is off, or a knowing that something is about to happen to them, only to discover later that it had. Such was the case many years ago, when I had a dream that my mother was about to have her wallet stolen, then the next evening she reported that it had been. Similarly, even though she had no terminal illness in 1988, I knew she would die within a year. This could be considered inconsequential; however, Radin (1997), with Sheldrake among others, demonstrated in his meta-research on all previous research on telepathy and other forms of extrasensory perception over the last 150 years or so that this experience is common, even if not accepted by mainstream, conventional science. Sheldrake has also found that most people have experienced the sensation of being watched, which he calls the seventh sense (2003: 416). He has found numerous cases of people whose jobs depended on honing this sense, such as private investigators, martial artists and soldiers. He has also discovered that the vast majority of cases of telepathy that featured any great frequency and accuracy occurred between people who were emotionally involved, like family, lovers and friends (2003: 52). In my previous exploration of telepathy through interactive media, I was also concerned with the concept of the extended mind, noting that,
169 Sheldrake says it is [] a kind of ESP, or extrasensory perception a form of perceptions beyond the known senses [ that] contradict the assumption that the mind is confined to the brain (2003: 19). 170 Sheldrake says the Greeks define it as distant feeling (2002: 19); coined in 1882 by W.H. Myers, to describe feeling at a distance (Ullman, Krippner and Vaughan, 2002: 10).

139

Sheldrake questions how we form conscious images inside our heads, within the standard neurophysiological framework of vision. His theory of vision and optics states that, our eyes not only receive images and light, but they also reach out to or extend outward into the world to collect images and information, and touch the outer environment, like a two-way projection system [] (Baker, 2004: 22)

Telepathy is a technique, sense or skill that is not widely understood or practiced actively. Evidence has demonstrated its existence [171] as a method to traverse distance, discreetly and invisibly with intimate others. It is tied to emotional connection or personal bonds and is the ultimate instance of remote presence. Radin, Sheldrake, Ullman et al. and many other scientists have shown that the key precondition for telepathy to work is an intimacy or close relationship [172], a deeply felt and engaged emotional bond between siblings, parent-child, lovers, etc., originally established during in-person exchanges, through close interaction, proximity or intimate relations. Thus, for the phenomenon of presence and feltness of others to be experienced ephemerally, it must originate from a pre-established physical connection, a mutually felt emotional exchange. In addition, there needs to be an openmindedness on the part of participants, an attitude of acceptance, a trust to receive and experience it and to be confident that there exists an authenticity of and belief in the experience; a suspension of disbelief, if you will. This is the same with any bodily or emotional sensation experienced in daily life. Perhaps all communication involves this trust and openness in order for the intended recipient to fully understand its content. Sheldrake believes that before all modern forms of communication and technology, body language, gestures and other forms of physical, non-verbal communication were all that was possible between humans when in proximity to each other (2003: 67, 7678). But as a bridge over vast distances he also hypothesises that telepathy evolved in all animals and humans (2003: 111 113). He also notes that telepathic occurrences increase during altered states of consciousness, such as meditation or dream states (2003: 78). In my experience, confounding the problem with telepathic exchange is that one cannot always know if others are actively sending messages; unless they have confirmation from the other person involved, this is not always possible if the first person did not consciously send the second person their message, which is very common. Convincing evidence shows that most people (and animals according to Sheldrake) have the potential to send and receive telepathically with others, as much as we have the capacity to sing. Yet it is not clear how much control we have over this sense or capacity, conscious or unconsciously, since few practice while others fake it for profit. I am not trying to prove telepathy, as I am not a scientist; rather, I am using it as a conceptual model for my practice. Motivated by my own conviction of the existence of telepathy as a human and animal ability, I practice
171 See Radin (1997); Ullman, Krippner and Vaughan (2002); Sheldrake (2003) and others. In each you can see their exhaustive studies and meta-studies of others experiments and research in this area, proving that telepathy exists (whether sceptics believe all the thousands of studies is something else), where they look at the many manifestations and occurrences, in a vast number of human and animal populations, where it occurs. 172 See previous research: Baker, C. (2004), Internal Networks: Telepathy Meets Technology in the DreamPod. Sheldrake discusses it (2002: 49-53) and Arnold Mindell, mentions it in his book Quantum Mind: The Edge Between Physics and Psychology (2000: 239).

140

sending my thoughts and love to people I care about (this is different from wishes or prayers [173]). I do this by visualising them while sending thoughts and feelings that I would like to express to them when they are not around, or when words are inadequate. Sometimes it seems that words cannot express the flutter that one feels when one sees a photo of someone one loves, or the visceral sensation of longing to wish them well, or a desire for them to visit or call. Sometimes one wishes not to have to ask for those things overtly. How tremendous it might be if we could just put our hands on each others heads or hearts and empathically or intentionally send and receive memories and emotions that we are experiencing to each other, without having to explain or badly translate these feelings and sensations into words. If we could, we might communicate with, understand and therefore treat each other better. How empowering it might be (or disempowering for some perhaps) to have a world where we could selectively read each others minds and bodies, in a more direct and honest way. This direct reading does not bypass the body to the mind, but assumes that the body and mind are one and that embodiment can be stretched beyond the confines of the skin. This vision of mind-reading incorporates the knowledge from the body necessary for such and complete reading to be possible, including nonverbal and gestural modes of communication. Dream space is a state of consciousness that can act as a conduit, or portal, for telepathy [174], as can near-death experiences (NDEs) (Beauregard, 2007: 158) [175]. This dream space is also where the consciousness is freer and more open (Wolf, 1994: 178, 180187; Sheldrake, 2003: 78; Ullman et al., 2002). Presence experienced in the alternate state of consciousness of dream or pre-sleep is elusive and hard to define [176]. My personal experience of it is as a permeable, thick viscosity of fog, where images, sounds and people just appear in seemingly disconnected ways that somehow make sense in context to the dreamer. Here the presence of others can often be felt, even while ones awareness is able to transition easily between wakefulness, the hypnogogic state and sleep. Below is a journal entry, musing about dreaming:
Dreaming is a state of exploration, to explore your current physical and emotional state, yet to find yourself in familiar places, but not, such as a shoe or clothing stores, with the most exotic and beautiful designs I could never otherwise think of; or in strange and horrible places such as a torture taking place in huge old house or hotel; or hallways with closed doors that seem familiar, but not; or with faceless lovers. But there's always a desire not to wake, to lay there and continue to explore the most recent dream, continue it, add to it, share it somehow, climb within it, keep playing the role I was in when I woke, lingering, hoping to fall asleep again and get carried away again but to stay aware, lucid (almost never happens). Dreaming is like being pulled into a movie that you sort of, or totally, know all the characters and can feel
173 The difference is that with a wish or prayer you want something to occur, whereas with telepathy it is an act of direct (if not always intentional) communication, not a hope, want or desire for something to happen. It is like calling someone versus hoping they will call you. In distant healing, groups of people actively wish and pray for someones recovery. This is sending a desire for change, not a message. See more from the Noetic Sciences Institute on distant healing online at http://www.noetic.org/research/dh/explore.html and http://www.noetic.org/research/overview/ and http://www.annals.org/content/132/11/903.abstract. 174 Again Ullman et al. discuss their research on this and it has also been found by Sheldrake and Radin in their studies. 175 Beauregard discusses this in the context of NDEs and encounters with deceased loves ones. 176 Farthing calls this transitional state between sleeping and waking hypnogogic (1992: 205209).

141

what they feel, but this movie has twists and turns you don't know are coming, yet feel compelled to experience for good or ill, and often the dream goes in such a strange direction that you wouldn't believe possible.

Throughout this research one goal, coming from long-held interests, was to encourage participants sensory awareness and mind-networking abilities in more controlled yet creative and playful ways [177]. This could be done in a way much like a visual phone call, to enable people to explore quietness and intimacy as a means to connect outwardly with others remotely. I explore this concept, using biosensor and media interfaces, to reach out through the virtual, or invisible, in-between space to find others to share and exchange with, in an embodied and affective experience.

7.5/ collective consciousness, spiritual presence and out-of-body-experiences

This artistic research attempts a small-scale version of Lvys (1998) collective consciousness [178], crossing boundaries of conceptual exploration and practical implementation. It does so by remixing or re-forming mental images and sensations into new collective or universal narratives of living, being, experiencing, emoting, expressing, feeling, performing and communicating via mobile media performance. Collective communication is facilitated during the collaborative remixing of the media elements of the performance, and the use of a multi-interaction approach. This investigation can also be seen to cross the boundaries of mind/body philosophies within the integrated individual body approach, because I posit that while the body is the fundamental site of the mind, it has been shown that the embodied mind can detach from the body, to witness and do things the body cannot otherwise physically do [179]. Shusterman refers to how Merleau-Ponty thinks we are limited because we cannot see outside of ourselves (Shusterman, 2008: 65). Yet doctors and other scientists have studied numerous patients who have had NDEs (Beauregard, 2007: 153-166), reporting out-of-body detachment, seeing their own bodies from high above themselves. I have had this experience as well. Others have researched how many people have had the experience, in a moment of crisis, of feeling and allowing themselves to be guided by a third man [180]. This is reported as occurring when one is under great threat of permanent detachment from ones body, through death. In the CBC Radio documentary The Third Man Factor (January 2009), it was found that many people had experienced this presence of a third man, guiding them to safety in times of extreme distress. One of the examples given was of a woman who was diving in the deep ocean and got lost in an underwater cave just as she was running extremely low on oxygen. She stated that she
177 My previous art research project sought to assist people to make a telepathic interchange. 178 Also in the Jungian sense, but more referencing the studies on modern physicists concepts of collective consciousness, such as Fred Alan Wolf, mentioned briefly herein. 179 I also have personally experienced this, and as this research comes from phenomenologically driven lived experience, my own experience is as valid as any research. 180 CBC Radio (January 27, 2009), The Current, documentary report The Third Man Factor, Toronto, Canada: The Canadian Broadcasting Corporation.

142

suddenly lost her fear and felt like she had her dead husband guiding her to the opening just before her tank ran out of air (CBC, 2009). What is astounding is that NDE accounts often cite that people have a review of their lives in the form of a visual experience, and during that review, not only do they know their own thoughts from their own viewpoints while seeing certain events of their lives again, but they also intuitively know the thoughts and feelings of everyone involved in the event, including knowing others points of view for the first time (Beauregard, 2007: 157158). So how does this spiritual sense of presence fit in? The Spiritual Brain: A Neuroscientists Case for the Existence of the Soul (2007) by Canadian neuroscientist Mario Beauregard with journalist Denise OLeary, discusses this and inspired aspects of my research. This discovery was similar to a revelation that I had several years ago upon reading the research of astrophysicists, physicists, Sheldrake and parapsychologists namely, that there are many scientists who are ready for a non-materialist paradigm shift. There are maverick scientists who have done numerous, credible studies to show that traditionally accepted materialist science does not have all the answers. These scientists believe, through their own investigations, that there are forces we cannot see or measure and that many mysteries of the universe cannot be understood by conventional science alone. These scientists are often shunned for sharing their discoveries (though less and less in recent years) [181]. But just because their ideas are unconventional does not mean they are religious fanatics, or Intelligent Designers. Nor does it mean that the universe beyond our consciousness or that the boundaries of our bodies present a duality. Nor is the overall debate a binary one. Nor does one have to take sides either atheist, materialist sceptic or religious, dogmatic zealot in order to explore these mysteries. That is why is it so refreshing to discover these voices. This work is about pushing the boundaries of communication through art and technology. In The Spiritual Brain, Beauregard presents reports describing bodies states first revealed by physician Van Lommel:
Van Lommel classified the experience his patients reported by type: Out-of-body experience (OBE). This is an experience of floating outside ones own body, while retaining ones identity and very clear consciousness. Most patients report looking down from above. As we have seen, in some cases, patients have reported information that was later verified. Holographic life review. In the popular phrase, I felt my whole life passing before my eyes. As Lommel describes it: [] All that has been done and thought seems to be significant and stored. Insight is obtained about whether love was given or on the contrary withheld. Because one is connected with the memories, emotions, and consciousness of another person, you experience the consequences of your own thoughts, words, and actions to that other person at the very moment in the past that they occurred. Encounter with deceased relations or friends. Deceased persons are recognised by their remembered appearance, but communications appears to be through direct transfer of thoughts.
181 Many have joined the Center for Consciousness in Tucson, Arizona, which studies quantum physics and the mind, neuroscience and philosophy of consciousness, and focuses on the concerns presented here. More on the centre is available online at www.consciousness.arizona.edu/ (Accessed May 30, 2010).

143

Return to the body. Some patients learn, through wordless communication with a Being of Light or deceased relative that they ought to return to life [...] (Beauregard, 2007: 157158, my emphasis)

It should be noted that Beauregard as a neuroscientist is more conservative in his claims than some other scientists mentioned herein, and goes to great lengths to qualify his statements. However, he demonstrates that the mind does not exist in only one part of the brain or in the body, but that it can be distributed beyond the corporeal flesh. He shows how mystics report feeling the experience of profound, full body sensations during transcendental or spiritual visitations of other presences:
Typically, a very intense mystical experience occurs only once or twice in a lifetime of contemplation. It can also include a number of other elements, such as the sense of having touched the ultimate ground of reality, the sense of the incommunicability of the experience, the sense of unity, the experience of timelessness and spacelessness, and the sense of union with humankind and the universe as well as feelings of positive affect, peace, joy, and unconditional love. (Beauregard, 2007: 262)

When trying to understand these experiences scientifically, he says:

[] some have argued that RSMEs [religious, spiritual, mystical experiences] are simply emotional experiences and nothing more. Distinguishing in a definitive way between two kinds of states and experiences would be useful. No it wont tell us whether God exists, but it may help us determine whether people who have mystical experiences enter a state of altered consciousness that is mostly related to emotion. (Beauregard, 2007: 268)

Even if these mystics sense of other presences cannot be proven in a materially tangible way [182], it cannot be denied that these people, who are not diagnosed as mentally ill, can and do report experiencing many profound, embodied sensations resulting from what they see as detached forms or manifestations of presence. This supports other research and claims revealing that there are a great many other ephemeral and experiential phenomena that we cannot explain, but cannot simply deny or easily dismiss. Even though these inexplicable phenomena might be rejected as the machinations of crazy people (or the possessed, in past times), such sceptical claims have since been disproved. Should we explore these experiences more seriously, as Sheldrake suggests in his talks [183], we might get further in solving some major world mysteries and problems of the world [184]. Similarly, Beauregard says,
[] once we understand the universe itself as a product of consciousness. We might expect living beings to evolve toward consciousness if consciousness underlies the universe. Consciousness is an irreducible quality [] But it will be stymied if the only purpose is to reduce consciousness to something it is not or to demonstrate that it is an illusion. (Beauregard, 2007: 277)

182 Although he took fMRI scans of Carmelite nuns (as discussed on the next page) to discover that it affects many different parts of their bodies with no two nuns having the same locations in the body light up in the scanner. (Beauregard, 2008: 266) 183 Sheldrake podcasts (online). 184 I suspect this, and new discoveries can lead to better understanding and solutions to problems.

144

Many physicists and other scientists mentioned here, as well as others in the yogic and eastern traditions [185], believe that the mind has the power to change and influence reality, and that our individual and collective consciousnesses can change reality. Whilst this is a controversial position to take, the idea can be seen to add weight to arguments for telepathy and other forms of embodied, non-corporeal presence, and in that sense can inform this thesis as a set of hypothetical and peripheral framing ideas. I do not cite this idea in order to make an extreme claim about the possible lack of material reality outside our own minds, but rather to suggest that as the placebo effect and numerous other phenomena seem to reveal our conscious minds may have the power, as shown in experimental physics, to influence reality [186]. The Spiritual Brain discusses and debunks much of the materialist science of the last few decades, which has sought to solve these mysteries by seeking the location of spirituality in the brain and body. Materialist science also tries to compartmentalise and create a disease, ailment, neurosis or genetic dysfunction of religious faith or spiritual practice to then cure it (much in the way that homosexuality was once regarded). Beauregard argues that science needs to shift how it thinks about mystical experience, that the scientific community, and indeed the general public, need to make a paradigm shift in how the unseen, invisible and immeasurable are viewed. Beauregard argues that materialism is limited and that proponents are as religious in their fervour as those who practice religion. He proposes that there must be something in between; that we must move away from this absolutist mentality to embrace unseen mysteries, seeking to understand and accept them, rather than pigeonholing them, filing the belief in their studies into the Diagnostic and Statistical Manual of Mental Disorders [187]. The Spiritual Brain is not an argument for the existence of God, but an argument for examining mystical experience more closely, and a validation of it as a real, felt, embodied, human experience; whether one is religious or not. Along with quantum mechanics and quantum physics, these invisible experiences might suggest that there are other options for how we explore and understand the world, beyond the science-versus-religion debate. Beauregard explains his own research, performing MRI scans on Carmelite nuns bodies and brains during their transcendental or relived transcendental experiences. He discovers and discusses these experiences as similar in magnitude and kind to those of other visceral experiences, such as falling in love, achieving orgasm or post-natal hormonal experiences, which take over the mind and body in a wash of chemical and electrical intensity. He says that mystical rapture is experienced with possibly more intensity than all of these combined. Beauregards work is highly relevant here, as it helps frame my own views on the different forms and
185 See Pema Chodron (1994) and other Buddhist texts. 186 The Pauli Effect a well documented parapsychological influence, which was named after Wolfgang Pauli, who was rumoured to have had an effect on technical equipment, causing phenomena such as laboratory bottles suddenly exploding for no reason in his presence. He was therefore not allowed in certain spaces when experiments were being performed. It is also similar to the observer effect, or how the act of observing in quantum mechanics can change the state of an atom from a wave to a particle. (Mindell, 2000: 508509). 187 This text provides a touchstone for paradigm shifts over time, as a frequently updated but always influential bible of the psychiatric profession. The changing interpretations of the phenomena discussed in the text can be seen as a literal textual transition over time from belief in one set of controversial ideas or dogma to another.

145

experiences of presence and embodiment, both in-body and virtually. However, it must be restated that this is not intended to prove or disprove telepathy or other mystical experiences of ethereal presence. Rather, these ideas are put forward as conceptual explorations of felt experiences of presence; they are instances of physical presence at a distance. The logical trajectory from Beauregard leads to a discussion of instantiations of presence or other presences, including themes of detached consciousnesses phenomena such as ghosts, god, extended minds, or other versions or dimensions of being (Wolf; Sheldrake; CBC Report). The physicist Fred Allan Wolf (1994) believes that there are various dimensional planes of reality that we transcend to after death, which are not known to us in this plane of reality but which some have glimpsed during dreams states, during higher level meditation and NDEs, in quantum consciousness and in quantum states of reality, which our ordinary sense modalities cannot grasp (Wolf, 1994: 340). Wolfs proposition about the substantiality or physicality of presence bears mention here:
Quantum physics and relativity have indicated that much of reality is hidden and mysterious, and this would be directly apparent if we could directly experience quantum reality. The universe is bizarre, and mind and meaning are as important as matter and energy. The mind appears to be present in matter at many levels. Not only does the mind appear to be present in living complex organisms such as humans and animals, but also at the level of cells and even at the level of molecules and subatomic matter. (Wolf, 1994: 340-341).

This suggests that our minds can impact other aspects of reality. Other non-western cultures and religions call this dimension transcendence, pure consciousness, Godhead or sometimes reincarnation. Wolfs position suggests that once our bodies die, we are able to see this other level of quantum conscious reality, without flesh: a pure consciousness. So important are these ideas to my ongoing research, I have cited Wolf as a major influence on my thinking in previous research as well:
[Wolf] explains that consciousness is a non-physical or quantum part of the brain/body and each persons consciousness is part of a greater universal whole. This quantum consciousness facilitates non-local communication between people across space, time and matter, consequently, we all are connected to a main cloud of consciousness. This cloud is not necessarily a god-like entity, but understood in quantum physics as an energy field that contains actions of the imaginal, actual and real within space and time, which dematerializes and rematerializes in various locations as it is created and perceived. His concept is of an invisible network of people, connected through cords to a mainframe of one consciousness, which connects all our collective lives. (Baker, 2004: 28)

This point brings us back to a concept of collective consciousness, referred to in Chapter 7.1. Embodiment for Glen Mazis in his book Earthbodies: Rediscovering Our Planetary Senses (2002) involves more than one individual body, as he claims we are part of a much bigger body comprising the planet and the universe. He says:
[...] it is important to see that there can only be a 'space' of cyberspace because it is a place we can inhabit with our bodies in this fluid, energetic, and circulatory sense of earth body. (2002:

146

126)

The question of presence, which necessarily informs this thesis, leads to a number of psychological and phenomenological areas of study that extend, in full, far beyond the scope of this thesis. While these areas do not compose the body of ideas for examination in this thesis, they do bear mention here as framing principles or questions that beg at the edges of this more academic discourse. Such questions, for this researcher, necessarily entangle some element of the experienced, embodied, personalised presence into the more theoretical and critical edge of analysis (and here I must admit that, like many who are interested in these subjects, I am informed by a personal sense of presence replacing loss, after the death of a central influence and loved one whose once-living presence has taken other forms of reassuring instantiation in ideas and experiences over the years). To digress for a moment, just far enough to allow such important influences to weave themselves more visibly into this narrative, it is important to at least mention or highlight some of the human questions that often go unspoken, such as: Is it possible that we are all part of the more immense quantum body of the earth as Wolf and Mazis believe with elastic minds (Sheldrake) possessing the ability to transit into and out of our individual encased (and assigned) bodies, roaming out of them when need be? Is the larger concept and reality of time not more elastic than now and in this moment and tomorrow and yesterday as others presence can be always with us, in ever-mutating, changing and transmuting forms of space and time? I posit, based on personal experience of sharing such personal information with others over the years, that many people who have experienced the loss of loved ones describe these feelings, even long after their losses. In terms of this thesis, I argue that in analysing this sense of loss alongside the concept of presence, it is important to at least consider (though of course one can not prove such) the phenomenological experiences of the ubiquitous, as well as the more rarefied experiences described by physicists and philosophers. In this context, with just this much personal experience woven into the fabric of my argument as a highlighting thread, perhaps the more distanced theoretical approaches to presence can better be explored with reference to the personal within the phenomenological. Perhaps, then, it is possible to find personal touchstones as well as critical understandings when it is said that sensing presence is an evolutionary response to keep us from feeling alone, and to that extent functions as a tool for psychological survival (Lewis et al., 2001; CBC, 2009; Sheldrake, 2002). As such, a sense of presence of others over distance may also help us as animals in times of crisis or in extreme conditions, and in some cases may guide us to safety. Sheldrake suggests that it is a survival sense, just like our other senses (2002).

147

Beauregard (2008) discusses the minds ability to leave the body and still be conscious enough to connect with other dimensions (as in NDEs) and to heal or harm the body (as in the placebo and nocebo effects in medicine), against the imperative for materialist scientists to locate it within the brain or body. A greater sense of mind is often experienced during transcendental experiences, one of a supreme presence of otherness or of an intelligent consciousness, often interpreted as God. Transcendence is experienced throughout the body, yet no two people experience ultimate presence within the same parts of the body and brain. This shows, as Sheldrake and others do [188], that there is an elastic nature or relationship between the mind, consciousness, the body and other consciousnesses, bodies and presences. This is relevant to the enquiry into liveness and presence because it connects to the exploration of virtual presence and extended consciousness through the network, and this discussion demonstrates variations on extended, virtual and extended presence. This chapter has covered the concept of presence in a myriad of permutations and manifestations as relevant to this research. While it does not discuss all forms and meanings of presence impossible within the scope of one PhD it attempts to articulate and connect those that guide the thinking and development process through the duration of the MINDtouch performance project, which is the main case study for this thesis.

188 As a media artist working with complex and abstract concepts, I find important corollaries and research references here which add a certain substance or weight to the explorations of my work. I cite these references in this context as framing references, but do not claim that the thesis can in itself begin to validate or prove the theories or studies of the controversial ideas offered by these scientists.

148

8/ embodiment and affect

This chapter unpacks the notion of embodiment, and particularly embodiment of or through technology, a concept that is key to the activities in this performance research project. This chapter does not attempt to cover all instantiations of uses of the word embodiment, but covers those relevant to the work.

8.1/ embodiment towards a working definition

Presence is often associated with the notion and experience of embodiment. Through explorations of the concepts previously discussed on extended mind, morphic resonance (Sheldrake, 2003) and embodied out-of-body experiences, it is apparent that the nature of the mind/body relationship is still greatly unknown. This makes the notion of embodiment more elastic. Conceptual explorations and research on embodiment, here and in the past, have led me to the conclusion that the mind and body are entangled as one, encased within the skin and enveloping the bones and organs, blood, the brain and other organs, fluids and tissue. Yet embodiment also incorporates the presence of a being or a spirit, a consciousness or soul that seems to be separate from the corporeal or fleshy encasement of the body, but living within it [189]. This is especially true in conjunction with the concept of virtual embodiment. The separation between mind/body and soul/spirit is evident when one has had the experience of seeing a dead body at an open-casket funeral the body that is left cannot be seen as an embodied being any longer. Along these lines, some also believe that embodiment is consciousness itself (Rettie, 2005; Beauregard, 2007) or an ephemeral life force, rather than a physical quality. As with other major themes of philosophy, such as Why do we exist?; What is the meaning of life?; and What happens after death?, the nature of the mind and the role of the body have long been pondered. Philosophers have previously thought that reason, logic and the mind are separate from the body. Especially held high are the ideas of Descartes, otherwise known as Cartesian split theory (Lakoff and Johnson, 1999). Before neuroscientists had a chance to examine the brain and nervous system more closely, debunking some of these notions, philosophers like Heidegger, Bergson, Merleau-Ponty and others were seriously questioning the validity of Cartesian split theory. In Philosophy of the Flesh: The Embodied Mind and Its Challenge to Western Thought, Lakoff and Johnson use modern neuroscience to explore philosophy past and present in order to more forcefully challenge the Cartesian paradigm point by point (1999). Their views might at first appear to contradict my own and those herein, as they take a more materialist approach to embodiment and the mind. However, I see that both perspectives can co-exist and compliment each other. My position is not at all in disagreement with science as a whole; in fact, neuroscience, physics and medicine together can answer many of our long-pondered philosophical and spiritual questions, while affirming some previously held speculations.
189 Various religions and other spiritual practices like yoga go much more deeply into this, but this history and exploration is beyond the scope of this thesis.

149

Francisco J Varela, with Evan Thompson and Eleanor Rosch, delve deeply into an examination of embodiment, evident in the title of their book The Embodied Mind: Cognitive Science and Human Experience (1991). They focus on embodiment in terms of consciousness, perception and cognition, reviewing western and eastern traditions of understanding the self and the world. They look at phenomenology and Buddhist philosophy from a science or neuroscience perspective, infused with a western European philosophical perspective. Their contemplation of embodiment and perception has shaped my understanding of presence in performance in the network, and in view of telepathy with the following terms:
By embodied, we mean reflection in which body and mind have been brought together [...] reflection is not just on experience, but reflection is a form of experience itself [...] (Varela et al., 1991: 27)

The concerted effort to counter and knit the previous concepts of Cartesian split discourse is also evident here,
It is a matter of simple experience that our mind and body can be dissociated, that the mind can wander, that we can be unaware of where we are and what our body or mind are doing. But this situation, the habit of mindlessness, can be changed. Body and mind can be brought together. We can develop habits in which body and mind are fully coordinated [through meditation and mindfulness]. (1991: 28, my emphasis)

What is intriguing about this is the acknowledgement of a tendency or inclination of the mind to wander and leave the body. Instead of trying to state this as a disconnect, as Merleau-Ponty and Kozel do, the authors suggest that ancient eastern meditation techniques keep the body and mind integrated.
By using the term embodied we mean to highlight two points: first, that cognition depends upon the kinds of experience that come from having a body with various sensorimotor capacities, and second, that these individual sensorimotor capacities are themselves embedded in a more encompassing biological, psychological, and cultural context [ as such] sensory and motor processes, perception and action, are fundamentally inseparable in lived cognition [] they have also evolved together. (1991: 172173)

Thus, Varela et al. demonstrate that consciousness and cognition are dependant on the body in its embedded contexts and vice versa even if the mind wanders, it is pulled back to the body, to continue to contextualise the experience. Recalling the Near-Death-Experience (NDE), Out-of-BodyExperience (OBE) studies and accounts of peoples consciousnesses independently roaming in the last chapter, one is led to wonder how long this wandering can last, and if it is only short in duration, how is the mind reintegrated into the body? If many eastern philosophies [190] and religions are correct,
190 Fred Allan Wolf is one of those scientists who believes this is what happens, and that consciousness goes to another plane of reality, free from materiality, very similarly to reincarnation concepts of many religions. See his book The Dreaming Universe (1995).

150

perhaps upon death the mind is released more fully formed, from a lifetime of embodied, lived experience, to roam other realms or dimensions of consciousness. As a non-religious but nonmaterialist spiritual agnostic, this notion is worth entertaining, as many more respected non-materialist scientists, physicists and neuroscientists are also doing [191]. As we are cut off from this invisible realm, we can only speculate from what we do know. This notion is tangential so, while curious, it will not be discussed further here. When looking at embodiment from cognitive science, phenomenology, and Buddhism, Varela et al. state early on:
[] we will focus on one such tradition, that which derives from the Buddhist method of examining experience called mindfulness meditation. We believe that the Buddhist doctrines of no-self and nondualism that grew out this method have a significant contribution to make in a dialogue with cognitive science [] (1991: 21, their emphasis)

This third position of a no-self or non-dualism, of integration through meditation, is a very progressive approach, validating not only the mind/body connection but also a broader focus on phenomena inside the body and outside in the world. This focus might also include non-materialist phenomena such as telepathic and extrasensory experience, which are still not well understood. Varela et al. describe how meditation is used to unify the mind and body and how, with practice, one can train oneself to be more in control. This in turn helps the practitioner to tame her/his awareness and the minds restlessness in order to become a more integrated body and being (1991: 26). Varela et al. add to this discussion on the integration of cognitive science, phenomenology and mindfulness/meditation, saying:
What we are suggesting is a change in the nature of reflection from an abstract, disembodied activity to an embodied (mindful), open-ended reflection. By embodied, we mean reflection in which body and mind have been brought together. What this formulation intends to convey is that the reflection is not just on experience, but reflection is a form of experience itself and that reflective form of experience can be performed with mindfulness/awareness. [] In our usual training and practice as Western scientists and philosophers we obviously proceed differently. We ask, What is mind? What is body? and proceed to reflect theoretically and to investigate scientifically. [] By not including ourselves in the reflection, we pursue only a partial reflection, and our question becomes disembodied [] theoretically confined, preconceptually entrapped somewhere. (1991: 27, their emphasis)

One can also see that Kozels phenomenological method is remarkably similar to the mindfulness practices of Buddhism. One might even argue that it is a form of mindfulness, designed for the western philosopher or theorist who is not interested in other aspects or practices of eastern religion. In reference to non-dualism, Kozel emphatically states that there is no mind without the body and the
191 I am aware of many popular sceptics like Daniel Dennett, Susan Blackmore and others who refute these ideas, but since this is art not science practice I am not concerned with these arguments, except to acknowledge that they exist. The Spiritual Brian contains an in-depth debunking of the sceptical, materialist science and media perspective, but there is not scope to address these sceptical claims here.

151

Cartesian split is a simplistic distinction. She proposes:

[] we need to have a method and an intent to receive information from our own bodies and from the bodies of others. In giving voice to diverse corporeal states we can overcome nave distinctions between matter and spirit, between body and mind. (2007: 23)

So Kozel has taken this task on and created a method for performance media practitioners (and others). She explains that this is needed because the forms of embodiment of mind, spirit and flesh in performance are intertwined, as such:
The body is a weave of different materialities, the body is a dynamic process, the body navigates the world at the intersection of a cluster of languages (verbal, physical, archetypal, mnemonic, unconscious). It is electric, biological, and cultural. (2007: 33)

Kozel stresses that the complexity of this is the corporeal, conscious and embodied experience of humanness. In a similar vein, Massumi states that,
Mind and body are seen as two levels recapitulating the same image/expression event in different but parallel ways, ascending by degrees from the concrete to the incorporeal, holding to the same absent center of a now spectral and potentialized encounter. (2002: 32)

Elizabeth Grosz in Volatile Bodies (1994) adds her voice to the debate, emphasising the intertwinement of dualities rather than disjointed separation, easing my own concerns. She also explores MerleauPontys notions of the corporeal and flesh:
The concept of flesh is [...] not the union or compound of two substances, but thinkable by itself [...] While it does not displace perception as the thematic object of investigation [... it is] the condition of both seeing and being seen, of touching and being touched, and of their intermingling and possible integration, a commonness in which both subject and object participate, a single thing folded back on itself. (Grosz, 1994: 95)

This curiously points to the possibility of being both subjective and objective, as well as of the body having the ability to have two simultaneous perceptual experiences, yet still be able to distinguish the differences. The concept of embodiment is compelling here, as it suggests a process which takes place both inside and out, crossing boundaries of the senses. Therefore, we must consider that the perception and the perceiver are one. The notion of the permeability and flexibility of consciousness brings the discussion full-circle, as a return to and embracing of the idea of an elastic embodiment, which speaks to that of an extended mind as well (Grosz, 1994: 96). The concept of permeable boundaries of perception and the body suggests that the body can separately function entirely outside of the mind (and vice versa), but in an elastic sense, not detached entirely from the body but, as Varela et al. Suggest, informed by it. On this Massumi comes in to address the boundaries of the body:
Brain and skin form a resonating vessel. Stimulation turns inward, is folded into the body, except that there is no inside to be in, because the body is radically open, absorbing impulses

152

quicker than they can be perceived, and because the entire vibratory event is unconscious, out of mind. (2002: 29) [192]

Massumis focus is on the unconscious as a means to move out of mind (body) [193]. This speaks to the Sheldrake and Ullman et al. findings, namely that most telepathy and other experiences of ESP happen during sleeping and dreaming (see pp. 141 and 142 in the previous chapter on this). To summarise then: within MINDtouch as well as previous work, my enquiry into how embodied experiences can be embedded into media art practices aims to uncover how such mediated work can be experienced by the body to access memory and alter consciousness. I surmise that controlled telepathic interactions can begin at a sensory level, facilitated below conscious awareness. One approach was to create an experience that facilitated re-embodiment through altered states of consciousness, enabling participants receptivity to new types of input. The intention was to tap into deeper levels of sensory experience and physical and emotional responses or states in the mediated space, then to enhance sensory awareness and mind-networking abilities in that space. This was intended to enable people to make a telepathic interchange, like a telephone call, assisting them through solitude, quietness, intimacy and privacy using biosensor and media interfaces. Participants were guided to connect outward to others remotely through the virtual, or invisible, space (Baker, 2004). This was not done in the current work, which continues from there, but I step away from trying to stimulate telepathic exchange, assuming it happens for some, and instead make a technological simulation of the process, focussed on the content of the simulated exchange; perhaps for those who do not naturally experience it, to join the experience through a mediated means.

8.2/ embodiment of space / absence and the space between

In this section, it is important to address and define/refine the concepts of absence and the space between in relation to an emerging working definition of embodiment for the purposes of this thesis. The focus here is on practical engagements between bodies in space and conceptual bodies, with a focus on the notion of extending the body beyond our physical boundaries, through engagement with our innate telepathic abilities. The body can be engaged in practice by facilitating a digital, artificial simulation of telepathy, in order to extend a real and actual embodied presence. This can be achieved using simple ubiquitous technology tools such as mobile phones to bridge the space between. To contextualise these concepts, especially in relation to embodying technology in order to traverse space and to minimise the experience of absence and physical distance, a definition of space that can be used comes from Paul Dourishs Where the Action Is: The Foundations of Embodied Interaction:

192 This is an important point repeated from page 23, but used with more emphasis and context here. 193 To clarify: when I speak of consciousness I include the unconscious as part of the range of states of consciousness, just as there is a wide range of states of corporeality.

153

Space is largely concerned with physical properties (or metaphorical physical properties). It concerns how people and artifacts are configured in a setting; how far apart they are, how they interfere with lines of sight, how actions fall off at a distance, and so on. (2001: 89)

Personal musings and felt experiences of this idea of space between have exposed a space that is not always or necessarily an absence or void. It is felt as a transition or transmission; a conduit, medium, or channel to move from one place to another, much like an alley or passage. This feltness includes telepathic exchanges, memes [194] and electromagnetic vibrations, and also the Internet, SMS/Chat, mobile and other telecommunications more commonly experienced within virtual spaces. Similarly, the space between can resonate as a memory or the moment of struggling to remember something; a pause, hesitancy or tension. These are all elements that extend our bodies beyond themselves, both as human natural abilities or traits, as well as artificial creations or simulations. They are fused together or intermingling, becoming a space-between moments, objects or people. It may also be understood as breathing space, thought space, meditation, imagination, dream space, being in limbo, and negative space (although not necessarily negativity). The space between is a sense of sustain, floating, suspension, or nowhere-ness; the ringing before answering, or waiting before arriving. It is falling asleep or trying to wake, the process rather than the attainment, the longing for an experience rather than the experiencing itself. Brian Massumi calls the in-between space potential or possibility (2002: 95). He offers notions that the body is in an ever becoming state of potential, a state of being which never arrives, and is always virtually becoming originally a Bergson concept, picked up by Merleau-Ponty and others: The field of potential is the effect of the contingent intermixing [...] (2002: 76) Related to embodiment, space and the understanding the bodys experience of presence, we need to understand the space it occupies, and how it senses and defines itself in order to position otherness or presence of others in a physical and virtual, intangible or ephemeral space. Merleau-Ponty addresses this issue:
Bodily space can be distinguished from external space and envelop its parts instead of spreading them out [... and] the body image is [...] a way of stating that my body is in the world [...] By considering the body in movement, we can see better how it inhabits space (and possibly time) because movement is not limited to submitting passively to space and time, it actively assumes them, it takes them up in their basic significance [] (2002/1962: 100102)

Merleau-Pontys discussion here references his concept of the here-body, which becomes a recurring motif for him and others, like Don Idhe. Idhe, in Bodies in Technology (2002), uses it in terms of placing the integrated conscious flesh within space and time, as against the notion of the untethered, yet embodied there-body or extended mind-body, which leaves the confines of the material flesh encased by skin. Significant also is Merleau-Pontys notion of body-image, as well as body space or body in
194 The scientist Richard Dawkins coined the word meme in The Selfish Gene (1976) as a concept for discussion of evolutionary principles in explaining the spread of ideas and cultural phenomena (1986: 192).

154

space and the space between [bodies]. Idhe categorises the here-body and now-me as the physicalised version, as opposed to out-of-body and virtual body states. He says:
If I am right about the secret norm of a here-body in action, it should be noted that such a body experience is one that is not simply coextensive with a body outline or ones skin. The intentionality of bodily action goes beyond ones bodily limits but only within a regional, limited range [] Ones skin is at best polymorphically ambiguous, and, even without material extension, the sense of the here-body exceeds its physical bounds. (Idhe, 2002: 6)

Grosz also explores the concept of bodies in space:

Our body is not in space like things; it inhabits or haunts space. It applies itself to space like a hand to an instrument, and when we wish to move about we do not move the body as we move an object. We transport it without instruments ... since it is ours and because, through it we have access to space [] (Merleau-Ponty 2002/1962: 5) In other words, we do not grasp space directly or through our senses but though our bodily situation. Space is not understood as a series of relations between different objectively located points, points of equal value; for one thing, this flattens and neutralises the positive contribution we ourselves make in the perceptions of objects. Rather, space is understood by us as a relation between these points and a central or organizing perspective which regulates perceptions so that they can occupy the same perceptual field. This perspective has no other location than that given by the body ... (Grosz, 1994: 90)

This perspective is key to understanding the notion of space between, because it shows how the body fills this space, and points to a notion of presence and embodiment existing without flesh. It is from our knowledge of space and ourselves within it that we see the basis from which to orient the body and from which it can move, as well as how our consciousness can and does wander. By looking at Human-Computer Interaction (HCI), Paul Dourish shows how a person can embody the technology they use. He also applies philosophy to computer science theory to demonstrate technological embodiment within certain contexts and in various manifestations. Dourishs theories are relevant here as he focuses on ephemeral beingness or (tele)presence of being, without a physical skin encasement which can manifest when users engage in various technological activities, such as when using flight simulators for pilot training or video gaming (2001: 100). Richardson also addresses technological embodiment by positing a form of non-corporeality within virtual contexts. She suggests this form of embodiment exists during invisible exchanges of sentiment, such as during online chatting and text messaging (Richardson, 2006: 6). These examples attempt to reconcile the non-Cartesian model of embodiment by correlating a technological form of extended mind with that of NDEs, out-ofbody and related experiences, as discussed by Beauregard in The Spiritual Brain (2007: 157). Beauregard suggests that the mind cannot be constrained to the flesh, or to matter or the body as an epiphenomenon (2007: 110) of the brain emerging through evolution, as is commonly held in the materialist scientific community. Rather, he is suggesting that neither scientists nor philosophers have resolved the issue of qualia (104 and 108) or how the mind can heal and control the body (such as with the placebo and nocebo effects found in medicine: 145), or can leave the body altogether (as in out-of-body and near-death experiences: 153154). Thus, the theorists who advocate a form of

155

consciousness or mind extension through technology are inadvertently advocating a sort of telepathy as a form of projection by sending the consciousness through technology. Kozel delves into concepts of the visible and invisible related to notions of presence and space between, as well as virtual and telepathic connection. She says:
The invisible does not make sense on its own, but gives depth and texture to what we see [...] the invisible can be seen as a dimension that cannot be isolated from others but permits the existing ones to intersect differently [...] (Kozel, 2007: 40)

Expanding on this, she says:

[...] the invisible is not simply nonvisible [...] Just as connective tissue does not appear on scans or x-rays but is integral to our structures and our physical communication of pain and pleasure, the invisible is that which is possibly visible; it spans temporality by being what has been or what in the future might be seen, and it is transsubjective [...] it is an absence that matters. Literally, it is an absence with a sort of materiality. (Kozel, 2007:41, quoting Merleau-Ponty, 1962a: 187)

This is an important argument that reconciles the dualities of perception and experience. Yet in light of numerous studies and accounts of near-death experiences, out-of-body experiences and other related extended-mind occurrences, which show the existence of consciousness beyond physical body and death, questions still linger. These arguments make it evident that all of these states can still be part of embodied consciousness; what is unclear is how or whether the consciousness can continue beyond the life of the perceptual, corporeal body. Kozel reflects further on the body, focusing on connective tissue and communication:
Connective tissue is an embodied way of understanding not only the pre-reflective within us but also the means of communication within one body and between bodies. Those patterns of tension communicate physicality and affect, pain, and peace, distortion and degeneration. They hold memory over time. Unpredictable, reshapeable, multilayered, affecting brain and body functioning, connective tissue is a fundamentally dynamic material and network. Language and the channels between people are material. They are connective tissue-tensile fibers, material and immaterial, imaginary and real, connecting and separating people, and connecting and separating meaning from itself. Beginning to view our bodies and the spaces between bodies differently is the condition for reworking in a Merleau-Pontian way our understanding of thought, the communication of ideas between people, and the relation between physicality and ideas, between self and world. (Kozel, 2007: 30, my emphasis)

This perspective on communication and spaces between bodies clearly places the body in the position of interpreter of all thought, emotional and unconscious states, as well as the site of physical expression. Kozels term connective tissue is similar in meaning to my conception of invisible threads. For this work, invisible threads translates as the invisible connection between people, framed around emotional connections and/or investment between people, knitting them together. Yet Kozels use argues that the bodys own fibres provide a more embodied understanding of human interaction,

156

thus implicating a more elastic, regenerative and alive system of connection than the word thread affords. I see these connections or threads as the stitching or weaving of ideas, emotions, and people together to create a new if ever-changing fabric. Both can co-exist as fabric-like tissue. The discussion and analysis regarding human connective tissue contextualises the visible and the invisible and makes the case for electro-magnetic, radio and other invisible waveforms and similar forms that are not visible to the eye. It also emboldens the argument that there are many elements in our environment and daily experience that are not easily seen, but exist and connect us. These include conscious, embodied extensions of our selves beyond our physical bodies. For the purposes of this discussion, embodiment and its various states are described as: 1) being in the body, and the mind/body integration; and 2) being in the body, but also sensing and extending the mind, during neardeath experiences, dreams and in altered states, VR space, and virtual worlds and consciousness. Idhe speaks of the embodied, in-person aspects of liveness and how our bodies connect with technologies of modern living. His notion of here-body describes the multi-sensory perspective or body in action/myself-as-body that is [] not simply coextensive with a body outline or ones skin []. He reiterates that the here-body is not just the one encased within the skin and surrounding the bones, but in the case of out-of-body experiences, the here-body is [] extended beyond the outline of the wearers body [] (2002: 57). Indirectly referencing Merleau-Ponty on body-image, Idhe adds,
[] image-body is identified with 'my body', but under the perspective of not being the 'nowme' that is implicitly identified with the floating perspective [...] now-me and the here-body [...] multisensory sense of the here-body would locate this in the floating location. (2002: 5)

Here, the discussion enters the domain of the body image, but also Idhe tries to place the floating or out of body into some sort of framework of here-body, now-body, which also brings in the time element, important here in terms of technology and the multisensory body in action and the experience of liveness. He continues: [] here-body in action that provides the centered norm of myself-as-body [] (Idhe, 2002: 6) He also categorises here-body now-me as the physicalised version versus outof-body and virtual bodies states. In a related way, Kozel speaks briefly of leaving ourselves existentially and conceptually (corporeally also), in that our consciousness has the ability to roam or extend literally outside the boundaries of our skin and bodies (consciously, rather than in Massumis unconscious sense). It could also be argued that we leave our bodies temporarily while we are thinking and daydreaming, or while our minds are simultaneously multi-tasking with numerous ideas, plans, worries, emotions and preoccupations. Unlike psychogeography or locative media projects that focus on the external environment and locativeness or situatedness in the geography out in the world. MINDtouch always returns to the body as the primary location of exploration, with the goal of situating and locating individuals back within themselves phenomenologically, engaging or listening to their bodies. Referencing body situatedness when embodying technology, Don Idhe states:

157

Bodies are necessarily situated, which is not to say that one cannot take account of interactions between (my) body and materiality [...] bodily situatedness. (2002: xix)

On technological embodiment through extended consciousness, he adds:

[...] machine muscle replac[es] human muscle [] Neither muscle nor mind has reached out into the open world except in human-technology symbiotic forms [...] the standard form of replacement worry [...] (2002: 3)

This is main thrust of Idhes argument that technology and the body meet and interface to extend beyond the skin (also suggesting this as the only way to extend the mind beyond the skin), and our engagement with technology in the process. I differ on this point, but see its connection to my work with the mobile phone and biofeedback sensors. Idhes argument focuses on efforts to connect the body to technology, very directly on the skin, as a means to reach out into the open world through a very symbiotic form. Idhe suggests a superficial outward reaching, through words as visual and spectacle, in contrast to embodied:
The full, multidimensional experience gestalts in the here-body of the embodied perspective, whereas the visual objectification out there is spectacle-like. (2002: 4)

This is a problematic position since it suggests that the external experience is merely visual and spectacle-like, as if the visual cannot be embodied. Additionally, it seems to suggest that this spectacle cannot induce a visceral or felt sense even if experienced outside of the body [195]. I would suggest that the extension of the conscious experience is sent back to the corporeal body of sensations and perceptions, even if perceiving beyond the skin (as in the case of out-of-body or near-death experiences). In this sense, therefore, out-of-body experiences, out there are felt and still embodied in here in the fleshy body. Idhe continues on the technological context of embodiment and in the realm of the in and out of bodily experiences:
[...] nontechnologized virtual body experience [...] some floating perspective [...] (2002: 5)

This position addresses the conundrum of the dualities of embodiment and then brings in the technological dimension, relating it to my work and exploration of embodiment in the virtual, mobile network. It also refers to the experience of embodying technology through virtual systems, like games or the Internet, as a form of out-of-body-experience or extended mind. This gets into the cyborg brain
195 Laura U. Marks, in her book The Skin of the Film: Intercultural Cinema, Embodiment, and the Senses (2000), might disagree, since she argues that film breathes, thinks and embodies the cultures and people it portrays. She argues that film contains tactile visual representations through sensual images evoking viewers memories of sensory experiences (such as water or the outdoors), or through film characters experiencing heightened states of sensory experience in the diegesis (such as smelling, tasting, hearing, etc.), or in the way that film cameras capture or pan across extreme body close-ups. This also relates to the symbolic order representation concept in Lacanian psychology used vastly in feminist film theory, which there is not space to discuss here.

158

in a vat ideas so popular in the 1990s, inspired by science fiction books and films of the time. This was when computer enthusiasts dreamed of a time when we would no longer need our bodies and could be pure mind [196], connected to a computer or streaming through a network or inhabiting a robotic form. However, the more we learn about the wisdom and distributed intelligence of the body and mind through neuroscience and biology, the less popular these fantasies become. Idhe brings in the notions of permeability of the body outline or skin boundaries, which help to support a view of the extended body:
[...] the sense of her here-body [... is] extended beyond the outline of the wearer's body. (2002: 7)

This is relevant not only to consciousness, but also in terms of digital virtuality and virtual connection through technological facilitation of body-mind extension, beyond the boundaries of flesh, but also necessitated by corporeality. To continue musing on the concept of extended self, one proposal is that when we are extending outward we send and receive threads of ourselves; sometimes they are not received or are ignored, or even received but not responded to. If a person sends their thoughts, love and emotional pain to someone else through their thoughts, sometimes the sender will know these are felt by the intended other, but sometimes the thread hits a wall. Digital connections, through text messages especially, seem like more tangible manifestations of those threads, and when the messages no longer come, their imprint is still left behind. The text itself still exists, until the phone crashes and erases them all, or a new phone is bought [197]. More resonant is the invisible emotional imprint felt inside, the felt absence of texts no longer incoming, the silence where the buzzing and beeping once lived. What does absence feel like? It has more nuances, essence and sensation than the expected model of a void, lack or emptiness. These are the threads between us when there is historical digital connection through technology (i.e. in the form of text-message or email exchanges), or a shared memory, shared perceptions and experiences, then these emotional ephemeral links intertwine people, and can be felt beyond the digital messages, beyond the physical body and physical/tangible space and time. How does one untether from that extended self, entwined with tattered, frayed sub-threads that are matted, enmeshed and entangled? Especially when the threads have endured, stretched, loosened and tightened over the years? How does one explain the excitement of getting those regular beeping, buzzing and blinking physicalised messages, especially when so many come causing a knee-jerk, Pavlovian need-to-answer response in a flirtation and in a frenzy when it is the only way to smile at another far away, to hug them, to touch them, to kiss and to laugh with them. When that ends, all that remains is the visceral imprint in the body of those sensations, sounds, memories and emotions.
196 Katherine Hayles critiques this in How We Became Posthuman: virtual bodies in cybernetics, literature, and informatics (1999). Donna Haraways Cyborg Manifesto (1984) also covers this territory. 197 This is a common experience with the technology, which can have the benefit of allowing one to move on more easily from the emotional baggage and memories such messages held.

159

Finally, this digital archive can be difficult to review, as it enacts the cellular memory and emotional intensity that can result in a temporary paralysis from reliving the sensations and moment again, leaving a painful longing, as one tries to recall why the interaction stopped. This leads to the question of how to distinguish between the reality and the fantasy created by these emotionally laden threads, spun by the text messages, which become imbued with expectation and desire. I have reflected on the experience of digital absence in another way (modified from a research journal entry):
The sense of digital absence can be explained from lived experience. My first cognisant experience of absence of virtual contact came a few years ago. Suddenly, following a very active, almost daily, exchange with a friend for several months. The abrupt change in the interaction was immediately disconcerting, leaving an immediate, affective sense of lack, void or silence, invisible but felt viscerally. This disconnect was like a tearing away of flesh. It was very frustrating since there had been a very close connection. What was most upsetting was the ability to see this friend online or available to chat often making it hard to resist attempts to reconnect. This example, the sense of anothers presence can be felt when they are visibly still online and can be seen in ones chat contact list. This is similar to working in the same room with them. In this case, the experience of digital presence created a real sense of loss or being rebuffed. Unlike when someone is in close proximity, or geographically local, where you can visit their home or run into them in commonly visited places, or even ask their friends how they are, when there is a digital absence one has no recourse, you are left dangling in the digital abyss, feeling the void affectively and emotionally.

This reflection speaks to Bordens earlier point on Social Presence for mobile chat applications, and also points to the negative ramifications of available to chat status as a means to increase emotional space and absence through digital means. However, the psychology of digital relationships is not an area that can be delved into more deeply here. I add it to show how technology can be embodied and have a physical or visceral impact on the body, in the sense that Massumi speaks of intensity and the triggering of emotions. Even if digitally manifested, these are actual and real experiences, felt in the flesh. Returning to the notion of absence, Merleau-Ponty explores presence through absence:
[...] an object[s ...] presence is such that it entails a possible absence. Now the permanence of my body is different in kind ... It defies exploration and is always presented to me from the same angle. Its permanence is the permanence in the world, a permanence from my point of view [...] Insofar as it sees or touches the world, my body can therefore be neither seen nor touched. What prevents it ever being an object, being "completely constituted," is that it is that by which there are objects. It is neither tangible nor visible insofar as it is that which sees and touches. (2002/1962, 2003: 9092)

In phenomenological experience of virtual absences, when a friendship or romantic relationship involves great geographical distance, followed by intermittent, proximal, then in-person interaction, interspersed with long spells of merely virtual interaction, even live text, voice or video chat cannot bridge or fix the fragile communication issues or misunderstandings in the same way that live interaction might. If the

160

person on the other end does not respond, they can vanish without a trace. This is one of several instances where it could be argued that there is a superiority of in-person liveness and presence, in terms of conflict resolution and problem solving approaches to interaction. To reiterate an earlier point: to maintain relationships and communicate effectively, distance must be reduced. Technological substitutions can be used to fill the gap, but in-person, corporeal presence is the most effective for complete human communication (Lewis et al., 2000: 205) [198]. This might appear to refute the argument that we embody space through our virtual consciousness. However, it supports the point that complete communication requires the real thing, since much is expressed that cannot be translated digitally. In summary, space, space between, and absence are interwoven as part of a continuum between physical space and virtual space. The MINDtouch project has been about traversing physical space or distance and embodying it with the aid of technology, though not replacing physical presence. I have attempted to exemplify how I have conceived and worked with space and space between, with physical bodies in physical locations connected through mobile phones.

8.3/ experience senses, perception and affect

To study participant experiences of feltness and presence using the body, we need to look at the senses and perception. Merleau-Ponty starts this discussion by pointing out how we think about the senses and perception:
We think we know perfectly well what 'seeing', 'hearing', sensing are, because perception has long provided us with objects which are coloured or which emit sounds. When we try to analyse it, we transpose these objects into consciousness [...] what we know to be in things themselves we immediately take as being in our consciousness of them. We make perception out of things perceived. (2002/1962: 5, my emphasis)

Therefore, although we think of our experience of objects as within us, we are actually reacting to these objects through our senses, which transpose or interpret the objects outside into perceptions inside our bodies and consciousness. This is similar to Sheldrakes extended mind theory (2003: 122). He postulates that our minds reach out to the world outside of us, and might agree that the body processes these perceptions in order for us to respond, as in the case of sensing imminent danger [199]. In philosophy, Merleau-Ponty explores the contrasting notion that we cannot understand our
198 To support Lewis et al, Cummings, Butler and Kraut did a study that shows that [] in one-to-one comparisons, an email message is not as good as a phone call or a face-to-face meeting for developing and sustaining social relationships [] Relationships sustained primarily over the Internet are not as close as those sustained by other means (2002: 19). 199 Sheldrake discusses studies with animals that show that telepathy is likely to be an evolutionary trait that keeps animals safe (2002: 111112).

161

senses and perceptions, because our main articulations of them are through the different interpretations of them through the disciplines of physiology and psychology. Yet our intellectual constructions fail to frame our experience of ourselves and our perception adequately. He says:
There is no physiological definition of sensation, and more generally there is no physiological psychology which is autonomous, because the physiological event itself obeys biological and psychological laws [...] 'sense-experience is a vital process, no less than procreation, breathing or growth' (Weizscker, quoted by Stein, J, 1928: 354) Psychology and physiology are no longer, then, two parallel sciences, but two accounts of behaviour, the first concrete, the second abstract. (2002/1962: 11)

To connect these two points, while there are different ways to articulate our sense experience and perceptions, our bodies have evolved to understand, respond and in some cases communicate them to others, all below our conscious awareness. Merleau-Ponty shows how psychologists interpret perception with respect to understanding the senses through the sciences:
[...] sensation is not experienced, and consciousness is always conscious of an object. We arrive at sensation when we think about perceptions and try to make it clear [...] Pure sensation, defined as the action of stimuli on our body, is the 'last effect' of knowledge, particularly of scientific knowledge, and its an illusion (a not unnatural one, moreover) that causes us to put it at the beginning and to believe that it precedes knowledge. (2002/1962: 4243, his emphasis)

Therefore, one can infer that our definitions of sensations are superficial and external to our experience of them, and are only the beginning of a true understanding. He points out that we need to live, feel, listen to, experience and describe them for ourselves to get the full picture of them. To do that, he says:
The function of the organism in receiving stimuli is, so to speak, to 'conceive' a certain form of excitation [...] exteroceptivity demands that stimuli be given a shape, the consciousness of the body invades the body, the soul spreads over all its parts, and behaviour overspills it is at the end of a chain of physical and physiological events which alone can be ascribed to the 'real body'[...] Cannot I find in the body message-wires sent by the internal organs to the brain, which are installed by nature to provide the soul with the opportunity of feeling its body? Consciousness of the body, and the soul, are thus repressed. (2002/1962: 8687) [200]

Massumi adds to this through his reflection and deconstruction of various theories on perception, neurobiology and physiology, phenomenology, and arts and entertainment. His interpretation of neuroscience discoveries and other studies is of particular interest, especially when he explains how perception and sensation work:
[] the feeling of having a feeling is what Leibniz called the perception of perception. (Massumi, 2002: 14)

and:

200 This is fascinating and fundamental, since he uses the word soul as a connecting or interpreting force for consciousness and body, which adds contention in the current climate of skepticism of spirituality, especially within the science community and mainstream media. This relates to my earlier discussion on spirituality.

162

[] the vast majority of the world's sensations are certainly non-conscious. (2002: 16) [201]

He later adds,
Sensation is the mode in which potential is present in the perceiving body [] Perception is never only impression [...] Studding each impression are shards of intentions and conscious memories, most presently bearing on pregame strategy shimmers of reflection and language [...] Toward that effect, in that sensation, a heterogeneity of levels contract into the body from which they reissue in an action [...] The play is the event-dimension doubling the empirical eventspace in which the substantial terms in play physically intermix. (2002: 75)

Massumi refers to physically playing football, which relates to the approach used in my work in assisting people to use their physical and other sensations and perceptions as a new form of nonverbal expression, responding to and acknowledging their bodies through a different modality or interface. These are the sensations to be harnessed in this research, through the wearable biofeedback sensors and using mobile media to translate them into visual expressions. Kozel continues on the experience of perception, the pre-reflective and the bodys way of communicating sensation and experience to itself. She says that the abstraction of the being of sensation (referring to Rajchman):
[] tends toward an ephemeral immateriality, yet its challenge to language and logic makes the pre-reflective seem to be the domain of corporeality. Can the pre-reflective be abstract and corporeal at the same time? [] The voice of physical experience spans registers of language: words, gestures, sound, colors, textures, smells. There is never any question of entrapping corporeality in the domain of the pre-reflective, as if it were forever mute, brute and material; bodies are at home at the fringes of language and have one foot squarely in the fertile ground from which new knowledge enters into existing conceptual paradigms, but they are also highly articulate. (2007: 21, my emphasis)

This sense of the physical spanning registers is a powerful concept explored in small degrees with participants in my workshops and performances. Kozel later adds:
Once the aesthetic experience is freed from these [shared structures of thought and judgement], what happens? Does it remain blissfully unfettered by all things rational and cognitive? No, after the escape from thought, further thought emerges from the experience, even concepts, frameworks, structures of sorts, in a move that seems to be contrary to the fluid being of sensation, contrary to art, contrary to phenomenology. It is impossible to prevent thought from attempting to pin down experience just as it is impossible to prevent experience from blowing holes in structures that attempt to codify it. (2007: 22, my emphasis)

This rings true even if the thought is in another form of expression from language such as gesture or drawing. What is freeing about this statement is that it clarifies the idea that experience and thought are
201 Again I want to stress that even though Massumi uses the word non-conscious, I define the non-conscious and unconscious as all part of the continuum of the mind/body consciousness it is not a case of literal awareness or acknowledgement at all times, but of the overall system of consciousness.

163

ever-evolving and shifting, forcing us continually to adapt and be open to wherever they take us conceptually, experientially and existentially. This, if one is open to a variety of perceptions and experiences, can expand ones approach to each moment of lived experience. This in turn can enrich and challenge new expressions, art practices, or further thought and philosophical exploration, each feeding into the next in exciting ways. The senses work together automatically in the body, so this involves the different dimensions of experience, each adding to the other to provide a larger picture of experience to the body and consciousness. Merleau-Ponty addresses this:
[...] the intelligence ensure no more than an apparent communication between different experiences, and the synthesis of visual and tactile worlds [...] the constitution of and intersensory world must be effected in the domain of sense itself, the community of significance between the two experiences being inadequate to ensure their union in one single experience. The senses are distinct from each other and distinct from intellect in so far as each one of them brings with is a structure of being which can never be exactly transposed. (2002/1962: 261)

and:
The senses translate each other without any need of an interpreter, and are mutual comprehensible without the intervention of any idea [...] With the notion of the bodily schema we find [] the unity of the senses and of the object [sensed]. My body is the seat or rather the very actuality of the phenomenon of expression (Ausdruck), and there the visual and auditory experiences [] and their expressive value is the ground of the antepredicative unity of the perceived world, and, through it, of verbal expression (Darstellung) [] My body is the fabric into which all objects are woven [] in relation to the perceived world, the general instrument of my comprehension. It is my body which gives significance [] (2002/1962: 273, his emphasis)

Not only is Merleau-Ponty focussing on the unification and intelligence of the body and our individual senses, but also upon our ability to translate sensation into the visual, audible and intellectual symbols of language and comprehend them in multiple dimensions of perception. Grosz adds:
[...] senses cannot be so readily separated off from each other in the attempt to understand their specific modalities and properties, for in lived experience the senses interact, form a union, and yield access to a singular world. Sight and touch are able to communicate with each other, to provide confirmations (or contradictions) of each other, because they are the sense of one and the same subject operating simultaneously, within one and the same world. (1994: 99)

and suggests,
Insofar as the body can no longer be seen as a unified and unifying organism, an organism centered either biologically or psychically, organised in terms of an overarching consciousness or unconscious, cohesive through its intentionality or its capacity for reflection and self-reflection [...] (1994: 168)

This underlines the greatly coordinated process of perception communication taking place within the body, and how the senses operate to report as much subjective experience from the objective world as

164

possible. Yet one could argue that there is no objective world; only our subjective experience of objects and of being in the world, which our senses engage with to help us make sense of the world. This does not mean that there is no objective reality out there, but that we only have our own senses, bodies and perceptions with which to experience it. While I am proposing that we can extend our minds and bodies, we cannot climb into another mind or experience the world through another bodys perceptions to confirm our own. We can only empathise, communicate and share experiences with other people and form a collective and shared understanding of objective reality, based on our individual, subjectively perceived, embodied experiences. This is what is explored in this project: shared subjective experience. Grosz perspective suggests that the body is a collective intelligence, modelled on notions of the collective consciousness (of humans and animals). She also highlights that phenomenological, reflective reporting and other forms of expression of lived experience may be essential in order to understand ourselves and each other, since science can only give us a part of the story. It reinforces the position that, as there are so many things we do not understand about our bodies, consciousness and the universe, a materialist view is no longer adequate. These passages address different aspects of an exploration of the senses that have been relevant to my work with participants, and efforts to aid them in tapping into their own inner sensations and perceptions. Biofeedback sensing devices were used in the performances to pick up on various body sensations and translate them into data, controlling the representations of the video interpretations of the inner sensations from the video group. These theorists discussions of how the body works have been salient in understanding and developing an approach to tapping into and exploring them more meaningfully. Massumi also posits a theory on the affective, sensory aspects and issues relevant to dealing with sensors and the embodied exploration of art and performance. He focuses much attention on perception itself, perception of sensation, and the concept that the body is always in a state of motion and potential. His analysis has been grounding and important for work on the body, simulated telepathy and mobile media performance, not to mention the visceral/embodied experience of performance and liveness. His definition of affect, based on the theories of Deleuze and Spinoza, is a rich area of thought and useful in framing this research:
Affect is autonomous to the degree to which it escapes confinement in the particular body whose vitality, or potential for interaction, it is. Formed, qualified, situated perceptions and cognitions fulfilling functions of actual connection or blockage are the capture and closure of affect. Emotion is the most intense [...] expression of that capture [...] (Massumi, 2002: 35, my emphasis)

This is how Massumi explains how the body understands and translates its own sensory experience, while he translates the biology, physiology and neuroscientific explanations of these activities into a

165

more philosophical version of how affect, emotion and perception work. This transition of science into a phenomenological and philosophical argument is at times also very vivid and poetic in its interpretations of experience and perception. He continues:
One's 'sense of aliveness' is a continuous, nonconscious self-perception (unconscious selfreflections or lived self-referentiality). (2002: 36)

Then, working with Merleau-Pontys idea of body image, Massumi adds:

[...] the body without an image is the involution of subject-object relations into the body of the observer and of that body into itself. (2002: 57)

This conception of body image, mentioned first by Merleau-Ponty, also comes up in other body and somatic, as well as neuroscientific and psychological, discussions. It is core to have a sense or image of self, like a map to ones body and perceptions. This body image enables us to move within the world and differentiate, not only one experience from another, but also ourselves and our bodies from other people, creatures, entities and objects encountered on a daily basis. In terms of presence corporeal or virtual and otherness, we must have a map, a sense and an image of self in order to differentiate ourselves from other presences. This is especially true within the less visible spaces of virtually networked, mobile and online environments, not to mention within the telepathic or non-tangible space between of emotional connection, over distance and any other threads that connect us less tangibly. Massumi introduces his theory on proprioception and other inner body senses relevant to biosensing devices in this research:
[...] there are other modes of perception involved [...] proprioception, defined as the sensibility proper to the muscles and ligaments as opposed to tactile sensibility [...] and visceral sensibility. Tactility is the sensibility of the skin as a surface of contact between the perceiving subject and the perceived object [...] enveloping the skin's contact with the external world [...] (2002: 58, my emphasis)

Then he delves into the felt and visceral:

Proprioception translates the exertions and ease of the body's encounters with objects into a muscular memory of rationality. (2002: 59)

and continues:
The immediacy of visceral perception is so radical that it can be said without exaggeration to precede the exteroceptive sense perception. (2002: 60)

What is crucial here is the notion of the senses experienced through translation of perceptions and physical response or further translation. Like a relay of experiences that eventually are conscious and felt both physically and emotionally.

166

Viscerality for me is an internally felt response in the gut, a gut feeling/reaction; a lump in the throat; butterflies in the stomach. These are the proprioceptive sensations of the internal body that involve hormones, pheromones, fluids and/or nervous reactions and responses triggered by emotions and thoughts, affect or situations encountered. Massumi says that [] visceral sensibility immediately registers excitations gathered by the five 'exteroceptive' senses even before they are fully processed by the brain [] (2002: 6061). A case also can be made when using biofeedback devices to measure senses, that these visceral, proprioceptive sensations and perceptions are the truest, most unadulterated or non-violated, pre-reflective experiences before they are translated to the mind. This also refers to how an emotional state can be critical for sending and receiving presence, in both live (performance or intimate) and virtual contexts, because it engages the individual in the flesh, which also creates an investment in engaging. Massumi adds,
[...] viscerality subtracts quality as such from excitation. It registers intensity [] The dimension of viscerality is adjacent to that of proprioception, but they do not overlap. The dimension of proprioception lies midway between stimulus and response [...] viscerality is the perception of suspense [...] the space of passion [...] degrees of intensity. (2002: 61, my emphasis)

Here Massumi is making the case that the emotional is physical, possibly more so than it is conscious or of the mind, due to translations and transmutations. He states that physical intensity is triggered by emotional stimulus, which serves to integrate the body and consciousness in a tangible, visceral way. As such, he points out,
[] an emotional qualification breaks narrative continuity for a moment to register a state actually to re-register an already felt state, for the skin is faster than the word. (2002: 25, my emphasis)

Intensity is then qualifiable as an emotional state. Further, on emotion as intensity, he explains:

An emotion is a subjective content [] the quality of an experience which is [] personal. Emotion is qualified intensity [] semantically and semiotically formed progressions, into narrativizable action-reaction circuits, into function and meaning. (2002: 28)

So emotion is not purely experienced, but instead intensity is the experience of emotion; it is translated by the body so that felt intensities are understandable to our consciousness. Emotion narrativises intensity, with the mind telling the body a story of what it has just experienced or transmutes the sensory to the conceptual space of the mind [202]. Such a complex series of processes can essentially help us get through our daily lives. He continues:
The body doesn't just absorb pulses or discrete stimulations; it infolds contexts, it infolds volitions and cognitions that are nothing if not situated. Intensity is asocial, but not presocial it
202 Benjamin Libet did studies to prove that we respond to situations with the body seconds before the mind catches up, but that when it does respond, the mind then back dates the brain to time stamp it and tell the memory what was registered and experienced when it was happening (Massumi, 2002: 30). See more on his recent work and findings on this in Mind Time: The Temporal Factor in Consciousness (2004: 75).

167

includes social elements but mixes them with elements belonging to other levels of functioning and combines them according to different logic. (Massumi 2002: 30, my emphasis)

The idea that the body includes the social and its response to the world is relevant to thoughts on intimacy and relationships brought forward by Lewis et al. On a related point on the experience of intensity, explaining the source and nature of it in relation to perception, he says:
The disconnection between form/content and intensity/effect is not just negative: it enables a different connectivity, a different difference, in parallel [...] Intensity is embodied in purely autonomic reactions most directly manifested in the skin at the surface of the body, at its interface with things. Depth reactions belong more to the form/content (qualifications) level, even though they also involve automatic functions such as heartbeat and breathing. The reason may be that they are associated with expectation [...] Modulations of heartbeat and breathing mark a reflux of consciousness into the autonomic depths [...] coterminous with rise of the autonomic into consciousness. They are a conscious-autonomic mix, a measure of their participation in one another. (Massumi, 2002: 25, my emphasis)

Further, on how intensity is experienced as emotion in conscious awareness and expresses viscerally, he says,
Passion, then, is best understood less as an abstract space than as the time-stuff of spatial abstraction [...] An emotion or feeling is a recognized affect, an identified intensity as reinjected into stimulus-response paths, into action-reaction circuits of infolding and externalization in short, into subjectobject relations. (2002: 61, my emphasis)

This is a curious, if clinical, description of the experience of passion, but comprises a breakdown necessary to understand it more clearly and objectively. Massumi adds:
Mesoperception is the synesthetic sensibility: it is the medium where inputs from all five senses meet, across subsensate excitation, and become flesh together, tense and quivering [] Mesoperception can be called sensation. (Massumi, 2002: 62, my emphasis)

This point argues for flesh as the ultimate interface for perception, and for communication with others. In light of this, all synthetic and virtual substitutes pale so live physicality is still superior. Idhe adds to the debate of the senses and embodiment within the technological context by stating:
The ultimate goal of virtual embodiment is to become the perfect simulacrum of full, multisensory bodily action. (2002: 7)

This is contentious. Can we not already shift our perception and extend our senses? I would argue that people do so in dream telepathy, albeit not with the same language of the simulacrum. Idhe is correct to say that a perfect simulacrum seems an ultimate goal, one that humanity has continually envisioned attaining in order to shift our embodied, sensing selves. He claims this perfect simulacrum could arrive digitally through technology, whereas I would argue that it already exists telepathically, but needs to be honed and enhanced. One could say that the MINDtouch project is an attempt to create a form of simulacra, yet in a more abstracted way externalising inner experienced senses and then translating them using external visual material, rather than creating a perfect mirror or copy of how they literally

168

appear in the world. In contrast to a focus on virtual and visual construction and representation, we might instead develop an external mirror view or body image map to represent us. This would then enable us to extend ourselves outside our internalised bodys image. So an internal, emotional and sensory approach has taken this research toward the visual representation of self, as opposed to an externalised visual representation of the extended body [203]. To follow from these arguments then, one could say that we experience sensations within the body when we are connected to others emotionally, whether they are in the room, on the phone or across the world we can still feel them, as we feel these intense sensations through regular contact with them. In the last ten years or more, emotional connections are often maintained through additional virtual presence. So where do we experience the connection? Inside of us, as the rest of the world is experienced, through our bodies and sensations. This discussion shows how emotional and sensory intensity affect the bodys basic automatic responses. For MINDtouch participants, this involved wearing the biofeedback sensors during live events, to induce and capture certain inner sensations and perceptions. This sense of intensity is registered as emotion in the awareness of the participants, then explored and transformed visually as a form of communication.

Figure 5 2010 Camille Baker key ideas and theorists on embodiment

This discussion has outlined the various theories on embodiment as it relates to sensation and perception, which are important to this research, since it involves both sensing and the body directly using technology.

203 I use this term from the point of view of Sheldrakes extended mind, framed with the argument herein that the mind and body are one, but recognising that embodiment does not always include the corporeal flesh that extends our consciousness and thus our body.

169

8.4/ liveness when is now?

The Oxford Dictionary definition of live is: [] of a performance, heard or watched at the time of its occurrence, as distinguished from one recorded on film, tape, etc. [] (Oxford Dictionary: 1989, 1047) Here I shall unpack a few different understandings of liveness in human interaction and with special focus towards liveness and real-time in performance. The intention of the practical performance aspect of MINDtouch has been to explore how liveness and presence factor into a form of mobile here and now. As such, the project has been an enquiry into whether one feels presence viscerally (as discussed previously) during mobile performance, and whether people can be enabled to feel it too and exchange it with others. The mind map below was an initial exploration of all the aspects of the quality and experience of liveness that are relevant to this research.

Figure 6 2009 Camille Baker

Any discussion and research on presence and liveness in performance must refer back to Philip Auslanders work, and his book Liveness, specifically on the quality of liveness in a performance setting. He puts forth that liveness contrasts with a mediatised performance on media technologies, and that the idea of liveness did not even come to be considered before the use of audio, film and video recording technology. For Auslander, it is important that liveness involves a certain tangibility and tactility within the context of performance. These qualities are considered the feature or selling point of liveness (Auslander, 1999: 55), allowing audiences the intimacy of witnessing from a particular performers point of view, in order to convince them that they are somehow physically interacting with the performer. Many performing arts patrons desire physical proximity with the performer; they tend to

170

report a sense of an immediate, intense, sensory and sensual reality. Auslander states that, [] liveness is ideological [... the] belief that live confrontation can somehow give rise to the truth in ways that recorded representation cannot [] (Auslander 1999: 128129). This sensory, sensual proximity during live, staged performances offers viewers smell, touch and biochemical reactions or exchanges between performers and audience, or amongst audience members, which provide a quality of authenticity in the performance. Auslander makes the assertion that a mediatised performance engages all senses in the body differently not only in magnitude, but also in kind. However, he fails to explain how or in what way. It is not evident that a mediatised event can be experienced via the olfactory, tactile, somatic and kinaesthetic or proprioceptive senses, although that does not imply that the viewer of a mediatised performance is not experiencing the event on a deeply emotional and visceral level that can influence the other senses. I would add that emotions are at the centre of a sense or understanding of presence, and to some extent liveness. In addition, sensory information from an event, from its visual and audio expression and reception, cannot have a profound affect on the body of the viewer unless they experience emotional engagement or have been affected with intensity, in the sense Massumi proposes. This in turn can trigger other senses to respond, and then the body can feel the event as intensity, and not just witness it (such as with sports fans and sporting events) (2002: 73). It could be argued that this triggered intensity can be missing during an in-person performance if the audience is not emotionally engaged; and on the other hand can be profoundly experienced during a mediatised performance if intensity is triggered, even if all other senses are not engaged during the experience. Proximity offers a certain quality of emotion and immediacy in hearing small changes in the tone or quality of a performers voice, in subtle body shifts, even in something as subtle as a glance or making eye contact a viscerality like a touch (Auslander 1999: 85, Notes #35). Much communication happens non-verbally through gestures and subtle body language [204]. Thus, much is communicated through proximity that is not possible in a mediated, non-live format. Yet the aim of this research is to get closer to the experience of proximity through differently-used communication and performance forms, especially in these times when we are less locally connected. As we are more engaged in distance relationships and global business, as global nomads, we are also trying to find alternate means of travel and connection to get away from our carbon-fuelled, planet-harming methods of being present with each other. There are many other factors not discussed by Auslander [205] regarding the experience of the viewer, such as context, the emotional and physical state of the viewer or people with whom the viewer experiences, or whether they are alone. Factors such as the nature of the event, the relationship between the viewer and the performer, the occasion, and other factors like alcohol consumption should

204 As discussed earlier in Chapter 7, Lewis et al. show us that we need physical interaction to continue to develop our brains. 205 However, Auslander is relevant in terms of this discussion, even as a counterpoint to exploring and understanding liveness and performance.

171

also be considered. These factors compound the complexity of the physical and emotional experience that accompanies the reception of a performance, whether live or mediatised, which Schechner and Boal address more deeply. The sense or experience of liveness that I have come to know originates from previous dance and music performances, but also from the use of digital technology. Liveness for me is being intertwined simultaneously in time and/or space and physicality. The concept is enmeshed with presence, but is not the same quality. Liveness for this project has also been about real-time. Connotations of liveness involve being physically present in person, in the here-and-now, at this moment, in front of an audience in a traditional performance context, live or remotely (as with live TV broadcast or an online event). Liveness often entails an event or experience that is taking place at a certain point in time, at an exact moment, at some location, somewhere, with someone witnessing it in person. For my definition, it is not a priority that people are experiencing it in person or even in the same physical locale, as long as the liveness is happening simultaneously for all wherever they are, through the medium of technology. Not everyone must be in person, in the flesh at the event location, but the event is experienced as taking place now and must be witnessed now, in real-time or within seconds. In a digital context, nowness is critical to live events, but being physically present is not: one can use a computer with an avatar as a stand-in to substitute for actual bodily presence, or use voice-over IP or video conferencing to replace the corporeal presence, depending on the contextlevel of granularity, and so forth. As a researcher, getting to a true understanding and identifying the effect, reception and experience(s) of liveness, aura, presence of the performer and the audience, to reveal their intrinsic embodiment within the network, can only be achieved phenomenologically through participants experience. Then, through my observation and experience of participants experience, a hetero-phenomenological lens can be applied. In the MINDtouch project, a goal has been to deliver this mixed liveness with realtime precision simultaneously or as close as is possible within technological and logistical limitations. This visceral liveness has been difficult to maintain in practice within the mobile network, but a form of viscerally felt embodiment has been experienced through the videophone. Thus, virtual presence can be created by emotionally and affectively experienced responses and sensations in the body, triggered by the activities during the performance event (cf. Massumi). In a related sense, Richardson speaks of technosoma, which she describes as follows:
[] in the context of mobile media, I suggest that both tool and body are covalent participants and coalesce as various technosoma in the making of meaning and environment. It is our openness to the spatially non-coincident flesh of the world not only bodily flesh but the 'stuff' of our environment that allows us to incorporate technologies and equipment into our own corporeal organisation [] Each soma blending can work to describe a way of being rather than a what of being; any soma is not an entity as 'the body' might sometimes be understood but a process/network or ontological schematic. This terminology aims to effectively combine the concepts of intercorporeality and medium specificity, and also to grasp the way in which individual bodies digest the collective embodiments or shared habits of the wider cultural milieu. (2005: 5)

172

Is there such a state of liveness or is it possible to be present now, in this moment? Does 'now' exist? Perhaps we are never really just being in the now. Pondering the expression be here now [206] exposes the underlying elements of the spatiality of here and the temporality of now [207], at this moment, and this one, and this one So how can we be here now, and where is here? Does it need to be a physical or tangible space/place or can it be the space/place of ones own body? Or can it be in transit in the ephemeral space between bodies and sensors and phones? In the telepathic or dream space between bodies, can others feel your being now? Can you play and perform with this? And when is now? How can we savour now and be present and live or in a specific space? Being in a specific place, in person, no longer matters during a virtual or mobile space/experience, but does time? How can liveness and presence factor into the form of mobile here and the never really now? Can we feel it in our bodies and can we help others to feel it too? Can we share it with others? Can we be here in our bodies now, as now slips away into future, as we are always becoming? These are questions that are taken from daily life into my media practice and this research. What makes a networked performance live in time, and not merely space? Is it the time specificity of an event, the fact that it occurs at a precisely agreed point on the clock? This is debatable if the event is online, since the people who join the event are in different time zones. Therefore, the number of the hour is different, but the minute and seconds are the same globally (as Mitchell explores see below). However, if the event takes place for a given length of time, starting and ending at a certain point, and is somehow transmitted in a manner that means that others need to tune in as it is happening so as not to miss the event, then the now-ness of temporality is vital. The key is that it takes place at a given time, for a given duration, and needs to be participated in or witnessed (by whatever means possible), as it is taking place on the net and in a real or actual place (whether digitally over the network or in a physical location, for example, streamed from a live concert in Wembley Stadium) [208]. Why is being live during performance, in real-time, important? How is it critical conceptually, technologically, physically, virtually or otherwise? Does it make the experience more authentic? Where does authenticity of a performance or experience come in? Is there such a thing? Is it even possible to actually be live? In an attempt to partially answers these questions, Merleau-Ponty discusses aspects of the body in time relevant to a conceptualisation of liveness and the importance of temporality to networked events:
The spatial synthesis and the synthesis of the object are based on this unfolding of time. In every focusing movement my body unites present, past and future, it secretes time, rather it becomes that location in nature where, for the first time, events [] present a double horizon of past and future [...] My body takes possession of time; it brings into existence a past and a

206 Alpert (1971) coined this expression with his book Be Here Now. 207 Which Lakoff and Johnson addresses more in terms of how we speak and think of them, which is culturally constructed (Lakoff and Johnson, 1999: 151). 208 During the Performing Presence conference (Exeter, 2009), such questions and issues arose and attendees presented their own unique interpretations. No conclusions were really drawn; only more explorations and ways to think about and address the issues.

173

future for a present [...] but creates time instead of submitting to it. But every act of focusing must be renewed, otherwise it falls into unconsciousness. (2002/1962: 278279)

Massumi adds:
Technologically assisted channeling of event-transitivity constitutes a qualitatively different mode of power than either the regulating codifications of the Static or the regularizing codings of the social or cultural [...] The network distributes. Interlinks. Relates. The network is the relationality [...] It is the being of collective becoming. (2002: 86)

This makes the imperative of liveness in a performance critical. It is also a perfect elucidation of the activity of facilitating virtual presence and transmitting it over a network, because it highlights the interpersonal connection through these electronic threads. In this sense, Massumi makes the idea of liveness less relevant since collective becoming is ongoing, so the body and such virtual events are ever arriving, always in the process of becoming. He continues:
Each event transmitter is sustained and delivered by a dedicated collective apparatus deploying at least one technology of channelling that gives it body in the interval, where it disappears into its own immanence [...] (Massumi, 2002: 87)

For my work, a reasonable measure of precision and faithfulness to an exact moment or attention to the timeliness of creating a precise moment of liveness or nowness is critical during a scheduled event. It needs to happen within fifteen seconds of the events start or transmission commencement (if online or mobile and much less in-person) this is considered an imperative during my performance events. It is important if exploring this experience conceptually within the practical context, as I have been. However, like many practitioners of networked, electronic and new media, I recognise the technological limitations, lags and delays that are inherent in such live media performance events. Potential and becoming are the virtual for Massumi [209], since we are never static, never actualised fully in the now and always in the state of becoming something or someone else, never arriving completely formed and always having potential to become. This argument for virtuality connects to liveness and the question of time When is now? and how an event can be live, as liveness pertains to a media transmission. To this he says:
[...] the event-space of the home must be seen as one characterized by a very loose regime of passage. As a regime of openness to sign circulation to the delivery, absorption, and relay of sounds, words, and visions home is a node in a circulatory network of many dimensions (each corresponding to a technology of transmission). (2002: 85)

Yoga and meditation focus strongly on being here now, with attention focused on the internal experience of the body, on the chanting of a mantra or on breathing while keeping the body still in a specific posture, trying to void the mind of thought, but being attentive to whatever sensations or
209 Though it is beyond the scope of this thesis to discuss Bergsons time and duration, it is worth mentioning that much of what Massumi says about being, becoming, time and actualisation originates from his work.

174

perceptions happen to arise in the moment. This relates to the discussion on Merleau-Pontys and Idhes here-body [210], although it manifests differently, as mentally focussing intently on clearing the mind of all thought. Pierre Lvy (1998) addresses liveness and becoming through the network by contributing his concept of virtuality, with real versus virtual, actual versus possible, in the context of how the body, text, economics and culture are virtualised. He explains what virtuality means philosophically and practically to modern society. He posits that, based on the Greek definition of virtualis, current philosophy adopts the virtual as potential rather than as actualization. He also believes that virtual has come to mean absences of existence rather than tangibility, but also much more than that (1998: 23). He puts a great emphasis on distinguishing between virtual senses and real senses, which may not be so divergent, since we engage the physical and affective when experiencing both virtual and real stimuli. Lvys emphasis is on how virtual collective intelligence on the Internet has the positive potential for freeing social activity, as well as enhancing knowledge production and dissemination. This aspect of collective intelligence relates well to MINDtouchs multi-interaction, collective/collaborative approach, and partially inspired the practice as a modality to bring people together to create and experience collectively. A personal interpretation and experience of virtuality reveals that it can be accessed through the imagination or the dreaming body, and conveyed from within an altered state, expanded sensory awareness, perception, imagination and daydream imagination, or extrasensory perception. It also includes embodied perceptions of reality, which create our natural, invisible, inner worlds and consciousness, the real experiences within an altered dimension, or dreams, psychic experiences, meditation, mysticism, near-death experiences, intuition, or vision quests involving physical pain (Baker, 2004: 33). Virtuality is about emotional connection with others not spatially or temporally linked. The concept of working in real-time must be addressed, as it is a goal of this project. Real-time in the networked digital world is a variable concept, and Mitchell addresses this by saying: [] there is more to the construction of time than the increasingly precise subdivision of the day. As clocks multiply and distribute themselves spatially, the relationships among them begins to matter. (2003: 12) and,
As transportation and telecommunication capacities have increased, we have entered the era of globalized network time of GMT, time zones, and sleep cycles decoupled form the solar day. (2003: 13)

The effect of this is that networked, digital real-time activities are not predicated on where something is happening, or on what time it is at any one location; it is more important that wherever and whenever you are on the planet, you are experiencing the event or activity on the network now, as the action is
210 Being physically in body, rather than the mind wandering from it.

175

unfolding. Mitchell adds to this [] if you take advantage of fast machines to compress process, you can elide the distinctions between simultaneity and sequence. (2003: 13) In this way, through mobile and networked media, we can be in real-time, anywhere, virtually, even if we are not there, where the event is physically produced or taking place. Mitchell speaks of time in terms of quantum time (2003: 14) and the state of limbo [211] that is experienced as information transfers through the network, or as someone travels through the air in a plane. He describes it as a sense of nowhere, a no-time space or a state where time and space is traversed, but not defined. In this respect, quantum time is very much like becoming in the Massumi sense. Addressing this state or space, Mitchell says: [...] my sense of continuity and belonging derives from being electronically networked to the widely scattered people and places I care about (2003: 17). Looking at time in terms of the digital realm of network delay, lag, or stops and starts in streaming, Kozel describes her experiences in creating and performing telematic and networked, interactive works. She tells of issues of delay and lag in the system or network, during a live interaction, as opportunities or even as features of the experience in this form of performance. She reveals at one point:
Performance using low-bandwidth Internet video conference links call attention to the neverceasing flow of interaction among bodies, spaces and equipment by means of consistent interruption. There is a shifting and juddering unpredictability to the movement of the image, and to the regular crashes or adjustments required for maintaining a link. Nothing is inert. The fragility of interaction across distances echoes the vulnerability of proximity [] All interaction can be seen in terms of the negotiation of flow [] (2007: 108109)

This reflection refers to a 1997 piece she performed in. However, using mobile video ten years later, I have encountered the same issues with telecommunications technology and the network for mobile video interaction. Kozels point is that it is important to view the pauses, delays and failures in transmission and interaction as metaphors for more immediate physical forms of human communication or interaction, which are as easily blighted with the same pauses, delays and failures in understanding. Thus, we continue to encounter such challenges with both technological and human interaction, and use these as opportunities to prompt solution-seeking. In summary, this chapter explores conceptual and practical issues around the notion of liveness, and its related quality of real time, or the more generalised shift in how time is perceived and played with when using new networked technologies for performance.

211 This is another type of space between. There is not space here to discuss it, as the term has a Christian origin, regarding the status of ones soul in oblivion. In common language, limbo means waiting in an indeterminate state, lacking resolution or certainty. It is a state between decisions.

176

Part III: conclusions & references for further study

177

9/ thesis conclusion:
This conclusion sets out to achieve three final aims: 1) to present a strategic summary analysis of participant feedback that has informed both the theory and practice of this thesis; 2) to demonstrate the original contributions to scholarship achieved by the thesis project itself; and 3) to outline a number of new areas for future research arising from this work, potentially of interest to future scholars and practitioners alike.

9.1/ implications and new directions

In the thesis as a whole, I have gathered and analysed individual experiences described by participants in the projects mediated participatory events and explained the event concept within the virtual space of a mobile. Networked performance has been analysed, interpreted and correlated to the technology used to enable the participatory experience. This includes the practice of facilitating the sensing of embodiment and liveness, within an in-person and remote, mobile participatory event. The contribution to knowledge includes the explorations and attempts to create and study distance perception and interaction of the sensation of liveness and felt presence of others (or lack of same), within real-time, inperson and telematic events, using mobile and wearable devices and interfaces. I posit that mobile media phones can become a medium or conduit for expression of non-verbal experience, and an extension of the body/mind in non-literal or abstracted ways: a vehicle to express embodied consciousness with others, over distance. With this research, I have explored and straddled various artistic and academic disciplines, communities, practices and methodologies: the new era of mobile arts, interactive, Internet performance and installation art, wearable technology and fashion, body practices and philosophies, as well as telecommunications technology. The unique knowledge contributed also includes a phenomenological perspective on the experience of the process of creating a mobile media, networked performance project and the complexities encountered when working with mobile networks as against the Internet. This exegesis has described the practical aspects of the media art PhD project MINDtouch, as manifested in the initial participant video collection workshops, conducted 20072008. The material

178

from these workshops was used during a few live events, staged from July 2009 to early 2010, with participants wearing biofeedback sensors, which interacted with mobile devices. Observations, discoveries and outcomes from all phases of the project, as well as the technological challenges faced, have been outlined. Theoretical notions on liveness, presence, embodiment, non-verbal communication, telepathy, participatory performance and mobile media, as well as live cinema, database cinema and networked performance were explored herein. These theoretical ideas guided my approach to the practical implementation and development, including: the workshops, iterative mobile and VJ software development, and the performance events. The ideas guided how participants were facilitated in the participatory activities and what would be focused on, as well as how the software was developed, and how the participatory events were staged overall. The intricacies of interweaving the theory and practice have been discussed here as well. Due to time and resource pressures it became difficult to complete the work of studying participants experiences of feltness or exchange of presence using mobile video, through the mobile network during participatory performative activities, as originally envisioned. However, phenomenological observations of my artistic and facilitation process, as well as the ethnographic observations and analysis of responses of the participants during the activities, have been greatly instructional. The actual video material created, as well as participant feedback on other significant outcomes, has been discussed. I learned that the embodiment of technology, the extension of presence, and consciously directed emotional and interpersonal connection through the mobile devices did occur for some through the activities described, but differently than anticipated. Through intent and desire to connect, participants were able to send and transform their presence through the device as one does through Internet engagement. I now see that through these qualities of intention and desire, we embody or send our presence through our technologies and over distance. I experienced a sense of liveness, of in-personness, during the last event, when preparing participants to start their video streaming. This was evident when, while dressing a participant in a biofeedback garment, several text messages suddenly showed up on my mobile, instantly, in real time, from the remote group who were waiting online on the qik website. These remote participants were asking, in real time, when we would be starting the live exchange; my response was to urge them to keep waiting a bit longer for us. These online participants did seem to experience some liveness of the stream of video, in real time, though they were unaware that the footage that they were seeing mixed was not live video upstreaming from the event. What they saw was a live mix of the archived videos, which they thought was live. These still account for manifestations of liveness that were created and experienced. This experiment continues, and is closer to manifesting whole connection, completing the technological loop of a body-to-video to bodythrough-video interaction, even though for the purposes of the PhD the technological development had to be paused to complete the written part of the research. More still needs to be done to investigate

179

the nature of presence and liveness experienced by participants in a mobile performance context: in future versions within various social contexts and with various different groups. The goal has been to interface technologies to aid in connecting remote groups, enabling them to reengage with each other, affectively and expressively in new non-verbal/textual, creative ways. Thus, this project brings together divergent areas of new media research and media art/performance practices, and contributes to additional ways of working with wearable devices, mobile phones and video in performance and participatory activities. It also contributes new ways of thinking about presence and embodiment in the digital media context with respect to interfacing the various technologies, practices and ideas. Knowledge contributed here includes a nuanced perspective of creating body-situated, participatory, mobile media performance work. This work could also have implications in the health and healing industries, education, affective computing/sensing domains, new forms of mobile media entertainment, and social and mobile computing fields. In addition, it provides recommendations for live, collaborative, locative and networked performance circles, in tandem with wearable tech art and other areas of interest. It could also have further applications in VJng/Live Cinema and database cinema practices. Yet its most practical contribution is the proposal of a new mobile social networking platform/product to engage and entertain people socially. After careful consideration of the concepts, similar art and research projects and MINDtouch, there remain questions for future consideration and research: is technological mediation as an extension of embodied presence attainable or fully desirable? Will 3D surround projection TV or other new visual and sensory technologies bring us closer to real, in-person presence? Do we want such a form of digital representation, or should we return to and continue to cultivate more personal approaches? My own answer to these questions, informed by this research and process, as well from conversation with people about digital versus in-person interaction, is that most people do not want the digital to replace in-person presence in life or in performance. As discussed herein, corporeal, fleshy physicality is the ultimate experience and condition for optimal interaction, communication and indeed survival, which we evolved to experience (Lewis et al., 2001). It has been made clear by A General Theory of Love [212], earlier in Bowlbys work, and other related relationship studies, that humans need in-person contact. This interaction establishes and maintains intimate bonds that continually develop neural pathways in the brain. As I study, think, feel and live, I am convinced that liveness and bodily, inperson, fleshy presence is the only the only real or authentic experience, the only way to clearly convey emotion and to have sustained and meaningful relationships with others. However, with the ever-increasing destruction and damage to our life-sustaining planet through excessive carbon-spewing travel, for business, food distribution, pleasure and trade, it is clear that we now need to find and use other solutions. Instead of face-to-face interaction and immediacy all the time, we need to continue to develop alternatives to fleshy contact. We need alternatives that feel real
212 See earlier discussion on this.

180

and authentic, and convey trustworthiness, both emotionally and physically. In my view, the only valid rational justification for substituting the fully embodied experience of in-person liveness and presence would be in order to save money, resources, lives and carbon emissions. This could prevent other destructive activities that threaten life under our current socio-economic, political and environmental frameworks. This is my conclusion and one that was echoed in one form or another by many artists and researchers who have discussed it during my research [213]. Where does this perspective leave the performance aspect of this research? This aspect remains under investigation since the work is still in progress. Cultivating mobile presence in the live and mobile performance context is still to be explored. I hope to continue this work and develop the technology further, creating more participatory performance activities and events, studying more participants and their perceptions, before making final assessments on liveness and presence in these contexts. But from the phenomenological discoveries regarding the mobile phone and digital communication and presence, I find myself still wondering, but with new insight to better prepare me for these continued studies. Throughout the PhD research process and the move to the UK, I have learned and experienced much that has underlined my concern for liveness and presence over distance. It is important within human relationships, communication and performance. This awareness has progressively become clear and pronounced. Coming to the UK from Canada would not seem a drastic move, but it has made me appreciate my former life and view it with a new perspective. This process has become very personal, as well as academic and artistic. All of my experiences during this time have become relevant and applicable to research into traversing distance and exuding presence over a digital and mobile network. In addition, during the period of my research, the world has gone through some radical transitions and shifts, highlighting the fact that globalisation has had a profound and sometimes problematic impact, causing people to be ever more distant from each other. I wonder, if we could learn to be present with each other, feeling and validating that presence, and try to communicate and empathise with one another better, perhaps then we could improve our world, as well as our personal experiences of life as we are living it, phenomenologically, and hopefully evolve further as a species and society. My motivation in this respect, and in my practice, is therefore one of social obligation, to make a contribution that has some social meaning and value, to affect and assist people, as well as to help them connect with each other more deeply. This work continues.

213 Such as at the Performing Presence Conference in Exeter, England, March 2009.

181

9.2/ in conclusion
This thesis benefits from access to the larger MINDtouch project and its original data set of processbased evidence files, in both video and transcript form (contained in the thesis Appendices). By analysing this unique data set and applying the theoretical contexts of kinaesthetic philosophies where appropriate, the thesis demonstrates both the practical and the critical/contextual effectiveness of media facilitation processes for the participants, and shares their senses of liveness and presence (of themselves and of others) when using technology to externalise visual expressions of internalised experiences. The thesis engages with the major MINDtouch case study and applies theories from fields of kinaesthetics and phenomenology, including current theories of embodiment, in considering notions of presence, liveness, feltness and telepathic exchange (all terms defined at length in these pages). The research brings together new thinking in the fields of media art theory, telecommunications, neuroscience, performance and new media practices. The practice-based approach to exploring theory through practice has involved designing and applying bespoke biofeedback sensors to interface with mobile phones, in the context of performance workshops incorporating theatre games and other creative approaches to being in both live and remote spaces. The thesis has combined methodologies, methods, practices and tools from Phenomenology, Ethnography, Practice-AsResearch and Experience Design, bringing together relevant aspects of these divergent areas of new media research and media art/performance practices. This thesis makes an original contribution to scholarship in the fields of performance and new media, with cognate contributions to the fields of philosophy and technology, and locates its arguments at the locus of the fields of Performance Art, Mobile Performance/Locative Media, and Philosophies of the body and Communications. The research (supported by original data gleaned from major case studies) demonstrates that there is a need for new technology tools to express viscerally felt emotion and to communicate more directly. The findings show that when intimate dream states are to be shared by participants, it is often the case that written and spoken language can block clear creative expression. On the other hand, more intimate media tools can help to enhance a sense of liveness and presence by supporting both the sharing and receiving of emotional messages via ubiquitous technologies linked by fairly simple online network protocols. It is hoped that this study will be of use to scholars in the arts and technology, and that it may help to demonstrate a way to communicate rich emotion through felt and embodied interactions with others across vast distances (thus supporting political movements aimed at reducing global travel in the age of global warming).

182

bibliography
Adams, K. (2003), The Threshold of the real: A Site for Participatory Resistance in Blast Theorys Uncle Roy All Around You In Body, Space & Technology Journal, 6(1) [Online] Available at http://people.brunel.ac.uk/bst/vol0601/kateadams/home.html (Accessed Jan 2010) London: Department of Performing Arts, Brunel University. Alpert, R. aka Baba Ram Dass (1971), Be Here Now, San Cristobal, New Mexico: Lama Foundation. Amerika, M. (2005/7), Excerpts From Portrait Of The VJ, Fibreculture Journal [Online] Available at http://journal.fibreculture.org/issue7/issue7_amerika.html (Accessed 12 July 2006). Amerika, M. (2009), Immobilit. [Online] Available at http://www.immobilite.com/ (Accessed 12 July 2010). Ansell Pearson, K. (2002), Philosophy and the Adventures of the Virtual, London and New York: Routledge. Argyle, M. (1975), Non-Verbal Communication in Human Social Interaction In Non-Verbal Communication, Hinde, R (ed), London and New York: Cambridge University Press. Auslander, P. (1999), Liveness: Performance in a Mediatized Culture, London and New York: Routledge. Auslander, P. (1997), Ontology vs. History: Making Distinctions Between the Live and the Mediatized, In Georgia Institute of Technology: The 3rd Annual Performance Studies Conference, Conference Archives, [Online] Available at http://webcast.gatech.edu/papers/arch/Auslander.html (Accessed 11 July 2006), Atlanta, Georgia: The School of Literature, Communication & Culture. Baars, B.J. (1997), In the Theatre of Consciousness: Global Workspace Theory, A Rigorous Scientific Theory of Consciousness, In Journal of Consciousness Studies Vol 4(4), Exeter, UK: Imprint Academic, pp 292-309. Baker, C., Schleser, M. and Molga, K. (2009), Aesthetics of Mobile Media Art, In International Journal of Media Practice, Volume 10 (2), to be printed July 2009 [Online] Available at http://www.intellectbooks.co.uk/journals/view-Article,id=8520/ (Accessed March 21, 2010), Bristol, UK: Intellect Press. Baker, C (2008), Liveness and Presence Bio-Networked Mobile Performance Practices: Emerging Perspectives, In The International Journal of Performance Arts and Digital Media, Volume 4 (2), [Online] Available at http://www.intellectbooks.co.uk/journals/view-Article,id=4976/ (Accessed March 21, 2010), Bristol, UK: Intellect Press, pp 117-136. Baker, C. (2004), Internal Networks: Telepathy Meets Technology in the DreamPod, Masters of Applied Science, Interactive Arts Thesis, Surrey, Canada: Simon Fraser University, pp 33. Baker, C. (2003), Sensorial Design for Interactive Art, unpublished paper / project proposal. Barker, C. (1977), Theatre Games: A New Approach to Drama Training, London: Methuen London Ltd. Barrett, E. and Bolt, B. (eds.) (2007), Practice As Research: Approaches to Creative Enquiry, London and New York: I.B. Tauris. Bartlem, E. (2005), Reshaping Spectatorship: Immersive and Distributed Aesthetics, In Fibreculture Journal: Distributed Aesthetics, Vol 7 [Online] Available at http://journal.fibreculture.org/issue7/issue7_bartlem_print.html (Accessed 20 March 2010) Melbourne, Australia: University of Melbourne.

183

Beauregard, M. and OLeary, D. (2007), The Spiritual Brain: A Neuroscientists Case for the Existence of the Soul, New York, NY: HarperOne, division of HarperCollins. Blast Theory (2003), Uncle Roy All Around You, performance group from Brighton, UK [Online] Available at http://www.blasttheory.co.uk/bt/work_uncleroy.html (Accessed July 2006). Blomberg, J., Giacomi, J., Mosher, A., and Swenton-Wall, P. (2003), Ethnographic Field Methods and Their Relation to Design, In D. Schuler and A. Namioka (eds.) Participatory Design: Perspectives on Systems Design. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc. pp 123 154. Bishop, C. (2006), Participation, London: Whitechapel Press and Cambridge, Mass: MIT Press. Boal, A., Jackson, A. (trans.) (1992), Games for Actors and Non-Actors, London and New York: Routledge. Borden, N. (2007), Research Insights: Social Networking and Mobile Communities, In Idlemode: Voices for Better Mobile Experience [Online] Available at http://idlemode.com/category/mobilepresence/ (Accessed May 21, 2007). Bourriaud, N. (1998, 2002), Relational Aesthetics, Dijon, France: Les Presses du Rel pp. 84-102. Bowlby, J. (1986), Attachment and Loss, Volume 1: Attachment, New York, NY: Basic Books. Bowlby, J. (1988), A Secure Base: Clinical Applications of Attachment Theory, Oxon, England: Routledge. Brandt, E. and Grunnet, C. (2000), Evoking the Future: Drama And Props In User Centered Design, In Proceedings of the Participatory Design Conference, New York: CPSR Conference Proceedings, pp 11-20. Bresheeth, H. (2007), Elasticity of Narrative Time: Time Travel in Science Fiction Cinema, In Journal of European Psychoanalysis, Vol 23 (2) [Online] Available at http://www.psychomedia.it/jep/pages/number23.htm (Accessed October 7, 2010), Rome, Italy pp. 59-84. Brouwer, J., Mulder, A. and Charlton, S. (eds.) (2003), Making Art of Databases, Rotterdam, The Netherlands: V2_Publishing / NAi Publishers. Burgess, M. (2002), Support For New Media Arts In Canada: What Place For Artists, In Sidestreet Art and Ideas, Online Journal, Fall 2002(2) pp 2. [no longer available online pdf copy available on request]. Byrne, B. and Carvalho, A. (eds.) (2006-2008), VJ Theory.net, journal and forum [Online] Available at http://www.vjtheory.org (Accessed October 2007), Falmouth, England: self- published. Carter, P (2004), Material Thinking, Melbourne University Press: Carlton, Victoria, Australia. Chodron, P (1994), Start Where You Are: A Guide to Compassionate Living, Shambhala: Boston. Christensen, M. S. (2006), As We May Feel: Interpreting The Cultural of Emerging Personal Affective Mobile Media, PhD Dissertation, Denmark: Department of Innovative Communication at the IT University of Copenhagen. Conway, F. (2005), Art of Decision, interactive multimedia exhibition [Online] Available at http://www.artofdecision.net/ (Accessed October 7, 2010) Costible, S. portfolio [Online] Available at http://www.sue-c.net/video.html (Accessed Jan 2010) Corino, G. (2006), Spatial issues and performative media in digital mobility: a network perspective, Digital Futures, MSc thesis, Institute of Digital Art and Technology [Online] Available at ! http://x.i-

184

dat.org/~gc2/digital_mobility.pdf (Accessed 13 July 2006), Plymouth, UK: University of Plymouth. Cummings, J.N., Butler, B. and Kraut, R. (2002), The quality of online social relationships, In How the Virtual Inspires The Real, Vol 45(7), Communications of the ACM Journal, New York, NY: ACM, pp103-108. Damasio, A. (1999), The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York, San Diego, London: Harcourt Brace & Company. Dark Fibre, performances (2009-2010), British Film Institute and onedotzero [Online] Available at http://www.onedotzero.com/ & https://www.bfi.org.uk/whatson/bfi_southbank/events/dark_fibre_0 (Accessed Jan 2010). Davis, M. (2000) A Curriculum Statement: Designing Experiences, Not Object, In Designing experiences, LOOP (1), American Institute of Graphic Arts, Virginia, USA: Virginia Commonwealth University Center for Design Studies. Dawkins, R. (2 ed.) (1989), The Selfish Gene, Oxford, England: Oxford University Press, pp 192. De Medeiros, M. B. (2003), Performance art and digital bodies, In Body, Space & Technology Journal, 3(2) Department of Performing Arts, Brunel University [Online] Available at http://people.brunel.ac.uk/bst/vol0302/mariabeatriz.html (Accessed 11 July 2006). Denizen, N.K. (2003), Performance Ethnography: A Critical Pedagogy and the Politics of Culture, Thousands Oaks, California, London, New Delhi: Sage Publications. Dixon, S. (2007), Digital Performance: A History of New Media in Theatre, Dance, Performance Art, and Installation, Cambridge, Massachusetts: MIT Press. Douglas, A. Scopa, K and Gray, C. (2003), Research Through Practice: Positioning the Practitioner as Researcher, In Working Papers In Art And Design, Vol 1[Online]. Available at http://www.herts.ac.uk/artdes1/research/papers/wpades/vol1/douglas2.html (Accessed 22 September 2006), Hertfordshire, UK: University of Hertfordshire, pp 5-6. Doruff, S. (2006), The Translocal Event and the Polyrhythmic Diagram, PhD dissertation, SMARTlab Programme in Performative New Media Arts, Central Saint Martins College of Art & Design, London: University of the Arts. Dourish, P. (2001), Where the Action Is: The Foundations of Embodied Interaction, Cambridge, Massachusetts: The MIT Press. Faulkner, M/D-Fuse (2006), VJ: Audio-Visual Art and VJ Culture, London: Laurence King. Farthing, G.W. (1992), The Psychology of Consciousness, Englewood Cliffs, New Jersey: Prentice Hall Press. Feijo, C., Gmez-Barroso, J. and Marn, A.A. (2007), Why YouTube Cannot Exist on a European Mobile: The European Regulatory Strategy on Mobile Content, In Mobile Media 2007: Proceedings of an International Conference on Social and Cultural Aspects of Mobile Phones, Media and Wireless Technologies, July 2-4, 2007, Goggin, G. and Hjorth, L. (eds.), Sydney, Australia: University of Sydney, pp 263-275. FILMobile (March 2008), Conference and Exhibition [Online]. Available at http://www.filmobile.net (Accessed Sept 2009), London, UK: University of Westminster. Freidman, D., Brogni, A., Guger, C., Antley, A., Steed, A., and Slater, M. (2006), Sharing and Analyzing data from Presence Experiments, In Presence Vol 15(5), Cambridge, Massachusetts: Massachusetts Institute of Technology.

185

Front, online magazine [Online] Vol 8, Available at http://front.bc.ca/research/texts/8 (Accessed Jan 2010) Gackenbach, J. (1998), From Sleep Consciousness to Pure Consciousness, In Presidential address at the annual meeting of the Association for the Study of Dreams in 1988, London, England. [Online] Available at http://www.sawka.com/spiritwatch/from.htm (Accessed 10 March 2003). See also A Continuum of Consciousness in Sleep [Online] Available at http://www.sawka.com/spiritwatch/gotoa.htm (Accessed 10 March 2003). Gates, C. (ed.) (2008), Rise of the VJ, In Vague Terrain Vol 9 [Online] Available at http://vagueterrain.net/journal09 (Accessed Jan 2010) Glaser, B with Holtor, J. (2004), Remodeling Grounded Theory, In Grounded Theory Review, Vol 4(1) [Online] Available at http://www.groundedtheoryreview.com/files/gtreviewvol4no1.pdf (Accessed 20 June 2007), Mill Valley, California: Sociology Press. Glesner, J. (2002), Internet Performances as Site-Specific Art, In Body, Space & Technology Journal, Vol 3(1) Department of Performing Arts [Online] Available at http://people.brunel.ac.uk/bst/documents/juliaglesner.doc (Accessed 10 July 2006), London, UK: Brunel University. Goldenberg, D. and Reed, P. (2008), What is Participatory Practice?, In Fillip Magazine Vol 8 [Online] Available at http://fillip.ca/content/what-is-a-participatory-practice (Accessed 23 January 2010), Vancouver, Canada: Projectile Publishing Society. Goffman, E. (1959), The Presentation of Self in Everyday Life, New York: Bantam Dell Pub Group. Green, N. and Haddon, L. (2009), Mobile Communications: An introduction to New Media, Berg New Media Series, Oxford and New York: Berg Publishers, a Division of Oxford International Publishers Ltd. Gumbrecht, H. U. (2004), Production of Presence: What Meaning Cannot Convey, Stanford, California: Stanford University Press. Hauber, J., Regenbrecht, H., Hills, A., Cockburn, A. and Billinghurst, M. (September 2005), Social Presence in Two- and Three-dimensional Videoconferencing, In International Society for Presence Research, Annual Conference Proceedings, [Online] Available at http://www.temple.edu/ispr/prev_conferences/proceedings/2005/presence2005.pdf (Accessed 7 July 2006), London, England: University College London, pp 189-198. Hegedus, C and Noujaim, J. (dirs) (2001), Startup.com, http://www.imdb.com/title/tt0256408/ (Accessed Jan 2010). film [Online] Available at

Idhe, D. (2002), Bodies in Technology, Electronic Mediations, Vol5, Minneapolis, Minnesota: University of Minnesota Press. Jaeger, T. (2005), live cinema unraveled: Handbook For Live Visual Performance, San Diego, California: self-published, with funds from UCSD Russell Grant. Jamieson, H. V. (2005), Mixing the Web, In Rhizome, an online media arts portal [Online] Available at http://rhizome.org/news/story.php?timestamp=20051121 (Accessed April 2006), New York: New Museum. Also see the WJ-S project [Online] Available at http://www.wj-s.org/ (Accessed April 2006). Johansson, T.D (2003), The Tactile Image: A Veronican Approach to Pictoriality in Michel Serress Aesthetics of the Senses, In Sense and Senses in Aesthetics, Backstrom, P. and Johansson, T.D (eds.) Helsinki, Finland: NSU Press.

186

Kait, C. and Weiss, S. (2001), Digital Hustlers: Living Large and Falling Hard in Silicon Alley, NYC, New York: HarperBusiness. Kaye, N., Giannachi, G., Slater, M., and Shanks, M. (2009), Performing Presence: From the Live to the Simulated, research project [Online] Available at http://presence.stanford.edu/. Also see more on that project [Online] Available at the Performing Presence Conference at the University of Exeter in March 2009 http://spa.exeter.ac.uk/drama/research/intermedia/conference.shtml (Accessed March 2009). Kolb, D. (1984), Experiential Learning, Englewood Cliffs, New Jersey: Prentice-Hall. Kozel, S. (2007), Closer: Performance, Technologies, Phenomenology, Cambridge, Massachusetts: The MIT Press. LaBerge, S. and Rheingold, H. (1990), Exploring the World of Lucid Dreaming. New York: Ballantine Books, a division of Random House, Inc. Ladly, M. (2007), Being there: Heidegger and The Phenomenon Of Presence In Telematic Performance, In International Journal of Performance Arts and Digital Media, Vol 3 (2&3) [Online] Available at http://www.intellectbooks.co.uk/journals/view-Article,id=4955/ (Accessed March 2010) Bristol, UK: Intellect Press, pp. 139150. Lvy, P. (1998), Becoming Virtual: Reality in the Digital Age, New York: Plenum Press. Lewis, T. MD, Amini, F. MD, Lannon, R. MD (2001), A General Theory of Love, New York: Vintage Books division of Random House. Libet, B. (2004), Mind time: The Temporal Factor In Consciousness, Cambridge, Massachusetts: Harvard University Press. Lord, M (2001), Fostering the Growth of Beginners' Improvisational Skills: A Study Of Dance Teaching Practices In The High School Setting, In Research in Dance Education, Vol (2)1 April 2001, London and New York: Routledge, pp 19 40. Makela, M. (2006), Live Cinema: Language and Elements, MA thesis, Media Lab, Helsinki University of Art and Design, Helsinki, Finland. Manovich, L. (2005), Soft-Cinema, interactive art project [Online] Available at www.manovich.net and or http://www.manovich.net/cinema_future/toc.htm http://www.softcinema.net/movies.htm (Accessed August 2009). Manovich, L. (2001), The Language of New Media, Cambridge, Massachusetts: MIT Press. Marks, L.U. (2000), The Skin of the Film: Intercultural Cinema, Embodiment, and the Senses, Durham and London: Duke University Press. Marshall, G. and Barthel, D. (eds.) (1994,1998), The Oxford Dictionary of Sociology, Oxford, New York: Oxford University Press. Massumi, B. (2002), Parables for the Virtual: Movement, Affect, Sensation, Durham and London: Duke University Press. Mazis, G (2002), Earthbodies: Rediscovering Our Planetary Senses, Albany, New York: SUNY Press, division of State University of New York Press. McCarthy, J. and Wright, P. (2004), Technology as Experience, Cambridge, Massachusetts: MIT Press. McKay, S and Jordon, J. (2006), portfolio [Online] Available at http://www.nomig.net/ (Accessed July 2006).

187

McKean, E. (2005), The New Oxford American Dictionary, 2nd Ed., [Online] Available at http://www.oup.com/us/noad (Accessed May 25, 2010), Oxford: Oxford University Press. Merleau-Ponty, M. (1962, reprinted 2002), Phenomenology of Perception, London and New York: Routledge. Mindell, A. (2000), Quantum Mind: The Edge Between Physics and Psychology, Portland, Oregon: Lao Tse Press. Mitchell, W (2003), Me++: The Cyborg Self and the Networked City, Cambridge, Massachusetts: MIT Press. Mixed Reality Lab, [Online] Available at http://www.mrl.nott.ac.uk/people.html (Accessed January 2010), Nottingham, UK: University of Nottingham. Munster, A. and Lovink, G. (2005), Theses on Distributed Aesthetics. Or, What a Network is Not, In Fibreculture Journal 2005(7), Fibreculture Publications [Online] Available at http://journal.fibreculture.org/issue7/issue7_munster_lovink.html (Accessed 12 July 2006). Murtagh, G.M. (2001), Seeing the 'Rules': Preliminary Observations of Action, Interaction and Mobile Phone Use. In Wireless World. Social and Interactional Aspects of the Mobile Age, Barry Brown, Nicola Green and Richard Harper, (eds.), London and Berlin: Springer-Verlag UK, pgs 81-91. networked performance, online blog (2005b), turbulance.org, an open online forum, topic: networkenabled performance [Online] Available at http://www.turbulence.org/blog/ (Accessed July 2005). Neumark, N. (2005). Introduction: Relays, Delays, and Distance Art/Activism, In At A Distance: Precursors to Art and Activism on the Internet, Chandler, A. and Neumark, N. (eds.) Cambridge, Massachusetts: MIT Press, pp 2-31. Nicholson, J.A. (2005), Flash! Mobs in the Age of Mobile Connectivity, In Fibreculture Journal 2005(6), Fibreculture Publications [Online] Available at http://journal.fibreculture.org/issue6/issue6_nicholson.html (Accessed 12 July 2006). Nowak, P. (November, 20, 2007) CBC News online [Online] Available at http://www.cbc.ca/news/background/tech/cellphones/economy.html (Accessed May 25, 2010). Oulasvirta A. Kurvinen, E., Kankainen, T. (2003), Understanding contexts by being there: case studies in bodystorming, Vol 7, Pers Ubiquit Comput, Springer-Verlag London, pp 125134 Pearson, C. (1992), Precognitive Dreaming, In A Public Experiment in Precognitive Dreaming, (1993) The Annual Conference of the Association for the Study of Dreams (1994) [Online] Available at http://www.nauticom.net/www/netcadet/npcg.htm (Accessed 10 March 2003), Pittsburgh, PA: The Pittsburgh Post-Gazette Dream Experiment. Phelan, P. (1993), Unmarked: The Politics Of Performance, London and New York: Routledge. Philips (2007), Design Probe Program: Skin, interactive presentation of research [Online] Available at http://www.design.philips.com/sites/philipsdesign/about/design/designnews/pressreleases/skin_red dot2007.page and http://www.design.philips.com/probes/projects/dresses/index.page (Accessed http://live.philips.com/index.php/nl_nl/video/design-probe-skin-dresses/25262423001 September 2009). Ponton, A. (2005), Vertigo: The Technophenomenological Body in Performance, Fine Art PhD Dissertation, London, UK: Goldsmiths College, University of London. Presence Project (2006-2009), research project, [Online] Available at http://presence.stanford.edu/ (Accessed July 2009), Stanford University, Palo Alto, Ca; Exeter, UK: University of Exeter and London: University College of London.

188

Radin, D. (1997), The Conscious Universe: The Scientific Truth of Psychic Phenomena. New York: HarperCollins Publishers Inc. Reason, P. and Bradbury, H. (eds.) (2001), Handbook of Action Research: Participative Inquiry and Practice, Sage: Thousand Oaks, CA. Rees, A. L. (1999), A History of Experimental Film and Video, London: British Film Institute. Rettie, R. M. (2005), Presence and Embodiment in Mobile Phone Communication, PsychNology Journal 2005, Vol 3(1), [Online] Available at http://www.psychnology.org/File/PSYCHNOLOGY_JOURNAL_3_1_RETTIE.pdf (Accessed 10 June 2007) pp 16-34. Richardson, I. (2005), Mobile Technosoma: Some Phenomenological Reflections On Iterant Media Devices, In Fibreculture Journal 2005, Mobility, New Social Intensities and the Coordinates of Digital Networks, Issue 6 [Online] Available at http://www.journal.fibreculture.org/issue6/issue6_richarson.html (Accessed 10 June 2007). Rogers, K. (1995), THE PSI NET PROJECT Psi-Phenomena - Telepresence - Virtual Reality, documentation on multimedia project, [Online] Available at (Accessed http://web.ukonline.co.uk/Members/kathleen.rogers/psinetweb/psimain.html 01/15/2003) Rose, G. (2007), Visual Methodologies: An Introduction to the Interpretation of Visual Materials, 2nd Ed, London, UK; Thousand Oaks, California, and New Delhi, India: Sage Publications. Sadler, S. (1998), The Situationist City, Cambridge, Massachusetts: The MIT Press. Schechner, R. (1988, 2003), Performance Theory, London and New York: Routledge. Schleser, M., Baker, C. and Molga, K. (2009) Aesthetics of Mobile Media Art, In Intellect: Journal of Media Practice, Vol 10 (2&3), [Online] Available at http://www.intellectbooks.co.uk/journals/viewArticle,id=8520/ (Accessed May 2010), Bristol, UK: Intellect Press, pp. 101122. Schiphorst, T and Anderson, K. (2004), Between Bodies: using Experience Modeling to Create Gestural Protocols for Physiological Data Transfer, International Conference For Human-Computer Interaction Systems, Vienna, Austria: Conference Proceedings for CHI 2004. Schiphorst, T. (2007), The Varieties of User Experience: Bridging Embodied Methodologies from Somatics and Performance to Human Computer Interaction, PhD Dissertation, School of Computing, Plymouth, United Kingdom: CAiiA Star at the University of Plymouth. Schroeder, R. (2006), Being There Together and the Future of Connected Presence, In Presence Journal, Vol 15 (4), August 2006, Cambridge, Massachusetts: Massachusetts Institute of Technology, pp 438-454. Schuler, D., and Namioka, A. (eds.) (1993), Participatory Design: Principles And Practices, Hillsdale, New Jersey: L. Erlbaum Associates. Sconce, J. (2000), Haunted Media: Electronic Presence from Telepathy to Television, Durham and London: Duke University Press. Shedroff, N. (1994) Information Interaction Design: A Unified Field Theory of Design, In Design, ed. Bob Jacobson. Cambridge, Massachusetts: MIT Press. Sheldrake, R. (2003), The Sense of Being Stared At: And Other Aspects of the Extended Mind, New York, NY: Crown Publishers. Sheldrake R., podcasts. [Online] Available at

189

http://www.sheldrake.org/B&R/audiostream/index.html (Accessed June, 2009). Shusterman, R. (2008), Body Consciousness: A Philosophy of Mindfulness and Somaesthetics, New York, NY: Cambridge University Press. Shusterman, R. (2000), Performing Live: Aesthetic Alternatives for the Ends of Art, Ithaca and London: Cornell University Press. Simpson, J. and Weiner, E. (eds.) (1989), Oxford English Dictionary, 2nd Edition, Oxford: Clarendon Press. Skoltz_Kolgen, portfolio [Online] Available at http://www.skoltzkolgen.com/ (Accessed Jan 2010). Smith, G.J. (2009) Variable Frame Rate: Multimedia Performance at MUTEK 2009, In Rhizome, online artists portal [Online] Available at http://rhizome.org/editorial/2668#more (Accessed Jan 2010) affiliated with New Museum: New York, NY. Smith, D.W. (2009) Phenomenology, In The Stanford Encyclopedia of Philosophy (Summer 2009 Ed.), E. N. Zalta (ed.), [Online] Available at: http://plato.stanford.edu/entries/phenomenology/#4 (Accessed May 4th, 2010). Spanos, W. V. (ed.) (1976), A Casebook on Existentialism 2, New York and Toronto: Harper & Row, Publishers, Inc. Sullivan, G (2005), Art Practice As Research: Inquiry In The Visual Arts, Thousand Oaks, California and London, UK: Sage Publications. Takahashi, Y., Uchida, C., Miyaki K., Sakai M., Shimbo, T., and Nakayama, T. (2009), Potential Benefits And Harms Of A Peer Support Social Network Service On The Internet For People With Depressive Tendencies: Qualitative Content Analysis And Social Network Analysis, In J Med Internet Res. 11(3):e29, National Institutes of Health, [Online] Available at http://www.ncbi.nlm.nih.gov/pubmed/19632979 (Accessed May 31, 2010) Bethesda, Maryland: National Library of Medicine. Tate Collections, online gallery and library [Online] Available at http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=276 and http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=155 and http://www.tate.org.uk/collections/glossary/definition.jsp?entryId=218 (Accessed January 2010) Terry, D. (2005), mo.vid.1, film [Online] http://www.deanterry.com/blog/index.php/2005/10/ [Accessed Dec 2008]. The Third Man Factor, The Current Part 3 (podcast) (27 Jan 2009), CBC Radio [Online] Available at http://www.cbc.ca/thecurrent/2009/200901/20090127.html or http://podcast.cbc.ca/mp3/current_20090127_11346.mp3 (Accessed March 10, 2009). Thompson, N. and Sholette, G. (eds.) (2004), Interventionist: Users Manual for Creative Disruptions of Everyday Life, North Adams, Massachusetts: Massachusetts Museum of Contemporary Art. Thorington, H. and Riel, M. (moderators) (2005a), soft_skinned_space -empyre-, listserve, topic: networked performance [Online] Available at http://www.subtle.net/empyre/ (Accessed 10 June 2007). Turkle, S. (1997), Life on Screen: Identity in the Age of the Internet, New York, NY: Simon & Schuster Inc. UK Video Art History, online archive [Online] Available at http://ukvideoart.tripod.com/ (Accessed January 2010).

190

Ullman, M. and Krippner, S. (1973, 2002), Dream Telepathy: Experiments in Nocturnal Extrasensory Perception. Charlottesville, Virginia: Hampton Roads Publishing Company, Inc. Varela, F. and Shear, J., (eds.) (1999), First-person Methodologies: What, Why and How, In The View from Within: First-person Approaches to the Study of Consciousness. Exeter, UK: Imprint Academic, University Press. Varela, F., Thompson, E., Rosch, E. (1991), The Embodied Mind: Cognitive Science and Human Experience, Cambridge, Massachusetts: MIT Press. Video Art in Canada, online archive and encyclopaedia [Online] Available http://www.thecanadianencyclopedia.com/index.cfm?PgNm=TCE&Params=A1ARTA0008366 (Accessed Jan 2010). VIVO (Video In/Video Out studios), organisation website http://videoinstudios.com/aboutus.php (Accessed Jan 2010). [Online] Available at

at

VJ Theory, online forum and journal [Online] Available at http://www.vjtheory.org (Assessed April 2008), Falmouth, UK: self-published. Wenders, W. (1991), Until the End of the World, film, Germany / France /Australia co-production: Argos Films. whisper(s) project (2002-2006), research project, lead by Thecla Schiphorst, School of Interactive Arts and Technology, Simon Fraser University, Surrey, Canada. Williams, J. P. and Smith, J.P. (2008), The Players' Realm: Studies On The Culture Of Video Games And Gaming, In Margin to the Center ++, New Media a Critical Introduction, Martin Lister, Jon Dovey, Seth Giddings, Iain Grant, Kieran Kelly (eds), London and New York: Routledge. WJ-S project, research project [Online] Available at http://www.wj-s.org/ (Accessed July 2008). Wolf, F.A. (1995), The Dreaming Universe, New York, NY: Simon & Schuster. Youngblood, G. (1970), Expanded Cinema, New York: P. Dutton and Co., Inc.

191

Part IV: Appendices (not for examination, but for elucidation and background information for the examiners)

192

Appendix I/ the MINDtouch project (BBC R&D sponsored)

193

A.1/ - technical appendix and background source material on mobile performance, practice and process
This Appendix is intended to provide a detailed background account of the major case study project MINDtouch upon which many of the thesis conclusions are based. This section describes the practical aspects of devising the MINDtouch Project, manifested through the initial video collection workshops, conducted from July 2007 Feb 2008, as well as the initial live experimental participatory events from July 2009 to February 2010. These workshops and events involved performers wearing biofeedback sensors while interacting with mobile devices. Also discussed are the technical issues of developing custom software for mobile phones, and how the mobile and wearable technologies were interfaced with each other technically, performatively and otherwise, as well as the design process of embedding the sensors in clothing. The practical project processes included developing techniques to encourage people to connect to remotely to each other, in order to re-engage with each other and with the world affectively. This was done through non-verbal/textual, non-linguistic activities in mobile, networked activities. As such, participants were asked to use their bodies as well as a range of expressive, creative, non-textual means to communicate visually using mobile videophones. This involved short activities during workshops and in organised social events, which were mainly improvised by participants through guided structured improvisation parameters, devised to assist participants perform the activities. The interaction between the mobile participants and those wearing the sensors was intended as a means to emulate the telematic sharing of dream imagery or other pre-linguistic communication. A creative, technological way to simulate and enable participants to imagine a telepathic exchange between themselves and other people was emphasised, much like lucid dreaming visually and affectively together (LaBerge and Rheingold: 1990),. This is something I hope we will all be able to do one day with or without technology.

A.1.1/ mobile video collection workshops

This section is dedicated to details of the phase one workshop process (see REF. Figure 1 on page 18). MINDtouch workshops were initially conducted as a preparation for creating the performative video events and collaborative artefacts. During the video collection process, participants in this first phase were asked to explore their internal images, emotional and affective senses and states, internal physical sensations, impressions of their immediate surroundings and body surfaces; inner thoughts, consciousness, pondering, internal conversations; reflections and impressions, in non-verbal, visual ways using the video capabilities of their phones. They were then encouraged to share these visual sensation explorations with others and myself. The archive of personal clips was used later during the

194

collective collage in the VJ performance events (see the script of activities in the Appendix II).

Images 33, 34, and 35 2007 C. Baker still images by participants in video collection workshops

Exploring the more emotive, sensory experience of presence has been the focus of this exploration, so I have endeavoured to tap into visceral, bodily expressions that might surface during body practices as a primary source of non-verbal, pre-conscious communication. Five video clip collection workshops were conducted from June 2007 to February 2008, each consisting of four to eight participants [214]. They were conducted in Vancouver, Canada (twice), Dublin, Ireland, (twice) and London (once). Participants of the video collection workshops were asked to express themselves using the video recording feature of each mobile phone, either using their own videophones or ones supplied. Briefly [215], the workshop procedure involved several people participating in a set of specific guided mind-quieting activities [216] before being guided through several video capturing exercises. These exercises asked them to visualise their internal experiences, thoughts, sensations, emotions and perceptions in a non-linguistic way, with the available surrounding and visual fodder captured on their video mobiles (or mine). In one activity in the workshops, participants were asked to introduce themselves to each other nonverbally, using the videophones. The participants each decided to approach the activity differently. Some decided to record parts of themselves, then show the recordings to each other as a way of introduction. In the last activity, participants were asked to work with a partner non-verbally, in order to create something that they might send to someone else, telepathically or through the mobile phone. This was intended to be something that would express their inner perceptions and experiences. As a result, several participants often went outside, each pair developing a different method to carry out the activity. Each outcome was fascinating. Another activity was introduced in later workshops, involving partners sitting back to back, trying to telepathically send an image or sensation they had to one another. After this they were asked to visually capture what they had sent. Each would then guess
214 Performances stared in July 2009, so the gap in the timeframe between conducting the workshops and conducting the performance events was due to the time spent in phase two, developing biosensing electronic devices and garments, as well as all the VJing / mixing software and network development. 215 There is a longer description on the workshops and analysis later in this Appendix. This is more for an overview understanding of how they were designed and conducted. 216 By this I mean a guided relaxation and visualisation exercise.

195

what the other person had sent, and then they would share their videos with each other to see if all three versions (the telepathic, the videophone and the guess) were similar. The activities were followed by showing the participants how to upload the videos online [217] to share with others in the group and myself, followed by an exit questionnaire and finally by an on-videophone interviews. More on the outcomes of these workshops is detailed below.

A.1.1a/ biosensors and electronics The biosensing system research commenced in November 2006, with acquisition and development of the biofeedback hardware, software and mobile phones. The biofeedback sensors initially used were very expensive but robust. They contained a professional medical grade system that monitored: EMG (electromyography: measuring muscle electricity), GSR (galvanic skin response), respiration and (initially) BVP (blood volume or pressure). This last was later changed to a Temperature sensor. These specific sensors were evaluated as the most responsive and easiest to work with, as those which could generate usable data, and ones which were least uncomfortable for the person wearing them [218]. The EMG measures muscle electricity to determine the muscle tension of larger muscles, like arms and legs. GSR response measures the electrical conductance or resistance of the skin that can change when people change their stress levels, usually worn on the fingers [219]. Respiration sensors are worn around the ribcage and monitor the inhalation and exhalation depth and frequency of the abdominal or chest expansion and contraction. BVP monitors the relative blood flow in the fingertips, using near infrared light. The heart rate can then be determined from the pulse or blood pressure. Although the first set included BVP, we changed to using temperature sensors, which read the body heat of an individual. These sensors were attached to the body via sticky electrode pads or Velcro attachments. Using wireless sensors was most desirable in order allow for ease of movement, but at the time they could only be acquired as wireless (in terms of not being connected by wires directly to the computer) using a Bluetooth hub (or Zigbee) carried or worn on the body, which sends the signals wirelessly to the computer. In this case, the data is sent to mobile phone or the computer to receive, process, and visualise the data. However they were ultimately rejected as unsuitable since as the images below demonstrate, there is still a great deal of cabling from the sensors themselves to the hub.

217 Initially we used a free public online video editing website eyespot.com, but it went defunct last year with no venue model, and other easily assessable online upload and video sharing tools have replaced it. More on the technology and technical development will be discussed in future papers. 218 Based on research and prior experience working with the whisper project. 219 A video game was made in 2002 called Journey to the Wild Divine (still available at http://www.wilddivine.com/wilddivinedemonstration-page.html) was made using the GSR; the lower one gets their stress levels, the higher the players score. The aim is to help kids or other player learn how to control their stress levels and their bodys responses.

196

Image 36 2007 Mind Media marketing images of the Nexus 4 Bluetooth Sensor transmitter

Images 37, 38, 39, 40 and 41 2007 Mind Media: electrode cable for EMG, sensor ground cable, Galvanic Skin Response, Respiration Sensor, Blood Volume Sensor (image source from Mind Media technologies) [220]

While these expensive medical grade sensors themselves were high quality, the short, thick, heavier and more restrictive cabling on the body encumbered movement and became unsightly for performance unless hidden within clothing. This first system came with simplistic visualisation software used for clinical stress reduction, not appropriate for MINDtouch in terms of how the body data [221]. In turn, this system was abandoned in favour of creating cheaper, more customisable, DIY sensors, and data visualisation software, which changed the approach to working with the sensors in the process.

Images 42, 43, and 44 2008 C. Baker: Michael Markert developing the custom biofeedback sensors & electronics.

220 Mind Media sensing systems [Online} Available at http://www.mindmedia.nl/english/sensors.php 221 We had to return this system for this reason and due to the time lost in miscommunication with the companys programmers and representatives when trying to get access to the SDK, which they had previously agreed to provide so we could develop our own custom data visualisation software for the mobile phones.

197

German electronic musician/creative electronics artist/ Apple, Java and mobile programmer Michael Markert joined me as a collaborator in Spring 2008, assembling and creating the DIY biofeedback sensors with a Bluetooth Arduino system. I then found other help to embed the sensors within customised clothing from two fashion designer colleagues. Effort was made to create garments that allowed for more flexible, unencumbered movement, and that were more aesthetic, ubiquitous and unseen on the body, creating a hidden body area network [222] or sensing system for performance contexts [223].

Image 45 2008 C. Baker: custom biofeedback visualisation software.

The intention was to develop subsequent mobile software for the live mixing and streaming of the videos and delivery to the server database for real-time collaborative video collage. This software was to allow people to share and interact with the visuals through their phones, in real-time. The concept was that the custom mobile VJ software would receive the data from the body, made in Mobile Processing [224] or another similar visual programming language or environment. This was envisioned to work somewhat similarly to eyespot.com (a very simple and elegant web-based, mobile video editing and sharing software, now sadly defunct) [225]. It seemed a realisable task, based on witnessing whisper[s] integration of biofeedback sensors with PDAs in 2005. With the software that Michael Markert developed to operate on the phones working well with the sensors, I then proceeded to work with another programmer to make the mobile VJ software, to work in tandem with the biosensing receiving software. However, to make VJ software of the sophistication I wanted, was not and is still is not possible to run on the operating systems of current mobile phone models on the market. Theoretically, it will be possible for the iPhone or others in the very near future.

222 This term, while used by many others was picked up from the whisper project, is used in medical and physiological sensing contexts. 223 Although this proved more difficult than expected and ultimately to have no wires, using conductive threads and materials would be the best approach, for another project as it was outside the time constraints and resources of the MINDtouch project. 224 Processing is a visual programming language designed by John Maeda at MIT as an easy language for artists to learn to programme to make Internet visual art and visualisations and has a mobile version available. 225 It is my impression that eyespot.com went offline and its technology bought out because they did not have a sustainable business model and could not maintain the costs that its huge popularity brought it, Sadly, it seemed that they did not manage to get a sponsor. Unfortunately, I have no screenshots of the web software interface, which was amazing. They went under suddenly and I did not even have time to rescue all the videos I and participants loaded there, but I was able to get the videos from some of the participants after the fact.

198

Image 46 2007 Brady Marks with permission (for a spin-off project): vision of the mobile interface for social VJing

Near the end of the development period, Manjit Bedi joined the technical team, specifically to create the VJ software system using Apples Quartz Composer programming environment. The software Manjit created uses the body data sent from the sensors within the lightweight garments to mix and stream the video into the database of clips, then adds live visual effects to the video clips. The software accesses and mixes these video clips or lives streams from four folders on the server into one mixed video. Lastly, QuickTime Streaming Server streams the video out the mixed video back out (ideally), in real-time to the web and to RealPlayer or web-enabled phones. This allows the remote audience members to share and interact with the resulting generative visual collage of other peoples video.

Image 47 2009 C. Baker: Quartz Composer early software patch view

199

A.1.1b/ garment design Fashion designers Tara Mooney and Rachel Lasebikan started working on the MINDtouch project as part of the SMARTlab wearable research group in July 2008, continuing through to the autumn of 2009. Their aim was to design lightweight, flexible and fun garments in which to embed the sensors. [226] What resulted in the first iteration of the MINDtouch garments, due to the limited time and resources available for this aspect of the research, was a set of attractive pieces of clothing that successfully hid the electronics but that were not, unfortunately, easy to self-fasten. As a result the power, batteries and Arduino connections were not easy to access by participants wishing to start the system. Due to extra wear and tear imposed by the dressing and undressing procedure, the electronics were gradually compromised and were not suitable for general use after a limited number of experimental events. [227] The second garment system was designed in such a way that it tightened around the chest of participants so that the breath sensor could measure the rise and fall of the ribcage with inhalation and exhalation. This was a key requirement, however it was constructed in such a way so as to pull too hard on the fragile sensor, so it was damaged. Due to scheduling conflicts, closer collaboration with the designers was not possible, so I was unable to attend to the fragility of the electronics. Due to the nature of the sensor components, the electronics (Arduino and the DIY sensor kits combined) were inexpensive and more fragile, especially when exposed to wear and tear, and were too easily damaged. In addition, while less expensive and more programmable, they still did not provide for easy movement, flexibility, durability and performativity necessary to truly work for the project. All of these factors would have to be taken into account in the construction of the garments for future designs, and I have since discovered that they are common stumbling block for artists and designers working with similar DIY sensors and electronics.

226 Both of the versions of the MINDtouch garments were designed with the inclusion of electronics in mind, but additional time for the user research would have enabled a more effective placement of the sensors for longer and more durable use. 227 The first set of garments with embedded systems has to date only been worn three times, and was not used for the last event as it was being repaired.

200

Image 48 2009 C. Baker: Alpha / Garment 1 with embedded sensors

It would, of course, have been an option to use other prefabricated garments with embedded systems (as such do exist in the open market, primarily for sports and health analysis), but most such existing garments are expensive and tied to proprietary software tools and exclusive research IP arrangements. Thus, none of the existing garments would have been suitable for this project without significant refitting and legal negotation, and after the difficulty with the first biofeedback sensor provider, this was not of interest or financially viable. In hindsight, working with an already researched, designed, and tested garment might have saved time but would have added significantly to the projects cost and would also have had an impact on the originality of the research outputs. This last factor in particular was to me, a considerable issue for purposes of use of the results in the original contribution of this thesis.

Images 49, 50, and 51 2009 C. Baker: Beta / Garment 2 with embedded sensors

201

A.1.1c/ performance events This section details the performance events design and staging. As explained, phase three performance events sought to create a social, abstract, non-linear narrative/montage or remix collaboration, to be streamed back to audience participants phones and to the Internet. These events attempted real-time video mixing and streaming to phones, although in practice there is significant lag and delay given the inherent network and security issues discussed earlier. With this in mind, there were multiple manifestations of presence intended to be experienced and witnessed by participants in the workshops and performances, such as: the engrossed attention during a mobile phone exchange and the in-between space of the network connection; the physical presence of participants and the in-between space as their data was sent to the phone or computer, followed by the in-between space of the computer and phone communicating with the database of video clips; the captured emotional, aesthetic presences embodied within the video expressions of the clips; the physical co-presences of the participants in the space of the staged network events the virtual multi- or co-presences of the remote participants and the global co-presences of these collaborators.

Figure 5 2009 C. Baker - early technical diagram of the networked performance project

202

The diagram above shows the three groups and technical interactions envisioned. In practice, Group 2 became the remote participants. During the initial event, three different groups were invited to enter the party-like environment in the space. Each group was greeted by a guide who walked them through various activities: 1) Group 1 participants wore the biofeedback sensor garments and were guided through various movement activities, to activate the sensors in various ways, triggering the remote systems visual effects to be applied to the live and archived video in the database. Simultaneously, the effected video was displayed on screens around the room. There were only two garments and two sets of sensors, [228] so this group was small, with only one or two participants each event [229]. The participatory activities for this group involved: a) movement activities, and b) theatre games (see guiding activities scripts in the appendix);

Figure 6 2009 C. Baker showing the staging and organisation of the performing groups in the live context.

228 As in the diagram, I had wanted more than or envisioned at least 4-6 in this group but with limited funding and resources was only able to acquire a limited number of sensor sets and make a limited number of garments. Also due to the Bluetooth pairing requirements each limited a garment / sensor set could only be paired with one computer or one phone at a time, an I also had a limited number of phones, which often were used for the video streaming aspect, since many participants did not have compatibles phones for streaming (and I did not want to prevent participation and tried to accommodate everyone. 229 In future iterations, I still hope to have more.

203

Images 52 and 53 2009 C. Baker stills from July 2009 first performance

2) Group 2: This group was guided in video streaming activities using on their mobile phones like earlier

workshops, but with fewer, shorter activities and simplified ways of speaking by visualising internal sensations. These were streamed using qik software on the phones to the qik.com website, to then be transferred [230] with a custom php script to the server, and then to mix and the stream-ready database folders;

Images 54 and 55 2009 C. Baker

3) Group 3: This initially was to be the group who mixed the video from the database via custom software

on their phone (downloaded as part of the activity). They would then interact with the biosensor participants (choosing live or archived video, created by Group 2). Their choices would be mixed and with visual effects added by the body data from Group 1. This activity would be guided so the participants could easily use the videophone software and interact quickly and decisively, making a body narrative in their own way. In practice this event had to be changed. I discovered that the phone models being used and their
230 This did not entirely work due to a part conflict on the server but has been rectified for future events.

204

operating systems were not yet capable of implementing/operating this type of custom software (see vision interface above on page 179). Instead, I adapted the event so that Group 3 became the remote group who waited and watched for the streams to arrive on the projects qik page or through the provided video streaming link for QuickTime, RealPlayer (or Internet browser with the player plugin). The streams could then be received on the Internet on a computer or peoples mobile phones (ideally), using Real Time Streaming Protocol (RTSP) from the streaming server. This group followed guiding instructions on the Facebook event invite or on the research web page, which described how to make their own videos to stream via qik. Alternatively, they could just watch. I discovered that most of the remote participants had more voyeuristic behaviour, as they chose mostly to watch, with only a couple remote participants contributing their own video streams in the last event. From this observation and the lack of remote video contributions on qik, it became evident that this group required more attention and guiding through an online moderator, which I did not have or consider until after the fact.

Images 56, 57 and 58 2009 C. Baker

This was meant to be ideally a two-hour event, but I discovered that it can be made shorter depending on the social event it is part of and the attention span, enthusiasm and creativity of the participants. The key focus was the performance, not on the technology, so the technology was meant only as an experience facilitator. The events were designed to be mainly about exchange, non-linguistic visual play, collaboration, participation and interaction, exploring movement and spontaneity. The exact details of the technical set up and preparation for each event is located in the Appendices.

A.1.1d/ participant expectations From the London, February 14th (2010) performance, I discovered that there was an expectation by some participants that the videos streamed as part of the project would be 'high quality'. These assumptions were made by a few of the remote viewers who had no training in video production processes [231]. It appeared that they were expecting that the video mix would be HDTV quality due to the cutting edge technology used in the project [232]. The one of the viewers, who commented on the

231 This is not an assumption, I discovered this directly from the participant whom I refer to. 232 Perhaps too much exposure to Sci-Fi and so much highly produced media exposure, such as TV shows like CSI.

205

videos uploaded from previous in-person event participants, said that what they saw was amateurish. Compared to what they are used to on TV or in the movies [233], which uses the highest quality, produced videoit may seem amateurism. What many without video training do not seem to realise is that TV and film are highly produced and polished, with a high number of video editors and postproduction experts, on a high salary on big budget productions which takes months and sometimes years to create. As a new tool, mobile video is a new way of working with and understanding video production. As stated in Chapter 5, the blurry and low quality mobile aesthetics include compression, network delays, low bandwidth at times, poor image quality, as they are created by the device that way to start withI argue this has its own appeal and authenticity. It can show participants (and others) that something meaningful can come from this technology. In this respect it is still early days in terms of quality and possibility. The education of audiences on the value of lower quality resolution as an aesthetic [234] is also important, I have discovered. Thus, audiences need to be shown that TV is a specific product highly produced to look as super slick as it does. Conversely, the low-fi phone video goes through many compression processes to make it small enough to move easily through the Internet and mobile networks. It is not the same product. Something can be gained, however, from mobile video through its immediacy and everyday personal imagery, versus other pre-produced, highly edited, and somewhat elitist artefacts. For the non-technological types, I would hope that this project has the potential to change their expectations from the removed polished, non-interactive media, towards getting involved in creating something themselves. Another important point is that the video may in fact be amateurish, since the goal was to have non-artists/non-professional videographers to make the video expressions. The inclusive nature of the facilitation and the event attempted to encourage other participants to refrain from judgment, which would result in further inhibiting participation. This last substantiation is thereforehow I responded to the one person who commented on the amateurish video.

A.1.2/ outcomes, analysis + interpretation: performances events

This outcomes section is dedicated to details of phase 1 of the MINDtouch workshops and performance events, observations and data analysis. It delves deeper into the discoveries of the practical research. It especially focuses ethnographically on participant feedback, as well as upon my phenomenological experience as creator of such a project.

233 Even compressed, originally high-definition video on YouTube looks better because it starts off much higher resolution before it is compressed for the web than mobile video. 234 While the manufacturers work hard to make it better see specifications on the newest video phone and all its enhanced video recording features available online at http://europe.nokia.com/find-products/devices/nokia-n8/specifications# Video and HD compatible hi-def resolution http://europe.nokia.com/find-products/devices/nokia-n8/features

206

A.1.2a/ workshop outcomes and observations As already discussed, participants of the phase 1 video collection workshops, conducted between June 2007 and January 2008, were asked to express themselves using the multimedia tools of mobile phones either using their own videophones or ones supplied.

Image 59 and 60 2007 C. Baker still images of participants in video collection workshops

Five video clip collection workshops were conducted in Vancouver (twice), Dublin (twice) and London (once). Each workshop involved meeting the participants, having them fill out consent for the research, a personal profile, and release forms for use of their video clips, an explanation of the research, conducting mind-quieting and body-tuning visualisations, and then four activities to capture video: two individual and two in pairs. The workshop procedure involved having participants experience a set of specific guided mind-quieting exercises to help them visualise their internal experiences, thoughts, sensations, emotions and perceptions, in a non-linguistic way, before being guided through several video capturing exercises to externally express these internal experiences by using the available surroundings and visual material. This approach or technique was used in a condensed way in the phase three performance event stage, but shortened and to a less immersive extent. The first workshop in Vancouver (16/06/07) was set in a big centre with a huge gallery space, many workshop rooms, a big black box theatre, and a gymnasium downtown next to the water. The initial mind-quieting activities took place in one of the workshop rooms and then after two individual videocollecting activities of five to ten minutes each, of which two were conducted in pairs. In one activity, participants were asked to introduce themselves non-verbally, using the videophones, to each other.

207

Images 61 and 62 2007 C. Baker - from Vancouver workshop June 2007

The participants each decided to approach the activity differently, some recording parts of themselves and then showing the recordings to each other, as a way to introduce themselves. In the last activity, participants were asked to work with their partner non-verbally in order to create something to send to someone else, through the mobile phone, something that would express their inner perceptions and experiences. As a result, several participants went outside. Each pair developed a different method to carry out these activities. The activities were followed by showing the participants how to upload the videos online [235] to share with others in the group and myself, followed by an exit questionnaire and finally by an on-videophone interview.

Image 63 and 64 2007 C. Baker - stills from Vancouver workshop June 2007

The second workshop took place in Dublin at the Beckett Theatre in Trinity College (22/07/07). The space for this second workshop was in a dance studio. The participants had more room to move around, and since a couple were dancers they tended to move more, which inspired the others to do so as well. The same four activities were facilitated (as in Vancouver), but in this instance the work was supported by a video documenter, which freed me as the researcher/workshop leader observe the participants and to assist when they were having trouble with the phones or needed more instruction. This group was, in general, more exploratory with their video expressions. There was also one

235 Initially we used a free public online video editing website, but it went defunct last year with no venue model, and other easily assessable online upload and video sharing tools have replaced it.

208

participant with Cerebral Palsy who had two assistants to facilitate his engagement in the workshop [236]. Participants were asked to interview each other with the videophones, rather than having one interviewer interviewing each of them individually, this was done partially to save time and to enable them to leave earlier after a long session. Even though participants did not know each other beforehand, and had usually come from different backgrounds, this proved to be successful. It might have also been the case since they had started to develop a rapport with each other in the workshop, sharing similar experiences, and this allowed them t become more open to discuss the experiences with each other.

Images 65 and 66 2007 C. Baker workshop still Dublin July 2007

The third workshop took place in Vancouver (21/08/07). Participants met in a waterfront park nearby on a beautiful hot, sunny day, which appeared to add to their enjoyment of the workshop. It was a much smaller group of four, thus, the first two individual activities were somewhat more difficult to conduct in such a public space, causing participants to seem somewhat self-conscious performing the internal and mind-quieting activities, with others in the park were throwing Frisbees and dogs were running about around them. One woman brought her child with her, so her focus was reduced, making it harder for her to relax completely and engage fully in the activities. Due to the bright sunshine and inspiring surroundings, the video imagery from this workshop was quite aesthetically pleasing.

236 Although assessing his experience was more challenging due to the additional factors involved in his experience of the workshop as conducted with an able-bodied partner, and although we had a follow-up discussion on Skype, it remains uncertain whether the workshop was totally effective and productive for this participant. To pursue this subject further, however, would have taken the study beyond its core focus. In future, a follow-on study might usefully focus specifically on use of the MINDtouch system by people with severe disabilities and/or communication needs.

209

Images 68 and 69 2007 C. Baker - stills from participants videos from Vancouver workshop August 2007

The fourth workshop was held once again at the Beckett Theatre in Trinity College Dublin in October (26/10/07). Like the previous workshops, these involved a mix of new participants from various backgrounds who attended out of interest. Some participants were quite imaginative and had unique approaches to the activities many looking for patterns from the structural environment, away from the group at times. Participants had access to several spaces, so the individual exercises were done in one studio and the pair exercises in a large black box performance space and outside, as it was an unseasonably warm autumn day.

Images 70, 71 and 72 2007 C. Baker - stills from participants videos from Dublin workshop October 2007

The images from this group stood out as very aesthetically stunning and exceptionally creative in the way in which the participants used the cameras special effects to invert the image, play with extreme close-ups and colours, as well as to further abstract the body parts and other object being recorded. The workshop process was quicker and many of the activities were improvised. As a result, a new activity was introduced involving partners sitting back to back, trying to telepathically send an image or sensation they had to one another, they then attempted to visually capture what they sent. Each would then guess what the other person had sent, and then they would share their videos with each other to see if all three versions (the telepathic, the videophone and the guess) were the same. This activity appeared to be enjoyed by most as it garnered more captivating visual explorations and creativity from participants, and so it was used again in the follow-up workshops. The surroundings became a large actor in the content that participants created expressively in their explorations within even though psycho-geography or locative media approaches were deliberately not extensively explored for this project. This was possibly both due to how comfortable participants were,

210

and what visual material they could work with in the vicinity to externalise and represent their inner explorations. This became a more significant factor than intended and has implications on how one can externalise with the videophone and be influenced by the surroundings. This is because it is difficult to replicate our inner eyes view, especially if the immediate environment has contrasting or unique visual fodder unlike our own internal experience and imagination. Using the available surroundings may have felt limiting and uncontrollable for participants, as they sometimes found it difficult to make the translation, yet it proved also to be a stimulus for intriguing visual explorations and experiments, as were the case with many of the video expressions. The process of conducting the workshops was very fruitful, and clarified the aims of the design/guiding process through observation, exploration and iteration. After studying the responses from participants and their interpretation of the activities and video expression/collection process, I was also able to better iterate subsequent workshops. My ability to guide participants effectively developed greatly, meaning that I could enable participants to have meaningful experiences while creating intriguing personal media expressions. The workshops became an end in themselves, and several groups asked for further practical engagement with these personal media empowerment tools (often for less technologically literate groups and individuals), in the form of future workshops. It appears that because mobile media is still a relatively new format, this hands-on engagement, with a human touch and an intimate form of communication, affected participants emotionally, intellectually and creatively. Thus, this stage of the project garnered unexpected results and served more than just to create clips for the visual database.

A.1.2b/ performance discoveries, results + analysis

The researcher discussing the work in relation to: lived experience, other works; application of results obtained; contribution to discourse; new / transgressive possibilities; obstacles encountered and the remaining problems to be addressed in future research. (Barrett, 2007:139 - From CHART: "Foucault's Author Function: Application in Practice as Research)

This section discusses the data collected from participants involved in a limited number of scratch performance events. The analysis here is partly ethnographic and used both qualitative and qualitative, with observational analysis crossing ethnography and heterophenomenology. It is based on Kozels definition mentioned in Chapter 2. There were 5 scratch events in total: 1. July 19th 2009 in the Matrix East Studios at UEL, with peers during SMARTlab seminar (20+ participants but only a 2/3rds filled in a survey - several left early - one documenter); 2. October 5th, 2009 Leeds Trinity College, in Leeds, with undergraduate students at with a student group as an Open Day activity (4 participants, and one semi-documenter);

211

3. December 4th, 2009 4M with peers and guests during SMARTlab Christmas party (4 participants and one documenter); 4. December 18th, 2009 Christmas Student Exhibition with a group of students at the Limkokwing University, London, during their student mobile project exhibition and Christmas party. (3 participants, no documenters); 5. February 13th, 2010 Valentines Apartment Dinner Party at a friend's dinner party with new acquaintances and unknown participants (to me) (6 participants, no documenters). The performance events were mostly conducted with student groups, peers or acquaintances, since these groups were most easily accessible. Each time, events or parties were compatible to combine with or piggyback the MINDtouch activities upon. These groups were most open to participating or allowing me to try out performative activities with them. However they may not have been necessarily representative of the general public, which I would have ultimately preferred to work with. If there were more time to conduct more of these events, there would have been a wider sample.

A.1.2c/ demographics There was a good split of gender overall, however there seemed to be more men in the performance events compared to the workshops, where there were more women. Workshops drew participants through email, while the performance events were part of other previously organised events. Therefore it could have been that women were more likely to voluntarily get involved or be drawn to these personal, creative activities more than men, but this is pure speculation. It would take further study to be sure of the motivations for involvement for each gender, but this was not considered relevant, since all participation was voluntary overall anyway. Similarly, there was a good range of ages, although it would be preferable in future to have more youth involved. Thus, in future versions of the project, effort will be made to seek out a wider age range of participants and more varied groups with which to work. The education level of participants was high and represents the fact that many of the participants were either PhD student peers or friends and acquaintances with similar backgrounds or education to my own. Again, in future it would be preferable to explore communities with wider educational range. Most performance participants were white British, some were mixed ethnicities raised in Britain (which I have counted as British), with a few other foreign citizens in the mix. The occupations were no surprise, since the participants from the first performance were all PhD student peers. Therefore, it seemed not unusual that so many were also lecturers and academics, with some participants also being the students of these peers. One concern was that this project was intended for non-trained performers (though not excluding trained performers either) and the general

212

public. Thus, with so many artists and performers in the group (esp. the first performance), results might be skewed and perhaps those participants had expectations much higher for such work. An alternative reading is that they might be more openminded or empathetic to participate in such work than other public groups, since it is close to their own practices. That being said, many untrained, nonartists participated in the last event, including a broader mix of backgrounds, occupations and training, as well as ethnicities, and it was thus more representative of the target audience/participant group. There was a range in academic backgrounds of the participants in the events overall, especially from the events after July 2009. However, due to the methods used to draw people to the events (unlike the workshops which were more random and self-selected), there was still a bias toward disciplines of study related to my own, such as design, fashion, digital media, film and TV, drama and dance,. Curiously, many had studied different disciplines than those related to their current job, showing that people's paths can sometimes take them far away from where they started. Since all but one of the events were conducted in London, it is not surprising that most participants came from or lived in London, unlike the workshops that were conducted in three different cities, which perhaps provided more differences. There also was wide range of incomes represented, since many participants were students and/or underemployed artists. However, half of those who completed the questionnaire chose not to disclose their income, so it is hard to entirely assess this range. Most of those who chose not to answer were either from my peer group or of the twenty-something, undergraduate students, who may not actually have had an income, but instead were possibly supported through loans, bursaries/grants, scholarships/studentships and/or family or other means. These events were definitely more of a prototype nature for a more broad mix in future.

A.1.2d / event survey feedback Overall the survey feedback was very positive. Most participants stated that they enjoy being involved in the participatory activities and several stated they most enjoyed the pre-activity, mind-quieting at the beginning. Curiously, some of the things some really liked or thought worked best were the very activities or aspects that others disliked or though needed improvement. In some cases, this was due to the fact that they were in the different events and locations. For example, when I had people assisting participants with the activities as guides, participants found that this guiding aspect helped them and was done well. While at other events when I was without the guides, participants felt the event was lacking because the activities needed more explanation. But this difference and this specific example was not a one-to-one correlation. Sometimes these contradictions in participant experience took place during the same event and can only be attributed to the differences in the participants themselves. The overall themes or patterns from the answers people provided can be seen in the appendices.

213

Most participants reported they had enjoyed being creative, but a few did not, especially during the main video activity (which I thought curious). It was fascinating that many enjoyed watching others but they did not like being observed. This was more significant in the first event where there was an observer group due to the large size of the group and fewer participants moving and movement activities in subsequent events. Most stated they enjoyed participating but not being observed, so this activity was cut in the other events. Most also felt that what worked best was the staging, organisation and facilitation of the event, particularly in the two events where I was able to find assistance in guiding the participants. Overall people stated that they thought the activities, especially the mind-quieting and video activities, worked well for them. Some participants felt the activities needed more explanation. This was a fair point as these comments came from were those events where I was managing it all on my own, including the technology set-up and other staging aspects, so my attention was split in running the event without guides to assist. Many participants from all events, stated that they wanted to see the project results more directly or immediately. I interpreted this to mean that they wanted to see the online group and the software receiving their body data trigger the videos, rather than just the outcome collage, which all could see. But most also stated that they felt this project was more interactive and participatory than most performances they had attended, with many not even regarding it as a performance at all. Unfortunately, due to the limited number of phones available, and most participants not having compatible handsets to use the biofeedback data visualisation software, the body data could not always be shown separately from the mix on the phones, since the video participants were usually using all the research phones. On one occasion the mobile software suddenly stopped working and had to be shown on computer instead. Some participants had trouble comprehending that their body data was influencing the visuals, which were the output or results; even when shown the software. Many only wanted to see their own data as they moved, having trouble grasping the relationship between the visualisation of the data itself, their bodies and the media collage displayed in the space mixed by it. However, it was not the intent or interest to show just the biofeedback data alone, since medical software and hardware is easily available to do thatthe project is not about just visualising physiological data. The project is more abstract, but this aspect likely needed more explanation and emphasis, so that the live results in front of participants, made more sense to them. It seemed that participants also wanted more immediate feedback on their videos, to see them in the collage more quickly in real-time, rather than have their videos in the queue or in the archive; this is something that was intended, but due to network set-backs and delays, was not possible at this time. Most participants misunderstood the following question: Did you feel the same sense of presence or interaction between performer audience in this event as you might in a conventional performance event? And How? Since the networked aspect was not fully connected or explicit to participants, it seems that people did not see the performance activities as performance or fully register that there were remote participants or audience. They answered in such as way that they experienced themselves and others in the space, involved in the activities in a personal, intimate and individual

214

sense. However, more stated that they were comfortable with this type of performance activity, although a few were unsure. The optional experience inventory received answers of a more individual nature, unique to participants about their experiences, although many either left this section blank or did not elaborate in great detail, in contrast to those in the workshops (see appendices for details). When asked: How did you perceive your bodys sensations, perceptions, thoughts and other internal experiences during the activities tonight? some answers were: 1) about the bodypulse, spiky crampy, lungs, head, heated, tired, ill health; or 2) about mood or mentalcalm, focused, racing thoughts, abstract, positive, heightened awareness. Most were fine with or enjoyed expressing themselves and sharing their sensations through the activities and similarly enjoyed participating in the activities using the mobile or wearable devices.

A.1.2e/ event analysis: production report

1) July 19th, 2009 with peers during the SMARTlab seminar:

This first performance was held in a 'black box' performance space on the university campus, in a state-of-the-art performance video studio [237]. However, I wanted the first performance experiment to be in a controlled space. I had a somewhat captive group with my PhD peers, who were attending our thrice-yearly seminar, with several of us showing our work. It seemed an ideal group to receive feedback from and a safe group to start the process with. I worked with the production technical staff and performance studio technician, receiving helpful support, accordingly I set up the equipment and tested it in advance. There were more than ample numbers of screens: one for the body data, and two mirroring screens for the output mixed collage of the mobile media. After consulting with the campus IT department and testing it all on campus only, it seemed that all of the networked side was working properly and I was given my own IP address for the server. However, on the day testing, it turned out that we could not get the network to connect, and just two hours before the event the IT technicians told me that, actually, they would not allow streaming off-campus. Even more frustratingly, they had also not managed to properly set up the performance space for the Internet as originally intended. This was very disheartening, since we had had several meetings and email exchanges on the project network requirements, and it was acknowledged that the performance event involved streaming video. However, it was later claimed that they did not know that it was necessary to stream videos on and OFF campus the, which would create a security hole that they could not allow. However, the show had to go on, even though there was no Internet connection, and video streaming
237 Contrary to the overall intention of these events, which was to be outside of conventional performance spaces.

215

to network was not possible that day. After all the preparation and testing, we could only stream from qik on the phones in the space and work with mixed video from the archive. Ultimately this is what took place. Another issue arose because even though the group had been asked to bring their mobile phones and to download the mobile software onto their phones in advance, most either did not have a compatible phone, or had not had time to prepare in advance [238]. Therefore, the only live streaming was done from the one phone available for the project (the other two were working with the biosensors), and one other that one participant with a compatible phone used. Due to the poor mobile reception on campus, the video streams by participants did not upload properly to the qik website and were only stored on the phone memory itself. Instead, later video recordings from the phones had to be collected and used after the event to contribute to the archive for future events. Additionally, only one phone was able to show the body data, and another phone was given to participants just to watch their body signals while they moved, the second set of sensors sent data to another screen. Understandably, all the participant feedback on the experience of presence was more about the experience of others in the space and the blurred lines between performing and being an audience, rather than what I was looking for (there was some misinterpretation of the question; see more on this further on). This again was due to the network circumstances, rather than the sensing the remote others not present in the room through network interaction, as was intended. Another distinctive facet of this first event was that second garment design, prototype 2, was not ready for the event, no matter how hard the designers worked to do so. Since there was only one garment and phone pairing with physiological data, the other paired set involved having the exposed sensors hanging off participants and shown on the larger screens [239]. Another unique feature of this event was that there was having the three guides or assistants to manage a large group of participants: 1) one guide working with two sensor garment-wearing participants, helping them to move actively enough to trigger body data changes; 2) one guide working with the video participants, and 3) a third guide with a placebo group (another unique feature used as a base or control group). This last guide also helped prepare the participants to pay attention to their bodies for the video activities and get used to moving for the sensors. This third guide also gave participants additional things to do while the other two groups were busy. Since there were many more participants than anticipated, I created a fourth group to observe the other groups in action and to note down (in a stream-of-consciousness way) their experiences of the overall event. I improvised with these last two groups, especially the last one, with the wish to include all who showed up, to keep it social and participatory. This also illuminated what to do in future if a similar situation arose.
238 Which understandably IS a pain for them to do and it would be preferable if all phones had some similar multimedia software and operating system on them so this was not an issue, such as QuickTime (which has editing capabilities as well), but it is also not a priority for some people and an extra cost. 239 As mentioned in the last chapter, each set of sensors needs to be paired with only one phone or computer, due to the limits of the BlueTooth one-to-one pairing requirements.

216

As a result of the network limitations, the collaborative video mix was partially a fake of the interaction based on the archive of clips from the earlier workshops then mixed and projected in real-time as the visual source. The body data from the sensors was live. Therefore, all the working technology was used, but there was no external, networked interaction. What also emerged from this was the discovery that there were some glitches with the mixing software, which had not been evident previously but would need to be rectified before the next event. In addition, a qik-to-server 'patch' needed to be made to bridge the gap. This faking it is common in interactive and new media projects, so I was prepared for its possibility. There is still always a great deal to be learned from such an event and that was learned in this iterative type of work.
Event Summary: Liveness (real-time) = people in the space, live body data, live mixing of archived video and one live streaming qik video camera (not received by qik) Presence (of others live and remote) = people in the space only ____________________________ 2) October 5th, 2009 with undergraduate students at Trinity College, Leeds; a peer's student group as an Open Day activity.

From August to October a few events that were lined up fell through, in addition to other constraints taking up time, much to my disappointment. When I was invited to go to Leeds to hold an event, I jumped at the opportunity. However, this turned into an unsuccessful event, but since all of them were 'scratch'/iterative/ work-in-progress, performances, there was an understanding that each would be a learning opportunity and each would build on each other. The 5th of October was part of an Open Day and my host had just recruited some of her students to help her. She wanted to give them something fun to do as a break in their day. Unfortunately, again, the IT department of this college did not understand the projects Internet needs. Even though they were provided with a detailed Tech Rider (technical specification), they did not seem to realise that streaming on and off campus was necessary, and so they only provided Ethernet cables to access the Internet, but not the ability to stream data from the sensors over the Internet to the server. On site, the mobile service reception was very weak and it took up to thirty minutes to upstream to the qik site. Again the event was NOT real-time or live and it was problematic to mix the videos out to the web. Thus, there was only the live (in-person) group, who participated in two aspects of the workshop; as with the previous workshops, this resulted in only about three to four participants making the video. At this event, none of the participants felt comfortable wearing the garment. The Internet connection issue would have prevented this data being sent over the Internet in order to trigger the mixing on the server anyway. As a result, the remote audience who were trying to view the live stream of real-time

217

mixed video saw only a live mix of the archived workshop video and in 'demo-mode' to fake receiving the body data over the network. Another unique aspect of this event was that, there was a remote helper at the location where the server was hosted, which was not necessary at the first event. He made sure that the stream was working for the remote audience while I was in Leeds. We kept in touch through mobile phone, although the mobile reception was so poor that this was difficult. Another constraint was that his help was only available for a thirty-minute duration, so the online aspect had to be short. During the inperson aspect I had two potential helpers, but with so few participants they were not really needed and one had other responsibilities hosting the open day, so the other became a participant.
Event Summary: Live (real-time) = people in the space making recorded video, remote audience watching live video mix from archived footage online and waiting for live qik video (that took 30 minutes), and one live streaming qik video camera. Presence (of others live and remote) = people in the space, people remote - but not connected or able to interact ____________________________ 3) December 4th, 2009 with peers and guests during the SMARTlab Christmas party (4M Studios, London)

This event was planned in advance as a celebration that concluded the performance series informing the thesis. In most respects, the event was highly successful [240]. It was also more controllable since the streaming server and the laptop receiving and sending body were both in the same space and were accessible to me for constant monitoring. What was distinctive about this event was that it was actually in a party context, as intended, and so unlike the other events it was not quiet or contrived; it was more spontaneous, unselfconscious and people were there to be social. Some attendees had no idea that others were participating and were curious when they saw others running about with cameras and doing 'odd' things in the space. There was one LCD monitor displaying the live mixes of video from the body data in a prominent place, and many assumed it was just visual 'eye candy' for the party; they did not appear to realise (or ask) that it was being mixed live. The unique attribute at this event was that both garments were available. However, the sensors of the beta garment had become damaged from wear and tear after the first event in July and were not usable, so we only used the newer version. There was one guide to help out and one participant wearing the sensor garment, and there were video participants. I conducted the meditative introduction activity and guided the video participants through the video 'assignments' and gave the sensor
240 My measure of success is all the of the technology working as intended, all of the participants having a good experience and some of them having a sense of presence of the remote participants.

218

participant some guided movement activities. This was the first party environment I had held the performance at, so participants were guided do the tasks, but could interact with the party attendees as they wished. And indeed, participants all reported that they really enjoyed it. However, there were still some glitches with the technology since the php fix script between qik and the server was not working properly on the server. It turned out there was a new conflict with the streaming ports. Again the live mix was only from the archived footage [241]. Also, none of the participants had the required phone or compatible software, so a research phone was again provided, but this meant that only one phone was streaming through qik. The participant using this phone was successful in the upstream of her videos, but most of the streams did not arrive to the qik and MINDtouch page until the next day. As this aspect was out of my hands it was a little annoying, and as a result there was no live video available for the server to stream. There must be something that slows down the qik servers: perhaps too many users streaming at once causes a bottleneck. This would validate the need for custom software to stream directly to research server, which there was no time or additional resources to develop.

Event Summary: Live (real-time) = people in the space making recorded video, live video mix in the space triggered from live body data, remote audience watching live video mix from archived footage online and waiting for live qik video (that never came), participants contributing video on qik remotely, and one live streaming qik video camera... Presence (of others live and remote) = people in the space, people remote - again only one way participation for online and in the space - still no interaction ____________________________

4) December 18th, 2009 with a group of students at the Limkokwing University, London, organised by a collaborator on other projects, during their student mobile project exhibition and Christmas party (3 participants, no documenters). This event was similar to the Leeds situation in that, despite best efforts of my host and myself [242], the IT department did not set up the Internet connection for the event in a way that was useful and then went away for holidays just before the event. Therefore, the Internet connection again did not work. Student participants were supposed to be organised in advance; however there were none available when I arrived; instead they were participating in other activities at the event. At first it was quite disappointing to be invited to show the project, then given little support from the host organisation. But again, the show must go on, and as such, an LCD screen was eventually organised and set up for my laptop to mix the video collage in 'demo-mode' as background visuals for the event. Once that was set up, students became more interested and three were recruited as participants: two making mobile video and one wearing the garment with the sensors. However, the students making video confessed
241 Although I had been adding footage into the archive manually from each event for the one to follow. 242 See the tech rider/ specification I provided them in Appendix II.

219

that they were unclear how to do the activity and focus with all the other festivities going on to distract them. Therefore, only a couple of videos were recorded and the participants could only engage with the activity for twenty minutes. It was evident that they wanted less to participate and more to be given a demonstration on the technology behind the project, and then get back to the party. The participant in the garment was more engaged and enthusiastic about it, putting in more effort to make her body trigger the video output and see the data on the phone interface. However, the mobile software began to crash the phone and not start up properly. Meanwhile, the computer version of body day software also was not receiving the data properly. As a result, this event was fraught with communication and technological issues. So without technical assistance or participant guides, I was not able to split my attention to stop engaging with the participants to fix or at least jerry-rig the technology. Since the students attention spans were short, it was a priority to keep them engaged as long as possible. Event Summary: Live (real-time) = people in the space, recording video, live video mixing in the space in 'demo-mode', the remote audience waiting for anything, participants contributing video on qik remotely via one live streaming video camera...
Presence (of others live and remote) = people in the space - only one way participation for those in the space ____________________________

5) February 13th, 2010 at a friend's dinner party for Valentine's, with new acquaintances and unknown participants (to me) (6 participants, no documenters) In an effort to collect more and varied data before having to close the research (due to time and financial constraints), and while the server was still available, I asked a friend if I could bring all the technology to his large Valentines Day dinner party, to see how the performance activities would fare in a more spontaneous environment. This was part of the effort to try different contexts for the event, and it was thought to be best suited for parties, concerts and festivals, so this was another opportunity to try that idea. However, one realisation surfaced in hindsight, which was that some help was needed even in this context. A remote operator with the server would have been useful, since the streaming still required manual operation until the automated or remote control functionality requiring networked operation was working smoothly. This was a fresh group, all whom were unfamiliar to me, and all from more varied backgrounds, experience and ethnicities than previously worked with, which could make a difference in results or feedback. Before the dinner party was fully under way, willing participants were recruited. One woman wore the garment and was an ideal candidate, with very little experience with technology or art/performance practices, and with a career in business consulting. The other participants included a cook, an osteopath, psychologist, and an IT specialist,. All were of different ethnicities: an Egyptian American, one Israeli French, a Cuban, a German. It was a great mix and none of them had done anything like this. However, one drawback was that it was more difficult to convey the project concepts

220

or explain the activities quickly (important before all other party-goers arrived), and to get participants engaged. The difference was that for example, during the first event, which had more artists, performers and digital media specialists (peers), it was easier to explain the project since they understood the language and domain of the performing arts and digital media. This difference was good practice for me however, and as with the event prior to this one, it made me realise that the way it was explained was too complex for the public. More work needed to be done to simply the explanation in order to really engage participants in future versions of this project. This underlines the value of guiding. There was only one attendee who had a compatible phone (the IT specialist), thus participants had to use the research phones again, and so there were none remaining for the sensor participant to see her own data, with only the laptop to show the live mixing as visual feedback of her body data. This was a bit too abstract for this participant and she found it hard to conceptually connect the idea that her body data was triggering the visuals and just wanted to see her body data alone. This was a side result that I had to contend with, and while I had intended to show people more direct visual representations of their body data to help them understand what was happening, it was not the goal of the project, so it was sacrificed for the other elements to operate effectively. Instead effort was made to help participants to see that their bodies were acting as a VJ or live cinema performer, and this was eventually understood. Again, without an assistant it was difficult to control the remote streaming, which was having some difficulties. However, the remote participants left me messages on the qik site, which in turn, were sent to my phone as text messages before we got started in the in-person space. This was an unexpected, but excellent and welcome addition to the live interaction, even though only I was aware of these messages within the local space. The remote participants were asking when we would get started, since they were waiting online and they were making comments on the videos from a previous event that were upstreamed and archived on qik. Had there been a moderator or online guide to work with the remote group, or a remote server operator who also moderated the remote group, it would have been easier to facilitate more interaction between each group. Again, the video from qik did not upload properly in real-time, taking over an hour (improved from the last time, but certainly not live), and the 'video pull' php script was not properly working to transfer the video from qik to the streaming server archive again either.

Event Summary:

Live (real-time) = people in the space making recorded video, live video mix in the space triggered from live body data, remote audience waiting for live qik video (that never came) and the mix did not work either, live participants contributing video on qik remotely with two live streaming qik video cameras, two remote participants making and contributing (new). Presence (of others live and remote) = people in the space, people remote watching and making video - again only one way participation for online and in the space - only networked interaction was

221

between me and the remote audience / participants via qik / mobile messaging.

post performance observations

These notes came after the December 4th event but seemed applicable to all of them as an overall assessment of the organisation of each event, discovered through iterations of them in practice: 1) Each was a manageable size when there was so much going on around at the party already; 2) Participants said they loved the activity of recording and streaming videoa private activity that is only theirs within the social environmentand reported that it felt like it was just for them; 3) It was not clear if the participant who was streaming the video via qik completely understood that she was connected to others online remotely (even though it was explained); 4) It became very clear that guiding is essential in this type of participatory event, but that I cannot conduct these events entirely alone, since one person must put the garment on the participant while someone needs to attend to the others, the technology, server and remote group, and each group needs guiding. Guiding needs time: time to orient participants to the experience they are diving into and explain what is happening overall (without dictating or influencing their experience). The guide's role is to orient the participants, to reassure them, to give tips or thoughts on how they can focus on the activity to illicit the video recording and how to can translate what they are feeling outwardly, into a visual manifestation. Guiding is more difficult with someone in the garment, because they have no direct feedback to watch their data, other than the visualisation waveforms from the custom software on the phone and the resulting video mixes. It was intended that there be a phone to do this each time, but we ran out of phones almost at every event due to people not bringing compatible models and not having enough research phones. It would have helped people wearing the garment, not otherwise able to make the mental connection to their bodies, what was happening with the sensors without this aid; 5) The online group needs more guiding in advance in future events. A message will be recorded for them, with more instructions to prepare them before the next one, so that they can see what is going on online more easily. A documenter who is streaming live from the event is also necessary for the remote group as well (intended this time but the documenters phone battery expired part way through). This should be on a different page on the qik site or custom software be made so that the files do not get mixed up with the files going to the server; 6) The visual display worked well and people at the party who were not directly involved in the activity enjoyed the display, especially when it was explained to them. Some became excited by what was taking place around them with video makers. It was as if there were invisible VJ's in the room which made it seem more compelling;

222

7) The documenters role is key to the documentation and they would be more thoroughly briefed and monitored to interview others not part of the activities for their reaction or impressions of on the event during and after. This would assist in discerning the external, incidental audience response to the work. 8) One major concern was the time needed to dress the person in the garments, and the delicacy of the sensors [243]. Effort had to be made to try not to further damage them. The way the beta garment was made it fragile and easily damaged (since the breath sensor is used to tighten it). However, discoveries like these were part of the research process. In future, more durable and flexible wearable electronics would be desired. It would need to be easier to put on and less cumbersome for the wearer: hiding the electronics is not the goal, rather working with conductive threads and fabrics in an easy to work with manner is (for another research project). Another major challenge was all of the network and security issues and passwords; complexity of the connectivity.

Guides and Assistants Necessary for Events include: A guide for the videophone group A guide for the person wearing the garment A documenter A guide for the online group is needed add info to my site and to qik / streaming

In future, my role would be to become the monitor of the whole event, more like a film or theatre director, moveing around the space to explain the activities and event to those not involved, as well as to explain how the visuals are created, especially if participants are not able to access the video directly on their phones (as intended).

A.1.2f/ participant response analysis After reviewing the comments from all participants, conflicting comments arose from the same aspects of the event. One example was when undertaking the 'telepathic' exercise, when many reported that they really enjoyed (as did those in the workshops), and yet two really did not like the idea (though did not say they did not like the activity itself). There were other conflicting responses to the video exercises from a couple participants in the moving activities during the first event [244]. As such, different activities worked for different people and not others, an expected variation. However, based on the
243 The alpha garments sensors were damaged and returned to the electronics collaborator Michael Markert to be fixed, so only one could be used for the performance. 244 See the appendices for the specific responses in conflict here to contextualise more.

223

written responses most participants enjoyed the events overall, yet shyness or self-consciousness could have interfered with the enjoyment for a few. The guiding process worked very well in the first and third performances because I had more assistance to manage all the groups and the technology. By contrast, in the second, forth and fifth performances, the participants reported that the activities and the project needed more explanation. This outcome is understandable, since recruiting assistants was difficult, so I was left managing too many things on my own, leaving participants on their own more. Overall, the network connection and interaction was more abstract or obscure in the minds of participants [245] as mentioned earlier. However, it would not have been technically possible to have displayed both the live video mixing from the laptop and while showing the activities of the online remote participants, simultaneously during the smaller events without additional computers. Nor would I wish participants to split their attention to do both, because they then would not be as fully engaged in either activity. Therefore, this aspect involving the engagement with others remotely over the network, evidently was not clear or felt by most, if at all unfortunately. In addition, most participants misinterpreted the question in the survey on presence, which in hindsight may have been obscurely written, asking them if they felt [] the same sense of presence or interaction between performer audience in this event as you might in a conventional performance event [][246]. There could be two reasons why this was misinterpreted, in addition to the wording, they are: 1) interactions between the live and remote audiences were not obvious (in the first event there was no remote audience, as discussed above), therefore, the only sense of presence felt would have been of each other in the physical space, and within the separate groups performing the activities; 2) the participants did not see themselves as performers (which was what I tried to foster in order to breakdown inhibitions, also mentioned earlier in Chapter 3), and the guides helped participants to feel comfortable and less intimidated. This was also the case whenever I was the only guide. During those events, participants expressed that they thought of presence in terms of live presence in the space and of those in the room physically, live in-person. Participants who knew about the networked aspect [247] or understood what was supposed to happen, reported that they did not feel the sense presence of the external interactors/audience and felt disconnected from them. However this would make sense, since those responses came from the first event where there was no network connection. Only in the last event, when text messages were sent to my personal phone from the remote audience, did the live audience have an awareness of the remote interactors, and only because I told them. Even then, due to the delays in qik's streaming service and due to the lack of additional equipment or support, participants were not able to see these remote messages themselves and we were not able to set up a second computer and connection on site (and it was inappropriate at the time) in the party context to show them.
245 It was 90% working this time, but still one piece of the exchange/ interaction loop was not connecting my programmer failed to get php qik script to work on the server even though it worked on all other machines we work working on and we did not have a network specialist with us to determine how to solve the port conflict where the server was housed. So this examining piece of the puzzle was never solved however I will change servers for upcoming events and hope to solve this part of the problem or make a custom tool to stream from the phone to my own server. 246 The primary question dealing with this sensation during the event. All the questions can be found in Appendix II. 247 This was explained to everyone, yet many may have forgot, or as seen in the responses, did not understand it.

224

Many intriguing discoveries were made when reading through the performance and workshop responses (and drawings), even though there were inconsistent network connections and there was never a complete connection loop, completely linked. So unfortunately, the full interaction was not possible to experience presence during a mobile, networked performance or to study. Yet one great discovery was that people really engaged with the technology in solo and pair activities, and in making very intimate and personal expressions of their inner experience. They did so within the social environments and contexts, and many found the experience very relaxing, some said 'therapeutic'. Like the workshops, the unexpected outcome was the willingness of participants to engage in something quite different for them, especially using mobile phones, and involving a fairly personal exploration within a social context. They did express a wish to be apart of a larger artistic endeavour. Still, if more of an interaction with the external/remote audiences was more obvious (and fully operational), there would likely be a whole new level of interaction between participants, which had been the aim. As it was, a fun yet personal, creative experience was enabled without the network, which still provided a participatory, expressive, interaction using mobile phones and wearable devices. Therefore, a majority of the aims were achieved. The social aspect was there, the engagement and a sense of interactivity were there, and a sense of liveness and real-time was definitely experienced (as noted after each event above). Missing was the live/real-time and collaborative networked interaction. Although one could argue that the project was still a collaboration between myself and the participants, as well as the participants with each other, since all visual material was sourced from previous workshops and events and was used in the live context. However, there remain missing elements I was aiming for in terms of closing the technological and experiential loop: real-time, networked presence to be felt and experienced. Overall, participants reported a definite experience of some sense of presence and embodiment within and through the mobile device, as well as the media artefacts they createdthis succeeded very well. Participants gave themselves completely over to the experience, almost like a therapy, which seemed enjoyable for most. It is not clear what influence I, as the researcher/ facilitator, had on them, since some of them were my peers and friends of friends. It is possible (and desirable) that my facilitation and that of the guides fostered a sense of emotional and participatory investment, trust or eagerness to engage. Since I am unable to assess this, one might assume that I had some effect on the experience, not only in the activities but also in the delivery, reception and experience for participants. This also affirms the aims to improve my facilitation skills. The differences that emerged between these five events became essential to the 'iterative' nature of the research. While each event featured many similar activities and processes, successive events were changed based on the circumstances of the host environment/context and the learning from each previous event. For example, the 'placebo' movement activities of the first event were dropped in future events, based on feedback and observations. In addition, the participant 'observer' role was also dropped, but this role did garner interesting results. For example, many participants were instructed to take notes of anything that came to their minds while watching the others. This resulted in many

225

expressing themselves through free-association. This appeared liberating for some, as it allowed them to use the familiar and comfortable modality of words and language, not to be used in the other activities. By contrast, a few participants did not like moving or being observed. Unfortunately, participants did not end up moving much during the physiological sensing activity, which the movement group was meant to prepare them for, without guidance and on-going encouragement. In addition, their movement did not result in any grand or energetic gestures that would register any significant fluctuations in the representative data visualisation. This was because most participants merely paced around the space and occasionally waved their arms around. As such, the sensor software had to be recalibrated to quite a low threshold to register any differences or changes to trigger the visual media mixing software and resulting visual output [248]. During the December 4th event, the lack of movement was very unfortunate and the mixing software had to be changed to demo-mode because the participant was not moving enough, and did not respond to guidance to do so to change the visuals (she ignored it or felt shy and inhibited with her workmates there). While it was intriguing to people conceptually that they could watch their body responses and signals, seeming to fascinate them that their bodies were triggering the visual effects and mixes, it became less engaging and too abstract for participants later in the event. Yet other participants reported that they would have liked to have had more information on their body data. The video activities and the idea that their own videos would show up in the mix at some point, as well as the intimate and personal experience of expressing their bodily and emotional sensations, primarily seemed to excite people and casued them to state that they had had a good experience. Not to mention that the relaxation element facilitated participants to connect with their own bodies, the activities, and to help them feel less self-conscious. In summary, as a personal, creative perception activity, the video expression exercises were quite successful and reportedly enjoyed by most participants, even though the networked interaction was missing (many did not notice). Yet even if the network connection completely functioned as expected, perhaps if participants in the social, live context would still be more absorbed in the video activity and personal expression aspect and more into their own creations than they would be in the interaction and collaborative creation aspects with an abstracted, remote audience. In addition, even if all had worked fully, the remote participants appeared to have been a voyeuristic. Only one online participant completed the online survey and only two contributed their own video to the database. Still others reported to have only watched the live streamed video, feeling unsure if was the totality of the experience/event (even though they were briefed in advance), yet they did not contribute. Perhaps having this aspect, with a more 'live' moderator, with a means to 'log in' [249] and see the live group in action (through live streaming documenter), could assist in motivating the remote group to participate more actively. In addition, having a more simply stated set of instructions and guidance might help
248 Originally there had been an intention to involve Tai Chi practitioners to perform their movements, though this did not end up being included, it might have resulted in more varied visual collages. 249 This was intended and an attempt to implement was started, but then it became more complex with time constraints to be completed and implemented online.

226

(although effort was made to make it easy). It could be inferred that a simpler or more guided facilitation, with a more engaging and simple online and mobile interface, which tracks users and allows for some moderation and motivation could improve interaction. From the overall feedback received when there were guides to assist, the experience was better for participants. In addition, during the three events where I was unaided, the participants felt that they needed more assistance to do the activities; also the issue with the remote group. Therefore, it could be said that these "scratch"/iterative performances were like user testing events in preparation for a more successful MINDtouch 2.0 series of participatory performances.

A.1.2g/ overall performance assessment Below are some overall observations from the staging of these initial iterative events that can be taken into future versions of this project. performance contexts From this research, using the various contexts for the events, it can be seen that each have provided a different motivating element: when events were organised in a contrived and purpose-specific way similar to the workshops, it was difficult to recruit many participants. Based on people spoken to in the recruitment process, many were a bit inhibited by the sound of the activities: fear of not being creative, fear of technology, fear of looking foolish, fear of being in the video [250]. When, by contrast, in a more controlled group like a class (still contrived), these fears may still have been there, but there was more complicity. In the party context or small group, people were more willing to try something new and even if they do not know each other, they become more open and social. In this party setting, people were coming to relax and socialise, but not expecting to do anything more challenging that drink, eat and converse, maybe dance, possibly sing, even play games. Either way, in these two contexts, the participants were more actively seeking new and challenging experiences. However, both times when caught unawares, there were people still ready to try something a bit foreign for the sake of potential fun. Of those I worked with, the younger participants were more cautious (to my surprise), more self-conscious (more expected) and less open-minded to the experience than the older counterparts.

body data Not only because it was part of the BBC funded work, but the physiological sensor use was added as a mechanism to change the video and add more 'embodied' effects and attributes to the presence experience, especially in the live context. But it was discovered that the sensor-embedded garments
250 Although that was not the point or even necessary, but people had to be reassured of this.

227

were also effective in engaging people with the concept of visual body mixing and the interactive aspect of the work. However, they quickly lost interest in their body data when they could not see it more concretely. One would assume that participants would feel more engaged with the work if all they needed to do was move and 'be', although that was not witnessed here. When guiding participants through the participatory activities it was important to explain what the sensors represented in the body and how the data was used in the performance. Some people were asked to wear the garments with the embedded devices measuring breath, temperature, stress, muscle activity, etc., so naturally they wanted to understand the data more. These participants were asked to move and express their sensations and perceptions, and were guided through the activities with their bodies, embodying the devices through their movement. While the other videophone participants did the same through the video imagery. Participants did wish to know more about the sensing aspect more directly and specifically, and which body activities were effecting the video/visuals, not a feature built into the current project. This was not something the programmer was asked do, and it would have added even more complexity to the current complex programming and weave of technologies and interactions. However, the sensors can add to the experience and should continue to be implemented, and perhaps more simply. Ideally I would use prefabricated garments (or drop the wearable aspect for simplicity), giving participants more direct and obvious feedback on their body actions.

MINDtouch 2.0 What was observed during the staged events was that people have trouble literally seeing a connection between the remote interactors and what they are doing within the live setting. To redesign this piece further, efforts and method would need to be implemented to demonstrate and visualise this connection to participants. This might include an additional interface to show both sides of the interaction for each group [251]. Therefore, a feedback mechanism is needed for each group to see all of the activities, such as an online and mobile application, with a much more powerful server behind it, and the aid of (hired) operators to manage the incoming and outgoing activities, live and remotely. As such, a proposal for a new project is arising from and informed by this research. Such a project includes guides and moderators for each group: live groups and remote mobile participants (on phones). Also necessary would be an ability to tag videos captured (as qik enables but for a custom software), to better enable the videos to stream into the correct folders. This would also guide them into appropriate (to their emotions, sensations and perceptions) or participant-made categories, including comments for the online interface (with moderation). In addition, this might involve the online guide requesting that lurkers and non-participants report what they see. experience it could be compared to those in the live or mobile video groups.
251 This also was part of the original video but also had to be abandoned due to time and resource limitations.

Then how participants

228

229

A.1.3/ visual mixing software + streaming development process

It has not been revealed yet as to how decisions were made with the video-clip database or footage folders in the development process of MINDtouch. This process was iterative and in collaboration with the creative programming experts/artists on the project. Below is a desktop screen shot showing the interface of the Quartz Composer programming environment and the specific patch or application that was made by Manjit Bedi, using another earlier created application to receive the sensor data made by Michael Markert. The process by which the visual mixing software was developed using the body data from the sensors, was as follows: 1) The first decision necessary was how to define how to work with, differentiate and categorise the video content collected from the workshops. This necessitated separating the videos into a small number of folders located on the server, each corresponding to the number of sensors used in the garment, worn on the body sending the body data to the custom mobile application;

Image 73 2010 C. Baker screen shot of the Quartz Composer patch for video mixing

2) From there, a decision was made about the process by which this video mixing software application transforms the body data and adds visual effects to clips, Metaphorical associations were made related to qualities of life-force and kinesthetics of a human body as correlated to each sensor, followed by how to visualise these life-force and kinesthetic qualities. The following were the choices made (figure 7):

230

sensor breath

life-force / kinesthetic quality life-force, internal winds (yoga) + well being

action = visual effect to apply Initial: slow or fast forward or back or change speed up according inhalations or exhalation speed Final: swells / an effect to stretch and contract the image Initial: colour changes or colour intensity or contrast Final: contrast (darkens and lightens) Initial: jittery or smoothness of playback, sharpness or blurring Final: from blurring to sharpness Initial: swells of opacity / mix overlay + transparency Final: colour changes or saturation of colour Initial: drawing + animation, mobile interaction/ changing the video or adding text or image Final: (same as EKG) colour changes or saturation of colour

temperature

fire, passion or ice, fear intensity emotions hot or cold

GSR

mental / nerves/ stress/ fear or calm

EKG EMG

life-force, blood, emotions, rhythm muscles, gesture strength, movement/

Figure 7 2010 C. Baker physiological sensors mapped to life-force qualities then to visual effects

3) Next we implemented the visual effects necessary to emulate or visually communicate the corresponding life-force metaphor (see chart above) within the possibilities of the software/programming environment of Quartz Composer. I brainstormed ideas, including personal assumptions of these qualities. To meet time constraints of project event delivery dates, Manjit implemented these ideas. He then interpreted these concepts within his programmatic decision-making and creativity, within the visual possibilities of Quartz Composer (see Image 73 below, which shows these implemented into the patch). 4) The videos were selected and categorised as they corresponded to each sensor (Appendix II for this spreadsheet). Each video was reviewed and a spreadsheet made of the visual content and qualities observed within each video, and were placed into the most appropriate of the four folders. The folders were labeled based on the predominant activities conducted with the participants in each workshop. One challenging aspect of this decision process was that this categorisation was required for the archived videos from the workshops, as well as to be applicable for the live video streams during each event, where the live video would stream directly into the folders on the server. Initially, the intention was to create an additional custom mobile software application that would tag each live streamed video (qik has this functionality, but it is only available for a few smartphone models. Tagging would

231

make it easier for people to add code that would direct the incoming streams of video into the appropriate folders, in order to then be mixed according to the body data and then streamed back out to participants [252]. Future iterations of the technology system will include tagging and folder interaction for the database. For past events, participants involved tended not have streaming/qik compatible phones, therefore the research phones were the only ones used and qik software available late in the research, thus participants were not instructed to tag. Making sure the associated content gets into the corresponding folder to be mixed according to the sensors and activity is crucial, but has been tricky in practice. Hence, archived clips from workshops have been mostly used for the first performances. More facilitation with participants will ensure that tagging takes place and that the content itself matches the participant's intent for the output video the database to work corresponding to the sensors in future events. This will complete the loop of: body data> sensation/life-force quality>visual effect >montage remix of body and body sensation video expression/dialogue >abstract visual collaboration (only 90% achieved for the events staged).

Image 73 2010 C. Baker zoomed in screen shot of patch for video mixing from sensor data 252 The software was in development all the way up to the last moments of the allowed timeframe for the PhD part of the project, so we did not yet complete this part, nor managed to get qik tagging function used in performances.

232

Another issue worth mentioning is the daisychain of tools, scripts and applications (see Figure 8 below), necessary and implemented to bring the vision some form of fruition.

Figure 8 2009 C. Baker demonstrating the technology connections

Even though the concept seemed simple, direct and beautiful to me: a simulated, embodied, mobile global, collaborative live cinema performance. In reality, ended up consisting of (see Figure 9 below to clarify): 1. body to sensor device to Arduino hub; 2. Arduino sending data to mobile phone or to my laptop computer [253], with only one-toone BlueTooth connection at a time (not multiple garments and sensor performers as desired)so one or the other but not bothI chose the computer to send the data to the server and had the second garment go to a second phone to visual the data for participants; 3. data then sent from the laptop through Quartz, through the Internet to the streaming server;

253 When using Quartz Composer, the body data ended up having to be sent to computer, since the mobile phones are not yet sophisticated to deal with complex applications like Quartz (also not compatible yet with Apple software that we all chose to work with for visuals); soon this will change and be possible with the iPhone.

233

4. on the server-side, the data received is processed by the Quartz Composer and another software simultaneously (Quartz Crystal) to encode the mix into one mini-clip to be streamable; 5. then it is taken by Apples QuickTime Broadcaster, in conjunction with the Apple Streaming Server software, and streamed to the Internet and phones (ideally) with the (see figure 9 below). On the streaming server there were the folders of movie clips receiving the incoming streams (as long as no firewall prevented the streams getting to the Internet and server, on both computers). Ideally, this would be with all web ports open on both local and remote computers, with an Ethernet connection and static IP address on both ends, to omplete the loop shown in Figure 10 below. However, we then needed to use XML on the server side, in order to prevent the information from being intercepted by a firewall and allow the stream to be received by any Internet enabled computer or Real Time Streaming Protocol enabled movie player, on any computer, or mobile phone, anywhere in the world. Screen sharing other intermediary tools also had to be implemented. A sketch of the server-side processes is below in Figure 9.

Figure 9 2010 C. Baker demonstrating the server-side processing

Additionally, the participants who were live streaming video with qik from their own phones had to send their videos to my qik page, otherwise I would not have had access to them and it would have

234

been harder to access them and they would not be sent to the streaming server for the mixing. It was necessary to have participants all log into qik using my account, where all the streams were collected and stored after streaming. There was a custom script written in php to pull those videos off the qik server and then convert them from .flv (flash video files) into QuickTime .mov files and put them in the right folders for the streaming server to process for streaming.

Figure 10 2009 Evan Raskob (collaborator) sketch of networked interactions

This figure demonstrates how the technological and networked processes interacted during these performance experiments. Naturally, there was a lag created by all this processing, thus, a delay of real-time or liveness occurred in actual events and qik alone could take up to thirty minutes to process each video in some cases. Also, the software Quartz Crystal encoded and rendered the mixed video from the data sent to the Quartz Composer patch to create a streamable video clip, which could take twice as long as the desired clip to render; the longest clip was two minutes. Smaller streams were preferable to ensure quicker, more live transmission, but this meant a technician was need to monitor and enable the rendering process mix by mix on the server-side for each performance. Since this was the only solution found to create a streamable piece, it again added to the overall processing time and again was not enabling a real-time experience, and was more like making a podcast. So at first, the stream was live, with hinted (preloaded) video, and then the process begins to slow down in the process of compression, rendering, then delays and lags in the streaming took over and these delays started to lengthen the whole stream. While in-person Live Cinema or VJing IS real-time, since all the technological production elements take place all in the same space and in-person, live, with all the network complications of this type of project

235

the real-time aspect is NOT quite there, not without delay. STILL, after all the years of online teleconferencing and videoconferencing, it can be unbearably slow. Tim Berners-Lee stated in a GuardianTech podcast in January 2010 [254] that the new version of the Internet is coming and will improve such issues, so very soon mobile video-Skyping and other such mobile Internet application should be lag free. However for this project, using a multiplicity of technologies to study a technological embodied, visual collaborative dream mixing: mobile phones, sensors, etc. with telematic and networked performance using video, has been very complex, more complex than intended or expected. In retrospect, it is clear that perhaps too many layers of complex interconnection were necessary to achieve the fully live-streamed video, VJ mixing and sensor technology integration that the project set out to achieve. In this way, the original aims were over ambitious. However, the attempt to achieve these big picture aims pushed the project to reach beyond the possible and to invent and test new boundaries for live interaction using real time networks. While the goal of achieving a real-time telepathic simulation using mobile media remains a strong motivator for further research, it would necessarily extend beyond the scope of this thesis. Technical collaborators on the MINDtouch Project from the BBC and external experts all confirm (see testimonials, in Appendix II) that what the project seeks to achieve, while exciting and innovative is also too extensive and ambitious to be completed within the few years of a fixed research project or PhD. This would, if extended require much more support and technical expertise on a greater scale. All the technical consultants have confirmed the success of the MINDtouch Project in its own terms as having made a significant contribution to knowledge, to be explored further by much larger teams in major industry bodies over years to come.

A.1.4/ video content categories + analysis

After reviewing all the video clips made by participants in the workshops, the main visual themes found were: 1. close-up and extreme close-up shots focusing on but abstracting the face and body so as to not be easily recognizable; 2. movement of camera either surveying the space or more vigorous movement like shaking, and fast movement of the camera and/ or the body to create blurring and streaking (walking feet very common), and other visual patterns resulting from the movement; 3. patterns of all varieties, especially lines and interior architectural features, and using movement of the camera to accentuate these visual patterns (lines very common): such as moving elevator doors, wires, radiator coiling, spiraling staircases, etc

254 This was heard in a podcast but an archive could not be found again but the story is available online to read at: http://www.fastcompany.com/1593583/tim-berners-lee-to-head-up-institute-of-web-science-in-uk-as-britain-goes-for-broke-onthe-w and http://www.ted.com/talks/tim_berners_lee_on_the_next_web.html

236

4. natural and outdoor elements: trees, water, clouds etc and any patterns to be captured from them. Below is a chart (Figure 11) that was made to connect the various pieces of the workshops to imagery created and then to the sensors of the body and database folder for the performance collage output. I wanted every aspect of the workshop and performance activities to have correspondence and a relationship to the sensor and video manipulation in the final performance context. Therefore, since there were probably more refined categories that could be made from the visual image content, resulting in further categories and folders in the database, this would be too complex for the Quartz Composer and other software developed. The four themes focused on were those corresponding to the four main activities in the workshops (although there were actually five in the workshops. I narrowed them to four categories for performances, for the four sensors that trigger the videos, with four streams of video from four folders in the database on the server.

main action

content/

camera

related activity

corresponding sensor

folder placement

close-ups and extreme closeups of face or body

use only yourself or environment in close proximity (representing an internal partner experience); also the introduce yourself exercise with a

temp

warmth/

folder 02

emotion = 1

moving camera survey room or space, gestural / blurring, feet moving abstraction of architecture + interior objects and decor, moving objects: elevators, wheelchairs, water long shot of exterior or /

near-by space, beyond the body (representing experience) with a partner, make something that you might send to others on your internal experience (collaborate without speaking) some from above activity and some from the telepathic exchange exercise (see workshop documentation) an internal

GSR nerves = sensor 0

folder 01

EKG/

EMG

folder 03

blood = 2

breath = 3

folder 04

outdoors, close-up of trees, clouds, grass natural world, outdoor shadow play

Figure 11 C. Baker chart of the activities, resulting content and connection to the database and sensors

A deliberate effort was made to connect the content themes found in the participant video imagery, regardless of the workshop location, to the metaphorical qualities assigned to the sensors and mapped to the visual effects to mix them with the software. This connection correlated to an emotion or physical

237

attribute to an activity in the workshop [255] and it was very clear that certain activities elicited certain types of visual content creation and visual imagery capture. This outcome directly addresses the earlier discussion on non-verbal communication through visual material, because it became evident that certain activities could be developed (in future) to help create a specific visual communication language, or grammar to represent certain emotions or internal experiences, but using external material as symbolic, semiotic or metaphorical representations of those internal experiences; for another research project. Below is a sample of the spreadsheet used to categorise and analyse the video content in order to appropriately assign it to a database folder on the streaming server.

Image 74 2010 C. Baker screen shot of one of the spreadsheet pages (see appendices for full spreadsheet)

A.1.5/ mobile/ networks and connectivity

During the last stages of phase three, concessions and modifications were made all along to the original vision of the project in order to get it to the performance stage. These occurred while organising the last scratch events before the project officially ended, and while reflecting on the whole process I realised that these were necessary in order to facilitate a networked connection through the mobile phone, This happens in most projects and in developing performance and other productions; it is part of the evolution of the work. Some of those concessions and modifications included: 1) having to give into the possibility of foregoing a mobile only interaction for all technical operations of the project as:

255 This was NOT based on participant feedback, as they were not (and were purposely) asked the literal connection between their visual expressions in the videos and their actual emotional, physical or perceptual experience that they were trying to express this was though to be too invasive and participants were reassured that this personal information collection was not part of the work, but merely for them to express abstractly. Therefore this connection was made was based on my own interpretation of the material and its correspondence to workshop activities it came from.

238

a) we could not close the loop from qik to the streaming server [256] before development time limit expired, due to unresolved server port conflicts. A network specialist was needed and not available to address this problem, which would need doing before future events are staged, but there was not the time or resources to do so for the purposes of the PhD); b) loading and playing the mixed videos on mobile phones using RealPlayer (the only player available for mobile phones currently) on the Nokia models used for the project [257] was problematic it turned out. Nokia online forums indicate that this has been a common problem for many of the Nokia N-Series models, including streams from corporate news servers, such as the BBC [258]. Thus, I decided that for these scratch events we would focus on encoding for the web, rather than waiting for the technology to be debugged allowing people to see the streamed the mixes on the Internet; and c) based on Michael's advice, it was decided early on that using the Internet for streaming would be a necessary part of the network equation, even though I had originally wanted all aspects of the interaction to operate only on the phones, except for the video database, including: all body data processing, all VJ mixing and all the streaming to and from the phones to the database. However, with current mobile operating systems and networks this technically is still not quite possible, or was not when we started and needed it. In addition, even though the custom software was developed to receive the body data for the phones, due to the nature of the pairing system of Bluetooth connection, only one connection to sensors could be paired to one system at a time, not the phone and the computer simultaneously. So this only made multiple connections impossible and together these issues made it an imperative to rely heavily on computers and that it was not possible for every aspect to be mobile. Mobile companies may be developing new technologies in their research labs that will change these limitations, but not in time for this project, however. So a big early concession was: MINDtouch is not all mobile. 2) the network issues are similar for mobile phones as Internet issues, since phones can use the Internet connectivity. A significant discovery was that presence might not be explored in the way envisionedexcept in one-to-one sense (i.e.voice/sms/video/text), since the networks for the internet and mobile systems are set up differently, therefore delays and severe lags are still experienced on mobiles with video streaming. Presence in this context may be felt in a video-conferencing/video telephone sense at one point soon, BUT at present, with the limitations of resources, time and literal barriers of firewalls and limited access to seamless connections (all mentioned earlier), not to mention the economic

256 This was the script that pulled the videos onto from qik to the MINDtouch server while it streams outward to the Internet from the 257 iPhones have not yet been tested. 258 See screenshots from the forums in the appendices to show that is a common problem using Real Player on Nokia smartphones.

239

and corporate agendas of service providers, tariffs, plans, services etc., government/political boundariesthe project envisioned is not yet possible without major delays and lags and without numerous translations and 'interchanges' (Mitchell 2003:60) or daisy-chained workarounds, which all add to the delay. So liveness, which necessitates a sense of felt presence, are diminished at the moment, but can continue to be explored in future versions of this project. These are part of the discoveries from this research.

240

Appendix II/ workshop materials + technical elements

241

A.2.1/ technical timeline for technical development

November 2006January 2007 research and decision-making on equipment suppliers JanuaryMarch 2007 acquisition of equipment and beginning to use and work with it JanuaryMarch 2007 search for qualified programmer to work with AprilJune 2007 Li Zhang begins to work with the sensors and mobile phones to start the programming MarchSept 2007 troubles getting access to SDK/ API from Mind Media to do the programming on the phones to receive the bio feedback data July 2007 Li Zhang cannot continue to work on project due to new job opportunities and relocation July 2007 Dr. Price and I spend some time working with biofeedback sensors and Mind Media software systems just to get a sensing of the movement and stillness activities and vocabulary for further Tai Chi explorations, until the phones are ready to sue with the sensors. AugustSeptember 2007 Jeremi Sudol works on the project and make great head way programming the phones, but is delayed due to communication lags between Mind Media programmers and himself, but finally start to make progress but leaves unfinished due to his own PHD commitments and deadlines says theres two weeks full-time work left before there is communication between sensors and phones, but the project and hardware is more sophisticated than he originally though and the end seems always to be further way. September 2007- February 2008 new search for another programmer November 2007 return of one set of the sensors and Nexus 4 transmitter unit this was done in order to try another product (not available) and see if we have less trouble with manufacturer and in the interface programming between phones and sensing hardware January 2008 found possible new programmer /collaborator in Michael Markert from Germany, who is planning to join the PhD programme in July 2008, but start on the programming in March February 2008 ordering new sensor system from Infusion Systems

242

A.2.2 / workshop participant consent form Participant Consent Form

The SMARTlab Digital Media Institute Practice-based PhD Programme, UNIVERSITY OF EAST LONDON Workshop Consent Form - Confidential data Title: MINDtouch : Biosensors, Liveness and Networked Performance with Mobile Devices - Emerging Perspectives in Embodiment Theory with User Interface Devices Investigator/Workshop Leader Name: Camille Baker, MASc. Interactive Arts I understand that my participation in this project will involve being observed in a participant-centred workshop, conducted by Camille Baker as investigator, who will record the workshop on video, and ask me to fill in a survey afterward, to determine the effectiveness of participatory performance approach, as a part of a larger research project on mobile media performance and the experience on presence on the network as part of the practice-based PhD. I understand that participation in this workshop is entirely voluntary and that I can withdraw from the workshop at any time without giving a reason. I am free to withdraw my participation at any time or to discuss my concerns with the workshop leader or her supervisor. I understand that I am free to ask any questions at any time. I am 19 years of age or older. I understand that at the end of the workshop I will be provided with additional information and feedback about the purpose of the research project, should I request it. I also understand that the activities during this workshop may involve minimal physical and personal risks to me, such as tripping or falling- depending on how active I choose to be in the workshop. I shall undertake to move safely about the space, with this in mind. By involvement in this workshop, I understand that I am giving my informed consent for information I provide therein to be included for my video contribution to be broadcast worldwide, and for the outcomes of the event be published widely, and in future studies. I understand that the information provided by me will be held confidentially, such that only the Investigator (workshop leader) can trace this information back to me individually. The information will be retained for up to 10 years, when it will be deleted/destroyed. I understand that I can ask for the information I provide to be deleted/destroyed at any time and, in accordance with the Data Protection Act, I can have access to the information at any time. I also understand that I may register any complaint with University Of East London, University Research Ethics Committee. I, ___________________________________(NAME) consent to participate in the workshop conducted by Camille Baker, SMARTlab Digital Media Institute PhD Programme, UNIVERSITY OF EAST LONDON, under the supervision of Professor Lizbeth Goodman. Signed: Date: Further information can be obtained from Research Ethics Committee, University Of East London, Docklands Campus, 4-6 University Way, London, UK, E16 2RD, or contact Debbie Dada at +144208 223 2976 or d.dada@uel.ac.uk. I may obtain copies of the results of this event, upon its completion by contacting:

243

Camille Baker Flat 14, 58 West End Lane, West Hampstead, London England NW6 2NE UK

+44(0) 78 4768 1127 Notes on participation:Your signature on this form will signify that you have received a document which describes the procedures, possible risks, and benefits of this research project, that you have received an adequate opportunity to consider the information in the documents describing the project or experiment, and that you voluntarily agree to participate in the project or experiment.
Any information that is obtained during this study will be kept confidential to the full extent permitted by the law. Knowledge of your identity is not required. You will not be required to write your name on any other identifying information on research materials. Materials will be maintained in a secure location. Risks to the participant, third parties or society: none known. Procedures: The project is intended to gather video clips for use in a larger collective interactive visual collage, using mobile phones and biofeedback sensor during a live performance. We will collect specific information regarding the experience of the participant performer while collecting video impressions. Exit interviews will be conducted with as many participants as is possible or participants asked to take a list of questions and record their own answers on the video phones. Participants will be sent an online survey to answer any outstanding questions as well if necessary especially for those unable to do on-camera interview. Participants in the video workshops or performance events will create the video archive/database using the recording feature of the mobile phones. They are invited to participate and to express themselves using of mobile media tools either using their own video phones or ones supplied by the researchers. Each person using one of our phones will be asked to leave all their identification, contact information and belongings with the researcher for security against theft of the phones Participants may either use one of the camera phones supplied or may use their own equipment. All participants will be given a theme to use to help them to collect video that explores: the senses (5 external + internal) - attend to them and try express them non-verbally with video words or concepts (without talking) emotions, feelings, worries (non-verbally) intimate thoughts or ideas (no talking) shapes + patterns people, places, environment emotions/expressions thoughts/confessions motion (abstracted?), sounds

Then the participants will be shown how to phone or transfer them into the database, initially using www.comvu.com, www.jumpcut.com or www.shozu.com and or our customised database for the project. If this proves too complicated for the participants, all the media will be collected via Bluetooth and/ or USB directly into the researchers computer for later upload to the server. Benefits of study to the development of new knowledge: This media art research involves creating one major project to investigate 'liveness' and 'presence' in mobile performance. This research uses biofeedback sensors in tandem with the moving meditation, movement and play, in order to uncover any new understandings that may have emerged from the use of networked and wireless technologies. Under investigation are concepts such as: telepathic performance, performing presence, live versus simulated/virtual, simulated telepathy in performance. The phenomenological perspective of the experience of creating a mobile, networked performance, as well as the production of a curated mobile, networked performance event of other artists will be my contribution to new media performance studies. The intent is also to study the process, reception, and participation in the experience(s) of liveness, aura, presence during mobile performance events and activities constructed. The research will also study the performance experience of participant / performers and the audience reaction, in terms of embodiment of liveness and presence, within the network and staged live.

244

245

A.2.3 / workshop participation and non-exclusive licensing agreement

By this agreement, you understand that Camille Baker is producing and presenting a SMARTlab PhD research project, involving mobile media collection, leading to a networked mobile performance involving, for some participants, wearing biofeedback sensors on the body. This research project and workshops have already taken place take place since June 2007 in Vancouver, Canada, Dublin, Ireland, in London, UK, Sao Paulo, Brazil and will be completed in spring 2010. This document sets forth the terms of your agreement to participate in a SMARTlab PhD mobile media research project. This agreement is between Camille Baker and you as a participating Participant (the Participant) in a SMARTlab PhD research project. This agreement will set out the terms governing the licensing of your video clips created/recorded for this SMARTlab PhD research project (Work) to Camille Baker. 1. Subject to the terms and conditions of this agreement, you the Performer/ Participant hereby grant to Camille Baker a royalty free, non-exclusive, license to exercise the rights to the Work for the full term of the project and thereafter, including any and all extensions, as follows: a. All rights necessary for Camille Baker to produce, publish, advertise and perform live the Work for admission sales, or give-away print and online catalogues and/or books relating to a SMARTlab PhD research project. i. The rights necessarily granted thus include the right to reproduce, publish, distribute, display, transmit, transcribe, record on video, audio, digital or electronic media or otherwise use your Work, in all forms, formats and media whether now known, including without limitation, in print, digital and electronic form, or hereafter developed, throughout the world; ii. Insofar as any such catalogue or book may be deemed a Collective Work, because it may be comprised of a number of separate and independent works, you also grant to Camille Baker the right to incorporate the Work into Collective Works and to exercise the same rights in your Work as incorporated in that Collective Work, as granted in the preceding section, although your name can be listed as a participant if you so wish(a)(i).

2.

The Participant warrants and represents the following:

a. That the Work is original and that the Participant is the sole author and sole owner of the copyright in the Work. If the Work includes materials of others, the Participant has obtained the permission of the owners of the copyright in all such materials to enable them to grant the license herein and shall provide copies of such permissions to Camille Baker; If the Work is previously published, the Participant has all necessary rights, permissions, licenses, and/or other authorizations necessary to allow them to present their Work publicly should it be used in this SMARTlab PhD research project; b. That the presentation or performance of the Work for the SMARTlab PhD research project, and any use of the Work as permitted by the license herein, will not be defamatory, violate any right of privacy or publicity, infringe on any intellectual property rights, or otherwise violate the laws of rights of any third party; c. The Participant signing this document has the full right, power and authority to present the Work publicly as well as for this SMARTlab PhD research project and grants Camille Baker all rights granted by this agreement. 3. The Participant understands and agrees that the inclusion of their Work in the SMARTlab PhD research

246

project is at the sole discretion of Camille Baker and that agreeing to the terms of this agreement in no way guarantees participation. The Participant further understands and agrees that Camille Baker reserves the right, at any time up to and including the scheduled date of their participation in a SMARTlab PhD research project, and in its sole discretion, to cancel the Performer/Participants participation in a SMARTlab PhD research project. 4. The Participant agrees to indemnify Camille Baker against any and all losses, liability, judgments, costs, including without limitation, reasonable attorneys fees arising out of any breach of the above representations and warranties. 5. The Participant agrees that to the extent their Work for a SMARTlab PhD research project requires approval of any law enforcement or administrative body, including but not limited to the acquisition of permits for projects taking place in public spaces, that acquiring such approval is solely the responsibility of the Camille Baker. Participant will not be responsible for obtaining such approvals and makes no guarantee as to whether such approvals are or are not necessary. 6. The Participant further agrees to indemnify and hold harmless Camille Baker for any and all liabilities, fines, claims, errors, omissions, losses, accidents, and expenses, including attorneys fees and court costs in any manner caused by, arising out of, or in connection with, either directly or indirectly, the Performer/Participants participation in a SMARTlab PhD research project, the Work, the license granted herein. 8. This agreement will be shall be treated as though executed, delivered and performed within London, England, without regard to conflicts of laws rules, and so governed by the laws of London, England; any disagreement regarding it must be heard by a tribunal in London, England, unless we both agree to a different location. This is our entire agreement and supersedes any and all oral understanding or agreements and may not be modified except by another written document signed by both parties.

IN WITNESS WHEREOF, this Agreement has been executed as of the day and year first above written.

By: (Performer/Participant Date

(Camille Baker)

Date

247

A.2.4/ Participant Profile Information

Introduction: As part of a larger research project conducted in these media collection workshops, you are asked to express your inner perceptions and sensations, non-verbally, through your mobile videophone. This is meant to act as a simulation of telepathic communication, and help you to do so by visually expressing your emotions, thoughts, daydreaming, physical sensations, and perceptions. The project then plans to use these video representations to create a performance installation simulating collective consciousness and creative exchange. Instructions: Please answer the following questions as honestly and clearly as possible to help us understand your user profile. Attitude Inventory:

How do you perceive your bodys sensations, as well as your thoughts, dreams and other internal experiences and perceptions? How do you feel about expressing yourself and sharing these sensations and experiences with others? What are your attitudes about participating in a performative exploration through mobile or wearable electronic devices? How do you feel about sharing these media expressions and performative experiences in a larger performance installation? What are your expectations about this project and the workshop activities?
Demographic Information:

Age: Highest Level of Education: Academic Major/Stream: Annual Salary:

Gender: Ethnicity/Nationality: Occupation: City of Residence:

248

A.2.5/ step-by-step videophone expressions workshop procedure

This procedure uses the practice-as-research, and ethnographic methods primarily. Individual performers/ participants, from untrained and trained backgrounds have an introduction to the research aims again, and then each is given a consent form and release form to sign. 1. 2. 3. Each participant performer is given consent forms and participant information surveys to complete, which takes 20-30 minutes, including the explanation of why they are needed. Participants in a group are led through a 10 minute meditation/ guided visualisation/ exploration of senses as preparation, Then each participants takes a video/ camera phone or use their own, and are given 3-4 separate activities to record expressive videos, each with a theme or goal to collect certain types of internal perceptions and sensations. Themes to use to help participants to collect video include exploring: 4. 5. 6. the senses (5 external + internal) - attend to them and try express them non-verbally with video words or concepts (without talking) emotions, feelings, worries (non-verbally) intimate thoughts or ideas (no talking) shapes + patterns people places/environment emotions/expressions thoughts/confessions motion (abstracted?) sounds

There are two activities that are done individually and two that are done in pairs, to help facilitate the act of communication with someone. After each video collection activity, participants are led through another mind-quieting, guided imagery visualisation. Then the participants are be shown how to videophone the clips they have created into the database, initially using www.comvu.com, www.eyespot.com or www.shozu.com and eventually a customised database for the project. When this proves too complicated for the participants, all the media is collected via Bluetooth and/ or USB directly into the researchers computer for later upload to the server. Participants complete exit questionnaires and they are asked to take a list of exit interview questions and interview their partners or record their own answers on the videophones (for the most honest, uninhibited and possibly performative answers), or the researcher conduct these video interview. Some participants might be sent an online post-survey, to answer any outstanding questions if necessary especially for those unable to do on-camera interview.

7.

8.

249

A.2.6/ workshop mind quieting script

Find a place in the space; lie down if you like, to be more comfortable [pause] Close your eyes and try to relax and tune into your body. [pause] Then, take several deep breaths down into ribs and throughout your body, be silent, and settle into your body. [pause] Begin to pay attention to your breath, heartbeat, sensations in your body and the images or colours behind your eyes. [pause] Try to listen to/feel the sensations inside your body, such as: hot or coldness, tingling, twitching, pain, gurgling in your stomach, the movement of muscles in your organs or limbs, or your blood flow or your thoughts, emotions, and other perceptions. [pause] Now focus on one area of your body that draws your attention, try to stay with that sensation and take a mental snapshot of it, as if to record it for future reference. [pause] Keep focusing your attention on that part of the body. [pause] Experience thus inner sensation for as long as you can. [pause] Think now how you might you express this sensation non-verbally physically and visually to someone else? [pause] Pay attention to the images or thoughts or emotions you are experiencing and how you might express them to someone else. [pause] How would you transmit these image sensations to them? As a feeling or image? As a thought or some other inner form? [pause] If you could send or transmit these sensations, images or emotions to this person, either telepathically or using the videophones, think again of your snapshot and what you would send them now [pause] Think now of who you might want to share them with near or far (non-verbally), doctor, colleague, friend, family or lover. [pause]

250

Hold these images and sensations with you now linger with them. [pause] When you are ready, open your eyes and try to take these sensations or images or emotions and experiences with you and use the camera/video phones now to express them, starting very close to your own body or space around you on the floor. Do this for 10 minutes.

251

A.2.7/ post-workshop questionnaire post workshop

252

253

254

A.2.8/ post-workshop interview questions

These questions are intended help me, as researcher, understand each participants experience and find out if the activity was successful in initiating a embodied experience 1) Describe what you just experienced? 2) What did your body feel like when you were performing or in the audience? 3) What did you think about during this experience? 4) What sensations did you perceive during this experience? 5) Did you sense or perceive anything of particular of interest or out of the ordinary? 6) Did you think about or feel anything of particular of interest or out of the ordinary? 7) Was there a sense of the other performers or audience during this experience, yes or no if yes what did it feel like? 8) What did performing feel like ? What did you sense, perceive or think while performing, and after? 9) Is there anything else you would like to say abut your experience?

255

A.2.9/ workshop video file content database

Date Workshop Location Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Participant Video#/ Name Length Name in Time Jin 180620073gp 00:10 00:10 00:09 00:09 00:07 00:09 00:09 00:14 00:22 00:12 00:16 00:11 00:11 00:31 00:09 00:09 00:32 00:31 00:28 00:24 00:27 00:27 00:23 00:30 00:20 00:17 00:13 00:09 Content Type / Imagery

16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007

Jin 180620070013gp Jin 180620070023gp Jin 180620070033gp Jin 180620070043gp Jin 180620070053gp Jin 180620070063gp Jim Jim Jim Jim Jim Jim Jim MOV000023gp MOV000043gp MOV000053gp MOV000063gp MOV000073gp MOV000093gp MOV0000103gp

eyes, close-up on face, slow motion close-up red, undefined object, slow long-shot, upward, interior structural, gestural waving long-shot, upward, interior structural, gestural waving close-up, red fabric, turning medium-shot, upward ceiling pipes, dark gestural medium-shot, upward ceiling pipes, dark gestural close-up on mouth, minimal movement close-up on mouth smoking, minimal movement extreme close-up of eye, from black extreme close-up of ear, from black close-up of TV with figure making gestures extreme close-up of ear with headphone extreme close-up of blurry text on book? side view of himself walking different views of himself walking zooming in and out to ceiling light zooming in and out to door stop moving up and down from wall to light moving side to side on wall to firebell moving up and down from light to interior ceiling structure scan around the interior building space extreme close-up of metal wall bump moving up and down from light to interior ceiling structure extreme close-up of dark space/ wall downward view of legs and feet close-up of arm tattoo extreme close-up of bare feet

Jim MOV0000113gp Jim MOV0000123gp Nels 06182007029mp4 Nels 06182007030mp4 Nels 06182007031mp4 Nels 06182007032mp4 Nels 06182007033mp4 Nels 06182007034mp4 Nels 06182007035mp4 Nels 06182007036mp4 Nels 06182007037mp4 Nels 06182007038mp4 Nels 06182007040mp4 Tiina 18062007095

256

16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007

Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver

Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Tiina Rina Rina Rina Rina Rina Rina Rina Rina Rina Rina Rina Rina Rina

18062007096 18062007097 18062007098 18062007100 18062007101 18062007102 18062007103 18062007107 18062007108 18062007109 18062007110 18062007111 18062007112 18062007113 18062007115 18062007116 18062007117 c-1.mov c-2.mov c-3.mov c-4.mov c-5.mov c-6.mov c-7.mov c-8.mov c-9.mov c-10.mov c-11.mov c-12.mov c-13.mov

00:01 00:08 00:11 00:02 00:07 00:28 00:13 00:13 00:05 00:01 00:01 00:13 00:41 00:41 00:02 00:08 01:12 00:10 00:47 00:22 00:41 00:20 00:15 00:27 00:23 00:11 00:24 00:42 00:17 00:22

extreme close-up of bare feet extreme close-up of bare feet extreme close-up, upside down slow pan along carpeted floor extreme close-up, upside down slow pan along carpeted floor extreme close-up, upside down slow pan along carpeted floor mid-shot, slow pan along wall and ceiling molding black + white close-up of metal wall bolts blurry close-up of interior structures extreme, very dark close-up of wall bolts extreme, very dark close-up of finger dark close-up of arm on a wooden stair rail very dark, black + white shot of floor, hand and legs very dark upside down shot of floor and legs outdoor pavement close-up outdoor pavement mid-range shot outdoor downward shot of feet walking outdoor upward + zoom shot of trees + sky moving camera focussed on grass + a tree moving camera focussed on pavement moving camera focussed on piping grid extreme close-up of a tree extreme close-up of participant's hand extreme close-up of gravel and leaves extreme close-up of concrete object extreme close-up of pavement designs extreme close-up of blurry ground extreme close-up of blurry grass and then pull out to long show of trees and lawn mid-shot of trees and leaves, park grounds zoomed in shot of trellis rooftop extreme close-up of blurry doors and surroundings

257

16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 16-Jun-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007

Vancouver Vancouver Vancouver Vancouver Vancouver Dublin Dublin Dublin Dublin Dublin

Rina Rina Rina Rina Rina

c-14.mov c-15.mov c-16.mov c-17.mov c-18.mov

00:32 01:35 00:36 01:07 00:29 00:12 00:41 00:24 00:20 01:34

close-up of pavement, doors and surroundings extreme close-up of copper sculpture close-up of pavement features extreme close-up of pavement and rail line metal ground level shot of pavement moving, turning, shot of roof moving, turning, running shot of wooden floor slow moving shot of corduroy trousers moving, turning, twirling shot of room, upside down at times extreme close-up of a slow pan of metallic furnace or heating vents, wall moldings and features close-ups and extreme close up of a woman and her digital camera image, moving around the space running shot around the room shots of a woman jumping with her digital camera turning of phone of woman and her camera upside down turning, close-up shot of woman extreme close-up of a dead fly on a ledge sideways shot of woman with her camera sideways follow shot of a pigeon zooming in and out to roof /ceiling structures zooming + panning in + out of darkened roof skylight erratic shaking, zoomed in, darkened roof skylight slow swinging + panning roof skylight + ceiling structures extreme close-up of white metal wall close-up of white concrete wall and moldings extreme close-up of white electrical wall plug side turning close-up of white water heater

Bobby 22072007149.mp4 Bobby 22072007150.mp4 Bobby 22072007151.mp4 Bobby 22072007152.mp4 Bobby 22072007153.mp4

22-Jul-2007

Dublin

Bobby 22072007154.mp4

01:18

22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007

Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin

Bobby 22072007155.mp4 Bobby 22072007156.mp4 Bobby 22072007157.mp4 Bobby 22072007158.mp4 Bobby 22072007159.mp4 Bobby 22072007160.mp4 Bobby 22072007161.mp4 Jo Lee Jo Lee Jo Lee Jo Lee Jo Lee Jo Lee Jo Lee Jo Lee MVI_0139.AVI MVI_0140.AVI MVI_0141.AVI MVI_0142.AVI MVI_0143.AVI MVI_0144.AVI MVI_0145.AVI MVI_0147.AVI

00:11 00:11 00:48 00:52 00:54 00:18 00:37 00:24 00:22 00:16 00:19 00:23 00:33 00:20 00:15

258

22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007

Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin

Jo Lee Jo Lee Jo Lee Jo Lee Jo Lee John John John John John John John John John John John John John John John John John John John John John John John John John John John

MVI_0148.AVI MVI_0149.AVI MVI_0150.AVI MVI_0151.AVI MVI_0152.AVI Video070.mp4 Video071.mp4 Video072.mp4 Video073.mp4 Video074.mp4 Video075.mp4 Video076.mp4 Video077.mp4 Video078.mp4 Video079.mp4 Video080.mp4 Video081.mp4 Video082.mp4 Video083.mp4 Video084.mp4 Video085.mp4 Video086.mp4 Video087.mp4 Video088.mp4 Video089.mp4 Video090.mp4 Video091.mp4 Video092.mp4 Video093.mp4 Video095.mp4 Video096.mp4 Video097.mp4

00:25 00:17 00:15 00:17 00:30 00:14 00:20 00:22 00:26 00:13 00:28 00:34 00:36 00:31 00:28 00:34 00:05 00:25 00:05 00:27 00:22 00:24 00:10 00:22 00:12 00:10 00:03 01:09 00:05 00:25 00:34 00:24

close-up of hand crawling on a white concrete wall close-up of port hole window and extreme close up of light extreme close-up of cabling shot of port hole window & woman through it shot of opening and closing elevator doors extreme close-up of black dot on white surface close-up of wood floor close-up of opening and closing of hand slow pan from wood floor to close up of legs side-ways shot of woman and others in the room close up of wood floor long shot of sky to quick pan to close up of wall close up of legs to pan of room back to legs blurry white/ grey indefinable image blurry indefinable blue/grey image extreme close-up of fabric/ jeans close-up of face close-up pan of clothing and body mid-range shot of wood floor sideways shot of sofa chair pan to window, zoomed in outdoors zoomed in shot of pigeon, pan to trees, back to pigeon outdoor shot of grounds zoomed in shot of pigeon extreme close up of floor, pan around the room, of woman taking photos close-up, zooming in and out, panning of porthole window downward shot of piano pedals and feet extreme close up of indefinable blue transparent object close up pan around a bicycle close up pan around a bicycle pan around a parking lot side shot of yard, tree and cryptlike object close up of mossy ground

259

22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007

Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin

John John John John John Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth Lizbeth

Video098.mp4 Video099.mp4 Video100.mp4 Video101.mp4 Video102.mp4 SSPX0019.mov SSPX0020.3g2 SSPX0021.3g2 SSPX0022.3g2 SSPX0023.3g2 SSPX0024.3g2 SSPX0025.3g2 SSPX0026.3g2 SSPX0027.3g2 SSPX0028.3g2 SSPX0029.3g2 SSPX0030.3g2 SSPX0031.3g2 SSPX0032.3g2 SSPX0033.3g2 SSPX0034.3g2 SSPX0035.3g2 SSPX0036.3g2 SSPX0037.3g2 SSPX0038.3g2 SSPX0039.3g2

00:12 00:10 00:07 00:27 00:07 00:08 00:14 00:30 00:25 00:09 00:14 00:30 00:30 00:30 00:30 00:30 00:30 00:30 00:30 00:30 00:30 00:30 00:30 00:30 00:30 00:30

close up of woman's slow walking feet along a car close-up of porthole window slowly turning shaking shot of fast dancing feet roving shot of metal bucket, old wood floor and antique sewing machine mid-shot of flowery trees opening with upside-down face, then moving around room close-up moving along the body and wood floor phone waving vigorously around the room phone waving slowly around the room and along the body phone waving slowly around the room phone waving slowly around the room upside-down shot moving from close-up of face to long shot of outdoor view zooming into close-up of friend in wheelchair shot moving from close-up of face to view around the room shot moving from close-up of foot and face to view outside shot moving from close-up of leg and face to view outside and room shot moving from close-up of leg and body to view of the room in background shot of reflection on the wood floor and leg waving through shot of friend in wheelchair and around the room shot of reflection on the wood floor shot of another person shooting, scan to person in wheelchair shot of person in wheelchair and his computer screen close-up of person in wheelchair and back to her own face and body close-up of another phone video, the person in wheelchair and extreme close-up of hand mid-shot the person in wheelchair close-up of the person in wheelchair and his computer

260

22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007 22-Jul-2007

Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin

Lizbeth Lizbeth

SSPX0040.3g2 SSPX0041.3g2

00:30 00:30 00:24 00:07 00:24 00:16 00:24 00:24 00:24 00:24

close-up of the computer and the person in his wheelchair close-up of the person in wheelchair, close-ups of face, hands eyes mid shot of the person in wheelchair and others around him long shot of outdoors, garden, lawn zooming into close-up of friend in wheelchair zooming into close-up of friend in wheelchair and extreme close up of hand mid shot of the person in wheelchair woman with him shot of woman recording in front and moving toward screen mid shot of the person in wheelchair and woman with him mid shot of outdoors, garden, hedge zoom out of shadow on the grass zoom in of a branch mid shot of shadow on pavement shot of people on a blanket and mid shot of legs slow zoom out from extreme close-up of glasses and eye to face and face long shot of shadows on the grass extreme close up shot of blanket and grass extreme close up shot of girl's face/head and activity, zoom out to a woman's face/head close up of grass, flower and hand close up of dandelion fluff long shot of shadow figure and tree on grass long shot of shadow figures on grass long shot of shadow figures dancing on grass and sideways mid shot of top of woman's head panning around close up of a circular window or object

James/ ChrisjamesMINDto Chris uch.mov James/ james_video.mov Chris James/ Jamescloseup2.m Chris ov James/ Jamesintro.mov Chris James/ Jameslizbethtrees. Chris mov James/ JamesMINDtouch Chris 3.mov James/ JamesMINDtouch Chris 4.mov James/ Jamesoutsidetrinit Chris y.mov Amalia Amalia Amalia Amalia Amalia Amalia Amalia Amalia Lori Lori Lori Lori Lori Lori Video109.mp4 Video110.mov Video111.mov Video112.mov Video113.mov Video114.mov Video115.mov Video116.mov SSPX0000.3g2 SSPX0001.3g2 SSPX0002.3g2 SSPX0003.3g2 SSPX0004.3g2 SSPX0005.3g2

17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007 17-Aug-2007

Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver Vancouver

01:06 00:04 00:11 00:08 00:31 00:18 02:08 01:13 00:30 00:30 00:30 00:30 00:30 00:30

26-Oct-2007

Dublin

Blair 20071026013.mp4

00:21

261

26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007

Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin

Blair 20071026014.mov Blair 20071026015.mov Blair 20071026016.mp4 Blair 20071026017.mp4 Blair 20071026018.mov Dermot Dermot Dermot Dermot Dermot Dermot Dermot Dermot Dermot Dermot Video118.mp4 Video119.mp4 Video120.mp4 Video121.mp4 Video122.mp4 Video123.mp4 Video124.mp4 Video125.mp4 Video126.mp4 Video127.mp4

01:40 00:25 00:25 05:50 00:58 00:06 00:26 13:53 00:12 00:03 00:09 00:41 00:26 00:29 00:58 00:18 00:10 00:26 00:35 00:30 00:25 00:12 00:10 00:23

downward shot of stairwell in a circular pan moving close up shot of wall moldings to extreme close up of boot moving extreme close up shot of boot, another person, the room and a wheelchair moving shots of outdoors and other women in the leaves, in parking lot close ups of objects and words around the space, ending on piano keys zoomed in close up of finger tip moving shot of wooden floor moving shot of wooden floor, people in the room, close ups of jeans and hand moving shot of wooden floor, a chair and a hand moving shot of wooden floor and hand moving shot of wooden floor, around the room and the roof window moving shot around the room and the roof window pan of outdoor parking space /alley sideways shot of tree, straighten to lawn downward moving shot of legs walking and reflection on pavement sideways moving shot of building extreme close-up of fingers extreme close-up of finger tips extreme inverted colour close-up of fingers and hand extreme close-up of unknown object extreme inverted colour close-up of unknown object blurry view through a window extreme close-up of lips, mouth and cheek extreme close-up of white/grey wall

Dermot Video128.mp4 Participant 27102007210.mp4 F Participant 27102007211.mp4 F Participant 27102007212.mp4 F Participant 27102007213.mp4 F Participant 27102007214.mp4 F Participant 27102007215.mp4 F Participant 27102007219.mp4 F Participant 27102007220.mp4 F

262

26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Oct-2007 26-Jan-2008

Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin Dublin London

Participant 27102007221.mp4 F Participant 27102007222.mp4 F Participant 27102007223.mp4 F Participant 27102007224.mp4 F Participant 27102007225.mp4 F Participant 27102007226.mp4 F Participant 27102007227.mp4 F Marcus Experiment1.mov Marcus Experiment1b.mov Marcus Experiment1c.3gp Marcus Experiment2.3gp Marcus Experiment3.3gp Marcus Experiment4.3gp Marcus Experiment5.3gp Babak 20080128022.mp4

00:15 00:25 00:11 00:12 00:15 00:02 00:25 01:46 00:56 01:01 00:44 00:44 01:10 01:38 01:32

extreme inverted colour close-up of lips, mouth and cheek extreme sepia tone close-up of hands mid-long shot through a window of sky and building long shot through a window of sky and cars below mid shot of reflection on wood floor close up of head shot outside with sun reflecting moving close up of pavement zooming in and out of radiator coils through window shot of sky/ clouds and outside buildings close up of wall writing; zooming in and out of shoes zooming in and out of porthole window close-up of hand playing piano interior close-up on wall and long shot of room moving shot of kitchen mix of images: ceiling, lights, computer screen, moving dot reflection, trees outside, close up of patterned fabric and screensaver close ups of heads and hair moving shot around the room shot of turning on water, sink, running water shot of others videoing each other, close-ups of faces and cameras moving shot of floor, legs, and hand moving shot legs close up on fingers and hand downward view of moving legs close up of teeth and mouths extreme close-up of eyes extreme close-up of words on a paper

26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008

London London London London London London London London London London London

Babak 20080128023.mov Babak 20080128024.mp4 Babak 20080128025.mp4 Babak 20080128026.mp4 Babak 20080128026.mp4 Vesna Vesna Vesna Vesna Vesna Vesna Video135.mp4 Video136.mp4 Video137.mp4 Video138.mp4 Video140.mp4 Video141.mp4

00:18 00:20 00:51 00:39 00:22 00:08 00:21 00:17 00:12 00:49 00:16

263

26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008 26-Jan-2008

London London London London London

Zoe 29012008239.mp4 Zoe 29012008240.mp4 Zoe 29012008241.mp4 Zoe 29012008242.mp4 Zoe 29012008243.mp4

00:15 00:13 00:22 00:07 01:30

medium shot of yellow and black line on floor medium shot of feet moving around on floor medium shot of feet moving very little medium shot of feet and the room medium shot of water filling a container from tap and moving the container and hands playing around and in the water, making patterns on the counter close up of fingers - to make it seem like they are walking on the ground and around the room medium shot of hands moving and animating against the floor as background follow shot of someone walking down the hall from neck down, then finger obscuring medium shot of torso, making various gestures, then turning upside down, plus hand gestures shot of floor medium shot to zoom in of reflection recording zoom out to medium shot of reflection recording blurred moving shot of floor pattern moving upward angled headshot with background changing moving downward headshot with background changing moving sideways headshot surroundings changing moving mid shot of woman with coke long shot of canal water and plane taking off zoom in shot of woman looking at the view moving shot of pointed finger and woman's feet moving camera through a door to encounter a woman video recording long shot of hallway with a woman video recording

26-Jan-2008

London

Zoe 29012008244.mp4

00:36

26-Jan-2008 26-Jan-2008

London London

Zoe 29012008245.mp4 Zoe 29012008246.mp4

00:54 00:44

20-Mar-2009

London

Alec

ar-Smart-3.mov

00:39

20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009

London London London London London London London London London London London London London

Ivan Ivan Ivan Ivan Ivan Ivan Ivan Ivan Ivan Ivan Ivan Ivan Ivan

Video000.3gp Video001.3gp Video002.3gp Video003.mov Video004.3gp Video005.3gp Video006.3gp Video007.3gp Video008.3gp Video009.3gp Video010.3gp Video011.3gp Video012.3gp

00:01 00:08 00:08 00:07 00:08 00:08 00:08 00:08 00:08 00:08 00:08 00:08 00:08

264

20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009 20-Mar-2009

London London London London London

Ivan Ivan Ivan Ivan

Video013.3gp Video014.3gp Video015.3gp Video016.3gp

00:08 00:08 00:08 00:08 00:07

side shot of hallway with a woman in the foreground video recording shot of wall zooming into a headshot of a woman smiling zoom in a circular graphic object zoom out and fast moving shot of circular graphic object shot of feet and carpet

JeeHee 20090320035.mp4

265

A.2.10/ technical updates and software guides

A.2.10a/ Arduino and mobile software guides
After a meeting end of February 2008, the current project status is as followed:

Mobile Phone Visualisation invoking a Video Database: The following restrictions apply for J2ME applications: - no video streaming via Bluetooth (but with WiFi) -no pixel-manipulations on videos, due to missing codec implementation(but on photos), see http://java.sun.com/javame/reference/apis/jsr135/ -access from and to video-database only via WiFi multiple Bluetooth connections from (multiple) sensor-boards to - no one lesize-restriction for application-embedded videos (but videos stored on the mobile or just -jar phone recorded) no video-out on any phone / handheld (-> -normally audience?) - no Video-Mixing (Overlays, Blends) possible with phones; neither J2ME apps (incl. those on the available Nokia devices) - even the iPhone has no support for live-video-mixing (in terms of smooth blending) and/or Quartz Patches andthis is a very suitable, brand-new mobile device with a noticeable processing power!

Conclusion: The processing powers of mobile handheld devices are simply too low to enable pixel-based video manipulations.However, 2D Animations are possible (Pixel / Flash Lite / Text / 3D), as well as simple consecutive video playback and/orvideo with graphic overlays. For more extreme manipulations and smooth transitions, a solution invoking a desktopcomputer (I denitely recommend a Macintosh with Quartz Composer) is required.I can imagine these possible solutions: 1) Video Streaming via WiFi & browser, I recommend data processing with Quartz Composer (& Video DB Access) and streaming QT Web-Server: SENSOR <-Bluetooth -> Mac / Quartz Composer -> Network -> QT-Server -> WiFi/GPSR/UMTS/EDGE -> Phones the sensor device sends its values by bluetooth to a computer which takes these values and uses them to mix visualsand videos from the database, then streams the results to a Quicktime Streaming Server that feeds this stream onto theweb, where it can be streamed to WiFi-enabled mobile devices. 2) Videos are stored on the phone / inside the .jar and accessed by the J2ME application: SENSOR <-Bluetooth -> Phone If the device is able to access the web, more videos could be downloaded from a database and used randomly in abackground process. However, I think the application should produce results even when its not connected to the web. Id strongly recommend adapting the premises to the available technology, the other way is full of obstacles! 3) Videos are recorded live by the performer (eg. 4 small clips), then live-rendered and triggered by biofeedback. This could constantly produce new visual output. The visuals can be merged by in-betweengraphic animations, 2D-graphic overlays etc If the performer uses a WiFi enabled device, a background upload into a database should be possible.

266

-What body function is controlling what visual reaction? Body states such as pulse, breath and skin resistance arent actively controllable events and their change is usually slow and almost impalpably. -When theres an audience mentioned: what does the audience do: does it watch streamed videos? does it actively participate, send videos? mobile phones dont have no Video-Out, so you cannot beam the screen.

Things to remember: A mobile phone (even PDAs or gaming devices like the Nokia N-Gage or the new iPhone) has low processing power:In contrast to 32 or 64 bit desktop computers with averagely 1 GB RAM, were talking of 8 to 16 bit microcontrollers with 16 kB to 8 MB RAM A SPP (serial connection) via Bluetooth is only possible between two paired devices!

Web connections arent realtime anymore, a delay has to be calculated. more downsides to a web-based application is slow timing and expensive costs for GPSR devices that have no built-in WiFi module. The gain of J2ME-universal access is lost by the restriction of WiFi.

Michael michael@audiocommander. de

267

After working through the code-sources, getting information on the Nexus-System and trying to compile the currentproject J2ME compatible, these are my current conclusions: Mobile Phone Application: -Code currently uses Java5 syntax (e.g. use of generics, which makes sense) - Code is based on super-new MIDP2.1, which is extremely seldom (e.g. Android -> no hardware yet!) -Java5 syntax requires additional tools like J2ME Polish (for example) -J2ME Polish cannot (or not easily) work with MIDP2.1, just MIDP2.0 -The targeted Nokia N80 (and others) uses Symbian OS S60, which provides only support for MIDP2.0 -S60 supports C and C++ programs (access to security related hardware features requires Symbian Signedauthorisation ($$) Mobile Phone Visualisation: Video Streaming via Bluetooth is far over my programming powers Video Streaming via WiFi & browser easily accomplishable Mobile Phones have low processing power, video playback wont be realtime and has to be low-resolution and/or efciency-optimised 2D Animations possible (Pixel / Flash Lite / maybe 3D)

Nexus Sensor Device: Complicated signal decoding, poor API documentation due to proprietary code protection (its a security and trust-demanding medical device that needs to make sure no lives can be endangered directly or indirectly). Proprietary sensory device that can be paired with only one Bluetooth receiver at a time. The huge advantage of working with mobile phones is that they are available in masses without any cost (everyone has one). By coupling oneexpensive & maybe-not-even-yet-available phone to the Nexus and communicating with difcult encoded messages,the mobile application has to be tightly bundled to this and only this Nexus model. Besides the risk of total dependenceon the manufacturer to continue sales and support for this product, I dont see any technical reason why it has to be arestricted J2ME phone and not an open device running Linux Embedded (like Openmokos Neo1973). Conclusion: I agree 100% with Jeremis nal comment about going for an alternative approach with a custom sensor board.My proposals: 1. Sensors -> Arduino BT -> Mobile Device (with Processing J2ME) See arduino.cc and libelium.com for further information Arduino BT Specs: 8 AIN (USB: 6 AIN), DIN/DOUT: 14 (PWM:6) It s cheap: 1 Arduino BT assembled costs 100,- , multiple devices possible!Its easy: no complex protocol decoding, standard protocols, open libraries available => new features instead of reinventing the wheel!Its open: and therefore potentially popular => {Arduino+MyPhone=100$} OR {Nexus+HiTechPhone=$5000}? Arduino is commonly used: plenty of information and code available Processing seems very straightforward, huge community as well, BT lib available The application will run on many phones => user/audience feedback can be implemented 2. Sensors -> ACSensorizer -> MIDI -> Mac -> QuartzComposer -> Web-Broadcasting incl. possible Bluetooth Feedback from local devices sending in custom media and/or dataSee audiocommander.de and midibox.org for further information ACSensorizer Specs: 8 AIN (expandable to 32), DIN:64, DOUT:64 (no PWM) It s even cheaper: 1 ACSensorizer unassembled costs between 25,- to 250,- (with or without HUI)Its easy: no host software needed, direct signal feed into QC (intuitive realtime VJ application)It s open, DIY, non-commercial (due to MIOS licence) Reliable: I spent 2+ years developing. Signal enhancement & ne control avbl. at the lowest processing levelExpandable with hardware modules and software (custom QC Plug-ins) By using the World Wide Web many-to-many and local-to-distance relations could widen the project scope

Michael michael@audiocommander.de

268

This is a bit of a summary of where we are with the MindMedia project (alreadyspoke with Camille about this). Essentially, I kind of felt like Alice inwonderland at some point, because the deeper I needed to dig to get this towork, the more opened up that needed to be implemented due to the complexityof their underlying design. What we have at the moment is an understanding of the communications protocolenough to communicate with the device, get status and device information,initiate and stop the streaming from all the sensors attached. While that'sgood it's not good enough, because the raw data is encoded using a variablelength encoding scheme of their proprietary design (more complex data gets tohave more bits, very smart and efcient design). But to get that requires alittle bit more work -- I have outlined the next steps in the fullassessment/readme le (which is the one attached). In my estimate it wouldtake about two weeks full time work to get that sorted. If there's someoneelse that will pick this up after me, I would gladly answer any questions tobring them up to speed. The overall assessment is that it's a) denitelyworkable, and b) 85% there. The reason for it taking so long, is that it'sreally a high end medical device and they had to from the getgo use a complexdesign to handle a very large amount of data streams over very limitedchannels (bluetooth is fairly low bandwidth and their new devices using thesame protocol handle more than 10 channels at insane sample rates-it's prettyhot). And since we're doing this custom for portable devices we can't usetheir DLL -- which takes care of these details for us.

Unfortunately with the amount of work I have on my hands at the moment I can'tcontinue working on this -- even though I'd really like to see this through tothe end -- especially since it's so close. So instead of holding up theprocess the sensors went back with Jacki on Wednesday. The box also containsthe N80 cell phone, charger, the BioTrace CD, and the CD with all associatedresources and source codes and documentation (a copy is also attached in thezip in this email). I'm around to answer any and all questions, technical or otherwise.

As always -- all the best greetings, ___jeremi

269

This is an asynchronous call and response system communicating over the serial port over a bluetooth connection. The bluetooth library takes care of all the connection details (as long as the particular device is present). I included the code (although it's not being used) for searching for other BT devices.

First all general information about the device (Nexus, also referred to in their documentation as the Front End) needs to be received by the PC (or the cellphone). These are the getFrontEndInfo, getVLDataInfo, and getIDData. The frontEnd information is the general information about the device. the VLdatainfo contains information that will be helpful in decoding the variable length data-- the raw sensor readings(sampling rates,and accuracy) IDDataInfo -- contains the critical information for each channel -- it's type, units, the gain value, For debugging purposes I have written a set of functions for easy conversion between string hex representation and actual hex values -- so it's easy to just send a message like "AAAA 2202 0010 0008" + checksum. The API indicates that some of the messages are unidirectional and some are bidirectional. This is how to communicate with the system. take the bidirectional messages, change the bits you want, and send them back -the return should be an acknowdledgment of that command. i din't get put in a robust system for checking that but you can see the simple examples in getting front end info for instance. also the PortiSerial code has a great suggestion on how to implement a generic ack system. Some of the datastructures seem weird -- and they are sometimes. trial and error and a lot of patience helps to pick this protocol apart. At this moment I am in the middle of parsing the IDData. important note, this is not yet implemented but needs to be done immediately -- iddata is too long to potentially fit in manageable messages. so it contains a mechanism where you specify in the request the starting address of the full structure and a number of words, and then the will recieve some of those pieces. you then need to assemble all the received messages and when the datastructure is complete, then parse it for all the critical data --that data is KEY to decoding the sensor readings. unfortunatley. Once the connection is established two threads are spawned, a reader and a writer that will handle all the low level communications. Java uses network byte order, while C++ the original design language here doesn't so at the lowest level all the bytes have to be reversed (see this in both the reader and the writer) -- this is critical, but after that conversion all byte operations are almost the same (special cases due to unsigned vs signed conflicts of java vs c++). see the extracted code in the cpp folder -- that is code from the portiserial project extracted to only deal with the decoding part -- but since i was missing the IDData information the decoding does not work. A lot of the functions in the API are not even implemented in the hardware here --it's not a problem, they're not really necessary. Take a look at the API (comm.pdf) and search for the related bytecodes in the source -- or search for the functionnames -- i tried to keep those nearly the same.

ACNexus
2008 Michael Markert http://www.audiocommander.de http://www.smartlab.uk.com

270

3. CODE STRUCTURE The code is aimed to maximize parallelism between doing development on the pc and having code that is simulatanously functional on a cellphone. That's why the code is shared and there are only two different wrapper classes one for each platform --Bluefin for the PC and TunaApp for the cellphone (although for debugging purposes I started adding a lot of System.out.println statements which don't work on the cellphone, so for real depolyment they will have to be removed, and some of that structure would have to be cleaned up). So the main app is Bluefin / or TunaApp Then the main logical control over the device is the SensorManager, it deals with high level operations like initialize, start/stop dataflow, and disconnect.In turn it contains a BTClient which is responsible for the actual transactions, manages the sender and receiver threads. BTClient extends the Nexus class, which contains all the nexus specific information, like message parsing, the necessary variables about the state of the system. This is where a lot of the nitty gritty happens. util.Hex provides the helpful functionality in conversions and managing bit operations. There's been a lot of experimentation in there and so multiple approaches to the same issue may be visible. Temp class --that what i use for quick prototyping and development and testing of new functions. it used to be in an outsider package, but due to scope issues i brought it in to air. VERSIONING I used darcs versioning while developing this system. so if you want to go back and see the incremental stages of development it's all there. darcs allows you to hold the repository locally --yey. (info and tutorials at darcs.net) -- note there may be a few stages before i added a lot of the "PortiSerial" c++ code ported over to java in the attempts to decode the variable length data. prior to those massive functions it was way cleaner and more manageable. it would be great if that code could be streamlined before adding it to the project -- but that's why the side version of the cpp experiments.

windows (and even if the system says they're in use, you can override it and use them anyways) just -- use the default microsoft stack driver for the bluetooth. it's more reliable -- take a look at proce55ing for mobiles at processing.org -- good for easy nice and quick visualization -- java does not have unsigned ints or shorts. just remember that when you're doing byte conversions and something's not right --in iDData parser i just started adding stuff, and there are casts to floats -- those are definitely wrong. just wanted to get something going -- there is a mindMedia forum on yahoo. it's mostly useful to the general users though, not so much low level developers. but potentially worth a try. i made us an account: user: mindmedia password: smartlab -- the chief engineer Pieter Ermers Email: Pieter@mindmedia.nl Phone: +31 (0)475 41 01 23 Fax: +31 (0)475 33 06 02 www.mindmedia.nl --next in the developing sequence for me -- finish requesting iDDataInfo --this means multi would be: message assembly with multiple requests, starting at various addresses until the whole large structure is received --parse that big structure -- use the information from iDDataInfo to seed the demo c++ VLData parser in the /cpp folder. get that data stream decoded properly -- try to streamline that code (their spaghetti mess is really less than appealing and very bloated) -- bring it in to the java project -- decode the variable length data. -- make sure the correct units also go right with the correct sensors --- this would be a great place to be at. -- then make a simple visualization -- create simple simulator with data (this could be done now already from data from bioTrace ) for visualization 5. OTHER THINGS This is an awesome device --very capable. It's just maybe a bit complex for very quick prototying innitially, especially when we have to redevelop the low level communications protocol (the generic windows developers get the DLL and don't need to worry about any of that) --so for that kind of development this would be easier. However with a bit more patience this will be done. For some encouragement just hook yourself up to the sensors and see your body vital signs on the screen. pretty amazing -now on the cell phone. The EEG sensors would really make this the bom-diggity. multi threading is key, and keeping the jar file size down. also important for good runtimes on the cellphone -- less division and costly operations. more bitshifts. Another alternative approach -- buying and wiring up cheaper sensors tocustom board with bluetooth. Much cheaper solution with full flexibilityto design a simple and reasonable communications protocol. instead ofhaving to deal with such an intense system. advantages include widerappeal, potential existing open source solutions (modules, both in termsof software and hardware).

MIDLET development to deploy the code (a jar ) onto the phone you will need the Sun JAva Wireless toolkit 2.5.1+ Sun Java Wireless Toolkit 2.5.1 (download free, requires Java 5.0+) the projects sit in C:\WTK2.5.1\apps (or where the WTK is installed) required project settings for the wtk Target platform JTWI Configuration CLDC 1.1 Mobile media JSR 135 Bluetooth OBEX JSR 82 when making a wtk project make sure you specify the package name for the class that will be the main midlet (ie. air.TunaApp ) ECLIPSE I have been using eclipse to develop this project. it's been really great. i highly recommened it. eclipse.org the .project and the .classpath files are for eclipse CURRENT STATUS if you run the Bluefin application right now it should connect to the nexus, get the frontend info and the vldatainfo, and for 5 secods pull data from all available channels and then terminate. you can save that output and use those logs for quicker development than directly with the device everytime.

4. OTHER THINGS Curious and randomg things I've come across during this project. -- sometimes when you're getting incorrect or weird responses from the nexus the batteries may be dead -- check those once in a while, as the LED light does not indicate low power. -- you can assign your own com ports to suit you better if you need in

271

Preface

System Preferences: Java enabled device with the following implemented protocols: JSR 082 for Bluetooth Connections JSR 135 1.2 and above for MIDI Sound ACSpecs can be used to determine if and which versions of these JSRs are available. -MIDP2.0 / CLDC 1.1 for Floating Point Math

272

I. Main Menu II. General Control Module Overview All main modules (see I.) have a unied menu control. Use the cursor keys:UP/DOWN to switch between menu itemsLEFT/RIGHT to toggle the selected parameter ACPlotter Graph plotting values ACPlotter Screen Menu -Units & Color ACSensorizer MIDI synth sonies values -Enhancement Method -Samplesize Scale (From / To) ACConsole Serial Terminal to connected devices Find & connect to ACNexusBT devices Presets Load and Save Patches (& Settings) Specs Show Information about the phone Incl. MIDP, JSRs, Multimedia Features & Bluetooth About Information & Weblinks

273

274

III. Calibration Settings

ACEnhancer is a subclass of ACSensor that outputs ACUnits as well. The Unit-conversion is not fully implemented yet! Calibration If you press the softbutton Calibrate you can select two options: 1. Calibrate: gives you 4 seconds to dene a minimum and maximum value operating thecurrently selected sensor. After these 4 seconds, the values are scaled between these max and mins. Thought the saving and restoring of these calibrations are prepared in the rmware, theyre not yet fully implemented in ACNexus. That means this setting is volatile, once the ACNexusBT is switched off. 2. Reset: resets previously occured calibrations for the selected sensor. ACSensorizer Screen Menu -Invert -Release Detect Pressing the Clear-Button C results in an immediate reset of all parameters to defaultvalues (eg., scale from 0 to 127...).Additionally, the sensors calibration is being reset.

-Harmony (Base / Scale) -Beats per Minute - Note Length Value -Enhancement Method -Samplesize Restrict (Ignore Below / Above) Scale (From / To) -Invert -Release Detect -MIDI Channel -Program (Instrument) - Channel Volume Numeric Keys 1,2,3,4,5,6 : Select Sensor (& toggle on/off if selected) 7: Reserved for further use (could have debug settings)8: Panic! (ACSensorizer), Start/Pause Sensorizer (ACPlotter)9: Scale value to t screen maximum 0 : Toggle Screen Info */# : Previous/Next Type (Units & Color)

Special Keys Back: Return to Main Menu Button 1: Return to Main Menu Button 2: Calibrate selected sensor C (Clear): Reset (! Resets **ALL** customized settings for this sensor immediately !)

275

IV. Enhancement Types

If you open up ACSensorizer or ACPlotter you are being asked if you want to connect to the ACNexusBT Bluetooth device: NO:the application goes into DEMO-mode, where all values are randomly generated.YES:the application connects to the default BT-ID Hardware Address

You can get the default BT-ID Hardware Address if you open ACConsole, select the ACNexusBT ID:-Field and choose More -> Search. Be patient, it sometimes takes up to1 minute until the search process is being shown. If completed and devices found, pleasechoose a device out of the shown list. The Hardware-ID of the selected device will be stored with your patch (and autorestored next launch).

The Images in the top row show Raw / Average / Smooth / Relative,the second row shows all Drop Enhancement with different parameters If you are in DEMO-mode and wish to connect, open ACConsole, select the ACNexusBTID: - Field and choose More -> Connect.The connection indicator now shows weather the connection is okay or not.

276

V. Screenshots

Various Screenshots from different modes:

277

* Search new Arduino Bluetooth Device # Edit Arduino Bluetooth Hardware ID directly Connect(Softkey) Connect to Arduino BT with shown Hardware ID Exit (Softkey) Quit Application Clear (Softkey) Clear ScreenLEFT/UP remove last sensor RIGHT /DOWN add next sensor FIRE Auto-Calibrate (In Sensor-Mode ONLY!)

* show numeric info # toggle Sensor Mode / Enhancer Mode

@ Sensor Mode: 0 show all sensors 1 show only sensor #12 show only sensor #23show only sensor #34show only sensor #45show only sensor #56show only sensor #6

@ Enhancer Mode: 0 RAW (raw signal)1 AVG (Average, nice & smooth curve) 2 REL(Relative, only up or down)

Enhancer Modes can be set for all or for

AUTO-CALIBRATION: @ Sensor Mode: -if not already in sensorMode (S#0): select sensorMode (#) select sensor to calibrate (1-6) press FIRE -Now you have 4 seconds to create a minimum & maximumvalue. These values get stored and will be interpolated. Values below the minimum & above the maximum will be ignored. -Press the C-Button to clear all previously calibrations

278

(J2ME MOBILE APP) _0.2.2 J2ME Applications for the Mobile Phones Tested for SonyEricsson & Nokia Phones ACNEXUSMOBILE System Requirements -CLDC 1.1 / MIDP 2.0 -Bluetooth & JSR-082 -MMAPI & JSR-135 Version 1.2 ( for ACSensorizer module! note that the N80 only has version 1.1 !)

Description: This is the main software that runs on phones! It is able to connect to ACNexus devices and receive sensor values. ACNexusMobile has currently two main modes: a Graph Plotter (ACPlotter) and a musical representation (ACSensorizer). The sensors can be congured and calibrated in both modes. ACConsole is there to search for new devices and transmit various debug strings.

Installation: Send ACNexus.jar to your mobile phone and install. Some phones require addionally the .jad le

ACNexus Suite v0.2 Quickstart Overview Michael Markert http://www.audiocommander.de 2009-03-17279

ACNEXUSQC (QUARTZCOMPOSER PLUGIN) _0.4.0

Quartz Composer Plugins incl. Examples (signal & video)

System Requirements: - Mac OS X 10.5.x - Installed Developer Tools

Description: QuartzComposer is a realtime OpenGL application, a so called "visual programming environment". Without the need to program texts, dierent objects ("patches") can be connected by wires.ACNexusQC is a custom developed plugin ("patch") that enables the communication with an ACNexus device (ArduinoBT) and makes it as simple to use as for example the Mouse-Coordinate Source-Patch. If installed properly, a new Plugin "ACNexusQC" can be dragged from the Patch Creator (Menu: Edit > Show Patch Creator / or press RETURN). The general aim for this plugin is that it can read videos from a folder on the computer and manipulate these videos by lters and eects which are in return controlled by the values received from the ACNexus sensors. The output is a realtime video that can be projected. Outlook for later versions: When this runs ona Mac OS X Server, the generated video can be transcoded to a rtsp-webstream and broadcasted to the web, which can then be received by phones that support this streaming video protocol. The broadcast will be near-realtime only.

Installation: Install & Run these plugins as described in readme.txt: - ACNexusQC.plugin -ACProcessorSuite.plugin (contains ACSignalEnhancer)

"Examples/ACNexusBT_06.qtz" Shows the six sensor values as bars (and numeric values) "Examples/ACNexusBT_Video_01.qtz" playback of the video "27102007211.mp4" while it's hue and exposure are controlled by the sensor values of the connected ACNexusBT device "Examples/ACNexusBT_Video_02.qtz" like the above example, but a variation and other (more realistic video). Installation and Usage is a bit complex, see readme.txt and various pages with video-tutorials introducing QuartzComposer. Once the plugins are installed successfully, you can check out the included Examples by

simple double-clicking: 281

ACNexus Suite v0.2 Quickstart Overview Michael Markert http://www.audiocommander.de 2009-03-17

ACNEXUSX (MAC OS MIDI APP) _0.1.0

Mac OS X Stand-Alone Application that connects to ACNexusBT devices and transmits the sensor values as MIDI (CH1, CC#20-#25) v0.1 lacks support of an Enhancement module and MIDI congurations settings (planned for v0.2)

Description: ACNexusX is a Mac OS X 10.5. stand-alone application. Once opened, it can communicate with any ACNexusBT device and receive the current sensor data. This data is then converted to MIDI-Messages and sent to the selected out port. Any application on the Mac (or on the network) or connected MIDI devices can then receive these MIDI messages and interpret them, that means: use these values to manipulate and control something (normally music or video). The general usage for this program is to enable inter-application communication of the sensor values. For example: the MIDI signals can be received by any VJ application that supports MIDI. It is not required to write a plugin for these applications, because nearly any VJ-app takes MIDI controller messages.

Installation: Copy ACNexusX to the /Applications folder and double-click. To actually see anything, a correct MIDIsetup is required! In other words: if no application or synthesizer or VJ-tool is actually receiving the MIDI messages, the sent values are simply invisible!

282

A.2.10b/ QuartzComposer patch guide

Overview
Summary
This document describes the function, installation and use of the Quartz Compositions created for Project Mindtouch. The compositions have been tested on a Mac Power PC running OS 10.5 and on a Intel Mac running OS 10.6. The compositions run on Quartz Composer version 3.1 and up.

Composition Name MoviePlayer_and_publish_body_se nsor_ data_using_OSC.qtz

Description This composition communicates with the Arduino using Bluetooth, plays back Quicktime movies ltered using the sensor readings and publishes the sensor readings to a web server using open sound control (OSC) This composition recreates the movie performance at a remote computer by downloading the sensor data in an XML le from a web server and using the data to apply the same visual treatments to the Quicktime movies.

MovieLoader_using_XML_feed.qtz

Camille BAKER, PhD thesis final September 2010

SMARTlab, CITE- University of East London

General Notes
Note: the directory scanner patch used in the plug-in only works with Quicktime movies ending with the .mov extension.

284

MoviePlayer_and_publish_body_sensor_data_using_OSC.qtz

http://sourceforge.net/projects/acnexus/ The download will contain instructions on how to install the plugin. It is possible to test the video composition by simulating sensors using random data. To do so, activate the Random @ Ofine setting checkbox in the compositions patch parameters. This and other settings in the patch parameters are described below.

Show Numeric Info when activated, this setting displays the raw sensor data, or the randomly generated test data. Random @ Ofine when activated, random numbers are generated to simulate sensor data. This should be switched off for performance. Quartz Compositions for Project Mindtouch

285

OSC
Scan Signal tells the composition to scan given directories for movie les; you should not need to change this. OSC stands for Open Sound Control, a protocol used for conveying data. OSC is being used to send the body sensor data from the local composition to a web server. This is shown in the diagram near the beginning of this document. The OSC data is picked up by an application called OSC2HTML which takes the sensor feeds and uploads them to a web-site. Location 1 to Location 4 tells the composer where to search for Quicktime les on the local machine. If multiple Quicktime les are in a directory, the composition will play through all of them. Important note: the le path needs to end with a forward slash /. The OSC patches setting should not need to be changed. Width and Height - this controls the size of the nal video window after compositing and applying effects.

The sensor data gets published a 4 oats with the OSC format of /sensor{sensor number} {value}. The OSC2HTML application expects the data on port 7003. Shufe when activated, the composition will play the movies in each folder at random. Show random debug - this shows the random order in which movies will be played.

ACNexus BT patch
The current set-up supports four body sensor datas. In theory, it appears that up to six sensors can be supported.

286

MovieLoader_using_XML_feed.qtz Image Processing Effects in Quartz Composer

The following screen grab shows the part of the composition that applies data from the four body sensors to the Quicktime movies.

Patch Parameters

]]

The image processing system applies different brightness, color saturation, and Gaussian blurs to the Quicktime movies according to the Arduino sensor data, then overlays all four movies to a single screen.

287

The compositions are applying real-time effects while 4 Quicktime movies are being played and composited, and this requires signicant processing power. Make sure all unnecessary applications are closed while running composition.

Troubleshooting
Movie playback is slow Scan Signal tells the composition to scan given directories for movie les; you should not need to change this. Location 1 to Location 4 tells the composer where to search for Quicktime les on the local machine. If multiple me Quicktime les are in a directory, the composition will play through all of them. Important note: the le path needs to end with a forward slash /. Width and Height - this controls the size of the nal video window after compositing and applying effects. Random Shufe when activated, the composition will play the movies in each folder at random. Show random debug - this shows the random order in which movies will be played. Show Numeric Info when activated, this setting displays the raw sensor data, or the randomly generated test data. XML Location - this is the location where the XML containing sensor readings are downloaded from. Period - this controls how frequently the composition checks for new data from the server. The lower the period, the more frequent the updates are. Sensor data is not being received by the server Check that the ACNexus patches are installed. Check that Bluetooth is enabled on the performance computer.

No sensor data coming through and compositions is reporting errors Check the app OSC2HTML is running. Check the internet connection.

Sensor data doesnt appear to match performance

Make sure that the Random@Ofine option in the Patch Preferences is deactivated.

No Movies Playing, Only blue screen showing Make sure that directories are properly formatted and notated in the Location 1-4 text boxes. Make sure that Quicktime movies are used, and that they have an .mov extension.

288

289

A.2.10c/ XML solution for webserver

Evan's Mindtouch Guide

Overview:
This document describes how to use the OSC app and webserver (see diagram) to connect the two Quartz Composer sketches, one running on the performance laptop and one running on the server.

OSC2HTML.app

About:

This application, running on OS X 10.5 and above, receives messages from the performance laptop's Quartz Composer composition and sends them to the webserver, where they are stored in a MySQL database and picked up later by the Quartz Composer composition running on the server.

To Run:
On the performance laptop, simply double-click the application to run it anytime before you run the Quartz Composer composition on the performance laptop. When finished, quit the app. There is no visual feedback.

Technical Details:
The app expects OSC data on port 7003 of the form: /sensor[INDEX] [float value] For example, to send some data (3.1415) to the app from the 1 sensor: /sensor1 3.1415
st

Note: Sensor indices start at 0.

Web Server
The Quartz Composer sketch running on the server will automatically retrieve data from the webserver. Note: Data is never erased from the webserver database, and will need to be periodically cleaned out, or exported and saved for future use. Technical Details:

290

To retrieve data from the database via a web link is easy: http://swampgirl67.net/mtxml/putSensorData.php?p=m1m1nd&mode=display This will retrieve an XML file with the last 20 sensor data entries in it. All other information can be found in the Php source code for the single file that handles the saving and display of data: PutSensorData.php

<?php /***** db fields: +--------+--------------+------+-----+-------------------+---------------+ | Field | Type | Null | Key | Default | Extra | +--------+-------------+------+-----+-------------------+----------------+ | id | int(11) | NO | PRI | NULL | auto_increment | | | varchar(255) | YES | | NULL | | +--------+--------------+------+-----+-------------------+----------------+ data ******/ | | timestamp | NO | | CURRENT_TIMESTAMP | | time // test password! if (isset($_REQUEST["p"]) && $_REQUEST["p"] == "m1m1nd") {
$dsn = array( 'phptype' => 'mysqli', 'username' => 'swamp_mtouchw', 'password' => 'm1ndxna', 'hostspec' => 'localhost', 'database' => 'swamp_mindtouch',

); $db = mysql_connect($dsn["hostspec"], $dsn["username"], $dsn["password"] ); if (!$db) { die(mysql_error()); } $db_selected = mysql_select_db ($dsn["database"], $db); if (!$db_selected) { die ('Can\'t use ' . $dsn['database'] . ': ' . mysql_error()); }

// set default mode for page $mode = "save"; //display mode if (isset($_REQUEST["mode"])) { $mode = strip_tags($_REQUEST["mode"]); }

// do we display results? if ($mode == "display") { echo '<?xml version="1.0" encoding="ISO-8859-1"?>';

291

echo "\n";

$limit = 20; $result =& mysql_query ('SELECT * FROM `sensordata` ORDER BY `time` DESC LIMIT 0,' . $limit, $db); if (!$result) { die('Could not query:' . mysql_error()); } /* while ($obj =& $result>fetchRow (DB_FETCHMODE_OBJECT)) printf ("%s, %s\n", $obj->name, $obj->category); */ /* print("<p>SAVED MSGS:</p> <p>\n"); while ($row =& mysql_fetch_array ($result)) { printf ("id: %s<br> data: %s<br> time: %s<br>", $row["id"], $row["data"], $row["time"]); print("\n</p>\n"); } */

print("<sensordata>\n"); while ($row =& mysql_fetch_array ($result)) { printf (" <entry id=\"%s\" time=\"%s\"> <data>%s</data> </entry>", $row["id"], $row["time"], $row["data"]); }

print("</sensordata>\n"); mysql_close($db); } else {

/* $_POST["d"] // data $_POST["ts"] // timestamp */

$data = strip_tags($_REQUEST["data"]); $ts = strip_tags($_REQUEST["ts"]); $result =& mysql_query ("INSERT INTO sensordata (time, data) VALUES ('$ts', '$data')", $db); if (!$result) { die("error inserting: " . mysql_error()); } mysql_close($db); // create timestamp, insert into table msgs in db txtmsgs (text, #, timestamp) // create other page to display last 10 txts } } ?>

292

Appendix III/ performance event materials

293

A.3.1/ streaming server set-up and operational notes Logging into the server username admin password SMARTlab (note capitalisation)
Services Running (already setup) Web Quick Time Streaming Web Publishing to put a document in the right place to be served via the web server - use the alias on the desktop (Documents for Web Server) on the desktop - drop the file on this and it will be served immediately Quicktime Publishing to put a document in the right place to be served via the web server - use the alias on the desktop (Documents for Quicktime Server) on the desktop - drop the file on this and it will be served immediately IP Address At this research phase, the server does not have a name or fixed IP address - to find the IP address, click System Preferences on the Dock, select Network, Select Ethernet 2 and the IP address will be shown in the pane on the right To check a file is being served via the web server Go to Safari (on a Mac, especially if you are serving QC patches) type http://<IP Address>/filename so if your IP address is 161.76.254.33 and the file is quartz.qtz the URL would be http://161.76.254.33/solar.qtz To check a file is being served via the Quick Time Streaming server (not live) Go to Quicktime Player - Click File > Open URL type rtsp://<IP Address>/filename so if your IP address is 161.76.254.33 and the file is sample_300kbit.mov the URL would be rtsp://161.76.254.33/sample_300kbit.mov Quartz Composer Notes If your patch requires access to any audio/video/Bluetooth, then it will not work via a web browser. Podcast Producer, please note: The Podcast Producer service is not currently viable for the MINDtouch project as it does not offer any service beyond that already supplied by the streaming server and web server. Quicktime Streaming Server Set up as service there is no need to change this. Quicktime Broadcaster - Live stream

294

To stream the contents of the web cam of the server (or other live streams from cameras) - click on Quicktime Broadcaster in the dock - rather than go through all the options - if you want to broadcast the isight camera live, check the monitor (on the left) is showing the output from the isight - click broadcast - this can then be seen on any Mac in the lab via QuickTime player Click File> Open URL type rtsp://<IP Address>/untitled2.sdp this is currently a 25 frames per second 500kbit/sec stream click stop at the end or the broadcast will continue for ever ! Backup The main hard drive of the machine is backed up to one of the internal drives - the other drives aren't at present (but this is made clear in the name of the disc)

295

A.3.2/ technical workplan (December 2009): Evan : - set up MySQL on Camille's (hosted) server yesterday (to store the sensor data for a php script to retrieve), the database is swamp_MINDtouch, username is mtouchw and pass is m1ndxna. 1) Write a Php script that runs on Apache on your hosted server called "handleSensorData.php" that will receive an XML string as input (sensor data, in a format to be worked out between Pawel and Manjit) and store it in a MySQL database (he will have to work out how, such as what database tables to use. I'd suggest unique id, timestamp, sensor name, sensor data) - to make a brief specification for me on the data format from the Quartz Composition that gets uploaded to the web server for SOAP for Manjit. 2) Write another script (Php on Apache) called something like "retrieveSensorData.php" to retrieve it in XML form for Quartz Composer (Manjit) to handle. 3) Then, there needs to be an application running on the streaming server (or a script, regularly triggered by something such as Quartz or a scheduled Cron job) that rips the Qik movies from their website and converts them to a Quicktime format that Quartz can readily use. There are commandline-tools to do this that can be scripted, such as ffmpeg.

Manjit : Modifying the send / receive patches in QC to handle the translate the sensor data into XML code: 1) using a SOAP patch in the quartz composition that runs on Camille's laptop. This sends data to the web server. He needs a brief from on what the XML looks like. 2) I will be changing the quartz composition that resides on the server to handle XML via RSS. This is with respect to sending and receiving the body sensor data.

296

A.3.3/ performance preparation checklist - contact external participants with task list - create remote "task activity list + promo - contact past participants to watch and participate - put software (QC patches and phone) and videos on appropriate machines - Instructions on phone software out to participants to load - charge phones --N80, N93i - find a documenter - print consent forms - post-event surveys printed - have extra batteries available - have script created (PHP) for video pulling - put new videos in folders - make .movs

297

A.3.4/ performance preparation outline

Tech checklist phones etc Testing procedures: o o Computers : server and laptop / software and networks on both Garment electronics / batteries etc

Venue (make a timeline of a day of performance) Networks Ethernet and network set up general connection and streaming server set up and configurations People o o o o o Questionnaires and consent forms Promotion: Info and invites on Facebook and my own website (get screen shots) Prepping remote group Documentation person Prepping Guides

The Event Process Double-check technology Introductions to guests Working with Guides / guides working with participants Dressing the person/ people in the electronic garment Sending video group off Post activity discussion and / interviews Questionnaires Journaling (my observations) Collection of phones and media Post Event Send notes of appreciation for participation via email View of media and documentation Journaling Post mortem Review of what worked and what did not

Reading of the questionnaire

298

A.3.5/ event technical preparation

1) taking biofeedback data collected by the phones previously and video mix them using custom software on a local computer in Quartz Composer's programming environment and send it out to the web server for others to download via web or WiFi or WLan; 2) to have a web vide streaming server set up to use RTSP (Real Time Streaming Protocol) to serve the live and recorded video from the web, then back to the phone web browser; 3) embed the RTSP streams into a website that is a) be viewable / displayed and interacted with or selected them on a phone (using qik or livecast application) and; b) be able to select videos inside QC, mix them and send them back to web/ then to the phones.
video documentation

The digital documentation and journaling includes an online and DVD presentation, intended to give readers various, accessible ways to understand the project process. It will include a website overview of the thesis research and ongoing processes of: conducting research, sourcing and costing sensors and phone equipment used in the research, other tools and people for the development of the media, events and technologies explored and software created; on-going audio/video, visual and textual documentation; meditation and movement research experiments, practices, processes and experiences; Initial planning concepts, sketches and images of the initial building steps; programming and exploration of the phones with the biofeedback sensors, work-in-progress planning and staging of performance events and activities; and video and text transcriptions of interviews with media collection performers, Tai Chi practitioners, VJ mixing performers/ participants and event audiences. http://www.smartlab.uk.com/2projects/MINDtouch.htm http://www.youtube.com/v/2qUSp1yl7T4&color1=0x6699&color2=0x54abd6&hl=en&feature=play er_embedded&fs=1 There has already been an exhibit of the mobile video edit taken from workshops and performances from the research project, as abstract mobile video art work on the phones they were created on installed in a gallery, which was shown in London, April 4, 2008 FILMobile. These videos can stand alone as artwork / output from the project on its own, as apart of the practice-based performance media research for the PhD. http://www.swampgirl67.net/lmobile.html

299

A.3.6/ additional logs of facebook, website and online interaction

300

301

302

303

304

305

306

307

308

309

310

A.3.7/ first performance: introductions script

This is the first of a series of experimental participatory performances and it will be mostly wobbly, because much of the key networking/ streaming are just getting to work in the last few days and not enough time to test much to find any issues yet. There were to be two identical garments to house the biofeedback sensors however we have to two different versions instead. So all the pieces of the complex puzzle Ive constructed are having their maiden voyage here today with you :) I want to acknowledge and thank profusely, all the great people who have been working on this with or assisting me, mostly for free or for my adoration alone - in no particular order: 4MGroup and Rama Prparim for hosting my server, Fashion Designers : Rachel Lesebikan, Tara Mooney, Networking and Quartz Composer Specialists: Manjit Bedi, and Huw Williams, the great Michael Market: electronics and mobile programmer, and others over the last two years previous programmers: Li Zhang and Jeremi Sudol, as well as at one point Clilly, not to mention the support and guidances of my supervisors Lizbeth, Susan Kozel, and Marc Price. And tonight I thank my lovely host and guide: Chrissie Poulter and Trinity Leeds. So before we get started I will say that will put you into 2 groups of active participants and one remote online group for this iteration - no is allowed to be totally inactive or passive throughout. Only two people at a time can use the biofeedback sensors sets within the gorgeous garments that Tara and Rachel have created. Please be gentle with them!!!! One group will stream / record via qik during their guided activities with me - these videos will be used in a live interaction with everything tonight. Anyone without the qik software can video or pictorially document it for me. We might need an observer group to be involved with at least one or two of these groups (Im hoping people might share phones if not enough - I have one extra one) and maybe all will get to be observers as well. The observer group is asked to just take notes and not analyse - be recorders not critics -- not just of what you see but what you experience as well. ONE RULE: NO TALKING at all throughout -- laughing and giggling is permitted and only the guides voices and maybe mine will be allowed - this is meant to explore using technology to assist in non-verbal and embodied ways of communicating, so try use the tools and options available to you here other than your words and voice. The video group will be coming in and out but everyone else stays in the room please and video group try be quiet coming in and out. One other note: This is intended to be fun and playful but also reflective, expressive and exploratory. Yet no one is expected to be a professional performer, even if some in the room are -- this is meant to be for the public, for anyone and all - so please have no fear and try not to self-exclude if you have no performance experience - that's the whole point / idea for it to be for non-performers. And Id like you to all stick around for ten minutes after to fill in a short post-performance survey afterward, but to save your tech or other project question for my process/ practice presentation tomorrow.

311

A.3.8/ performance mind-quieting script [in a soft voice]: Find a place in the space - standing or sitting. Close your eyes and be very still. Then, take several deep breaths, be silent, and settle into your body. Begin to pay attention to your body and the images or colours behind your eyes. Go inside your body and listen to/feel the sensations, such as: your breath, your heart beat, areas of heat or coolness inside your body, electrical energy, your thoughts or brainwaves, your sensations, the movement of muscles in your organs or limbs, or your blood flow. [pause] Focus on one area of your body or sensation that draws your attention now. [pause] Keep focusing your attention on that part of the body or sensation. [pause] Experience the inner sensations for a few breaths. [pause] Feel the blood flow, the expansion and contraction of your breath in the body, and any other sensations or perceptions like stomach gurgling or tickling toes, heat or cold, or any physical manifestations of emotions or thoughts.... [pause] Pay attention to any images or thoughts or emotions you are experiencing with these sensations or perceptions [pause] Think of how you might visualise these sensations or perceptions, thoughts or emotions [pause] Hold these mental images in your minds' eye now linger with them.

312

[pause] Now think how you might express them to share them with others near or far --non-verbally but visually? [pause] When you are ready, open your eyes ....and try to take these sensations or images or emotions and experiences with you and now and use the camera/video phones to express them, starting very close to your own body or space around you on the floor. Do this for 5 mins.

313

A.3.9/ group guiding information

Performance Guide Info - Group 1 (biofeedback garment)

Participatory Performance Aspects There will be three different groups/ activities in a social / game-like environment / participatory performance - so designing the staging the event and the logistics are currently in development. Guides need to prepare themselves physically, aesthetically, and emotionally to be a nurturing, empathetic and aware of participants and create a ritualized, performative approach when working with participants. As guides, we need to keep our intention on creating a social, intimate, safe and playful experience for participants. Guides will assist a small group of participants through the activities: group 1 - Activity Overview Participants will be wearing the biofeedback sensors for this activity and guided through various movement and emotional/ sensory activities = playful both calming and vigorous or "stressful" - to activate the sensors in various ways - which trigger video from the database and the effected video is displayed on the big screens around the room; Your job is to help the two (until we have more sensors sets) participants put on the garment: 1) invite a couple or people in the group to put the garment - in a playful way; 2) explain to them that they will not be asked to do anything they are not comfortable to do and they can stop at anytime and take the garment off; 3) help them put on the garment; 4) connect the battery to the Arduino (make sure they are fresh batteries); 5) start the same as the video group -- ask them to close their eyes and do a short guided imagery / mental visualisation - 5 mins max 6) guide them through the telepathic activity - sending and receiving - 5 mins -- each (telepathic activity is: partners sitting back-to-back, try to telepathically send an image or sensation in their mind to one the partner, each afterward can guess what the other person sent, and share what was sent were the similar to what was received) Paper and pens

314

7) guide participants to do some tensing and releasing of muscles on the spot - 2-3 mins max 8) then guide them to do a slow walking / movement activity individually (see the Augusto Boal book) - 2-3 mins max 9) ask them then to do a weight sharing activity back to back (see Laban exercise) - 5 mins max 10) guide them through faster movement (maybe some running or jumping or something more strenuous) through the space -- weaving through the other participants, interacting with others in the space if they want as long as they don't stop too long, don't speak and then keep moving. 3 mins max MINDtouch: Embodied Transference/ Transcendence mobile performance event July 2009

315

Performance Guide Info - Group 2 (no tech)

Participatory Performance Aspects There will be three different groups/ activities in a social / game-like environment / participatory performance - so designing the staging the event and the logistics are currently in development. Guides need to prepare themselves physically, aesthetically, and emotionally to be a nurturing, empathetic and aware of participants and create a ritualized, performative approach when working with participants. As guides, we need to keep our intention on creating a social, intimate, safe and playful experience for participants. Guides will assist a small group of participants through the activities:

group 2 - Activity Overview Participants will be doing something similar to group 1+ 3, but without any technology and paying more attention to the screen imagery. Your job is to: Start the same as the video group-- ask them to close their eyes and do a short guided imagery / mental visualisation (5 mins max) This activity will be broken down into guided 4 sections of short explorations: 1) close range movement slow walking / movement, expressions/ gestures of internal sensation/ feelings or body itself, etc. (3 mins max) 2) in partners, participants practice expressing themselves non-verbally / gesturally: they are asked to introduce themselves non-verbally without speaking - using body language and gestures -- no speaking allowed !!! (5 mins max); 3) collaboration of experiences: partners sitting back-to-back, try to telepathically send an image or sensation in their mind to one the partner, then guess what they saw in their mind and was sent from the other taking turns (3 mins each person). Afterward each can tell what the other person sent, sharing their mental images with each other to see if the exchange was successful (they can talk in this part) 4) Observation + interaction exercise : guide them to observe the videos mix and bodydata visualization made by the biosensor group and then try to NON-VERBALLY find ways to interact or send mental messages to those participants (non-verbally) to see if it changes the data outputs from them (5 + mins) -- guide them to move freely around the space [Tell them: see script at the end of group 3 ideas]

316

MINDtouch: Embodied Transference/ Transcendence mobile performance event July 2009

317

Performance Guide Info - Group 3 (video capture)

Participatory Performance Aspects There will be three different groups/ activities in a social / game-like environment / participatory performance - so designing the staging the event and the logistics are currently in development. Guides need to prepare themselves physically, aesthetically, and emotionally to be a nurturing, empathetic and aware of participants and create a ritualized, performative approach when working with participants. As guides, we need to keep our intention on creating a social, intimate, safe and playful experience for participants. Guides will assist a small group of participants through the activities: group 3 - Activity Overview Start the same as the other groups -- ask them to close their eyes and do a short guided imagery / mental visualisation (5 mins max) This activity will be broken down into guided 4 sections of short explorations: 1) close range movement expressions/ explorations of internal sensation/ feelings or body itself, etc. (5 mins max) 2) then go further: expressions/ explorations of external experience or what they see outside of themselves and their own body; 3) in partners, participants explore exchanging with each other and the themes below, exploring body sensations, internal feelings (keep all non-verbal / visual); external elements of the body and surrounding it. exercise : participants are asked to introduce themselves non-verbally using the videophone without speaking -- no speaking allowed -- one example: recording parts of themselves to show each other 4) collaboration of experiences make something together 5 minutes each activity (time it) to prepare and make. Options (choose one per group) exercise 1: participants are asked to work with their partner, also non-verbally, to create something to send to someone else not in the room, telepathically or through the mobile phone, something that would express their combined inner perceptions and experiences. exercise 2: partners sitting back-to-back, try to telepathically send an image or sensation in their mind to one the partner, then attempt to visually capture what they saw and sent from their mind

318

- taking turns, 3mins each. Each afterward can guess what the other person sent, and share their videos with each other to see if all three versions (the telepathic, the videophone and the guess) were the similar. ____________
Tell them: Consider yourself a performer not just a documenting observer in this event. Try to use a different sense of seeing, reading through your own body in order to discern and select moments to capture but try maintain enough outside perspective to be aware of the world while you explore this sensory tool. Be sensitive and aware of yourself, others and the world. [see workshop script for the creative framing suggestions]

MINDtouch: Embodied Transference/ Transcendence mobile performance event July 2009

319

A.3.10/ performance event participant consent form

The SMARTlab Digital Media Institute Practice-based PhD Programme, UNIVERSITY OF EAST LONDON Consent Form - Confidential data I understand that my participation in this project will involve being observed in a participant-centred event, conducted by Camille Baker as investigator, who will record the event on video, and ask me to fill in a survey afterward, to determine the effectiveness of participatory performance approach, as a part of a larger research project on mobile media performance and the experience on presence on the network as part of the practice-based PhD. I understand that participation in this event is entirely voluntary and that I can withdraw from the event at any time without giving a reason. I am free to withdraw my participation at any time or to discuss my concerns with the event leader or her supervisor. I understand that I am free to ask any questions at any time. I am 19 years of age or older. I understand that at the end of the event I will be provided with additional information and feedback about the purpose of the research project, should I request it. I also understand that the activities during this event may involve minimal physical and personal risks to me, such as tripping or falling- depending on how active I choose to be in the event. I shall undertake to move safely about the space, with this in mind. By involvement in this event, I understand that I am giving my informed consent for information I provide therein to be included for my video contribution to be broadcast worldwide, and for the outcomes of the be published widely, and in future studies. I understand that the information provided by me will be held confidentially, such that only the Investigator (event leader) can trace this information back to me individually. The information will be retained for up to 10 years, when it will be deleted/destroyed. I understand that I can ask for the information I provide to be deleted/destroyed at any time and, in accordance with the Data Protection Act, I can have access to the information at any time. I also understand that I may register any complaint with University Of East London, University Research Ethics Committee. I, ___________________________________(NAME) consent to participate in the event conducted by Camille Baker, SMARTlab Digital Media Institute PhD Programme, UNIVERSITY OF EAST LONDON, under the supervision of Professor Lizbeth Goodman. Signed: Date: Further information can be obtained from Research Ethics Committee, University Of East London, Docklands Campus, 4-6 University Way, London, UK, E16 2RD, or contact Debbie Dada at +144208 223 2976 or d.dada@uel.ac.uk. I may obtain copies of the results of this event, upon its completion by contacting: Camille Baker Flat 14, 58 West End Lane, West Hampstead, London England NW6 2NE UK

+44(0) 78 4768 1127

320

A.3.11/ post performance audience survey

Please tell us about yourself! Answer the following questions as honestly and clearly as possible, in order to help us understand your experience better. Information gathered from this survey will assist in planning future events. Your responses will remain confidential, and will be used for statistical purposes only; all questions are optional of course.

Demographic Information: Age: Highest Level of Education: Academic Major/Stream: Annual Household Income: Gender: Ethnicity/Nationality: Occupation: City of Residence:

Event Experience Feedback: What was the most intriguing part of the event / activity? Why? What was the least intriguing part of the event / activity? Why?

What aspect of this event / activity do you think worked well?

What aspect of this event / activity do you think could be improved and how?

How is the experience of this type of performance different than a more conventional performance (stage dance, music, theatre, etc)?

Did you feel the same sense of presence or interaction between performer and audience in this event / activity, as you might in a conventional performance event? If so, how?

How did you feel about being a performer, rather than an audience?

Other comments about the event / activity, as a whole?

321

Experience Inventory (optional): answer as briefly or extensively as you like. If/when you were making your video. . . - How did you perceive your bodys sensations, perceptions, thoughts and other internal experiences during the activities tonight?

- How did you feel about expressing yourself and sharing these sensations and experiences with others through these activities tonight?

- What did you think about participating in these activities using mobile devices?

- How did/ do you feel about sharing these media expressions and experiences in the larger live video installation?

- Other experience comments:

Thank you for your participation!

322

A.3.12/ event participant experience feedback themes

What was your favourite part of the event tonight? Why? (themes) participating relaxing being creative watching others moving making images What was your least favourite part of the event tonight? Why? (themes) didn't understand (2) moving (2) expressing / filming self - self-conscious (3) being observed by others (3) What aspect(s) of this event do you think worked well? (themes) guiding / facilitation / organisation (3) staging / set up / space (2) relaxation / meditation (4) exercises (different ones for different responses) (6) making videos / visuals (6)

What aspect of this event do you think could be improved and how? (themes) more explanation (5) want to see the project results (5) more info on the body sensors and what is happening with the body data (2)

How is the experience of this type of performance different than a more conventional performance (stage dance, music, theatre, etc)? (themes) participative / participatory (7) more interactive (3)

323

 more personal (3) more improvised (3) Did you feel the same sense of presence or interaction between performer audience in this event as you might in a conventional performance event? And How? (themes) * didn't perceive difference between performer and audience (4) not a performance (2) more confident / less self-conscious (3) *most misunderstood the question as intended, since the networked aspect wasn't working fully nor explicit. people didn't connect with the performance activities as performance or that there were remote participants or audience > they just experienced themselves and others in the space, involved in the activities in a personal, intimate and individual sense. How did you feel about being a performer rather than an audience? (themes) yes/ ok/ happily/ enjoyed (even if not usually performer) (16) unsure (2) uncomfortable (4) didn't realise or feel it was performing/ they were a performer (4)

Experience Inventory (optional): answer as briefly or extensively as you like. How did you perceive your bodys sensations, perceptions, thoughts and other internal experiences during the activities tonight? senses noted(themes): bodily - pulse, spiky crampy, lungs, head, heated, tired, ill health mood or mental - calm, focussed, racing thoughts, abstract, positive, heightened awareness

How did you feel about expressing yourself and sharing these sensations and experiences with others through these activities tonight? (themes) good/ fine/ enjoyed (7) nervous / uncomfortable (4) other = neutral, low-key, safe (3)

What did you think about participating in performance activities using mobile or wearable electronic

324

devices? (themes) fun/ cool/ enjoyed it (7) interesting / new (3) no problem but... (3)

How did/ do you feel about sharing these media expressions and performative experiences in the larger performance installation ? (themes) fine/ sure/ good (8) unsure (2) need more info / or to see results first (3) Other comments about the event as a whole? Other experience comments: (themes) good / enjoyed it / great work / interesting (12) problems or focus on gadgets (2) interested in the screens, video, media, or live feeds (5)

325

A.3.13/ performance survey feedback

326

327

328

329

330

331

332

333

Appendix IV/ collaborator and technical feedback

334

A.4.1/ testimonials on network issues

335

A.4.2/ screenshots of forum discussions on Nokia Realplayer issues

336

337

Appendix V/ DVD Support Materials: Table of Contents

338

A.5/ Notes
There are five media DVDs to support the research exegesis/ thesis: 1) Participant workshop video clips* 2) Workshop documentation videos and stills 3) Performance videos and final VJ mixes 4) Promotional materials 5) Technical materials

These media support materials were split into 5 parts due to the great amount of video and media generated from each phase of this project.

A.5.1/ Disc 1 - participant workshop video clips

This disc is the only one that does not have a designed interface and interactive menu presentation of the videos created by participants in workshops from 2007-2009. After many months and many attempts was unable to get the software to encode these mobile phone videos in any DVD creation software without crashing. This is due to the differing compression put on the video clips for each different phone model that is incompatible with the DVD Encoding protocols. However, videos can be viewed individually in Apples QuickTime video player software. N ote: that while there were many more participants in these workshops, the video here is all the was shared or useable. Disc Table of Contents Main Menu 1. Participant workshop videos i. June 2007 Vancouver a. Participant 1 b. Participant 2 c. Participant 3 d. Participant 4 e. Participant 5 ii. July 2007 Dublin a. Participant 1 b. Participant 2 c. Participant 3

339

d. Participant 4 e. Participant 5 iii. August 2007 Vancouver a. Participant 1 b. Participant 2 iv. October 2007 Dublin a. Participant 1 b. Participant 2 c. Participant 3 d. Participant 4 v. January 2008 London a. Participant 1 b. Participant 2 c. Participant 3 vi. November 2008 London - Dana Centre demonstration a. Participant 1 b. Participant 2 vii. March 2009 London a. Participant 1 b. Participant 2 c. Participant 3 d. Participant 4 e. Participant 5 f. Participant 6

A.5.2/ Disc 2 - Workshop documentation videos and stills

This disc and the remaining 3 discs were easily encoded with the original layout and interactive menus designed for all. Disc Table of Contents Main Menu 1. Workshop documentation video i. ii. Workshop documentation June 2007 Workshop participant interviews July 2007

iii. Workshop documentation August 2007 iv. Workshop participant interviews October 2007

340

v. Workshop participant interviews January 2008 2. Workshop images i. ii. Workshop still images Participant 1 workshop stills March 2008

iii. Participant 2 workshop stills March 2008

A.5.3/ Disc 3 - Performance videos and final VJ mixes

This disc and the remaining 2 discs were easily encoded with the original layout and interactive menus designed for all. Note: that while there were many more participants in these performances, the video here is all the was shared or useable. Disc Table of Contents Main Menu 1. Performance participant video i. July 2009 (first performance) Matrix, UEL campus, London a. Participant 1 b. Participant 2 c. Participant 3 d. Participant 4 ii. October 2009 Leeds Trinity College, Leeds a. Participant 1 b. Participant 2 iii. December 2009 4M Studios, London a. Participant 1 b. Participant 2 c. Participant 3 iv. December 18, 2009, Limkokwing University, London a. Participant 1 v. February 12, 2010 Latitudinal Cuisine Dinner, London a. Participant 1 2. Final performance mixed collages i. ii. Collage 1 Collage 2

341

iii. Collage 3 iv. Collage 4 v. Collage 5 3. First performance documentation video (July 2009)

A.5.4/ Disc 4 - Promotional materials

This disc and the remaining disc were easily encoded with the original layout and interactive menus designed for all. Disc Table of Contents Main Menu 1. Promotional video and posters i. ii. iii. iv. 2. i. ii. iii. iv. 3. i. ii. iii. 4. Promotional video 2007 Mindtouch Promotion 2008 (long version) SMARTlabPhD Plumbline 2009 Ars Electronica Video Application 2010 Filmobile edit: workshops July 07 Filmobile edit: workshops June 07 Filmobile edit: workshops August 07 Filmobile edit: workshops Oct 07 MobileFest08 - test VJ mix Art video edit from workshops 2008 (long) Art video edit from workshops 2008 (small)

FILMoble

MobileFest 2008

Promotional posters

342

A.5.5/ Disc 5 Technical materials

This disc was easily encoded with the original layout and interactive menus designed for it. Much of the technical development is documented in the main body of the thesis text. This disc just contains additions items relevant to the technical description. Disc Table of Contents Main Menu 1. Technical process i. ii. iii. Garment design and development a. a. a. Garment design image slideshow Biofeedback sensor testing video Electronics development image slideshow Biosensors and electronics development Software design

343

Appendix VI/ published papers relevant to the thesis

344

A.6.1/ Liveness and presence in bio-networked mobile media performance practices: emerging perspectives (Dec 2008)

The International Journal of Performance Arts and Digital Media, Volume 4 (2), Intellect Press

Liveness and presence in bio-networked mobile media performance practices: emerging perspectives
Camille Baker University of East London

Liveness and presence in bio-networked mobile media performance practices: emerging perspectives
Abstract If you could share and exchange your dream imagery, feelings and sensations with your friends and loved ones, how would you do it? If could not only share and exchange, but remix and collage them, what would it look like or feel like? How would it work? Explored through the new media performance project, MindTouch, uses biofeedback sensors and mobile media phones in live, staged events. MindTouch is a BBC R+D sponsored PHD research, to connect people remotely through the mobile phone, allowing them to re-engage with each other and with the world, affectively and expressively in new creative, non-verbal/non-textual ways through mobile video performance events. The aim of this article is to discuss current practice-based PhD research underway at the SMARTlab Digital Media Institutes UK base at the University of East London, under the direction of Professor Lizbeth Goodman. The MindTouch project involves the pursuit of a better understanding and facilitation of the individuals mode of expression in relation to the collective experiences of liveness and presence, as observable and capturable within the virtual space of a live mobile performance event. This article reflects on meta-concepts of this research and on past video collection workshops completed in the first phase of the project, including observations from these workshops, and the on-going practice, development and technical issues involved. It discusses practical processes and concerns (such as methods for encouraging people to connect remotely, that are allowed to re-engage each other and the world affectively using the mobile phone), both in and through participants bodies, and through a range of expressive, creative, non-verbal/nontextual means, through networked performance activities.

Keywords liveness and presence mobile media body-computer interfaces interactive performance practices

345

Introduction The aim of this meta-research project is to uncover new understandings of the sensations of liveness1 and presence that may emerge from the use of mobile technologies and wearable devices in performance contexts. A secondary exploration, funded by BBC R&D (2007-present), involves finding new ways to interface between the body/mind and technology. Through the use of biofeedback sensors on the bodies of performers and moving meditators, sending data to mobile phones, I intend to find meaningful ways to communicate non-verbally and through the body directly to technology. Critical to this work is to enable the perception and embodied sense of liveness and presence within the virtual, non-space of mobile networks. A visceral, sensory, emotional and philosophical understanding is key in this research as to whether or not liveness and presence are perceived by mobile performers, participants and audiences, in a networked performance event. Also of interest is whether this perception has changed for them or is experienced differently during such an event, compared to traditional performances. To explore these concepts a practice-based investigation has been undertaken, using paradigms of participatory performance and other live visual experiments and improvizations. This research set out initially to explore the body as interface and has both a focused and extended concern through practical testing, to look at the specific conditions of silent communication between people who cannot speak, or who choose not to speak. Thus, the findings consider new ways to simulate, emulate, or even facilitate, a non-verbal or telepathic connection and the sense of feltness, presence or liveness, co-presence or collaboration, within a mobile performance in this case through the added use of biofeedback sensors to increase the embodied interaction. Metaphorical underpinnings The desire to gain a visceral, sensory, emotional and philosophical understanding of how participant performers and audiences perceive liveness and presence1 within real-time, inperson and virtual domains, during mobile performance settings by participants is central. The impetus for much of my work are the inspirational sensations and imagery taken from my dreams, intended to be induced in others, helping them to experience a receptive state of creative consciousness to connect with others non-locally. The notion of non-verbal, embodied communication or telepathy, is often felt through dreams is integral to the metaphor that shapes and guides this research. Definitions of telepathy often describe it as presence at a distance; others call it thought communication.1 It is the experience of sending or receiving a feeling, sensation, experience, thought, idea, news or traumatic event from someone else (generally someone close or intimate to you), non-verbally or through ones mind, across distance, and often during dreams (Baker 2004).1 Consciousness Researcher, Dean Radin has done extensive research on telepathy and defines non-locality, as physical objects (or people) that may seem unrelated or separate but are connected, transcending the limitation of space and time

346

or taking place simultaneously, either through space and time, or in space or time (Radin 1997: 277). Mobile media phones are used in the MindTouch research as the medium or conduit for self-expression, especially to convey non-verbal experiences and sensations as an extension of the body/mind1 in both non-literal or abstracted ways, as well as more direct representations. Phones could be seen as the vehicle to express embodied consciousness with others, through movement, or creative processes between participants or with self. This embodied mixing exchange or interaction between the mobile users and the body of the person wearing the sensors can then be seen as a means to emulate the sharing of dream imagery or other non-verbal, pre-language, telepathic communication or conversation between the unspeaking body and the mobile phone users.1 It is surmised here that controlled telepathic interactions could begin at a sensory level, facilitated through dreams, below conscious awareness. Thus, simulation of telepathy is a key aspect of my performance exploration: employing a creative, technological way to help others imagine a telepathic exchange between two or more bodies who are lucid dreaming (LaBerge and Rheingold 1990) visually and physically, together. In doing so, current available technology is used to illustrate the concept that we can all do it one day, with or without technology. The intent is to represent a small-scale collaborative consciousness or presence, which can be accessed and remixed or re-performed into new collective or universal narratives of living, being, experiencing, emoting, expressing, feeling, performing, and communicating via mobile media performance. Project description From 2007-present SMARTlab was commissioned to undertake a project on somatics and the body as interface, now called MindTouch, in collaboration with Dr Marc Price of BBC R&D. Camille Baker is the researcher, Dr. Susan Kozel is an additional supervisor, and the technology team includes Michael Markert, Jeremi Sudol, Jana Riedel and Dr. Li Zhang. Performance collaborators at SMARTlab include: Bobby Byrne, James Brosnan and Chrissie Poulter. The project in progress brings together diverging areas of new media research and media art/performance practices, in terms of use of biofeedback sensors interfacing with mobile phone technologies in a unique way. Embodied, non-verbal communication and interaction is explored, using wearable biosensing devices as interfaces to mobile media experiences, within participatory performance events. In addition, a small, curated exhibition event is being produced for the Fall 2008, including works from other media performance artists, collecting responses on their experiences and those of their audiences, as well as similar performance media projects to date, for a contrasting sources of data, to compare and contrast them with my own project. The effectiveness of my own facilitation or enablement of sensing liveness and presence in these contexts is also studied.

347

Participants in the first phase video collection workshops are asked to share internal images, feelings, thoughts, and impressions using the phone in a non-verbal, visual way, using the mobile video recording cameras and online video editing tools such as www.eyespot.com, then share these with others. This is collection of personal clips is then to be used later in a collective collage/performance of this experience. During the video collection process of the project, participants are asked to explore their own: consciousness, dream states and relaxation; non-verbal emotional and affective senses and states; internal sensations and embodiment; inner feelings, pondering, internal conversations expressed through abstractions; reflections, either literal mirror images or more intangible manifestations; representations of experiences of their immediate surroundings; external (spatial), eternal (temporal) and psychic (non-tangible) worlds, as well as mundane routines versus profound realizations or epiphanies. The media performance aspect of the project involves staging various mobile media event(s) that utilize a database of archived of these streamed and/or archived video clips, created by the video recording mobile phones, then retrieved, streamed and remixed during live visuals performance(s). The phones in this project act as controllers, or essentially as the computer processor, as well as a receiver and visual performance mixer taking the biofeedback data from the body via the biofeedback sensors. Participants are invited to wear the biosensors and perform or video jockey (VJ) the visuals live, creating the unique mixes from a video archive. Various groups are invited to participate, from party-goers, to Tai Chi practitioners and meditators, to groups with various types of communication disabilities, from deaf/dumb to cerebral palsy and locked-in syndrome, to childrens groups, to the general public to see what types of communication exchanges and collaboration experiences they might have.

Figure 1: 2006 C. Baker Early technical diagram of the networked performance project.

VJing mixing techniques, using software on the phones, instigate a mobile means to collage the video expressions in a live performance setting, then the video clips are mixed and manipulated live (or as near to live as we can), accessing a remote database. This software reads body data (breath flow, heart rate, skin response), from the biofeedback

348

sensors on a performer or participant in the audience. The live modified video/visuals are sent back to the phones of the in-person, or remote audience, like a conversation between the body/performer and the audience. Thus, the body data controls the creative interactions of video mixing, displayed in the live setting, while remotely accessed on phones or Internet. This video collage is meant to become a collaborative, non-linear narrative or remix, ultimately to be globally streamed via mobile-casting, converging these previously distinct technologies and practices, and by bringing all the different presences together to collaborate on the visual mix, simulating collective consciousness. This live video performance collage is the main performance event, combining the technological and experiential context of mobile videophone screens and possibly larger screen, with the body-data responses taken from the biofeedback sensors in a continuous loop or conversation between the body(ies) moving in the live space and the local and remote audience/interactors. Body states, body interface, affect and embodiment In my conceptual explorations and research of embodiment, the mind and body are seen as one, encased within the skin, containing bones and organs, blood, the brain and other organs, fluids and tissue. Rettie (2005) believes that embodiment is consciousness itself1 or an ephemeral essence, rather than physical quality. My work is an enquiry into how embodied experiences could be embedded into media art practices, as well as how they could be experienced in the body, to access memory and altered consciousness. The assumption for me is always that the body is the integral site of the mind, so the experience must take place within the body. This research intended to return to the body for interpersonal exchange, wisdom, transcendence, and exploration, to immersively attune people to their own bodily sensations to harness the virtuality of sensation, imagination and a dreamlike imaginative state. In aim is to understand how people can find alternate, embodied ways to connect to each other, wireless biofeedback sensor technology is used to assist as the embodied interface with mobile devices. From there, it might be determined whether such an interface can simulate or facilitate the telepathic exchange and what we can learn from the body-asinterface in the exchange, possibly serving as a means to interface with technology or as a conduit for unique social interactions. Body data, such as breath, blood pressure, heartbeat, muscle electricity, stress and sweat and other bodily functions, impulses and behaviours are used to interact with mobile software, then be interpreted, or interacted with, as the conduit for creative play and expression. Massumis Parables for the Virtual: Movement, Affect, Sensation (2002) is relevant in terms of presenting theory on affective, sensory aspects and issues, when dealing with sensors and the embodied exploration of art and performance. He focuses much attention on

349

perception itself, perception of sensation, and the concept that the body is always in a state of motion and potential.1 His analysis, in reference to other prominent thinkers and scientists, is grounding and fundamental for work on the body, simulated telepathy and mobile media performance, not to mention the visceral/embodied experience of performance and liveness. Massumis definition of affect, based on the theories of Deleuze and Spinoza, is a rich area of thought. It states that:
Affect... as couched in its perceptions and cognitions... implying a participation of the senses in each other: the ability to transform the effects of one sensory mode into those of another (tactility and vision interoceptive senses, especially proprioception, are crucial)Formed, qualified, situated perceptions and cognitions fulfilling functions of actual connection or blockage are the capture and closure of affect. Emotion is the most intense... expression of that capture. (Massumi 2002: 35)

Massumis ideas of sensation, potential, becoming and movement are critical to embodied mobile media and sensor artwork. His notion of virtuality will be explored more in future writing as well. Presence perspectives Philip Auslander suggests that presence has an emotional impact, but is ephemeral and evanescent an intangible expression that only exists in the transitory present moment (Auslander 1999: 132). While agreeing with this statement, my notion of presence is also a physical, conceptual, phenomenological sensibility almost akin to a vibration or physical agitation. Presence can be felt, viscerally and intuitively. It is the feeling or sensation of another person and their engagement in some activity or with someone. It is when one has engaged actively, emotionally, intellectually, playfully, or is otherwise involved in an activity or experience with others and can feel them also engaged in the experience. According to Shusterman (2000), presence can also mean, to be physically present, inperson, in bodily attendance, in a physical space. Yet, for me, it is about an embodied engagement and the felt or intuitive sense of others, in time and/or space (physical or virtual), it has an emotional quality that transcends physical attendance, including time but not necessarily space.1 Presence for this research is concerned with embodiment and of feeling others on an intuitive level, as well as emotionally and physically, but not necessarily in-person, like one feels anothers presence when speaking to them on the telephone or across the table.1 The sense of presence is critical in my attempt to tap into the visceral, bodily expressions that surface during body practices; Tai Chi, yoga, dance and meditation for example, as a primary source of non-verbal, pre-conscious communication. Physical presences and sensations then are intended to be transformed or transduced into a digital form and used to touch and play with others, remotely or non-locally, using their mobile media devices to represent their presence in a live networked context.

350

Presence experienced in the alternate state of consciousness of dream or pre-sleep is rather elusive to define, but could be described as the permeable, thick viscosity of a fog, where images, sounds and people just appear and whose presence can often be felt, while awareness is able to transition easily between wakefulness or sleep.1 Pierre Lvys1 views on the concept of virtuality are compelling in terms of real versus virtual, actual versus possible, how the body, text, economics, and culture are virtualized and what that means philosophically and practically to modern society. He posits that, based on the Greek definition of virtualis that current philosophy adopts, the virtual is about potential rather than actualization. Yet he believes that virtual has come to mean absences of existence rather than tangibility (Lvy 1998: 23). He focuses on how virtual collective intelligence on the Internet has positive potential for freeing social activity, as well as enhancing knowledge production and dissemination. This aspect of collective intelligence drives the concept of mixing of the mobile media in a live performance setting, using a multi-interaction approach. Lvy also puts a great emphasis on distinguishing between virtual senses and real senses; however, it is my view that these may not be so divergent, as we engage the physical and affective when experiencing both virtual and real senses. My experience of virtual presence is less technological. My interpretation and experience of virtuality is that it can be accessed through the imagination or the dreaming body, conveyed from within an altered state, expanded sensory awareness, perception, imagination and daydream imagination, or extrasensory perception. This is the very embodied, immersive, almost liminal space of the pre-waking/sleep state. It also includes embodied perceptions of reality, which create our natural, invisible, inner worlds and consciousness, the real experiences within an altered dimension, or dreams, psychic experiences, mediation, mysticism, near-death experiences, intuition or vision quests involving physical pain (Baker 2004: 33). When discussing presence, it is valuable to also address the concepts of absence and the space-between, in order to clarify the concept and experience of presence and its evolving definition. This can only explained this sense of virtual absence in my own experience of it, which includes sudden absences of virtual contact, where once there were previously very active, almost daily, exchanges. This kind of change in the interaction can be disconcerting and create an affective sense of lack, void or silence, which was experienced emotionally and viscerally as a disconnect. Hence, unlike when someone is in close physical proximity and geographically local, and you can go to their home, run into them in commonly visited places, or ask their friends how they are, with digital absence, one has no recourse: one is left dangling in the digital abyss, still feeling the void affectively. When a relationship involves great geographical distance followed by intermittent, proximal, then in-person interaction, and often interspersed with long spells of merely virtual interaction even live text, voice or video chat cannot bridge or fix the fragile communication issues or misunderstandings in the same way that live interaction might. If the person on the other end does not respond, they can vanish without a trace. This is where it could be argued that there is a superiority of in-person liveness, in terms of conflict resolution and problem solving approaches to virtual interaction, which is worth exploring further but not at this time.

351

With these ideas in mind, there are multiple manifestations of feeling the presence explored, facilitated and observed of participants in performance research workshops, such as: The engrossed attention during a mobile phone exchange and the in-between space of the network connection. The physical presence of performers and their body data and the in-between space of the data being sent to the phone, followed by the in-between of the phone accessing the database of video clips. The possible telepathic presences during the moving meditation. The captured emotional, aesthetic presences of the video performers within their clips. The physical co-presences of the audience members in the space of the stage events. The virtual multi- or co-presences of the remote audience performers and the global co-presences of these collaborators.

This conceptual exploration is critical to gain insight into what presence is and is not, in order to facilitate it in practical ways in my experimentation and performance events. Liveness perspectives The sense or experience of liveness, originates from previous dance and music performances, and also from using digital technology. This liveness is intertwined with time and/or space, and physicality, or all simultaneously. Connotations of liveness1 are with literally being in-person, physically present, in the here-and-now, happening at this moment, in front of an audience in a traditional performance context, live or remotely (as with live TV broadcast or online event). In a digital context, now-ness is critical, but being physically present is not: for example, one can use a computer with an avatar as a stand-in to substitute for actual physicality and representing a physical player, or use voice-over IP, or video conferencing to replace the physical presence, depending on the contextlevel of granularity, and so forth. For many it is important that liveness involve a certain tangibility and tactility, especially in the context of performance. Tactility and physicality are considered a feature or selling point of liveness according to Auslander, allowing audiences the intimacy of witnessing from a particular performers point of view, convincing them that they are somehow physically interacting with the performer (Auslander 1999). Many performing arts patrons desire physical proximity with the performer; they tend to report a sense of an immediate, intense, sensory and sensual reality. Auslander states that, liveness is ideological...[the] belief that live confrontation can somehow give rise to the truth in ways that recorded representation cannot (Auslander 1999: 128, 129). In this way, proximity during live, staged performances offers viewers smell, touch, biochemical reactions or exchanges between performers and audience, or audience members amongst themselves. Proximity offers a sense of emotion (not only visual), such as hearing small changes in the tone or quality of a performers voice, or subtle body shifts, conveying

352

emotion and immediate feedback, sometimes as subtle as a glance making eye contact a viscerality (Auslander 1999: 85, Notes #35), like a touch. Viscerality for me is an internally felt response in the gut, that is a gut feeling/reaction, lump in the throat, butterflies in the stomach. These are proprioceptive sensations of the internal body that include: hormones, pheromones, fluids and/or nervous reactions and responses triggered by emotional or other thoughts, affect or situations encountered by a person. Brian Massumi says that,
visceral sensibility immediately registers excitations gathered by the five 'exteroceptive' senses even before they are fully processed by the brainThe immediacy of visceral perception is so radical that it can be said without exaggeration to precede the exteroceptive sense perception... viscerality subtracts quality as such from excitation. It registers intensity. (Massumi 2002: 60-61)

This liveness has viscerality that might be difficult to cultivate within the mobile network, but at least a viscerally felt embodiment might be possible,1 since it seems to take place during telepathic experiences,1 and as discussed, virtual presence can cause emotionally and affectively experienced responses and sensations in the body. Understanding and identifying the effect, reception and experience(s) of liveness, aura, presence of the performer and the audience, in order to reveal their intrinsic embodiment within the network, within the construction of the events and activities, can only be accessed phenomenologically. Thus, only by receiving feedback from audiences and participant performers can one learn of their first-person impressions, and then analysis of these experiences can determine whether they felt or experienced liveness differently from traditional performances. 1 Participatory performance Richard Shusterman1 explores the pragmatic, aesthetic experience of performance, suggesting that traditional audiences and art viewers feel that the art establishment has lost interest in trying to satisfy viewers need for affective, emotional, transformational or aesthetic experiences, and so they have gravitated to popular art and entertainment to meet these needs. All of these are important voices to ground investigations on mobile and networked performance practice. Performance in my projects is more in the tradition of public participatory art or performance art, whereby the performance is like a structured improvization, involving available, untrained participants, who are guided by trained performers in semi-structured activities. This form is used and intended as a way to explore the mobile phone and mobile media, as a means to re-engage people in performance and new media art, is aligned with mobile games as much as it is with new media and fine and performing arts. My own notion and experience of performance has evolved from a traditional understanding as a performer in music and dance, a media and performance curator, and as

353

audience member, to a more interventionist (Thompson and Sholette, eds. 2004), sitespecific, performance art and now mixed-reality or pervasive games1 and networked performance. My path is forged by fascination with new domains and experiences that help people to explore and share, which I am always seeking in my own life. This is a fertile, timely area for both new media and performance explorations. Mobile media The current notion of mobile media involves images, sound or videos created by, existing on, or received through mobile phones or other portable devices.1 In networked new media performance practices, four areas of practice have been identified by various media artists,1 some explored in various combinations.1 They are as follows and incorporating most of these practices into the work: 1. Telematic performance and events: telematics connect people to people or people to objects through a network, such as live-video steams of performances or collaborations (or even performance tele-conferencing) across geographic distances. 2. Locative media or psycho-geography: locative media provides a location aware connection through GPS or mobile phones. 3. Wearable projects: wearables extend the body's senses through technological sensor devices. 4. Active objects and responsive environments: smart environments enable spaces and objects to respond to environmental changes of state, generated by people with within them. Martin Sonderlev Christensens recent PhD thesis, As We May Feel (Christensen 2006), thoroughly explores the affective qualities of mobile phones in the context of their recent multimedia creation properties, and the intimate, social uses that have been applied to them. This perspective is fundamental in its exploration of embodied, personal expression possible using mobile, wearable devices, rather than in merely geographical and locative ways. His ideas really ground mine in terms of the intimate nature of mobile phones and their appropriateness for intimate, embodied, participatory performance activities with the public. Otherwise, theoretically and in practice, current works and research in the field of mobile performance and locative media are few but growing, and many are locative or involving mixed reality1 or pervasive games. The approach taken by many locative media projects is to focus on urban space, environmental, ecological and geographical activities and games, and most do not reference traditional performance practices or studies, or performance history, but often come from a gaming or HCI perspective. Although this project uses mobile phones, it is not as concerned with the space, location, and geographic or psycho-geographic, mapping aspect of mobile devices in locative games and other similar projects1. The fascination here is more with the emotive, affective perception or embodied experience possibilities of mobile media, the situatedness1 (Idhe

354

2002: 68) of the person and their experience within and attentiveness to themselves. Also curious is participants individualized, visceral experience of the space, or locativeness within their own bodies, as opposed to situating them geographically, for example, the mapping of their own body parts or an abstracted mapping of their internal thoughts and emotions, or their physical sensations. Participants are encouraged to share the intimate nature of a person describing their own process/practice of living or being. Alternatively, they might describe the experience of performing their lives, activities and thoughts, share them with others, and collect and collage this experience.1 Performance video/vjing The practice of VJing or video jockeying has been chosen as the methods of mixing video in a live context. Live performance video has its roots in the rave culture of the late 80s and 90s, bridging electronic music, new media installation, and experimental film and video, audio/visual or VJ performance is coming into its own as a media art form and practice.1 There are very few academic voices in this domain presently; however, a group called VJTheory have recently shed light on the art process and practice, as well as the dynamic nature of this visual performance. Stunning live performance works have come from the artists worldwide including Canadian artists nomIg, skoltz-kogan, yet many artists are now calling their practice live cinema, audio-visual performance or performance video to disassociate from the perceived negative connotations of VJs as just party visuals, (see Image 1 below).

Image 1: Still from nomIg live performance (permission granted by Stephanie McKay and Ed Jordan).

Another interesting development is that of Web Jockying:

While the DJs and VJs of the world remain tethered to the remix of sound and image, WJs (web jockeys) have at their fingertips on an infinite and diverse pool of material-sound, image, text, code, web cams, blogs, and more--all constantly changing and expanding. A new software development, WJs, offers artists a tool to create live multimedia performances from this digital soup ... the world of artistic algorithms, and computer art in a live performance...1

This has tremendous potential to be used with mobile platforms as well. Related to live video performance is non-linear, non-narrative, ambient or generative video, as well as software and database systems developed for them. Many artists and filmmakers are finding new ways to generate narrative or non-narrative cinema online, using

355

computer databases and programming to display random film clips and audio, online, for ambient home backgrounds, and in installation artworks. In this way, they are always creating something new and live. These two types of nonlinear, live, generated and performed video forms are compelling ways to perform with video, which is what motivates my ideas around using the style and form of VJing practices in my project but involving the mobile phone. The MindTouch project does not adhere strictly to the form of VJing more commonly practiced in clubs and parties and does not incorporate music or sound at all to drive the video mixing, which is fundamental to the art of VJing. Instead it uses the body rhythms and data to drive the mixing. Still, it is important to acknowledge VJ practice as an influence, since this project appropriates its form and some of its style or visual language to construct the collaborative collages. Combining VJing or live video performance mixing, within the technological and experiential context of mobile phone screens, triggered by body data, is the project means to emulate the sharing of dream imagery. This is a key aspect of the work and a way one can imagine a telepathic exchange of two or more bodies, lucid dreaming visually together, while using the current available technology to simulate the concept and at the same time create a live and collaborative experience that puts the VJing in the hands of the participants. Participatory mobile media artifact collection Participants of the video collection workshops that have been conducted are asked to express themselves using the multimedia tools of the mobile phones either using their own videophones or ones supplied. Low-grade video/media on mobile phones are valued for their immediacy, poor image quality and pixilated imperfectness with its own aesthetic quality, but also used for the innate encouragement of spontaneity, speed of thought, including: rewriting, superimposing, remixing of ideas, flashes, clashes of images and emotion, layering of meaning and stream of consciousness; or simulation of telepathy/collective, if chaotic, intelligence. Five video clip collection workshops have been conducted to date, each of four to eight participants. They were conducted in Vancouver, Canada (twice), Dublin, Ireland (twice) and London, England. These workshops have been successful, if in smaller groups than anticipated, but this has proved useful. Each workshop to date has been different due to the number of participants showing up, the available space and date to conduct them, and the environment each has offered in terms of visual materials or fodder for participants to work with. The first workshop in Vancouver (16/06/07) was set in a big centre with a huge gallery space, many workshop rooms, a big black box theatre, a gymnasium downtown next to the water. The initial mind-quieting activities took place in one of the workshop rooms and then after two individual video-collecting activities of 5 to 10 minutes each, and two were conducted in pairs. In one activity, participants were asked to introduce themselves nonverbally, using the videophones, to each other. The image below (Image 2) presents how participants decided to approach the activity, recording parts of themselves and then

356

showing the recordings to each other, as a way to introduce themselves. In the last activity, participants were asked to work with their partner, non-verbally, to create something to send to someone else, telepathically or through the mobile phone, something which would express their inner perceptions and experiences. As a result, several participants went outside and each pair developed a different method to carry out these pair activities and each outcome was fascinating; one of the results can be seen in below.

Image 2 and 3: 2007 C. Baker Stills from participants videos from Vancouver workshop June 2007.

The second workshop took place in Dublin at Trinity College (22/07/07). The space for this second one was in a dance studio and the participants had more room to move around, and, since a couple was dancers they tended to move more, which inspired the others to do so as well. The same four activities as in Vancouver were facilitated, but were video recorded and I was able to observe the participants and assist when they were having trouble with the phones or needed more instruction on the activity. This group was more exploratory with their video expressions and also had one participant with cerebral palsy and two helpers/participants to work with him.1 Participants were asked to interview each other with the videophones, rather than having one interviewer for each of them individually to save time. This proved to be successful, as they seemed to develop a rapport with each other, shared similar experiences, and thus, seemed were more open to discuss them with each other them.

357

Image 4: 2007 C. Baker workshop still Dublin July 2007.

The third workshop took place in Vancouver (21/08/07). Participants met in a waterfront park nearby on a beautiful sunny, hot day, which added to their enjoyment of the workshop. It was a much smaller group of four, thus the first two individual activities were somewhat more difficult to conduct in such a public space, causing participants to seem somewhat selfconscious performing the internal and mind-quieting activities, with others in the park were throwing Frisbees and dogs were running about around them. One woman brought her child with her, as such her focus was reduced, making it harder for her to completely relax. Yet she attempted to do the activities with her child as to the best of her ability in this circumstance. The bright sunshine and inspiring surroundings resulted in the video imagery from this workshop being quite lovely.

Image 5 and 6: 2007 C. Baker stills from participants videos from Vancouver workshop August 2007.

The fourth workshop was again in Dublin, at Trinity Collage again in October (26/10/07). Like the other workshops before it, there was mix of new people from various backgrounds, attending out of interest. Some participants were quite imaginative and had unique approaches to the activities the ways they used the space and so forth, many looking for patterns from the structural environment, away from the group at times. Participants had access to several spaces, thus the individual exercises were done in one studio and pair exercises in a large black box performance space and outside, as it was an unseasonably warm fall day.

358

The images from this group were stunning and exceptionally creative in the way in which the participants used the camera special effects to invert the image, extreme close-ups and colours to further abstract the body parts being recorded. The process was quicker and the activities were improvized somewhat, thus a new activity was introduced involving partners sitting back to back, trying to telepathically send an image or sensation they had to one another, then attempting to visually capture what they sent. Each would then guess what the other person had sent, and then they would share their videos with each other to see if all three versions (the telepathic, the videophone and the guess) were the same. This activity was enjoyed by most, to be used again in future workshops, as it garnered more captivating visual explorations and creativity from participants.

Image 7, 8 and 9: 2007 C. Baker - stills from participants videos from Dublin workshop October 2007.

Each of the workshops involved different types of participant (not just because of the country location, but that might factor in) and different types of environments. The surroundings became a large actor in the content that participants created expressively in their explorations within. This was possibly both because of the comfort level of participants, and what types of visual material they could work with in the vicinity to externalize and represent their inner explorations. This became a more significant factor than intended and has implications upon how one can externalize with the videophone and was influenced by the surroundings, since its difficult to replicate our inner eyes view if the immediate environment provides contrasting or unique visual fodder unlike our own internal experience and imagination more limiting and uncontrollable, yet a stimulus for intriguing explorations. The process of conducting the workshops has been very fruitful, in clarifying my aims of the process, through observation, exploration and iteration, and studying the responses from participants in their interpretation of the activities and video expression/collection process. My ability to guide participants effectively has also developed greatly, in order to assist participants to have meaningful experiences in creating intriguing personal media expressions. The workshops have become an end in themselves, as many groups have asked for practical engagement with these personal media empowerment tools (often for less technologically literate groups and individuals). Since mobile media is still a relatively new format, this hands-on workshop engagement, with a human touch and an intimate form of communication, has touched participants both intellectually and creatively. Thus, this stage of the project has garnered unexpected results.

359

System design and software development The research commenced in November 2006, with acquisition and work with the biofeedback hardware, software and the mobile phones as appropriate for the research. The biofeedback sensors acquired for use in the research are be EMG (electromagnetic), GSR (galvanic skin response), respiration and BVP (blood volume or pressure), as they are the most responsive, easiest to work with in terms of generating usable data, and not uncomfortable to the person wearing them. Electromyogram measures muscle electricity to determine muscle tension of any larger muscle, like arms and legs. Galvanic skin response is work on the fingers and measures the electrical conductance or resistance of the skin that can change when people change their stress levels. Respiration sensors are worn around the ribcage and monitor the inhalation and exhalation depth and frequency based on the abdominal or chest expansion or contraction. Blood volume monitors the relative blood flow in the fingertips, using near infrared light, and from the pulse or blood pressure the heart rate can be determined. These sensors are attached to the body via sticky electrode pads or Velcro attachments (for image source of sensors, see http://www.mindmedia.nl/english/sensors.php). Wireless sensors are desired in order allow for ease of movement. Starting the project we were unable to get sensors from the preferred company, since they were unable to complete their line of biofeedback sensors by the start of the research. So much more expensive and more sophisticated medical grade sensors, than needed, were purchased, with their heavier and more restrictive cabling on the body. However, after some frustration in their use, we more recently abandoned them in favour of creating our own cheaper and more customizable sensors. The sensors initially used were high quality, but the short, thick cables restrict movement and are visually unsightly for performance purposes, unless hidden in customized clothing. The system also came with simplistic software for clinical stress reduction, inappropriate for this project in terms of the visualization of the body data. Thus, because of the time lost in miscommunication with the manufacturer, and in order to develop our own customer software for the mobile phones, we have changed our approach to working with sensors. Most recently, the German electronic musician/creative electronics/java programmer Michael Markert has been working as a collaborator on the project, assembling and creating biofeedback sensors to work with a wireless Arduino system, to embed them into some customized clothing in order to be flexible for unencumbered movement for dance and Tai Chi, as well as to be ubiquitous or unseen on the body, so that is less obvious the body area network sensing system in performance contexts.

360

Image 10: custom biofeedback visualisation software. We are now creating additional mobile live VJ software for streaming video, accessing the database of clips, for editing, mixing and add live effects to this video in real time and then allow audience members to share and interact using these visuals as a communication exploration, as well as working on embedding the custom sensors into lightweight, adaptable garments. Conclusion This article has described the practical aspects of the media art PhD work, new media performance project called MindTouch, as manifested in the initial participant performance video collection workshops conducted last year, intended to be used with performers wearing biofeedback sensors that interact with mobile devices during live events later this year. Some of the workshops, initial observations and outcomes, as well as some of the technological difficulties of working with biofeedback sensors have been outlined. The theoretical notions on liveness, presence, non-verbal communication, telepathy, performance and mobile media explored herein are integrated into the more practical implementation of the workshops, custom mobile software and performance events, in terms of what is important to ask participants to focus on how the software is developed and what its meant to do, and how the participatory performances are guided and staged. The intricacies of the interweaving of theory and practice will be further explored at later stages of completion. Once the research and the video software is completed, the sensors garments are made and a few scratch performances are staged in coming months, more will be known of nature of the presence and liveness being experienced by participants. In addition, more will be known in terms of what is meaningful in the body data from further work with meditators and various groups of performers, and its application for performance media events. The goal is that interfacing technologies will assist in connecting people remotely and allow them to re-engage with each other affectively and expressively in new nonverbal/textual ways. Thus, this project brings together diverging areas of new media research and media art/performance practices, in terms of use of biofeedback sensors interfacing with mobile phone technologies in a unique way.

361

The unique knowledge contributed by this research is a body-situated, theoretical perspective of the experience of creating a participatory mobile media performance. There will be a contribution to the understanding of individual and collective experience of liveness in performance within the virtual space of a mobile, networked performance. More will be known about the perception and experience, both participatory and observatory, of liveness and sensed perception (or not), within these performed, real-time, in-person and virtual domains, using mobile and wearable devices and interfaces. References Auslander, P. (1999), Liveness: Performance in a Mediatized Culture, London and New York: Routledge. Baker, C. (2004), Internal Networks: Telepathy Meets Technology in the DreamPod, Masters Thesis, Surrey, BC, Canada: Simon Fraser University, pp. 33. Byrne, B and Carvalho, A (eds.) (2006-2008), VJ Theory.net, Falmouth, England. Online at www.vjtheory.org Accessed October 2007. Christensen, M.S. (2006), As We May Feel: Interpreting The Cultural of Emerging Personal Affective Mobile Media, PhD Dissertation, Denmark: Department of Innovative Communication at the IT University of Copenhagen. Corino, G. (2006), Spatial Issues and Performative Media in Digital Mobility: A Network Perspective, Digital Futures, MSc thesis, Institute of Digital Art and Technology, University of Plymouth [Online], http://x.i-dat.org/~gc2/digital_mobility.pdf. Accessed 13 July 2006. Damasio, A. (1999), The Feeling of What Happens: Body and Emotion in the Making of Consciousness, New York, San Diego, London: Harcourt Brace and Company, pps. 113125. Idhe, D. (2002), Bodies in Technology, Electronic Mediations, Volume 5, Minneapolis, Minnesota: University of Minnesota Press. Jaeger, T. (2005), Live Cinema Unraveled: Handbook For Live Visual Performance, San Diego, California: Self-published, with funds from UCSD Russell Grant. Jamieson, H. Varley. (2006), online http://rhizome.org/news/story.php?timestamp=20051121, accessed April 2006. Kozel, S. (2007), Closer, Cambridge, Massachusetts: The MIT Press. LaBerge, S. and Rheingold, H. (1990), Exploring the World of Lucid Dreaming. New York: Ballantine Books. Lvy, P. (1998), Becoming Virtual: Reality in the Digital Age, New York: Plenum Press. Massumi, B. (2002), Parables for the Virtual: Movement, Affect, Sensation, Durham and London: Duke University Press. McKay, S and Jordon, J. (2006), http://www.nomig.net/. Accessed July 2006. at

362

Pearson, C. (1992), Precognitive Dreaming, in A Public Experiment in Precognitive Dreaming, Presented at the annual conference of the Association for the Study of Dreams (1993) and The Pittsburgh Post-Gazette Dream Experiment (1994), http://www.nauticom.net/www/netcadet/npcg.htm. Accessed 10 March 2003. Radin, D. (1997), The Conscious Universe: The Scientific Truth of Psychic Phenomena, New York: HarperCollins Publishers. Rettie, R.M. (2005), Presence and Embodiment in Mobile Phone Communication, PsychNology Journal, 3: 1, pp. 1634, http://www.psychnology.org/File/PSYCHNOLOGY_JOURNAL_3_1_RETTIE.pdf. Accessed 10 June 2007. Richardson, I. (2005), Mobile Technosoma: Some Phenomenological Reflections on Iterant Media Devices, Fibreculture Journal, Issue 6: Mobility, New Social Intensities and the Coordinates of Digital Networks, http://www.journal.fibreculture.org/issue6/issue6_richarson.html. Accessed 10 June 2007. Sadler, S. (1998), The Situationist City, Cambridge, Massachusetts: The MIT Press. Schechner, R. (1988, 2003), Performance Theory, London and New York: Routledge. Shusterman, R. (2000), Performing Live: Aesthetic Alternatives for the Ends of Art, Ithaca and London: Cornell University Press. Simpson, J. and Weiner, E. (1989), Oxford English Dictionary, 2nd edn., Oxford: Clarendon Press. Thompson, N and Sholette, G. et al (eds.) (2004), The Interventionists: Users Manual for the Creative Disruption of Everyday Life, Cambridge, Massachusetts and London, England: The MIT Press. Ullman, M. and Krippner, S. (1973, 2002), Dream Telepathy: Experiments in Nocturnal Extrasensory Perception, Charlottesville, Virginia: Hampton Roads Publishing Company. Various authors (2005a), Thorington, H. and Riel, M. (moderators), from soft_skinned_space -empyrelistserve topic: Networked Performance, http://www.subtle.net/empyre/. Accessed 10 June 2007. WJ-S project, online at http://www.wj-s.org/. Accessed July 22, 2006. (2005b), networked performance blog, An Open Online Forum, Topic: NetworkEnabled Performance, http://www.turbulence.org/blog/. Accessed July 2005.

363

Contributor details

Camille Baker is a Ph.D. Candidate at the SMARTlab, University of East London, conducting research on Networked Performance Media, with funding provided by BBC R&D. Her research interests include: mobile devices, video art, live cinema, performance/interactive media, responsive environments and installation, telematics, new media curating and networked communities. Contact: Camille Baker, MASc. Interactive Arts, the SMARTlab Digital Media Institute, University of East London, Docklands Campus, 46 University Way, London. E16 2RD, UK. E-mail: camille@smartlab.uk.com

364

A.6.2/ aesthetics of mobile media art (July 2009)

INTELLECT: Journal of Media Practice http://www.intellectbooks.co.uk/journals/view-Article,id=8520/ Aesthetics of Mobile Media Art Camille Baker, University of East London, Max Schleser, University of Westminster and Kasia Molga, University of East London The current state of mobile media involves images, sound or videos created by, existing on, or received through mobile phones or other portable and telephonic devices, and includes mp3 players, portable digital assistants (Blackberry, Palm Pilot, etc), iPods and other such devices, sometimes using GPS or RFID components as well. This article argues for the existence of aesthetics unique to the mobile media. To examine what those aesthetics are, the mobile medias specific qualities of immediacy and intimacy, afforded by mobile devices, are explored and expressed by three media artists herein, via mobile documentary, mobile performance video and visual fine arts. The following reflects each artists experience and reflections in turn, starting with Max Schleser, then Camille Baker, then Kasia Molga. MOBILE SCREEN AND DOCUMENTARY FILM AESTHETICS The mobile phone can be seen as a viewing device for micro-movies [i]. In the age of High Definition, the mobile phone has introduced a new standard at the other end of the cinematic spectrum. Lisa Gye writes in Picture This: The Impact of Mobile Camera Phones on Personal Photographic Practices: Camera phones are not, however, just another kind of camera. Located as they are in a device that is not only connected to the telecommunications grid but that is usually carried with us wherever we go, camera phones are both extending existing personal imaging practices and allowing for the evolution of new kinds of imaging practices (Gye, 2007: 279). Although mobile media can be translated between different screen formats and viewing experiences, this translation should not be considered as a one-fits-all approach, but rather as having distinctive parameters according to each mediums specificity. Mobile phone moving image productions can be situated in the field of experimental documentary filmmaking and can, therefore, be categorized as mobile-mentary productions. This new category is in its early stages of development and further technological progress will continue to shape this mobile-mentary category. The category can be defined through the characteristics of an original aesthetic signified by pixilated video images. Thus, it is the mobile phones limitations that are the defining pattern for the establishment of this new format. Writing in High Techn, Rutsky outlines the aesthetic dimension within the contemporary definition of technology:

365

It is not only the conception of technology that has changed, but also the notion of aesthetics. The aesthetic can no longer be figured in traditional terms of aura and wholeness, nor in the modernist terms of instrumentality or functionality. Like technology, it too comes to be seen as an unsettling, generative process, which continually breaks free of their previous context and recombines them in different ways (Rutsky, 1999: 5). Aesthetics are here understood and analyzed as the mediation of an experience. Within this experience, which creates an original aesthetic, the significance of location is manifest in the mobile-mentary practice and encouraged by the mobility of the device. This notion can be exemplified through one of the most common phrases in mobile-phone conversations Where are you? or I am at These phrases emphasize the importance of location within voice communication, which transcends into the field of filmic communication. This London based (mobile phone) filmmaker refers to the new emerging mobile screen aesthetic as Keitai aesthetic. Keitai is Japanese meaning hand-carry, small and portable, carrying something, form shape or figure, mobile phone (Flaherty, 2007). The Keitai aesthetic is related to the mobile experience in three ways. First, on the visual level, it is expressed through the digital pixel compositions. In mobile phone filmmaking, the period between 2005 and 2008 is characterized by the advancement from the 3gp mobile phone video file format to the mpeg4 compression format. The city film Max with a Keitaiii, which was produced entirely on a mobile phone in Japan in 2006, depicts the new emerging pixel aesthetics. The experimental feature documentary explores the Japanese megapolis of the Taiheiy Belt [iii] through the lens of a mobile phone. It juxtaposes the advanced Japanese mobile video format, which is based on the mpeg 4 codec, with the former 3gp compression format. The result is visually distinctive as can be seen from the images below. Maxs images (max 1, max 2, max 3) Second, the Keitai aesthetic is expressed in the way mobile phones have impacted body language, and hence the way the body experience is incorporated in the screening and viewing process using mobile devices. The mobile phone has shifted electronic eyes from desktops to the palms of our hands (Mitchell, 2006: 175). Writing in New Philosophy for New Media, Hansen (2004) correlates the aesthetics of new media with a strong theory of embodiment. This experience can be understood as a form of mobile presence, which Ingrid Richardson (2000: 5) calls the technosoma of mobile phones, or the incorporation of the mobile devices into our evolving corporeality, both through the nature of the device itself, as well as its uses.iv As Hansen argues, [t]he image can no longer be restricted to the level of surface appearance, but must be extended to encompass the entire process by which information is made perceivable through embodied experience. This is what I propose to call the digital image (2004: 10). He states that contemporary media art has effected what amounts to a paradigm shift in the very basis of aesthetic culture: a shift from a dominant ocularcentric aesthetic to a haptic aesthetic rooted in embodied affectivity (Hansen, 2004: 11). Through this new sensorium, new media experiences emerge, witnessing the capacity

366

of the body to experience itself as more than itself and thus to deploy its sensorimotor power to create the unpredictable, the experimental, the new (Hansen, 2004: 6). Whether one can generalize this account to the extent argued by Hansen is one question, however I would like to adapt the unpredictable, the experimental and the new characteristic to the works produced for and by mobile devices. The mobile device incorporates the haptic notion as one touches the device actively to record/playback an audio-visual media file: As Greek etymology tells us, haptic means able to come into contact with. As a function of the skin, then, the haptic the sense of touch constitutes the reciprocal contact between the environment and us. It is by the way of touch that we apprehend space, turning contact into communicative interface (Bruno, 2008: 23). This physicality is a characteristic of the new media in general. Manovich writes that the new media change our concept of what an image is because they turn a viewer into an active user. As a result, an illusionist image is no longer something a subject simply looks at, comparing it with memories of represented reality to judge its reality effect. The new media image is something the user actively goes into, zooming in or clicking on individual parts with the assumption that they contain hyperlinks(2001: 180). However, with the mobile device, one is now able to (hyper)link any location to any (new media) image. The mobile device works against the tendency of digital imaging to detach the viewer from an embodied, haptic sense of physical location and being-there (Lenoir in Hansen, 2004: 2). This allows the audience/viewer to identify with the location. In this sense, Keitai can be seen as characterized by its colonization of the small and seemingly inconsequential in-between temporalities and spaces of everyday life. keitai connectivity is a membrane between the real and the virtual, here and elsewhere, rather than a portal of high fidelity connectivity that demands full and sustained engagement (Ito, 2005: 14). Finally, the Keitai aesthetic is connected to the qualities of a state of inbetweenness. In-betweenness here refers to how mobile media operate in-between photography, video and the internet, while simultaneously establishing new links [v]. The mobile phone merges communication and lens-based media. It can connect different online environments, agents and avatars [vi] in any location, so that the space in which communication happens is no longer physical or virtual; it is a hybrid (de Souza e Silva, 2006: 30): Hybrid spaces merge the physical and the digital in a social environment created by the mobility of users connected via mobile technological

367

devicesMobile and portable interfaces are embedded in physical space, promoting the blurring of boundaries between physical and digital spaces (de Souza e Silva, 2006: 19). The mobile device is a case of media converging in a spectacle of information/data, where the division between the media forms seems blurred, but where the location of consumption emphasizes the original experience. Mobile devices push the media experience into a new domain. MOBILE VIDEO AND PERFORMANCE AESTHETICS As a video artist performer and new media curator, who also teaches film and video history, aesthetics and practices, with an interest in VJing and Live cinema, I have been long interested in trying to use mobile devices to marry performance and video. The discoveries I have made of mobile screen aesthetics come from research specifically examining mobile video in a performance or participatory performance context, and revolve around the ease of expression in capture, the intimacy and affective nature of the devices, highlighted by Christensen (2004), the gestural qualities implied by the size and mobility of the device, and the accessibility of mobile phones for all. The project discussed below is a sponsored project called MindTouch.[vii] Theoretically, and in practice, current works and research in the field of mobile media and performance activities [viii ] are few but growing, and many are locative media, mixed reality [ix] and now called pervasive games. However, of interest to me is the emotive, affective perception or experience of performers and participants of these projects, as well as their individualized, visceral experiences, rather than the psycho-geographical (Corino, 2006), or Situationist nature (Sadler, 1998) or exploring the urban condition, or those of a social geography which has been so popular in recent years. The fascination with mobile phones in the MindTouch project is more with the embodied possibilities of mobile media. What is of interest to me is the situatedness [x] of the user in their bodies and their experience within and attentiveness to themselves, rather than the landscape or the world. As such, in addition to liveness and presence, also of interest is the individuals experience of the space or state of being located within their own bodies. In this sense they become more aware of and attuned to their own embodiment and can perceive and locate their own sensations. For example, how people map their own body parts or an abstracted mapping of their internal thoughts and emotions, or their physical sensations. Alternatively, non-verbally illustrating the experience of performing their lives, activities and thoughts, then sharing them with others visually, through the collection and collage of this experience. Performance in the MindTouch project is more in the tradition of public participatory art or performance art, whereby the performance is more a structured improvisation involving available, untrained participants, who are guided by trained performers in semistructured activities. This form is used and intended as a way to explore the mobile phone

368

and mobile media as a means to re-engage people in performance and new media art, and is aligned with mobile games as much as it is with new media and fine and performing arts. My perspective is also that it is necessary to explore key thinkers in performance theory in order to understand the basis for new directions in performance practices, and to discover whether there are any new insights to be found. In this way, Richard Schechner (1988 [2003]) is a definitive voice on performance, as he challenges previously held views in this field, demonstrating the existence of performance in everyday life and practices, such as in play, sports, legal proceedings, rituals, and pop culture. Philip Auslander (1999), on the other hand, focuses on how liveness is in opposition to mediatization in performance, claiming that liveness does not exist outside the context of mediation. While philosopher Richard Shusterman (2000) explores the more pragmatic aesthetics of the experience of performance, suggesting that traditional audiences and art viewers feel that the art establishment has lost interest in satisfying viewers need for affective, emotional, transformational or aesthetic experiences, and so, he claims, they have gravitated to popular art and entertainment to meet these needs. However, my own notion and experience of performance is evolving, moving from a more traditional understanding as a performer in music and dance, a media and performance curator, and as audience member, to a more interventionist, site-specific, performance art and now mixed-reality games (another name for locative media and psycho-geographic and networked performance). The mobile media phones used in the MindTouch research are the medium or conduit for self-expression, especially to convey non-verbal experiences and sensations as an extension of the body/mind [xi] in both non-literal or abstracted ways, as well as more direct representations. Phones are explored as vehicles or methods to express thoughts, perceptions and emotions to friends and between participants, simulating telepathic exchanges, like exploring and sharing a dream. Martin Sonderlev Christensens recent PhD thesis, As We May Feel, thoroughly explores the affective qualities of mobile phones in the context of their recent multimedia creation properties, and the intimate, social uses that have been applied to them. This is an exploration of embodied, personal expression made possible using mobile, wearable devices. For Christensen, [a]ffect is expressed through human actions and constructed in the threshold between our body and everybody that is the others bodies... Looking more specifically at affect, we might see it as primal a psychical and biologically given part of how we function as a physical being. Affect is in the body -- it is our readiness for the world, or readiness to act and it is on the body as expression or actions (2004: 12). It is easy to find oneself engaging in this practice as well: What can I capture?, (looking more carefully), What can I do with this moment/image?, Who shall I send it to? or Where shall I put it, on Flickr or Facebook?, Who am I sharing it with?, Am I doing this for me or to finding a way to connect with others? Christensen provokes a compelling point, articulating something I was feeling and experiencing.

369

Mobile images are often experienced as personal, intimate, private, and once sent, the immediacy of them feels like a giving or blowing a kiss to another through the network or sharing pieces of yourself (depending on the content) (Christensen, 2004). This experience seems common. As with a text message, they are like thought transfers. However, with video, it is like sending your sight, your experiences, thoughts, feelings visually, but also your unique expression, insight and perspective. MindTouch explores embodied, non-verbal communication and interaction, using wearable biosensing devices as interfaces to mobile media experiences within the future planned performance events. In turn, MindTouch studies the effectiveness of the facilitation or enablement of sensing liveness and presence in these contexts.

Image 1 2007 C. Baker still images by participants in video collection workshops Participants in the first phase video collection workshops are asked to share internal images, feelings, thoughts, and impressions using the phone in a non-verbal, visual way, using the mobile video recording cameras, then share these with others. This collection of personal clips is to be used later in a collective collage/performance of this experience. Image 2, 3 and4 2007 C. Baker - stills from participants videos from Dublin workshop, October 2007 Low-grade video on mobile phones is valued for its immediacy, poor image quality, and pixilated imperfectness -- its own aesthetic quality -- but it is also used for the innate encouragement of spontaneity, speed of thought, including rewriting, superimposing, remixing of ideas, flashes, clashes of images and emotion, layering of meaning and stream of consciousness, and simulation of telepathy and collective, if chaotic, intelligence. The key aspects that have been noticed in this research, through both observing participants and own use of the mobile media phone, are four key features: 5. 6. 7. 8. innate performativity or movement/gestural qualities of device portability factor (watching/shooting anywhere)/mundane is interesting viewing world through camera/camera vision of the phone intimacy/expressivity of device/predilection for close-ups

370

When participants are using and working with the phones, it seems that their immediate impulse is to wave them around in their hand and make the images blurred to see the patterns that result from this blurring, so it inspires a playful, gestural or performative exploration that I do not suggest but see people engaging in (I am inclined to do the same). Coming from a video art background this intrigues me, since using the older, bigger video cameras for more professional purposes, and even consumer home-movie camcorders, demanded shake stabilization and an effort NOT to blur the image. Yet a new shooting aesthetic may be unfolding from the mere size of the device and how it lends itself to movement, gesture and exploring the blurring effect. This is of obvious interest to me, since it combines the performance aspect with video and the mobile phone. Portability also encourages intimacy, since one can take their phone to bed with them, to the toilet, or to places where cameras are not allowed, and can communicate much more personal images and videos than standard video cameras enable due to their size, and thus encourage complexity in what they represent to the world. In my workshops, participants are asked to try to find imagery that represents, in an abstract way, how they are feeling or sensing, in a way that they could communicate to others. Dean Terry says about his work, mo.vid.1, a mobile video project from 2005: The project is centered on the idea of remapping private spaces into public ones, of reversing scale, of inverting and rejecting the consumerist idea of quality and its technological expression in ever higher resolutions by exploiting the limits of the devices. (The videos are really pretty poor with all the compression artifacts, and these limitations are exposed by using two Apple Mac G5s to drive the system and two high resolution projectors to display it) (Terry, 2005). This is key to how I work with video from my video collection workshops, allowing ordinary people to become artists by inverting what is considered quality and using the limitations and pixilation as an asset or tool, rather than a hindrance. He also states: The project also examines what it means to project very private space immediately surrounding the body into meta-space. Many of the videos show objects little more than a few centimetres beyond the tiny lens, often some body part, like hands or forearms that obscure an unknown, overexposed background space. Other pieces are gestural performances, recording the movements required when following a line, or when trying to create shapes by moving the camera in certain ways (Terry, 2005, my emphasis). This seems a common approach to mobile video and what I have found myself creating. The device inherently encourages movement, and thus blurry, abstracted patterns and a splattering effect. It affirms my theory that this intimate approach to mobile video

371

production is facilitated or afforded by the device itself. It will be a shame as the manufacturers inevitably are putting in more image stabilization and anti-shake technology to compensate for this. What has also become evident in my work is that the portability factor, which enables users to explore the world around them -- as Christensen suggests, with new eyes, as well as with the ability to watch and shoot anywhere, the mundane becomes compelling: a way of viewing world through the camera vision or through the lens of the phone. Christensen inspires one to think of the mobile phone as a window or microscope to life around us -helping us to see ourselves in the world more clearly and communicate/connect this insight outward to others: to find the world and ourselves in it (Christensen, 2004: 11). Thus, we are becoming more engaged or reengaged with the environment, looking at the world around us more closely, and sharing our experience of it. People find themselves, as do I, shooting video or taking images of their world as they encounter it, and perhaps, seeing differently. The phone becomes a window on the world that has users seeing it differently, or returning to truly observing the world in order to capture it, and then finding details that they might not have otherwise noticed (Christensen, 2004: 6-7). In this way, not only are they learning how to be amateur digital videographers, but they are also reengaging with and interacting with the world again, in a way they may not otherwise do. Christensen ties many elements together and bolsters the argument for mobile devices as affective, social and as important new new media tools and technological developments: mobile device technologies are reinserting the real world experiences into new media context ... offer[ing] locative and corporeal embodiment of technology... (Christensen, 2004: 11). During the FILMOBILE conference [xii] in April 2008, Steve Hawley presented his own work and discussed how, with the advent of HD and High Resolution video cameras and televisions, the world is now shown to us in larger than life reality and quality, creating the illusion of somehow getting closer to the world, close to places or things we could never get close to or see clearly. However, he pointed out that, in this way, HD presents more of a spectacle of the world and still keeps this presentation firmly in the hands of the expert filmmaker or videographer, with access and training in this medium. The images and work using this high-resolution medium are usually the close up of creatures in the Amazon forest, in a National Geographic sense, or of huge vistas in the desert. Meanwhile, Steve Hawley implicated that, the mobile phone, with its poor, imperfection of resolution and pixilated quality and small aspect ratio is in some ways more real, in that everyone has access to the medium and can learn it easily, but more importantly it captures what I am doing now, in this moment, and has that personal immediacy that the imperfection of the image actually authenticates through its lack of production values--of high resolution, clean editing, and large aspect ratio. It is a more intimate and everyday life document that the everyday person can relate to and hence feels more real. Through its messiness, it at once encourages a more personal, non-expert/every-person, frank expression not possible with the HD video image.

372

MOBILE VISUAL ART AESTHETICS In an article written in 2006 and published in the proceedings of the EVA conference, I concluded that one could not (however) find a sufficient way of creating visual art forms exclusively for the mobile phone handsets (Molga, 2007: 16). This was mainly because, during the time that the article was written, even though mobile devices were advanced enough to carry visual content, MMS [xiii] as the most appropriate carrier of this sort of content was not popular enough among users. Moreover, handsets produced by different manufacturers were often incompatible, and sending or exchanging larger sized data files than a simple text message incurred much higher costs. The other important factor was the size and quality of resolution of the mobile phone screen. However, during last two years, the mobile phone market, and the way people use it, have changed. Compatibility and convergence are the top marketed concepts: No technology ever, not electricity, not the automobile, not the personal computer, not the internet, not credit cards, has had such a fast cannibalization of an existing industry (Ahohen 2004: 3). Small screen resolution is higher, nearly HD in dimension and format, although still tiny, and is expanding. Data sent and received can be larger and is cheaper to exchange. Thanks to Bluetooth, data can be exchanged free of charge with phones or other Bluetooth-enabled devices (computers, printers). Illuminations is a body of work consisting of a number of art installations concerned with the various ways of adopting mobile phones in artistic expression as well as art experience. Dealing exclusively with visual content, informed by fine art practice, this work treats the mobile phone as a portal between the audience, the artwork and the artist. In installations such as Mirror of Infinity or Breaking News,[xiv] the mobile phone is an avenue of communication between a spectator and a visual art piece, exhibited in a semiconventional way as an image on the wall of the venue.[xv] In Little Heavens, however I employed a mobile phone as the main platform of exposition of visual artworks, delivering a unique and intimate experience to the user. LITTLE HEAVENS
A couple of decades ago in Poland, children had a peculiar game called Little Heavens. The concept was to create beautiful collages from flower petals and pieces of colorful candy wrappers and place them under a piece of glass in a hole in the ground. The hole would be then covered by soil, leaves and grass to make it look like it was untouched. A child who made the collage would remember this location and then share it with their closest friends. Children with whom the secret was shared were let into the circle of trust. There were no winners in this game, however, children would compete with each other in the creation of the most beautiful collages. Sometime collages disappeared, dug out and destroyed by kids who belonged to more than one circle of trust or by those making Little Heavens on their own who wanted to get rid of the competition for the most beautiful creation.

In the art installation Little Heavens, I attempted to convey the meaning of this game, from the intimate hole in the ground to the intimate screen of ones mobile phone.

373

Spectators will be able to send and receive fragments of text or images which then combined with my drawings will be send back in a form of the picture as an intimate token of trust. Before embarking onto the creation of this art installation, I had to examine the aesthetics of a mobile phone screen to ensure the best quality of art experience. THE MOBILE PHONE SCREEN AS A MEDIUM OF DISPLAY AND EXPERIENCE The digital screen as a mean of display of works of fine art created with traditional techniques has not been met with too much enthusiasm among artists to date. Paul Zelensky, an artist and writer living in Los Angeles noted: basic problems endemic to viewing art on screen. For the artwork: the limitations of size and scale, the elimination of texture, materiality, shadow, reflected light; the absence of other images in the visible environment with which to measure and compare size, scale, texture, and so forth. For the viewer: the denial of the bodys and the eyes immediate perception, coordination, and assimilation of all of the above (Zelevansky, 1997). The article by Zelensky was written more than 10 years ago. His opinion of art viewing on a screen is still valid; however, since then, not only has the display technology improved, but also the spectators gaze has become more sophisticated in understanding digital presentation. Through Little Heavens I sought to exploit those limitations and turn them into advantages for creation and viewing of mobile art. I attempted to establish this on three levels: 1. The mobile phone screen display attributes and its position within the framework of art history, i.e. light as a one of the most apparent features and the effort of the artists from the past to achieve the effect of light and glow; 2. The mobile phone screen surface attributes and its influence on formal qualities and technique of picture making, i.e. size in pixels and number of colours. 3. Mobile phone screen display and the spectators interaction as part of a creative and emotional experience, i.e. the way spectators operate their phones to have a look at the picture. Both of my art installations for this project aimed to reintroduce the picture as a window to another world, finding their roots in the history of art, when images were first made to conjure up the appearances of something that was absent (Berger 1972: 10) and to convey the concept of the Sublime and the Beautiful through the Renaissance focus on flat, rectangular surface intended for frontal viewing (Manovich 2001: 95). The small size of the screen surface draws an analogy to miniature paintings of the Middle Ages medieval illuminated manuscripts. Decorated with gold or silver leafing,

374

those tiny illustrations meant to illuminate lighten up the page. An analogy to illuminated manuscripts can be seen not only in a small size of the surface of the screen but also an internal light appearing when the phone is active. Light has another function. The colours of a painting are visible, thanks to the light reflected from the painted surface. The light rays having various lengths, make the human eye see different colours. The digital screen of a mobile phone has a light within, creating a very characteristic aspect of glow to this medium. The image glows with its lights. The intensity of colours is deeper. Paul Zelensky noticed the benefits of such presentation: the clean, concentrated colour and light of the screen buttresses the technological and authorial power of the message (Paul Zelevansky 1997: 46). When creating a picture and then digitizing it for the use of the miniature screen, the importance of the technique becomes apparent. In order to achieve the glow effect and to use the feature of internal light of the small screen, simplicity is the key. Over-complicated drawings or paintings done on a textured surface or with thick brush-stroked paint might not work. The texture on the surface of paper or canvas would almost be lost, looking rather like dirt; brush strokes would be invisible or create shadows; too many fine lines would merge into one or become entirely invisible. Spectators answering the questionnaire about this project, after being subjected to a week-long experiment, seemed to prefer the smooth, figurative, colour pictures, which were either made entirely using vector graphic applications or high resolution scans of paintings or drawing on smooth surface, and using either very decisive lines and spots of colours or polished paint. Image KM_02 Art experiment# 1 Technique: black ink pen, watercolor pens on watercolor paper Image KM_03 Art Experiment # 2 Technique: black ink pen, watercolor pens on watercolor paper Image KM_01 Art Experiment # 3 Technique: vector graphic and digital collage made using Photoshop

Interestingly, while seeking the effect of Renaissance depth, the mixed technique pictures, combining both traditional and vector/digital media, seemed to convey the effect best. An important aspect of visual mobile phone art making is the way in which the picture is digitized. If it is done with traditional techniques, it can be scanned and then transformed to the right size and right format for a mobile phone. However, the more obvious method would be to use the phones own built-in digital photo camera. The latest digital camera phones are of a decent quality (above 5 Mg px) and have various settings. This approach enables immediacy and is more personal when the picture is photographed and sent, rather than scanned and manipulated and then sent. Although most of the spectators taking part in the experiment claimed that they could not see any difference between those two methods, when asked about glow and light they pointed to paintings photographed or images made entirely with vector graphic applications.

375

Image KM_04

Image KM_05

Image KM_)6

Little Heavens # 4 Little Heavens # 5 Little Heavens # 6 Technique: acrylic ink Technique: acrylic ink on Technique: ink on tracing paper on tracing paper tracing paper The emotional attitude of viewers to the personal aspect of the mobile phone is a very important characteristic in both experience and creation of visual art forms for a mobile phone screen. The experience of receiving messages of the art piece in ones private space on the private window of a mobile phone makes a strong statement about this concept, which might make daily contemplation on art a seamless experience. This discovery also can help create a new set of rituals of experience of this type of art form one of most important factors in the appreciation of work of art as noted by Walter Benjamin: The uniqueness of a work of art is inseparable from its being imbedded in the fabric of tradition It is significant that the existence of the work of art has its basis in ritual, the location of its original use value (1936 [2000]: 677). In his article Interaction as an Aesthetic Event, Lev Manovich examines the visual qualities of mobile devices and their effect on a user: Have you realized that the phone that you own assuming it is a model that came out in the last couple of years constantly plays games with you? It seduces you with its animated icons and sounds, the shape and surface finishes, the feel of its buttons and every other detail of its material and media definitions (Manovich 2007: 1, my emphasis). According to Manovich, the modernist design formula form follows function came to be replaced by new formulas such as form follows emotion (ibid). Christensen (2004) also noted in his thesis the intimate and affective nature of this device. Both statements made by Manovich and Christensen show an analogy to another aspect of miniature paintings from the previous centuries namely miniature portraits intimately exchanged by lovers. It is obvious that people always have had a need for emotional connection with others and a shared secret or an intimate object is a token of their connection. The way miniature portraits were treated is very similar to the Little Heavens game and the way we swap text messages with people we are close to. Direct interaction with the art piece is another crucial element in such an art practice. It creates an opportunity for a deeper, memorable experience and a better understanding of content. The unique interaction with the visual art piece can enhance and reach further than just static art experience within the space of the art gallery. The mobile phone picture here the visual artwork not only can substitute something absent as stated by Berger; it also makes a spectator directly and intimately connected to an artist, even if anonymous. There is clearly a felt state of connected presence a term coined by Christian Licopp, especially if

376

pictures are sent to spectators every day during a certain period of time. The connectedness through the private communication device makes spectators feel special a very important and unique aspect of this type of art experience. With the mobile phone, spectators have to actively engage in un-packing the parcel in order to see the artwork. First they hear the sound or feel the vibration of the phone notifying them that someone or something has sent a message. Usually during the same time or when a user takes the phone into their hands and presses a button to look at the message, the phone lights up. Then there is a journey through the interface to retrieve the message. People taking part in the experiment noted that the picture an artwork was a very positive surprise and most of them saved it to their phones galleries to look at it on a neutral background, with no interruption from other elements of the interface. If in a space with other people friends they quite often shared the excitement and the picture with others. As mentioned previously, mobile media enable a new mobile experience, which can be linked to the embodied sensorium of performance works and early cinema. Why is it important to utilize a mobile phone as a carrier for visual arts? Because nowadays the disappearing experience of ritual as something special and the ubiquity of visual art imagery in advertising, make such messages epidemic. Visual imagery has become too mundane thus, people stop noticing it. So perhaps to use a mundane object to deliver an art piece can help us to remember that [t]he visual arts have always existed within a certain preserve; originally this preserve was magical or sacred. But it was also physical: it was the place, the cave, the building, in which, or for which, the work was made. The experience of art, which at first was the experience of ritual, was set apart from the rest of life precisely in order to be able to exercise power over it (Berger, 1972: 32). CONCLUSION In summary, in our individual practices we have made converging discoveries to begin to define the new mobile screen aesthetics. These discoveries include the following. In order to understand mobile video projects, it is important to evaluate them in a category of their own, which this article suggests is within the new emerging mobilementary (mobile documentary) and performance video categories. Mobile devices position us as agents not in cyberspace, but in the everyday environment. The new emerging pixel aesthetics include low-resolution digital images and video, digital pixel compositions, constrained and condensed screen dimensions and file size. This aesthetic emphasizes the importance of location, space and also non-space being here, being anywhere, but here is where the phone is and not any fixed place, via immersion. The individuals experience of the space, or state of being is located within their own bodies. This is expressed as a haptic aesthetic, rooted in an embodied affectivity, manifested through gesture and movement of the device while documenting or videoing innate performativity or movement, which is

377

afforded by the phones themselves. Through the emergence of mobile devices as a tool for creative production, an alternative space for the creation of artwork has emerged. This article has outlined the potential of mobile media for the fields of mobile performance video, visual fine arts, and mobile documentary filmmaking. This environment provides choice for creatives (artists and mobile phone users alike) to express themselves beyond the realm of the media industry. The new aesthetic drives innovation and illustrates the potential of mobile media within the evolving mediascape. Mobile media can be accessed and utilized by a large number of people everyday. There is an unintended intimacy and affective nature of the device. Expressivity with the device constitutes a predilection for close-ups and a sense of immediacy instant, realtime, being here now. The portability factor allows for the watching and shooting anywhere, with the intimacy of enabling one to take their phone to private and personal spaces. Mobile devices make the mundane interesting, the everyday confronted, providing a new lens for viewing the world through a new camera vision. The mediazation of the world through the phone magnifies and brings the everyday world back into focus, having an amplified reality and quality, creating expectations of somehow getting closer to the world. The everyday relation with this device creates a new set of rituals which can be used in appreciation of delivered content whether it is a piece of visual art or a picture sent by a friend. The sound or a vibration made by this device creates a sense of expectation, even urgency (McLuhan 2004: 289) a crucial part of a ritual followed by manual unpacking of a message. Visual display is made available thanks to a small LCD screen; its resolution is constantly improving allowing the exhibition of consistently more complex imagery. What is also important is the use of light light within the screen produced when the phone is activated a vital component of the represented image. The sense of intimate connectedness to the message, its subject and the author, has emotional implications in everyday art experience making one feel special, important and inspired. As this article has illustrated, the mobile media device is a new medium for expression and experience, with its own unique aesthetics.

378

References Ahonen, T.T. (2004), From Interesting Ideas to Compelling Mobile Services, Vodafone Receiver, Issue 12 [Online] http://www.vodafone.com/flash/receiver/12/articles/index04.html Accessed September 11, 2008 Auslander, P. (1999), Liveness: Performance in a Mediatized Culture, London and New York: Routledge. Benjamin, W (1936 [2000]) The Work of Art in the Age of Mechanical Reproduction, in McKeon, M (ed.), Theory of the Novel: A Historical Approach, Baltimore: The Johns Hopkins University Press, pp. 673-695. Berger, J (1972) Ways of Seeing, London and Suffolk: British Broadcasting Corporation and Penguin Books Ltd Bruno, G (2008) Atlas of Emotion: Journeys in Art, Architecture, and Film, London: Verso. Christensen, M S (2004), As We May Feel: Interpreting the Culture of Emerging Personal Affective Mobile Media, unpublished PhD thesis, Copenhagen: Department of Innovative Communication, IT University of Copenhagen. Corino, G (2006), Spatial Issues and Performative Media in Digital Mobility: A Network Perspective, Digital Futures MSc dissertation, Institute of Digital Art and Technology, University of Plymouth [Online] http://x.i-dat.org/~gc2/digital_mobility.pdf Accessed 13 July 2006 Davenport G, R Evans and M Halliday (1993), Orchestrating Digital Micromovies, Leonardo, 26: 4, pp. 283-288. De Souza e Silva A (2006) Interfaces of Hybrid Spaces, in Kavoori, A P and N. Arceneaux, (Eds.), The Cell Phone Reader: Essays in Social Transformation, New York: Peter Lang Publishing, pp. 19-44. Gye, L (2007) Picture This: The Impact of Mobile Camera Phones on Personal Photographic Practices. Continuum: Journal of Media & Cultural Studies, 21: 2 (June), pp. 279288. Flaherty, S. (2007) Keitai culture [online] http://www.keitaiculture.com Accessed August 07, 2007. Hansen, M (2004) New Philosophy for New Media, Cambridge, MA: The MIT Press. Idhe, D (2002) Bodies in Technology, (Electronic Mediations, Volume 5), Minneapolis, Minnesota: University of Minnesota Press. Ito, M (2005). Introduction, in Ito, M., D Okabe and M Matsuda, (Eds.), Personal, Portable, Pedestrian: Mobile Phones in Japanese Life, Cambridge: MIT Press, pp. 1-16. Manovich L (2001) The Language of New Media, Cambridge, MA: The MIT Press Manovich, L. (2007), Interaction as an Aesthetic Event, Vodafone Receiver, Issue 17 [Online] http://www.vodafone.com/flash/receiver/12/articles/index04.html Accessed August 02, 2008 Mitchell, W (2006) Networked Eyes, in Jones, C A. (ed.), Sensorium Embodied

379

Experience, Technology and Contemporary Art, Cambridge, MA: MIT Press, pp. 174-9. McLuhan, M (2005) Understanding Media, London and New York: Routledge. Molga, Kasia (2007) Wireless Art: the Possibilities of Application of Mobile Communication Media to a Modern, Visual Art Practice and as a Medium of Expression?, Proceedings, EVA Conference, July 2007, pp. 16.1-16.6. Richardson, I. (2005), Mobile Technosoma: some phenomenological reflections on iterant media devices, Fibreculture Journal 2005, Issue 6: Mobility, New Social Intensities and the Coordinates of Digital Networks. [Online] http://www.journal.fibreculture.org/issue6/issue6_richarson.html. Accessed 10 June 2007. Rutsky, R L (1999) High Techn: Art and Technology from the Machine Aesthetic to the Posthuman, Minnesota: University of Minnesota Press. Sadler, S (1998) The Situationist City, Cambridge, MA: The MIT Press. Schechner, R. (1988 [2003]), Performance Theory, London and New York: Routledge. Shusterman, R. (2000), Performing Live: Aesthetic Alternatives for the Ends of Art, Ithaca and London: Cornell University Press. Terry, D (2005) mo.vid.1. [Online] www.100lies.com or http://www.deanterry.com/blog/index.php/2005/10/ Accessed September 12, 2008 Wolf, R W (2009) Micromovies Pocket Short, Part 1. Das Kurzfilmmagazin [Online], Available: http://www.shortfilm.de/index.php?id=414&L=2&print=1. Accessed 20 March 2009. Zelevansky, P (1997) Shopping on the Home Image Network, Art Journal, 56: 3, p. 64 [Online] http://www.questia.com/read/5000561018 Accessed September 04, 2008 Endnotes According to Reinhard W. Wolf, The term micro-movie was coined in the 1980s in Nicholas Negropontes Architecture Machine Group at MIT and expanded in 1993 by Glorianna Davenport (Interactive Cinema Group at MIT Media Lab) in the context of interactive video databases which were introduced in 1997 at the International Short Film Festival Oberhausen. (Wolf, 2009). In Orchestrating Digital Micromovies, Davenport, Evans and Halliday (1993) describe a micromovie as a short piece of video with descriptive information attached to it.
ii iii i

Max with a Keitai (2008) Japan/UK, 3gp and mpeg4 video.

The megapolis Taiheiy Belt, the Tokaido corridor, includes the Ibaraki Prefecture in the north of Japan and reaches to the Fukuoka Prefecture in the south in Japan. This Pacific belt includes the major cities of Tokyo, Osaka, Kyoto, Kobe, Himeji and Hiroshima amongst others Such as in the gestural work of MindTouch, discussed below.

iv

380

An example of this is semacodes, a mobile barcode technology which allows mobile phone users to connect the online environments through taking a picture of the specific mobile barcode. An avatar is ones representation in space in which one is not (de Souza e Silva, 2006: 29) MindTouch workshop video edits are available online at www.swampgirl67.net in the Media Research section Such as projects like Uncle Roy All Around You from the performance group Blast Theory, http://www.blasttheory.co.uk/bt/work_uncleroy.html. I have noticed many locative game projects are now calling themselves mixed reality, since they mix digital gaming interfaces with live performative tasks and game play, using mobile devices. In Don Idhes sense that to be situated entails that the knower is always embodied, located, is a body (Idhe, 2002: 68)
xi x ix viii vii vi

This use of body/mind acknowledges the studies and theories of Brian Massumi, neurobiologist Antonio Damasio, Benjamin Libet and other philosophers, showing that the mind not only exists in the brain but also in parts of the body, especially the senses. FILMOBILE is run by a networked team of professionals working in telematic film, leading a major project developed at the University of Westminster. The Project brings together the work of many international artists, filmmakers and professionals working in the mobile phone industry. In April and May 2008, FILMOBILE organized a major international event consisting of a gallery exhibition, cinema screenings and an international conference. This event explored the cultural and economic impact brought about by new mobile technologies and initiate debates between artists, the media and the new mobile industry (www.filmobile.net). Multimedia Messaging Service

xii

xiii xiv

Art installations engaging the mobile phone as an avenue of experience, developed within the last 3 years. Mirror of Infinity and Breaking News interactive art installations pictures available online at www.kasiamolga.net in the Interactive Installation section
xv

The visual manifestation of this installation is either in a form of a projection or on LCD display.

381