Visual programming of intelligent agents for virtual archaeology

Undisclosed

Abstract Historians and archaeologists have used virtual worlds in their research to provide them with a laboratory to test hypothesis based on excavation data, site pictures, historic documents and images, and similar elements. Known as virtual archaeology, this new set of methods to visualise historic sites has so far focused on depicting the architecture and artefacts found on the sites. More recently, however, historians and archaeologists have employed virtual worlds to also study social interaction in those sites, thus increasing the level of immersion for researchers, students, and visitors, but also presenting new challenges: simulation of behaviour and social interaction commonly uses intelligent agents, an area extensively researched, but which requires relatively advanced programming skills which might not be universally available to historians and archeologists. Work done for this thesis will attempt to provide them with simple, visual tools to allow researchers to describe profiles and rules enabling the simulation of social interaction inside heritage sites recreated in virtual worlds using Second Life/ OpenSimulator technology. Keywords: Virtual archaeology; Second Life; OpenSimulator; visual programming; intelligent agents

included mostly computer experts, software engineers, and 3D modellers. Several decisions were made during the creation of the models that were poorly documented (or not at all) and the criteria were mostly favouring an interesting, quasicinematographic experience, regardless of the actual level of historical accuracy of the models. Additionally, the costs were prohibitive for most installations, mostly due to the hardware requirements, many of which were still experimental prototypes, custom-built for a specific type of experience, and not easily available. This lead historians and archaeologists, recognising the immense potential of 3D computer models for their research, to start proposing research methodologies focused on the historian/archaeologist (one of those methodologies is detailed in the London Charter [3], but others exist). These documents specified guidelines for adding documented justifications of all modelling decisions made during a virtual archaeology project, described metadata types to identify the 3D objects used in the project, and suggested methods to keep track of successive revisions and versions of the project. The adoption of such a method would allow researchers to validate models as to their historic accuracy, by allowing further researchers to build their own models based on the documentation provided, and eventually confirm or reject the model independently. Thus, current research in virtual archaeology tends to have historians and/or archaeologists as project leaders, with technically qualified researchers contributing to the project development under their guidance. This lead to historically more correct models, which, however, might be visually less attractive. The new model is not free of problems. On one hand, historians might not be technologically proficient enough to understand how to use and employ appropriate computer hardware and software to model the heritage site [4] or have difficulties in conveying what they wish to create to the technical team, overestimating the capabilities of currentgeneration hardware and software. Research teams and technical teams might be separated, working in isolation of each other, and only with difficulty manage to complete a project that is fulfilling to both and is also historically validated. Fortunately, such examples abound (Rome Reborn is a typical example of a successful project [5]). Since many of those projects might not be so visually appealing as the first-generation projects, the movie and game industry have been more reluctant to adopt them, preferring instead to continue to use historically inaccurate models, since they have a different audience in mind entertainment and not necessarily education. This raises the question of how to

INTRODUCTION Virtual archaeology is a field of research that employs techniques and methods used by historians and archaeologists, using computer-generated models, to visualise cultural artefacts and heritage sites [1]. These days, most of those models are done in 3D and have become increasingly popular as the cost of sufficiently powerful hardware to render 3D synthetic environments dropped to affordable levels. Historically, this area of computer science focused mostly on the technology to render 3D artefacts and heritage sites[2]. Research focused on realism and immersive interfaces, usually using haptic devices, and the goal was to provide an experience of the past that was as realist as possible. This captured the attention of the movie and games software industry which were interested in the degree of realistic detail provided by the technology employed on the models. On the educational side, most of the installations were made in museums or similar environments, where whole rooms could be dedicated to the required machinery providing the illusion of walking across heritage sites; more light-weight solutions, using portable helmets and augmented reality glasses, started to appear on some historical sites, providing additional information to visitors. However, Frisher reports that the focus of those initial attempts was on the spectacularity of the technology, and not on its historical accuracy. Frequently, historians and archeologists were not part of the research teams, which

I.

distribute the results of a virtual archaeology project once all 3D models have been created; very often, the costs of distribution have to be included in the project as well. II. VIRTUAL WORLDS A. Solving distribution problems, but raising new ones Virtual worlds are synthetic computer-generated 3D environments used simultaneously by thousands or even hundreds of thousands of users, each of which represented by their avatar, and who can interact, in real time, with the simulated environment. These allow for the same model to be instantly visualised by several users sharing the same virtual space, and are thus appropriate both for entertainment and educational purposes (e.g. a guided tour to Ancient Rome, which can be visited alone or in a group with or without a guide [6]). Among all virtual worlds, Second Life, and its open source counterpart, OpenSimulator, has gained popularity among the research community. Second Life has around twenty million users registered with their avatars, and all the displayed content is created by its users. 3D content is created and visualised in real-time; no special external software is required, since the open source Second Life Viewer includes all building and programming tools to create content, which appears immediately for all users in the same scene, without requiring rendering. Content is created collaboratively: several users can work on the same models at the same time, and each will see immediately the results of adding further content. The underlying system will provide object persistence by storing content on the so-called grid servers, which are hosted by the company Linden Lab in the case of the Second Life Grid a collection of several servers running a single, visually contiguous virtual world. While creating appealing content requires some skill and talent, the creation tools were designed with amateurs in mind. All this has contributed to its widespread use as a residential entertainment platform, a business tool, an education platform, and as a research and simulation platform for many different areas; virtual archaeology neatly fitted in the kind of possible uses for Second Life. When working on a virtual archaeology project, teams can log in from geographically distinct areas but work collaboratively on the same virtual site; due to the nature of immediate visualisation of the created content, historians can use Second Life as a research lab: modelling heritage sites based on researched documentation and images and deploying a possible virtual reconstruction of the site. Researchers can immediately validate their assumptions visually and proceed to correct them. Changing models or moving it to different places is very easy to accomplish even for an unskilled user; by collaborating with skilled 3D modellers, historians can, in the virtual world environment, using communication tools like Voice over IP (VoIP) or simple text chat, instruct technicians to position content accurately. This interactive mode of close collaboration allows several hypothesis about the layout of the heritage site to be quickly evaluated and approve or reject them. Simultaneously, while the site is being reconstructed, the area can be opened to visitors; thus, it is possible to dynamically adapt and change the whole site, or start with an area but expand it gradually, while the site is being actively visited. Virtual worlds like Second Life neatly exploit this

facility to distribute content to a relatively large audience very easily; unlike other platforms, it eliminates long preparation phases before modelling begins, followed by content actually getting modelled using specialised and expensive 3D tools like Maya, 3DS, Blender, or others; 3D models get rendered (which usually takes a lot of time); digital video footage is captured and then edited; and finally, the resulting images are published, burned to DVDs, or sent to websites for streaming. Any corrections required under a more traditional approach take a lot of time until a new version is produced and distributed to the end users. By contrast, using virtual world technologies, heritage sites are immediately available once modelling is finished, and they can be changed dynamically in real time and adjusted instantly without any further delays. Morgan [4] explains how this process even allows historians to forfeit the use of specialised 3D modellers; historians and archaeologists might be able to acquire the required skills to do the modelling themselves. While the results might not be as visually appealing as the ones produced by talented professionals, they will be historically correct, and the elimination of a further layer of communication with nonhistorians might lead to faster results. B. Virtual archaeology is not only about architecture Nevertheless, traditional methods of creating virtual archaeology projects that lead to video documentaries have one important feature: they show humans (in this case, human actors) interacting within the environment of a heritage site. Archaeology is not only about architecture and art, but also about the people who populated the site; from an educational perspective, it is important to contextualise the architecture and art and show how it has been used by its past inhabitants. In virtual worlds, to convey the human dimension, one valid approach is to hire actors, who, through their avatars, will demonstrate human interaction within the virtual reconstruction. The problem in this case is that virtual worlds are visited 24 hours a day, 7 days a week; staffing a virtual reconstruction full time, around the clock, is often way beyond the resources of most virtual archaeology projects. A tradeoff is to announce specific events on certain dates, and expect visitors only to arrive at the specified hour. This, however, will exclude many potential world-wide users who might be interested in the project but live in timezones that disallow them to participate at the appointed hours; by contrast, a game or a video can be played at a convenient hour for all interested parties, and repeated as often as the viewer wants. A possible solution is to model the behaviour of inhabitants of the heritage site using intelligent agents. III. INTELLIGENT AGENTS IN VIRTUAL ARCHAEOLOGY The immediate advantage of using intelligent agents in virtual archaeology projects is that they are always available. They also permit the researchers to model behaviour easily and fit it better to the environment, or even apply the same approach to architecture and formulate what-if scenarios about the use of certain spaces by human inhabitants. Unlike a video documentary, which usually just presents one possible way of human interaction with the heritage site, virtual worlds using intelligent agents to model human behaviour can show a whole range of different behaviours, and allow researchers to

evaluate them, picking the ones that better reflect the findings from researched data. Intelligent agents are also very-well documented and have many implemented solutions (e.g. Bogdanovych [7]). The problem is that they require programming, a skill that might not be available to many historians or archaeologists. In this case, a common method is to use programming by imitation, where human operators teach intelligent agents by demonstrating what they should accomplish (e.g. walk across a street, carry this object, sit here, turn around, and so forth); this technique was employed by Goertzel et al. in Second Life [8]. The problem is that simulating human behaviour by carefully programming each intelligent agent by imitation takes a lot of time and can be tedious. IV. SCOPE OF THE DOCTORAL THESIS To summarise the above issues, a virtual archaeology project with scientific accuracy requires having an historian or archaeologist as a project leader. Better projects will not only include artefacts and architecture, but propose models of human behaviour interacting with the heritage site; these will also increase the immersive nature of the project for potential visitors or students. The problem is that historians and archaeologists might not be technically qualified to do complex programming of intelligent agents; while some forms of programming by imitation exist, they can be slow, painful, and tedious. Historians can already validate architectural models in a laboratory setting, but the question remains open on how to validate models of human behaviour and interaction in those very same heritage sites. Thus, ultimately, the issue is how to allow historians to develop those models without requiring them to learn programming languages and acquire proficiency in those. A possible solution might involve developing a new set of simple visual tools; but those might be either too simple and not powerful to replicate the desired behaviour models, or, conversely, they might make the task overwhelmingly difficult for historians. V. PRACTICAL CASES In my work at the company Beta Technologies, which develops content and applications for the virtual world of Second Life (or on OpenSimulator-based virtual worlds, which, for all purposes, work similarly), I have participated in three virtual archaeology projects, all of them involving not only the 3D modelling of a reconstruction of a historical site, but also requiring human interaction (simulated using avatars) with the environment. In Kings Visualisation Lab Theatron 3 project [9], a recreation of about 20 European theatres from across a wide span of eras and countries, the required human behaviour simulation was confined to simulating stage directions for a play; a simulator was developed to allow students to deploy choreographies to automatically animate avatars according to a pre-programmed plan. The City & Spectacle: a Vision of Pre-Earthquake Lisbon project by CHAIA (U. vora) aims to recreate the environment and the spectacles of Baroque Lisbon before the 1755 earthquake utterly destroyed it [10]; spectacles will include human visitors and participants, actors representing roles according to scripts, but also intelligent agents in the background contributing to the

overall picture. Finally, in the University of Bristol Crystal Palace Project [11], intelligent agents driving Second Life avatars will replicate the behaviour of Victorian visitors to a reconstruction of the Pompeian Court inside the Sydenham Crystal Palace of 1854. All these projects, at some point, have required to provide historians with a simple and friendly tool to model the behaviour of intelligent agents in Second Life or OpenSimulator. While the approach used in the Theatron 3 project was enough for its intended purposes, new tools, designed from scratch, shall be employed on the Crystal Palace Project and may be subsequently used on the City & Spectacle project as well. Future research may include a qualitative analysis on the impact of behaviour and social interaction models using intelligent agents to better provide an immersive experience to visitors or to enhance the training and retention levels of students exploring the virtual heritage site. Another analysis may be made on the complexity and efficacy of visual programming tools for intelligent agents when compared to more traditional programming tools. VI. METHODS, GOALS AND TASKS

My doctoral thesis is focused on the Crystal Palace Project and will begin with a series of questionnaires made to the historian researchers of University of Bristol in order to define the functional specifications of the required tools. The overall generic problem definition will be developed by interacting with those researchers. The state of the art study shall uncover documentation about ongoing research on intelligent agent programming and visual programming tools, specifically those aimed for nonprogrammers (for instance, children). This is a very vast field which will require identifying what research method is more adequate to tackle with the problem, and, once identified, what technologies and solutions exist that provide historians with the necessary tools to accomplish their own intended goals with the application. The next stage is to implement a prototype, possibly defining a technological model, and validate it according to the specifications but also the intended purpose of the overall historical research in the context of the Crystal Palace Project, and write the dissertation document based on this research, its findings, and conclusions. The expected result is a working prototype that not only demonstrates the validity of this approach in the context of the research being done by the University of Bristol, but that can be successfully employed on subsequent virtual architecture projects using the Second Life or OpenSimulator platforms, namely, on the City & Spectacle project, at a later stage. ACKNOWLEDGMENT Special thanks to Prof. Leonel Caseiro Morgado of the University of Trs-os-Montes and Alto Douro (UTAD), who has shown relentless encouragement to me to proceed with my academic studies after finishing my mastership in software engineering under his supervision.

Many thanks to Prof. Eduardo Solteiro Pires from UTAD and Dr. Nic Earle from the University of Bristol who gently agreed to become co-supervisors of my thesis. Their patience with a student that has to constantly struggle with lack of time due to work at three different organisations is much appreciated. Finally, a tender word of thanks to my partner, Silvana Moreira, who agreed years ago to work together with me on virtual worlds, opening a company to commercially explore this fascinating new technological area, but certainly never expected that I soon got the wish to engage more fully in researching some aspects of the science behind virtual worlds. This often takes me away from my regular duties at home and at my three day jobs which she has patiently tolerated over the past years. REFERENCES
[1] Barcel, J. A., Forte, M., and Sanders, D. H. 2000 Virtual Reality in Archaeology. Oxford: Archaeopress. [2] Frisher, B., Niccolucci, F., Ryan, N. S., and Barcel, J. A. 2002. From CVR to CVRO: the Past, Present and Future of Cultural Virtual Reality. VAST 2000. BAR International Series 1075, 7-18. [3] Beacham, R. 2006. The London Charter. [4] Morgan, C. 2009. (Re)Building atalhyk: Changing Virtual Reality in Archaeology. Archaeologies. 5, 3, 468-487. DOI=10.1007/ s11759-009-9113-0. [5] Frisher, B. 2010. Rome Reborn: A Case Study in the Digital Reconstruction of Historic Cities. Virtual historic cities: reinventing urban research. [6] Dylla, K., Frisher, B., Mueller, P., Ulmer, A., and Haegler, S. 2009. Rome Reborn 2.0: A Case Study of Virtual City Reconstruction Using Procedural Modeling Techniques. CAA 2009. Making History Interactive. 37th Proceedings of the CAA Conference, 62-66. [7] Bogdanovych, A., Rodriguez-Aguilar, J. A., Simoff, S., and Cohen, A. 2010. Authentic Interactive Reenactment of Cultural Heritage with 3D Virtual Worlds and Artificial Intelligence. Applied Artificial Intelligence. 24, 6, 617-647. DOI=10.1080/08839514.2010.492172. [8] Goertzel, B., Pennachin, C., Geissweiller, N., Looks, M., Senna, A., Silva, W., Heljakka, A., and Lopes, C. 2008. An Integrative Methodology for Teaching Embodied Non-Linguistic Agents, Applied to Virtual Animals in Second Life [9] Proceeding of the 2008 conference on Artificial General Intelligence 2008: Proceedings of the First AGI Conference. 161-175 [10] 2009 THEATRON Final Report - September 2009. [11] Cmara, A., Pimentel, A., Murteira, H., and Rodrigues, P. 2009. City and Spectacle: A Vision of Pre-Earthquake Lisbon. 1st International Conference On Virtual Systems and Multimedia (VSMM 2009). [12] Earle, N. and Shelley, H. 2009. Pompeii in the Crystal Palace: Comparing Victorian and Modern Virtual, Immersive Environments. Electronic Visualisation and the Arts (EVA 2009).