Autonomous Robots 7, 57 (1999) c 1999 Kluwer Academic Publishers. Manufactured in The Netherlands.

Articial Neural Networks for Robot LearningGuest Editors Introduction

TOM ZIEMKE Department of Computer Science, University of Sk vde, Box 408, 54128 Sk vde, Sweden o o
tom@ida.his.se

NOEL E. SHARKEY Department of Computer Science, University of Shefeld, Shefeld S1 4DP, UK

N.Sharkey@dcs.shef.ac.uk

Introduction Since the early development of articial neural networks (ANNs) in the 1940s, they have been used for controlling autonomous robots. Grey Walter used hardwired neural network controllers for his turtles, Elmer and Elsie, built in the late 1940s (Grey Walter, 1950). These were simple reexive or reactive robots that mainly operated by phototaxis. Grey Walter also developed the CORA system (Grey Walter, 1953) that trained an ANN-controlled robot by classical conditioninga not often acknowledged piece of work that predates current reinforcement learning methods by over 20 years. His interest was in what we now call biorobotics (e.g., Ziemke and Sharkey, 1998). Sadly, this type of work all but vanished after the untimely death of neural networks in the late 1960s. It was not until the reincarnation of neural networks in the late 1980s and 1990s that ANNs began to reappear as controllers for autonomous robots (Bekey and Goldberg, 1993). Modern ANN techniques, due to their exibility, robustness to noise, and capacity for adaptation, have successfully been used for control and learning of various robotic tasks dealing with uncertain and partial information in the interaction with unpredictable and dynamic environments. Thus ANNs have found one of their most successful, but also one of the most challenging areas of application in the eld of robotics and autonomous agents, and autonomous robotics is discovering its roots. Clearly, the use of ANNs for robot learning is now much too broad a eld to be covered exhaustively in a single special issue. There are now a number of

collections on different aspects of the topic (Brooks et al., 1998; Omidvar and van der Smagt, 1997; Sharkey, 1997; Ziemkey and Sharkey, 1998). The papers in this issue are a representative sample of current work, and as a whole they illustrate the state of the art and the diversity of the eld. The robotic tasks addressed range from mobile robot navigation to arm control and the types of agents discussed range from teams of simulated soccer players to physical robots. Some of the papers reect biological inspiration while others have an engineering motivation.

The Papers in this Issue The rst two papers in this issue deal with the problem of forming representations of the environment for autonomous agent navigation, and both use inspiration from biology in the presented network architectures. Li and Svensson present a dynamic network architecture that allows autonomous agents to map landmarks, places and the spatial relation between them, and in particular they discuss the use of focus in environment mapping. Herrmann, Pawelzik and Geisel discuss the closely related problem of self-localization of autonomous robots, and present a framework for the construction of spatial representations from sensory input. The next three papers discuss conditioned or reinforcement learning in autonomous agents. Balkenius and Mor n present a model of classical conditioning, a e basic learning mechanism found in most animals. They argue that their model is well suited to robotic implementation since it can deal with arbitrarily complex

Ziemke and Sharkey Tony Savage, The Queens University of Belfast, UK Nestor Schmajuk, Duke University, Durham, USA Patrick van der Smagt, German Aerospace Center, Wessling, Germany Tom Wadden, Royal Institute of Technology, Stockholm, Sweden Jeremy Wyatt, University of Birmingham, UK Uwe Zimmer, GMDJapan Research Laboratory, Kitakyushu, Japan. References
Bekey, G. and Goldberg, K.Y. (Eds.) 1993. Neural Networks in Robotics, Kluwer: Boston, MA. Brooks, R., Grossberg, S., and Optican, L. (Eds.) 1998. Neural Control and Robotics: Biology and Technology, special issue of Neural Networks, Vol. 11(78). Grey Walter, W. 1950. An imitation of life. Scientic American, 182:4254. Grey Walter, W. 1953. The Living Brain, Norton: New York. Niklasson, L., Bod n, M., and Ziemke, T. (Eds.) 1998. ICANN 98 e Proceedings of the 8th International Conference on Articial Neural Networks, Springer: London, UK. Omidvar, O. and van der Smagt, P. (Eds.) 1997. Neural Systems for Robotics, Academic Press: Boston, MA. Sharkey, N.E. (Ed.) 1997. Robot Learning: The New Wave, special issue of Robotics and Autonomous Systems, Vol. 22(34). Ziemke, T. and Sharkey, N.E. (Eds.) 1998. Biorobotics, special issue of Connection Science, Vol. 10(34).

stimulus sequences. The work by Martin and Mill n, on a the other hand, has an engineering focus. They present an ANN controller for a robot arm that learns collisionfree motion from local sensory data while executing high-level descriptions of tasks. Wiering, Salustowicz and Schmidhuber present a novel approach to the integration of reinforcement learning and incomplete world models, and they compare their results to other reinforcement learning methods and evolutionary techniques. In the nal paper, Nol and Floreano present an extensive review of neurocomputational and robotic studies of the interaction between learning and evolution. They discuss the issue from two perspectives: (a) how the combination of learning and evolutionary techniques can lead to better performance in autonomous robotics, and (b) what insights about the interaction of learning and evolution in living organisms can be gained from robot studies. Acknowledgments The papers contained in this special issue are extended and revised versions of selected oral, poster and tutorial presentations that were part of the Autonomous Robotics and Adaptive Behavior track at the 8th International Conference on Articial Neural Networks (ICANN 98), Sk vde, Sweden, September 14, 1998 o (Niklasson et al., 1998). We would like to thank all speakers and participants in this event for interesting presentations and discussions. Furthermore, we would like to thank the members of the international review panel for their help in the preparation of this special issue. The panel consisted of: Charles Anderson, Colorado State University, Fort Collins, USA Neil Burgess, University College London, UK Inman Harvey, University of Sussex, Brighton, UK Manfred Huber, University of Massachusetts, Amherst, USA John Kazer, University of Shefeld, UK Ben Kr se, University of Amsterdam, The o Netherlands Guang Li, Chalmers Institute of Technology, Gothenburg, Sweden Ulrich Nehmzow, University of Manchester, UK Tony Prescott, University of Shefeld, UK

Tom Ziemke is a parallel and distributed processor: He currently works as a researcher and lecturer (US equivalent: assistant professor) at the University of Sk vde, Sweden, while completing a Ph.D. o in robotics and embodied cognition at the University of Shefeld, UK. He received a German diploma degree in Informatics in 1993 and a Swedish M.Sc. in Computer Science in 1994. During 1998 he co-chaired the ICANN 98 conference, co-organized an IEE workshop on biorobotics, co-edited a special issue on the same topic for the Connection Science journal, and also joined its editorial board. His main research interests are adaptive robot behavior, recurrent neural nets, cognitive representation, and the interaction between embodied agents (natural and articial) and their environments.

Guest Editors Introduction

Noel Sharkey is an interdisciplinary researcher. Currently a professor of Computer Science, he holds a Doctorate in Experimental Psychology from the University of Exeter, UK (1982). He is a Fellow of the British Computer Society, a Fellow of the Institution of Electrical Engineers, and a member of the British Experimental Psychology Society. He has worked as a research associate in Computer Science (AI) at Yale University, USA, and as a senior research associate in Psychology at Stanford University, USA, where he has also twice served as a visiting assistant professor. In 1984 he took up a new blood lectureship (English assistant professor) in Language and Linguistics at Essex University, UK and in 1989, became Reader

in Computer Science at Exeter, before accepting a chair in Computer Science at Shefeld University in 1994. He is Editor-in-Chief of the journal Connection Science, was Chairman of the IEE professional group A4 (Articial Intelligence) and is founding chairman of a new IEE professional group on Neural and Evolutionary Computing. Noel has published on a number of topics from experiments on human reaction time and memory to the fault diagnosis of engines. He is best known for his work on Neural Computing within AI and Cognitive Science (particularly psychological modelling, language and representation). At present, Noel is working in the area of self-learning and biologically inspired robotics as well as struggling with theories of interactive cognition.