Initial Research Plan

BiologicallyInspired
LearningofToolUsebyImitation
YAN WU
yan.wu08@imperial.ac.uk

Introduction
The natural desire of humans to explain and influence the world around them has
led to the development of complex human behaviours in sophisticated tool grasping and
manipulation. Since the birth of Unimate, the first fully autonomous electro-mechanical
system (commonly named robot), humans started relegating ever complex tasks to such
machines to assist their explorations of the world. The tasks that humans tend to give to
robots typically can be divided into two categories:
1. Tasks that robots perform better than humans robots designed to place
hundreds of thousands of electronic components per hour onto printed circuit
boards with much greater efficiency, accuracy and reliability.
2. Tasks that human dislike robots deployed to defuse roadside bombs or
improvised explosive devices which are dangerous to humans.
At the end of 2007, there were more than 1 million robots in operation
worldwide, with roughly half in Asia and 32% in Europe [1]. While humans are capable
of manipulating objects for a huge spectrum of tasks, most of these contemporary robots
are can only make use of a very limited set of pre-installed tools which lack the ability
to explore and acquire the knowledge and skills of using a novel tool. To harness the
massive latent labour power of these robots in novel tool use becomes an ever
increasingly challenging area of research.
Page 1 of 6

Initial Research Plan

EEE | ISN | BioART | Yan WU (00403784)

Motivation
Although deploying robots for task-specific applications has received much
attention in both the industry and the research community, little progress has been made
on methodologies to grasp and manipulate novel objects and tools. However, this scope
of research remains as a very competitive area for many industrial and military
applications as well as research in brain science.
For instance, in the mission of decommissioning a nuclear power plant, robots
are preferred to execute those tasks that can potentially subject the agents to harmful
radioactive radiations. Remote control of the robots in such environment might not be
feasible due to the presence of strong radioactivity. In unforeseen circumstances, robots
with the ability to manipulate new objects and tools can quickly learn to use new tools
and solve the problems without any hardware and software modifications.

Background
Much of the research work in the area focused on object grasping and
manipulation using gripper-based robots. Saxena et al addressed the problem of novel
object grasping by the use of 2D templates and techniques in computer vision [2] while
Grosso and Vercelli tackled it by 3D reconstruction of novel objects [3]. Indubitably,
these works presented effective methodologies in certain applications of object
grasping. However, extension of these works into the problem of tool use is pessimistic
due to either the assumptions imposed by the authors or the tremendous computational
complexity for the robot to handle in real-time.
In contrast, the work by Dearden and Demiris approached a similar problem
from the fundamentals of computational neuroscience. In [4], Demiris and Dearden
presented the concept of a robot developmental pathway, which couples exploration
(unsupervised learning), and hierarchical learning by imitation (reinforcement learning)
at the different stages of robotic learning analogous to that of a child. This direction of
work has received ever increasing attention in the research community owing to the
great advantages of exploiting the effective and efficient biological learning systems for
robots. Mas et al presented their work on the extended grasping behaviour for robot
using the human grasp taxonomy [5]. However, their proposed methodology was only
evaluated in a simulated environment. At the other end, Mouri et al proposed their work
Page 2 of 6

Initial Research Plan

EEE | ISN | BioART | Yan WU (00403784)

on imitation of human grasping reflex for robotic object grasping strategy [6]. This
research implemented a haptic-sensor based decision rule for the object grasping
mechanism while ignoring the importance of visual information.
Around the globe, the number of humanoid robots with comparable
functionalities of human hands is very limited. A large fraction of these robots belong to
the Japanese universities and companies which make the research much of a secret.
Moreover, there has been little work done directly on tool use and most related work
was done in a simulation environment. In 2005, Amant et al presented their simulated
work on tool use for autonomous agents [7]. However, this work stays very much
theoretical and philosophical which might not be immediately applicable to a physical
robot. Undeniably, a handful of robotic laboratories have been pushing the frontiers of
model design for learning novel tool-use. Nabeshima and Kuniyoshi et al implemented
an adaptive model for robotic tool use based purely on visual feedback [8]. This work
demonstrated its only limited ability of the robot to use a new but much simplified tool
to reach distant objects. In comparison, the learning model of tool use proposed by
Stoytchev powered his robot to explore the right tool (the shape of the sticks) to move
an object to a desired location [9]. However, the authors in both works did not evaluate
their models for the use of more complex tools, such as scissors and screwdrivers.

Objectives
The area of computational neuroscience studies the physiology and dynamics of
functional and biological neural systems. Hypotheses and models of these studies
capture essential features in learning, memory and signal processing etc. which have
direct impacts on humanoid robotics. For example, Iriki discussed how monkeys
learned to use a new tool [10] which is potentially useful if a computational model can
be drawn from his hypotheses.
Machine learning approach in robotics is capable of bridging the difference
between biological systems and electro-mechanical systems. Moreover, an extensive
range of the machine learning models is biologically inspired. The HAMMER
architecture developed by Demiris and Khadhouri [11] is an illustration of actionperceptual model using reinforcement learning. In fact, the use of machine learning
techniques in vision-based learning (computer vision) has been a matured model widely
Page 3 of 6

Initial Research Plan

EEE | ISN | BioART | Yan WU (00403784)

accepted in the field.

The area of HRI investigates algorithms for enabling robots to assist human
activities outside the traditional factory floor, for example in rehabilitation and
assistance for disabled people, as well as domestic activities. This approach gives rise to
possible discovery of new algorithm and heuristics in assisting the robot to learn to use
new tools.
Weighing the novelty and limitations of previous research, we believe that
combination of computational neuroscience principles, plausible machine learning
models and human-robot interaction (HRI) will lead to the creation of an integrated
biologically-inspired model for humanoid robot to acquire skills in using novel tools.
This research will focus on neurally-inspired algorithms to enable Learning by
imitation. Machine vision, machine learning and motor control algorithms that mimic
the human ability to imitate the actions of other people will be developed to allow a
human to demonstrate a task, and have the robot learn it by imitating. The research will
develop a computational model of the human neural processes of translating visual and
haptic signals (for example picking an object) to the corresponding motor signals that
will allow a robot to imitate the task of tool use. This model will then be validated on
the iCub, a newly acquired humanoid robot developed by the RobotCub Consortium1
funded by the European Commission.
The proposed research work can be classified into the follow three main
components. The time frame for the investigation of each component with the listed
objectives is approximately a year.

Autonomous Robot-Object Interaction This section investigates possible

models for robot to recognise, discriminate between, grasp and manipulate
objects without human instructions.
1.

Devise an effective algorithm to recognise and discriminate different objects

placed at the same background

2.

Generate suitable model to manoeuvre the robot arm to reach objects

3.

Collect human haptic and visual data on grasping novel objects

4.

Generate an inductive computational model based on the collected data and

henceforth a learning architecture for the iCub

http://www.robotcub.org/
Page 4 of 6

Initial Research Plan

EEE | ISN | BioART | Yan WU (00403784)

5.

Propose approaches for classified object grasping

6.

Integrate the shortlisted approaches by HAMMER architecture and test the

implemented model

Object-to-Tool Transition This section aims to translate the human behaviours

of learning a new tool to computational models that describe such behaviours.
1.

Collect human haptic and visual data on novel tool exploration

2.

Collect human haptic and visual data on familiar tool use

3.

Generate computational models for the above two human behaviours and
propose a hypothesis on the mechanism bridging from Point 1 to Point 2

Autonomous Novel Tool Use This section integrates the research findings in
the above mentioned objectives and puts forth a learning architecture for novel
tool use in robots.
1.

Build a number of models for use of specific tools to be tested on the iCub
based on the proposed hypothesis

2.

Propose, implement and test an integrated model for novel too use by
HAMMER approach

Reference
[1]

International Federation of Robotics: Robots Today and Tomorrow: IFR presents

the 2007 World Robotics Statistics Survey, World Robotics, 29 Nov 2007.

[2]

Saxena, A., Driemeyer, J., and Ng, A.Y., Robotic Grasping of Novel Objects
using Vision, International Journal of Robotics Research (IJRR), 2008.

[3]

Grosso, E., and Vercelli, G., Grasping strategies for reconstructed unknown 3D
objects, Intelligent Robots and Systems '91. 'Intelligence for Mechanical Systems,
Proceedings IROS '91. IEEE/RSJ International Workshop on, Vol. 1, pp. 70 -75,
Nov 1991.

[4]

Demiris, Y., and Dearden, A., From motor babbling to hierarchical learning by
imitation: a robot developmental pathway, Proceedings of EPIROB-2005, pp. 3137, Japan, July 2005.

Page 5 of 6

Initial Research Plan

[5]

EEE | ISN | BioART | Yan WU (00403784)

Mas, R., Boulic, R., and Thalmann, D., Extended Grasping Behavior for
Autonomous Human Agents, ACM Conference on Autonomous Agents'97, pp.
146-160, 1997.

[6]

Mouri, T., Kawasaki, H., and Ito, S., Unknown Object Grasping Strategy
Imitating Human Grasping Reflex for Anthropomorphic Robot Hand, Journal of
Advanced Mechanical Design, Systems, and Manufacturing, Vol. 1, No. 1, pp.111, 2007.

[7]

St. Amant, R., and Wood, A. B., Tool use for autonomous agents, Proceedings
of the National Conference on Artificial Intelligence 2005, pp. 184-189, 2005.

[8]

Nabeshima, C., Kuniyoshi, Y., and Lungarella, M., Adaptive body schema for
robotic tool-use, Advanced Robotics, Vol. 20, No. 10, pp. 1105-1126(22), 2006.

[9]

Stoytchev, A., "Learning the Affordances of Tools using a Behavior-Grounded

Approach", Affordance-Based Robot Control, Springer Lecture Notes in Artificial
Intelligence (LNAI) 4760, pp. 140-158, 2008.

[10] Okanoya, K., Tokimoto, N., Kumazawa, N., Hihara, S., and Iriki, A., "Tool-use
training in a species of rodent: the emergence of an optimal motor strategy and
functional understanding", PLoS One, 3(3), e1860: 1-8, 26 Mar 2008.
[11] Demiris, Y., and Khadhouri, B., Hierarchical attentive multiple models for
execution and recognition of actions, Robotics and autonomous systems, Vol. 54,
pp. 361-369, 2006.
[12] Dayan, P., and Abbott, L.F., Theoretical Neuroscience: Computational and
Mathematical Modeling of Neural Systems, The MIT Press, 2001.

This initial research plan was written on the 30 November 2008 by Yan Wu, the undersigned first year PhD student
from the BioART team of the Intelligent Systems and Network group, Electrical and Electronic Engineering
Department. This proposal has been read, corrected and endorsed by Yan Wus supervisor Dr. Yiannis Demiris.

_________________________

_________________________

Yan Wu

Dr. Yiannis Demiris

Page 6 of 6