A MOBIUS AUTOMATON: AN APPLICATION
OF ARTIFICIAL INTELLIGENCE TECHNIQUES
by
N i l s J . Nilsson
Stanford Research I n s t i t u t e
Menlo Park, C a l i f o r n i a
Summary
A research p r o j e c t applying a r t i f i c i a l i n t e l -
ligence techniques to the development of i n t e -
grated robot systems Is d e s c r i b e d . The experimen-
t a l f a c i l i t y c o n s i s t s of an SDS-940 computer and
associated programs c o n t r o l l i n g a wheeled v e h i c l e
t h a t c a r r i e s a TV camera and other sensors. The
primary emphasis is on the development of a system
of programs for processing sensory data from the
v e h i c l e , f o r s t o r i n g relevant i n f o r m a t i o n about
the environment, and f o r planning the sequence of
motor a c t i o n s necessary to accomplish tasks in the
environment. A t y p i c a l task performed by our
present system r e q u i r e s the robot v e h i c l e to r e -
arrange (by pushing) simple objects in i t s e n v i r o n -
ment .
A novel f e a t u r e of our approach is the use of a
formal theorem-proving system to plan the execution
of h i g h - l e v e l functions as a sequence of o t h e r ,
perhaps l o w e r - l e v e l , f u n c t i o n s . The execution of
these, i n t u r n , requires a d d i t i o n a l planning a t
lower l e v e l s . The main theme of the research is
the i n t e g r a t i o n of the necessary planning systems,
models of the w o r l d , and sensory processing sys-
tems i n t o an e f i i c i e n t whole capable of performing
a wide range of tasks in a r e a l environment.
Key Words
Robot
Robot System
V i s u a l Processing
Problem-Solving
Question-Answering
Theorem-Proving
Models of the World
Planning
Scene A n a l y s i s
Mobile Automaton
Acknowledgment
At l e a s t two dozen persons at the Stanford
Research I n s t i t u t e have made s u b s t a n t i a l c o n t r i b u -
t i o n s to the p r o j e c t that tne author has the good
f o r t u n e t o describe i n t h i s paper. A l l o f u s ex-
press our a p p r e c i a t i o n to the Rome A i r Development
Center and the Advanced Research P r o j e c t s Agency,
who supported t h i s research under Contract No.
F 30602-69-C-0056.
I Introduction
At the Stanford Research I n s t i t u t e we are im-
plementing a f a c i l i t y f o r the experimental study of
robot systems. The f a c i l i t y c o n s i s t s of a t i m e -
shared SDS-940 computer, several core-loads of p r o -
grams, a robot v e h i c l e , and s p e c i a l i n t e r f a c e
equipment.
-509-
Several e a r l i e r reports' 1 " and papers2- 4 de-
s c r i b i n g the p r o j e c t have been w r i t t e n ; in t h i s
paper we s h a l l describe i t s status as of e a r l y
1969 and discuss some of our f u t u r e p l a n s .
The robot v e h i c l e i t s e l f is shown in F i g . 1.
It is propelled by two stepping motors independent-
ly d r i v i n g a wheel on e i t h e r side of the v e h i c l e .
It c a r r i e s a v i d i c o n t e l e v i s i o n camera and o p t i c a l
range-finder in a movable " h e a d . " Control l o g i c on
board the v e h i c l e routes commands from the computer
to the appropriate a c t i o n s i t e s on the v e h i c l e .
In a d d i t i o n to the d r i v e motors, there are motors
to c o n t r o l the camera focus and i r i s s e t t i n g s and
the t i l t angle of the head. (A motor to pan the
head is not yet used by present programs.) Other
computer commands arm or disarm i n t e r r u p t l o g i c ,
c o n t r o l power switches, and request readings of
the status of various r e g i s t e r s on the v e h i c l e .
Besides the t e l e v i s i o n camera and r a n g e - f i n d e r
sensors, several " c a t - w h i s k e r " touch-sensors are
attached to the v e h i c l e ' s perimeter. These touch
sensors enable the v e h i c l e to know when it bumps
i n t o something. Commands from the SDS-940 computer
to the v e h i c l e and i n f o r m a t i o n from the v e h i c l e to
the computer are sent over two s p e c i a l radio l i n k s ,
one f o r narrow-band t e l e m e t e r i n g and one f o r t r a n s -
mission of the TV video from the v e h i c l e to the
computer.
The purpose of our robot research at SRI is to
study processes f o r the r e a l - t i m e c o n t r o l of a
robot system that i n t e r a c t s w i t h a complex e n v i r o n -
ment. We want the v e h i c l e to be able to perform
various tasks t h a t r e q u i r e it to move about in i t s
environment or to rearrange o b j e c t s . In order to
accomplish a wide v a r i e t y of tasks r a t h e r than a
few s p e c i f i c ones, a robot system must have very
general methods. What is required is the i n t e g r a -
t i o n in one system of many of the a b i l i t i e s t h a t
are usually found separately i n i n d i v i d u a l A r t i f i -
c i a l I n t e l l i g e n c e programs.
We can group most of the needed a b i l i t i e s i n t o
three broad classes: (1) p r o b l e m - s o l v i n g , (2) mod-
e l l i n g , and (3) p e r c e p t i o n :
(1) Problem-Solving
A robot system accomplishes the tasks given it
by performing a sequence of p r i m i t i v e a c t i o n s , such
as wheel motions and camera readings. For e f f i -
ciency, a task should f i r s t be analyzed i n t o a
sequence of p r i m i t i v e actions c a l c u l a t e d to have
the desired e f f e c t . This process of task a n a l y s i s
i s o f t e n c a l l e d p l a n n i n g , because i t i s accom-
p l i s h e d before the robot begins to a c t . Obviously,
in order to p l a n , a robot system must "know" about
the e f f e c t s o f i t s a c t i o n s .
 (2) Modelling
A body o f k n o w l e d g e a b o u t t h e e f f e c t s o f a c -
t i o n s i s a type o f model o f t h e w o r l d . A robot
p r o b l e m - s o l v i n g system uses t h e i n f o r m a t i o n s t o r e d
i n t h e m o d e l t o c a l c u l a t e what sequence o f a c t i o n s
w i l l cause t h e w o r l d t o b e i n a d e s i r e d s t a t e . As
t h e w o r l d c h a n g e s , e i t h e r b y t h e r o b o t ' s own a c -
t i o n s o r f o r o t h e r r e a s o n s , t h e m o d e l must b e u p -
dated to record these changes. A l s o , new i n f o r m a -
t i o n l e a r n e d a b o u t t h e w o r l d s h o u l d b e added t o
the model.
(3) Perception
Sensors are necessary to g i v e a r o b o t system
new i n f o r m a t i o n a b o u t t h e w o r l d . B y f a r t h e most
i m p o r t a n t sensory system i s v i s i o n , s i n c e i t a l -
l o w s d i r e c t p e r c e p t i o n o f a good s i z e d p i e c e o f
t h e w o r l d beyond t h e r a n g e o f t o u c h . Since we a s -
sume t h a t a r o b o t s y s t e m w i l l n o t a l w a y s have
s t o r e d i n i t s model e v e r y d e t a i l o f t h e exact c o n -
f i g u r a t i o n o f i t s w o r l d and t h u s c a n n o t know p r e -
c i s e l y the e f f e c t s o f i t s every a c t i o n , i t also
needs s e n s o r s w i t h w h i c h t o c h e c k p r e d i c t e d c o n -
sequences a g a i n s t r e a l i t y a s i t e x e c u t e s I t s p l a n s .
The i n t e g r a t i o n o f s u c h a b i l i t i e s i n t o a
s m o o t h l y - r u n n i n g , e f f i c i e n t system p r e s e n t s b o t h
i m p o r t a n t c o n c e p t u a l p r o b l e m s and s e r i o u s p r a c t i c a l
challenges. For e x a m p l e , i t would b e i n f e a s i b l e
f o r a s i n g l e p r o b l e m - s o l v i n g system ( u s i n g a s i n g l e
model) t o attempt t o c a l c u l a t e the long chains o f
p r i m i t i v e a c t i o n s needed t o p e r f o r m l e n g t h y t a s k s .
A way a r o u n d t h i s d i f f i c u l t y i s t o p r o g r a m a number
o f c o o r d i n a t i n g " a c t i o n - u n i t s , " e a c h w i t h i t s own
p r o b l e m - s o l v i n g s y s t e m and m o d e l , and e a c h r e s p o n -
s i b l e f o r p l a n n i n g and e x e c u t i n g a s p e c i a l i z e d
function. I n p l a n n i n g how t o p e r f o r m i t s p a r t i c u -
l a r f u n c t i o n , e a c h a c t i o n - u n i t knows t h e e f f e c t s o f
executing f u n c t i o n s handled by various of the other
action-units. W i t h t h i s k n o w l e d g e i t composes i t s
p l a n a s a sequence o f o t h e r f u n c t i o n s ( w i t h t h e
a p p r o p r i a t e a r g u m e n t s ) and l e a v e s t h e p l a n n i n g r e -
q u i r e d f o r each o f t h e s e f u n c t i o n s u p t o t h e
a c t i o n - u n i t s r e s p o n s i b l e f o r e x e c u t i n g them a t t h e
time they are to be executed.
Such a s y s t e m o f i n t e r d e p e n d e n t a c t i o n - u n i t s
i m p l i e s c e r t a i n a d d i t i o n a l p r o b l e m s i n v o l v i n g com-
m u n i c a t i o n o f i n f o r m a t i o n and t r a n s f e r o f c o n t r o l
between u n i t s . When such a s y s t e m i s I m p l e m e n t e d
o n a s e r i a l c o m p u t e r w i t h l i m i t e d c o r e memory, o b -
v i o u s p r a c t i c a l d i f f i c u l t i e s a r i s e connected w i t h
s w a p p i n g p r o g r a m segments i n and o u t o f c o r e and
handling interrupts in real time. The c o o r d i n a t e d
a c t i o n - u n i t scheme s e r v e s a s a u s e f u l g u i d e i n e x -
p l a i n i n g the o p e r a t i o n o f o u r system, even though
p r a c t i c a l n e c e s s i t i e s have d i c t a t e d o c c a s i o n a l d e -
v i a t i o n s f r o m t h i s scheme i n o u r i m p l e m e n t a t i o n .
In the next s e c t i o n we s h a l l d i s c u s s the problem-
s o l v i n g p r o c e s s e s and m o d e l s a s s o c i a t e d w i t h some
s p e c i f i c f u n c t i o n s o f t h e p r e s e n t SRI r o b o t s y s t e m .
-510-
II SOME SPECIFIC FUNCTIONS OF THE ROBOT
SYSTEM AND THEIR ASSOCIATED PROBLEM-
SOLVING PROCESSES AND MODELS
A. Low L e v e l F u n c t i o n s
The r o b o t s y s t e m i s c a p a b l e o f e x e c u t i n g a
number o f f u n c t i o n s t h a t v a r y i n c o m p l e x i t y f r o m
the simple a b i l i t y t o t u r n the d r i v e wheels a c e r -
t a i n number o f s t e p s t o t h e a b i l i t y t o c o l l e c t a
number of b o x e s by p u s h i n g them to a common a r e a
of the room. The o r g a n i z a t i o n o f t h e s e f u n c t i o n a l
a c t i o n - u n i t s i s not s t r i c t l y h i e r a r c h i c a l , a l -
though f o r d e s c r i p t i v e convenience we w i l l d i v i d e
them i n t o two c l a s s e s : l o w l e v e l and h i g h l e v e l
functions.
Of the f u n c t i o n s that we s h a l l mention h e r e ,
the s i m p l e s t are c e r t a i n p r i m i t i v e assembly l a n -
guage r o u t i n e s f o r m o v i n g t h e w h e e l s , t i l t i n g t h e
h e a d , r e a d i n g a TV p i c t u r e , and so o n . Two e x -
amples of t h e s e a r e MOVE and TURN; MOVE c a u s e s t h e
v e h i c l e to r o l l in a s t r a i g h t l i n e by t u r n i n g both
d r i v e w h e e l s i n u n i s o n , and TURN c a u s e s t h e v e h i c l e
t o r o t a t e about i t s c e n t e r b y t u r n i n g t h e d r i v e
wheels in o p p o s i t e d i r e c t i o n s . The a r g u m e n t s o f
MOVE and TURN a r e t h e number of s t e p s t h a t t h e
d r i v e w h e e l s a r e t o t u r n (each s t e p r e s u l t i n g i n a
v e h i c l e m o t i o n o f 1/32 i n c h ) and " s t a t u s " a r g u -
ments t h a t a l l o w q u e r i e s t o b e made a b o u t w h e t h e r
o r n o t t h e f u n c t i o n has been c o m p l e t e d . *
Once b e g u n , t h e e x e c u t i o n o f any f u n c t i o n
proceeds e i t h e r u n t i l i t I s completed i n i t s
n o r m a l manner o r u n t i l i t i s h a l t e d b y one o f a
number o f " a b n o r m a l " c i r c u m s t a n c e s , s u c h a s t h e
v e h i c l e bumping i n t o u n e x p e c t e d o b j e c t s , o v e r l o a d
c o n d i t i o n s , r e s o u r c e e x h a u s t i o n , and s o o n . Under
o r d i n a r y o p e r a t i o n , i f e x e c u t i o n o f MOVE r e s u l t s
i n a bump, m o l l o n i s s t o p p e d a u t o m a t i c a l l y b y a
s p e c i a l mechanism o n t h e v e h i c l e . T h i s mechanism
can be o v e r r i d d e n by a s p e c i a l i n s t r u c t i o n from
t h e c o m p u t e r , h o w e v e r , t o e n a b l e t h e r o b o t t o push
ob J e c t s .
The p r o b l e m - s o l v i n g s y s t e m s f o r MOVE and TURN
a r e t r i v i a l ; t h e y need o n l y t o c a l c u l a t e what s i g -
n a l s s h a l l b e sent t o r e g i s t e r s a s s o c i a t e d w i t h
t h e m o t o r s i n o r d e r t o c o m p l e t e t h e d e s i r e d number
of steps.
At a l e v e l j u s t above MOVE and TURN is a f u n c -
t i o n whose e x e c u t i o n c a u s e s t h e v e h i c l e t o t r a v e l
d i r e c t l y to a point s p e c i f i e d by a p a i r of ( x , y )
coordinates. T h i s f u n c t i o n i s implemented i n the
FORTRAN r o u t i n e LEG. The model used by LEG c o n -
t a i n s i n f o r m a t i o n about the r o b o t ' s p r e s e n t ( x , y )
l o c a t i o n and h e a d i n g r e l a t i v e t o a g i v e n c o o r d i n a t e
s y s t e m and i n f o r m a t i o n a b o u t how f a r t h e v e h i c l e
t r a v e l s f o r each step a p p l i e d t o the s t e p p i n g
motors. This information is stored along with
some o t h e r s p e c i a l c o n s t a n t s i n a s t r u c t u r e c a l l e d
t h e PARAMETER MODEL. Thus f o r a g i v e n ( x , y )
* Our i m p l e m e n t a t i o n a l l o w s a p r o g r a m c a l l i n g
r o u t i n e s l i k e MOVE o r TURN t o r u n i n p a r a l l e l
w i t h the motor f u n c t i o n s they i n i t i a t e .
d e s t i n a t i o n as an argument of LEG, LEG's problem-
s o l v i n g system c a l c u l a t e s appropriate arguments
f o r a TURN and MOVE sequence and then executes
t h i s sequence. Predicted changes in the r o b o t ' s
l o c a t i o n and heading caused by execution of MOVE
and TURN are used to update the PARAMETER MODEL.
Ascending one more l e v e l in our system, we e n -
counter a group of FORTRAN " t w o - l e t t e r " r o u t i n e s
whose execution can be i n i t i a t e d from the t e l e -
t y p e . Our a c t i o n - u n i t system ceases to be s t r i c t -
ly h i e r a r c h i c a l at t h i s p o i n t , since some of the
t w o - l e t t e r commands can cause others to be exe-
cuted .
One of these t w o - l e t t e r commands, EX, takes as
an argument a sequence of ( x , y ) coordinate p o s i -
t i o n s . Execution of EX causes the robot to t r a v e l
from i t s present p o s i t i o n d i r e c t l y t o the f i r s t
p o i n t in the sequence, thence d i r e c t l y to the
second, and so on u n t i l the robot reaches the l a s t
point in the sequence. The problem-solving system
f o r EX simply needs to know the e f f e c t caused by
execution of a LEG program and composes a chain of
LEG r o u t i n e s , each w i t h arguments provided by the
successive p o i n t s s p e c i f i e d in the sequence of
points. Under o r d i n a r y o p e r a t i o n , if one of these
LEG r o u t i n e s is h a l t e d due to a bump, EX backs the
v e h i c l e up s l i g h t l y and then h a l t s . A s p e c i a l
f e a t u r e o f our implementation i s the a b i l i t y t o
arm and s e r v i c e i n t e r r u p t s (such as caused by
bumps) at the FORTRAN programming l e v e l .
Another t w o - l e t t e r command, P I , causes a p i c -
t u r e to be read a f t e r the TV camera has been aimed
at a s p e c i f i e d p o s i t i o n on the f l o o r . The problem-
s o l v i n g system f o r PI thus c a l c u l a t e s the appro-
p r i a t e arguments f o r a TURN r o u t i n e and a head-
t i l t i n g r o u t i n e ; PI then causes these to be exe-
c u t e d , reads in a p i c t u r e from the TV camera, and
performs processing necessary to e x t r a c t informa-
t i o n about empty areas on the f l o o r . ( D e t a i l s of
the p i c t u r e processing programs of the robot sys-
tem are described in Sec. I l l below.)
The a b i l i t y to t r a v e l by the shortest route to
a s p e c i f i e d goal p o s i t i o n along a path c a l c u l a t e d
to avoid bumping i n t o obstacles is provided by the
t w o - l e t t e r command TE. Execution of TE i n v o l v e s
the c a l c u l a t i o n of an a p p r o p r i a t e sequence of
p o i n t s f o r EX and the execution of EX. This appro-
p r i a t e sequence is c a l c u l a t e d by a s p e c i a l problem-
s o l v i n g system embodied in the t w o - l e t t e r command
PL.
The source of i n f o r m a t i o n about the world used
by PL is a planar map of the room c a l l e d the GRID
MODEL. The GRID MODEL is a h i e r a r c h i c a l l y o r g a -
nized system o f f o u r - b y - f o u r g r i d c e l l s . I n i t i a l l y
the "whole w o r l d " is represented by a f o u r - b y - f o u r
array of c e l l s . A given c e l l can be e i t h e r empty
(of o b s t a c l e s ) , f u l l , p a r t i a l l y f u l l , o r unknown.
Each p a r t i a l l y f u l l c e l l i s f u r t h e r subdivided i n t o
a f o u r - b y - f o u r a r r a y o f c e l l s , and s o on, u n t i l a l l
p a r t i a l l y f u l l c e l l s represent areas o f some s u i t -
ably s m a l l s i z e . (Our present system s p l i t s c e l l s
down to a depth of three l e v e l s , r e p r e s e n t i n g a
smallest area of about 12 I n c h e s . )
-511-
Special "model maintenance" programs insure
t h a t the GRID MODEL is a u t o m a t i c a l l y updated by
i n f o r m a t i o n about empty and f u l l f l o o r areas
gained by e i t h e r successful execution or i n t e r -
r u p t i o n of MOVE commands.
The PL program f i r s t uses the GRID MODEL to
compute a network or graph of "nodes." The nodes
correspond to p o i n t s in the room opposite corners
of o b s t a c l e s ; the shortest path to a goal p o i n t
w i l l then pass through a sequence of a subset of
these nodes. In F i g . 2 we show a complete GRID
MODEL of a room c o n t a i n i n g three o b j e c t s . The
r o b o t ' s p o s i t i o n , marked " R , " and the goal p o s i -
t i o n , marked " G , " together w i t h the nodes A,B,C,D,
E , F , H , I , J and K are shown o v e r l a i d on the GRID
MODEL. The program PL then determines that the
shortest path is the sequence of p o i n t s , R , F , I ,
and G by employing an o p t i m a l graph-searching a l g o -
r i t h m developed by H a r t , et a l . 5
If the GRID MODEL map of the world contains
unknown space, PL must decide whether or not to
t r e a t t h i s unknown space as f u l l or empty. Cur-
r e n t l y , PL m u l t i p l i e s the length of any segment of
the route through unknown space by a parameter k.
Thus if k = l , unknown space is t r e a t e d as empty;
values of k g r e a t e r than u n i t y cause routes through
known empty space to be p r e f e r r e d to p o s s i b l y
s h o r t e r routes through unknown space.
Execution of TE is accomplished by f i r s t r e a d -
ing and processing a p i c t u r e (using PI w i t h the
camera aimed at the goal p o s i t i o n ) and t a k i n g a
r a n g e - f i n d e r r e a d i n g . The i n f o r m a t i o n about f u l l
and empty f l o o r areas thus gained is added to the
GRID MODEL. A route based on the updated GRID
MODEL is then planned using PL, and then EX is exe-
cuted using the arguments c a l c u l a t e d by PL. If the
EX c a l l e d by TE is h a l t e d by a bump, a procedure
attempts to manuever around the i n t e r f e r i n g ob-
s t a c l e , and then TE is c a l l e d to s t a r t over a g a i n .
Thus, v i s i o n is used o n l y at the beginning of a
journey and when unexpected bumps occur along the
journey.
Although our present robot system does not
have manipulators w i t h which to pick up o b j e c t s ,
it can move o b j e c t s by pushing them. The funda-
mental a b i l i t y to push o b j e c t s from one place to
another is programmed i n t o another t w o - l e t t e r
FORTRAN r o u t i n e , c a l l e d PU. Execution of PU causes
the robot to push an o b j e c t from one named p o s i t i o n
along a s t r a i g h t l i n e path to another named p o s i -
t i o n . The program PU takes f i v e arguments: the
( x , y ) coordinates of the o b j e c t to be pushed, the
" s i z e " or maximum extent of the o b j e c t about i t s
center of g r a v i t y , and the ( x , y ) coordinates of the
spot to which the o b j e c t is to be pushed. The
problem-solving system f o r PU assembles an EX, a
TURN, and two MOVE commands i n t o a sequence whose
execution w i l l accomplish the d e s i r e d push. F i r s t
a l o c a t i o n from which the robot must begin pushing
the o b j e c t is computed. Then PL is used to p l a n a
route to t h i s goal l o c a t i o n . The sequence of
p o i n t s along the route serves as the argument f o r
EX t h a t is then executed. (Should EX be stopped by
a bump, PU is s t a r t e d over a g a i n . ) N e x t , PU's
p r o b l e m - s o l v i n g system (using the PARAMETER model)
c a l c u l a t e s an argument f o r TURN t h a t w i l l p o i n t
the robot i n the d i r e c t i o n t h a t the o b j e c t i s t o
be pushed. A l a r g e argument is provided f o r the
f i r s t MOVE command so t h a t when it is executed, it
w i l l bump i n t o the o b j e c t to be pushed and a u t o -
m a t i c a l l y h a l t . A f t e r the bump and h a l t , the
automatic stopping mechanism on the v e h i c l e is
o v e r r i d d e n and the next MOVE command is executed
w i t h an argument c a l c u l a t e d to push the o b j e c t the
desired distance.
B. Higher Level Functions
As we ascend to h i g h e r l e v e l f u n c t i o n s , the
r e q u i r e d p r o b l e m - s o l v i n g processes must be more
powerful and g e n e r a l . We want our robot system to
have the a b i l i t y t o perform tasks p o s s i b l y r e q u i r -
i n g q u i t e complex l o g i c a l d e d u c t i o n s . What is
needed f o r t h i s type of p r o b l e m - s o l v i n g is a gen-
e r a l language in which to s t a t e problems and a
powerful search s t r a t e g y w i t h which t o f i n d s o l u -
t i o n s . We have chosen the language of f i r s t - o r d e r
p r e d i c a t e c a l c u l u s i n which t o s t a t e h i g h l e v e l
problems f o r the r o b o t . These problems are then
solved by an a d a p t a t i o n of a "Question Answering
System" QA-3, based on " r e s o l u t i o n " theorem-
proving methods.6-9
As an example of a h i g h l e v e l problem f o r the
r o b o t , consider the t a s k of moving (by pushing)
t h r e e o b j e c t s to a common p l a c e . This t a s k is an
example of one t h a t has been executed by our p r e s -
ent system. If the o b j e c t s to be pushed a r e , say,
OBI, 062, and 0B3, then the problem of moving them
to a common place can be s t a t e d as a " c o n j e c t u r e "
f o r QA-3:
C7p,s){POSITION ( 0 B l , p , s )
A POSITION (OB2,p,s)
A POSITION ( 0 B 3 , p , s ) }
(That i s , "There e x i s t s a s i t u a t i o n s and a
place p, such t h a t OBI, 0B2, and 0B3 are a l l at
place p in s i t u a t i o n s . " ) The task f o r QA-3 is to
" p r o v e " t h a t t h i s c o n j e c t u r e f o l l o w s from "axioms"
t h a t describe the present p o s i t i o n of o b j e c t s and
the e f f e c t s o f c e r t a i n a c t i o n s .
Our f o r m u l a t i o n of these problems f o r the
theorem-prover i n v o l v e s s p e c i f y i n g the e f f e c t s o f
a c t i o n s in terms of f u n c t i o n s t h a t map s i t u a t i o n s
i n t o new s i t u a t i o n s . For example, the f u n c t i o n
PUSH ( x , p , s ) maps the s i t u a t i o n s i n t o the s i t u a -
t i o n r e s u l t i n g by pushing o b j e c t x i n t o place p.
Thus two axioms needed by QA-3 to solve the push-
i n g problem a r e :
(Vx,p,s){POSITION ( x , p , PUSH ( x , p , s ) ) }
and
(Vx,y,p,q,s){POSITION ( x , p , s ) A ~ SAME ( x , y )
*=> POSITION (x,p,PUSH ( y , q , s ) ) } .
The f i r s t o f these axioms s t a t e s t h a t i f i n a n
a r b i t r a r y s i t u a t i o n s, an a r b i t r a r y object x is
pushed to an a r b i t r a r y place p, then a new
-512-
s i t u a t i o n , PUSH ( x , p , B ) , w i l l r e s u l t i n which the
o b j e c t x w i l l be at p o s i t i o n p. The second axiom
Btates t h a t any o b j e c t w i l l stay i n i t s o l d place
in the new s i t u a t i o n r e s u l t i n g by pushing a d i f -
ferent object.
In a d d i t i o n to the two axioms Just mentioned,
we would have o t h e r s d e s c r i b i n g the present p o s i -
tions of o b j e c t s . For example, i f OBI i s a t c o -
o r d i n a t e p o s i t i o n ( 3 , 5 ) i n the present s i t u a t i o n ,
we would have:
POSITION (OBI, ( 3 , 5 ) , PRESENT) .
(This i n f o r m a t i o n is provided a u t o m a t i c a l l y by
r o u t i n e s t h a t scan the GRID MODEL, g i v i n g names to
c l u s t e r s o f f u l l c e l l s and n o t i n g the l o c a t i o n s o f
these c l u s t e r s . )
I n proving the t r u t h o f the c o n j e c t u r e , the
theorem-prover used by QA-3 also produces the
place p and s i t u a t i o n s t h a t e x i s t . That i s , QA-3
determines t h a t the d e s i r e d s i t u a t i o n s i s :
s = PUSH (0B3, ( 3 , 5 ) , PUSH (0B2, ( 3 , 5 ) , PRESENT)).
A l l of the i n f o r m a t i o n about the world used by QA-3
i n s o l v i n g t h i s problem i s stored i n the form o f
axioms in a s t r u c t u r e c a l l e d the AXIOM MODEL. In
g e n e r a l , the AXIOM MODEL w i l l c o n t a i n a large num-
ber of f a c t s , more than are necessary f o r any
given d e d u c t i o n .
Another LISP program examines the composition
of f u n c t i o n s c a l c u l a t e d by QA-3 and determines
those lower l e v e l FORTRAN t w o - l e t t e r commands
needed to accomplish each of them. In our present
example, a sequence of PU commands would be assem-
bled. I n order t o c a l c u l a t e the a p p r o p r i a t e a r g u -
ments f o r each PU, QA-3 is c a l l e d a g a i n , t h i s time
to prove c o n j e c t u r e s of the form:
(^p,w){POSITION (OB2,p,PRESENT) A SIZE (OB2,w)) .
Again the proof produces the p and w t h a t e x i s t ,
thus p r o v i d i n g the necessary p o s i t i o n and size
arguments f o r PU. (Size i n f o r m a t i o n is a l s o a u t o -
m a t i c a l l y entered i n t o the AXIOM MODEL by r o u t i n e s
t h a t scan the GRID MODEL.)
In t r a n s f e r r i n g c o n t r o l between LISP and
FORTRAN (and a l s o between separate large FORTRAN
segments), use Is made of a s p e c i a l m i n i a t u r e moni-
t o r system c a l l e d the VALET. The VALET handles
the process of d i s m i s s i n g program segments and
s t a r t i n g up new ones using a u x i l i a r y drum storage
f o r t r a n s f e r r i n g i n f o r m a t i o n between programs.
The QA-3 theorem-proving system allows us to
pose q u i t e general problems to the robot system,
but f u r t h e r research is needed on adapting theorem-
p r o v i n g techniques t o robot p r o b l e m - s o l v i n g i n
order t o Increase e f f i c i e n c y . * The g e n e r a l i t y o f
* We can e a s i l y propose l e s s f o r t u n a t e a x l o m a t i z a -
t l o n s f o r the " c o l l e c t i n g o b j e c t s t a s k " t h a t
would prevent QA-3 from being able to solve i t .
theorem-proving techniques tempts us to use a
s i n g l e theorem-prover (and axiom s e t ) as a problem-
s o l v e r (and model) f o r a l l h i g h l e v e l robot a b i l i -
t i e s . We might conclude, however, t h a t e f f i c i e n t
o p e r a t i o n r e q u i r e s a number of c o o r d i n a t i n g a c t i o n -
u n i t s t r u c t u r e s , each having i t s own s p e c i a l i z e d
theorem-prover and axiom set and each responsible
f o r r e l a t i v e l y narrow classes o f f u n c t i o n s .
Another LISP program enables commands s t a t e d
in simple E n g l i s h to be e x e c u t e d . 1 0 , * 1 1 It also
accepts simple E n g l i s h statements about the e n -
vironment and t r a n s l a t e s them i n t o predicate c a l -
culus statements to be stored as axioms. E n g l i s h
commands are o r d i n a r i l y t r a n s l a t e d i n t o p r e d i c a t e
c a l c u l u s c o n j e c t u r e s f o r QA-3 to solve by produc-
ing an a p p r o p r i a t e sequence of subordinate f u n c -
t i o n s . For some simple commands, the theorem-
prover is bypassed and lower l e v e l r o u t i n e s such
as PU, TE, e t c . , are c a l l e d d i r e c t l y .
The E n g l i s h program also accepts simple
E n g l i s h questions t h a t r e q u i r e no robot a c t i o n s .
For t h e s e , it uses QA-3 to discover the answer,
and then i t d e l i v e r s t h i s answer i n E n g l i s h v i a
the t e l e t y p e w r i t e r . (Task execution can also be
reported by an a p p r o p r i a t e English o u t p u t . )
Ill VISUAL PERCEPTION
V i s i o n is p o t e n t i a l l y the most e f f e c t i v e means
f o r the robot system to o b t a i n i n f o r m a t i o n about
i t s w o r l d . The robot l i v e s i n a r a t h e r a n t i s e p t i c
but nevertheless r e a l world of simple o b j e c t s —
boxes, wedges, w a l l s , doorways, e t c . I t s v i s u a l
system e x t r a c t s i n f o r m a t i o n about t h a t world from
a c o n v e n t i o n a l TV p i c t u r e . A complete scene a n a l y -
s i s would produce a d e s c r i p t i o n of the v i s u a l scene,
i n c l u d i n g the l o c a t i o n and i d e n t i f i c a t i o n o f a l l
v i s i b l e o b j e c t s . C u r r e n t l y , we have two separate
o p e r a t i n g v i s i o n programs. One of these produces
l i n e drawings, and has been used f o r some time to
i d e n t i f y empty f l o o r space, regions on the f l o o r
i n t o which the robot is f r e e to move. The o t h e r ,
which is more r e c e n t , locates and i d e n t i f i e s the
major, non-overlapping o b j e c t s . In t h i s section
we s h a l l give b r i e f d e s c r i p t i o n s of how these p r o -
grams o p e r a t e .
A. Line Drawing Program
The l i n e drawing program produces a l i n e draw-
i n g r e p r e s e n t a t i o n of a scene by a s e r i e s of essen-
t i a l l y l o c a l o p e r a t i o n s on a TV p i c t u r e . * F i g . 3a
shows a t y p i c a l d i g i t i z e d p i c t u r e , which is stored
in the computer as a 120 x 120 array of 4 - b i t (16-
l e v e l ) i n t e n s i t y v a l u e s . The scene shown is f a i r l y
t y p i c a l , and i n c l u d e s some of the problems of m i l d
shadows and r e f l e c t i o n s , some f a i n t edges, and ob-
j e c t s not completely i n the f i e l d o f view.
* Most of these o p e r a t i o n s were adaptations of
e a r l i e r work by R o b e r t s . 1 2 D e t a i l s of our p r o -
cedures, t o g e t h e r w i t h a d e s c r i p t i o n of s p e c i a l
hardware f o r doing them e f f i c i e n t l y , are given
in R e f s . 1 and 4.
-513-
The f i r s t o f the l o c a l operations i s a d i g i t a l
d i f f e r e n t i a t i o n used to f i n d p o i n t s where there are
s i g n i f i c a n t changes i n l i g h t i n t e n s i t y . These
changes u s u a l l y occur at or near the boundaries of
o b j e c t s , as can be seen from F i g . 3 b . The next
step i s t o determine the l o c a l d i r e c t i o n o f these
boundaries. This is done by s y s t e m a t i c a l l y p l a c -
i n g small masks over the d i f f e r e n t i a t e d p i c t u r e ,
and l o o k i n g f o r places where the masks l i n e up w e l l
w i t h the g r a d i e n t . F i g . 3c shows the l o c a t i o n s and
o r i e n t a t i o n s of masks t h a t responded s t r o n g l y .
The next step is to f i t these short l i n e seg-
ments w i t h longer s t r a i g h t l i n e s . This is done by
f i r s t grouping the short l i n e segments, and then
f i t t i n g a s i n g l e s t r a i g h t l i n e t o a l l o f the seg-
ments in a group. Grouping is a systematic p r o -
cedure in which short segments are l i n k e d if they
are s u f f i c i e n t l y close i n l o c a t i o n and o r i e n t a t i o n .
F i g . 3d shows the r e s u l t s of f i t t i n g longer l i n e s
to the segments in each group. The f i n a l step is
to J o i n the endpoints of these long l i n e s to p r o -
duce a connected l i n e d r a w i n g . This is done by
c o n s i d e r i n g the endpoints one at a time and c r e a t -
i n g candidate c o n n e c t i o n s — s t r a i g h t connections to
neighboring e n d p o i n t s , e x t r a p o l a t i o n s t o p o i n t s o f
intersection, extrapolations to T-junctions, etc.
The candidate that best f i t s the corresponding p a r t
of the d e r i v a t i v e p i c t u r e is the one s e l e c t e d . The
f i n a l l i n e drawing produced by t h i s procedure is
shown i n F i g . 3 e .
While the l i n e drawing preserves much of the
i n f o r m a t i o n in the quantized p i c t u r e in a compact
form, i t o f t e n contains flaws due t o missing o r
e x t r a l i n e segments t h a t complicate i t s a n a l y s i s .
C u r r e n t l y , the only i n f o r m a t i o n we e x t r a c t from
the l i n e drawing is a map of the open f l o o r space.
A program c a l l e d f l o o r boundary f i r s t f i n d s a path
along those l i n e segments t h a t bound the f l o o r
space in the p i c t u r e . These l i n e s are t y p i c a l l y
the places where the w a l l s or o b j e c t s meet the f l o o r ,
or where sides of o b j e c t s obscure our view of the
f l o o r . F i g . 3f shows the f l o o r boundary e x t r a c t e d
from the l i n e drawing.
Now corresponding to any p o i n t in the p i c t u r e
is a ray going from the lens center of the camera
through the p i c t u r e p o i n t and out i n t o space. This
ray is the locus of a l l p o i n t s t h a t can produce the
given p i c t u r e p o i n t . If we f o l l o w the rays going
through p o i n t s on the f l o o r boundary to the p o i n t s
at which they pierce the f l o o r , we o b t a i n an i r -
r e g u l a r polygon on the f l o o r t h a t bounds space
known to be empty. In t h i s way the l i n e drawing
is used to i d e n t i f y empty f l o o r space, and the
v i s i o n system enters i n f o r m a t i o n about open area
i n t o the GRID MODEL.
B. Object I d e n t i f i c a t i o n Program
Were the l i n e drawing program able to produce
a p e r f e c t l i n e drawing, the a n a l y s i s needed to
locate and i d e n t i f y o b j e c t s in the scene would be
r e l a t i v e l y s t r a i g h t f o r w a r d . However, the l i n e
drawing o f t e n contains flaws t h a t s e r i o u s l y com-
p l i c a t e i t s a n a l y s i s . Some of these flaws could
be c o r r e c t e d by more elaborate l o c a l p r o c e s s i n g .
 However, t h e r e is a l i m i t to how w e l l l o c a l p r o -
cessing can p e r f o r m , and when s i g n i f i c a n t edges
cannot be t o l d from i n s i g n i f i c a n t edges on the
basis of l o c a l c r i t e r i a , the goal of producing a
p e r f e c t l i n e drawing in t h i s way must be abandoned.
The o b j e c t i d e n t i f i c a t i o n program l o c a t e s and
i d e n t i f i e s non-overlapping o b j e c t s b y g a t h e r i n g
and i n t e r p r e t i n g evidence s u p p l i e d by l o c a l opera-
t o r s . The program c o n s i s t s of two p a r t s : a r e p e r -
t o i r e of l o c a l operators and an e x e c u t i v e . The
l o c a l o p e r a t o r s process the g r a d i e n t p i c t u r e t o
perform t e s t s such as d e c i d i n g whether or not there
i s a l i n e between two p o i n t s , o r f i n d i n g a l l l i n e s
l e a v i n g a given p o i n t . Each operator r e t u r n s not
a s i n g l e answer, but a set of p o s s i b l e answers w i t h
associated confidences ( i d e a l l y , p r o b a b i l i t i e s )
t h a t each answer is in f a c t c o r r e c t . The executive
explores the scene by c a l l i n g l o c a l o p e r a t o r s and
e v a l u a t i n g the r e s u l t s i n the l i g h t o f both p r i o r
t e s t r e s u l t s and b u i l t - i n knowledge o f the w o r l d .
The e x e c u t i v e program is organized as a d e c i -
s i o n t r e e . Each node in the t r e e s p e c i f i e s t h a t a
p a r t i c u l a r t e s t is to be performed. The branches
l e a v i n g a node correspond to the p o s s i b l e t e s t o u t -
comes, and since each outcome has an associated
c o n f i d e n c e , these confidences are attached to the
branches.
A g i v e n node in the t r e e can be thought of as
r e p r e s e n t i n g a hypothesis about the contents of the
scene. This hypothesis is simply t h a t the scene is
p a r t i a l l y described b y the t e s t r e s u l t s s p e c i f i e d
by the path from the s t a r t node to the given node.
The hypothesis is given a confidence by combining
the confidences of these t e s t r e s u l t s . The t e s t
c a l l e d f o r at the given node is designed to provide
a n answer t h a t w i l l tend t o c o n f i r m o r i n f i r m the
hypothesis.
An a n a l y s i s of a scene proceeds as a search of
the d e c i s i o n t r e e d e s c r i b e d . At any stage in the
search we have a p a r t i a l l y expanded t r e e c o r r e s -
ponding to the t e s t s already performed. The nodes
a t the t i p s o f t h i s p a r t i a l t r e e , which w e s h a l l
c a l l open nodes, present us w i t h a choice of pos-
s i b l e next t e s t s ( o r , a l t e r n a t i v e l y , hypotheses t o
be f u r t h e r i n v e s t i g a t e d ) . The open node w i t h h i g h -
est associated confidence i s selected f o r expansion,
i . e . , the t e s t c a l l e d f o r b y t h a t p a r t i c u l a r node
is performed. The search proceeds u n t i l the open
node w i t h h i g h e s t confidence is a t e r m i n a l node of
the t r e e . A t e r m i n a l node t y p i c a l l y represents a
complete d e s c r i p t i o n of at l e a s t a p o r t i o n of the
scene, and hence c o n s t i t u t e s at l e a s t a p a r t i a l
"answer," ( C e r t a i n t e r m i n a l nodes correspond to
impossible p h y s i c a l s i t u a t i o n s ; i n t h i s e v e n t , the
search Is resumed at the next most c o n f i d e n t node.)
A f t e r r e t u r n i n g a p a r t i a l answer the p o r t i o n of the
scene c o n t a i n i n g the o b j e c t found is d e l e t e d and
the a n a l y s i s begins a g a i n . Thus the t r e e search is
i t e r a t e d u n t i l the scene c o n t a i n s n o f u r t h e r o b -
jects.
The d e c i s i o n t r e e i t s e l f embodies the s t r a t e g y
f o r searching the scene. The basic ideas behind
t h i s s t r a t e g y are s i m p l e , and w i l l be i l l u s t r a t e d
by f o l l o w i n g the o p e r a t i o n of the program on the
-514-
scene o f F i g . 3a. The f i r s t o p e r a t i o n c a l l e d f o r i s
a search f o r v e r t i c a l l i n e s , since these are u s u a l l y
both r e l i a b l y d e t e c t a b l e and s i g n i f i c a n t . F i g . 4a
shows t h a t the a p p r o p r i a t e operator found three v e r -
t i c a l l i n e s , which happened to rank in confidence as
numbered.
S t a r t i n g w i t h the highest confidence l i n e and
checking to see t h a t i t s lower endpoint was w i t h i n
the p i c t u r e , the program next looked f o r o t h e r
l i n e s l e a v i n g t h a t e n d p o i n t . A f a i l u r e here would
have led to the c o n c l u s i o n t h a t something was
s t r a n g e , and t h e r e f o r e to a t r a n s f e r to the next
most c o n f i d e n t node in the t r e e . However, a " s p u r "
was d e t e c t e d , as shown in F i g . 4b. Hypotheses t h a t
the lower endpoint was connected to the lower end-
p o i n t s of o t h e r v e r t i c a l s were r e j e c t e d because the
d i r e c t i o n of the spur was not c o r r e c t . Thus, at
t h i s p o i n t a t t e n t i o n was s h i f t e d to the top of the
v e r t i c a l . A spur was found t h e r e , as expected, and
t h a t spur was f o l l o w e d to i t s endpoint as shown in
F i g . 4c. The f a c t t h a t i t s endpoint was on the
p i c t u r e , coupled w i t h the f a c t t h a t the program had
f a i l e d i n i t s attempt t o connect the v e r t i c a l t o
other v e r t i c a l s , provided s t r o n g evidence t h a t a
wedge had been found. A f u r t h e r check of the angle
at the top of the v e r t i c a l confirmed the wedge hy-
p o t h e s i s , and the same c a l c u l a t i o n s used in the
f l o o r boundary program were used to l o c a t e the lower
vertices.
At t h i s point one o b j e c t had been found and
i d e n t i f i e d , and a search f o r other o b j e c t s began.
On the second i t e r a t i o n , the remaining v e r t i c a l s
were s u c c e s s f u l l y Joined at t h e i r lower e n d p o i n t s ,
as shown in F i g . 4d, and v a r i o u s spurs were found,
as shown in F i g . 4e. A s i m i l a r attempt to spot a
wedge f a i l e d to produce s t r o n g evidence, as shown
i n F i g . 4 f , and the f i n a l output i n d i c a t e d the ex-
i s t e n c e and l o c a t i o n of an o b j e c t p a r t l y out of
view w i t h o u t s p e c i f y i n g what it was. At t h i s p o i n t
there were no more v e r t i c a l l i n e s , and the a n a l y s i s
was completed.
The o b j e c t i d e n t i f i c a t i o n program is capable of
l o c a t i n g and o f t e n i d e n t i f y i n g non-overlapping ob-
j e c t s on the b a s i s of p a r t i a l i n f o r m a t i o n . There
are a number of obvious ways in which it can be
improved and extended, and f u r t h e r research w i l l be
devoted to these t a s k s . However, even as it stands
it can provide the robot w i t h much v a l u a b l e i n f o r -
mation about the r o b o t ' s w o r l d .
IV CONCLUSIONS
There are s e v e r a l key q u e s t i o n s t h a t our work
has helped to put i n t o f o c u s . Given t h a t a robot
system w i l l i n v o l v e the s u c c e s s f u l i n t e g r a t i o n o f
p r o b l e m - s o l v i n g , m o d e l l i n g , and p e r c e p t u a l a b i l i t i e s ,
there are many research questions concerning each
of t h e s e . Let us discuss each in t u r n .
A. Problem-Solving
Our somewhat h i e r a r c h i c a l o r g a n i z a t i o n of
problem-solvers and models seems a n a t u r a l , even if
ad hoc, s o l u t i o n to o r g a n i z i n g complex b e h a v i o r .
Are t h e r e a l t e r n a t i v e s ? W i l l the use of theorem-
p r o v i n g techniques provide enough g e n e r a l i t y to
permit a s i n g l e general-purpose problem-solver, or
w i l l s e v e r a l " s p e c i a l i s t " t h e o r e m - p r o v e r s b e needed
to gain the required e f f i c i e n c y ?
O t h e r q u e s t i o n s c o n c e r n t h e use o f t h e o r e m -
p r o v i n g methods f o r p r o b l e m - s o l v i n g . How d o t h e y
compare w i t h t h e " p r o d u c t i o n m e t h o d s " a s used b y
t h e G e n e r a l P r o b l e m S o l v e r (GPS) o r w i t h t h e p r o -
c e d u r a l language approach as developed by F i k e s ? 1 3
P e r h a p s some c o m b i n a t i o n o f a l l o f t h e s e w i l l p r o v e
s u p e r i o r t o any o f t h e m ; p e r h a p s more e x p e r i e n c e
w i l l show t h a t t h e y a r e o n l y s u p e r f i c i a l l y d i f -
ferent .
Another question i s : T o what l e v e l o f d e t a i l
s h o u l d b e h a v i o r a l p l a n s b e made b e f o r e p a r t o f t h e
p l a n i s e x e c u t e d and t h e r e s u l t s c h e c k e d a g a i n s t
perceptual information? Although t h i s question
w i l l n o t have a s i n g l e a n s w e r , w e need t o know
upon what f a c t o r s t h e answer d e p e n d s .
Our p r o b l e m - s o l v i n g r e s e a r c h w i l l a l s o b e
d i r e c t e d a t methods f o r o r g a n i z i n g even more com-
plex robot behavior. W e hope e v e n t u a l l y t o b e
a b l e t o d e s i g n r o b o t systems capable o f p e r f o r m i n g
complex assembly t a s k s r e q u i r i n g the i n t e l l i g e n t
use o f t o o l s and o t h e r m a t e r i a l s .
B. Modelling
S e v e r a l q u e s t i o n s a b o u t m o d e l s can b e p o s e d :
Even i f w e c o n t i n u e t o use a number o f p r o b l e m -
s o l v e r s , must e a c h have i t s own model? T o what
e x t e n t c a n t h e same model s e r v e s e v e r a l p r o b l e m -
solvers? When a p e r c e p t u a l s y s t e m d i s c o v e r s new
i n f o r m a t i o n about the w o r l d , should it be entered
d i r e c t l y i n t o a l l models c o n c e r n e d ? I n what f o r m
s h o u l d i n f o r m a t i o n b e s t o r e d i n t h e v a r i o u s models?
S h o u l d p r o v i s i o n s b e made f o r f o r g e t t i n g o l d i n -
formation? Can a r o b o t s y s t e m be g i v e n a s i m p l e
m o d e l o f i t B own p r o b l e m - s o l v i n g a b i l i t i e s ? En-
s u i n g r e s e a r c h and e x p e r i e n c e w i t h o u r p r e s e n t
system should h e l p us w i t h these q u e s t i o n s .
C. Visual Perception
The i m m e d i a t e v i s i o n p r o b l e m s i n v o l v e i n c l u d -
i n g more t e s t s i n t h e o b j e c t i d e n t i f i c a t i o n p r o g r a m
t o c o m p l e t e u n f i n i s h e d a n a l y s i s , and r e m o v i n g t h e
r e s t r i c t i o n to non-overlapping objects. Beyond
t h e s e improvements t h e r e are s t i l l l o n g e r range
problems to be s o l v e d . The scene a n a l y s i s p r o g r a m s
i m p l i c i t l y s t o r e i n f o r m a t i o n about the w o r l d i n
their structure. Changes i n t h e r o b o t ' s w o r l d c a n
r e q u i r e e x t e n s i v e changes t o t h e whole p r o g r a m .
What p r o g r a m o r g a n i z a t i o n w o u l d m i n i m i z e t h e s e
problems? How c a n t h e scene a n a l y s i s p r o g r a m i n -
t e r r o g a t e and use f a c t s s t o r e d i n t h e m o d e l t o
advantage? Since " f a c t s " obtained from e i t h e r the
model o r t h e s u b r o u t i n e s are s u b j e c t t o e r r o r , i t
i s n a t u r a l t o accompany them b y a c o n f i d e n c e mea-
sure. How s h o u l d t h e s e c o n f i d e n c e s b e computed
and how s h o u l d t h e y b e c o m b i n e d , s i n c e , l o o s e l y
s p e a k i n g , we o p e r a t e under c o n d i t i o n s of s t r o n g
s t a t i s t i c a l dependence? How c a n w e augment t h e c u r -
r e n t r e p e r t o i r e o f s u b r o u t i n e s w i t h o t h e r s t o make
use o f s u c h p r o p e r t i e s a s c o l o r , t e x t u r e and r a n g e ?
Future v i s i o n research w i l l be devoted to answering
q u e s t i o n s such as t h e s e .
-515-
The m a i n theme o f t h e p r o j e c t has b e e n , and
w i l l c o n t i n u e t o b e , t h e problem o f system i n t e g r a -
tion. I n s t u d y i n g r o b o t systems t h a t i n t e r a c t w i t h
t h e r e a l w o r l d , i t seems e x t r e m e l y i m p o r t a n t t o
b u i l d and p r o g r a m a r e a l s y s t e m and t o p r o v i d e i t
w i t h a real environment. Whereas much can b e
learned by s i m u l a t i n g c e r t a i n of the necessary func-
t i o n s (we use t h i s s t r a t e g y r e g u l a r l y ) , many i m -
p o r t a n t issues are l i k e l y not to be a n t i c i p a t e d at
a l l in simulations. Thus q u e s t i o n s r e g a r d i n g , s a y ,
the f e a s i b i l i t y o f a system o f i n t e r a c t i n g a c t i o n -
u n i t s f o r c o n t r o l l i n g a r e a l r o b o t can o n l y b e c o n -
fronted by a c t u a l l y attempting to c o n t r o l a r e a l
r o b o t w i t h such a s y s t e m . Questions r e g a r d i n g the
s u i t a b i l i t y o f c a n d i d a t e v i s u a l p r o c e s s i n g schemes
c a n most r e a l i s t i c a l l y b e a n s w e r e d b y e x p e r i m e n t s
w i t h a s y s t e m t h a t needs t o " s e e " t h e r e a l w o r l d .
T h e o r e m - p r o v i n g t e c h n i q u e s seem a d e q u a t e f o r s o l v -
i n g many " t o y " p r o b l e m s ; w i l l t h e f u l l g e n e r a l i t y
o f t h i s approach r e a l l y b e e x p l o i t a b l e f o r d i r e c t -
i n g the automatic c o n t r o l o f mechanical equipment
in real-time?
The q u e s t i o n s t h a t w e have posed i n t h i s s e c -
t i o n a r e among t h o s e t h a t must b e answered i n o r d e r
t o d e v e l o p u s e f u l and v e r s a t i l e r o b o t s y s t e m s .
E x p e r i m e n t i n g w i t h a f a c i l i t y such a s w e have d e -
s c r i b e d a p p e a r s t o b e t h e b e s t way t o e l i c i t t h e
p r o p e r q u e s t i o n s and t o w o r k t o w a r d t h e i r a n s w e r s .
REFERENCES
1. N. Nilsson, et a l , "Application of Intelligent
Automata to Reconnaissance," C o n t r a c t AF30(602)-
4 1 4 7 , SRI P r o j e c t 5 9 5 3 , S t a n f o r d R e s e a r c h
I n s t i t u t e , Menlo P a r k , C a l i f o r n i a ( f o u r I n t e r i m
R e p o r t s and one F i n a l R e p o r t d a t e d December
1968).
2. C . A . Rosen and N . J . N i l s s o n , " A n I n t e l l i g e n t
A u t o m a t o n , " IEEE I n t e r n a t i o n a l C o n v e n t i o n
Record, Part 9 (1967).
3. B. Raphael, "Programming a R o b o t , " P r o c . IFIP
Congress 6 8 , E d i n b u r g h , S c o t l a n d (August 1968).
4. G . E . F o r s e n , " P r o c e s s i n g V i s u a l Data w i t h a n
Automaton E y e , " i n P i c t o r i a l P a t t e r n R e c o g n i -
t i o n (Thompson Book Company, W a s h i n g t o n , D . C . ,
1968).
5. P . H a r t , N . N i l s s o n , and B . R a p h a e l , " A F o r m a l
Basis f o r the H e u r i s t i c Determination of M i n i -
mum C o s t P a t h s , " IEEE T r a n s , on Systems S c i e n c e
and C y b e r n e t i c s , V o l . SSC-4, N o . 2 , p p . 1 0 0 -
107, ( J u l y 1968).
6. C . Green and B . R a p h a e l , "The Use o f T h e o r e m -
Proving Techniques i n Question-Answering
S y s t e m s , " P r o c . 1968 ACM C o n f e r e n c e , Las V e g a s ,
Nevada ( A u g u s t 1 9 6 8 ) .
7. B. Raphael, "Research on I n t e l l i g e n t Question-
Answering Systems," F i n a l Report, Contract AF
1 9 ( 6 2 8 ) - 5 9 1 9 , SRI P r o j e c t 6001, Stanford
R e s e a r c h I n s t i t u t e , Menlo Park, C a l i f o r n i a
(May 1 9 6 8 ) .
 8. C. Green, "Theorem-Proving by R e s o l u t i o n as a
Basis f o r Question-Answering Systems," Machine
I n t e l l i g e n c e 4, B. M e l t z e r and D. M i c h i e , Eds.
(Edinburgh U n i v e r s i t y Press, Edinburgh, S c o t -
l a n d ; to appear 1969).

9. C. Green, " A p p l i c a t i o n s of Theorem-Proving to

P r o b l e m - S o l v i n g , " P r o c . o f the I n t e r n a t i o n a l
J o i n t Conference o n A r t i f i c i a l I n t e l l i g e n c e ,
Washington, D.C., (May 1969).

10. L. S. Coles, "An On-Line Question-Answering

System w i t h N a t u r a l Language and P i c t o r i a l
I n p u t , " Proc. 1968 ACM Conference, LAS Vegas,
Nevada (August 1968).

11. L. S. Coles, " T a l k i n g w i t h a Robot in E n g l i s h , "

Proc. o f the I n t e r n a t i o n a l J o i n t Conference
o n A r t i f i c i a l I n t e l l i g e n c e , Washington, D.C.,
(May 1969).

12. L. G. R o b e r t s , "Machine Perception of Three-

Dimensional S o l i d s , " O p t i c a l and E l e c t r o -
O p t i c a l I n f o r m a t i o n Processing (MIT Press,
Cambridge, Massachusetts, 1965).

13. R. F i k e s , "A 9tudy in H e u r i s t i c Problem-

S o l v i n g : Problems Stated as P r o c e d u r e , "
Proc. of F o u r t h Systems Symposium, held at
Case Western Reserve U n i v e r s i t y , C l e v e l a n d ,
Ohio, November 1968 ( t o be p u b l i s h e d ) .

-516-
FIG. 2 A GRID MODEL OF A ROOM WITH THREE OBJECTS

-518-