CIS 496 / EAS 499 Senior Project

Project Proposal Specification

Instructors: Norman I. Badler and Joseph T. Kider. Jr.

A Cognitive Model for an Embodied Conversational Agent

Ian Perera

Advisor: Mitchell P. Marcus

University of Pennsylvania

ABSTRACT

Intelligent Virtual Agent cognitive models often use a series of abstractions to split different tasks into
manageable and solvable problems. For example, language is translated from a sentence to a parse tree, and then to a
semantic representation. The semantic representation is then used with a knowledge base to transform the semantics into
a temporal logic, and then the logic is transformed into statements which can be evaluated. However, such a pipeline has
limitations because each of the constituent parts could aid in evaluating other parts for pronoun reference,
disambiguation, prepositions, and pragmatics, yet are kept separate in a pipeline model.
I propose a cognitive model that consists of a cross between a semantic spreading activation network and finite
state machine, which is embodied in a virtual world by means of callback functions expressed as nodes in the network.
Each node in this network represents a concept that is mapped to other nodes with a relationship. This system allows for
conceptual relationships found in a semantic network to coexist with and fill in the information needed for the functional
callback nodes associated with particular actions. Gates are used to control shortest path and spreading activation
calculations when nodes are queried. Learning can take place through the addition of connections either from language
input or through automatic learning (such as Long-Term Potentiation - adding connections between nodes that activate
together). The FSM aspect is used to model sequences of actions while maintaining conceptual information at each step of
the process.

Project Blog: http://virtuallyembodied.blogspot.com

1. INTRODUCTION I propose a system that addresses these weaknesses by

Intelligent Virtual Agent cognitive models often use a
series of abstractions to split different tasks into
manageable and solvable problems. For example,
language is translated from a sentence to a parse tree, and
then to a semantic representation. The semantic
representation is then used with a knowledge base to
transform the semantics into a logic, and then the logic is
transformed into statements which can be evaluated.
However, this design poses a few problems. First, it is
often the case that each of these parts can only be
maintained and developed by experts in the respective
field of the component. Computational linguists are
required to change syntactical and semantic parsing,
logicians are needed to change and interpret logic
representations, and programmers are required to change
the controller behavior to add functionality or adapt to a
new domain. Second, although splitting these pieces into
separate parts makes the task easier, it also neglects the
information that could be gleaned from outside processes.
For example, the knowledge base could be used to reject
parses that are unintelligible, or multiple parses can be
considered to determine the most favorable action for the
agent. Responses from the controller could also be used
to update the knowledge base or provide a context based
on information from the environment. Finally, such a
system is not attuned to learning actions. Since each
process is a fairly mechanical transformation from one
input to an output, there are no structures in place that can
be used for induction.
unifying the semantic, conceptual, and experiential
aspects of virtual agents. The core of this system can be
seen as a combination of a spreading activation network
(for learning and action determination), a semantic
network (for defining concepts hierarchically), and a
finite state machine (to store and execute sequences of
actions). The system is a network of nodes, where each
node represents a concept that is mapped to other nodes
with a relationship. Nodes can also consist of callbacks to
written code, which would be executed depending on the
desired action of the node. This system allows for
conceptual relationships found in a semantic network to
coexist with and fill in the information needed for the
functional nodes associated with particular actions.
1.1 Design Goals
The goal of this project is to create a cognitive model
that could be used for an embodied conversational agent,
but also could be useful in other artificial intelligence
tasks that can utilize both symbolic and instance-based
data, such as computer vision and speech recognition.
I plan to create an interface for easily adding symbolic
knowledge to the system, as well as a test interface
consisting of a small virtual environment and a terminal
to give text commands to the agent using this model.
1.2 Projects Features and Functionality
This project will be able to be implemented in a virtual
environment through callback functions. It will be

CIS 496/EAS499 Senior Design Project

controlled through natural language input and react to the
environment. Furthermore, it will learn both through
natural language and by inference over observations in
the virtual world.
2. RELATED WORK
Various artificial intelligence applications contain
pieces applicable to this design. The concept of
conversing with an agent in a virtual environment stems
from a motivation towards embodied AI, as well as
practical applications for an embodied conversation
agent, such as training simulations and natural language
interfaces for robots. Spreading Activation Networks are
a natural way of achieving this goal, combining a model
of cognition with task-planning in a virtual or robotic
environment.
2.1 Embodied AI
Although the motivation behind this system is
primarily to design agents that can provide a human-like
interface in virtual worlds, it is also motivated by a desire
to experiment with the theory of “embodied” artificial
intelligence. [Chr03] explains the embodied AI argument
as follows: the assumption that there are global semantic
truths about the world that could be stored in a knowledge
representation is flawed – instead, meaning is stored in a
way that is unique to our experiences. Therefore, rather
than trying to create conversational agents that are
disembodied programs, we must give them an
environment and sensory facilities to form their
perceptions about the world. Work in embodied AI is
typically done either in robotics or virtual environments. I
will focus on the latter in evaluating past efforts.
2.2 Virtual Embodied Agents
[Win71] details the embodied conversational agent
SHRDLU, which was one of the first examples of a
natural language interface for a virtual agent. The agent
was placed in a limited environment of various shapes of
blocks, and could manipulate and describe them
according to the user’s input. My model should provide
similar possible commands to the agent and generalize to
different environments.
[Bur09] demonstrates current work in evaluating the
advantages of embodiment in a virtual world such as
Second Life, where users can interact with the agent at
any time. This provides a way for users to experience
human-like interactions without the need for actual
human users. [ISB11] assert that agents that demonstrate
environmental awareness are judged more convincing in
conversation than those without such awareness. They
implement a conversation system that integrates
environmental knowledge in a limited domain, but more
importantly, show through user studies that
environmental awareness does improve a user’s
evaluation of the “humanness” of a virtual agent. This
study gives credence to the embodied AI theory and
highlights relevant applications of embodied
conversational agents.
2.3 Spreading Activation Networks
[CL75] explain and expand upon the Spreading
Activation Theory of Semantic Processing first proposed
by Collins. Spreading Activation Theory suggests how
one could implement cognition in a computer program,
while also accounting for certain psychological
phenomena such as priming. The base structure for a
Spreading Activation Network is typically a semantic
network, a hierarchical knowledge representation that
stores concepts in terms of their relationships to each
other. Semantic networks were first introduced in [Col69]
as a cognitive model to explain semantic retrieval times
from psychological studies. However, their hierarchical
representation is useful for generalizing and comparing
concepts based on their similarity, and they are therefore
used for knowledge representations in many applications,
such as ConceptNet [LS04]. ConceptNet also generates
semantic information in its network based on language
input from users.
Given a base network, spreading activation starts with
one or more “source” nodes. Nodes connected to the
source node receive activation with some decay rate. This
spreading continues until the activation for the next set of
nodes reaches a certain threshold. Spreading activation in
task planning is done by performing two activations
simultaneously – one for the current state of the
environment, which will activate possible actions, and
one for the desired state of the environment, which will
activate which actions will satisfy the desired goal. The
path with the greatest activation from both sources
describes the sequence of actions required to reach the
goal.
[BBK00] demonstrate the use of a spreading activation
network for an agent architecture that carries out tasks
even under changing environment conditions. They
describe a case study with a robotic feeding arm, but the
principles are general enough to apply to any embodied
agent.
[Bac03] describes a virtual agent architecture based on
a spreading activation network, with additional
components to handle virtual inputs and behavior.
However, this architecture does not include a natural
language interface. While this architecture shares many
similarities with my model, my model treats language as
the core of the conceptual model, allowing for both
learning and user querying of the agent’s plans and
beliefs.
[Mae90] describes an architecture that uses an
activation/inhibition network for goal-directed behavior.
However, it also does not include a language component,
and I also plan to extend the concepts of activation and
inhibition by implementing a simulated Long-Term
Potentiation model to facilitate structural learning in the
semantic spreading activation network.
3. PROJECT PROPOSAL
There are multiple components to this project, some of
which are integrated less tightly than others. The
semantic spreading activation network forms the core of
the system, and is therefore the first priority. The
language interface is second, and the embodiment in the
virtual environment is third (and can be substituted with a
text environment in the case of time constraints). Once
these essential parts are implemented, additions and
improvements can be made in any order.
© SIG Center for Computer Graphics 2010.
CIS 496 / EAS 499 Senior Project
Project Proposal Specification
Instructors: Norman I. Badler and Joseph T. Kider. Jr.
3.1 Anticipated Design
3.1.1 Knowledge Representation (KR)
Knowledge in this agent system will be represented in
terms of a semantic network. Each node in the semantic
network represents a concept, gate, or function callback.
Concepts are actions, ideas, and objects that comprise the
agent’s beliefs and environment, such as tree, run, and
color. Gates modify the spreading activations or shortest
path calculations used in reasoning and goal-planning.
Some examples would be a logical OR, a logical AND, a
delay, or an accumulator. Finally, function callbacks take
function arguments from the spreading activation and
trigger if the activation is high enough.
Edges between nodes represent both explicit and
implicit relationships between them. Explicit
relationships are specified by the KR designer or
generated based on textual input. Implicit relationships
are caused by concurrent activation of stimuli in the
environment, and simulate Long-Term Potentiation – the
Hebbian theory of “neurons that fire together, wire
together”.
3.1.2 Memory Model
The memory model for this architecture consists of
three parts, but all three parts use the same semantic
spreading activation network model.
Long-term knowledge contains general relationships
between concepts and actions that the agent is familiar
with. Working memory contains information about the
current environment, and is used for fulfilling goals in the
environment and forming new relationships based on
environmental stimuli. Abstract memory contains
possible theoretical scenarios that could be evaluated to
optimize the path towards a goal.
3.1.3 Parsing
First, the input string is determined to be a command,
query, or a fact. Commands are either direct calls to
actions or goals that the agent should try to achieve.
Queries are questions that the agent will respond to by
traversing its KR. Facts are additions to be made to the
KR.
A syntactic parser will then take this input string and
convert it into a parse tree, labeling the syntax structure.
From this tree, I will split clauses and resolve them
separately. Coordinating (e.g. and, for) and subordinating
(e.g. while, before, after) conjunctions are used to form
relationships between the clauses, such as preconditions
or concurrency requirements.
3.1.4 Traversal Process
3.1.4.1 Commands
First, find the node corresponding to the action
requested. Then, determine if any further information is
needed from the user. If there are any preconditions of the
action, initiate activation at the action node and at the
environment root node (the agent itself) to determine the
shortest path sequence of actions to complete the action.
CIS 496/EAS499 Senior Design Project
3.1.4.2 Queries and Facts
Determine the object or concept in question, then say
or modify the attribute that was requested.
3.1.5 Grounding
Nodes will be grounded with information from the
environment in various ways. Code grounding links an
action with a callback function to execute prewritten
code. Mathematical grounding would use gate nodes to
compare values of attributes within the network. Verbal
grounding links a particular attribute with a phrase that
could be output, or phrase frame that could be output with
various attributes filled in.
3.1.6 Code Grounding
Code grounding provides developers with the ability to
use existing code for interacting with a virtual
environment or any other program. By mapping concepts
and attributes to functions and their parameters, one can
embed this model into any program to provide a natural
language interface. Code will be written in C++ and will
utilize the same return types and parameters as a regular
C++ program.
Different concepts can be linked to the same function
but with different parameters. For example, the concepts
“walk” and “run” might be linked to the same
move(vec2) function, but with different magnitudes.
Functions embedded in the network can also return
objects to be used by other functions in the network. For
example, one function may return a path between two
points, and another function would take that path as an
input to move the agent along it.
3.2 Anticipated Approach
The first step in this project will be establishing a
method of storing the knowledge representation. I may
use a third-party tool for initial implementations, but
tools such as Protégé would not be able to provide all of
the functionality I would like in designing my network,
such as viewing implicit connections and connections
created during real-time interactions. For my initial work
I can hard-code test KRs, and migrate to a more robust
solution given enough time.
Next, I will implement the spreading activation
algorithm in the network. The spreading activation
algorithm will allow me to specify commands, queries,
and facts as if they have already been parsed into node
references to test the system. From here I will develop a
basic parsing system that performs a simple analysis on
input strings, holding the user to certain constraints.
Robust language processing will be delayed to later in the
project.
Code grounding will be the final essential part of the
architecture. The “callback” system will simply be calls
to predefined methods for demonstration purposes.
Once the above essential parts are complete, I will
work on integrating the agent into a simple virtual test
environment, along with various tests to evaluate the
functionality of the agent.
Once the agent is working in the virtual environment, I
will implement more robust learning, parsing, and
knowledge in that order.
 3.3 Target Platforms • Emotion modeling through gates and emotional
spreading activation
This system will run on any machine capable of
• Improve natural language inputs
executing C++ code and that has the required graphics
libraries for the virtual environment. • Incorporate feature vector learning for sensory
input
3.4 Evaluation Criteria

To evaluate the success of the project, I will devise (remove line)

various scenarios for the agent to complete in the virtual
environment. These will range from simple tests to more
complex experiments. A simple test may include a few
simple objects, and have certain actions defined for them. You will fill these sections in as you complete your
This environment would also allow me to test actions and project for the alpha review and the final document, these
plans defined through the language input. One example sections give psedo-code, charts, images, examples etc to
would be: “Pick up the green box.” This requires the show what you’ve done over the course of the semester.
agent to choose the green box among other objects, find a We are leaving this section open to creativity too, feel
path to it, and then move along that path and finally free to add whatever you feel is necessary to relate your
execute the code associated with picking an object up.
Variations on this scenario could illustrate other 5. Method
aspects of the system. For example, perhaps the agent
does not know that it has to be touching the object to pick
it up. The agent would then say that it was unable to 6. RESULTS
complete the request, and the user could specify, “You
must be touching an object to pick it up.” This input fact
would add a precondition to the “pick up” action and 7. CONCLUSIONS and FUTURE WORK
would then allow the agent to formulate a new plan which
included touching the object before picking it up.
APPENDIX
More complicated scenarios could test problem solving
and observation skills. For example, perhaps the green A. Optional Appendix
box is behind a closed blue door, and there are different Some text here. Some text here. Some text here.
colored switches within reach of the agent. Given the
same command, “Pick up the green box”, the agent would
currently be unable to form a path to the box. The agent
would determine that the door is an obstacle to the path
when it is closed, and would then either ask the user how
to open it, or try various actions to see which would clear (remove line)
the path.

4. RESEARCH TIMELINE

Project Milestone Report (Alpha Version)

• Completed all background reading References
• Knowledge representation functions with [Bac03] Bach, J. The MicroPsi Agent
spreading activation Architecture.
• Simple example of language input triggering a 2003.
function tied to KR [BBK00] Bagchi, S.; Biswas, G. & Kawamura, K.
Task planning under uncertainty using a
• Virtual environment example (time permitting)
spreading activation network. Systems,
Project Milestone Report (Beta Version) Man and Cybernetics, Part A: Systems
• More robust natural language input and Humans, IEEE Transactions on,
2000, 30, 639 -650
• Additions to KR through natural language input
[Bur09] Burden, D. J. Deploying embodied AI
• Natural language responses into virtual worlds. Knowledge-Based
• Virtual environment example Systems, 2009, 22, 540 – 544.
[Chr03] Chrisley, R. Embodied artificial
Project Final Deliverables
intelligence. Artif. Intell., Elsevier
• Additional improvements to natural language Science Publishers Ltd., 2003, 149, 131-
• More complicated actions specified and tested 150.
[CL75] Collins, A. M. & Loftus, E. F.
• Implicit learning through observation (time
A spreading activation theory of semantic
permitting)
processing. Psychological Review, 1975,
Project Future Tasks 82, 407-428
[ISB11] Ijaz, K.; Simoff, S. & Bogdanovych, A.
Enhancing the Believability of Embodied
Conversational Agents through

© SIG Center for Computer Graphics 2010.

CIS 496 / EAS 499 Senior Project
Project Proposal Specification
Instructors: Norman I. Badler and Joseph T. Kider. Jr.

Environment-, Self- and Interaction-

Awareness. Proceedings of the Thirty-
Fourth Australasian Computer Science
Conference (ACSC 2011), 2011
[LS04] Liu, H. & Singh, P. ConceptNet - A
Practical Commonsense Reasoning Tool-
Kit. BT Technology Journal, Springer
Netherlands, 2004, 22, 211-226
[Mae90] Maes, P. Situated agents can have goals.
Robot. Auton. Syst., North-Holland
Publishing Co., 1990, 6, 49-70

CIS 496/EAS499 Senior Design Project

 User input: “Jump.”

Parse: (S (NP (-NONE- *) <no subject specified, assumed to be agent>)

(VP (VBP Jump)))

Find verb “jump” in database, get the node:

Agent

Attribute
Jump Execute

Directio Is-A bool jump(agent, direction, speed)

n { agent.velocity += direction *
speed; }

Up Attribute
Movement
Directio
n

Precondition ?
Parameter substitution

Clearanc Directio
? Direction
n
e

AND

Objects in
Zero Of Nearby
Location:

Figure 1: Example KR chunk representing the “jump” action and its relationship to the “movement” concept

© SIG Center for Computer Graphics 2010.

CIS 496 / EAS 499 Senior Project
Project Proposal Specification
Instructors: Norman I. Badler and Joseph T. Kider. Jr.

24- 31- 7- 14- 21- 28- 7- 14- 21- 28- 4- 11- 18- 25- 2-
Jan Jan Feb Feb Feb Feb Ma Ma Ma Ma Ap Apr Apr Apr May
r r r r r
KR Design

Spreading
Activation
Algorithm
Natural Language
Input

Code Grounding

Embody Agent in
Virtual
Environment
Design/Implement
Test Scenarios

Improvements

Figure 2: Proposed schedule

CIS 496/EAS499 Senior Design Project