A Theoretical Framework of Pedagogical Agents Based on Psychological

Incentive Mechanism and Artificial Psychology Theory

TAO Xiaome1, 2, NIU Qinzhou1, Mike Jackson2, Bin Hu2

1. Guilin University of Technology, Guilin, 541004，P. R. China；
2. Birmingham City University, Birmingham, B42 2SU, UK
taoxiaomei@glite.edu.cn; djx@glite.edu.cn; mike.jackson@bcu.ac.uk; bin.hu@bcu.ac.uk

Abstract the Artificial Psychology theory, in the traditional E-

This paper proposes an innovative teaching and
learning theoretical framework based on
psychological incentive mechanism and Artificial
Psychology theory. Related works about these two
areas will be addressed in section 2. This framework
includes three type pedagogical agents, namely the
Expert agent, the Mentor agent and the Motivator
agent. This framework is intended to create a positive
and personalized study environment for the learners
with the sufficient consideration to the learner’s
emotion ingredient, learner’s personal preference and
the appropriate application of the pedagogical
psychology. The goal of this framework is to stimulate
the learners’ internal drive to promote them to be
active in the learning process.
1. Introduction
The research on pedagogical agents in E-learning
has been undergoing a long time development with the
computing techniques development. The empirical
research works [1] have indicated that pedagogical
agents are effective in improving the learners’
performance in an E-learning environment. The
traditional pedagogical agents offer help and hints,
demonstrate principles, procedures and give examples,
and also can help learners to process and store
information. The advantages of the pedagogical agents
are obvious and include on-demand learning, low cost,
distance learning, etc. The learner can study with the
help of pedagogical agents anytime, anywhere with the
support of pervasive computing techniques. Also with
the support techniques of multimedia, cross-media and
virtual reality, the pedagogical agents are more
attractive to the learners.
However, without the sufficient consideration to
pedagogical psychology and without the support from
learning framework, the teaching side and the learning
side cannot communicate very well from the aspect of
affective communication. The designs of E-learning
systems are based on the universal rules and features
of most learners. And the nature of these systems is
either teaching-centric or learner-centric. Because the
learners are individuals and the states of the learner are
dynamic, the traditional E-learning systems mentioned
above cannot meet the requirements on the psychology
level between the tutors and the learners. The
principles of pedagogical psychology state that [2], the
teaching side and learning side are in an interplay
relationship. An integrated teaching and learning
environment should include the tutors, the learners,
and the positive study atmosphere. The tutor should
give appropriate response in terms of the learners’
performance, and then the learners will respond
positively to the tutors’ instruction. The whole study
environment should offer a positive atmosphere to the
learners to stimulate the learners’ internal drive.
Therefore, the communication and interaction at the
psychology level between the learner and the tutor
should be considered further.
To solve the issues above, an innovative teaching
and learning theoretical framework based on Artificial
Psychology and pedagogical psychology is proposed
in this paper. This framework is intended to create a
positive and personalized study environment for the
learners with the sufficient consideration to the
learner’s emotion ingredient, learner’s personal
preference and the appropriate application of the
pedagogical psychology. The goal of this framework is
to stimulate the learners’ internal drive to promote
them to be active in the learning process. This
framework includes three type pedagogical agents, the
Expert agent, the Mentor agent and the Motivator
agent according to the classification in [3]. And it
consists of two databases, one is knowledge base, the
other is learner’s profile. In section 3, the framework
will be addressed in detail.
2. Related Research Works
2.1. Artificial Psychology
The definition of Artificial Psychology is “The
realization of an artificial machine that contains
computer model algorithms that comprehensively
mimic human psychological activities including
emotion, willingness, character, creativity mainly by
means of information techniques”. The theory of
Artificial Psychology is aimed at a comprehensive
understanding of human psychology, (especially
feelings, willingness, character, and creativity) by
means of a systematic integration of psychology, brain
science, and information science, neuroscience,
computing science and novel theories of automation
science[4][5]. Recently, there has been a gradual
growth in the application of Artificial Psychology in
education. For example, in [6], based on human
cerebrum control pattern, a relationship model between
emotion and gesture is presented, and with this model,
an E-learning assistant system was developed that
offers the function to get the learners’ change in
emotional states through the analysis to the learners’
behavior or gesture. In [7], it describes an individual-
centered E-learning tool taking into account the full
spectrum of Web mining techniques and activities, and
presents a multi-agents architecture based on artificial
psychology techniques, which will make the E-
learning process both adaptable and dynamic. In [8], it
proposes an emotion-based affective cognitive
architecture for meditative E-Learning that gathers
biofeedback psychological signals to reflect the
learners’ emotional state and then delivers content to
the learner by appropriate way.
2.2. Pedagogical agents
Pedagogical agents are animated characters
designed to operate in an educational setting for
supporting or facilitating learning [18]. Pedagogical
agents normally take the shape of a cartoon animated
character or an interactive, artificial intelligence driven
2-D or 3-D image. They can offer help and hints,
demonstrate principles and procedures, and show
examples to help learners to process and store
information.
Pedagogical agents also have some important
features that all the agents have [9], such as:
z Autonomy, including goal-oriented, collaborative,
flexible, and self-starting.
z Temporal continuity, including personality,
communication ability and adaptability.
Pedagogical agent research and development has
made significant strides over the past few years,
incorporating animated computer characters that are
increasingly more realistic and human-like with
respect to their dialogue, appearance, animation and
the instructional outcomes they produce [19]. The
literatures concerning to the pedagogical agents are
abundant, such as [3], [18].
2.3. Pedagogical psychology
Pedagogical psychology, also named as educational
psychology, is a branch of applied psychology that
studies the application of the conclusions of theoretical
psychology to the process of education and teaching
[2]. The appropriate application of pedagogical
psychology to teaching practice can improve learners'
performances. The incentive mechanism in
pedagogical psychology takes a role as a stimulator
that can produce or strengthen the motivation of the
learners and make them in a positive emotional state
during the learning process. Four main sub-
mechanisms to enhance the learners’ internal drive in
the incentive mechanism are concluded below [2]:
z Goal mechanism, this is the foundation of the
incentive mechanism. Setting a reasonable and
clear goal to the learners and let the learners to
perceive it and produce the motivation to achieve
it.
z Competition mechanism, this is the propulsion in
the incentive mechanism. In group study, the
appropriate application of competition can
strengthen the learners’ motivation and promote
the learners’ performance to approach the best.
But in order to reach the positive effect, the
learners should treat the competition results
correctly.
z Co-operation mechanism, this is an effective
method in incentive mechanism. Learning might
be facilitated through participating in the co-
operative activities. For example, similarly
competent peer can invite the learner to active
argumentation or suggest ideas to cause the
learner’s cognitive conflict.
z Evaluation or feedback mechanism, this is the
core of the incentive mechanism. The tutors
should give the learners reasonable evaluation at
the appropriate opportunity. Through the
felicitous evaluation, the learners’ passion,
interests and confidence will be improved vastly.
In another aspect, through the appropriate
 evaluation, the learners’ depression and
impatience could be eliminated at some level.
2.4. Combination of the pedagogical
psychology into the pedagogical agents with
the support of artificial psychology theory
Artificial psychology theory provides the theory
foundation and computing model for adding the
pedagogical psychology into the pedagogical agents.
For example, in [6] the gesture-emotion model offers
the interplay relationship between gesture and emotion
and the computing model to know the learners’
emotion state. According this model, the learners’
emotion states have influence on the gesture or
behavior, so the gesture, facial expression or other
behavior can be used to detect the current emotional
states of the learners. In [8], an E-learning framework
integrated with the techniques and knowledge from the
modern cognitive science framework was proposed to
automatically respond to learners’ dynamic
psychological reactions.
Pedagogical agents inherit the features from the
normal software agents such as autonomy,
communication ability. The incentive mechanisms can
be integrated in the design of pedagogical agents. For
example, the goal mechanism can be realized by the
Expert and the Mentor agent. The Expert agent is used
to demonstrate the goal to the learner and let the
learner to perceive it. A Mentor agent can be used to
create the personalized study schedule and mode. With
regard to co-operation mechanism, the Mentor and
Motivator Agents can be used to organize the group
work. The Mentor agent can be used to create the
competition for the learner with other learners and the
motivator agent takes the role of a virtual competitor,
or a study companion. In relation to evaluation
mechanism, the Mentor agent and the Motivator agent
can be used to give the appropriate feedback to the
learner to stimulate them to strive for further
improvements.
Artificial psychology is the theoretical foundation,
pedagogical agents are the technical methods, and the
incentive mechanism is the design concept. Therefore,
this paper proposes an E-learning framework
combination of the pedagogical psychology with the
pedagogical agents underpinned by the support of
artificial psychology theory.
3. An innovative E-learning theoretical
framework
Based on the related work listed in section 2, an
innovative teaching and learning framework is
proposed in this section and the framework structure is
showed in Figure 1. The design of framework is to
sufficiently motivate the learners’ internal drive in the
learning process. Its theoretical foundation is artificial
psychology theory and the incentive mechanism in the
pedagogical psychology. This framework includes
three type pedagogical agents, the Expert agent, the
Mentor agent and the Motivator agent. And it has two
databases, knowledge base and learner’s profiles base.
The interface between this system and the learner is
based on Bio-feedback and/or Behavior. The Bio-
feedback signals, such as brain wave, heart rate, BVP,
skin conductivity and respiration, can reflect the
learner’s emotional state. This method needs the
support of the special devices such as biofeedback
instruments to collect the bio-feedback signals from
the learner, so this method increases the cost and
implementation complexity. This is only recommended
in highly specialised training environments such as the
training for pilots or cosmonauts. A Behavior based
method is low-cost and the learner does not need to
wear the sensors for the bio-signal collecting. The
gesture and the facial expression can be collected by a
video camera, the speech can be received by
microphone or headphone, and mouse and/or keyboard
usage can be monitored
An Analysis Module based on the Artificial
Psychology Computing Model is designed to realize
two chief goals as below:
z Receive the input both by the Bio-feedback
and/or Behavior modes and provide the
computing processing to transfer the different
input data to the learner’s emotional state or
character to store in the learner’s profile database.
The research of bio-feedback signal extraction
and analysis are the key techniques. Artificial
psychology offers the theory foundation and
computing methods.[6][8]
z Provide an artificial psychology computing
module to offer the optimal stimulation event to
let the learner’s emotional state reach the best
status for study based on the current emotion. In
terms of the results of the psychology
experiments, the optimal emotional state in study
process means positive, relaxing, confident,
opportune exciting, concentrated and so on.
Therefore, if the learner’s emotional state is
depression, the system should give the learner
some encouraging signal and also can adjust the
difficulty level of the test to help the learner to
get the confidence and positive emotional state
back. The existing research works about how to
 model and calculate the emotional states
conversion are in [10] [11] [12]. Further research
work should focus on the relationship between
the stimulation events and the following
emotional state.
around the incentive mechanism, for instance, the
positive evaluation and feedback, and encouragement
in the learning process. Apart from this, the Motivator
Agent can also play the role as a virtual competitor or a
study companion. These three agents work in a multi-

Bio-feedback Behavior
----------------------------- -----------------------------
Brain Wave, Gesture,
Heart Rate, Facial Expression,
Blood Volume Pressure, Speech Content,
Skin Conductivity, Speech Intonation,
Respiration. Actions to the Computer.

Analysis Module based

on Artificial Psychology Learner’s Knowledge
Computing Model Profiles Base

Motivator Agent Mentor Agent Expert Agent

------------------ ------------------ ------------------
Virtual Competitor, Preference Setting Demonstration
Group Competition, Goal Setting Exercise and
Positive Evaluation and Study Strategy Selection Test Bank
Feedback. Exam paper setting
Test and Mark

Figure 1. The Framework of the Virtual Teaching and Learning

The knowledge base is designed to store the study
contents, the exercise and test bank. The learner’s
profiles base is designed to store the learners’ profiles,
such as their basic personal information, character,
mood, emotional state records, and preference. This
database is used to provide necessary information for
the calculation in the artificial psychology module.
The pedagogical agents are divided into three types
in terms of the three functions of a tutor in real
teaching and learning environment. They are
knowledge delivering, study activities arrangement and
organization, the stimulation to the learner’s emotional
state respectively. Corresponding pedagogical agents
are designed as the Expert agent, the Mentor agent and
the Motivator agent. The Expert agent focuses on the
demonstration and training like in the traditional E-
learning system. The Mentor Agent focuses on the
organization work on the ground of the learner’s
preference. The Motivator Agent focuses on the work
agent mode, independence mutually, cooperation and
coordination, communication, and negotiation.
The flow chart of the study scenario is showed as
Figure 2. The implementation steps are explained as
follows:
1. Personal Information and Emotional State
Collecting, the personal basic information only
need to be inputted once initially, and the
emotional state will be collected in real time.
2. Preference Setting, the learner can set the virtual
study environment or other parameters, for
example, the skin of the system, according their
own preference.
3. Study Strategy Selection, the Mentor Agent will
select appropriate study strategy for the learner
according their character and their knowledge
structure and level. If the learner’s basic
knowledge is not good, the system can add a
review part for the foundation knowledge.
 Personal Information and Emotional
State Collecting
Knowledge
Base
Preference Setting

Expert Agent
Study Strategy Selection

Goal Setting
Mentor Agent

Demonstrating
Analysis
Learner Module
Profiles based on Practice: Co-operation/Competition
AP
Exam paper setting
Motivator Agent
Test

Evaluation/Feedback

Figure 2. The Flow Chart of the Study Scenario

4. Goal Setting, the Mentor Agent sets a goal for the
learner in one unit based on the learner’s previous
performance and current emotion state.
5. Demonstrating, the Expert Agent plays the role of
an introducer or a demonstrator in a well-defined
subject.
6. Practice: Co-operation/Competition, the Mentor
Agent and the Motivator Agent set the co-
operation and competition activities for the
learners coordinately to stimulate the learners’
internal drive. The motivator agent even can
produce a virtual competitor or companion for the
learner to stimulate their learning motivation.
7. Exam paper setting, the Expert Agent sets the
content and difficulty level of the exam paper in
collaboration with the Mentor Agent.
8. Test, the learner attends the test and submits the
answers back to the system. The test can be held
by the Mentor Agent.
9. Evaluation/Feedback, the Mentor Agent marks
the answers submitted by the learners and the
Motivator Agent according to the learner’s
performance and their emotional state to give
positive and encouraging evaluation and feedback
to the learner.
4. Future work
This paper proposes an innovative teaching and
learning theoretical framework based on psychological
incentive mechanism and artificial psychology. This
framework aims at creating a positive and personalized
study environment for the learners with consideration
to the learner’s character, emotional ingredients, and
preference. In the study process, the learners’ internal
drive is expected to be promoted to be more active by
the techniques and mechanisms in this framework.
And in this E-Learning framework, the learners are
easy to perceive the sense of achievement and
enjoyment in the learning process. But this framework
is just a start, and some key problems need to be
solved in the future:
1. Research in the area of Artificial Psychology is in
its initial stages, the analysis module to be based
on artificial psychology still has many theoretical
and technical issues that need to be further
explored.
2. This framework is based on multi-agents system,
so it is necessary to undertake further research on
the coordination and communication issues
among these agents.
3. Further research should be focused on the on Artificial Psychology Theoretics”, Artificial Intelligence,
methods of how to deliver the feedback from the 2006, pp.264-266.
virtual pedagogical agent to the learner, including [13] W. Lewis Johnson, Jeff Rickel, Randy Stiles, Allen
the virtual affective production and representation. Munro1, “Integrating Pedagogical Agents into Virtual
4. Further developments in pedagogical theory and Environments”, Teleoperators and Virtual Environments
experiment will be incorporated into the 7(6): December 1998. pp. 523-546.
framework [14] Chioma Okonkwo, Julita Vassileva, “Affective
5. Finally, the framework should be realized to test Pedagogical Agents and User Persuasion”, "Universal
the effect and it should be evaluated by the Access in Human - Computer Interaction (UAHCI)", 9th
International Conference on Human- Computer Interaction,
learners. The experiment results should be used
New Orleans, USA, Laurence Erlbaum, pp.397-401.
to modify the framework further.
[15] Makis Leontidis and Constantin Halatsis, “An Affective
Model for Personalized Learning in Adaptive Educational
5. References Hypermedia Systems”, Proceedings of Workshop on
Personalisation in E-Learning Environments at Individual
[1] Clark, R., and Mayer, R.E., e-Learning and the and Group Level, 11th International Conference on User
Science of Instruction. San Francisco, CA. Pfeiffer. Modeling, 2007, pp. 77-78
2003 [16]Barry Kort, Rob Reilly, Rosalind W. Picard, “An
[2] Feng Zhongliang, Wu Xinchun, Yao Meilin, Wang Affective Model of Interplay Between Emotions and
Jianmin, Pedagogical psychology, People’s education Learning: Reengineering Educational Pedagogy—Building a
Learning Companion”, Advanced Learning Technologies,
Press, 2000
Proceedings. Madison, WI, USA, 2001,pp.43-46.
[3] Baylor, A. L. & Kim, Y, “Simulating instructional roles
[17] Selker, T., Coach: A Teaching Agent that Learns,
through pedagogical agents”, International Journal of
Communications of ACM, Vol. 37. No. 7, 1994, pp.92-99.
Artificial Intelligence in Education, 15(1) , 2005,pp.95-115.
[4] WANG Zhiliang, “Artificial Psychology”, Human [18] Shaw, E., Johnson, W. L., & Ganeshan, R. Pedagogical
Interface, Part I, HCII 2007, LNCS 4557, 2007. pp. 208 – agents on the web, The third international conference on
217. autonomous agents, Seattle, WA, 1999.
[5] WAN G Zhiliang, XIE Lun. Artificial psychology —an [19] Baylor, A. L. , Cole, R. Graesser, A. & Johnson, L.
attainable scientific research on the human brain, IPMM’99 Pedagogical agent research and development: Next steps and
( KEYNOTE PAPER), Honolulu, USA ,1999. pp.1067-1071. future possibilities, Proceedings of AI-ED (Artificial
Intelligence in Education), Amsterdam, 2005.
[6] Wang,Xiaowei Wang,Zhiliang “E- Learning
Assistant System Based on Artificial Psychology”,
Artificial Intelligence, 2006, pp.275-277.
[7] Quan Lu, Jing Chen, JunPing Qiu, “Individual-Centered
E-Learning Based on Artificial Psychology”, Service
Systems and Service Management, June 2007, pp.1-4.
[8] H. Ekanayake, D.D. Karunarathna, K.P. Hewagamage,
“Emotion-based Affective Cognitive Architecture for
Meditative E-Learning”, 8th International Information
Technology Conference, Colombo, Sri Lanka ,2006.
[9] Choonhapong Thaiupathump, John Bourne, J. Olin
Campbell, “Intelligent Agents for Online Learning”,
Journal of Asynchronous Learning Networks Volume 3,
Issue 2 - November 1999,pp.1-18.
[10] CHENG Ning, FAN Yumei, LIU Jiwei and WANG
Zhiliang, “Application of Basic Emotions Theory in
Construction of Artificial Psychology Model”, Computer
Engineering, November 2005, pp. 175-177.
[11] GU Xuejing , WANG Zhiliang , LIU Jiwei, LIU Shan,,
“Research on Modeling Artificial Psychology Based on
HMM”, Computer Application and Research，2006，pp.
30-32.
[12] Gu , Xuejing Wang , Zhiliang Wei, Zhehua Wang ,
Chao, “The Research of Constucting Affective Model Based