AgentSpotter: A MAS Profiling System For Agent Factory

AgentSpotter: a MAS Profiling System for
Agent Factory
Dinh Doan Van Bien
Thesis presented for the degree of

Masters in Advanced Software Engineering
to the
School of Computer Science & Informatics
College of Engineering, Mathematical & Physical Sciences
University College Dublin
Research Supervisor: Dr. Rem W. Collier

Head of School: Dr. Joe Carthy
October 2008
Abstract
Recent advances in Agent Oriented Software Engineering (AOSE)

have focused on providing agent-oriented frameworks and integrated
tools in order to facilitate the creation of multiagent systems (MAS).
The inherent complexity of MASs means that performance and scala-
bility issues are likely to arise. Yet there is a lack of industry-grade
tools that enable software engineers to understand and debug these
crucial problems.
In this dissertation, we address this deficiency through the devel-
opment of a novel MAS performance analysis system entitled Agent-
Spotter, based on the mapping of conventional profiling concepts
to agent-oriented concepts. AgentSpotter’s platform-independent
architecture is composed of a MAS run-time profiling data collec-
tion infrastructure and a suite of interactive visualisation tools (Flat
Profile, Call Graph Tree View, and Space Time Diagram) that use
the collected data to provide new insights into a MAS run-time
performance.
We present our implementation of AgentSpotter for Agent Factory
(AF), a Java-based agent-oriented toolkit developed by researchers
at the PRISM Laboratory (University College Dublin).
Finally, we demonstrate the effectiveness of AgentSpotter by
using it to profile a set of representative MASs developed with Agent
Factory.
Contents
Abstract iii
Contents v
List of Figures ix
List of Tables xi
Acknowledgements xiii
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Agents and multiagent systems 5

2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 FIPA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Agent Factory . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 JADE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 Cougaar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3 Profiling 9
v
vi Contents
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Origins of profiling . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 First generation profilers . . . . . . . . . . . . . . . . . . . . 13
3.5 Second generation profilers . . . . . . . . . . . . . . . . . . . 14
3.6 Modern Java profilers . . . . . . . . . . . . . . . . . . . . . . 15
3.7 Profiling in perspective . . . . . . . . . . . . . . . . . . . . . 16
3.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4 Performance concepts and solutions in MASs 19

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2 Fundamental performance issues in a MAS context . . . . . 21
4.3 MAS scalability issues . . . . . . . . . . . . . . . . . . . . . 23
4.4 Facilitating MAS performance case studies . . . . . . . . . . 27
4.5 Cougaar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.7 Towards a set of requirements for a performance analysis toolkit 34
5 A blueprint for AgentSpotter, the Agent Factory profiling

system 37
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.2 Platform-independent AgentSpotter architecture . . . . . . . 39
5.3 Platform-specific AgentSpotter architecture for Agent Factory 43
5.4 AgentSpotter Station . . . . . . . . . . . . . . . . . . . . . . 45
5.5 Session Summary Tables . . . . . . . . . . . . . . . . . . . . 46
5.6 Call Graph Tree View . . . . . . . . . . . . . . . . . . . . . 49
5.7 Space-Time Diagram concept . . . . . . . . . . . . . . . . . 56
5.8 Space-Time Diagram user interface . . . . . . . . . . . . . . 60
Contents vii
5.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6 AgentSpotter design and implementation 69

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.2 AgentSpotter implementation architecture: an overview . . . 70
6.3 AgentSpotter profiling infrastructure . . . . . . . . . . . . . 72
6.4 AgentSpotter File Manager . . . . . . . . . . . . . . . . . . 75
6.5 AgentSpotter Station . . . . . . . . . . . . . . . . . . . . . . 76
6.6 AgentSpotter Visualisation Components . . . . . . . . . . . 77
6.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7 Agent Spotting: an evaluation 85

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7.2 Benchmark application . . . . . . . . . . . . . . . . . . . . . 86
7.3 Agent Sherpa . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.4 Agent Herders . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7.5 Evaluation summary . . . . . . . . . . . . . . . . . . . . . . 98
8 Conclusions 103
8.1 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . 103
8.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
8.3 General conclusion . . . . . . . . . . . . . . . . . . . . . . . 105
Appendices 109
A Building AgentSpotter from the subversion repository 109
B Getting started with AgentSpotter 111

B.1 Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
B.2 AgentSpotter Station . . . . . . . . . . . . . . . . . . . . . . 112
viii Contents
C AgentSpotter Snapshot File Database Schema 115
Bibliography 117
List of Figures
3.1 Profilers time line . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.1 AgentSpotter abstract (platform-independent) architecture . . . 40

5.2 AgentSpotter for Agent Factory concrete architecture . . . . . . 40
5.3 Call Graph Tree View of a fictional C program which removes
duplicate lines from a file . . . . . . . . . . . . . . . . . . . . . . 49
5.4 Agent message impact concept diagram . . . . . . . . . . . . . . 51
5.5 AgentSpotter Call Graph Tree View levels . . . . . . . . . . . . 53
5.6 Lamport Space-Time Diagram . . . . . . . . . . . . . . . . . . . 56
5.7 AgentSpotter Space-Time Diagram specification . . . . . . . . . 59
5.8 Agent Factory agent activity representation in the Space-Time
Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6.1 AgentSpotter implementation architecture . . . . . . . . . . . . 71

6.2 Agent Spotter Station sample screen shot showing all the available
views in a single take . . . . . . . . . . . . . . . . . . . . . . . . 78
6.3 Call Graph Tree View screen shot . . . . . . . . . . . . . . . . . 79
6.4 Space-Time Diagram screen shot . . . . . . . . . . . . . . . . . 84
7.1 Benchmark application control window . . . . . . . . . . . . . . 90
ix
x List of Figures
7.2 Benchmark application sample space-time diagram (18 minute

long session) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
7.3 Comparison of Agent Sherpa call graphs . . . . . . . . . . . . . 95
7.4 Comparison of Agent Sherpa space-time diagrams . . . . . . . . 96
7.5 Agent Herders’ call graph . . . . . . . . . . . . . . . . . . . . . 100
A.1 Netbeans “Open Project” dialog box . . . . . . . . . . . . . . . 110
C.1 Database schema diagram of the AgentSpotter profile data file 115
List of Tables
4.1 Common MAS scalability problems and solutions . . . . . . . . 25
5.1 OOP and AOP execution units mapping . . . . . . . . . . . . . 47
6.1 TraceEvent types generated by the AF-RTE sub-systems . . . . 73
7.1 Benchmark application flat profile . . . . . . . . . . . . . . . . . 91

7.2 Agent Sherpa application flat profile for 3 drone session . . . . . 94
7.3 Agent Sherpa application flat profile for 12 drone session . . . . 97
7.4 Agent Herders’ flat profile . . . . . . . . . . . . . . . . . . . . . 99
7.5 Comparison of the profiled sessions in terms of number of events 99
7.6 Comparison of AgentSpotter visualisations load times on a 2GHz
Dual Core Pentium machine . . . . . . . . . . . . . . . . . . . . 101
xi
Acknowledgements
I would like to thank my thesis supervisor Rem W. Collier for his outstanding
support and guidance.
My thanks also go to my company, Intelligo Software Ltd, for sponsoring
my Master’s degree.
Finally, I am deeply grateful to my wife Stéphanie for everything she has
done to make it all happen.
xiii
CHAPTER 1
Introduction
1.1 Motivation
Recent advances in agent frameworks and methodologies have helped bridg-

ing the gap between agent-oriented programming theory and practice. Yet
there is still a lot of work to do before agent-oriented software engineering
(AOSE) can truly break into industry, notably in the area of development
and debugging tools.
To this end, agent-oriented frameworks have been integrated into existing
mainstream Java development environments (IDEs) like NetBeans or Eclipse
to allow programmers to use existing tools and techniques originally designed
for an object-oriented context.
This reuse can work well at the source code level or the agent code level,
because the required activities’ nature and granularity are quite similar:
syntax highlighting, project building, stepping through the code. In other
words, the ancillary tasks of project and build management are conceptually
the same in object-oriented and agent-oriented software engineering, and
can be performed using the same tools and techniques. For example, in
1
2 Introduction
Agent Factory (AF) [11], a Java-based agent-oriented toolkit, a project is

constituted of a set of source code files, written in Java or in AFAPL1 , plus
a set of configuration and deployment files. An AF project can be built into
an executable set of JAR files using the standard Java build tool Ant.
At run-time, the analogy between object-oriented and agent-oriented can
be pushed further. As long as the object-oriented level is the appropriate
abstraction level for the problem at hand, an agent-oriented programmer
can efficiently use all the run-time debugging, memory inspection, and
performance analysis services provided by modern IDEs.
In fact, the AOSE proponents generally agree that the full integration
with existing IDEs is an important step towards the widespread adoption
of the agent-oriented paradigm, because it allows novice agent-oriented
programmers to build on their existing object-oriented skills in order to
create multiagent systems (MASs).
Nevertheless, we argue that modern object-oriented IDEs do not cover
all the needs of agent-oriented programmers “out of the box”, notably when
they are dealing with issues at the higher agent-oriented abstraction level.
Despite the recent development of AOSE tools like the AF Debugger
tool [7], which maps the traditional debugging concepts of state stepping
and breakpoint setting onto intelligent agents, tools that target software
developers whilst providing new insights into agent-oriented systems are
scarce. Moreover, after surveying the literature, we have determined that
one area is particularly lacking: MAS performance analysis tools.
It is commonly admitted that the behaviour of complex and distributed
MASs is very difficult to apprehend without adequate tools. To address this
problem, existing MAS toolkits usually offer a more or less advanced logging
1
Agent Factory Agent Programming Language
1.2. Objectives 3
system to help developers debug and understand their systems [10, 19].
While logging can deal with simple scenarios, software developers expect
more advanced tools to tackle more elusive and complex performance issues.
Ideally, MAS developers should be able to use a very popular performance
analysis technique called profiling. Profiling is based on the idea that in a
program, only a few places, called bottlenecks or hot spots, can be accounted
for the majority of the execution time of a program. Hence, by fixing
these bottlenecks only, the performance of a program can be substantially
improved. Profiling was introduced 40 years ago by Donald E. Knuth
in his empirical study of FORTRAN programs [26], and has since then
been successfully adapted to different languages, platforms and software
architectures, including large distributed systems.
We strongly believe that a profiler could truly help MAS developers,
novice or experimented. Yet, as far as we know, profiling concepts have
never been applied to agent-oriented programming. Therefore, we propose
to implement the first profiling system for a MAS.
1.2 Objectives
Our main objectives are as follows:
1. To map profiling concepts to agent-oriented concepts.
2. To elaborate a conceptual model for generic MAS profiling.
3. To realise and evaluate a profiling system based on this conceptual

model for a specific agent platform, namely Agent Factory.
In short, we aim to build a comprehensive MAS performance analysis

system for Agent Factory, based on profiling.
4 Introduction
1.3 Thesis outline
We have classically organised this dissertation in two main parts:
1. Fundamental Concepts and Background Research, which includes the

following chapters:
• Chapter 2 – Agents and multiagent systems.
• Chapter 3 – Profiling.
• Chapter 4 – Performance concepts and solutions in MASs.
2. Project Presentation, which includes the following chapters:
• Chapter 5 – A blueprint for AgentSpotter, the Agent Factory

profiling system.
• Chapter 6 – AgentSpotter design and implementation.
• Chapter 7 – Agent Spotting: an evaluation.

CHAPTER 2
Agents and multiagent systems
In this chapter, we present the fundamental notions of software agent and

multiagent system. We also briefly provide background information about
the different systems that we will reference later.
2.1 Definitions
For our purposes, a multiagent system (MAS) is a software system, situated

in a given environment, and composed of interacting coarse grained software
entities called agents. The most advanced agents are capable of intelligent
goal-directed autonomous behaviour (deliberative agents), whereas less
capable agents can only react to external events (reactive agents), but still
retain their autonomy by controlling their own thread of execution. Agents
have no direct control over their environment, instead they sense it through
sensors and act upon it using actuators. The interaction between agents
happens through an agent communication language which allows an agent
to influence the behaviour of another agent by sending messages based, for
instance, on the theory of speech acts1 .
1
See the FIPA ACL message structure specification in [15].
5
6 Agents and multiagent systems
In the perspective of the two-tier properties-based approach to agent

definition proposed by Wooldridge and Jennings in [52], our definition fully
covers the weak notion of agency (autonomy, social ability, reactivity, pro-
activeness), and relies on the stronger agency concepts of rationality and
intentionality.
Recent advances in MAS research have led to the emergence of agent-
oriented software frameworks, mostly Java-based, that allow the development
and the deployment of multiagent systems using standard software infras-
tructures and development tools. These frameworks promote the concepts of
agent-oriented programming (AOP) and agent-oriented software engineering
(AOSE).
Our work is based on the Agent Factory (AF) framework, described later
in §2.3. So, by all means, our working definition of an agent matches the
one underlying AF.
2.2 FIPA
The Foundation for Intelligent Physical Agents (FIPA) is an organisation

created in 1996 in Switzerland to lead the standardisation effort in agent-
based technologies [23]. Since 2005, FIPA has been officially accepted by
the IEEE as its eleventh standards committee. FIPA addresses a wide range
of agent-oriented technologies. The most important standards are surely the
Abstract Architecture specification [16] and the FIPA Agent Communication
Language (ACL) specification [15]. FIPA standards are widely adopted and
guarantee a minimum level of interoperability between the different agent
platforms.
2.3. Agent Factory 7
2.3 Agent Factory
AF is an open source Java-based, comprehensive toolkit for the creation

and deployment of agent-oriented applications. It is actively maintained by
researchers at the PRISM Laboratory in the School of Computer Science
and Informatics at University College Dublin. It has successfully supported
advanced research in a diverse range of problem domains: mobile & ubiqui-
tous computing, distributed sensor networks, social robotics, and augmented
reality [8].
It is structured around a FIPA-compliant run-time environment, which
offers basic system services (agent management, messaging, yellow pages),
and a system of pluggable agent development kits. The default AFAPL2
development kit provides a language and a reasoning engine for BDI (Belief
Desire Intention) agents, but other agent models can be plugged in like
the Reactive Message Agent model for example (RMA development kit).
Beyond its integration with the Java subsystem, a strong point of Agent
Factory is its integration with the mainstream Java IDEs NetBeans and
Eclipse.
2.4 JADE
JADE stands for “Java Agent DEvelopment Framework”. It is a major

open source project that benefits from an industrial level support through
the http://jade.tilab.com web site [50]. An extensive set of academic
papers, tutorials, and programming guides is available on-line, and recently
a comprehensive book geared towards developers was published [2]. JADE
has been used in a wide range of applications by companies and academic
bodies. It is so popular that on their web site the JADE maintainers claim
that “its spread surpasses even JADE team knowledge” and that they
8 Agents and multiagent systems
“apologize in advance for all those [companies or institutions] that have not
been listed” [50].
2.5 Cougaar
Cougaar is an open source Java-based platform particularly suited to the

construction of large scale MASs. Cougaar was originally funded by the
American Army (DARPA) and developed for military grade logistics deploy-
ment planning applications. In 2001 it was transferred to the commercial
domain as an “intelligent middleware” and has since then benefited from an
active commercial support. Cougaar is the only existing MAS framework
which integrates a performance measurement infrastructure right into the
system architecture.
CHAPTER 3
Profiling
3.1 Introduction
In this chapter, we describe profiling and explain why it matters in software

engineering. Our objective is to provide a solid understanding of profiling
concepts before we try to apply them to agent-oriented programming.
To this end, we start by defining what profiling is. Then, we explore
the elaboration of profiling concepts throughout history. Finally, we place
profiling in the greater perspective of software engineering.
3.2 Definitions
Most of the time in the software engineering literature, the definition of

profiling delves directly into technical aspects such as differentiating in-
struction counting profilers from sampling profilers, and it is assumed that
its underlying principles are well known and obvious. We believe a useful
definition should instead focus on these principles in order to understand
why profiling is so useful and so popular . Hence, we propose our own
9
10 Profiling
definition of profiling which tries to incorporate the elements that make

profiling an effective performance analysis technique.
General Definition. We define profiling as a performance analysis process

which consists in collecting, processing and analysing run-time data about
where a program spends its time, or how it spends its resources, in order
to gain a fresh insight into the main factors of the program’s performance,
or to detect hidden behavioural and resource usage patterns. The primary
motivation for profiling is performance debugging or tuning, which involves
iteratively identifying and removing the hot spots (areas that use much more
processing time than expected) or bottlenecks (areas that use processing time
or resources inefficiently and introduce unnecessary delays) in a program
until a satisfying performance level is reached.
Profiling involves the use of a profiler, which is composed of two equally
important logical parts:
• An instrumentation apparatus which is directly weaved into the pro-

gram under study or run side-by-side to gather and record execution
data;
• A post-processing system which uses the recorded data to generate

meaningful performance analysis listings or visualisations.
Limited Definition. A more limited definition of profiling designates only the

process of collecting aggregated performance data over the whole execution
of a program. The data generated this way is called a profile and is
constituted of counters, aggregated timings, or summaries. A profile must be
distinguished from a trace, which records the actual chronology of execution
events and generates a growing amount of data over run-time.
3.3. Origins of profiling 11
The limited definition does not contradict our general definition, but
it fails to emphasise the key value of profiling which is the eye-opening
quality of the information it extracts from raw performance data, providing
real insight into what drives or hampers the performance of a program and
revealing hidden patterns.
As explained in the next section, that eye-opening quality is what has
made profiling so remarkable right from its inception.
In the following sections, we summarise the history of profiling, following
the time line presented in Figure 3.1.
3.3 Origins of profiling
Nearly forty years ago, in his 1971 paper on the empirical study of FORTRAN
programs [26], Donald E. Knuth introduced a simple yet powerful dynamic
program statistical analysis technique that is still widely in use today:
program profiles.
Originally, a program profile or “frequency counts” was an annotated
program listing showing for each statement the number of times it was
actually executed and its estimated cumulative cost in terms of machine
cycles.
The profiles Knuth generated for his FORTRAN sample programs re-
vealed that “less than 4% of a program generally accounts for more than
half of its running time” [26], which implies that a programmer can make
substantial improvements to a program by using the program’s profile to
identify and remove the hot spots or bottlenecks that account for the ma-
jority of the execution time. This principle is sometimes called the “Pareto
Principle” or the “80/20 law”; it has been universally observed in domains
other than software development [40], and stated as the “vital few and trivial
12 Profiling
prof
Unix profiler ATOM JVMTI
gprof
Customized program New tool interface
Call graph execution
analysis tools API, JVMPI replacement
profiler
Knuth
JVMPI
Profilers
First Java profiler
First profilers
interface API
1970 1980 1990 2000 2010
Figure 3.1: Profilers time line
many” [48] — meaning that in anything a few are vital (20%), and many
are trivial (80%).
Interestingly, Knuth also discovered that often the programmers’ mental
map of the bottlenecks in their programs did not match the actual findings
of program profiles. In other words, profiles were real “eye-openers” and
gave them an unexpected insight into the actual run-time behaviour of their
programs (as opposed to the predicted behaviour). That fact advocates the
necessity of using tools that can help the programmer break the limitations
of their minds and analyse information about running software, which
is otherwise intractable because of the sheer amount of data and events
produced over run-time.
In retrospect, Knuth’s findings have been very influential and have
implicitly driven most of the performance analysis approaches in software
development in the last three decades.
3.4. First generation profilers 13
3.4 First generation profilers
3.4.1 Knuth profilers
Technically, the first profilers were the programs that Knuth used for his 1971
study [26]: FORDAP, based on instruction counting, and PROGTIME, based
on program status sampling. Their underlying principles were applied with
nearly no modifications to the first generation of profiling tools represented
by prof and pxp.
FORDAP would take a FORTRAN program as input and would output
an equivalent program that would also maintain frequency counts and write
them into a file. After being compiled and run, the modified program would
produce an annotated listing showing for each statement the number of
times it was actually performed and the estimated total execution time.
PROGTIME was based on a different approach called program status
sampling. PROGTIME would spawn the user program as a subtask and
stop at regular intervals to record with a time stamp the place where the
program under observation was executing. Instead of frequency counts,
PROGTIME would produce execution time listings.
Theoretically both approaches should reveal the same bottlenecks. How-
ever, program status sampling gives a more realistic view of the run-time
behaviour of a program by showing more clearly the low-frequency but time
consuming operations like input and output.
3.4.2 UNIX prof
In 1979, the UNIX implementers introduced a sampling profiler for the C

language called prof.
The main output of prof is a “flat profile” which shows how many times
a program spends in each function, and how many times that function is
14 Profiling
called. The sampling method used by prof means that it is less accurate
than an instruction counting profiler, also called “exact” profiler. When
the execution time of a function is smaller than the sampling interval, it
won’t be properly recorded. Despite these shortcomings, prof is reasonably
accurate for most uses.
Its use involves several steps:
• Compile the program using cc with the “-p” switch to instrument the
program.
• Run the program. After completion it generates a “mon.out” file

which contains run-time statistics.
• Set the current shell directory to the “mon.out” file location.
• View the statistics by typing “prof” on the command line.
prof is the most widely available profiler on UNIX systems, and is indeed
so popular that it is still in use today.
3.5 Second generation profilers
3.5.1 gprof: a call graph execution profiler
The main contribution of gprof, introduced in the 80’s by Graham et al. [17],
is the call graph. This concept improves on prof’s flat profile. It not only
summarises the time spent in different functions, but also recursively presents
all the call stacks annotated with the time spent in the different functions
called. Another name for the call graph is “hierarchical profile”, which
conveys the idea that gprof provides information to understand the impact
of one function in relation with all the functions that have called it. To put
3.6. Modern Java profilers 15
it crudely, a call graph shows the “where” to help understand why the “how
long” is a problem.
The textual output of gprof is very dense and requires some practice to
understand. We shall see in §3.6 how modern profiler user interfaces have
made the call graph more tractable.
3.5.2 ATOM
This instrumentation framework, created in the 90’s by Srivastava and

Eustace [47], has popularised the concept of allowing software engineers
to create custom program analysis tools. In short, ATOM provides an
infrastructure and a simple API which enables the creation of different tools
without having to start from scratch. ATOM has been used to create profilers,
execution tracers, run-time optimisers etc. That is to say, programmers who
are not satisfied with the service offered by standard tools can relatively
easily implement a tool tailored to their needs. This idea is the basis of
modern program analysis APIs as we shall see in the next section.
3.6 Modern Java profilers
The Java Virtual Machine Tool Interface (JVMTI) [49] and assorted tools
inherit from the systems and concept we have previously mentioned. The
main motivation of JVMTI is to provide a standard API like ATOM both
for specialised tool vendors and for casual developers with punctual instru-
mentation needs. The API gives access to JVM internal information items
(call stacks, class metadata, CPU usage counters etc.) that can be pieced
together to create a monitoring application, a profiler or even a debugger.
Modern Java profilers use the JVMTI as their back-end, and offer ad-
vanced technical features like “on-demand” profiling, remote profiling, thread
profiling etc. However, from a conceptual point of view, modern profilers
16 Profiling
have not introduced new ideas, they have just improved on the existing
concepts we presented: flat profiles, call graphs, profiling API. To be fair,
the adoption of graphical user interfaces has made profilers more usable
than ever. For instance, the visual representation of call graphs in tree view
controls has made their interpretation much easier.
3.7 Profiling in perspective
Performance is an important attribute of good software. Efficient software

should not make wasteful use of system resources, and should be as responsive
as expected by its users.
Having said that, from a business point of view it does not make sense to
be overly concerned by optimal performance when designing software, unless
it either poses a genuine problem (e.g. a program is not responsive enough
to satisfy its users or overruns its resources limitations), or is a primary
system goal per se (as in real time software).
That explains why the majority of programmers choose to use perfor-
mance analysis tools only punctually, when confronted with severe perfor-
mance problems. In other words, programmers will favour easy to use tools
that produce a substantial amount of improvements with the minimum
amount of effort.
3.8 Conclusion
In summary, we defined profiling and explained why it matters for program-

mers. We also retraced the history of profiling and concluded that modern
profilers are still using concepts invented one or two decades ago. All things
considered, programmers have a predilection for profiling when conducting
performance analysis, because it has a high benefit to effort ratio.
3.8. Conclusion 17
Clearly, profiling could be very useful for MAS developers. So how should
we proceed to map profiling concepts to agent-oriented programming1 ?
In a traditional OOP context, profiling is focused on methods, and
restricted to program or JVM boundaries, with the primary objective of
detecting bottlenecks by looking at method calls frequency counts, method
execution timings, call graphs, memory usage graphs etc.
In a MAS context, however, the profiles and the associated visualisations
are generally too fine-grained and too low level for a relevant analysis of
MAS run-time performance and activities. We shall see in Chapter 5 how
these concepts can be mapped to a MAS context. Beforehand, we need to
provide more theoretical background about the performance issues in MASs.
1
We must mention that profiling has been successfully ported to different programming
paradigms. We think the most striking example is the work of Brassel et al., who describe
in [3] how they have applied run-time profiling to functional logic programs.
CHAPTER 4
Performance concepts and
solutions in MASs
4.1 Introduction
In the light of the history and the current state of performance analysis
tools in non agent-oriented systems, and with the aim of developing a MAS
performance analysis system, the next logical step is to proceed to a survey of
the existing work on MAS performance, with a particular focus on integrated
tool-based solutions.
A first glance at the literature reveals a contrasted picture. On the one
hand, there is a good amount of literature on scalability, which is commonly
admitted as the most significant performance related concept in multiagent
systems. The main motivation of these studies is to provide solutions to the
scalability issues of potentially large MASs that enterprise level applications
require. The solutions described borrow heavily from the distributed systems
research area [13, 4] while trying to come up with concepts truly unique
to the agent-oriented paradigm like self-organising [51, 38, 6, 30] and self-
19
20 Performance concepts and solutions in MASs
tuning systems [30, 6]. On the other hand, it is harder to find papers
about comprehensive solutions that try to reduce the gap between theory
and practice and help non-expert, or impatient, MAS designers conduct
meaningful performance analyses of their system under construction. The
only example of an integrated performance analysis infrastructure that we
found is the Cougaar system described in [19].
Our point is that although MAS performance, specifically scalability, is a
seemingly well studied subject, there is still some work to do to translate the
general conclusions into a comprehensive and usable performance analysis
toolkit integrated with a modern agent platform, which we insist on as being
an important requirement to fulfil if we want to allow non-specialist software
engineers to design enterprise level MASs. Indeed, a growing number of
MASs are leaving the experimental arena for the production environments,
and given the more stringent quality of service requirements of enterprise-
level production systems and the likeliness of emergent complex behaviour to
exhibit performance issues, it is obvious that performance analysis will be an
important area in AOSE even if performance is not a primary requirement,
as in parallel programming for instance.
What’s more, the software engineers are much wiser now and will not put
up with reams of logging statements or time-consuming wheel-reinventing
experiments anymore. Even when using the modest JavaScript browser
language, they now are just a mouse click away from a wide choice of
profilers and other powerful performance analysis tools that provide detailed
performance reports and summarised performance indicators [37], and can
even suggest performance improvements automatically [53].
As a result we chose to structure our survey in a way that provides
a strong foundation for a practical approach to MAS performance anal-
4.2. Fundamental performance issues in a MAS context 21
ysis. Rather than simply enumerating the existing approaches to MAS

performance in a thematic fashion, our survey is organised around a prob-
lem/solution discussion that helps us progressively motivate the creation of
a MAS performance analysis toolkit.
In the first part of this chapter, we present a study of the fundamental
performance issues in MAS context and identify the major approaches
promoted in the literature to address these issues.
In the second section of this chapter, we show how MAS performance
analysis case studies could benefit from the existence of an integrated
performance analysis system.
In the third part of this chapter, we describe an existing performance
measurement infrastructure that could be used as the basis for a performance
analysis system.
4.2 Fundamental performance issues in a MAS context
Our goal is to present a theoretical view of the main issues of MAS per-
formance, and to identify the main solutions that have been proposed to
address these issues. We start by a necessary characterisation of the perfor-
mance of agent-oriented software and a definition of essential concepts like
performance and scalability. Then we proceed to a more detailed analysis
of scalability which is commonly viewed as the central performance related
issue in MASs. Finally we expose the different approaches proposed to
address scalability issues in MASs.
4.2.1 Basic performance concepts
In [45], Smith and Williams define performance as being a measure of how

quickly a software system or component executes compared to some prede-
fined time objectives. In other words, performance designates any aspect of
a software system that you can measure using a stopwatch. The authors
also state that the most important performance concepts are responsiveness
and scalability, and propose the following basic definitions:
Responsiveness measures how well a software system’s response time or

throughput meets a user’s performance requirements.
Response time measures how much time a software system takes to fulfil
an end-user request.
Throughput quantifies the number of requests that can be handled in a

given time slice, usually a minute.
Scalability indicates how well a system can maintain its response time or
its throughput under its performance objectives despite an increasing
workload. Actually, scalability is not a number but a function which
puts in relation the response time with the load, and can be plotted as
a scalability curve. Ideally the scalability curve should grow linearly.
These general definitions provide the basic concepts in order to char-

acterise the performance of software systems. Furthermore, they allow us
to clearly state that the performance of a system is perceived as problem-
atic if the system does not meet its performance objectives in terms of
responsiveness and scalability. To deal properly with performance issues, the
performance objectives that are missed must therefore be clearly identified,
so that the responsiveness and scalability can be fixed and improved.
4.2.2 MAS performance at different levels
In a MAS, performance must be viewed at different levels. At the agent level,

when considering only the autonomous computational entity abstracted
away from the interaction with its peers, the focus is on responsiveness. It
4.3. MAS scalability issues 23
is obvious that the main influence on the responsiveness of an agent is the

amount of computation it requires to carry out the set of tasks required
to meet its design objectives. Agreeing with the BT researchers in [29],
we make a further distinction based on the rationality level of the agent
architecture. For reactive agents this set of tasks includes only purely
reactive behaviour which is similar to the behaviour of a traditional process.
For deliberative agents that implement a reasoning system, the computation
cost of the reasoning activity must be considered separately from the actual
task execution.
As a side note, at the agent platform level, the different platform services
have a running cost that must be accounted for. The main subsystems to
consider include the Agent Management Service and the Message Transport
Service as defined in the FIPA Abstract Architecture Specification [16].
At the risk of stating the obvious, a performance toolkit should clearly
provide access to the essential performance indicators we have defined in
this section.
4.3 MAS scalability issues
At the MAS level, scalability is by far the most studied aspect of agent-
oriented systems performance in the literature, surely because it is commonly
recognised as the main performance issue in distributed systems. We have
summarised in table 4.1 the wide array of scalability issues that can arise
in a MAS. There is not a single approach available to solve these issues,
however the solutions shown in the table can be broadly described as
approaches to manage and reduce the load and the complexity of the system.
These approaches can be further classified depending on their level of agent
orientation. Solutions that propose the use of proved techniques from
the distributed systems area can be contrasted with solutions that focus
specifically on agent concepts like improved coordination algorithms, specific
MAS performance models, and at the other end of the scale, solutions that
empower agent systems to deal autonomously with scalability issues.
Of course both points of view are equally valid and interesting, but ulti-
mately we favour the truly novel agent-oriented solutions that demonstrate
the added value of the agent-oriented paradigm.
Additionally, performance case studies of common MAS structures and
architectures constitute an important research direction that can help MAS
designers make more informed design decisions. In the following paragraphs
we provide a more detailed development of each one of these approaches.
4.3.1 Solutions inspired by distributed systems
As explained in [4] and [13], well-known scaling techniques from the dis-
tributed systems area must not be overseen. These techniques include hiding
communication latencies, data and tasks distribution, and data replication.
In practice, Brazier et al. suggest in [4] that MAS designers should base
their implementation of scalable naming or messaging services on existing
middleware architectures or protocols (e.g. CORBA, DNS).
4.3.2 MAS performance prediction and evaluation models
Following the software performance engineering proactive approach [44],

models can be used to predict agent systems performance and evaluate
designs. For instance, in [34] Mur et al. elaborate a mathematical model
that allows the dynamic computation of the optimal number of agents
required to handle a given load in the Highly Organised Team of Agents for
Information Retrieval (HOTAIR) architecture. In [39], Rana et al. propose
a more generic model based on Petri nets, which uses the metrics gathered
Table 4.1: Common MAS scalability problems and solutions 4.3.
Characteristic Problems Solutions
Number of agents Too many concurrent tasks Schedule agents differently [13]
Add more CPU if the system scales linearly
Schedule tasks differently (e.g. by priority)
Self-tuning systems [30, 6]
Number of messages More messages than necessary are exchanged Change the coordination protocols to reduce
the number of messages
Messages take too much time to arrive Change the transportation method
Size of input data or rules Retrieval or processing time too high Reduce the amount of data
database
MAS scalability issues
Redistribute data processing

Cache data
Proactive agents Reasoning takes too much time or too much Change the reasoning engine
space
Reduce pro-activity
Redistribute roles
Reactive agents Too many events Change event propagation strategies e.g. use
broadcast [30]
Decouple agent interaction e.g. use a black-
board
Society Organisation Too much interaction Adopt a more structured organisation [29]
Self-organising agents [6, 51]
Centralised architecture does not scale up Replicate central components [34, 13]
Type of coordination Planning issues Use scalable planning algorithms [5]
Task delegation issues Use the Contract Net protocol [46, 24]
Administrative overhead Naming or Location service does not scale up Use existing middleware services [4]
25
about a subset of known agents to predict the performance of a much larger

system.
4.3.3 Self-building and self-organising agents
Numerous examples of agent-oriented approaches to scalability issues are

available. Turner and Jennings show in [51] how self-building and self-
organising agents can maintain and adapt their own organisational structure
in order to dynamically improve the global performance of a MAS. A series
of papers about the HOTAIR project [38, 6, 30], a scalable document
indexing system based developed at University College Dublin, provide a
good report on a concrete application of self-organisation to help a system
scale dynamically depending on the load.
4.3.4 Self-tuning agents
The HOTAIR system also contains a good example of self-tuning agents

that optimise their own behaviours depending on system or application
performance metrics [30, 6]. One motivation for self-tuning is to allow an
agent to degrade its individual performance in order to maximise the overall
performance of the system.
4.3.5 Performance analysis of common MAS structures
How does a mesh structure compare to a hierarchical structure? What’s the

impact on the task allocation computational load when an auction protocol
is used instead of a contract net protocol? These questions are examples of
what a MAS designer may ask when selecting an appropriate coordination
structure. A growing number of studies provide answers to this kind of
questions and will ultimately constitute a precious body of knowledge to
guide aspiring MAS designers. Among the most interesting studies, an early
work by Lee et al. [29] demonstrates that hierarchical structures are more
4.4. Facilitating MAS performance case studies 27
scalable than mesh structures; not surprisingly it also shows that an auction
protocol is more computationally expensive than a contract net protocol
because an auction is like a repetitive contract net. In that study, Lee et al.
also conclude that “the benefits of carrying out such investigations become
increasingly important as MASs mature” [29]. We completely agree with
their analysis, but we want to go further and add that carrying out these
performance studies should be greatly facilitated by extending the existing
agent frameworks with the appropriate infrastructure. In other words, as we
explain further in the next section, carrying out MAS performance analysis
should be made much easier in order to encourage the publication of more
studies.
4.4 Facilitating MAS performance case studies
As more MASs are leaving the experimental arena for the production envi-
ronments, performance becomes a greater concern, and it is important to
multiply the investigations into the performance of existing MAS designs.
To represent this type of work, we have selected the following papers:
• In [25], Juhasz and Paul compare the performance of two leading

open source agent platforms, Grasshopper and JADE, running the
same MAS. Their evaluation focuses on the data transfer rate between
agents and the time taken to complete some determined tasks.
• In [12], Cubillos et al. study the performance of a multiagent passenger

transportation planning system implemented with JADE in terms of
planning request average processing time.
In both studies the goal is to provide implementation specific insights into

the performance of a particular MAS so that implementers of similar systems
are aware of the potential pitfalls or advantages of the design decisions that
are made and of the platform that is used. Our main interest is not the
content of these studies. From a theoretical point of view, we actually think
that the scope of these studies is narrower than the generic architectural
performance analysis in [29], which we have described earlier (see §4.3.5).
However, from a more practical point of view, these studies are the sign
that MAS platforms and implementations are mature enough to serve as
background for other studies instead of being objects of study themselves.
We believe that the maturity of the MAS platforms and implementations is
an essential prerequisite to allow us to bridge the gap between the theory
and the practice of MAS performance analysis, because it highlights the
need for a generic performance analysis toolkit and it also makes the creation
of such a toolkit technically possible.
In this view, the fact that both systems run under the same platform,
JADE, has motivated our selection of these papers. While the performance
analysis objectives and measurements were completely different, the authors
have probably used the same platform services and API to set up their
experiments and gather the data for their performance analysis. Surely both
teams of researchers had to reinvent the wheel and create themselves the
tools they needed. This is a common problem in the academic world, and
it is not usually viewed as a serious one since it happens in the context of
one-off projects. Yet in production environments where performance analysis
is carried out more routinely as part of the software development life cycle,
MAS engineers will require clear methodologies and general toolkits to avoid
reinventing the wheel, and more importantly, to obtain meaningful results
they can apply immediately to improve their designs.
The availability of such a system might foster more interest in the
4.5. Cougaar 29
evaluation of the performance of existing MAS designs by making the results

easier to obtain. This has happened in the traditional software engineering
community thanks to the introduction of historical profilers like gprof [17],
or performance analysis APIs like ATOM [47] and the Java Virtual Machine
Tool Interface (JVMTI) [49], which have made performance analysis more
accessible for researchers and software engineers.
Without detailing the actual requirements for such a system, we can
nevertheless immediately identify the necessity of a solid performance mea-
surement infrastructure, directly integrated into the agent architecture. A
good example of that architecture is the Cougaar platform that we describe
further in the next section.
4.5 Cougaar, a MAS with a built-in performance

measurement infrastructure
In this section we examine an advanced military grade Java agent platform

called Cougaar, presented earlier in §2.5, which integrates a performance
measurement infrastructure right into the system architecture. While appre-
ciating the great depth and scope of the Cougaar performance measurement
system, we identify the further requirements which we believe are missing
in Cougaar to make it completely relevant in the modern landscape of
multiagent platforms. Indeed, the need for non-expert friendly and time
saving tools has recently risen as a primary concern in multiagent platforms,
which is a direct consequence of the recent development of AOSE.
4.5.1 Requirements
In [19], Helsinger et al. describe the performance measurement infrastructure

they have built into the Cougaar agent architecture. Their approach is based
on a rigorous analysis of the challenges of performance measurement in
distributed systems, particularly in distributed multiagent systems, namely

the potentially voluminous amount of performance data, the problem of
coordinated collection of widely distributed data, the necessity of minimising
the impact on the measured system and finally the ability to cope with
changing measurement needs and priorities. To tackle these challenges they
state a clear set of general performance measurement requirements. Their
ideal system should be extensible so as to allow the gathering of performance
data at any place in the system. The impact on system performance should
be minimal. The collected data should be readily available and accurate
for internal or external use. The system should be able to deal with large
amounts of performance data. Eventually the system should be able to
dynamically adapt to changes in measurement needs and priorities.
Helsinger et al. refine these requirements further by classifying perfor-
mance metrics according to their different users (internal or external), usages
(state information, trend analysis, change notification), method of propaga-
tion (in-band or out-of-band) and abstraction level (computer system level,
generic agent-system level, highest level application-specific measurement).
Indeed, the different uses of performance metrics have an impact on how
they are collected and then delivered to their intended recipient.
4.5.2 Design and implementation
The creators of Cougaar claim to have comprehensively incorporated these

requirements into the design and implementation of the Cougaar agent archi-
tecture by developing sub-systems specifically geared towards performance
measurement (“Metrics Service”, “Logging Service” and “Cougaar Events”
mechanism) and allowing performance measurement extension points at
different levels in the system. One particularly interesting feature of Cougaar
agents is their self-tuning ability through an “Adaptivity Engine” under the
4.5. Cougaar 31
supervision of higher level system policies dynamically set by other agents

or by a human operator.
Without detailing the full implementation of the Cougaar performance
measurement infrastructure, we can summarise its main characteristics as
follows:
• Different performance data channels provide metrics of different scopes

and different types. The primary channels are raw polling sensors
at the heart of the system execution engine which gather simple low-
impact data elements such as counters and event sensors that are
triggered when the system steps through predefined places in the code.
The secondary channels provide more elaborate information: status
sensors which summarise the state of Cougaar components; history
(or trend analysis) channel which aggregates and stores performance
data over long running times.
• Computer level metrics (CPU, network, memory usage statistics) are

collected inside the Cougaar execution engine and made accessible as
platform-independent data through the “NodeMetricsService”.
• Agent architecture level metrics are made externally available through

the “BlackboardMetricsService” and can be accessed internally via a
raw sensor channel.
• The message transport service captures statistics on the messages that

flow through it.
• A servlet-based extension mechanism allows the addition of visualisa-

tion plug-ins that feed on the performance metrics data source.
• Cougaar provides an event and logging service based on the Log4J

logging framework that can be used to log performance metrics.
• The instrumentation services can be dynamically enabled and disabled

either from the outside or from the inside through a self-steering
mechanism.
• The flexible design of the Metrics service (use of call-backs and listeners)
ensures that it has no impact on the system when it is not in use.
4.5.3 Critique of Cougaar
The comprehensiveness of the Cougaar system approach to MAS performance

measurement has currently no equivalent that we know of. Thanks to a
strong theoretical background, the Cougaar framework seems to cover all
the potential scenarios of performance measurement in a MAS. It is difficult
not to agree with the creators of Cougaar that “the Cougaar performance
measurement system offers a great range of performance data, through a
variety of channels, with dynamically modifiable operating modes, with low
system impact.” [19]
However, if we had to re-implement the Cougaar performance measure-
ment system, while keeping the core infrastructure, we would add a more
explicit provision for an interface that facilitates the creation of performance
analysis tools. In essence, the extended interface should allow tools to gather
run-time metrics data and to control the execution of the MAS at the same
time in a more unified way.
As a concrete example, the interface should allow a tool to connect to a
running MAS, hook up to some events of interest to gather run-time metrics,
then maybe pause and restart the system at some critical points, and finally
stop monitoring the system to present analysis results to the user.
4.6. Discussion 33
4.6 Discussion
4.6.1 Lack of a tool-based MAS performance analysis system
The breadth of our survey has clearly shown the extent of MAS performance
issues and the variety of solutions proposed to solve these issues. We have
also discovered that the tool-based approach to performance analysis we
advocate in this dissertation has been surprisingly neglected in the agent-
oriented paradigm. Yet, as we have demonstrated in Chapter 3, this approach
is favoured in all the other major programming paradigms and is usually
backed up by a solid theoretical foundation. In fact, tools ultimately bring
together performance analysis theory and practice.
The relatively recent focus on the adaptation of mainstream software
engineering practices for the fabrication of agent-oriented systems could
explain the lack of a tool-based approach to performance analysis. Now that
AOSE has become a practical possibility, it is time to integrate the practices
that will help make agent-oriented programming a viable proposition for
mainstream enterprise-level software projects.
To put it another way, we have seen how the MAS community has done
the important work of elaborating a set of concepts to tackle performance
problems in MASs. However, the prescriptions and ideas will only be truly
useful when they can be applied, evaluated or enforced through a set of
practices, methodologies, or even better, tools.
4.6.2 Motivation for a MAS performance analysis toolkit
Modern software performance engineering (SPE) promotes a proactive ap-

proach to performance issues by integrating performance-oriented principles
and best practices into the engineering process from start to end, which
can entail a significant additional upfront cost [45]. In practice, a reactive
approach is the norm, where corrective measures are taken only when the
performance issues are discovered late in the process, which usually means
at best a significant redesign, or at worst the failure of the project due to
intractable issues in the upfront design of the system.
As with most software practices, we contend that the reasonable approach
certainly lies half way and should follow the so-called Pareto principle. Our
idea is that for 90% of the code, software engineers should focus primarily
on functionality. For the remaining 10% that are usually linked to non-
functional problems like performance, 90% of the potential performance
issues should be addressed as automatically as possible by the software
itself, either passively by code reuse that integrates the best practices, or
proactively by using agent-oriented software solutions. The remaining 1%
(10% of 10%) can be addressed manually using an easy to use performance
analysis toolkit which gives as much information as possible to software
engineers about the problems at hand.
In other words, a MAS naturally generates complex emergent behaviours
that are likely to exhibit performance issues. Therefore, despite the embed-
ding of the best performance management principles and practices right into
the software, still in a lot of instances the intervention of a human software
engineer is unavoidable. Hence the necessity of providing a comprehensive
toolkit to help software engineers tame the complexity of MAS performance
analysis.
4.7 Towards a set of requirements for a performance

analysis toolkit
In this chapter we have surveyed the MAS performance concepts and solu-
tions and offered an overview of the most common MAS scalability issues in
4.7. Towards a set of requirements for a performance analysis toolkit 35
table 4.1.
Among the cited approaches, we have focused on performance case
studies as a good way to disseminate knowledge about the performance
characteristics of different MAS designs and architectures. After noticing
that this research direction is not as developed as it should be, we have
suggested that the reason for that scarcity could be the current lack of
comprehensive MAS performance analysis system.
Then, as a fundamental requirement for a performance analysis system,
we have identified the necessity of a performance measurement infrastructure
similar to the one built right into the Cougaar agent architecture that we
have reviewed. From our review of the Cougaar system we have concluded
that while a solid performance measurement infrastructure is a good basis for
a performance analysis system, we should also provide an unified interface,
in the form of a toolkit, to facilitate the creation of performance analysis
tools like profilers.
The important point now is to define extensively the requirements of
such a toolkit, which is the main subject of the next chapter.
CHAPTER 5
A blueprint for AgentSpotter,
the Agent Factory profiling
system
5.1 Introduction
In the previous chapter we have broadly studied the different aspects of

multiagent systems performance and scalability. We have concluded that
despite the subject’s good theoretical coverage in the literature, and the
existence of performance-minded infrastructures, there is a lack of tools that
facilitate MAS performance issues’ evaluation and analysis from a software
developer’s point of view.
Ideally, agents and MASs should be empowered to deal autonomously
with performance and scalability issues, which is a research area in itself.
However, until generic solutions are found for this problem, the human design
factor remains preponderant, and some specific challenges must be addressed
at the conceptual level and at the infrastructure level to build a toolkit that
37
38 A blueprint for AgentSpotter
will truly help software developers understand and fix performance issues in
MASs.
In this chapter, our goal is to detail our vision of such a toolkit based
on the well-known profiling technique, from the abstract concepts to the
user interface. Our approach is as platform-independent as possible, even
though in the end, we have chosen the Agent Factory platform to implement
a prototype of that system, which we have named AgentSpotter.
We must emphasise that while there are many visualisation programs for
MASs, AgentSpotter is the first desktop profiling system which deals with
agent-oriented concepts and is completely integrated with a mainstream
AOSE development framework.
5.1.1 AgentSpotter core objective
In brief, the core objective of AgentSpotter is to map the traditional concepts

of profiling to agent-oriented concepts in order to build a profiler tool for
MAS developers.
Profiling is indeed the dominant software performance analysis tech-
nique in use in traditional software development paradigms. As we have
explained in Chapter 3, profiling’s key value is the eye-opening quality of the
information it extracts from raw performance data, providing unexpected
insights into what really drives or hampers the performance of a program
and revealing hidden patterns.
5.1.2 Why an agent-specific profiler?
Before describing what the ideal profiler for a MAS should be, we must first
address the main objection that could be expressed against our approach: as
most MASs are based on Java platforms and are built around Java classes,
why cannot we just use the profilers that come with these systems?
5.2. Platform-independent AgentSpotter architecture 39
Our answer is that we need a specific profiling system for multiagent sys-
tems because a Java profiler does not operate at the appropriate conceptual
level. While there can be a technical mapping between objects and agents,
it is usually difficult to connect method calls with specific agent activities
like messaging or deliberation.
Actually, our first attempt at the problem would have been to use the
output of a traditional Java profiler and to try to infer agent-specific activity
from low-level method profiles. The fundamental flaw with that technique is
that it wouldn’t allow us to group the different statistics by agent instance,
so we finally chose to design an agent-specific infrastructure.
5.1.3 Analysis outline
Our analysis starts with a discussion about the required infrastructure for
AgentSpotter. First, we describe AgentSpotter’s abstract, i.e. platform-
independent, architecture. Then we adapt that abstract architecture to the
Agent Factory platform.
Once the overall architecture has been clearly defined, we then set to
detail extensively the requirements for our main visualisation tool called
AgentSpotter Station. Our requirements analysis is particularly focused on
the concepts underlying the novel visualisations that we have introduced.
5.2 Platform-independent AgentSpotter architecture
5.2.1 High-level overview
We provide an overview of AgentSpotter’s abstract architecture in Figure 5.1

using the following graphical conventions:
• Top-level architectural units are enclosed in dashed lines and are titled
in slanted capital letters, e.g. AGENT PLATFORM
AGENT PLATFORM
Profiled Application messages Run-Time Environment AgentSpotter Service

activity
events Profiler
events
System Monitors
data
AGENTSPOTTER STATION (visualisation application) QUERY ENGINE
Snapshot File (.aspot)
Session Summary Tables queries Session summary query
Agent activity query
Space-Time Diagram data Events
Messages sent/received query Sessions
Call graph query

Call Graph Tree View Messages
System activity query
Agents
Figure 5.1: AgentSpotter abstract (platform-independent) architecture. Top-

level architectural units are enclosed in dashed lines. Self-contained software
packages are enclosed in solid lines. Arrows denote data or processing
interactions.
AGENT FACTORY PLATFORM
Profiled Application Agent Factory Run-Time Environment AgentSpotter Service
Custom Metrics Metrics Service Profiler
Agent Management Service

CPU Monitor
Agents Scheduler
Heap Monitor
Message Transport Service
data
AGENTSPOTTER STATION (visualisation application) QUERY ENGINE
Session Summary Tables queries Session summary query
Space-Time Diagram data Events
Messages sent/received query Sessions
Call graph query

Call Graph Tree View Messages
System activity query
Agents
Figure 5.2: AgentSpotter for Agent Factory concrete architecture: note that
only the upper part of the diagram is different from Figure 5.1.
5.2. Platform-independent AgentSpotter architecture 41
• Self-contained software packages are enclosed in solid lines e.g Profiler
• Logical software modules (groups of packages) are titled using slanted

capitalised names e.g. AgentSpotter Service
• Arrows denote data or processing interactions e.g. queries -
At the highest level, the AgentSpotter Service should communicate with

the Run-Time Environment to capture the profiling data from a Profiled
Application running inside an Agent Platform. The captured data should
be stored into a Snapshot File which would then be processed by a Query
Engine to generate the input data for AgentSpotter Station, the visualisation
application.
5.2.2 Profiled Application
The Profiled Application should not be directly aware of AgentSpotter

Service’s presence at run-time, just as it should not be aware of a debugger’s
presence for example. Therefore, no direct line of communication should
exist between the AgentSpotter Service and the Profiled Application. This
decoupling ensures that profiling is a concern completely separate from the
normal MAS development process.
In some rare cases it might be necessary for the Profiled Application to
communicate with the AgentSpotter Service. This should be done through
a special infrastructure like the Metrics Service described in §5.3.2.
5.2.3 AgentSpotter Profiler Service
The AgentSpotter Profiler service should monitor performance events gener-

ated by the Agent Platform’s Run-Time Environment. These events should
reflect the different things that happen inside the platform: agent manage-
ment events, agent scheduler activity, messaging, and other platform services
activity (e.g. agent specific input/output, reasoning engine). As explained

above, the Profiler has no direct line of communication with the Profiled
Application.
Additionally, through modules called System Monitors, the AgentSpotter
Service could optionally record some system performance counters like the
CPU load, the memory usage, or the network throughput, so as to provide
a general context for the Agent Platform performance events.
5.2.4 Sessions and Snapshot File
We call the result of one uninterrupted data capture operation applied

to a Profiled Application a session. A Snapshot File should be able to
record run-time data for multiple sessions. Co-locating sessions this way
will facilitate ulterior manipulation.
Ideally a Snapshot File should be stored as a single file. As a result, a
Snapshot File could be more easily moved around the file system, shared
with other developers, and transported across operating systems.
5.2.5 Query Engine
Basically, snapshot files contain raw instrumentation data. For instance,

every time an agent sends or receives a message, it should be recorded as
a unique “message sent” or “message received” event with a time stamp
and a duration. Similarly, whenever an agent executes a task, it should
be recorded as a single “agent activity” event with a time stamp and a
duration.
Large MAS applications could potentially generate hundreds of per-
formance events per second, which would mean hundreds of thousands of
records over a 10 minute session for example. However, a simple listing of
these events would not give much information about the system. Therefore, a
5.3. Platform-specific AgentSpotter architecture for Agent Factory 43
special data manipulation infrastructure, i.e. a Query Engine, is required to

allow the extraction of meaningful information from the raw instrumentation
data.
The ideal Query Engine should be able to query hundreds of thousands
of performance event records in a few seconds. It should also provide a data
manipulation language similar to SQL (the Structured Query Language
for relational databases) to facilitate the specification of rich and complex
queries.
5.2.6 AgentSpotter Station
The AgentSpotter Station is the profiler front-end application and should

implement the following set of visualisations based on queries provided by
the Query Engine: the Session Summary Tables, the Call Graph Tree View,
and the Space-Time Diagram. We describe AgentSpotter Station in more
detail in §5.4 and onwards.
5.3 Platform-specific AgentSpotter architecture for Agent

Factory
We have applied the AgentSpotter abstract architecture requirements to

Agent Factory in order to produce the concrete, i.e. platform-specific,
architecture illustrated in Figure 5.2.
From an architectural point of view, only the agent platform details
change, as pictured in the upper part of Figure 5.2. In contrast, the
AgentSpotter file processing and visualisation pipeline, as pictured in the
lower part of Figure 5.2, is exactly the same as in Figure 5.1. Essentially,
we are arguing that only the platform-specific data capture apparatus of
AgentSpotter should vary, while the snapshot file processing and visualisation
modules should stay the same, regardless of the Agent Platform on which
the Profiled Application is running.
The immediate benefit of this architecture is that it will provide a testbed
for comparing the performance of applications running on different platforms,
say JADE and Agent Factory.
5.3.1 Instrumented AF-RTE subsystems
The AgentSpotter instance for Agent Factory requires that the following
Agent Factory Run-Time Environment (AF-RTE) subsystems generate
profiling events: the Agent Management Service, the Scheduler, and the
Message Transport Service. The events generated by these subsystems
should provide enough information to support the standard AgentSpotter
visualisations described in §5.4.
5.3.2 AF-RTE Metrics Service
The standard, “out of the box”, performance data should cover most of
developers’ profiling needs. However, in some special cases, developers
might need to record specific metrics that are not already handled by the
system. Thus a new service, the Metrics Service, should be introduced in
the Agent Factory Run-Time Environment, so that the Profiled Application
can generate custom events that will be observed by the Metrics Service.
The Metrics Service should then forward the events to the AgentSpotter
Profiler in order to get them saved to the Snapshot File.
This seemingly roundabout architecture should enable developers to keep
the instrumentation code in a deployed application with little impact on
the system performance and no direct classpath link with the AgentSpotter
system.
5.4. AgentSpotter Station 45
5.4 AgentSpotter Station
AgentSpotter Station is the profiler front-end application that develop-

ers should use to analyse the result of a MAS profiling session. In line
with the modern profiler tools described in Chapter 3 (see §3.6), it should
be a user-friendly GUI application providing both textual and graphical
interpretations, i.e. visualisations, of the profiling data.
5.4.1 Visualisations
Developers expect profilers to generate flat profiles and hierarchical profiles

(also known as “call graphs”), so AgentSpotter should offer the agent-oriented
versions of these visualisations.
The AgentSpotter version of a flat profile is described in §5.5.
The AgentSpotter version of the hierarchical profile is the Call Graph
Tree View, exposed in §5.6.
Despite their relevance, these basic visualisations fail to fully convey the
distributed complexity of a MAS application; consequently AgentSpotter
should also implement a more advanced visualisation called the Space-Time
Diagram, presented in §5.7.
5.4.2 Side-by-side display of visualisations
AgentSpotter Station should allow developers to open visualisations of

different sessions from different snapshot files at the same time for direct
comparison.
For example, a developer could first profile an application to create a
baseline session. After studying the session, the developer could make a
few changes to the application code or run-time parameters and profile
the application again to generate a second session. The developer would
then open the same visualisation for both sessions, and lay them out in two
windows side by side on the screen in order to visually compare them, and
immediately see the effects of the changes on the run-time behaviour of the
application.
Along the same lines, developers should be able to open different visuali-
sations of the same session at the same time, i.e. view the Call Graph Tree
View and the Space-Time side by side to see the correlations between them.
5.5 Session Summary Tables
5.5.1 Flat Profile principles
The flat profile is the minimal service that a profiler should provide. For
each profiled application’s identifiable basic execution block, it should clearly
list the following information items:
• the number of times the block was invoked;
• the total amount of time the application spent inside the block;
• the percentage of the total recorded time represented by that amount.
Just by looking at that apparently simple listing ordered by total time

spent, developers should be able to spot bottlenecks immediately i.e. execu-
tion blocks that use up more computing time than they should in absolute
time or compared to other execution units.
5.5.2 Selection of the appropriate execution unit
But what exactly is the appropriate execution unit for a MAS? To answer
that question, we have pitted the OOP execution unit concepts against
their AOP counterparts as shown in table 5.1. Note that we have assumed
agents are implemented as single execution threads, which is the approach
5.5. Session Summary Tables 47
Table 5.1: OOP and AOP execution units mapping

OOP Execution Unit AOP Execution Unit
Process Platform
Thread Agent
Object N/A
Method N/A
adopted by Java-based Agent Factory and JADE [11, 2]. The table shows
that agents are the lowest-level meaningful execution unit in a MAS.
One could argue that in practice agents are based on objects and carry
out their tasks by invoking methods, but from a conceptual point of view it
is irrelevant.
5.5.3 Vital indicators
In Chapter 4, we have described the vital indicators of a MAS performance

(see §4.2.1 and §4.2.2). AgentSpotter Station should obviously compute and
display these indicators.
5.5.4 Agent statistics
Concretely, AgentSpotter Station should present a table that lists for each
agent the following indicators:
Agent description: name, role, type (deliberative or reactive) of the agent.
Cumulative activity: cumulative computation time used by the agent.
Perception time: perception time used by a deliberative agent.
Action time: task execution time used by a deliberative agent.
Reasoning time: reasoning time used by a deliberative agent.
% session activity: percentage of the global session computation time

used by the agent.
Number of iterations: number of non-zero duration iterations (i.e. sche-

duling quantums or time slices) used by the agent;
Number of time slice overshoots: number of times where an agent has

overused its time slice allocation;
Maximum and average iteration duration: maximum and average du-

ration of time slices;
Total number of sent/received messages: total number of messages

exchanged by the agent.
5.5.5 Session statistics
Moreover, AgentSpotter Station should provide a session-level statistics

table with the following items:
Total duration: session run-time recorded.
Total activity: amount of computation time recorded over the session.
Total number of messages: number of messages sent or received by the

platform — the numbers could possibly not match in case of inter-
platform communication.
Average number of active agents per second: this number gives an

idea of the level of concurrency of application.
5.5.6 Conclusion
In this section, we have described the requirements for AgentSpotter’s

session-level and agent-level summary tables composing what is called a flat
profile. These statistics should give MAS developers a sense of the order of
magnitude in the profiled application’s run-time parameters. Additionally,
5.6. Call Graph Tree View 49
Figure 5.3: Call Graph Tree View of a fictional C program which removes
duplicate lines from a file
developers should be able to spot immediately the most active agents in

terms of computation or messages exchanged.
Although the flat profile is very useful for a preliminary analysis, it must
be used in conjunction with more detailed visualisations like the Call Graph
Tree View or the Space-Time Diagram described in the following sections.
5.6 Call Graph Tree View
5.6.1 Overcoming the limitations of Summary Tables
Summary tables give developers a sense of how much time agents have spent
doing things, but they do not give much detail about the chain of events
that has led the agents to do these things.
Suppose for example that an agent is found to use more that half of
a MAS’s computation time while it was expected to use only a quarter.
Looking at the agent’s internal resource use could give a good idea of the
cause of the problem e.g. “I think the agent X took too much time doing a
bubble sort on a list of 10000 integers”.
However, we have shown in Chapter 4 that analysing the behaviour of
an agent in isolation is not sufficient to understand fully the cause of a MAS
performance problem. Indeed, the real complexity of a MAS comes from
the distributed interactions between the agents.
Consequently, AgentSpotter should implement a visualisation that allows
developers to understand the impact of interaction on the performance of a
MAS. In traditional profilers, this kind of visualisation is called a hierarchical
profiler or call graph (see Chapter 3, §3.5.1).
5.6.2 Traditional call graph
A call graph is basically a tree displaying all the calls made by a program.
These calls are grouped by call target, and are annotated with the cumulative
time spent in each call target.
The example in Figure 5.3 shows a typical call graph of a completely
fictional C language program which is supposed to remove duplicate lines
from a file. The root node is the main function and it represents 100% of
total execution time. The idea is that each tree’s level displays recursively all
the calls made by the current node as child nodes along with the percentage
of time they represent and so on until the top of the call stack is reached.
Therefore, if main is calling process file, it is represented as a sub-tree with
leaves describing its own calls to bubble sort, load file and remove duplicates.
The call graph tree view’s key benefit is the extended context it gives to
the performance information. For instance, our simple example reveals that
Agent A
message Mα at time α
impact of MΩ
{
Agent B α β Ω
{
impact of Mα
message MΩ at time Ω
Agent X
Legend:
Message event
Agent activity
Agent life line

time
Figure 5.4: Agent message impact concept diagram
the program spends 90% of its time processing a file. The tree shows that
one of the top-level function process file’s callees, the bubble sort operation,
eats up 80% of its caller time. A flat profile would have shown the time
for these functions separately without explicitly showing the hierarchical
link between them. In other words, what matters is to know the “where” to
understand why the “how long” is a problem.
5.6.3 Agent-oriented call graph model
At this point, the problem of mapping a traditional programming concept

to an agent concept arises again. Specifically, what is the equivalent of a
function call for an agent? This is a much more difficult question than it
seems, and would necessitate a long discussion that we have decided not to
develop in this dissertation for space reasons. Instead, we invite the reader
to allow us to make a conceptual leap without a proper demonstration.

In short, we assert that we can measure TMα ,B , the impact of a message
Mα sent from an agent A to an agent B and received at time stamp α,
by measuring the total amount of computation time used by the agent B
until agent B receives a message MΩ from another agent X at time stamp
α ≤ Ω. Let b be the duration of an activity by agent B at time stamp t
where α ≤ t ≤ Ω. The impact of message Mα on agent B, TMα ,B , is then
given by the recurrent equation:
Ω
X
TMα ,B = bt (5.1)
t=α
In Figure 5.4 we have tried to summarise this concept in a graphical form.

The diagram clearly shows that the three activity stars that lie between α
and Ω make up the total impact of Mα on agent B. Note that the outgoing
message at time stamp β does not break the computation sequence.
It is now easy to determine the total impact Tx,y of all the messages
sent by a given agent x to another agent y. Let M be the total number
of messages sent, 1 ≤ m ≤ M a single message impact identifier, αm the
reception time stamp of message m from x to y, and Ωm , where αm ≤ Ωm ,
the next reception time stamp message coming right after m from any other
source. The total impact Tx,y is then given by the equation:
Ωm
M X
X
Tx,y = bt (5.2)
m=1 t=αm
By applying the equations recursively, we can compute the total impact

Tx of an agent x on N other agents numbered 1 ≤ a ≤ N as follows:
Ma X
N X
X Ωm
Tx = bt (5.3)
a=1 m=1 t=αm
Figure 5.5: AgentSpotter Call Graph Tree View levels
Finally, the total impact TS of all the K agents numbered 1 ≤ k ≤ K of

a session S is given by the equation:
Nk X
K X
X Ma X
Ωm
TS = bt (5.4)
k=1 a=1 m=1 t=αm
It must be noted that the total activity time AS of the session S is given
by the equation:
k0 −1
K αX
X
AS = TS + bt (5.5)
k=1 t=αS
where αS is the first recorded time stamp in session S and αk0 the time
stamp of the very first message received by agent k. To put it differently,
the total impact for each agent can be computed only after it has received
its first message.
A traditional call graph’s depth can be as deep as the deepest function
call stack. In contrast, the graph representation of our model is a fixed-depth
tree.
5.6.4 Call graph model application
The model we have just defined is the basis for the graphical representation
that AgentSpotter Station should provide to help developers understand
better the impact of agents interactions on a MAS performance.
This conceptual model deals with the session level, the emitter agent level,
the receiver agent level and the message level. The graphical translation
of the model, outlined in Figure 5.5, should be a tree view representing
the levels we have previously enumerated plus an additional level for the
FIPA ACL message content. A message content is defined as a performative
plus an expression e.g. “request:doSomeThing(123)”. This additional level
should give developers necessary contextual information for the messages.
Of course the model must be slightly extended to accommodate for this new
level, but this is left as an exercise for the reader.
The session at the root of the tree should add up to 100% of all emitter
agents’ impact as defined by equation 5.4. Then at each level, each node
should recursively total the impact of its child nodes down to the message
leaf nodes. These leaf nodes simply report their impact as defined by
equation 5.1. More precisely, at each level, for each node, the following
values should be displayed:
Label: informative text associated with the node. The structure of the
label depends on the level as follows:
• session: “capture date and time - duration”;
• emitter agent: “from: agent id”;
• sender agent: “to: agent id”;
• FIPA ACL: “performative: contents”;
• message: “sent: time stamp rec: time stamp”.
Total impact time: sum of impact times of all the current node’s children.
% parent time: percentage of the current node total impact time over the
node’s parent total impact time.
% session time: percentage of the current node total impact time over
the session total impact time.
Ideally, developers should be able to order the intermediary tree levels

differently so as to produce different call graph interpretations. For example,
moving the FIPA level right above the emitter agent level would list for
each FIPA ACL entry their total impact for all the emitter/receiver pairs.
5.6.5 User interface requirements
Even if it has a fixed depth, a call graph tree view could potentially be
very wide at the leaf level for sessions that produce thousands of messages.
Therefore, to help developers navigate easily through the tree, AgentSpotter
Station should offer an advanced tree navigation user interface that would
develop only the part of the tree that is currently explored to reduce the
visual clutter. The currently explored part of the tree should be highlighted
in a different colour to give the developer a visual feedback.
Moreover, to speed up information retrieval, a special search feature
should allow developers to enter a keyword (an agent name or a performative
for example). Doing so would have the effect of highlighting in a special colour
all the visible nodes which contain that keyword, improving significantly the
visual retrieval speed of a node.
Ideally, to enable developers to locate items even more quickly, zooming
and panning around the tree view should be possible.
5.6.6 Conclusion
To summarise, in this section we have proposed a new visualisation, the

call graph tree view, in order to overcome the main limitation of the Agent-
Spotter flat profile, namely the lack of detailed information about the agents
interactions.
Figure 5.6: Lamport Space-Time Diagram extracted from [28] ( 1978

c
ACM).
After exposing the concept of call graph in a traditional programming

context, we have then mapped it into an agent-oriented concept based on
the idea that when an agent A sends a message to an agent B, its impact
TMα ,B can be measured by the equation 5.1.
We have then extended this notion to a tree model with multiple levels:
session, message emitter, message receiver, message.
Finally, we have described the advanced user interface that would allow
developers to interact with this model in the form of a zoomable and
searchable tree view.
5.7 Space-Time Diagram concept
5.7.1 Motivation
The Call Graph Tree View is an information-rich visualisation that should

make AgentSpotter easily comparable to existing visual Java profilers. Fur-
thermore, it is fully agent-oriented and should provide real insights into the
5.7. Space-Time Diagram concept 57
performance behaviour of a MAS.

Nonetheless, we believe the call graph does not fully capture the richness
of the concurrency and the distribution of performance in MAS systems.
To put it bluntly, the call graph reveals the link between the “where” (the
agents), the “what” (the messages) and the “how long” (the performance
timings), but fails to convey a sense of the “when” in an obvious way.
Consequently, we need to add to AgentSpotter a comprehensive graphical
representation that conveniently conjugates space and time, if we want to
picture out how phenomenons like message passing and performance timings
– the “what”, evolve across time – the “when”, among a community of
distributed agents – the “where”.
5.7.2 Lamport Diagram
Distributed systems specialists commonly use a graphical representation

called a “space-time diagram” or “Lamport Diagram”. It was made popular
by L. Lamport in the seminal paper “Times, Clocks, and the Ordering of
Events in Distributed Systems” [28]. In his paper, L. Lamport suggests the
space-time diagram as a useful view for the “happened before” relation that
he defines as follows:
Definition. The relation “→” on the set of events of a system

is the smallest relation satisfying the following three conditions:
(1) If a and b are events in the same process, and a comes before
b, then a → b. (2) If a is the sending of a message by one process
and b is the receipt of the same message by another process, then
a → b. (3) If a → b and b → c then a → c. Two distinct events
a and b are said to be concurrent if a → b and b 6→ a. We assume
that a → a for any event a. (Systems in which an event can
happen before itself do not seem to be physically meaningful.)

This implies that → is an irreflexive partial ordering on the set
of all events in the system.
The author then extends the partial ordering defined by the “→” relation
to a total ordering which he uses to describe a distributed algorithm to
synchronise a system of clocks.
To simplify a little bit, synchronising clocks this way allows the definition
of a “global” reference clock which ticks at the same time for all the processes
of the system. Eventually, such a global clock makes the representation of
distributed processes in a common space-time diagram possible, since a given
clock tick means the same time for all the processes. A typical Lamport
diagram extracted from [28] is pictured in Figure 5.6. Vertical lines represent
process time lines, while the dashed horizontal lines mark the ticks of the
global clock. The squiggly arrows denote message passing events. Thanks
to the synchronised global clock ticks’ markings, the causality relations
between the different distributed concurrent events appear very clearly.
5.7.3 Principles
The Lamport Diagram should constitute the core for the comprehensive
MAS performance visualisation we need. In contrast with the flat profile
and the call graph, our goal is not to reduce the data to a few meaningful
dimensions.
On the contrary, our plan is to display as much detail and context as
possible about a MAS performance, and let developers pan the view around,
zoom in and out in order to reveal hidden patterns or focus on minute
details.
As an analogy, it is like having a zoomable satellite view of the city of
5.8. Space-Time Diagram user interface 59
AGENT FACTORY PLATFORM
Profiled Application Agent Factory Run-Time Environment AgentSpotter Service
Custom Metrics Metrics Service Profiler
Agent Management Service

CPU Monitor
Agents Scheduler
Heap Monitor
Session Time Line
CPU Line
Agent Life Line ♣♦♠

intra-platform message lines simple events (no duration)
Agent Life Line
timed events (with duration) inter-platform messages lines

External Platform Life Line
current viewport
information window on mouse hover
“Bird’s Eye View”

Session Time Line
CPU Line
Agent Life Line ♣♦♠
Agent Life Line
External Platform Life Line
Figure 5.7: AgentSpotter Space-Time Diagram specification annotated with

AgentSpotter for Agent Factory infrastructure links (see 5.2).
Dublin with a map view overlay showing the street names, the points of
interest and some geolocated statistics (e.g. number of pubs per square
kilometre): the mix of raw visual data and value added geolocated data
generates much more information.
5.8 Space-Time Diagram user interface
Basically, the visualisation should appear as a Lamport diagram where

the time flows horizontally towards the right instead of flowing vertically
towards the top. At the risk of stating the obvious, the distributed processes
pictured in the diagram should be agents. Moreover, developers should be
able to interact with the diagram using the mouse.
It can be awkward to try to describe a complex visual construction using
words only, so we invite the reader to use Figure 5.7 as a visual map to our
AgentSpotter Space-Time Diagram specification. In the following sections,
we describe the required diagram elements, starting from the top of the
picture.
5.8.1 Session Time Line
Represents the cadence of the global system clock.
• A pixel point on the time line should map to a millisecond precision

time stamp. The pixel coordinate of a given time stamp is obtained
by applying a certain factor to avoid stretching the display too much
(1 pixel for 25 milliseconds seems to produce a good display).
• The session time line should always be visible whatever the position and
the scale of current view port, so as to provide an essential temporal
context.
• Helpful labelled tick marks should be drawn so that developers can

quickly make an approximation of the current time stamp at a given
position on the time line. Long ticks should mark the seconds, and the
current time should be displayed every 5 seconds. Shorter tick marks
should be displayed every 250 ms.
• On mouse hover, a pop up information window should indicate the

exact time stamp at the point under the mouse down to the millisecond.
5.8.2 CPU Line
Optional graphical plot of the global CPU load of the host system during a
session.
• A vertical gradient going from green (low CPU usage) to red (high
CPU usage) should give a sense of the system load.
• On mouse hover, a pop up information window should indicate the

exact CPU load percentage at the point under the mouse.
5.8.3 Agent Life Lines
Represent all the performance events and message passing activity that
occurred for a given agent during a session.
• The life line should start only at the time stamp where it was created
or resumed in order to make fluctuations in a MAS population more
apparent.
• The life line area grey scale colour percentage should be proportional
to the agent’s contribution to the global system activity: the more
an agent uses computation time, the darker its life line area’s colour
should be.
• By default, agent life lines should appear from top to bottom in

descending order of total computation time. Ideally, developers should
be able to change this default order by moving the life lines up or
down in order to visually bring together life lines with particularly
interesting interactions.
1 time slice
Time
{
0%
% time slice
50%
75%
{
100%
{
proportional dimensions
150% of time slice (exceeded allocation: RED)
80% of time slice (over 75%: ORANGE)
50% of time slice (under 75%: GREEN)
Figure 5.8: Agent Factory agent activity representation in the Space-Time

Diagram
• The life line caption (the agent name) should always be visible whenever
a part of the life line is visible so as to provide an essential contextual
information.
• When the mouse pointer hovers on the life line caption, a pop up
information window should display the agent statistics appearing in
the flat profile (see §5.5.4).
• Developers should also be able to hide temporarily certain life lines in

order to reduce the visual clutter.
5.8.4 Performance Events
All the performance events related to an agent should be displayed along its
life line as they happen.
• Events that have a time stamp only, i.e. simple events, should appear
as little glyphs centred at the location corresponding to their time

stamp of occurrence. For example, “message received” events could
be displayed with a little envelope glyph.
• Events that have a duration, i.e. timed events, should be drawn with a
size proportional to their duration and should provide as much visual
information as possible.
Consider an Agent Factory agent activity event: it has a measured

actual execution time and an allocated “time slice” duration that the
agent is expected not to exceed. If an agent exceeds its time slice
allocation, it may prevent other agents from getting some CPU time.
Thus, it would be interesting to represent visually the fact that an
agent activity has exceeded its time slice allocation in a way that
would underline its punishing effect on performance. Our idea is to
make AgentSpotter use colour coded rectangles to represent agent
activity.
As illustrated in Figure 5.8, the rectangle size should be proportional

to the event duration and its allocated time slice use percentage.
The colour code should indicate how the agent has used the time
slice. Green would mean that it has used only 3/4 (75%) of its time
allocation. From 75% up to 100%, orange would signify that the agent
has nearly used up most of its allocated time. When the agent exceeds
its time allocation, the rectangle should be painted red. Figure 5.8
demonstrates how different time slice usages should be represented:
50% (green), 80% (orange), and 150% (red).
• On mouse hover, a pop up information window should display specific

details about the event.
• Other representations should be invented for other performance events,

which could be the subject of an ulterior study. Imagine for example
an extended activity event display for an Agent Factory BDI agent
that would show the proportion of reasoning, perception and action
for each event.
5.8.5 Message Lines
All communication between agents should be rendered as message lines that

link the emitter with the receiver.
• A little arrow end should be added to the message line to show clearly
the direction of the message.
• Messages that stay inside a platform (intra-platform) should be distin-

guished from messages that are going to or coming from outside the
platform (inter-platform).
• On mouse hover, a pop up information window should display all the

FIPA message headers and its content.
5.8.6 External Platform Life Line
When agents communicate with an external platform, a placeholder life line

for the platform should be drawn.
• The AgentSpotter infrastructure is mono-platform only, which means

that it cannot profile two different platforms at the same time. Yet
it should be possible to profile multiple communicating platforms at
the same time by running profiling sessions separately. The resulting
data could then be merged a posteriori through the Query Engine.
The main task of the merge operation would be to synchronise the
timestamps in the different sessions using a Lamport clock as explained

earlier (see §5.7.2). For space reasons we won’t examine this process
in detail, although it is well worth further studying.
• As the platform life line is a placeholder only, messages lines that

are linking to it are assumed to have the same sending and receiving
timestamps. As a result, they should always render as straight lines.
5.8.7 Tooltip Window
Most diagram elements should pop up a yellow information window (also

called Tooltip Window) to display relevant data about the currently pointed
location. This seemingly minor feature is in fact the most efficient way to
provide developers with the right information in the right context.
5.8.8 Bird’s Eye View
Agent space-time diagrams for long running sessions could potentially span
tens of thousands of pixels. For example it would require (10∗60∗1000)/25 =
24000 pixels across to display a 10-minute long session, which is equivalent
to more than 24 screenfuls of a 1024 pixels wide monitor.
Thus, to facilitate the navigation around such a large view, a zooming
and panning facility is an imperative requirement.
Moreover, AgentSpotter should implement a reduced global view in low
resolution of the full diagram, called a “bird’s eye view”, where developers
can quickly select areas to explore by moving around a little rectangle
representing the current view port. The view port is depicted as a transparent
blue area on Figure 5.7: its current position is perfectly mirrored by the
little blue rectangle in the bird’s eye view window.
5.8.9 Links with the AgentSpotter for Agent Factory

infrastructure
At the risk of overloading the reader with too much information, we have
traced on Figure 5.7 the links between the AgentSpotter data capture
infrastructure for Agent Factory and the space-time diagram. The dashed
arrows show how the data coming from each Agent Factory Run-time
Environment sub-system, as well as the data coming from system monitors
(CPU usage), is represented on the diagram. Our point is to demonstrate
the incredible richness of our visualisation.
Experimented human computer interface (HCI) designers will surely
object that such density could lead to a confusing picture. We will address
this objection later in our evaluation of the system (see Chapter 7).
5.8.10 Conclusion
In this section, we have developed the requirements for AgentSpotter’s pièce

de résistance: the Space-Time Diagram. It is inspired from the Lamport
Diagram which we have extended to agents, and made interactive with the
aim of providing a rich contextual visualisation for a MAS profiling session.
The diagram’s key value is the sheer amount of information it manages to
convey. On a single diagram, developers should be able to visually connect
together the performance and interaction history of a MAS, both in space
and in time. They should also be able to instantly zoom from a global
overview of the system down to a millisecond detail.
We strongly believe that the diagram’s rich information presentation
and advanced interactivity features will be essential tools to inform MAS
designers better about performance and the behaviour of their system.
5.9. Summary 67
5.9 Summary
In this chapter we have extensively described the requirements for Agent-

Spotter, our comprehensive MAS profiler, in a way that can be used as a
blueprint to develop a software prototype.
First, we have managed to define an abstract architecture that should
allow AgentSpotter’s development for another Java-based MAS framework
like JADE.
Second, we have introduced a low-impact, decoupled profiling infrastruc-
ture specifically tailored for Agent Factory. Our infrastructure should make
it possible for developers to instrument without difficulty their systems and
store data capture sessions into readily manipulable snapshot files.
More significantly, we have explained the concepts underlying the Agent-
Spotter Station application, and detailed the visualisation tool suite it should
offer.
Initially, we have presented the most basic visualisation tool, called Flat
Profile or Summary Tables. It should be used to get a preliminary report
on agents vital performance indicators.
Then, for more detailed performance analysis, we have imagined more
advanced tools: the Call Graph Tree View and Space-Time Diagram.
The Call Graph Tree View is a multi-level tree representation of the
impact of agents interactions on other agents performance. Its main use is
to point out interesting interaction-related performance issues.
Finally, the Space-Time Diagram, based on the well-known Lamport
Diagram, is the richest visualisation of the suite. In essence, it displays a
MAS’s full history of performance and message passing events over a given
profiling session. The diagram fully exploits the human mind’s ability to
process efficiently dense information when it is presented in a spatio-temporal
context. It should allow MAS developers to obtain unexpected insights into

the performance behaviour of their systems.
In short, we have specified a novel system that will truly help MAS
developers understand better the performance of their programs.
In the next chapter we will show how we have used our requirement
analysis as a blueprint for developing a prototype of the AgentSpotter
profiling system for the Agent Factory platform.
CHAPTER 6
AgentSpotter design and
implementation
6.1 Introduction
Following the requirements’ analysis elaborated in the previous chapter,

we have implemented a fully working beta version of AgentSpotter for the
Agent Factory platform. Our goal in this chapter is to expose the high
level design and implementation details of AgentSpotter, then present and
comment on screen shots of the final application. In fact, we have managed
to implement nearly all the features we envisioned in Chapter 5, so we will
broadly follow the same structure to describe the resulting system.
The first part of this chapter is devoted to the profiler infrastructure. We
explain how we have extended the Agent Factory Run-Time Environment
to support our profiler service. This is followed by a discussion about
the profiler service, and the implementation of the profiling data capture
pipeline.
The second part focuses on AgentSpotter Station, the profiler visualisa-
69
70 AgentSpotter design and implementation
tion tool suite, and consists mostly in commenting on screen shots of the
application.
6.2 AgentSpotter implementation architecture: an overview
Building upon the concrete architecture we defined in Chapter 5 (see §5.3), we

have implemented the software system illustrated in Figure 6.1. Naturally,
we have based this implementation diagram on Figure 5.2. Besides the
graphical conventions explained in §5.2.1, this new diagram adopts the
following additional ones:
• Software modules are decorated with their corresponding Java package

com.agentfactory
or class names, e.g. Agent Factory Run-Time Environment
• Open source off-the-shelf components that we have used are identified

by their logo e.g.
• Software modules that we have developed from scratch are titled in

red e.g. AgentSpotter Service
To implement the core AgentSpotter profiling infrastructure, we have

slightly extended the existing Agent Factory Run-time Environment to
generate performance-related events at some strategic points in the Agent
Management Service, the Scheduler, and the Message Transport Service.
These extensions use classes introduced by the new Metrics Service. The
run-time subsystems communicate profiling events to the Profiler.
The AgentSpotter Service relies on the AgentSpotter File Manager to
store the profiling data in a Snapshot File. AgentSpotter snapshot files
are actually single-file databases managed by the public domain SQLite
Database Engine [22]. As a result, profiling data is stored as queryable
relational database tables.
6.2. AgentSpotter implementation architecture: an overview 71
AGENT FACTORY PLATFORM com.agentfactory

com.agentfactory.agentspotter
Agent Factory Run-Time Environment
Profiled Application AgentSpotter Service
com.agentfactory.service.metrics
Configuration file APP.CFG com.agentfactory.agentspotter.AgentSpotter
TraceEvent Class TraceEvent’s
... Profiler
PLATFORM_GUI com.agentfactory.
agentspotter.AgentSpotter Agent Management Service
... com.agentfactory.agentspotter.cpuinfo
Scheduler CPU Monitor

Agents
com.agentfactory.agentspotter.module com.agentfactory.agentspotter.file data
AGENTSPOTTER STATION RCP APPLICATION AgentSpotter File Manager

com.agentfactory.agentspotter.visualisation org.sqlite
QUERY ENGINE
AgentSpotter Visualisation Components queries
Session summary query SQLite Database Engine
Session Summary Tables
Session Summary Tables
Space-Time Diagram Messages sent/received query

data
Space-Time Diagram
Call graph query

Call Graph Tree View
Call Graph Tree View
System activity query DDL

org.prefuse
data
SQL Templates SQL
org.piccolo2d org.antlr.stringtemplate
Piccolo2D
Sessions Events
ANTLR StringTemplate
org.openide
Messages
Netbeans Platforms APIs Agents
Actually a SQLite database file
Figure 6.1: AgentSpotter implementation architecture
Snapshot files are processed by AgentSpotter Station visualisation tool

suite, which is implemented as a “rich client” application based on the
Netbeans Platform APIs [33]. In an IDE-like environment, AgentSpotter
Station encapsulates the AgentSpotter Visualisation Components. These
components use the AgentSpotter File Manager to extract the profiling in-
formation from the snapshot files, and exploit the advanced 2D visualisation
features of the Piccolo2D [1] and the prefuse [18] graphical toolkits.
6.3 AgentSpotter profiling infrastructure
6.3.1 AF-RTE performance data channels
Our first implementation task for the AgentSpotter profiler was to ex-
tend the Agent Factory Run-Time Environment (AF-RTE). Thanks to the
Agent Factory framework’s modular architecture, it was easy to identify
the strategic points in the code where the instrumentation code had to be
injected (the “point-cuts” in aspect-oriented programming terminology).
Concretely, we have introduced a new com.agentfactory.agentspotter.
service.metrics package which simply contains a new data TraceEvent
class. Thus, the AF-RTE sub-systems were made to generate the profiling
events by creating TraceEvent instances.
As was done in Cougaar [19], we have applied the Publisher-Subscriber
pattern in order to implement the data communication channels between
the AF-RTE and the Profiler service. In effect, the AF-RTE sub-systems
can publish performance data regardless of the actual presence of a profiler.
Then, when it is activated, the Profiler service can subscribe to the different
well-known performance data channels in order to collect trace events and
persist them to an AgentSpotter snapshot file.
The protocol we have used is very straightforward and is based on simple
hierarchical slash-separated (“/”) string event identifiers. For example, an
agent scheduler activity event is published as a TraceEvent instance with a
Type property value of “scheduler/activity” where the first path element,
“scheduler”, denotes the origin of the event, and the rest, “activity”, indicates
the nature of the event. Table 6.1 presents the full list of the events currently
published by the AF-RTE. We expect this list to grow as more data channels
will be added to the system.
Note that for lack of time, we did not implement the Metrics Service
6.3. AgentSpotter profiling infrastructure 73
Table 6.1: TraceEvent types generated by the AF-RTE sub-systems

AF-RTE Sub-System Event Type
Agent Scheduler scheduler/activity
Message Transport Service message/send
message/receive
Agent Management Service agentContainer/resume
agentContainer/suspend
agentContainer/terminate
Metrics Service custom/simpleEvent
(to be implemented) custom/timedEvent
which is supposed to handle custom profiling data.
6.3.2 FIPA ACL message identification
One interesting problem we had to solve is how to link up a send event

of an ACL message with the corresponding receive event. The obvious
solution is to identify uniquely a message and use the same identifier for
the send and receive events. Contrary to our expectations, however, the
FIPA ACL Message Structure Specification [15] says nothing about a unique
message identification field. As a stopgap solution, we have decided to
use the conversation identifier field to store the required unique message
identifier.
The drawback of this solution is that some applications might need to
use the conversation identifier field for their own purpose. Fortunately, from
a technical point of view, this new problem can be solved easily by having
the Message Transport Service (MTS) use a special wrapper mechanism
which would allow the addition of system information to a message for
instrumentation purposes 1 . But, from a conceptual point of view, it is not
1
Very simply, at emission time, the MTS could prepend the conversation identifier
field with a message unique ID followed by a special separator. At reception time, the
MTS would remove the message ID prepended to the field, and hand over the original
message to the receiving agent. This solution should be readily adaptable to different
agent platforms.
really a clean solution. Ideally, the FIPA ACL message standard should
include a “message identifier” field reserved for system use.
6.3.3 AgentSpotter profiling service
The Profiler service contains two parts: the profiler class, AgentSpotter,
which is implemented as an Agent Factory platform service 2 , and an
operating system specific CPU usage monitor.
Profiler
As explained in the previous section, the AgentSpotter class is a perfor-

mance data channel subscriber. Being an Agent Factory platform service,
it can be directly plugged into an existing Agent Factory application by
following these simple steps:
1. Add the required AgentSpotter JAR files to the application classpath.
2. Add a single line to the platform configuration file:

PLATFORM GUI com.agentfactory.agentspotter.AgentSpotter
3. Run the application normally to generate the profiling data.
When AgentSpotter Service is started up, it opens a data connection using

the File Manager, subscribes to the AF-RTE performance data channels,
and then waits for the data publishers to prompt it about a channel update.
When prompted, it pulls down the updated data and forwards it with
no further processing to the File Manager. The loop goes on until the
2
According to FIPA standards, the purpose of a platform service is to “provide
support services for agents” [16]. Platform services are at the core of the Agent Factory
infrastructure: they are the recommended mechanism for implementing new system
functionality [9].
6.4. AgentSpotter File Manager 75
application terminates. The most important point is to keep processing time

to a minimum, so as to avoid affecting the system performance too much3 .
CPU Monitor
To demonstrate a system monitor implementation, we have developed a CPU

monitor that records the CPU usage percentage every second. Accessing
system information like the current CPU load in Java requires some operating
system specific code. Therefore, the CPU monitor is an optional feature. At
the moment, it works only for the Windows and Linux operating systems.
Nonetheless, the additional contextual information provided by the CPU
monitor is worth the effort. That is why we plan to add support for more
operating systems like Mac OS X in the future.
6.3.4 Running the Debugger and the Profiler together
Interestingly, because AgentSpotter is a mere Agent Factory platform service,

it is entirely possible to run it along with the Agent Factory Debugger tool [7].
This combination can make a lot of sense in some scenarios where developers
want to keep a trace of a debugging session.
6.4 AgentSpotter File Manager
The File Manager is the bridge between the data capture infrastructure and
the visualisation toolkit. In accordance with our requirements, we needed
a compact and reliable snapshot file format, and a queryable and flexible
data retrieval system. Given the fact that profiling data is highly structured,
and that we do not expect shared access to a snapshot file, it was obvious
that a single file database system was the best basis for our File Manager.
3
For lack of time, we could not conduct an evaluation of the actual impact of the
profiler at run-time. In practice, the instrumentation does not seem to significantly slow
down the profiled applications.
Informed by our own experience with similar systems, we have selected

the SQLite Database Engine [22]. SQLite is uncomplicated, very fast, and
implements a sizable subset of the SQL-92 standard.
Our approach is to be contrasted with a log file based persistence solution
like the one used by Cougaar [19]. Whilst a log file seems easy and quick to
produce, we expect subsequent automated information retrieval operations
to require some significant programming efforts.
It must be stressed that the File Manager does not directly expose
to its software clients its acquaintance with a particular database system.
Instead, it provides an API that deals with domain specific data record
classes representing profiler data like SessionRecord, AgentRecord, and
MessageRecord.
We won’t examine the characteristics of the File Manager in further
detail, since we have already explored the rationale of a solid file and query
management sub-system (see §5.2.4 and §5.2.5).
In conclusion, AgentSpotter records profiling data in snapshot files that
are in reality SQLite database files4 . These compact database files are easy
to query, manipulate and share with other developers or systems.
6.5 AgentSpotter Station
From the developers’ point of view, the AgentSpotter Station desktop

application is the profiler central attraction. It can be seen in the screen
shot in Figure 6.2, which demonstrates in a single take all the Station’s
features numbered as follows:
1. Snapshot Explorer : list of snapshot files under study, organised in a

hierarchical explorer interface.
4
See Appendix C for a detailed database schema.
6.6. AgentSpotter Visualisation Components 77
2. Summary Tables: flat profile listings.
3. Call Graph Tree View : hierarchical profile visualisation.
4. Space-Time Diagram: comprehensive performance behaviour visuali-

sation.
5. Bird’s Eye View : navigation mini-map window for the focused Space-
Time Diagram.
AgentSpotter Station was assembled as a standalone modular application

using the Netbeans Platform APIs, the same “rich client” framework that
powers the Netbeans IDE [33]. The main reason behind this choice was the
preexistence of the Agent Factory Netbeans IDE plugin. It seemed logical
to use the same architecture to build the profiler standalone interface with
the aim of converting it later to a set of plugins.
Netbeans Platform’s most spectacular feature is the provision of a versa-
tile window layout manager which supports the composition of sophisticated
views similar to the screen shot shown in Figure 6.2. As previously explained
in §5.4.2, a flexible window layout system not only enhances the user ex-
perience, but also expands the user’s analysis possibilities by facilitating
side-by-side comparisons.
While the flat profile view implementation was trivial and does not
warrant an extended discussion — the summary tables are simple listings,
the implementation of the graphical user interfaces was a more complex task
which we describe in the next section.
6.6 AgentSpotter Visualisation Components
To promote modularity and reuse, we have designed the AgentSpotter Call

Graph Tree View and Space-Time Diagram visualisations as self-contained
Figure 6.2: Agent Spotter Station sample screen shot showing all the available views in a single take
AgentSpotter design and implementation
78
and reusable graphical components. In addition, following good engineering

practices, we did not develop these visualisations from scratch using raw
Java2D API calls. Instead, we have utilised two leading graphical toolkits
particularly suited to the development of interactive zoomable user interfaces:
prefuse [18] and Piccolo2D [1]. The resulting Visualisation Components
library can be reused in any Java Swing application, not just AgentSpotter
Station.
6.6.1 Call Graph Tree View
Figure 6.3: Call Graph Tree View screen shot
In order to construct our Call Graph Tree View, we have adapted a special
tree display component included in prefuse toolkit. We have essentially
managed to implement all the features described in §5.6, including the
keyword search facility which highlights nodes containing a given keyword.
A sample screen shot of the resulting display can be seen in Figure 6.3.
Screen shot comment
The element numbered 1 on the screen shot is the tree root, i.e. the session
level which represents 100% of the recorded activity time. The tree root
is highlighted in blue because it is the current tree selection. As such, it
determines the branch that is expanded; as stated in §5.6.5 the idea is
to reduce the visual clutter. The tree expansion algorithm ensures that
all the children of a selected node are visible as well as its grandchildren.
Consequently, when the tree root is selected, only the first two subsequent
levels are expanded, that is the emitter agent level, and the receiver agent
level. Hence, selecting an emitter agent node should make the FIPA ACL
message level visible, and so on. As an illustration, the call graph numbered
3 shown in Figure 6.2 screen shot, has an agent receiver node selected; as a
result, this branch is fully expanded down to the message impact level.
The element numbered 2 is a text area used to enter a search keyword.
The number of nodes matching the keyword is displayed and all the matching
nodes that are visible are highlighted in pink, just like the element numbered
3, which is one of the nine nodes containing the “explorer” keyword. In a
large expanded tree, the pink nodes really do stand out.
Controls
The view can be controlled using the mouse or the keyboard as follows:
Select: Click an item to expand its tree branch.
Pan: Left-click and drag the background to pan the display view.
Zoom: Right-click and drag the mouse up or down or use the scroll wheel
to zoom the display view.
Zoom-To-Fit: Right-click once to zoom the display to fit the whole tree.
Search: Type in a search query to search over the currently visible nodes.
Layout: Ensure the view window has the focus then press one of the
following layout control keys: Control-1 (left to right tree, the default
one), Control-2 (top to bottom tree), Control-3 (right to left tree) or
Control-4 (bottom to top).
6.6.2 Space-Time Diagram
The screen shot in Figure 6.4 shows a sample Space-Time Diagram View.
In contrast with the Call Graph, this diagram was not adapted from a
pre-existing visualisation component. All the major features specified in §5.8
were included in the beta version except the external platform messaging
display (see §5.8.6). The following paragraphs describe the resulting view as
numbered in Figure 6.4. However, to avoid repeating the contents of §5.8,
only the interesting implementation details are reported here.
1. Session Time Line. This element was implemented exactly as described

in §5.8.1.
2. CPU Load % Line. As specified in §5.8.2, if the system supports CPU

monitoring, for each second, the CPU line plots the current system load
as a gradient area graph. Above the 50% load line, the plotting colour is
gradually shifted from green to red.
3. Agent Life Line. Our sample screen shot (figure 6.4) clearly shows how
the background colours of the life lines vary with the impact of each agent
on the global performance, as stipulated in §5.8.3. Also, the life line captions
“stick” to the left border of the view port so that developers know whose
life line is currently displayed. For time reasons, we didn’t implement the
advanced life line manipulation operations, i.e. moving life lines around and
hiding them selectively.
4. Message Lines. Implemented exactly as stated in §5.8.5.
5. Performance Timed Events. Implemented exactly as specified in §5.8.4.
6. Tooltip Window. This element is mentioned in §5.8.7. In our sample

screen shot (figure 6.4), the tooltip window gives the exact time stamp
associated with a position on the session time line.
7. Message Send/Receive Events. The outgoing mail box represents a

message send event, and the little opened envelope represents a receive
event. To reinforce the message lines visual expressiveness, the icons are
respectively used as origin and destination for the message lines (see §5.8.5).
8. Bird’s Eye View. As made obvious in our sample screen shot (figure 6.4),
the Bird’s Eye View provides important visual information about the current
view port global position whilst enabling developers to navigate quickly
around a large display by dragging the translucent blue rectangle (see §5.8.8).
9. Navigation Scroll Bars. Another way to navigate swiftly around the

diagram is provided by the horizontal and vertical scroll bars.
Controls
Developers can interact with the diagram using the mouse as follows:
Tooltip: Hover the mouse pointer over an item to display a tooltip window.
Pan: Left-click and drag the background to pan the display view.
6.7. Conclusion 83
Zoom: Right-click and drag the mouse left or right to zoom the display
view.
Zoom-To-Fit Item: Left double-click any item to zoom the display to fit
the item.
Zoom-To-Fit Enclosing Item: Right double-click any item to zoom the

display to fit the enclosing item.
Scroll: Use the scroll bar controls to scroll the display view.
Navigate: Left-click and drag the translucent blue rectangle in the Bird’s
Eye View to change the current view’s focus.
6.7 Conclusion
To sum up, we have documented the implementation details of the Agent-

Spotter profiler system for Agent Factory. We have shown that we have
managed to cover 95% of the required feature set in the beta version of
AgentSpotter: only the multi-platform aspect was left out.
From an engineering point of view, the main lesson of the AgentSpotter
system development is the high level of reuse that can be achieved nowadays,
thanks to the wide availability of high quality free libraries and frameworks
in every functional domain. We have, in a very short time, successfully
produced a beta level application, not a prototype.
From a functional point of view, our program has proved it provides
a superior user experience. Admittedly, user experience should not be a
primary concern in a standard academic software project. In the domain of
interactive visualisation programs however, user experience is the essential
element that transforms a tool into an extension of a developer’s mind,
Figure 6.4: Space-Time Diagram screen shot. Obtained from the Agent
Herders application [14].
unobtrusively unlocking the access to areas or dimensions that are otherwise

hard or even impossible to reach. Developing multiagent systems is a complex
task, therefore MAS developers need smart tools to tame this complexity.
In the next chapter, with the intention of validating our approach, we
have evaluated the effectiveness of AgentSpotter on a set of representative
Agent Factory applications.
CHAPTER 7
Agent Spotting: an evaluation
7.1 Introduction
In order to validate AgentSpotter, we have used it to profile a set of repre-

sentative Agent Factory applications, namely:
• AFBenchmark: a benchmarking application that we have created

specifically for AgentSpotter.
• Agent Sherpa: a simulation designed by fellow students Iain Hull,

Chris Walsh and Aidan Morrissey [21].
• Agent Herders: a complex program that was entered to the Multi-

Agent Programming Contest 2008 by the Dublin Bogtrotters [14].
The goal of our evaluation was threefold:
• To ensure that the program works as expected.
• To demonstrate that AgentSpotter really provides unprecedented

insights into agent-oriented applications.
85
86 Agent Spotting: an evaluation
• To verify AgentSpotter’s ability to cope with high volume and complex

profiling data.
7.2 Benchmark application
7.2.1 Specification
While developing AgentSpotter, we had to create a specific testbed Agent

Factory application that could exercise the whole feature set. The require-
ments for this application can be summarised as follows:
• Load history: a normally distributed random load history should be

generated so that we can get an idea of a “normal” profile which can
be contrasted with “abnormal” profiles where, for example, a single
agent is monopolising all the load, or the load is spread equally among
all agents.
• Agents: the number of active agents should be changeable dynamically

to simulate process escalation.
• Interactions: in addition to direct interactions, the application should

exercise some task delegation scenarios. The idea is to generate
multiple hops messaging scenarios and see their impact on performance.
• Messages: agents should generate a steady flow of messages with

occasional bursts of intense communication.
• Performance events: all three performance behaviours described in

§5.8.4 should be represented, i.e. green (t ≤ 50% time slice), orange
(50% ≤ t ≤ 75% time slice), and red (t > 100%).
We managed to fulfil these requirements by creating a MAS with overseer

agents that request worker agents to execute small, medium or large tasks.
7.2. Benchmark application 87
Worker agents that have been recently overloaded will simply refuse to carry
out the tasks (in a real application they would inform back the requester
about their refusal). From time to time, overseer agents would request
agents to delegate some tasks. In this case, worker agents will behave as
overseers just for one round.
In terms of user interface, the benchmark application is not very spec-
tacular. It only offers a control window shown in Figure 7.1 which allows
developers to start and pause the process, with a slider control that sets the
number of active worker agents.
7.2.2 Evaluation scenario and objective
In effect, the benchmark application has allowed us to test the following

scenario:
• Start the session with a dozen of worker agents and two overseer
agents.
• After 10 minutes add 15 worker agents to spread the load.
• After 4 minutes, suspend the process for 20 seconds.
• Reduce back the number of worker agents to a dozen.
• Run for 5 minutes more then stop the session.
The objective of the performance analysis is twofold: to see the effect

of increasing the number of worker agents on a profile, and to see how
AgentSpotter deals with a relatively long running session.
7.2.3 Flat profile
The resulting flat profile of this test is reproduced in Table 7.1. For the
reader’s convenience, the maximum value for each column is identified by
an enclosing box. Overseer agents are called “master1” and “master2”. The
worker agents are called “agent” followed by a number e.g. “agent007”.
Firstly, the benchmark appears to make a good job of producing a load
history following a normal distribution1 .
Secondly, we can draw the following conclusions from a quick study
of Table 7.1:
• The most active agents in terms of number of iterations are the overseer
agents, “master1” and “master2”, however in terms of CPU load and
overload, three worker agents are topping the list with 30% of the
total activity: “agent001”, “agent009”, and “agent003”.
• The agents with the highest CPU load also display a high number of
time slice overshoots, and a high average time slice duration.
• As expected, the overseer agents were very busy exchanging messages

with the workers. But it seems that messaging is not CPU intensive2 ,
so their activity percentage is very low, at 1% only. In other words,
bossing other agents around is not really tiring!
In the end, as the performance is distributed normally, the flat profile

does not reveal that much, it just confirms with numbers our original design
principles. Which is, of course, exactly the result we expected.
1
Applying the Shapiro-Wilk test for normality on the activity percentage column
yields W = 0.81 with p < 0.0001. The closer to 1 W , the more probable our null
hypothesis that the data set comes from a normally distributed population.
2
However, it would be really interesting to get an idea of the Message Transport
Service load by adding a special performance data channel.
7.2. Benchmark application 89
7.2.4 Call graph and space-time diagram
In contrast, the visualisations provide far more intriguing insights. For a

start, the session call graph3 indicates that messages sent by “master1” have
induced 74.3% of the session activity, whereas “master2” has commandeered
only 20.5%. To put it differently, “master1” was a more effective boss than
“master2”. Unsurprisingly, the agents that “master1” had the most impact
on were: “agent001” (10.1%), “agent009” (7.3%), and “agent003’ (6.2%).
These agents are of course the top performers revealed by the flat profile.
Both overseers were running the same code, and even shared the same
random seeds, so why 70/30 instead of the intuitive 50/50?
Maybe a look at the space-time diagram displayed in Figure 7.2 will
help us unravel this mystery. The numbered areas on this diagram can be
described as follows:
1. This portion of the diagram shows what happens when only 12 workers
are active. The large red rectangles that usually come in groups of
three represent the time-consuming tasks ordered by the overseer
agents. These blocks never overlap because of the way Agent Factory
schedules agents4 , and they come in bursts, because both overseers
send the same order to the same worker at the same time5 .
2. At this point, 15 more workers are added to the fray, after a slight
pause revealed by the absence of message lines.
3
We did not provide a screen shot for the call graph due to its size: the resolution
loss from scaling down a bitmap would have made the text illegible.
4
The Agent Factory scheduler does not preempt agents that have exceeded their time
allocation.
5
This was revealed by zooming into what appeared to be a single message line. At a
high magnification level, there were in fact two messages lines within a few microseconds
interval to the same worker.
3. This third portion shows the impact of the new workers. The red
blocks are still there, but they are better spread among the agents, so
they don’t come as clumps of three anymore. It looks as if the system
has successfully scaled up.
4. The Bird’s Eye View reveals the bigger picture, and reminds us that
we are looking only at one third of the overall session.
7.2.5 Conclusion
There would be much more to say, but we have clearly demonstrated the
explaining power of AgentSpotter. Unfortunately, we didn’t solve the 70/30
overseer time sharing mystery.
After obtaining these very positive results within a carefully controlled
environment, we were motivated to try agent spotting applications in the
“wild”, as exposed in the following sections.
Figure 7.1: Benchmark application control window
7.3 Agent Sherpa
Agent Sherpa is a simulation of a system “surveying hostile environments

with remote patrol agents” [21]. It was written by fellow students as an
assignment for an agent-oriented course we have taken part in. We have
7.3. Agent Sherpa 91
Table 7.1: Benchmark application flat profile
Total Session Time 18:50.691

Total Activity 10:29.164
Messages Sent 1206
Messages Received 1206
Time Slice Duration 1000 ms
Agent T > 0 T > 100% Activity % Session M ax(T ) Average(T ) Msg. Msg.
iterations overload mm:ss.ms activity ss.ms ss.ms sent rec.
agent001 338 22 1:08.564 10.90 3.740 0.202 6 57
agent009 365 21 1:04.257 10.21 3.425 0.176 13 77
agent004 349 22 1:01.529 9.78 3.235 0.176 10 69
agent014 284 14 46.413 7.38 3.148 0.163 2 36
agent003 401 13 43.881 6.97 3.323 0.109 12 76
agent006 361 12 40.141 6.38 3.279 0.111 12 73
agent005 367 12 34.903 5.55 3.325 0.095 17 76
agent013 301 9 34.716 5.52 3.190 0.115 14 71
agent007 378 11 31.864 5.06 3.356 0.084 21 71
agent008 357 7 30.850 4.90 3.201 0.086 14 72
agent010 330 8 30.280 4.81 3.147 0.091 21 81
agent015 285 9 29.382 4.67 3.257 0.103 4 42
agent002 348 8 23.196 3.69 3.147 0.066 9 70
agent011 357 5 19.363 3.08 3.095 0.054 4 39
agent012 225 3 13.172 2.09 3.049 0.058 9 41
master2 901 0 6.681 1.06 0.183 0.007 504 86
master1 873 0 6.485 1.03 0.227 0.007 514 82
agent024 46 2 6.281 1.00 3.045 0.136 3 7
agent019 31 1 4.449 0.71 3.014 0.143 0 5
agent026 42 1 4.400 0.70 3.084 0.104 0 4
agent030 26 1 4.002 0.64 3.132 0.153 2 8
agent017 46 1 3.811 0.61 3.031 0.082 0 3
agent025 40 1 3.767 0.60 3.006 0.094 0 3
agent027 31 1 3.694 0.59 3.103 0.119 0 2
agent018 38 1 3.384 0.54 3.044 0.089 2 7
agent020 39 0 1.762 0.28 0.547 0.045 0 3
agent022 47 0 1.523 0.24 0.559 0.032 5 13
agent021 32 0 1.300 0.21 0.555 0.040 2 7
agent016 219 0 1.194 0.19 0.555 0.005 2 6
agent029 38 0 1.039 0.17 0.550 0.027 2 8
agent028 45 0 0.749 0.12 0.546 0.016 1 4
agent032 34 0 0.749 0.12 0.561 0.022 1 4
agent031 36 0 0.742 0.12 0.545 0.020 0 2
agent023 40 0 0.598 0.10 0.543 0.014 0 1
agent033 30 0 0.043 0.01 0.003 0.001 0 0
Figure 7.2: Benchmark application sample space-time diagram (18 minute long session)
92
7.3. Agent Sherpa 93
selected Agent Sherpa because it explicitly endorses a “Command & Control”

architecture where a central unit “monitors the drones and predicts locations
of other IED’s6 or ‘hotspots’ of enemy activity.” [21]
Such a centralised architecture is a potential problem for scaling up, as
we have explained in Table 4.1, Chapter 4. Hence our interest in profiling
a first run with 3 drones only, and then a second run with 12 drones, in a
view to study the scaling behaviour of the system.
The 3-drone session profile listed in Table 7.2 confirms the centralisation
of the application. The “CandC” agent represents 70.83% of the computation
time. Despite 7 time slice overshoots, the system performs well, and has
efficiently executed its assigned task. The call graph numbered 2 displayed
in Figure 7.3 reveals that outgoing messages from “drone2” have the highest
impact because they determine 74.2% of the overall computation time. This
impact consists entirely in “CandC” processing map data using sensor data
returned by the drone. As shown in the call graph, map data is returned
as “mapData(...)” using the inform performative. Conversely, outgoing
messages from “CandC” account for 25.8% of the total computation time.
This impact is distributed among the drones broadly the same way it is
distributed in the profile.
A quick study of the space-time diagram, numbered 2 in Figure 7.4, first
shows that the process is rather CPU intensive (red gradient throughout),
but also reveals the existence of a regular rhythm in the execution of the
system. The very dark “CandC” life line is the visual expression of the
agent’s monopoly on computation time.
The second session, running 12 fully active drones, has a strikingly
different space-time diagram, numbered 1 in Figure 7.4. First the “CandC”
6
Stands for “Improvised Explosive Devices”.
life line is nearly black, which is confirmed by the 96% computation time
share in the flat profile listed in Table 7.3. Then the “CandC” life line starts
with an uninterrupted series of green boxes, which turn completely red after
one minute of run-time, and end up saturating the life line. Clearly, as
confirmed by the CPU usage line, the system is on its knees. A quick glance
at the call graph numbered 1 in Figure 7.3 shows that the outgoing messages
from “drone1” have determined 93.3% of the total computation time! Does
that mean that “drone1” is the culprit?
Since we only aimed to show that AgentSpotter can help discover prob-
lems, we won’t really try to solve the questions raised by the Agent Sherpa
profiles study7 . However, we have confirmed the effectiveness of Agent-
Spotter tools on a “real” application. In the next section we will see how
AgentSpotter behaves when confronted with a much more complex MAS.
Table 7.2: Agent Sherpa application flat profile for 3 drone session

Total Activity 25.367
Messages Sent 320
CandC 177 7 17.968 70.83 0.623 0.101 237 83
drone3 419 1 3.467 13.67 0.354 0.008 37 79
drone2 424 0 2.753 10.85 0.264 0.006 37 79
drone1 220 0 1.179 4.65 0.140 0.005 9 79
7
Our theory is that “CandC” is overwhelmed by the number of requests sent back by
all the drones at the same time.
7.3.
Agent Sherpa
Figure 7.3: Comparison of Agent Sherpa call graphs

95
Figure 7.4: Comparison of Agent Sherpa space-time diagrams – 1: 12 concurrent drones; 2: 3 concurrent drones
96
7.4. Agent Herders 97
Table 7.3: Agent Sherpa application flat profile for 12 drone session

Messages Sent 372
CandC 627 421 4:47.707 96.31 1.074 0.458 244 128
drone1 202 1 2.104 0.70 1.058 0.010 14 20
drone7 225 1 1.363 0.46 0.308 0.006 14 20
drone6 228 0 1.292 0.43 0.132 0.005 14 20
drone12 167 0 1.017 0.34 0.096 0.006 14 20
drone9 136 0 0.995 0.33 0.162 0.007 14 22
drone2 142 0 0.804 0.27 0.166 0.005 14 20
drone4 160 0 0.661 0.22 0.128 0.004 14 21
drone11 95 0 0.658 0.22 0.109 0.006 8 20
drone8 115 0 0.588 0.20 0.097 0.005 8 20
drone3 75 1 0.568 0.19 0.308 0.007 3 20
drone10 121 0 0.521 0.17 0.070 0.004 3 21
drone5 109 0 0.448 0.15 0.077 0.004 8 20
7.4 Agent Herders
Agent Herders is a complex application that we won’t try to describe fully

here. In a few words, two teams of 6 agent herders compete in herding cows
on a two-dimensional board using different strategies. We invite the reader
to refer to [14] for a more extensive discussion.
What interests us in Agent Herders is the complexity of its coordination
structure, namely a system of auctions to share tasks between the herd
leader, the map explorer and the herder agents. The comparison of the
different AgentSpotter profiles in Table 7.5 indicates that Agent Herders is an
order of magnitude more complex than the other profiled MASs: thousands
of interactions and activities versus hundreds in around 5 minutes.
Our main goal is to evaluate the performance of AgentSpotter when
loading the profile of a complex application. The comparison of visualisa-

tions load times in Table 7.6 provides another evidence of Agent Herders’
complexity: it takes AgentSpotter 16× more time to load the call graph for
Herders than to load the call graph for Benchmark! Given that it should
require at least K × Nk × Ma × Ωm operations8 to build Herders’ call graph,
it is quite a good performance (see §5.6.3 in Chapter 5 for a discussion about
the call graph model).
The time taken by the space-time diagram is more linear9 , therefore it
takes only 3× the time to load it for Herders than for Benchmark.
For the sake of completeness, we have reproduced the Agent Herders’
flat profile in Table 7.4, and call graph in Figure 7.5, but we will leave them
uncommented. We have also taken a screen shot of the space-time diagram
to support an earlier discussion in Figure 6.4 on page 84.
7.5 Evaluation summary
To summarise, we have conducted a thorough evaluation of AgentSpotter

by using it to profile a benchmark application of our creation, a “real” MAS
with scalability issues, and a very complex system that was entered in a
research contest. Each evaluation has confirmed the effectiveness of our tool
in different areas. Firstly, our tool suite works exactly as we have designed
it, and correctly produces all the results we have specified, whatever the
application profiled. Secondly, we have successfully demonstrated that the
novel analysis tools we have created truly provide new insights into the
8
Based on the following approximations: K = 9, Nk = 9, Ma = 4349/9, Ωm = 3, it
took around 5.7 × 107 operations, i.e. 5.7 × 107 ÷ 80secs = 7.1 × 105 /sec. Of course this
does not stand for a rigorous algorithmic analysis, but it gives a good idea of the order of
magnitude involved.
9
Most of the complexity of the space-time diagram lies in the setup of the graphical
scene. Indeed, each small element of the scene requires a Java object. Consequently, a
large diagram requires a large amount of memory.
7.5. Evaluation summary 99
Table 7.4: Agent Herders’ flat profile

Messages Sent 4349
explorer 768 0 36.099 34.44 1.037 0.047 1515 1562
herdLeader 1543 0 23.736 22.64 0.329 0.015 275 994
botagent2 780 0 8.871 8.46 0.238 0.011 547 1095
botagent4 765 0 7.992 7.62 0.268 0.010 489 1062
botagent5 717 0 7.425 7.08 0.324 0.010 442 1068
botagent6 755 0 6.954 6.63 0.284 0.009 465 1063
botagent1 700 0 6.354 6.06 0.266 0.009 307 1055
botagent3 658 0 5.531 5.28 0.361 0.008 309 997
hManager 375 0 1.860 1.77 0.320 0.004 0 910
Table 7.5: Comparison of the profiled sessions in terms of number of events
Event type Herders Sherpa-12 Sherpa-3 Benchmark

agentContainer/resume 9 13 4 105
agentContainer/suspend 0 0 0 22
cpu 280 342 169 1130
message/receive 9806 372 320 1206
message/send 4349 372 320 1206
scheduler/activity 7061 2402 1240 7680
Session time 4:40.069 5:42.244 2:49.363 18:50.691
Figure 7.5: Agent Herders’ call graph

100
7.5. Evaluation summary 101
Table 7.6: Comparison of AgentSpotter visualisations load times on a 2GHz

Dual Core Pentium machine
Agent Herders Benchmark
Flat profile < 1 sec < 1 sec
Space-time diagram 15 secs 6 secs
Call graph 1 min 20 secs 5 secs
behaviour of a MAS. Finally, we have verified the ability of our system to

handle high volume and complex profiles.
CHAPTER 8
Conclusions
8.1 Related work
To the best of our knowledge, AgentSpotter is the first profiler system

specifically designed for a MAS using agent-oriented performance concepts,
although there is a strong MAS research history in the main domains covered
by AgentSpotter: MAS developer tools, and MAS visualisation tools.
In the domain of interactive tools specifically geared towards MAS
developers, we can cite, for example, other tools developed for Agent Factory:
the AF Debugger [7], and the VIPER agent protocol editor tool [41].
A fundamental approach on visualisation and debugging tools architec-
ture for MASs can be found in [35] where Ndumu et al. state that visualising
MASs is a “notoriously difficult task” because of the information overload,
and present a system comprising a suite of tools, each tool providing a
different perspective on the studied MAS.
Common techniques used for MAS visualisation are based on clustering
and classification algorithms as in [42] where Shroeder and Noy compare the
relative merits of data dimension reduction techniques in order to generate
103
104 Conclusions
summary visualisations. In contrast, AgentSpotter stresses the importance

of exploiting the human visual processing abilities.
Lam and Barber correctly argue in [27] that “software comprehension ...
is essential for developing, maintaining, and improving software”, notably
MASs. Their global approach, which combines a methodology (Tracing
Method ) and a tool (Tracer Tool ), has inspired the design of AgentSpotter.
As mentioned in Chapter 4, while MAS performance study is well repre-
sented in the literature, only the Cougaar system [19] offers a performance-
minded infrastructure.
As Lynch and Rajendran remind us in [32], the ultimate goal is “breaking
into industry”, which requires the creation of industry-grade development
tools prefigured by AgentSpotter.
AOSE, and a fortiori MAS profiling, is ultimately just one aspect of
the bigger roadmap for agent-oriented technologies defined in [31] by the
AgentLink organisation.
Outside the MAS domain, AgentSpotter must be related with the recent
explosion of infovis (information visualisation) research, and more specif-
ically the domain of performance visualisation in parallel and distributed
applications as described in [43]. Graph visualisations like the Call Graph
are surveyed by Herman and al. in [20]. The Pajé visualisation system
described in [36], developed by France Telecom researchers, is very close to
AgentSpotter in its architecture and visualisation offer, but is targeted at
traditional distributed systems.
8.2 Future work
We envision four main axes of improvement for AgentSpotter.

Firstly, we would like to port our tool to another major Java-based MAS
8.3. General conclusion 105
platform like JADE. As we explain in Chapter 5, only the data capture

apparatus should require a specific implementation.
Secondly, we should expand the range of data captured and displayed by
AgentSpotter. For instance, the performance of additional system services
should be recorded, and more details should be collected about agents’
performance events like the distribution of the execution time among the
sensors, actuators, reasoning engine etc.
The third task would consist in extending AgentSpotter to a multi-plat-
form profiling tool as we suggest in §5.8.6.
The last, and more practical, task would be to make the AgentSpotter
tool even more usable by providing, for example, a way to save pictures and
tables to files, and to execute custom queries on the underlying performance
data1 .
8.3 General conclusion
In this dissertation, we have presented our implementation of AgentSpotter

for Agent Factory. It is the first generic MAS profiling system destined to a
Java-based, FIPA compliant, agent-oriented framework, and provides the
following features:
• Mapping of the conventional profiling concepts to agent-oriented con-

cepts.
• Platform-independent architecture.
• Integration of the performance analysis infrastructure right into Agent

Factory’s run-time environment.
1
This is already possible outside AgentSpotter by using one of the existing query
tools for SQLite databases.
106 Conclusions
• Implementation of a suite of novel interactive visual tools that aid

MAS developers understand and debug performance issues.
We have also successfully demonstrated how AgentSpotter sheds new

light on MAS performance behaviour, by using it to evaluate a set of
representative systems designed with Agent Factory.
It is our hope that this contribution, which facilitates the conduct of
MAS performance analysis, will foster more interest in the comparative
performance studies of MAS designs.
Appendices
107
Appendix A
Building AgentSpotter from the subversion

repository
AgentSpotter is available from the Agent Factory (AF) area on SourceForge

at http://sourceforge.net/projects/agentfactory. It is released un-
der the same licence as AF, i.e. the GNU Lesser General Public License,
and comes with full source code.
If you wish to build it from source, here is the procedure to follow. First
ensure that your system meets the following requirements:
• Netbeans IDE version 6.1.
• JDK version 1.6+.
Then download the GNU tarball from http://agentfactory.svn.sourceforge.

net/viewvc/agentfactory/agentspotter/trunk/, or checkout the project’s
trunk using SVN by using the following command-line:
svn co h t t p s : / / a g e n t f a c t o r y . svn . s o u r c e f o r g e . n e t / s v n r o o t /
a g e n t f a c t o r y / a g e n t s p o t t e r / trunk a g e n t s p o t t e r
Open the Netbeans IDE, then open the src/AgentSpotterStation

directory as a project (File→Open Project... menu). Ensure that the “Open
Required Projects” check box in the displayed “Open Project” dialog box is
checked, as shown on Figure A.1.
109
110 Building AgentSpotter from the subversion repository
Figure A.1: Netbeans “Open Project” dialog box
Once the project is opened use the usual Netbeans commands to build
the application.
Appendix B
Getting started with AgentSpotter
B.1 Profiling
Here are the steps required to profile an existing AF application:
• First ensure that your application is compatible with AF version 1.2.
• Also, if your application has a main frame, verify that the default
close operation is to exit the application i.e. add the following code if
needed:
...
JFrame mainFrame = new MyFrame ( ) ;
mainFrame . s e t D e f a u l t C l o s e O p e r a t i o n (
WindowConstants . EXIT ON CLOSE) ;
...
The idea is to make sure that the application shuts down gracefully so
that AgentSpotter has a chance to flush its database records.
• Add the following line to the platform configuration file (.CFG):

PLATFORMGUI c o m . a g e n t f a c t o r y . a g e n t s p o t t e r . A g e n t S p o t t e r
Alternatively, you can specify:

PLATFORMGUI c o m . a g e n t f a c t o r y . a g e n t s p o t t e r . A g e n t S p o t t e r
AUTOCOMMIT
111
112 Getting started with AgentSpotter
This will flush the database at each write, which is rather slow but is
necessary if the MAS has not determined exit point.
• Assuming that you have downloaded the distribution from http:

//sourceforge.net/projects/agentfactory and that you have un-
compressed it to a directory called agentspotter, add all the JAR
files from agentspotter/lib to the application class path.
• Run the MAS.
• When the MAS has terminated, load the generated “profile.aspot” file
into AgentSpotter Station. This file is generated in the default current
directory of the profiled AF application.
B.2 AgentSpotter Station
The AgentSpotter Station GUI is rather intuitive and tries to follow the
user interface conventions used by the Netbeans IDE. Here are a few points
that will help you getting started:
• To load a snapshot file, right click on the root element titled “Agent-
Spotter Snapshots” in Snapshot Window (element numbered 1 shown
in Figure 6.2 screen shot on page 78), and select the Add Snapshot
File... in the contextual menu.
• In the displayed open file dialog, locate and load the “profile.aspot”
file you have just generated.
• If any window is missing from the main display, you can make it appear
by going to the Window menu and selecting the appropriate menu
entry i.e. Snapshot Window or Bird’s Eye Window.
B.2. AgentSpotter Station 113
• To display a visualisation, expand a snapshot file loaded in the Snapshot

Window, then right click on a session to pop up a contextual menu,
then select the desired visualisation.
• Check the Netbeans documentation for an explanation of how the

window layout system works.
Appendix C
AgentSpotter Snapshot File Database Schema
Figure C.1: Database schema diagram of the AgentSpotter profile data file
115
Bibliography
[1] Benjamin B. Bederson, Jesse Grosjean, and Jon Meyer. Toolkit design
for interactive structured graphics. IEEE Transactions on Software
Engineering, 30:535–546, 2004.
[2] Fabio Luigi Bellifemine, Giovanni Caire, and Dominic Greenwood.

Developing Multi-Agent Systems with JADE. Wiley Series in Agent
Technology. Wiley, 2 2007.
[3] B. Brassel, M. Hanus, F. Huch, J. Silva, and G. Vidal. Run-time

profiling of functional logic programs. In Proceedings of the Interna-
tional Symposium on Logic-based Program Synthesis and Transforma-
tion (LOPSTR04), pages 182–197. Springer LNCS, 2005.
[4] F. Brazier, M. van Steen, and N. Wijngaards. On MAS scalability. In

T. Wagner and O. Rana, editors, Proceedings of Second International
Workshop on Infrastructure for Agents, MAS, and Scalable MAS, pages
121–126, 05 2001.
[5] Bradley J. Clement and Anthony C. Barrett. Continual coordination

through shared activities. In AAMAS ’03: Proceedings of the second
international joint conference on Autonomous agents and multiagent
systems, pages 57–64, New York, NY, USA, 2003. ACM.
117
118 Bibliography
[6] R. W. Collier, M. J. O’Grady, G. M. P. O’Hare, C. Muldoon, D. Phelan,

R. Strahan, and Y. Tong. Self-organisation in agent-based mobile
computing. In DEXA ’04: Proceedings of the Database and Expert
Systems Applications, 15th International Workshop, pages 764–768,
Washington, DC, USA, 2004. IEEE Computer Society.
[7] Rem W. Collier. Debugging agents in agent factory. In Rafael H.

Bordini, Mehdi Dastani, Jürgen Dix, and Amal El Fallah-Seghrouchni,
editors, PROMAS, volume 4411 of Lecture Notes in Computer Science,
pages 229–248. Springer, 2006.
[8] Rem W. Collier. Web site for Agent Factory, 2008. http://www.
agentfactory.com/ (accessed October, 2008).
[9] Rem W. Collier. Web site for Agent Factory, Platform Service De-
velopment Guide, 2008. http://www.agentfactory.com/index.php/
Platform_Service_Development_Guide (accessed October, 2008).
[10] Rem W. Collier. Web site for Agent Factory, The Logger API Guide,
2008. http://www.agentfactory.com/index.php/The_Logger_API_
Guide (accessed October, 2008).
[11] Rem W. Collier, Gregory M. P. O’Hare, T. D. Lowen, and Colm

Rooney. Beyond prototyping in the factory of agents. In Vladimı́r
Marı́k, Jörg P. Müller, and Michal Pechoucek, editors, Multi-Agent
Systems and Applications III, 3rd International Central and Eastern
European Conference on Multi-Agent Systems, CEEMAS 2003, Prague,
Czech Republic, June 16-18, 2003, Proceedings, volume 2691 of Lecture
Notes in Computer Science, pages 383–393. Springer, 2003.
Bibliography 119
[12] Claudio Cubillos, Franco Guidi-Polanco, and Ricardo Soto. Performance

analysis of a multiagent architecture for passenger transportation. In
Jan van Leeuwen, Giuseppe F. Italiano, Wiebe van der Hoek, Christoph
Meinel, Harald Sack, and Frantisek Plasil, editors, SOFSEM (1), volume
4362 of Lecture Notes in Computer Science, pages 591–600. Springer,
2007.
[13] R. Deters. Scalability and Multi-Agent Systems. In AGENTS ’01: Pro-

ceedings of Workshop “Infrastructure for Scalable Multi-Agent Systems”,
pages 127–134, 2001.
[14] M. Dragone, D. Lillis, C. Muldoon, R. Tynan, R. W. Collier, and

G. M. P. OHare. Dublin bogtrotters: Agent herders. In Proceedings of
the 6th International Workshop on Programming Multi Agent Systems
(PROMAS-2008), Estoril, Portugal, 2008.
[15] FIPA. FIPA ACL Message Structure Specification. FIPA, 2001.
[16] FIPA. FIPA Abstract Architecture Specification. FIPA, 2002.
[17] Susan L. Graham, Peter B. Kessler, and Marshall K. Mckusick. Gprof:

A call graph execution profiler. SIGPLAN Not., 17(6):120–126, 1982.
[18] Jeffrey Heer, Stuart K. Card, and James A. Landay. prefuse: a toolkit
for interactive information visualization. In CHI ’05: Proceeding of
the SIGCHI conference on Human factors in computing systems, pages
421–430, New York, NY, USA, 2005. ACM Press.
[19] Aaron Helsinger, Richard Lazarus, William Wright, and John Zinky.
Tools and techniques for performance measurement of large distributed
120 Bibliography
multiagent systems. In AAMAS ’03: Proceedings of the second interna-

tional joint conference on Autonomous agents and multiagent systems,
pages 843–850, New York, NY, USA, 2003. ACM.
[20] Ivan Herman, Ieee Cs Society, Guy Melanon, and M. Scott Marshall.
Graph visualization and navigation in information visualization: A
survey. IEEE Transactions on Visualization and Computer Graphics,
6:24–43, 2000.
[21] Iain Hull, Chris Walsh, and Aidan Morrissey. Web site for Agent
Sherpa, Surveying Hostile Environments with Remote Patrol Agents,
2008. http://code.google.com/p/agent-sherpa/ (accessed October,
2008).
[22] Hwaci. Web site for the SQLite Database Engine, 2008. http://www.
sqlite.org/ (accessed October, 2008).
[23] IEEE. Web site for FIPA, 2008. http://www.fipa.org/ (accessed

October, 2008).
[24] Zoltan Juhasz and Prasenjit Paul. Scalability Analysis of the Contract
Net Protocol. ccgrid, 0:346, 2002.
[25] Kresimir Jurasovic, Gordan Jezic, and Mario Kusek. A Performance

Analysis of Multi-Agent Systems. ITSSA, 1(4):335–342, 2006.
[26] Donald E. Knuth. An empirical study of FORTRAN programs. j-SPE,

1(2):105–133, April/June 1971.
[27] D. N. Lam and K. S. Barber. Comprehending agent software. In

AAMAS ’05: Proceedings of the fourth international joint conference
Bibliography 121
on Autonomous agents and multiagent systems, pages 586–593, New

York, NY, USA, 2005. ACM.
[28] Leslie Lamport. Time, clocks, and the ordering of events in a distributed
system. Commun. ACM, 21(7):558–565, July 1978.
[29] L. C. Lee, H. S. Nwana, D. T. Ndumu, and P. De Wilde. The stability,

scalability and performance of multi-agent systems. BT Technology
Journal, 16(3):94–103, 1998.
[30] David Lillis, Rem Collier, Fergus Toolan, and John Dunnion. Evaluating
communication strategies in a multi agent information retrieval system.
In Proceedings of the 5th European Workshop on Multi-Agent Systems
(EUMAS’07), Hammamet, Tunisia, December 13-14th 2007.
[31] M. Luck, P. McBurney, O. Shehory, and S. Willmott. Agent Technology:

Computing as Interaction (A Roadmap for Agent Based Computing).
AgentLink, 2005.
[32] Simon Lynch and Keerthi Rajendran. Breaking into industry: tool
support for multiagent systems. In AAMAS ’07: Proceedings of the 6th
international joint conference on Autonomous agents and multiagent
systems, pages 1–3, New York, NY, USA, 2007. ACM.
[33] SUN Microsystems. Web site for Netbeans Platform, 2008. http:
//platform.netbeans.org/ (accessed October, 2008).
[34] Angel Mur, Liu Peng, Rem Collier, David Lillis, Fergus Toolan, and
John Dunnion. A HOTAIR scalability model. In Proceedings of the 16th
Irish Conference on Artificial Intelligence and Cognitive Science (AICS
2005), pages 359–368, Portstewart, Northern Ireland, 2005. University
of Ulster.
122 Bibliography
[35] Divine T. Ndumu, Hyacinth S. Nwana, Lyndon C. Lee, and Jaron C.

Collis. Visualising and debugging distributed multi-agent systems.
In AGENTS ’99: Proceedings of the third annual conference on Au-
tonomous Agents, pages 326–333, New York, NY, USA, 1999. ACM.
[36] F.-G. Ottogalli, Cyril Labbé, V. Olive, Benhur de Oliveira Stein,

Jacques Chassin de Kergommeaux, and Jean-Marc Vincent. Visualisa-
tion of distributed applications for performance debugging. In ICCS
’01: Proceedings of the International Conference on Computational
Science-Part II, pages 831–840, London, UK, 2001. Springer-Verlag.
[37] Parakey Inc. Firebug JavaScript debugger and profiler. http://

getfirebug.com/js.html (accessed August 4th, 2008), 2008.
[38] Liu Peng, Rem Collier, Angel Mur, David Lillis, Fergus Toolan, and
John Dunnion. A self-configuring agent-based document indexing
system. In Proceedings of the 4th International Central and Eastern Eu-
ropean Conference on Multi-Agent Systems (CEEMAS 2005), Budapest,
Hungary, 2005. Springer-Verlag GmbH.
[39] Omer F. Rana and Kate Stout. What is scalability in multi-agent

systems? In AGENTS ’00: Proceedings of the fourth international
conference on Autonomous agents, pages 56–63, New York, NY, USA,
2000. ACM.
[40] W.J. Reed. The Pareto, Zipf and other power laws. Economics Letters,
74(1):15–19, 2001.
[41] C. F. B. Rooney, R. W. Collier, and G. M. P. Ohare. Viper: Visual

protocol editor. In Proceedings of COORDINATION 2004, pages 279–
293. Springer Verlag, 2004.
Bibliography 123
[42] Michael Schroeder and Penny Noy. Multi-agent visualisation based on

multivariate data. In AGENTS ’01: Proceedings of the fifth international
conference on Autonomous agents, pages 85–91, New York, NY, USA,
2001. ACM.
[43] Eric Shaffer, Daniel A. Reed, Shannon Whitmore, and Benjamin Scha-
effer. Virtue: Performance visualization of parallel and distributed
applications. Computer, 32(12):44–51, 1999.
[44] Connie U. Smith. Performance Engineering of Software Systems.

Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA,
1990.
[45] Connie U. Smith and Lloyd G. Williams. Performance solutions: a

practical guide to creating responsive, scalable software. Addison Wesley
Longman Publishing Co., Inc., Redwood City, CA, USA, 2002.
[46] R.G. Smith. The contract net protocol. IEEE Transactions on Com-
puters, 29(12):1104–1113, 1980.
[47] Amitabh Srivastava and Alan Eustace. Atom: a system for building
customized program analysis tools. In PLDI ’94: Proceedings of the
ACM SIGPLAN 1994 conference on Programming language design and
implementation, pages 196–205, New York, NY, USA, 1994. ACM.
[48] Kenneth S. Stephens and Joseph M. Juran. The Non-Pareto Principle—

Mea Culpa. In Juran, Quality, and a Century of Improvement: The
Best On Quality, volume 15, pages 185–190. American Society for
Quality, 2004.
124 Bibliography
[49] Sun Microsystems, Inc. JVM Tool Interface (JVMTI), Version

1.0. Web pages at http://java.sun.com/j2se/1.5.0/docs/guide/
jvmti/ (accessed August 4th, 2008), 2004.
[50] TILAB. Web site for JADE, 2008. http://jade.tilab.com/ (accessed

February 28th, 2008).
[51] Phillip J. Turner and Nicholas R. Jennings. Improving the scalability of

multi-agent systems. In Revised Papers from the International Workshop
on Infrastructure for Multi-Agent Systems, volume 1887, pages 246–262,
London, UK, 2001. Springer-Verlag.
[52] Michael Wooldridge and Nicholas R. Jennings. Intelligent agents: The-

ory and practice. Knowledge Engineering Review, 10:115–152, 1995.
[53] Yahoo! Inc. Speed up your web pages with YSlow. http://developer.
yahoo.com/yslow/ (accessed August 4th, 2008), 2008.

AgentSpotter: A MAS Profiling System For Agent Factory

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

AgentSpotter: A MAS Profiling System For Agent Factory

Uploaded by

Copyright:

Available Formats

AgentSpotter: a MAS Profiling System for

Dinh Doan Van Bien

Thesis presented for the degree of

Research Supervisor: Dr. Rem W. Collier

Recent advances in Agent Oriented Software Engineering (AOSE)

2 Agents and multiagent systems 5

4 Performance concepts and solutions in MASs 19

5 A blueprint for AgentSpotter, the Agent Factory profiling

6 AgentSpotter design and implementation 69

7 Agent Spotting: an evaluation 85

A Building AgentSpotter from the subversion repository 109

B Getting started with AgentSpotter 111

C AgentSpotter Snapshot File Database Schema 115

3.1 Profilers time line . . . . . . . . . . . . . . . . . . . . . . . . . . 12

5.1 AgentSpotter abstract (platform-independent) architecture . . . 40

6.1 AgentSpotter implementation architecture . . . . . . . . . . . . 71

7.1 Benchmark application control window . . . . . . . . . . . . . . 90

7.2 Benchmark application sample space-time diagram (18 minute

A.1 Netbeans “Open Project” dialog box . . . . . . . . . . . . . . . 110

4.1 Common MAS scalability problems and solutions . . . . . . . . 25

5.1 OOP and AOP execution units mapping . . . . . . . . . . . . . 47

6.1 TraceEvent types generated by the AF-RTE sub-systems . . . . 73

7.1 Benchmark application flat profile . . . . . . . . . . . . . . . . . 91

Recent advances in agent frameworks and methodologies have helped bridg-

Agent Factory (AF) [11], a Java-based agent-oriented toolkit, a project is

Our main objectives are as follows:

1. To map profiling concepts to agent-oriented concepts.

2. To elaborate a conceptual model for generic MAS profiling.

3. To realise and evaluate a profiling system based on this conceptual

In short, we aim to build a comprehensive MAS performance analysis

1.3 Thesis outline

We have classically organised this dissertation in two main parts:

1. Fundamental Concepts and Background Research, which includes the

• Chapter 2 – Agents and multiagent systems.

• Chapter 4 – Performance concepts and solutions in MASs.

2. Project Presentation, which includes the following chapters:

• Chapter 5 – A blueprint for AgentSpotter, the Agent Factory

• Chapter 6 – AgentSpotter design and implementation.

• Chapter 7 – Agent Spotting: an evaluation.

In this chapter, we present the fundamental notions of software agent and

For our purposes, a multiagent system (MAS) is a software system, situated

In the perspective of the two-tier properties-based approach to agent

The Foundation for Intelligent Physical Agents (FIPA) is an organisation

2.3 Agent Factory

AF is an open source Java-based, comprehensive toolkit for the creation

JADE stands for “Java Agent DEvelopment Framework”. It is a major

Cougaar is an open source Java-based platform particularly suited to the

In this chapter, we describe profiling and explain why it matters in software

Most of the time in the software engineering literature, the definition of

definition of profiling which tries to incorporate the elements that make

General Definition. We define profiling as a performance analysis process

• An instrumentation apparatus which is directly weaved into the pro-

• A post-processing system which uses the recorded data to generate

Limited Definition. A more limited definition of profiling designates only the

3.3 Origins of profiling

1970 1980 1990 2000 2010

Figure 3.1: Profilers time line

3.4 First generation profilers

3.4.1 Knuth profilers

3.4.2 UNIX prof

In 1979, the UNIX implementers introduced a sampling profiler for the C

• Run the program. After completion it generates a “mon.out” file

• Set the current shell directory to the “mon.out” file location.