Bushfire Surveillance Using Dynamic Priority Maps and Swarming Unmanned Aerial Vehicles

Bushfire Surveillance Using Dynamic
Priority Maps and Swarming Unmanned

Aerial Vehicles
David John Howden,
DipInfTech, BSc(Computer Science and
Software Engineering)
Swinburne University of Technology
2012
A dissertation submitted in fullment of the requirements of the degree of
Doctor of Philosophy.
Abstract
Bushres are large or destructive conagrations that occur in areas of wilderness, their
remote location serving as a barrier to rapid detection or response. As a result of their
inaccessibility, these res can grow to unsuppressible proportions and not only cause
signicant economic damage to an area, but also endanger the lives of communities and
their re ghters.
Fast and eective detection and monitoring is a key factor in bushre ghting, however
the areas bushres typically occur in are remote, with minimal to no communication in-
frastructure. Fire managers often have to rely on archaic information gathering methods
such as re towers and physical patrols by re ghters at the scene. Having accurate
knowledge of the status of a bushre is indispensable for enabling accurate re predic-
tion modelling, maintaining the safety of re ghting crews and allowing eorts to be
focused on areas of the highest risk such as urban areas with strong human presence.
This problem is ideal for the application of multiple, small unmanned aerial vehicles
(UAVs). The advantages of deploying UAVs as a swarm include massive scalability, low
communication overheads, reduced need for human supervision, and resilience against
individual failure. For deployment of multiple UAVs to be practical, a method for
controlling their actions autonomously and cooperatively is required.
This dissertation presents a swarm intelligence approach to exhaustive and contin-
uous surveillance of large areas. Digital pheromone maps are used to represent the
environment, facilitating both information sharing and collective decision making. In-
dividual pheromone maps, stored on-board, are communicated via short range, peer to
peer broadcasts, allowing for environmental knowledge to be synchronised and spread
through the swarm. This enables emergent coordination in an area without the in-
frastructure needed to provide global communication, and the resilience against loss of
individual vehicles that operations in the vicinity of unpredictable and volatile bushres
requires.
i
Acknowledgements
Professional
With sincere and profound gratitude, I thank my advisor and friend, Professor Tim
Hendtlass. Without Tims, frankly, extraordinary advocacy and support, this thesis
would not exist, and I would not be where I am today. As an undergraduate, Tims
childlike wonder and boundless enthusiasm inspired the same from us, his students.
Even then, his door was always open and he, without exception, made time to humour
my eager but undirected interest in the eld of articial intelligence. As one of Tims
postgrads, his hands-o style of advising helped me grow as a condent, independent
academic, while his eagerness to listen to any problem I may have had, or even just to
idly chat while clearing my mind, meant at the same time I never felt lost or without
guidance.
I was additionally privileged to have received the assistance and support of Dr. James
Montgomery. Some of my fondest memories of Swinburne involve hallway to oce
conversations long after everyone else had left for the night: either discussing work and
sharing the feeling of not enough done, or drinking, knowing that there was always
tomorrow. His almost inhuman ability to locate and correct grammatical aws in my
work, over the course of what feels like a dozen drafts, has resulted in a standard of
quality I never could have achieved alone.
On a personal note, with love to my parents...
I am sincerely thankful for, and wish to earnestly acknowledge, the unconditional support
and love of my father, Kevin Howden. He has stoically come to my aid far more often
than could be considered reasonable over the course of my postgraduate experience.
Knowing that he was there, proud and in the background, in case I ever got myself in
too far over my head, not only allowed me to take risks and push myself further than
would have been otherwise possible, but allowed me to enjoy the ride.
iii
Finally, I would like to express heartfelt gratitude to, and genuine appreciation of,
my mother, Karen Fraser. Despite our continued disagreement over how long in front
of a computer is too long (its never), you did everything in your power to give me
opportunities and push me to succeed at every age. You taught me to always be curious,
and the determination to succeed that instilled, while not always visually evident (to
your despair), has stuck with me to this day.
iv
Declaration of Originality
I hereby declare that this dissertation contains no material which has been accepted for
the award of any other degree or diploma, except where due reference is made; that to
the best of the my knowledge this dissertation contains no material previously published
or written by another person except where due reference is made; and that where work
is based on joint research or publications, the relative contributions of the respective
authors has been disclosed.
David John Howden
v
Contents
1. Introduction 1
1.1. Bushres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2. Unmanned Aerial Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3. Swarm Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4. Flocking and Stigmergy . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2. Existing Work 13
2.1. Potential Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2. Pheromone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3. Realised Swarm Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3.1. Beckers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3.2. Pherobots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3.3. I-SWARM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3.4. iRobot SwarmBots . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3.5. PheGMot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4. Implementation Issues with UAV Swarms . . . . . . . . . . . . . . . . . . 19
2.4.1. Spatial Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.4.2. Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3. Algorithm Implementation 23
3.1. Pheromone Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2. Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2.1. Update Waypoint . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2.2. Merging Pheromone Maps . . . . . . . . . . . . . . . . . . . . . . 26
3.2.3. Storing the Direction of Incoming Broadcasts . . . . . . . . . . . 27
3.2.4. Attraction Points . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
vii
Contents
3.2.5. Pheromone Map Evaluation . . . . . . . . . . . . . . . . . . . . . 29
3.3. Diversity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.4. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4. Simulation Setup 35
4.1. Environmental Representation and Perception . . . . . . . . . . . . . . . 35
4.2. Node Spacing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.3. UAV Specications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.4. Simulator Limitations: Granularity and Abstractions . . . . . . . . . . . 37
4.5. Fire Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5. Performance Baselines 41
5.1. Baseline Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.2. Scaling with Swarm Size . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.3. Stable States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.4. Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
5.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6. Varied Priority Levels 51
6.1. Complex Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
6.2. Null Priority and No-Fly Zones . . . . . . . . . . . . . . . . . . . . . . . 56
6.3. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7. Target detection and tracking 61
7.1. Fire location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
7.2. Fire tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
7.3. Secondary Fire Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
7.4. Discreet Mobile Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
7.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
8. Communication 73
8.1. Scaling with Communication . . . . . . . . . . . . . . . . . . . . . . . . . 73
8.2. Information Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
8.2.1. Data Synchronisation . . . . . . . . . . . . . . . . . . . . . . . . . 76
8.3. Not All Communication Is Good . . . . . . . . . . . . . . . . . . . . . . 79
8.3.1. Deterministic versus Stochastic Behaviours . . . . . . . . . . . . . 81
viii
Contents
8.4. Communication Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
8.5. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
9. Comparative Results 87
9.1. Exhaustive Swarming Search Strategy . . . . . . . . . . . . . . . . . . . . 87
9.2. Uniform Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
9.3. Lake Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
9.4. No-Fly Zone Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
9.5. Analysis of Comparative Results . . . . . . . . . . . . . . . . . . . . . . . 94
9.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
10.Conclusions and Final Remarks 97
10.1. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
10.2. Research Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
10.3. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Appendices 110
A. Glossary 113
B. Comparison of Currently Deployed Unmanned Aircraft Systems 115
C. Publications Arising from this Study 117
ix
List of Algorithms
3.1. A mock UAV control loop, containing the core surveillance algorithm
function, UpdateWaypoint . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2. UpdateWaypoint, the main update function for UAVs using the surveil-
lance algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3. Merging pheromone maps so that each nodes most recent observation is
retained. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4. StoreBroadcastDirection probabilistically stores the position of broad-
casting agent when communication occurs within repulsion range . . . . 28
3.5. GetAttractionPoint calculates a point of attraction by using the UAVs
position as the origin for the sum of the direction vectors contained in
directionList. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.6. EvaluatePheromoneMap heuristically evaluates the UAVs pheromone
map for a destination waypoint using pheromone values and information
derived from previously known peer UAV locations (i.e. the attraction
point). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
xi
List of Figures
1.1. Collection of small US Navy unmanned aerial vehicles . . . . . . . . . . . 3
1.2. Local, and consensus, autonomy levels . . . . . . . . . . . . . . . . . . . 5
2.1. Pheromone maps for swarm robotics overlay a digital grid over the envi-
ronment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1. Visualisation of attraction point calculation and search space occlusion . 30
4.1. Calculation of optimal node separation . . . . . . . . . . . . . . . . . . . 36
4.2. Anatomy of a bushre: head, back, and ank res . . . . . . . . . . . . . 39
4.3. Satellite view of Wilsons Promontory, an Australian national park . . . . 40
5.1. Method for approximating the maximum survey speed of a UAV . . . . . 42
5.2. Decrease in average survey time with increased UAVs . . . . . . . . . . . 43
5.3. Average survey time of the algorithm as a percentage of baseline . . . . . 44
5.4. Increase in pheromone and subsequent levelling o at a stable state . . . 45
5.5. UAVs exhibiting edge following behaviour on the rst pass of a uniform
map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.6. Scale representation of the path a UAV follows when all non-surveyed
nodes have the same pheromone level . . . . . . . . . . . . . . . . . . . . 46
5.7. Typical sequence of moves for a UAV . . . . . . . . . . . . . . . . . . . . 47
5.8. Diagonal movement when node spacing is smaller than the survey radius 47
5.9. An example of diagonal moves leading to interesting, yet unstable, spi-
ralling behaviour. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.1. Priority map with two priority levels . . . . . . . . . . . . . . . . . . . . 51
6.2. Average survey period of the baseline and simulation results using the
two priority map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
6.3. Average survey period on the two priority map as a proportion of the
baseline value (i.e. survey period divided by baseline period) . . . . . . . 53
xiii
List of Figures
6.4. Priority map with four priority levels . . . . . . . . . . . . . . . . . . . . 53
6.5. Priority map with eight priority levels . . . . . . . . . . . . . . . . . . . . 54
6.6. Average survey period of the eight priority level environment, broken
down into individual priority zones. . . . . . . . . . . . . . . . . . . . . . 54
6.7. Eect of UAV density on survey ratios . . . . . . . . . . . . . . . . . . . 55
6.8. Increasing severity of priority levels on identically sized environments to
increase local agent density . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.9. Heat maps generated using the eight priority map . . . . . . . . . . . . . 57
6.10. Comparison of heat maps after the priority map is rearranged into a more
dicult to survey arrangement . . . . . . . . . . . . . . . . . . . . . . . . 58
6.11. Comparison of the three dierent priority map congurations proportion-
ate to baseline performance . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.1. Average detection time of targets based on location . . . . . . . . . . . . 61
7.2. Average detection time of targets in a map with two priority levels . . . . 62
7.3. Average detection time of res at the edge of the search space using a
scaled priority map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.4. Progressive images of the simulation in progress . . . . . . . . . . . . . . 64
7.5. Comparison of the percentage of re area discovered using two dierent
classes of UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
7.6. Progressive images of the secondary re simulation in progress. . . . . . . 66
7.7. Percentage of res found using the secondary re simulation with grass
and forest res . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
7.8. Forest re: dierence in detection between the primary waypointed and
secondary unknown re for various priority levels . . . . . . . . . . . . . 67
7.9. Grass re: dierence in detection between the primary waypointed and
secondary unknown re for various priority levels . . . . . . . . . . . . . 68
7.10. Elapsed time before initial detection of targets in a square 22.5 km envi-
ronment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
7.11. Percentage of time during the simulation that the swarm knew the targets
location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
7.12. Average pheromone of the nodes directly under targets . . . . . . . . . . 71
8.1. Communication eects on average survey period . . . . . . . . . . . . . . 74
8.2. Diminishing returns on communication range over large environments . . 74
xiv
List of Figures
8.3. Convergence of environment pheromone level and pheromone level of
nodes which are synchronised . . . . . . . . . . . . . . . . . . . . . . . . 77
8.4. Eect of communication period on synchronisation level and environment
pass time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
8.5. Percentage of synchronised nodes given varying communication periods . 78
8.6. Average search time, scaled relative to a swarm with zero communication 79
8.7. Communication period scaled by environment size and communication
range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
8.8. Communication period scaled by environment size and swarm size . . . . 80
8.9. Agent density divided by communication range . . . . . . . . . . . . . . 82
8.10. Variance in pheromone levels when adding random noise to the pheromone
map in a sparsely populated environment . . . . . . . . . . . . . . . . . . 83
8.11. Time between surveys of high priority nodes as a function of how fre-
quently one UAV broadcasts random pheromone information . . . . . . . 84
8.12. The time between surveys of medium priority nodes as a function of how
frequently one UAV broadcasts random pheromone information . . . . . 85
8.13. The time between surveys of low priority nodes as a function of how
frequently one UAV broadcasts random pheromone information. . . . . . 85
9.1. Flowchart for the behaviour of Erginacs algorithm . . . . . . . . . . . . 88
9.2. An example of emergent contour following observed during the execution
of Algorithm E using a uniform map. . . . . . . . . . . . . . . . . . . . . 89
9.3. Comparative average mean times of Algorithms E and H, since cells last
visit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
9.4. Lake map featuring four priority levels, inclusive of null priority areas . . 91
9.5. Comparative average survey times using the lake map . . . . . . . . . . . 91
9.6. Mean time since visit for highest priority areas using the lake map . . . . 92
9.7. No-y zone map featuring four priority levels, inclusive of no-y areas . . 93
9.8. Mean time since visit for highest priority areas using the no-y zone map 93
9.9. Heatmaps of node surveys for the no-y zone map . . . . . . . . . . . . . 94
xv
1. Introduction
1.1. Bushres
Bushres are conagrations that start in remote undeveloped areas of the environment,
and have the potential to spread out of control causing immense damage. Bushres, also
know as wildres or forest res in dierent parts of the world, dier from other res in
that the wooded areas they originate in serve as an abundant source of fuel. As bushres
more often occur during summer months, and with greater frequency during droughts,
fuel is often desiccated and extremely combustible, allowing an unmanaged re to turn
into a restorm. During a restorm, a bushre generates its own self-sustaining weather
system: hot deoxygenated air evacuates upwards fast enough to draw in fresh air from
surrounding areas, serving to feed the re with yet more oxygen. The radiant heat
(infrared radiation) of a restorm is great enough to ignite fuel at a signicant distance
ahead of the re front itself (Cheney and Sullivan 2008). Under such circumstances
bushres can result not only in property and environmental damage but severe loss of
human life.
Due to its unique environment, Australia is one of the most bushre prone countries in
the world, disaster-level bushres
1
alone causing an average of A$77,000,000 of damage a
year (Gentle, Neil; Kierce, Sharyn; Nitz, Alistair 2001; Ganewatta and Handmer 2006).
The most signicant reason that bushres so frequently result in this level of destruction
is due to the limited accessibility of the undeveloped forested or mountainous regions
in which ignition often occurs. Before re management can fully mobilise an incident
can already be at an unmanageable size, at which point it can easily spread and threaten
nearby populated areas.
The way that res spread can be modelled with a damage-time function showing the
amount of damage caused increasing exponentially the longer the re is allowed to burn
(Restas 2006a). With this in mind, the speed with which re management can begin
1
The Gentle, Neil; Kierce, Sharyn; Nitz, Alistair (2001) denes disaster-level bushres as those with
a total insurance cost of more than A$10,000,000.
1
Chapter 1: Introduction
is often the decisive factor in how much damage a re causes. To begin re ghting in
earnest, re management requires three things. First, the re manager needs to discover
that there is a re in progress. Second, men and equipment need to reach the re and,
third, preliminary reconnaissance needs to be performed.
Reconnaissance is the most important initial activity by re ghters arriving on the
scene. Knowledge of the res extent and status is necessary both for the safety of
personnel on the ground and for accurate re modelling for use in eciently allocating
resources. The problem with this is that reconnaissance too often requires actually
touring the aected area and the terrain is unlikely to allow for ground vehicles to
be used eectively. The cost of manned aircraft is prohibitive so they are not usually
deployed until after the initial reconnaissance suggests that it is necessary. Satellite re
imaging may seem like a promising alternative, but because of the polar orbit used by
weather satellites and current technological limitations, they are best used for strategic
observations.
2
Strategic observations provide a regional view of the res progress, a mission often
best suited to weather satellites equipped with radiometers and re detection algorithms
(Kant et al. 2000). Tactical observations on the other hand are ideally localised, frequent
and more detailed (Ambrosia et al. 2003), being traditionally performed by re ghters
on the ground. However, even assuming conditions allow for re ghters to arrive while
the re is still contained within an optimistic 150 metre radius, there will already exist
a perimeter of over a kilometre which needs to be surveyed. This process takes time,
and as described the longer a re is allowed to burn uncontrolled, the faster it spreads.
Additionally, as observations are performed manually by individual re commanders,
subjectivity can become a factor, distorting the information the re manager has to
work with.
1.2. Unmanned Aerial Vehicles
Fire managers are interested in possible applications of unmanned aerial vehicles (UAVs)
for use reconnoitring res which are in progress, as well as the initial detection of igni-
tions. A UAV is essentially any vehicle that can y without a human pilot on board,
ranging from the remote controlled helicopters purchasable from toy stores to the General
Atomics MQ-9 Reaper, a four and a half tonne semi-autonomous war plane. Generally
speaking however, the most common type of UAVs are aeroplanes with an approxi-
2
A satellite on a polar orbit has a round trip time of nine hours.
2
Figure 1.1.: Collection of small US Navy unmanned aerial vehicles, the largest of which
being the RQ-2B Pioneer with a wingspan of 5 metres (Source: public do-
main image from http://www.navy.mil, 2005)
mate wingspan of three meters, with partial autonomy, a number of which are shown in
Figure 1.1.
Autonomy, in reference to UAVs, is the ability for the vehicle to y itself through
application of on-board articial intelligence algorithms. Due to technological limitations
and safety regulations, even UAVs which are capable of fully autonomous operation
have historically been limited to use as remote controlled platforms piloted by humans.
This state of aairs is slowly changing with initiatives like the ASTRAEA programme,
which has brought together a consortium of major aerospace companies for the purpose
of developing UAV certication, and the UKs document CAP 722Unmanned Aerial
Vehicle Operations in UK Airspace: Guidance which for the rst time provides guidelines
for verifying autonomous vehicles for use in unrestricted airspaces (Hutchings et al. 2007;
Civil Aviation Authority (UK) 2012).
There are many reasons for interest in UAVs as both a replacement for traditional
aeroplanes, as well as in entirely new domains. The main advantages for UAVs come
from a value standpoint: small UAVs can be purchased for A$5,000 to A$10,000, while
traditional light aircraft start at A$100,000. These values do not factor in the costs
of hiring certied commercial pilots and maintaining the vehicle to meet the additional
safety regulations required for carrying humans. This reduced cost and lack of human
presence is what leads UAVs to be ideal for tasks which are Dull, Dirty (or) Dangerous
(Barber et al. 2006, p. 1).
In most cases the missions that UAVs would be used for, when abstracted, have the
unifying goal of searching a bounded problem space. Work in this eld has so far been
3
mostly focused on discrete searches, where one or more targets exist within an area, and
once they are located the search is complete. The discrete search approach is sensible for
missions such as search and rescue, mapping a static area such as in agricultural survey-
ing, battle damage assessment (BDA), and for short duration Intelligence, Surveillance
and Reconnaissance (ISR) missions (Sauter et al. 2005). Continuous state-space explo-
ration (i.e. surveillance), such as would be required for problems such as re spotting,
border surveillance, or long duration ISR missions, is a research area with a smaller
body of published work.
The idea of utilising aerial surveillance to reconnoitre wildres is an old one, as replac-
ing ground crews in reconnaissance would mean the eect of rugged terrain on survey
time was negated. However, getting a manned aeroplane to the scene of a wildre both
takes too long for the initial survey and is not cost eective for monitoring the progress
of the re if the initial survey showed it to be a minor event.
3
At the other extreme,
where a wildre has grown out of control and generates a restorm, the use of a manned
aircraft is warranted though the environment posses an inherent danger to the vehicle
and pilot. It is apparent then that depending on the state of the re, this reconnaissance
can fall into either the previously mentioned dull or dangerous category, making re
reconnaissance an ideal application for UAVs.
Preparatory work for the rst deployment of a UAV for re monitoring in operational
service was done by Restas (2006a). Of the more interesting things found by getting
the equipment into the hands of actual re ghters was that not only were black and
white images of the res progress sucient, it was even claimed that black and white
was preferable to coloured images for the sake of clarity. The online experiments also
proved the utility of having a UAV for re spotting when an unknown secondary re was
discovered while using the UAV to observe a current re which was in progress. Benets
aside, Restas also pointed out the limitations of a single UAV in that it is insucient
for large res, and that while re monitoring was accomplished, it was unsuccessful at
conducting re spotting patrols.
The advantages of using multiple UAVs over a single platform are similar to the
original advantages of using a UAV instead of a manned vehicle; namely that if there
is danger involved, having multiple vehicles is preferable due to redundancy, so that
the loss of an individual does not have an inordinate eect on mission performance.
Secondly the response time issue, where an on-site UAV is able to reach a re quicker
3
Wildres are reasonably frequent, and the extinction of a medium-sized forest re generally takes
from a couple of hours to a day (Restas 2006b).
4
Figure 1.2.: a) UAVs operating with consensus level autonomy using long range com-
munication, and central control, b) Swarming UAVs operating with local
autonomy using only short range communication
than a manned aircraft can be made active. Similarly, multiple UAVs are able to map a
re more quickly than a single UAV. If each additional UAV needs to be controlled by a
human operator however, this approach rapidly becomes unmanageable as the number
of UAVs increases.
1.3. Swarm Intelligence
For the deployment of large groups of UAVs to be practical, it is necessary to utilise the
autonomous capacities of modern robotics. There are currently two main approaches
to using autonomous robots in large groups, shown in Figure 1.2. The rst is a model
where the robots have long range communication and are able to synchronise their
actions as a group to good eect (Sirigineedi et al. 2009). This is known as a consensus
level of autonomy, where agents work as a team to devise actions (Martin and Barber
1996). As long as communication bandwidth is plentiful and guaranteed, this method
is able to produce optimal or near optimal search patterns. However, some of the
main disadvantages of direct control are still present, such as lack of scalability and long
range bandwidth overheads. The other model utilises local autonomy and is inspired by
biological systems such as ocks of birds or colonies of termites (Bonabeau et al. 1999;
Kennedy 2006; Sahin 2005). This model translates into each agent having only local
knowledge of its environment and planning its own actions based on some combination
of the state of any kin robots in sensor range, the environment it is in, and information
gained via indirect communication.
The biological approach to multi-agent systems falls within the sphere of Swarm In-
telligence (SI), a eld which studies the emergence of complex behaviours from the
interactions of multiple simple entities (Beni 2005). Swarm Intelligence algorithms are
dominated by two key mechanisms, ocking and stigmergy. In the earliest computer
5
simulations of ocking (Heppner and Grenander 1990; Reynolds 1987), it was demon-
strated that complex behaviours exhibited by groups of individually simple creatures
could be replicated via very simple rule-based algorithms. In Reynolds seminal work on
Boids
4
the ocking behaviour of birds was simulated, fundamentally, by simply having
each individual Boid agent try to maintain an optimal distance from its neighbours. Go-
ing a little deeper, this optimal distance was calculated through three interacting forces:
separation, alignment, and cohesion. Separation pushes the Boid away from its closest
neighbour, alignment steers the Boids heading to try and match that of its neighbours,
and cohesion pulls the Boid towards the ocks centre of mass. It is the interaction of
these rules with each other from which the desired ocking behaviour emerges. While
originally designed as an exercise in computer graphics, it would later serve as inspira-
tion for algorithms in other elds, including the well known Particle Swarm Optimisation
(PSO) (Kennedy and Eberhart 1995).
PSO exists at the other end of the swarm intelligence abstraction spectrum; a meta-
heuristic which uses a swarm of particles to y through a tness landscape and converge
on a good solution. Each particle is initialised with a random position and velocity, and
remembers its best found value and position which had resulted in that value. Upon
each update of the algorithm, particles accelerate towards the best position they have
personally seen, as well as the best position seen by any particle in the swarm. The
algorithm, while extremely simple, is surprisingly eective at optimising a wide range of
functions. Unifying these two examples, along with other algorithms in the eld of SI,
are certain traits (Parunak 1997):
Small Agents - Individual agents are not a large or irreplaceable part of the
whole, giving rise to resilience and resistance to loss.
Short Sighted - Agents make decisions predominantly based on knowledge of
their immediate surroundings. They do not make long term plans or individually
keep detailed histories.
5
Decentralised - Decisions are made by the individual, on an agent-by-agent basis.
Swarms never have a centralised control structure.
These traits on their own suce to produce a collection of simple, independent indi-
viduals but will not result in the emergent behaviours which are symbolic of a swarm.
4
The name Boids refers to Bird-like objects.
5
This does not preclude modication of the environment to collectively store information as this is
communal rather than individual.
6
The following additional traits, also suggested by Parunak, are what enable emergent
behaviour:
Ability to Share Information - For agents to interact with each other, there
needs to be some mechanism to spread information. This can be as simple as an
agent observing where a peer is in relation to itself and the environment, to the
pheromone trails left by ant colonies or the waggle dance done by bees (Frisch
1968).
Diversity Mechanism - A homogeneous collection of individuals, sharing knowl-
edge and behavioural rules, is prone to rapid convergence. To counteract this, a
method of injecting diversity is a vital yet subtle element to emergent behaviours.
At a minimum this usually means the presence of a stochastic process (i.e. a
random element).
While small, simple, decentralised individuals describes what a swarm is, the hidden
complexity and emergent behaviours are the result of communal information ow and
counteracting diversity processes. Broadly speaking there are two main types of swarm:
ocks, and colonies. Reynolds (1987) put forward that, at a minimum, to be considered
a ock, agents need to know the relative direction and distance to at least some of their
neighbours. This spatial awareness of kin agents is all the information spread that is
required for a ock to function, however some colony type swarms are able to function
without even this basic information. In colonies, communication is most often done
through modifying the environment itself, a mechanism known as stigmergy.
Stigmergy as a term was rst coined by Grasse in 1959 in relation to the nest building
behaviour of termite colonies (Grasse 1959). In this paper it was discovered that phero-
mone was largely responsible for the ability of the simple individual termites to perform
useful tasks in a coordinated fashion. A pheromone is a chemical that is secreted into
the environment by a swarming creature that elicits a response by other members of the
swarm that sense it. By selectively applying pheromone chemicals to their environment,
individuals in the swarm are able to indirectly communicate with their peers.
In the case of termites, it was found they will occasionally scoop up a ball of earth
which is then coated with pheromone and dropped at random. However if there is already
a pheromone coated mud ball nearby, there is an increased chance that the second will
be placed alongside. As this stack increases in size and the corresponding quantity of
pheromone grows, so does the chance it will be further added to, which eventually results
in a structured termite mound, complete with arches and chambers. This rst action of
7
placing a mud ball which cascades into a full termite mound is an example of a positive
feedback loop, a frequent result of stigmergic communication (Izquierdo-Torres 2004).
Thus while a colony is made up of individual termites seemingly pursuing their own
interests, the shape and state of their surrounding environment inuences their actions
and enables swarm wide coordination.
Positive feedback behaviours are often benecial in the short term but must be bal-
anced by a diversity mechanism (negative feedback) to enable the system as a whole to
continue to function over the long term. In the termite example the pheromone which
was added to the mud balls evaporates over time, eventually disappearing entirely. Other
common diversity mechanisms are repulsion from other agents, limiting the agents scope
to a local subset of the environment, and stochastic elements in the control algorithms
(e.g. the random nature of the termites actions).
As with the Boids/PSO ocking example, stigmergic systems were abstracted to in-
spire algorithm design. The earliest work on ant based systems was by Ebling et al.
(1989), with the most commonly cited algorithm in this sub-category being Ant Colony
Optimisation (ACO) described by Dorigo et al. (1996). In ACO the behaviour of forag-
ing ants nding an optimal path between nest and food source is emulated. The basic
concept is that foraging ants wander randomly, leaving a constant stream of pheromone,
and when food is found they head back to the nest. As with termites, the random nature
of their wandering is inuenced by pheromone quantity, and the pheromone evaporates
over time.
The more circuitous the route taken back to the nest, the weaker the average phero-
mone as it will have had more time to evaporate. Therefore an ant that wanders o
this trail and ends up nding a shortcut will leave a stronger pheromone trail than the
ant travelling by the original route. Over time, the long route will disappear and only
the short one will remain. Eectively what ants solved in nature, and ACO through
biomimicry, is the Shortest Path Problem.
1.4. Flocking and Stigmergy
The two concepts of ocking and stigmergy are central in the implementation of UAV
swarms as they both represent mechanisms for indirect communication and subsequently
self-organisation. Communication within a swarm in general, and especially in robotic
swarms, is almost inherently via one-to-many communication. In addition, the prevailing
opinion in published research is that robotic swarms will require, or at the least only
8
be practical with, low range, local communication (McLurkin and Demaine 2009; Schill
2007).
Broadly speaking there are three reasons for this, the rst being maintaining scala-
bility. Robots should be able to be added and removed from a swarm with no dispro-
portionate eect on performance. If communication is global, then the communication
overheard increases quadratically with each additional agent; a limited communication
range (e.g. short range line-of-sight to kin robots) means only the local density of the
swarm is important, not the ultimate size.
The second reason is that having limited awareness is an important diversity mecha-
nism. Without a method of inserting diversity, agents will increasingly make the same
decision and the system as a whole will converge at some local optimum, especially with
homogeneous swarms but also to a lesser extent with heterogeneous systems.
6
In the
words of Reynolds (1987, p. 30), An interesting result of experiment[ation] is that
the aggregate motion that we intuitively recognise as ocking (or schooling or herd-
ing) depends upon a limited, localised view of the world.
7
Essentially, the better an
individuals decision, the closer it will match the decision made by everyone else. Forc-
ing globally bad, but still locally good, decisions on the basis of limited knowledge
improves the swarms diversity and resilience against failure.
Finally, short range communication requires only minimal hardware. Long range
communication equipment requires an increase of weight, power use, and cost of the
payload required to be carried by the UAV as well as the general additional complexity of
the UAV itself (Varga 2003). There is also a documented shortage of satellite bandwidth
which is currently a key component in any long range communication implementation.
It was noted that if all 12 Predator systems were active and dependent on satellite
communications, they would consume a signicant portion of the available bandwidth.
By comparison, Global Hawk requires over three times the bandwidth of the Predator
(Varga 2003, p. 72).
While the rst two reasons are philosophical and essentially immutable, the hardware
reasons may be partially eased in the future with the use of ad hoc networks formed
by mounting Joint Tactical Radio Systems (JTRS) to aerial vehicles (U.S. Government
Accountability Oce 2005; Osborn 2011). Using this method, communication ranges of
60 km have been quoted using radio data transmission which could allow for eective
swarm wide communication.
6
In heterogeneous systems there are multiple agent-roles which can either be statically or dynamically
assigned based on environmental information (McLurkin and Yamins 2005).
7
Authors emphasis.
9
1.5. Summary
Bushres are potentially disastrous events which occur around the world, and are es-
pecially problematic in the Australian bush. Each bushre has the potential to cause
serious loss of life and property damage if it remains unchecked and spreads out of con-
trol. One of the key issues in bushre management is in both the initial reconnaissance
of the event, and in monitoring the res extent if it becomes necessary to deploy re
ghting personnel to the scene.
Due to the remoteness and inaccessibility of the terrain where bushres usually begin,
small and rapidly deployable unmanned aerial vehicles (UAV) have been suggested as a
possible solution to the problem of re surveillance. Studies in using a single remotely
controlled UAV have provided promising results, though there were coverage problems
found with only having a single plane airborne. These limitations could be overcome
by using multiple vehicles, but then the problem becomes one of having enough skilled
operators on the ground to y them all.
Multiple UAVs can be made practical if a mechanism exists for the vehicles to coordi-
nate their actions autonomously as a group instead of individually. The eld of swarm
intelligence deals with systems of multiple agents which can collaborate on tasks without
need for a centralised controller or group-wide coordination. Inspiration for these sys-
tems, and name of the eld, is from animal swarms found in nature such as ant colonies
or ocks of birds.
The problem addressed in this thesis is that of coordinating multiple UAVs to provide
persistent surveillance for the purpose of detecting and then monitoring bushres in an
area without the infrastructure needed to provide long range communication. Lack of
long range communication necessitates that a solution be decentralised. The solution
must also take into account the loss of individual planes due to the inherent dangers of
overying a bushre.
This dissertation begins by examining the current state of the art regarding multiple
robot systems in Chapter 2, followed by introducing an original contribution to the
eld, in the form of an algorithm proposed in Chapter 3, which aims to solve the stated
problem. The capacity of the algorithm to perform area surveillance is comprehensively
tested via simulation, using a simulator design discussed in Chapter 4, against theoretical
maximum performance goals in Chapters 5 and 6. Chapter 7 analyses the proposed
algorithms ability to monitor bushre events which are in progress. Chapter 8 discussed
how dierent communication constraints aect swarm performance, and then general
10
performance is compared against an existing eld-leading swarm surveillance algorithm
in Chapter 9.
11
2. Existing Work
Currently, two major branches of swarm robotic research are behaviour design and com-
munication, both of which are heavily inuenced by biologically inspired collective intelli-
gence (Bayindir and Sahin 2007). It was recognised early on that behaviour design using
traditional methods when dealing with groups of autonomous agents was a dicult prob-
lem that can be made tractable to the simple and mathematically elegant approaches
found in nature (Yamaguchi and Arai 1994). The related challenge in implementing this
type of behaviour design lies in developing a suitable medium for interaction between
elements and in deriving the appropriate modes of information exchange (Payton et al.
2005, p. 1).
2.1. Potential Fields
Two of the most common techniques for communication between vehicles in swarm
robotics are articial pheromone and articial potential elds (Parunak et al. 2002a).
The articial potential eld, rst applied in the eld of robotics in the mid 1980s (Hogan
1984; Khatib 1986; Krogh 1984), is a temporary structure created over an analogical
representation of the world. The structure consists of vector elds which can either
attract or repell robot movement (Steels 1993, p. 47). Compared to earlier work which
required movement to be preplanned inside pre-mapped static environments, potential
elds allowed robots to autonomously plan paths in real time even inside dynamic en-
vironments (Freund and Hoyer 1988). While their initial design was mathematically
inspired, the technique was found to be useful in emulating biological concepts.
The main use of articial potential elds, as with pheromone based algorithms, is
pathnding. Navigation of dynamic environments is made possible by assigning repul-
sion vectors from the edges of found objects and attraction vectors at target destinations.
Providing the path does not get stuck at a local optimum,
1
robots can generate plans on-
line (i.e. reactively) by iteratively summing surrounding vectors and performing gradient
1
There exist many common methods for avoiding this pitfall.
13
Chapter 2: Existing Work
descent. The similarities between potential elds and pheromone can be seen in work
done by Mei et al. (2006) where a combination of pheromone
2
and articial potential
elds were used for global and local pathnding, respectively.
A contemporary example of pathnding with potential elds is in communication re-
lays. In communication relay problems, groups of heterogenous robots are required to
form communication relays between a ground station and some target receiver, opti-
mising point to point data throughput (Horner 2004; Deok-Jin and Richard 2010). By
utilising various gradient descent methods and heuristics, a stable relay emerges in a
distributed fashion with minimal computational overhead. It is in problems of this na-
ture, where robots move from their initial location to a nal environmental equilibrium
state, that potential elds excel. When the environment changes, such as through a
robot becoming non-functional or a receiver moving, the individual robots simply fall
into the new equilibrium.
2.2. Pheromone
There are two prevailing methods of representing pheromone within pheromone based
systems: pheromone maps, where pheromone originates from the environment, and direct
communication, where pheromone originates from swarm members to form gradients.
Of the two, the method seen most often in robot swarms that have been physically
assembled (as opposed to simulated) has individual swarm members using peer-to-peer
communication to transmit pheromone values which represent signal strength at their
current position. The robot who initiates these messages usually does so in response
to stimulus, such as target detection or boundary discovery. By having peers, which
receive this data, propagate the message at decreased strength (in relation to distance),
potential elds can be built up without any prior knowledge of the size or shape of the
environment. In eect, each robot becomes a mobile waypoint or node in a dynamic
map. The term pheromone robotics is sometimes used when combining swarm robotics
and this type of pheromone communication (Payton et al. 2001).
With pheromone map based systems an internal digital pheromone map is used to
both represent the environmental knowledge of each individual agent and as the main,
or only, means of communication between peers. The specic implementation of the
pheromone map varies, but can be summarised as overlaying a geographic area with
a digital lattice grid and storing pheromone data at the vertices, referred to as cells or
2
Mei et al. (2006) used a form of Ant Colony Optimisation (ACO).
14
Figure 2.1.: Pheromone maps for swarm robotics overlay a digital grid over the
environment.
nodes, as visualised in Figure 2.1. Communication is usually in the form of indiscriminate
broadcasts of information, ranging from simple visible light signals, to wireless transmis-
sion of pheromone maps or pheromone values, to real chemical pheromone (Ducatelle
et al. 2009; Purnamadjaja and Russell 2005a).
In traditional pheromone maps node values are initialised with zero pheromone and
then digital agents are placed representing areas of interest (AOI) (Parunak et al. 2002b;
Sauter et al. 2005). These AOI agents pump Interest pheromone into the environment,
which diuses into neighbouring cells, creating a gradient which can be ascended to
locate the source.
3
When a node is visited by a UAV, all pheromone is removed from
that cell; when an AOI is visited it stops producing pheromone. This type of algorithm
can be further improved by including deterrent pheromones Threat and Repulsion
(Walter et al. 2006). Algorithms of this nature are sometimes referred to as lawn
mowing algorithms due to the similarities of an agent removing pheromone with the act
of cutting a lawn with a lawnmower (Arkin et al. 2000).
Repulsion pheromone can be added to the location of AOI agents which have been
recently visited to discourage subsequent visits in the short term. It can also be added
to the physical location of each agent to discourage convergence. Threat pheromone is
placed at areas which are actively dangerous, such as directly over a restorm or around,
in the case of military applications, the location of surface-to-air missiles. In all cases,
the pheromone placed slowly evaporates over time the same as in the biological model,
and for the same reasons as in the standard ACO model (forgetfulness, decreasing local
order, etc.) (Parunak 1997).
3
These implementations are similar to inuence map algorithms, an AI technique traditionally used
in strategic computer games (Millington 2006).
15
Dasgupta (2006) discusses algorithms for distributed automatic target recognition
(ATR). The proposed application is one where a swarm is performing surveillance and the
task of target identication is improved by using multiple platforms to take and compare
images of a target from dierent angles. As is the trend with swarm applications, a
pheromone based implementation is used with individual pheromone maps
4
distributed
via stigmergic broadcasts. Initial target detection is via a simple dispersion algorithm,
but once target information is known a mixture of attractive (at known target locations)
and repulsive (at previous target locations) pheromone is coupled with a hill climbing
algorithm to saturate target locations with agents (Chen et al. 2007). Good results are
shown even with simulated environmental eects which add noise to the images.
Area coverage using pheromone essentially creates an optimisation landscape, and
as such it can be appropriate to use optimisation terminology when describing the be-
haviour of algorithms in these environments. Specically, the problem with heuristics
based on diusion is that agents can become stuck in local minima, the diusion and
evaporation rates need to be precisely calibrated to minimise wandering (usually using
an oine method) and, most signicantly, they cannot guarantee exhaustive coverage
(Erignac 2007). A way of getting around these issues is by taking a less literal inter-
pretation of nature and using raw Euclidean distance to cells that need to be observed,
rather than pheromone diusion and evaporation. Using this method, cells are either
explored or unexplored, with explored cells containing the Euclidean distance to the
closest unexplored cell. The heuristic presented by Erignac (2007) is essentially a greedy
hill descent method, but using a representation of the environment more similar to a
potential eld than one strictly pheromone based. If there is an adjacent unexplored
cell, move to it; if all adjacent cells are explored, move into the one that has the lowest
distance to an unexplored cell.
While re surveillance requires a persistent presence, the algorithms described so far
are primarily designed to perform a discrete search, where the mission is considered
complete once the environment has been fully searched and targets have been located.
The standard way of modifying these approaches to accommodate continuous search
is to switch nodes from an inert/explored state to an active/unexplored state after an
arbitrary period (Altshuler et al. 2005). The critical problem with this approach is that if
the interval is too short then exhaustive coverage is not guaranteed: nodes will reactivate
for their second pass before all nodes have been seen on their rst pass. Alternatively,
if the interval is too long, then the environment becomes very sparse and the swarm
4
Dasgupta (2006) refers to these pheromone maps as pheromone landscapes.
16
takes straight, long distance routes to newly active nodes instead of eciently winding
through local nodes.
An additional problem lies in the distinction between monitoring a known re which
is in progress, and detecting new res. While long term surveillance for new res is a
problem which can be calibrated for known variables (size of the environment, num-
ber of active UAVs, etc.), transitioning into active re surveillance means the problem
space becomes highly dynamic. Pheromone dispersion algorithms can transition be-
tween surveillance and monitoring through the addition of articial point of interest
(POI) agents, however algorithms which require a priori target knowledge will be lim-
ited in their usefulness for this particular domain (Flint et al. 2002).
2.3. Realised Swarm Robotics
2.3.1. Beckers
Beckers (1994) was one of the rst projects to take the theoretical idea of stigmergic
robots and build a real swarm. The swarm of ve robots were equipped with grippers
for picking up pucks, and two IR sensors. In his work, the behaviour of termite nest
building was replicated, where, without direct communication, the swarm was able to
collect a scattered collection of objects and stack them together. This was achieved with
purely stigmergic communication (moving the pucks) and a very small set of simple rules
encoded into a nite state machine (FSM).
2.3.2. Pherobots
Another swarm robotic system used for research purposes is the Pherobot swarm, de-
signed by Payton et al. (2005). Pherobots use an infrared communications ring that is
used for both navigation as well as communication. One interesting advantage of in-
frared communication is that it approximates real world pheromone, in that its strength
decreases over distance providing a natural gradient for peer robots to interact with.
With Pherobots, the pheromone analogue used is peer-to-peer messaging which in-
cludes hop count, pheromone type, and a multi-purpose data eld. Typically when a
Pherobot receives a pheromone message, it increases the hop count and resends the
message in all directions. In the (almost guaranteed) event of multiple pheromone mes-
sages being received, only the strongest signal is propagated. In this way, each Pherobot
acts as a node in a pheromone map. A typical application of this approach is robotic
17
mapping, where the swarm can spread incrementally and individual agents can maintain
position indenitely.
2.3.3. I-SWARM
I-SWARM was a project funded by the European Commission to build a large scale
robot swarm of up to 1,000 vehicles (Seyfried et al. 2005). Using optical communication,
early results of this project allowed for multiple types of pheromone messages to be
sent optically via LEDs to enable dynamic task allocation (Woern et al. 2006). The
inspiration for this particular robot Swarm was collective perception in honeybees:
By evaluating trophallactic
5
contacts forager bees can indirectly assess the current ratio
of brood demand to pollen supply in the colony without inspecting brood area and pollen
stores individually (Camazine 1993; Schmickl and Crailsheim 2004).
Simulating the I-SWARM using the LaRoSim (Large Robotswarm Simulator) plat-
form, Schmickl et al. (2007) are able to utilise collective perception to measure and
compare the sizes of two distant targets without long distance communication or sens-
ing (Valdastri et al. 2006). As with the previously mentioned Pherobots, communication
is via peer-to-peer communication which incorporates hop count data, where only the
strongest signal is propagated in the case of multiple messages. Similar approaches have
also been used with this system to perform pathnding and lawn-cutting algorithms
(Schmickl and Crailsheim 2006; Valdastri et al. 2006).
6
2.3.4. iRobot SwarmBots
The SwarmBot is a four-wheeled, 12 cm cube with spatial location capability, infrared
and radio communication incorporating unique ID chips (McLurkin 2008). Primarily
used by McLurkin and Smith (2004) in work on distributed algorithms, applications
have ranged from simple swarm dispersal behaviour to complex environmental bound-
ary detection by using the swarm as a mobile sensor network (McLurkin and Demaine
2009). Work by Shaw et al. (2010) used the SwarmBot platform as a test case to show
the eciency of the input-based consensus algorithm in addressing the problem of com-
munication failure in collective perception.
While Pherobots were designed with the view that coordination schemes that require
unique identities for each robot [and] explicit routing of point-to-point communication
5
Trophallaxis is the transfer of uid food among members of a colony through orice to mouth feeding.
6
In I-SWARM publications the synonym collective oor cleaning is used for lawn-cutting, and optimal
route nding for pathnding.
18
between robots [...] can be overwhelmed when dealing with extremely large numbers
(Payton et al. 2003, p. 1), SwarmBots have successfully been deployed under these
constraints with swarm sizes exceeding 100 vehicles without any reported diculties
(McLurkin 2004).
2.3.5. PheGMot
The pheromone-guided mobile-robot (PheGMot-III), designed by Kuwana et al. (1996),
simulates the behaviour of the male silkworm moth by attaching living moth antennae
to robots for the purpose of detecting pheromone. The resulting cybernetic organism
was able to achieve target location in a wind tunnel by gradient descent sensing of real
world chemical trails (Kuwana et al. 1999) .
Similar robots have been designed by Russell (2009), who experiments with using real
world pheromone to coordinate robot movement. This work is an extension of earlier
work, conducted in collaboration with Purnamadjaja and Russell (2005a), in building
robots which could simulate the necrophoric behaviour of bees.
7
In other work by Pur-
namadjaja and Russell (2005b, 2007), robots with gas sensors and chemical fans where
able to take on queen and worker roles, where the queen could indirectly coordinate
the actions of the worker swarm.
Sugawara et al. (2004), using the virtual dynamic environment for autonomous robots
(V-DEAR), experimented with using a less literal approach to real pheromone. In his
work, a projector is used to display trails where robots have passed, and these markings
are in turn observed by the swarm and used to prompt behaviours such as recruitment
and searching.
2.4. Implementation Issues with UAV Swarms
One of the key development challenges in creating a functional UAV swarm is the lim-
ited processing capabilities that can be carried on board. As endurance is a critical
requirement of UAVs, power usage, weight and heat generation all need to be kept to a
minimum. These constraints limit the computational hardware that can be carried. One
suggested method for combating this is the use of eld-programmable gate arrays (FP-
GAs), which have been used in low-level electronic applications for many years but only
in the last decade reached a level on complexity where their use as processing devices
7
Necrophoric behaviour is a pattern where dead members of colonies are transported by live members
of the same colony.
19
in their own right has been seriously considered (Sanderson 2003). A modern FPGA,
with embedded processors, RAM, and serial IO, has a superior processing performance
to power consumption ratio when compared to a conventional processors.
Extending these advantages, sharing the FPGA computing and power resources of
individual UAVs across the swarm has been shown to be a possibility (Kearney and
Jasiunas 2007). With this distributed computing approach, computationally intensive
tasks can be broken up and executed by other agents in the swarm individually or
in parallel. To implement this system in practice requires the creation of a purpose
built operating system to handle the migration of tasks between UAVs without undue
eort required by application developers (Jasiunas 2009), as well as robust intra-swarm
communication. However, the benet of allowing a swarm to pool its resources, and
overcome the individually weak computational power of each agent, may make this a
desirable, and maybe even necessary, eld of inquiry.
There has also been some work done in using a graphics processing unit (GPU) to
perform the pheromone calculations of a UAV swarm. Depending on the algorithm used,
pheromone updates can be the most computationally expensive subsystem the UAV has
to deal with outside of on-board image recognition. Using a GPU to run the calculations
shows an order of magnitude improvement over similar CPU code. However the current
main use of this is for robot simulation software, where swarm sizes could be measured
in tens of thousands (Walter et al. 2006).
2.4.1. Spatial Awareness
Identifying an agents location in its environment is one of the fundamental challenges
in swarm robotics. The most common and practical method of determining position
for outdoor UAVs is through the use of the Global Positioning System (GPS), with the
only real drawback being relatively low precision (within approximately 20 meters for
civilian applications) (Stirling et al. 2010). The main alternative is an inertial navigation
system (INS), a dead reckoning process where current location can be calculated based
on a previously determined location and subsequent distance travelled. The drawback
to an INS is that a method of ascertaining initial positions is still required, and it is
often necessary to have a shared frame of reference throughout the swarm to make
communicated data useful. In known environments (usually an indoor testing facility)
research has been done in determining a UAVs position using visual input alone using
markers in the environment, though it is theorised that this could be extended to using
known landmarks in an outdoor search environment (Yamada et al. 2003).
20
While largely impractical for the task of monitoring bushres, an alternative to calcu-
lating spatial awareness on-board is delegating the task to the environment itself using
sensor networks (Batalin et al. 2003). In this scenario, the environment contains a sensor
network that acts as a series of virtual signposts with which agents align themselves.
Navigation directions can then be ooaded from the robots to the external network.
The distributed sensor network itself can determine spatial coordinates based on the
node location in the network topology (McLurkin 1999). McLurkin and Smith (2004)
show how pheromone communication can be used to produce diusion gradients, with
peer-to-peer communication incorporating hop count used to achieve this in a manner
very similar to pheromone robotics. Intanagonwiwat et al. (2000) used a similar method
of peer-to-peer messaging and diusion with distributed sensor networks, calculating
ecient paths for information ow.
2.4.2. Control
Researchers have looked at ways in which control can be exerted on a swarm once it is
deployed. While a swarm will have inherent behaviour, a human operator providing high
level direction or objectives could be benecial in some areas (Parunak 2003). One way
of enabling this work is through the development of programming languages which cater
to the specics of swarm computing. Swarm is an early domain specic programming
language in which a hierarchical structure is used for agents. In Swarm, an agent can
consist of a swarm of other agents (Minar et al. 1996). Similarly, LISP based work
named Protoswarm considers a swarm as a single computer occupying an environment
space and calculations are performed by individual agents inside the area with limited
local interaction (Bachrach et al. 2008). These approaches have not progressed past
the stage of using potential elds, and are not yet well suited for doing complex swarm
behaviours.
2.5. Summary
Swarm robotics is a growing eld with many applications in both the civilian and mil-
itary spheres. The advantages of deploying robots as a swarm instead of a centrally
controlled system include massive scalability, low communication overheads, reduced
need for human supervision, and resilience against individual failure. Algorithms which
are designed to work in a swarm environment are still in their infancy, yet a promising
21
branch of inquiry has been biomimicry, specically the use of articial pheromone.
Using pheromone maps to represent the environment a swarm occupies allows algo-
rithms to combine the information sharing and collective decision making elements of
swarm behaviour into a single intuitive concept. The problem with pheromone as a
concept, however, is that the analogy only holds so long as the information is stored in
the environment itself. With the problem of detecting and then monitoring bushres in
an area without the infrastructure needed to provide long range communication there is
no immediately apparent medium to store this information.
The next chapter will present an algorithm which solves this problem by storing a
unique pheromone map within each individual agent and then broadcasting the phero-
mone map locally as a means of synchronising with other agents internal model.
22
3. Algorithm Implementation
The goal of the presented algorithm is to enable a swarm of UAVs to cooperatively
engage in aerial surveillance, primarily for bushres, while being resilient to vehicle loss
and under the constrain of utilising only short range communication. Areas of interest
should be speciable, so that high risk areas can be given increased levels of attention.
The proposed algorithm uses a pheromone map, where pheromone is used to quantify
the need for a survey of a particular location: the higher the pheromone level the greater
the need. The map consists of a grid graph whose vertices correspond to virtual nodes
with a pheromone value and environmental coordinate. Pheromone increases automati-
cally over time by an amount that is proportional to the required survey frequency, and
is reset to zero when a survey is made. Survey frequency (i.e. priority) of a location can
be set a priori or changed in response to environmental observations depending on the
mission. UAVs are attracted to the node most due for survey while taking into account
distance from their current location.
Each UAV maintains an individual pheromone map, resetting the pheromone on each
node it visits. This map information is broadcast periodically, allowing any nearby
UAVs within range of the transmission to synchronise the pheromone level of all nodes
on their own pheromone map to the lower of their current or received value. This enables
a UAV to propagate its environmental knowledge to surrounding agents, using xed-size
storage, without loss of any information relevant to decision making.
To maintain swarm diversity, UAVs in close proximity are pulled away from each
other if their location becomes known via a broadcast, discouraging multiple UAVs from
surveying the same point shortly after each other. The net eect of this is that the
UAVs spread out rather than converge, surveying locations with the highest pheromone
levels preferentially so that all areas on the map are surveyed as frequently as possible
and in the required frequency ratio.
23
Chapter 3: Algorithm Implementation
3.1. Pheromone Maps
The algorithms cornerstone is the pheromone map, a grid graph of nodes consisting of
positional data, pheromone quantity, and priority level.
1
A literal implementation of
this pheromone map would store pheromone as a real value and periodically update it.
A more ecient implementation, and the one used by the algorithm, is a time-priority
product based pheromone map (Howden 2009). In this design the stored pheromone
level is replaced with the time it was last surveyed, initialised to the time at which the
surveillance mission began. By multiplying the elapsed time since the last survey with
the priority of that node, the quantity of pheromone can be calculated. This minor, but
important, substitution is show in Equation (3.1) where the quantity of pheromone at
node n is the product of the nodes priority and time unobserved (i.e. the nodes last
visit time subtracted from the current time). Pheromone quantity is inherently reset
when surveyed by a UAV as a result of time becoming zero.
2
pheromone
n
= priority
n
time (3.1)
A literal pheromone map, that stores actual pheromone levels, requires both matrices
(priority and pheromone) to be completely traversed at each time step to increment the
stored values. The alternative, implementing the pheromone map in the more abstract
time-priority representation, reduces performance overhead by avoiding this continuous
periodic incrementation in favour of calculated values only when a new waypoint needs
to be selected, and only for nodes under consideration. Storing survey times instead of
pheromone directly has the additional advantages of:
Tracking survey times directly is conceptually elegant.
The pheromone map does not need to be updated to the current time stamp before
broadcasting.
Updating the pheromone map does not require synchronisation of data before
comparisons: timestamps are simply compared and the most recent is stored.
This has the added benet of allowing survey data (pheromone maps) to be shared
between UAVs with dierent priority maps if required.
The pheromone level of nodes is calculated as few times as possible, on demand,
without extra storage overhead.
1
The terms cell and node are interchangeable synonyms in the context of this, and related, work.
2
It is assumed that agents, such as UAVs, have time synchronisation (e.g. through GPS time).
24
The pheromone map serves as both the UAVs primary environmental representation
and as their means of communicating with nearby agents. Agents could include other
UAVs in the swarm, the re managers base of operation, and deployed re teams via
portable computing devices. While the necessary communication protocols for data
transmission are beyond the scope of this thesis, the assumed capabilities are minimal.
It is assumed that the UAVs will be capable of a line of sight, short range (at least
one kilometre) broadcast of their pheromone map and position, at a frequency of at
least once per 10 minutes. Similarly, it is assumed that agents are able to intercept
broadcasts of pheromone maps by other agents at a similar rate. No handshake protocol
is assumed, with the algorithm designed so UAVs are not required to request information,
or maintain awareness of any agent that is not actively and currently broadcasting its
own pheromone map.
3.2. Implementation Details
This work is primarily focused on the manipulation and evaluation of pheromone maps
for the purpose of coordinated area surveillance. This goal is achieved through gen-
erating waypoints which can be passed to a UAVs motion control routines (i.e. an
autopilot). While implementation of a full UAV control system is beyond the scope of
the proposed algorithm, a mock UAV control loop is presented in Algorithm 3.1 to help
provide context for the surveillance algorithm. In this control loop, the UAV monitors
for broadcasts coming from other agents in the function ReceiveBroadcasts, and stores
them for later use. When the UpdateWaypoint (Algorithm 3.2) function is called, the
list of newBroadcasts is passed through to be processed, as well as a list of all nodes
the UAV has surveyed since the algorithm was last called. The waypoint returned by
this function is used by the UAVs motion control routines as a target destination.
Algorithm 3.1 A mock UAV control loop, containing the core surveillance algorithm
function, UpdateWaypoint
1: function ControlLoop
2: while true do
3: newBroadcasts ReceiveBroadcasts()
4: observedNodes ProcessCameraData()
5: waypoint UpdateWaypoint(newBroadcasts, observedNodes)
6: MotionControl(waypoint)
In an actual implementation of the control loop of an autonomous UAV, such function-
25
ality would not be strictly sequential or even necessarily handled by the same hardware
systems. In particular, it can be assumed that the UAVs motion control will essen-
tially always be running, piloting a safe path towards the currently selected waypoint.
Concurrently, the UAV should be able to continuously monitor for, and subsequently
store, pheromone map broadcasts.
3
The UpdateWaypoint function itself can run asyn-
chronously as a low priority process, integrating the queue of received broadcasts into
the UAVs own pheromone map as time allows.
3.2.1. Update Waypoint
The surveillance algorithms primary function, UpdateWaypoint, is shown in Algo-
rithm 3.2. In this function, incoming data is integrated into the UAVs existing phero-
mone map, and then if neccersary the pheromone map is heuristically evaluated to
update the current waypoint, then the waypoint is returned to the UAV.
In the body of this function, rst any broadcast pheromone maps received since the
last update are used to update the UAVs own pheromone map with the MergeMaps
function (Algorithm 3.3). The position of the broadcasting agent is then passed to
StoreBroadcastDirection (Algorithm 3.4) for use in repulsion. After the UAVs phero-
mone map has been updated using broadcast data, nodes that the UAV has directly
observed have their survey time set to the current system time.
4
If any of the proceeding steps resulted in the waypoint nodes timestamp being
updated, either through direct or third party observation, the pheromone map is re-
evaluated for a new target. When called, EvaluatePheromoneMap iterates through
the nodes of the pheromone map and returns the one with the highest tness, as a
function of distance and pheromone level. Finally, if scheduled to do so, the UAV is
instructed by BroadcastMap to indiscriminately broadcast its own pheromone map and
position.
3.2.2. Merging Pheromone Maps
The function to merge pheromone maps is shown in Algorithm 3.3. MergeMaps is used
to selectively update the UAVs internal pheromone map with more recent information
3
Of note, Chapter 8 shows that lossless data transmission is not a requirement for cooperative surveil-
lance. The swarm is able to coordinate its actions even when periodically broadcasting and storing
pheromone maps consisting of randomised data.
4
Ideally the nodes timestamp (i.e. survey time) would be set to the time that the node was actually
observed, but representing this in the psuedocode example is cumbersome and the dierence, in
practice, would be minor.
26
Algorithm 3.2 UpdateWaypoint is the main update function for UAVs us-
ing the surveillance algorithm. Incoming data is integrated via MergeMaps,
StoreBroadcastDirection, and UpdatePheromoneMap. EvaluatePheromoneMap is
then called to calculate a new waypoint.
1: function UpdateWaypoint(newBroadcasts, observedNodes)
2: newBroadcasts getBroadcasts()
3: for each broadcast in newBroadcasts do
4: MergeMaps(broadcast.pheromoneMap)
5: StoreBroadcastDirection(broadcast.position)
6: for each node in observedNodes do
7: UpdatePheromoneMap(node)
8: if waypointNode.time was updated then
9: waypointNode EvaluatePheromoneMap()
10: if (currentTime lastBroadcastTime) > broadcastPeriod then
11: BroadcastMap()
12: return waypointNode
contained within a received, broadcast pheromone map. For each node, the timestamps
of the two pheromone maps are compared and the most recent one is stored. Addi-
tionally the presence of re at the most recent observation is stored (target status of
the most recent observation is kept). Depending on the exact scenario the UAVs are
deployed for, and the reliability of the onboard re detection algorithms, this step could
be implemented as setting the fireDetected ag to true if either map has its ag set.
Algorithm 3.3 Merging pheromone maps so that each nodes most recent observation
is retained.
1: function MergeMaps(broadcastPMap)
2: for i in range(nodeCount) do
3: if phermoneMap[i].time < broadcastPMap[i].time then
4: phermoneMap[i].time broadcastPMap[i].time
5: phermoneMap[i].fireDetected broadcastPMap[i].fireDetected
3.2.3. Storing the Direction of Incoming Broadcasts
The purpose of function StoreBroadcastDirection (Algorithm 3.4) is to determine the
position of broadcasting agents relative to the position of the UAV at the time of trans-
mission, and store this information as a directional vector for later use in maintaining
swarm diversity. The direction vector, direction, describes a line beginning at the po-
sition of a broadcasting agent, and ending at the UAVs own position. These direction
27
vectors are stored in a queue data structure, directionList, and are manipulated to cal-
culate a point of attraction away from other UAVs. The attraction point mechanism is
discussed in the following section and implemented in Algorithm 3.5.
Algorithm 3.4 StoreBroadcastDirection probabilistically stores the position of broad-
casting agent when communication occurs within repulsion range. V ector2 returns a
two-element vector representing oating point coordinates.
1: function StoreBroadcastDirection(broadcastPosition)
2: if Distance(position, broadcastPosition) < REPULSION RANGE then
3: if Random() < REPULSION CHANCE then
4: direction Vector2(position broadcastPosition)
5: direction Normalise(direction)
6: directionList.Push(direction)
In storing direction vectors, function StoreBroadcastDirection works in a proba-
bilistic manner. When called, the broadcast origin position is rst checked if it is
within a predened, constant repulsion range (REPULSION RANGE). If the broad-
casting agent is distant, no repulsion is required and the function terminates. If the
broadcast originates from inside the UAVs repulsion range, there is a random chance
(REPULSION CHANCE) that a direction vector is calculated and stored.
5
3.2.4. Attraction Points
While direction vectors are calculated and stored when broadcasts are received, the ac-
tual attraction point is calculated dynamically as needed in the function GetAttractionPoint
(Algorithm 3.5) using the UAVs current position as an origin. By taking the sum of
all stored vectors, an approximation of the centroid position of nearby swarm members
(relative to the UAV) is calculated. The directional data of this vector is then isolated
by normalising its magnitude, then multiplying by a xed constant.
6
The attraction
point is calculated by adding this vector to the UAVs current position (i.e. using the
UAVs position as the vectors origin). The function call removeExpiredDirections is
used to remove direction vectors that are older than a predened age, this aspect, and
attraction points in general, are further discussed and analysed in Section 3.3.
5
It was found experimentally that, for the UAV specications described in Chapter 4, a
REPULSION CHANCE of 20% and REPULSION RANGE of 1000 metres worked well over
the range of simulations described in the following chapters. In general when UAV density increases,
performance can be slightly improved by increasing these values, and vice versa.
6
Experimentally, it was found that an OFFSET DISTANCE of 1000 metres worked well, though
the exact value was not critical and had only a minor eect on performance.
28
Algorithm 3.5 GetAttractionPoint calculates a point of attraction by using the UAVs
position as the origin for the sum of the direction vectors contained in directionList.
1: function GetAttractionPoint()
2: attractionPoint Vector2(0, 0)
3: removeExpiredDirections(directionList)
4: for each direction in directionList do
5: attractionPoint attractionPoint +direction
6: attractionPoint Normalise(attractionPoint)
7: attractionPoint attractionPoint OFFSET DISTANCE
8: attractionPoint attractionPoint +this.position
9: return attractionPoint
3.2.5. Pheromone Map Evaluation
The function EvaluatePheromoneMap, shown in Algorithm 3.6, brings together the al-
gorithms various elements to perform a heuristic search of the pheromone map to select a
node to be the UAVs new waypoint. The search initialises by calling GetAttractionPoint
to calculating and return an attraction point.
7
The second step of the search is to check
that the node being evaluated is closer to the attraction point than the UAVs position
(Figure 3.1).
Algorithm 3.6 EvaluatePheromoneMap heuristically evaluates the UAVs pheromone
map for a destination waypoint using pheromone values and information derived from
previously known peer UAV locations (i.e. the attraction point).
1: function EvaluatePheromoneMap()
2: maxWeight 0
3: bestNode null
4: attractionPoint GetAttractionPoint()
5: for each node in pheramoneMap do
6: posDistance Distance(node.position, this.position)
7: apDistance Distance(node.position, attractionPoint)
8: if apDistance posDistance then
9: weight Pow(node.pheromone, 2)/(posDistance +apDistance)
10: if weight > maxWeight then
11: maxWeight weight
12: bestNode node
13: return bestNode
Providing the node is a valid target, weight is calculated as a tness function of
pheromone level over distance. Specically, weight is calculated as pheromone level
7
As can be seen in Algorithm 3.5, in the case of directionList being empty, the attraction point
defaults to the UAVs current position.
29
Figure 3.1.: a) UAV detects a broadcast, calculating and storing a direction vector in
response. b) The UAV calculates an attraction point based on its current
position and the stored direction vector. c) The UAV evaluates nodes which
are closer to the attraction point than its current position, all other nodes
are occluded from its search space.
squared, divided by the sum of distance between UAV and node, and distance between
node and attraction point. This heuristic is summarised in Equation 3.2, using the
distance function to represent the distance between two points, the UAV position p,
attraction point ap, and node n.
f(node) =
pheromone
2
node
distance(p, node) +distance(ap, node)
(3.2)
3.3. Diversity
An agents state consists of its environmental knowledge and its physical location. Multi-
ple homogeneous agents will perform no better than a single one if they share an identical
state. In the specic problem of continuous area coverage addressed by this dissertation,
diversity is the degree to which a swarm of agents is able to maintain disparate states,
especially when acting upon the same, or similar, environmental knowledge(Parunak
1997). In the specic case of the problem addressed by this thesis, continuous area cov-
erage, diversity is the ability of UAVs, sharing similar positions and pheromone maps,
to generate unique waypoints.
8
Being homogeneous, UAVs acting without diversity mechanisms will always be at-
tracted to the node with the best pheromone to distance value. When acting individu-
ally this is will tend to produce the best results,
9
however adding additional UAVs will
8
N.b. in the context of diversity with homogeneous agents, perfect environmental knowledge can
actually be detrimental. As state is the combination of position and knowledge, the more similar
knowledge is, the more dissimilar position needs to be to produce unique decisions.
9
Chapter 5, Performance Baselines, describes metrics for what good results entails: in summary the
30
be of decreasing benet as the swarm will converge around mutually recognised best
areas, rather than spreading out to cover as many nodes as possible. To reduce or pre-
vent convergence and maximise the ability of the group to perform area surveillance,
it is important to utilise tools such as repulsion (separating UAVs that are too close
together) and randomness (adding an element of non-determinism).
As described in the previous section, the implemented repulsion mechanism utilises
attraction points. This method anticipates a point in the environment perceived to have
a low agent density and then increases the heuristic weight of nodes in the vicinity. This
attraction point is calculated using a historical list of directional vectors which describe
the relative position of the UAV to broadcasting agents at the time of their broadcast.
Each individual vector is stored for a period of time equal to the predened broadcast
period before being discarded. As was shown in Algorithm 3.5, the direction vectors are
summed, set to a constant magnitude, and then added to the UAVs current position.
This information is then utilised by the node evaluation heuristic to select waypoints of
the UAV.
Attraction point repulsion was designed to take into account two somewhat unusual
properties of the problem domain. Firstly, that the position of peers is only known
indirectly, through infrequent snapshots of the environment. Secondly, the swarms goal
is to perform repeated, exhaustive searches of the environment as quickly as possible,
so it is benecial for UAVs under the inuence of repulsion to still be performing useful
movements.
Lack of knowledge about the exact location of nearby UAVs is dealt with through the
memory of relative broadcast directions. Upon exchanging broadcasts at close range,
two or more UAVs will have nearly identical states (position and knowledge) if repulsion
is not implemented. Further, repulsion of an insucient duration, such as to the UAVs
next decision, will not be sucient to overcome the attraction to nearby high pheromone
nodes. The result of this would be for the group of similar-state UAVs to redundantly
overy the same nodes in short succession. This behaviour can only be amplied by the
potentially long broadcast period, which leaves UAVs unaware that their environmental
knowledge is out of date.
Temporary memory of relative broadcast direction is used rather than broadcast po-
sition again due to the period between positional updates. If the similar-state group
of UAVs were in close proximity during a previous broadcast and continued together in
the same general direction, they will both experience repulsion in the same direction,
performance of a surveillance algorithm can be evaluated as the average time nodes go unobserved.
31
negating any benecial eect. This can be visualised by imagining two UAVs broad-
casting at the same time: initially, using broadcast positions, the UAVs will experience
repulsion directly away from each other. If they select similar waypoints (not unlikely,
considering their now similar states), their movement in the same direction will cause
their repulsion directions to become increasingly aligned. By storing the initial vectors,
repulsion stays aimed in separate directions and also applied to successive movements,
helping break apart the joined pair with articial psuedo-hetrogenity.
The second property of the problem, the desire to have UAV movements made while
under the eect of repulsion be as productive as possible, is primarily facilitated by the
use of attraction points that change the weight of nodes rather than directly manipu-
lating the UAVs movement. It should be noted that repulsion itself is an inherently
sub-optimal process at an individual level. The goal of diversity mechanisms is to pro-
mote group tness at the expensive of the individuals. In the instance of this algorithm,
group performance is improved by guiding the individual UAV to select waypoints which
may have lower pheromone levels or be further away than would otherwise be most ef-
cient.
Attraction points achieve these design goals in a number of ways. Firstly, and most
severely, is search space occlusion. Nodes which are closer to the UAV than the attraction
point are not evaluated during waypoint selection. In eect this draws an occluding line
perpendicular to the midpoint between the UAV and attraction point (Figure 3.1). Using
a constant magnitude for the sum of direction vectors simplies the calculation of the
occlusion line, and also keeps the distance penalty relevant to nearby nodes.
10
Within the
subset of the environment that a UAV under the eect of repulsion is allowed to evaluate,
diversity is further promoted by the node weights that the UAV is evaluating being
slightly varied to other nearby agents due to the UAVs uniquely generated attraction
point.
Combined, these two factors of attraction points, occlusion and tness landscape mod-
ication, are very powerful. While one or the other would be sucient for two agent
problems, taken together the occlusion mechanic can forcibly break up large groups of
UAVs into largely disparate sections of the environment, while the node weight mod-
ication helps keep UAVs that arent aware of each others presence, or are otherwise
evaluating the same area, from selecting the exact same series of waypoints. This is
achieved while still leaving the pheromone over distance heuristic as the sole arbiter of
10
When dividing pheromone by the sum of two distance-to-point values, all nodes in a line between the
two points have an equal, minimal weight.
32
waypoint selection.
The nal piece of the repulsion mechanic is the addition of randomness. Attraction
points are strong enough that if a pair of UAVs meet only one needs to come under the
eects of repulsion for both to maintain separate states. When broadcasts are received
by an agent, there is a stochastically determined chance that repulsion information will
not be kept in memory, even if the broadcasting agent is within repulsion range. This
has the eect of allowing some UAVs to perform individually optimal movements, while
still preventing extended convergence.
3.4. Summary
The algorithm proposed in this dissertation uses a pheromone map to represent envi-
ronment knowledge and as the medium of communication. UAVs perform exhaustive
surveillance by iteratively evaluating their individual pheromone map using a distance
based heuristic to select destination waypoints. Convergence is avoided through use of
a novel diversity mechanic: attraction points. Using attraction points, homogeneous
agents are able to be repelled from each other while still maintaining the ability to
perform eective surveillance. The ecacy of this method is shown in subsequent Chap-
ters where it is demonstrated that UAVs maintain their individual eectiveness against
increases in swarm size.
33
4. Simulation Setup
The proposed algorithm is evaluated by simulating swarms of UAVs. This approach is
also used to explore and describe properties of the algorithm by varying parameters and
environments. To this end, this chapter describes the most important elements of the
simulator as well as the decisions and assumptions behind them.
4.1. Environmental Representation and Perception
Agents perceive the environment as a pheromone map in the form of a lattice graph, with
quantities of pheromone accumulating at the nodes. Node arrangement is in a square
grid pattern to streamline map storage and representation, and evaluation functions. In
a real world deployment a hexagonal representation would allow for a slight increase in
mission performance at the cost of an increase in code complexity.
The agents can sense their own position in the simulated environment as a Cartesian
coordinate via GPS. Positions of peer agents are sensed indirectly through Cartesian
coordinates attached to periodic information broadcasts. It is assumed that the agents
have additional sensors required to handle emergency collision avoidance with other
objects, as this is outside the scope of the algorithm presented.
Fire detection is done through use of a low power visible CMOS camera (Kearney
and Jasiunas 2007). When a possible target is identied the UAV switches to a high
powered thermographic camera, also known as a forward looking infrared (FLIR) camera.
The cameras specications are based on current and common o-the-shelf components:
640480 resolution, eld of view of 60
and optical 10zoom. Using these specications,

and adding a requirement for high resolution 5 cm
2
pixel clarity, the UAV has a 176m
footprint radius when using the FLIR camera at an altitude of 305 metres.
Speed of search is a factor of UAV speed, and the specications of the cameras used:
a larger footprint allows UAVs to travel a shorter distance to attain coverage. For
this reason, along with the swarm principle of individuals being simple and cheap, the
specications of the camera are consciously conservative.
35
Chapter 4: Simulation Setup
Figure 4.1.: Calculation of optimal node separation(NS). Camera radius (CR) is the
radius of the ground area that the UAV can see at any point in time, while
arrival radius (AR) is how close the UAV must be to a node before it counts
as being surveyed. Node separation is calculated based on these two numbers
to ensure that camera coverage is complete once imperfect UAV navigation
is taken into account.
4.2. Node Spacing
To guarantee complete coverage of the environment with minimal overlap, the node
separation must be matched to the UAVs physical specications. Node separation (NS)
is calculated as a function of the radius of the cameras footprint (CR) and an arrival
radius (AR) of the UAV, which indicates that a UAV is acceptably close to the desired
location. The arrival radius is necessary as it is unrealistic to assume that a UAV will
be able to constantly perform perfect yovers. Using these variables the optimal node
separation can be calculated using Equation (4.1).
NS = (
2)(CR AR) (4.1)

As can be seen in Figure 4.1, the amount of overlap is tied to the value of the arrival
radius. In a real world deployment, this would be a key variable to optimise based on
performance of the UAVs used.
Another potential improvement which was briey mentioned in the chapter introduc-
tion is to switch to a hexagonal grid over the default square one. This would give a small
improvement on overlap between areas, albeit at a slight cost to conceptual neatness and
36
ease of implementation and calculation speed.
4.3. UAV Specications
The algorithm is primarily designed with swarms of miniature UAVs (MAVs) in mind,
however the current state of the art sees UAVs in the three metre wingspan category
as the most practical choice presently, thus the variables reect a compromise between
the two. A comparison of currently deployed UAV systems is shown in Appendix B.
There is a correlation between vehicle dimensions and performance, with larger planes
able to travel faster and stay aloft for longer periods of time. The specications for a
UAV to be simulated were chosen by projecting this trend downwards towards smaller
UAVs (Miller 2006).
The theoretical MAV used in simulations has an average cruising speed of 35 knots and
ies at an altitude of 305 metres. This specic altitude is required for the thermographic
camera described in Section 4.1 to reach its footprint and resolution goal. UAV size is
not specically simulated, but is assumed to be below two metres in wingspan and
economical to mass produce and thus y as a swarm. In re simulations, a second,
larger, theoretical UAV is also used, with a three metre wingspan and a maximum speed
of 105 knots. As refuelling is not normally simulated, UAVs by default have unlimited
endurance.
1
UAVs broadcast their pheromone map once per 60 seconds via radio modems with
3 km line of sight radius. It has been shown in earlier work that this type of algorithm
maintains functionality even with much longer broadcast periods and malfunctioning
agents broadcasting random information (Howden and Hendtlass 2008). Simple o-the-
shelf parts are sucient for peer-to-peer communication as dropped connections and
latency are not a major issue.
4.4. Simulator Limitations: Granularity and Abstractions
The simulation has an update resolution of one second intervals. This aects all functions
which would usually be continuous, e.g. movement and UAV logic calculations.
Turning circles, acceleration, heading and atmospheric conditions (e.g. wind speed)
are not expressly simulated as accurate ight physics is outside the scope of the work,
1
However, as will be shown later, a UAV can leave the swarm, for example to refuel, without signi-
cantly degrading the swarms performance.
37
and is thus loosely approximated by a model which uses a single force vector. As the
algorithm is designed as one which accepts input in the form of a pheromone map and
coordinates, and outputs desired destination coordinates into a UAVs autopilot system
this is simply a matter of calculating a vector between the UAVs position and the
current waypoint and adding this to the UAVs current velocity, capped at the UAVs
cruising speed.
This is another reason for the conservative cruising speed selected in the preceding
section, a concession to the inherent performance increase in a UAV with a near perfect
turning radius. Another factor is that randomness injected by wind and other natural
phenomena, which have not been simulated, are not solely negative factors in perfor-
mance. Randomness is benecial in injecting diversity into the swarm (see Section 3.2.1),
mitigating the required strength of whichever repulsion mechanism is used.
UAV recovery is not simulated, while UAV launch is only simulated for re surveillance
scenarios. When not explicitly stated otherwise, UAVs start the simulation randomly
dispersed throughout the environment and data is not gathered until the entire environ-
ment has been seen at least once.
2
This also helps to decrease noise in the simulation
results and means usable data can be generated in less time. In all other regards, this
manipulation of data has no net eect on generated results as the algorithm reaches an
equilibrium after the entire environment has been searched at least once regardless, as
will be shown in Section 5.3.
4.5. Fire Simulation
Accurate simulation of bushres is a complex and ongoing eld of research, and fully
replicating state of the art results is outside the scope of the dissertation. To provide a
framework for the testing of the algorithm on tracking a spreading target (Section 7.2),
such as a bushre, an approximated worst case scenario is used as a proxy (Howden
2013).
The anatomy of a bushre is traditionally broken up into three distinct sections relative
to wind direction as shown in Figure 4.2. Heading res are aligned with the mean wind
direction, resulting in a re which is blown towards fuel. Backing res go against the
wind and thus spread slowly, igniting new fuel at its base. Flank res, as the name
suggests, are the long edges between heading and backing res. The main danger of
2
While the second and subsequent passes of an environment are relatively constant in duration, the
rst pass of the environment is normally the fastest: the factors behind this are discussed in Chapter
5, Performance Baselines.
38
anking res is that they can become wide heading res if the wind direction shifts. It
has been shown that, in areas of homogeneous fuel and terrain, when a re begins at a
single point the head re spreads in a fan shape which widens as it progresses, resulting
in an elliptical re shape (Wagner 1969).
Figure 4.2.: Anatomy of a bushre: head(ing), back, and ank res
It has been shown that the maximum speed of a heading re is between 1218 km/h, in
forests and grasslands respectively (Cheney and Sullivan 2008). To test the eciency of
the algorithm in tracking a moving re front, res are simulated using an ellipse with one
focus on the res ignition point and an edge (i.e. heading re) which spreads from that
point at 18 km/h. For simulations which contain a modelled re spread component, the
environment used is homogeneous in both fuel and terrain, with a square 22.5 km area,
chosen as an analogue to the Wilsons Promontory National Park in Victoria, Australia
(Figure 4.3). The national park is the largest coastal wilderness area in Victoria, and
39
is frequented by bushres: in 2009 a lightning strike began an event which burned over
250 km
2
(50% of the parks area).
Figure 4.3.: a) Satellite view of Wilsons Promontory, an Australian national park lo-
cated at 39
02
S 146
23
E. b) A corresponding priority map, where each pixel

corresponds to a node. Higher priority levels are represented by brighter
colours.
4.6. Summary
The simulator presented is primarily concerned with allowing evaluation of the waypoint
selection heuristic in the proposed algorithm. Due to this focus, low level aerodynamic
and bushre mechanics have been abstracted. Where abstractions will result in arti-
cially increasing the apparent performance of the underlying algorithm, measures have
been taken to mitigate this, such as consciously conservative UAV speeds and worst-case
re velocities.
40
5. Performance Baselines
This chapter describes typical scenarios the proposed algorithm was designed for, and
the scaling of performance associated with particular algorithm parameters. The list of
parameters which have an eect on search speed can be broadly divided into software
variables which are easily changed, and available hardware which is often a constraint
which needs to be worked within. Hardware constraints include swarm size, communi-
cation range, target detection radius and general ight characteristics of the UAV. With
regard to software, variables include the frequency of information broadcasts, pheromone
map topology, evaluation heuristics and repulsion methods. When describing the base-
line performance of the presented surveillance algorithm, the UAV platform is assumed
to be an inviolate constraint and algorithm modications are the focal point.
5.1. Baseline Calculation
The proposed algorithm, in order to evaluate performance, is compared against two
metrics. First against a calculated maximum theoretical performance level, and after,
in Chapter 9, against another state-of-the-art algorithm.
The calculated value, referred to as the baseline, is derived by dividing the size of the
environment by the amount of new area a UAV can scan per second. The baseline is
calculated by taking two circles with radii equal to that of the UAVs camera footprint,
and separating their centres by the distance a UAV can travel in one second. The non-
overlapping area of either one of the circles is an approximation of the amount of ground
a UAV can survey per second (see Figure 5.1).
Standard UAV conguration has an individual footprint area of separation of 18 metres
and camera radius of 176 meters, resulting in an area of 6333.97m
2
scanned per second.
Dividing the size of the environment in square metres by this number gives the minimum
amount of time a single UAV could complete a survey, for example a square environment
with sides of 30 km would take (30000
2
/6333.97) seconds (39.47 hours) for a single UAV.
To derive a value for a swarm of UAVs, the value is divided by the swarms size. Thus
41
Chapter 5: Performance Baselines
Figure 5.1.: Method for approximating the maximum survey speed of a UAV, given the
distance travelled in one second d, camera radius r, and the new area scanned
a.
2 UAVs would take 19 hours and 100 UAVs would take 23 minutes.
This stated minimum time inherently assumes that there is no overlap between coop-
erating peer agents. Other assumptions include the ability for a UAV to turn without
overlapping its own previously scanned areas, as well as ignoring physical limitations on
the starting locations of the UAVs.
Using this lower bound, the average amount of time a node will go unobserved is
exactly half the time it takes to survey the full environment. Average time unobserved,
i.e. period between surveys for a given node (also referred to as survey period), is the
standard gauge used in judging performance of surveillance algorithms in this disserta-
tion. With simulated results, the average time unobserved is calculated by using the
raw timestamps at each node.
5.2. Scaling with Swarm Size
One of the most important measures of eectiveness with regard to swarm algorithms
is how well performance scales as the size of the swarm changes. In a perfect swarm,
doubling the number of agents would double the performance, which in the case of
surveillance would mean halving the average survey time. Additionally, due to the
inherent lack of a central controller in a swarm paradigm, changes in swarm size should
not require modication of algorithmic constants. While tweaking of variables will
invariably increase performance for any explicit scenario, a swarm algorithm should be
robust enough to uctuate in size while still maintaining viability.
To test the viability of changing swarm sizes in the presented surveillance algorithm,
an experimental simulation was run in a 30 km square environment with uniform priority,
42
Figure 5.2.: Decrease in average survey time with increased UAVs in a 30 km square
environment using a log-log scale.
with results shown in Figure 5.2. As the graph is log-log, the straight line seen indicates
that a power relationship exists between swarm size and performance (i.e. doubling
UAVs halves the survey time). It can also be observed by the trend seen in the simulation
that results reasonably match those shown in the baseline, theoretical minimum time,
estimates.
Relative performance of the swarm compared to the baseline can be more easily seen
in Figure 5.3. With swarm sizes between 1 and 150, performance stays at a linear 225%
of baseline. At 400 UAVs this margin increases to 300% of baseline and the gap widens
thereafter. A point of interest in the graph is that a lone UAV, i.e. a swarm of one, is
able to get within 180% of the baseline. The reason for this is that an individual UAV
operating in isolation conducts negligible redundant searches and thus is able to perform
highly ecient search patterns.
5.3. Stable States
The increase in total environment pheromone quantity, and the amount of pheromone
being removed by agents in the environment, reaches an equilibrium when the rst pass
of the map is complete, where a pass is dened as each node having been surveyed at
43
Figure 5.3.: Average survey time of the algorithm as a percentage of baseline
least once by a member of the swarm. The rate of increase in pheromone and sub-
sequent levelling o is shown in Figure 5.4, where the environment starts with zero
pheromone and the amount increases directly proportionate to the amount of time the
simulation is run until the point where every node has been searched once. The shape of
the curve is representative regardless of variables such as swarm and environment size,
communication radii, and node spacings.
5.4. Patterns
While there are no specic behaviours hard-coded into the swarm, there is one pre-
dominant one which emerges, with a few minor variations. This behaviour is an edge
following one, which is an ecient behaviour for the class of lawn-mowing problems. As
the name implies, edge following is the phenomenon whereby an agent surveys an area
by doing a circuit of the perimeter and slowly spiralling in. This behaviour is often seen
as desirable enough to hard code into algorithms, such as in Erignac (2007). Figure 5.5
shows a simulation in progress in which the UAVs are exhibiting this edge following
behaviour. Nodes start bright green when they have zero pheromone, and fade to black
(and then red) as pheromone increases. The small multi-coloured squares which appear
at the head of zero-pheromone trails represent the positions of the swarms UAVs.
At the root of this behaviour is the ratio between node spacing and the UAVs arrival
44
Figure 5.4.: While the environment starts with zero pheromone at the beginning of the
search, the pheromone level rises until the rst pass has been completed, at
which point it uctuates around a stable average (i.e. pheromone equilib-
rium between removal and addition). 20 UAVs, 25 km square environment
radius. As described in Section 4.2, arrival radius is how close the UAV needs to be to a
node before it counts as being fully surveyed. The spacing used in the nal version of the
algorithm is one where node spacing is larger than twice the arrival radius (i.e. arrival
diameter). If node spacing and arrival radius are such that a UAV can survey more
than a single node at a time, the search patterns produced are shown in Figure 5.6. In
this diagram nodes are represented by grid cells, the node has its pheromone reset when
the survey radius of the UAV, represented by the large white area, covers the centre
of the cell (i.e. not when the UAV is directly overhead). The individual moves in the
patterns are shown in Figure 5.7 for arrival radii which are smaller or larger than the
node spacing.
What is apparent when deconstructing this apparent edge following behaviour is that,
at its core, it is basically an inclination to travel in a straight line as long as pheromone
levels are not signicantly diverse. For this reason, the behaviour becomes progressively
less pronounced over the duration of a search as pheromone levels become less uniform.
In a single UAV simulation, edge following persists almost indenitely and typically
for the rst pass of a map with any arbitrary number of UAVs, edge following is the
predominant behaviour. Refer back to Figure 5.5 for an example of a typical scenario.
Depending on the number of UAVs, and the degree of communication, after the second
pass the behaviour deteriorates as the pheromone levels of neighbouring cells become
45
Figure 5.5.: A group of UAVs (represented by small, multicoloured squares) exhibiting
edge following behaviour on the rst pass of a uniform map.
Figure 5.6.: Scale representation of the path a UAV follows when all non-surveyed nodes
have the same pheromone level. The dierence between survey radius and
survey diameter node spacing is shown. Nodes count as observed once the
UAVs footprint reaches the centre of a given node.
46
Figure 5.7.: a) Typical sequence of moves when node spacing is equal to or greater than
the arrival radius but smaller than the arrival diameter. b) Typical sequence
of moves when the spacing of nodes is larger than the arrival diameter.
Figure 5.8.: An example of node spacing smaller than the survey radius, zoomed into a
corner of the search space. UAVs follow a complex series of moves resulting
in diagonal movement.
increasingly dissimilar. This disrupts edge following and results in UAVs sporadically
traversing low pheromone areas to reach high pheromone ones.
With node spacings smaller than the communication radius (the UAVs eective broad-
cast distance), artefacts such as diagonal moves with heavy overlap of the paths start
to appear.
1
Figure 5.8 shows a simulation in which UAVs are exhibiting edge following
behaviour. These diagonal moves can sometimes result in a path which spirals outwards,
as shown in Figure 5.9, which would be an optimal behaviour if the overlap is ignored.
This behaviour is relatively fragile and once a UAV nishes its initial spiral it is rare for
a new one to be started, though diagonal moves in general are still favoured.
1
The proportion of paths which are diagonal to the pheromone maps orientation is dependant on the
degree of dierence in sizes.
47
Figure 5.9.: An example of diagonal moves leading to interesting, yet unstable, spiralling
behaviour.
48
5.5. Summary
This chapter has described the theoretical maximum performance against which the al-
gorithms performance is evaluated. The mechanics and factors including as the relation
between node spacings and the arrival radius have also been described, which are re-
quired for understanding the results which will be presented in the following chapters.
The capacity of the algorithm to perform exhaustive coverage on an area with varied
swarm sizes, without serious loss of eciency, was validated. Artefacts were found dur-
ing simulation such as emergent edge following, and rapid stabilisation of pheromone
levels.
49
6. Varied Priority Levels
It is not always desirable to have the entire environment receive equal attention: for
example dry grassland is more likely to combust than a rocky hillside, and proximity
to a populated area increases the attention required regardless of terrain. Setting the
pheromone to increase at varied rates at individual nodes causes the frequency at which
they are surveyed to change proportionally. In Figure 6.1 a priority map is shown with
two dierent priority levels. The area with the highest priority (Priority 1) increments
pheromone at twice the rate of the lowest priority (Priority 2) area.
To calculate baseline numbers for a multiple priority map such as this, the environment
size used in the equation is modied: the size of each prioritys subsection of the map is
multiplied by its relative priority level. The reason behind this is that a single pass of
the search space will not be complete until the every node has been seen at least once.
Due to the low priority areas having the slowest pheromone increment rates, they will
be the last ones to be seen. To use the dual priority map as an example, where the
high priority area has twice the pheromone as the low, the high priority area will need
to be visited twice in the time it takes the low priority area to be surveyed. Using this
method gives the same baseline number for uniform priority maps (everything is simply
multiplied by 1) yet adjusts appropriately for multiple priority environments.
To show the result of this, Figure 6.2 presents a simulation with high priority areas
having half the survey period of low priority areas (i.e. high priority areas are visited
Figure 6.1.: Priority map with two priority levels. The light areas have a pheromone
increment of twice the dark areas.
51
Chapter 6: Varied Priority Levels
Figure 6.2.: Average survey period of the baseline and simulation results using the two
priority map from Figure 6.1
twice as often). It can be seen from the data that the respective survey rates are
comparable to the theoretical baselines generated by this new method. In Figure 6.3,
where results are plotted as a proportion of the baseline performance, it can be seen
that for swarm sizes under 200 the average survey frequency is less than the baseline by
a factor of 0.7, the same as was observed in the previous chapter with uniform priority.
Figure 6.4 shows a continuation of this trend, with the result of using the same map
layout containing four distinct priority levels instead of two.
6.1. Complex Environments
In the previous section it was shown that environments with two and four areas of rela-
tively similar priorities are able to be surveyed at independent rates without signicant
loss of eciency. This section will look at more complex environments with a larger
spectrum of priority dierences. Figure 6.5 shows an environment with nine priority
levels, inclusive of one null-priority area. A null priority area is one without pheromone
or pheromone increment.
When compared to the baseline results for environments with two and four priority
areas, discussed in the previous section, Figure 6.6 shows that while the survey period
ratios are noisier, they still follow the trends noted in the previous section. Of interest is
that at low densities, the low priority areas are over-surveyed which is to the detriment of
52
Figure 6.3.: Average survey period on the two priority map as a proportion of the base-
line value
Figure 6.4.: Priority map with four priority levels (increment rates of 4 / 3 / 2 /
1), with the highest priority area incrementing pheromone at four times
the rate of the lowest.
53
Figure 6.5.: Priority map with nine priority levels (eight plus a null priority zone).
Lighter areas have a greater priority, with the lightest areas having a phero-
mone increment of eight times the least area. The null priority area is blue,
nominally representing water.
Figure 6.6.: Average survey period of the eight priority level environment, broken down
into individual priority zones.
54
Figure 6.7.: Eect of UAV density on survey ratios: a) 31815 m
2
(150%) environment,
b) 15079 m
2
(50%) environment
high priority areas. This over-surveying is the degree that algorithm survey frequencies
exceed the theoretical maximums calculated in the baseline. After a critical density of
UAVs has been reached however, in this scenario around 70 UAVs, all priorities converge
to the baseline predicted ratios.
The eect of agent density on the performance convergence point can also be seen
when comparing the same simulation but with an environment size of 50% (15079m
square) and 150% (31815m square) respectively shown in Figure 6.7. It can be seen
that when the environment shrinks the shape of the performance curve is compressed
yet performance relative to the baseline prediction remains consistent.
In these specic examples the convergence to ideal ratios occurs at around 80 UAVs
when the environment is shrunk to 50%, while this does not occur until 150 UAVs for
the 150% environment. In either case performance at this milestone coincides with a
survey period of 60% of baseline for all priorities. The cause of this is a shifting of the
point at which theres no way to spread the swarm out well enough to match the lower
bound on period. Consequently, in the small environment, performance is reduced to
50% of baseline performance at 200 UAVs, while the large environment does not reach
this point until 1000 UAVs.
Loss of eciency, while still present in single priority search spaces, is exaggerated in
multiple priority maps as instead of the entire environment being saturated before loss of
eciency occurs, high priorities cause agent density to increase to saturation in discrete
regions. This can be demonstrated by using the same environment size (22.5 km square)
and priority map with nine regions used previously, and greatly enhancing the dierence
in priority levels so that the highest priority area generates 128 times the pheromone
of the lowest priority area. Figure 6.8 shows that despite having the same global agent
55
Figure 6.8.: Increasing severity of priority levels on identically sized environments to
increase local agent density. a) Maximum pheromone increment 128, b)
Maximum pheromone increment 8
density, increased local density causes eciency to drop-o at a steeper rate at high
UAV counts, as seen when global density was raised by shrinking the environment size.
Of note in both simulations, despite the extreme dierence in priority levels between
them, the survey period of the highest priority is consistent in both at approximately
0.6 of baseline. The dierence in eciency does not occur until extremely high agent
densities with swarms of over 100 UAVs.
6.2. Null Priority and No-Fly Zones
Null priority areas are, as the name suggests, nodes of the pheromone map which have
both zero pheromone, and a pheromone increment rate of zero. Because using a phero-
mone map which is a regular two dimensional matrix simplies many equations needed
to run the algorithm, having null priority areas enables the UAVs to ignore areas of the
environment which, while contained within the spatial coordinates of the pheromone
map, do not need to be searched. Practical examples of such areas, in the context of
re spotting, are bodies of water or stretches of sand/earth/gravel with no substantial
vegetation, or areas already burning or burnt out.
Areas of null priority are still counted as valid nodes for the purposes of waypoint
selection, they however have a weighted value of zero and thus are almost always avoided
unless own over on the way to somewhere else.
No-y zones are an extension of null priority areas, with the addition that the node
is never a valid destination, even en route to another location. The main use for no-y
zones would be to contain UAVs within unrestricted airspace, such as areas away from
56
Figure 6.9.: a) Ideal heat map which would be generated by the baseline algorithm, b)
heat map generated by the algorithm running an 80 UAV swarm. Ratios
are given as a proportion of the most surveyed node.
airports, residential areas, or ight lanes. It is also plausible that they could be used
for very tall or sheer obstacles, such as cli faces which are not safely navigable at the
height required for infrared ground surveillance, or very tall buildings in the event that
the algorithm was used near a city. Due to these later two scenarios, no-y zones are
simulated as blocking line of sight (with regards to communication) in addition to being
impassable to UAVs.
When average survey periods are analysed, the dierence in average survey period
between environments with null priority and no-y are almost imperceptible. The dif-
ference however is very apparent when the number of times individual nodes have been
surveyed during the simulation are viewed as heat maps. A heat map, in this context,
is a visualisation of the number of times individual nodes have been surveyed, scaled so
that the node with the least visits is blue, and the node with the most is red. Figure 6.9
shows the heat map representations of the original nine priority level simulation (using
null priority) alongside the ideal heat map generated by the baseline algorithm.
In this baseline example, with the null priority zone being in a corner and adjacent to
the two lowest priority areas, the heat map shows a complete lack of activity over the null
priority region, and little apparent eect on the visit frequency of the surrounding areas.
To better illustrate the eect of null priority versus no-y, the areas were rearranged so
that the null priority zone is an obstacle in the middle of the map.
57
Figure 6.10.: Comparison of heat maps after the priority map is rearranged into a more
dicult to survey arrangement. The centre blue region is a) null priority
and b) no-y. Ratios are given as a proportion of the most surveyed node.
Map B is strongly biased by the very high survey count at the top right
corner of the no-y zone.
Figure 6.10 shows the new conguration with null priority and no-y heat maps com-
pared side by side. The main change in the null priority heat map is the corners of the
null priority zone have been occasionally own over, most easily observable in the top
left between the highest priority areas. This artefact becomes especially noticeable in
the no-y heat map. Here the entire map appears muted due to the number of times the
nodes adjacent to the corner of the no-y zone, between the two highest priority areas,
were surveyed by UAVs funnelled around the no-y zone.
Having established how null priority areas aect the movement of UAVs through the
environment, the actual performance aspects are shown in Figure 6.11. The term orig-
inal denotes the original priority map conguration using null priority, while the other
two are from the rearranged priority map. As can be seen, the absolute performance
dierence between the three is minimal.
6.3. Summary
This chapter introduced the use of priority levels. Simulation shows that the limiting
factor for eciency is local density of UAVs. Until density thresholds are reached,
performance increases linearly with swarm size. Afterwards performance still increases,
58
Figure 6.11.: Comparison of the three dierent priority map congurations proportionate
to baseline performance
but individual UAVs become less ecient.
The proposed algorithm has been shown to maintain its performance when using
complex priority maps with greatly dissimilar priority levels. It was also shown that
regions of null priority and no-y zones have minimal adverse eect on performance.
59
7. Target detection and tracking
Ultimately the purpose of using pheromone maps for swarm UAV surveillance is for the
detection of some object. In the case of res, this includes both static and dynamic
targets. Static targets spontaneously appear inside the environment and do not move,
such as illegal camp res or lightning strikes, while a re that has spread can be con-
sidered a dynamic target which can move and change appearance and size. For the
algorithm to be eective, it must be capable of simultaneously locating unknown targets
and monitoring known targets for change.
7.1. Fire location
Figure 7.1.: Average detection time of targets based on location
In general, the average time taken to nd a new target, such as a re, is equal to the
average survey period of the priority level it appears in. The exception to this rule is if
the re appears at the very edge of the search space. When survey periods are plotted
per node rather than per area, as in Figure 7.1, it can be seen that the survey period
for edge nodes can be 20% longer than the overall region average. It is observable in
61
Chapter 7: Target detection and tracking
Figure 7.2 that this artefact is related to search space edges, and not priority region
edges. The reason behind this is simply because the area in the middle is in the way
as UAVs try to reach the other side.
Figure 7.2.: Average detection time of targets in a map with two priority levels
While this edge behaviour could be considered benecial if it is expected that res
will most often occur away from the edges of the search space, it is possible to suppress
this artefact. To achieve this, the priority map is weighted so that the priority of
edge nodes are increased proportionally to the amount they would otherwise be under-
surveyed. Figure 7.3 shows how scaling the priority map almost completely eliminates
the bias against edge nodes. The amount each node needs to be scaled can be calculated
through simulation of a uniform priority map. Each node in the simulated results divides
its average survey period with that of the best performing node. These values (which
were found to range from 1.0 in the centre to around 1.24 in the corners) are then saved
in a matrix, and used as priority level multipliers.
7.2. Fire tracking
Using a standard priority map, once a re has been discovered it will be passively tracked
at the same rate that the entire environment is surveyed. Tracking speed can be greatly
increased by making two changes to the priority map upon re detection. Firstly, when
a re is found, nodes immediately surrounding it have their priority level increased to
a high level (referred to as a re priority), and secondly the location of the re itself
62
Figure 7.3.: Average detection time of res at the edge of the search space using a scaled
priority map
is set to have a priority of zero. Taken together, this results in the swarm converging
at the edge of res, eectively tracking the re front, which makes observation of the
res speed and spread substantially more rapid. Going forward, priority maps with
non-static priority levels will be referred to as a dynamic priority map.
To show dynamic priority maps in action, a scenario is presented where a ground
site contains a swarm of UAVs and a re is ignited at a distance of 15 km. UAVs
are launched individually from the base at 120 second intervals, rather than starting
randomly dispersed throughout the environment. The re heads away from the launching
point until the heading re is 12 km away from the ignition point, where the simulation
ends.
When the simulation in progress is viewed (Figure 7.4), it can be seen that when the
unobserved res (nodes with re present) are on the edge of the re fronts, and that a
large proportion of the swarm has been emergently drawn to the heading re, where it
is spreading most quickly.
Performance of the algorithm in this scenario is measured by the percentage of the
res area which has been discovered, the results of which are shown in Figure 7.5. Due
to the launch period and distance that needs to be travelled, there is a hard limit of 16
UAVs or 11 MAVs which can reach even the ignition point before the simulation ends,
as can be seen in the plateauing of MAV performance at this point. The results consists
of averaged results from 300 random simulation restarts.
63
Figure 7.4.: Progressive images of the simulation in progress. Red areas are undiscovered
re, while green areas are res which have been found by the swarm. UAV
locations are shown (not to scale) as squares. The environment is a square,
as per the description, but has the bottom third cropped in these images.
64
Figure 7.5.: Comparison of the percentage of re area discovered using two dierent
classes of UAVs
7.3. Secondary Fire Detection
If we assume that there will only be a single re front to manage, the algorithm will
produce the best results when assigning an arbitrarily high priority to areas of re.
This results in UAVs eectively ignoring any nodes which were not currently tagged
as being adjacent to a re. Two issues with this are that rstly the assumption of
no additional res is unrealistic, and secondly that there are diminishing returns with
focusing additional resources on a small area.
By selecting an appropriate level of priority, it is possible to have a swarm which both
tracks moving re fronts as well as maintaining persistent surveillance of the remainder
of the environment for secondary res. To show the trade-o in re front tracking and
area surveillance, a second scenario is presented which is initialised with two points of
ignition. The swarm has a waypoint for the primary ignition, 10.5 km distant, while a
secondary ignition, 21 km distant, is unknown (visualised in Figure 7.6). The swarm
consists of 16 UAVs and launches at a rate of one vehicle per 120 seconds. The simulation
consists of averaged results of 300 random restarts.
Figure 7.7 shows the re detection percentages for forest res and grass res using a
swarm of 16 UAVs. Lowering the priority to allow secondary re detection amplies the
dierences seen with the single re example. While the slower moving forest re scenario
still manages excellent (over 90%) re detection, there are some diculties with faster
65
Figure 7.6.: Progressive images of the secondary re simulation in progress. Red areas
are undiscovered re, while green areas are res which have been found by
the swarm. UAV locations are shown as (not to scale) squares.
moving grass res. Figure 7.8 and Figure 7.9 show the breakdown of primary re versus
secondary re detection. When the re priority is too low the swarm has only limited
reaction to locating res and mostly continues a wide area search. Too high a priority
and the swarm converges on the rst re found and eectively ceases to search.
1
Specically addressing swarm performance on the grass re scenario, the main di-
culty the swarm has is that the simulation has a short duration
2
and the two res are
initially separated by a distance of 10.5 km. Distance between res is not a major issue
when the swarm is deployed and dispersed throughout the environment when the rst
re is located, but when launched from a base in response to an initial sighting the sec-
ond re is often not discovered until too late in the simulation for it to be fully mapped
out.
1
As was shown in Figure 7.8, a re priority of approximately 20 times that of the normal priority was
found to produce the best results.
2
40 minutes of real-time are simulated, and the res spread 12 km from their origins at a rate of
18 km/h.
66
Figure 7.7.: Percentage of res found using the secondary re simulation with grass and
forest res with a swarm of 16 UAVs
Figure 7.8.: Forest re: dierence in detection between the primary waypointed and
secondary unknown re for various priority levels
67
Figure 7.9.: Grass re: dierence in detection between the primary waypointed and
secondary unknown re for various priority levels
7.4. Discreet Mobile Targets
Dynamic priority maps have been shown in the previous section to perform well for res.
This section investigates the tracking of targets which are travelling instead of spreading
(e.g. an animal or vehicle). The simplest example of this is a single target which travels
in a straight line across the environment. To track this type of target, the priority
map requires a mechanism for reverting high priority areas back to their original state
once the target is no longer in the area. For this example specically, when a target is
detected all adjacent nodes have their priority set to 200 times that of the base value.
3
If a cell with an increased priority is visited and no target is found and there are no
known targets in adjacent nodes, the priority of this node is reset to the default level.
The results of detection time on a moving target and the eects of reactive priority
map changes are shown in Figure 7.10. It is an interesting artefact that slow moving
targets actually take longer to be found than stationary ones, yet fast moving targets get
found more quickly. While moving between nodes disrupts the ability of the algorithm
to nd targets by systematically checking every node in order, when moving fast enough
this becomes outweighed by the increased probability that the targets movements pass
it under a UAVs camera. This can be attributed to targets which are moving not
3
The value of 200 was arbitrarily selected, however any arbitrarily large value will produce the same
results.
68
Figure 7.10.: Elapsed time before initial detection of targets in a square 22.5 km envi-
ronment with 100 UAVs travelling at 18 m/s. 4 Target Baseline refers
to a simulation with four targets and with no priority update on target
detection.
conveniently waiting on a node with an appropriate amount of pheromone to allow it to
be seen as soon as possible, which has the eect of, on average, lengthening the time it
avoids detection. The mere act of moving, however, works to reduce the time undetected
as it consequently can make the target pass under a UAV en route to some place else:
the cross-over on the chart reects when these eects outweigh each other. Regardless,
the main eect changing the priority has on detection time is to degrade performance
when there are multiple targets in the environment.
Once the target has been found, the other measurement for performance is the time
between observations of the target from then on. Figure 7.11 shows results for observa-
tion time using dynamic versus static priority levels. With dynamic priority levels, in
general there is a marked increase in performance over the baseline. This falls o and
time between observations post-detection increases with faster targets. While UAVs are
travelling at 18 m/s (68.4 km/h), the percentage of time the targets were under direct
observation by a UAV falls to under 50% at a mere 5.5 m/s (19.8 km/h) target speed.
The only scenarios which resulted in targets being under observation for an extended
amount of time are those which increase the priority of nodes in front of the target at
the same rate that they were able to be surveyed. This occurs when the target to UAV
speed ratio is within certain values, else the UAVs would not be in the right position
when the target entered a new node and it would quickly get away from the swarm,
69
Figure 7.11.: Percentage of time during the simulation that the swarm knew the targets
location. Baseline results refer to a simulation where the priority was not
updated on target detection. square 22.5 km environment with 100 UAVs
travelling at 18 m/s. N.B. The one and four target baselines are eectively
identical due to being non-reactive, thus have overlapping data points on
the graph.
unobserved.
Further observations were made of the eects of using such high priority levels on
the average pheromone levels of the map (Figure 7.12). For targets slower than 5 m/s,
the algorithm still has a chance of keeping up and the average pheromone levels of the
environment more than double. Once targets increase in speed the values for average
pheromone level and pheromone level at target converge as the eect of priority level
changes become negligible.
7.5. Summary
This chapter has described the use of dynamic priority updates for use in tracking res
and other targets. This involved incorporating rules which automatically increase or
decrease the priority of an area under certain triggering conditions, such as targets
being found or a period of time passing.
The results presented indicate that eective wildre tracking can be performed by a
decentralised UAV swarm. Using dynamic priority updates has been shown to enable
persistent surveillance for re ignition, while at the same time closely tracking re fronts
70
Figure 7.12.: Average pheromone of the nodes directly under the targets, ignoring pri-
ority multipliers, compared to the average pheromone level of the map.
via exploitation of emergent behaviours.
Performance was not found to carry over to discrete moving targets such as vehicles.
Better target tracking might be possible by incorporating predictive target direction
algorithms, such as those used in radar systems.
71
8. Communication
There are two components to the swarms communication: broadcast range, and broad-
cast message frequency. As broadcasting data wirelessly is power intensive, it is benecial
to do so as infrequently as possible, providing mission performance is not signicantly
compromised. A longer broadcast period also minimises the amount of processing time
that needs to be used to process incoming information, which can be signicant at high
agent densities.
The communication range, on the other hand, is determined by the available hardware.
While low bandwidth communication between a UAV and a ground station can be long
range (10 to 100 km) due to use of large ground antennas and dishes, an optimistic
maximum range for direct MAV to MAV communication is 5 km, while the standard
communication used is a more restrained 3 km line of sight (LOS).
8.1. Scaling with Communication
Eects of varying communication range and frequency can be seen in Figure 8.1. Min-
imum and maximum values are given respectively by a swarm which does not com-
municate at all and a swarm which communicates every second with an innite range.
It is apparent in this particular scenario (a square 30km environment with 40 UAVs)
that the communication period has only a marginal eect when UAVs are operating
at the standard communication range of 3 km. The dierence in performance between
broadcasting once a second, and once every four minutes, is only 10%.
The benets of communication range are tied to the density of the swarm; if agents are
spaced 2 km apart, a communication range of 3 km is as good as 100 km. Figure 8.2 shows
the performance of dierent communication ranges based on environment size. As search
time will naturally increase for larger areas, regardless of other factors, performance is
expressed relative to performance using global communication.
As can be seen in Figure 8.2, the relative eect on performance of any particular
communication range is dependant on the size of the environment being surveyed. In a
73
Chapter 8: Communication
Figure 8.1.: Communication eects on performance (average survey period). Commu-
nication range of none is for a scenario with broadcasting completely dis-
abled. Square 30 km environment, 40 UAVs
Figure 8.2.: 40 UAVs, 60 second communication period: Diminishing returns on com-
munication range over large environments. Results are scaled by the perfor-
mance of a swarm with global communication and a 1 second communication
period.
74
square 60 km environment a 4 km range is needed to get the same performance as a 3 km
range in a 40 km environment. The dierence in performance between a 5 km radius
and an unlimited/global range is only 10% even at 90 km. This relative performance
gap will shrink as the agent density increases with the addition of agents, or grow as
agents are removed.
In summary, communication parameters do not have a notable eect on swarm per-
formance as long as relatively frugal minimums are met (e.g. 2 km range, 120 s interval).
While this is sucient for what is considered to be the algorithms normal parameters,
information ow within the swarm has interesting and noteworthy artefacts, especially
at extremely low or transitional values, which are examined in further detail below.
8.2. Information Flow
In swarm systems using distributed pheromone maps, any new information in a trans-
mission overwrites old information while travelling through the swarm in a series of
hops. This process can be modelled as a dynamic, ad hoc network which has nodes
and connections based on spatial coordinates and UAV proximity. The spread of new
information in the swarm can thus be analysed in the context of a shortest path problem
in a dynamic tree (Chabini 1998). In a swarm of n agents, assuming an innite search
space and nite communication range, the maximum hop count is n 1.
Information is not guaranteed to traverse the entire tree due to the possibility of data
becoming out of date before it reaches a given node, at which point it will progress no
further down that branch. In an optimised network, the maximum path length would be
minimised, and every node would be connected to every other node via a path with a hop
count of one, such as maximally connected through global peer-to-peer communication.
Issues with setting up such a network include agents being required to know the
location of all other agents in the swarm. In practice this is dicult to achieve without
global communication. If this long range communication, in the quantities of bandwidth
required, is a viable option then an algorithm that optimises using centralised control
should be seriously considered.
Depending on the connectivity of the swarm, itself dependant on factors such as swarm
density, communication rage, and knowledge of other UAVs, the network will resemble
a scale free or small world network (Newman 2003). In practice, communication ranges
are limited and the network formed between UAVs has a small-world topology, where
the majority of nodes are not neighbours, yet most nodes can be reached by a small
75
number of hops (Olsson et al. 2010).
More eective area search is achieved when factors such as repulsion are strong enough
to achieve a uniform distribution. This enables rapid information ow as there exists
only a single network.
8.2.1. Data Synchronisation
An individual UAVs knowledge of its general vicinity is up to date in most cases. The
chance of data being known by any particular individual is directly related to the age of
the data. Areas with high pheromone have the oldest information, and UAVs are drawn
to the highest pheromone areas. However as the agent density goes up, the rate of
information ow becomes increasingly important in maintaining this state of local area
knowledge. When a UAV is not alone in surveying a locality, the UAVs individually
observed information needs to be shared with greater frequency if performance is to be
maintained. Otherwise nodes go unvisited until their pheromone level is high enough to
attract a UAV (as normal) followed by multiple redundant searches in a small period of
time by multiple UAVs who have yet to receive knowledge of the rsts eorts.
The ow of information within a swarm can be measured by how synchronised the
data contained in pheromone maps of individuals in the swarm are. Synchronisation
is measured on a per-agent basis: a synchronised node of the pheromone map is one
in which the survey time known to the selected UAV is the most recent available in
the entire swarm. As an example, in a swarm of 20 non-communicating agents, each
agent will always have a minimum of 5% of its pheromone map in sync simply due
to having been the most recent agent to directly observe that subset of nodes. Adding
communication broadcasts increases this percentage, relative to broadcast frequency, by
enabling the incorporation of survey timestamps from peer agents.
Of the information in the environment, the age of data has an inverse relationship
to how likely it is to be synchronised with the entire swarm. Figure 8.3 plots the total
synchronisation level along side the average age of the synchronised and unsynchronised
information. In general, increasing the communication range lowers the average phero-
mone level of the environment, which in turn lowers the average age of the subset of
nodes that are synchronised. The percentage of unsynchronised information approaches
zero as the communication range becomes functionally global.
The mean age of synchronised nodes increases parallel to the time taken for the swarm
to complete its rst pass of the environment, shown in Figure 8.4. Scaling is based on
communication period instead of communication range between a period of 1, and a
76
Figure 8.3.: As the synchronisation level approaches 100%, the environment pheromone
level and pheromone level of nodes which are synchronised converge, while
unsychronised nodes are those a single time increment old as the entire
swarm will have knowledge of it on the next simulation update.
Figure 8.4.: Eect of communication period on synchronisation level and environment
pass time
77
Figure 8.5.: Percentage of synchronised nodes given varying communication periods
period of 4000. This behaviour is intuitive and not particularly interesting.
More interesting is what happens after the algorithm starts breaking down. At a
communication period of 4000 seconds (and only 30% of the pheromone map in sync,
as shown in Figure 8.5) performance falls to a comparatively stable level and redundant
surveying becomes predominant, yet the nodes being redundantly surveyed are at least
mostly in areas with generally high pheromone. To put it another way, the areas which
would have the highest pheromone on a pheromone map generated by a swarm with
constant, global communication are still being surveyed rst, however agents are sur-
veying the same area one after the other as they have not received information letting
them know it is no longer required.
The percentage of synchronised information contributed by rst hand knowledge
should be taken into account: in this example when 30% of the map is in sync, 5%
is direct observation and 25% is received information. The directly observed nodes have
the same relatively low mean age regardless of communication, however the 25% of
in-sync information contributed by the swarm is all relatively old data.
As broadcasts become less frequent, the 5% of the environment that the agents have
directly observed themselves becomes a larger component of the agents total in-sync
knowledge, until the point where there is no communication and it makes up 100% of
the total in-sync nodes. This is why the mean sync age counter-intuitively goes down
78
Figure 8.6.: Average search time (performance), scaled relative to a swarm with zero
communication. For example, a value of 1.2 represents nodes going 120%
longer between visits than if the swarm did not communicate at all. Swarm
size is 16 UAVs.
past breakdown, even though the time it takes the swarm to perform a complete pass
of the environment goes up.
It is noteworthy that while without communication nodes have an approximately
equivalent mean age as when broadcasting continuously, there is a much higher vari-
ance, with especially high maximum values. This is due to the fact that nodes become
desynchronised essentially at random, and sections of the map can go unobserved for as
long as it takes for a single agent to do a full circuit.
8.3. Not All Communication Is Good
Redundant searching of a node by multiple UAVs becomes a critical issue if the infor-
mation ow inside a swarm is extremely low, at which point communication is actively
detrimental to the performance of the swarm as a whole (Figure 8.6). The reason for
this phenomenon is that agents receive enough information in a single exchange to deter-
mine the approximate order in which features in the environment should be visited, but
without further communication there is no cooperation between peers and a signicant
amount of time is wasted on redundant searches.
Communication rates of a swarm of 16 UAVs is represented in Figure 8.7, where it can
be seen that there is a decrease in the received broadcast frequency as the communication
radius shrinks or the environment size increases. The graph shows that communication
79
Figure 8.7.: Communication period scaled by environment size and communication
range. The communication period is the average time that passes between
broadcasts received by a single UAV, arbitrarily chosen from the swarm.
Swarm size of 16 UAVs.
Figure 8.8.: Communication period, with a communication range of 2.8 km, scaled by
environment size and swarm size. Communication period is the average
time that passes between broadcasts received by an arbitrarily chosen UAV
in the swarm.
80
radius makes the most signicant dierence in this regard, with the dierence between
400 and 1200 metres being an order of magnitude improvement in how frequently broad-
casts are received. Increasing the number of UAVs, shown in Figure 8.8, has a similar,
yet reduced, eect as increasing the communication radius. The communication benets
gained by either of these actions is relative rather than absolute: increasing the com-
munication range by a kilometre has a much larger eect relative to the search space
size.
Using the size of the search area (map), the communication radius (range), and the
number of UAVs in the swarm (UAVs), the simplied equation (i.e. constants have
been removed) for calculating the expected communication period (p) takes the form of
Equation (8.1):
p =
(
map
range
)
UAV s
(8.1)
For a specic example, the data used to generate Figures 8.6 and 8.8 was found to have
95% of the variation in observed communication period explained by Equation (8.2).
1
p =
(
map7.08
range0.19
)
1.53
UAV s 0.68
(8.2)
Using this formula it is possible to estimate the threshold at which communication
becomes benecial. Figure 8.9 charts agent density over communication period, and
is coloured based on simulation averages showing when the performance of a swarm,
without communication, is exceeded. It can be seen that there is a trend of the swarm
outperforming the no communication scenario when the ratio exceeds 0.36. While not
a strong heuristic by itself, it demonstrates a correlation between these key values in
regard to swarm performance as well as showing that this crossover point is computable.
8.3.1. Deterministic versus Stochastic Behaviours
Unusual artefacts are found in very low communication scenarios due to the nature of
deterministic swarm algorithms which use homogeneous agents. By denition, as the
agents are both deterministic and homogeneous, each agent will make identical decisions
given the same initial state and inputs; in this case location and environment state.
1
This forumla was calculated by evaluating simulation data using the software tool Eureqa (Schmidt
and Lipson 2009).
81
Figure 8.9.: Agent density divided by communication range. Darker, green nodes repre-
sent a combination of search area size, number of UAVs, and communication
radius, where the swarm outperforms one that does not use communication.
This corresponds fairly closely to a ratio value of 0.36 and greater.
82
Figure 8.10.: Result of adding random noise to the pheromone map in a sparsely popu-
lated environment (16 UAVs, 80 km Uniform map). The percentage shown
is the maximum variance between real pheromone levels and the pheromone
levels used in the individuals heuristic evaluations.
When an agents internal model of the environment (i.e. pheromone map) is the same
as another UAV in close proximity, without frequent updates about that neighbours
activity, the agents will naturally mirror each others decisions and perform repeated
redundant searches. The addition of basic stochastic measures to the algorithm, in
this case applying small random weights to node priorities for each individual UAV,
shows this behaviour almost totally negated (Figure 8.10). Doing this has an overall
performance hit when information ow is high enough to negate this behaviour on its
own, and at zero communication where convergence is an impossibility. Between these
ranges, however, increased randomness shows a related decrease in average search time.
8.4. Communication Failure
In the real world it cannot be assumed that all UAVs will operate ideally at all times.
For example it is possible that environmental dangers may cause a vehicle to crash or
otherwise become non-functional. In this case the performance of the swarm rapidly
degrades to the normal performance that would be expected from the new number of
UAVs. More serious than complete non-functionality, is corrupted behaviour such as
the UAV broadcasting incorrect pheromone information to the other UAVs in reception
83
Figure 8.11.: Time between surveys of high priority nodes as a function of how frequently
one UAV broadcasts random pheromone information. Each plot contains
40,000 data points, outliers represent less than 1% of each data set.
84
Figure 8.12.: The time between surveys of medium priority nodes as a function of how
frequently one UAV broadcasts random pheromone information. Each plot
contains 40,000 data points, outliers represent less than 1% of each data
set.
Figure 8.13.: The time between surveys of low priority nodes as a function of how fre-
quently one UAV broadcasts random pheromone information. Each plot
contains 40,000 data points, outliers representing less than 1% of each data
set.
85
range. This has been simulated by making one UAV periodically broadcast random
pheromone values.
Figures 8.11 to 8.13 show the performance for high (4 pheromone), medium (2)
and low (1) priority regions respectively with a malfunctioning UAV broadcasting with
a swarm size of 40. Note the very dierent vertical scales for these three gures. The hor-
izontal axis shows how frequently one UAV broadcasts random pheromone information.
The last plot, which is labelled as zero iterations between random communication events,
shows the performance when no random pheromone values are ever communicated.
Inspection of these three gures shows a decrease in performance at local priority
areas, particularly in respect of the outliers. This decrease is caused by the high priority
nodes being over surveyed (slightly) as a result of the misinformation. The overall eect
on the majority of points (of any priority) is small and the algorithm seems able to adapt
well to a certain amount of noise being injected into the pheromone maps of the UAVs.
8.5. Summary
This chapter discussed how dierent communication constraints aect swarm perfor-
mance. It has been shown that, outside of fringe cases, the eects of communication
range, broadcast period, and outright communication failure have a limited inuence on
average survey times. Due to the distributed nature of the algorithm, the swarm is able
to function at near maximum capacity even when a low degree of information sharing
between individuals occurs.
The next chapter compares the performance of the algorithm presented in this disser-
tation with an existing state of the art area surveillance algorithm.
86
9. Comparative Results
Of the algorithms simulated to nd a baseline for comparison, the best performing was
one by Erignac (2007). Erignacs algorithm is designed to perform a single exhaustive
pass of an environment, and uses Euclidean distance to the closest unexplored node
as pheromone. Behaviour of the UAVs is determined by a nite state machine, which
defaults to performing gradient descent on the pheromone if no other conditions are met.
This chapter compares the performance of the proposed algorithm with Erignacs.
9.1. Exhaustive Swarming Search Strategy
The nite state machine used to determine UAV behaviour in Erignacs algorithm is
shown in Figure 9.1. Due to an interesting implementation of state-based behaviours,
the search pattern which emerges is highly ecient when communication is continuous.
The state-space that Erignacs algorithm is designed for is one with a uniform level of
priority, where each cell starts in the unexplored state, and needs to be visited at least
once to change it to explored. To be useful as a comparison to the proposed algorithm
in this work, a variant is required.
Firstly, the Euclidean distance pheromone map is used side-by-side with a modied
priority pheromone map which indicates a cell as being in the explored state if its phero-
mone is lower than one. When enough time passes to raise pheromone above one, the
state changes to unexplored. Even when cells are showing as explored, visiting them
during a random move or a repulsion move would reset their pheromone to zero as with
a normal pheromone map.
As absolute pheromone values are needed due to the binary nature of the node states,
the rate of pheromone increase needs to be optimised oine (pre-simulation). When
pheromone increases too slowly the swarm spends an inecient amount of time moving
long distances towards newly triggered nodes. Alternatively, when pheromone increase is
too fast, the inability of UAVs to distinguish old unexplored cells from new unexplored
cells becomes pronounced.
87
Chapter 9: Comparative Results
Figure 9.1.: Flowchart for the behaviour of Erginacs original algorithm, as presented in
U.S. Patent No. 7,856,314 (Erignac United States Patent 7606659, Issued
20, Oct, 2009).
88
Figure 9.2.: Pheromone gradients resulting from the emergent contour following observed
during the execution of Algorithm E using a uniform map.
Explicitly coded countour following behaviour was found to be largely redundant as
simulations show that countour following is an emergent behaviour of both algorithms.
An example of this emergent countour following can be seen in Figure 9.2. Avoidance
behaviour is handled through explicit short duration avoidance movements when UAVs
are within adjacent nodes, enabled by continuous UAV position broadcasts.
The variant of Erignacs original algorithm is referred to as Algorithm E, while the
algorithm presented in this dissertation is Algorithm H.
9.2. Uniform Map
Scenarios for generating comparison data were run in a 50km
2
environment, divided into
20164 (142
2
) nodes. The performance metric used was the length of time since each cell
was last visited, averaged for the whole of the map. This measurement was taken 2000
times and then averaged for each scenario, 200 times per pass of the map. A pass was
dened as the time taken for each cell to be visited at least once.
The rst scenario is an exhaustive and persistent search of an area with uniform
priority. As can be seen in Figure 9.3, with the addition of an explored/unexplored
mechanism though the priority map, the global-scope Euclidean distance pheromone
enables better results for a single UAV, and parity is held until around four agents.
89
Figure 9.3.: Uniform Map - Comparative average mean times of Algorithms E and H,
since cells last visit
After this, with higher agent densities, the emphasis of Algorithm H on local seeking of
pheromone of any value (not only past a threshold) provides a signicant decrease in
mean visit time.
9.3. Lake Map
The second scenario is a priority map used in (Howden and Hendtlass 2008) with three
levels of priority, referred to here as the Lake map (Figure 9.4). Each level is set to twice
the level before it, so the highest priority cells are the small white circles which need to
be surveyed four times as often as the baseline and the light grey squares need to be
surveyed twice as often, resulting in the average survey times for Algorithm H shown in
Figure 9.5. Figure 9.6 shows that, compared to Algorithm E, at any density of UAVs on
the lake map, Algorithm H provides a consistent 25% to 30% decrease in survey times
of the high priority survey areas.
90
Figure 9.4.: Lake map - A priority map featuring four priority levels, inclusive of null
priority areas. Black circles are null priority, light grey increases at a 4
rate, and middle grey at a 2 rate.
Figure 9.5.: Results of simulations run on the priority map shown in Figure 9.4 using Al-
gorithm H. The ratio of average survey time between priority areas remains
consistent with the scaling of UAVs.
91
Figure 9.6.: Lake Map - Mean time since visit for highest priority areas
9.4. No-Fly Zone Map
The third and most complex environment is the No-Fly Zone map shown in Figure 9.7.
This priority map has the same priority ratios as the re map, but with the addition of
no-y areas. An additional obstacle is the addition of complex null priority areas in the
form of spiralling lane ways.
Results for the No-Fly Zone map, shown in Figure 9.8, continue the trend seen in
the rst two environments. With its ability to exploit distant areas of the map, the
comparison algorithm was able to maintain parity for small swarm sizes of one to four,
but was unable to compete with larger swarm sizes. By UAV count 64, Algorithm H has
twice the comparison algorithm performance. Relative performance between priorities
for both algorithms remains similar to what was shown for the lake map.
The reason for the dramatic performance dierence at higher agent densities is in-
dicated by the comparative heat maps of node surveys for the two algorithms shown
in Figure 9.9. While the priority pheromone map allows both algorithms to perform
a continuous search with good results, the ability of Algorithm H to exploit this data
in a continuous, as opposed to binary, manner allows it to optimise its moves to a far
greater extent. As Algorithm Es implementation forces a binary representation to be
used, repeat visits to previously explored cells occur, essentially, at random.
92
Figure 9.7.: No-y zone map - Checkered areas are no-y zones, black is null priority,
light grey increases at a x4 rate, and middle grey at a x2 rate
Figure 9.8.: Mean time since visit for highest priority areas using the no-y zone map
93
Figure 9.9.: Heatmaps of node surveys for the no-y zone map using a) Algorithm E,
and b) Algorithm H
9.5. Analysis of Comparative Results
To its credit, in a search space it was not explicitly designed for, the variant of Erignacs
algorithm used for comparison performed equivalently when only individuals or pairs of
agents were used in two of the three environments. This is due in part to its higher
emphasis on long distance moves, compared to Algorithm H which strongly emphasises
local, short range decisions. The more sparse agent coverage is, the larger the UAVs
decision range needs to be for optimal results. Additionally, the repulsion method used
by Algorithm E was very light handed, which, as discussed in Section 3.2.1, is benecial
when agent density is very low.
In all other scenarios, Algorithm H provided greater average visit frequencies, often
in the range of double or greater. Aside from achieving the primary objective more
eectively, there are two other advantages to Algorithm H over Algorithm E for contin-
uous surveillance missions. Firstly, Algorithm H requires no o-line optimisation and
no adjustment on-the-y to accommodate for lost agents. As it works through relative
pheromone values, the absolute value is unimportant. For an algorithm that imple-
ments the binary abstraction of a priority map, the period between cells switching from
explored to unexplored needs to be calibrated o-line, as an inappropriate or poorly
chosen value prevents the algorithm from functioning correctly. A value which is too
94
high (where agents always move to the closest adjacent cell) or too low (where agents
spend most of their time using the random move behaviour) leads to results no better
than a random search.
The second advantage of Algorithm H is computation time. Utilising a Euclidean
distance pheromone map requires that each cell be populated with the distance to the
closest unexplored cell. This consequently requires the use of a wave front propagation
algorithm every time the map is changed, either via an agents visit, or through com-
munication of an updated pheromone map through the swarm. This is computationally
expensive, and occurs every few seconds in large swarms. Using the priority map algo-
rithm from this dissertation, only the few nodes on the agents immediate path need to
be checked when new information is received, and only a small and constant sized area
of the map needs to be evaluated when a new decision is required. Due to the constant
size of the evaluation area, the heuristics computation time does not increase with map
size, as opposed to an exponential increase for searching the entire map.
An interesting observation is that while the performance is good, the ratios observed
are not the 4/2/1 relationship that was set. Both algorithms are able to maintain
an exact relationship if that is the desired result: for Algorithm H the heuristic is
changed to negate the distance penalty; for Algorithm E the period between explored
and unexplored is increased. The side eect of these changes is that every area performs
worse as the agents spend a disproportionate amount of time in transit chasing global
maxima. The larger a swarm is, the worse this approach becomes as it is rarer that any
individual will be the one to reach the target rst. Even with the current experiment
conguration, the ratios approach their 4/2/1 ideal as the agent count increases, often
being almost exact by 512 UAVs.
9.6. Summary
In the majority of simulated scenarios, including all scenarios involving mid to large
swarm sizes, the proposed algorithm achieves signicantly higher survey frequencies than
the comparison algorithm. In addition to increased mission performance, the proposed
algorithm provides two additional benets. Firstly there is greatly reduced dependence
on pre-mission calibration of variables for optimal performance. Secondly less on-board
computational resources are required to execute the algorithm.
95
10. Conclusions and Final Remarks
10.1. Summary
Bushres are a problem around the world, especially common in countries such as Aus-
tralia where summers can be long, hot, and dry. Eective bushre management includes
the simultaneous goals of minimising risk to re ghters on the ground, and limiting the
potential of the re to spreading out of control. These goals are largely dependant on
surveillance and predictive modelling based on observational data, which is traditional
gained through personnel manually patrolling a res perimeter.
With recent advances in the eld of autonomous robotics, there has been interest in the
use of UAVs in re surveillance roles. For a problem with inherent dangers and in which
swift response is critical, UAVs are uniquely suited for this role due to being rapidly
deployable and relatively expendable. Initial studies with single, non-autonomous drones
have been promising, however problems were found with maintaining adequate coverage.
While adding more drone-type UAVs is largely impractical due to the need to main-
tain enough trained sta to individually operate each vehicle, swarm intelligence provides
mechanisms for coordinating the actions of autonomous entities as a group. In addition,
swarm intelligence enables collaboration and cooperation between individuals without
the need for centralised control, an important consideration when operating in environ-
ments that typically do not have existing communication infrastructure. The advantages
of deploying robots as a swarm instead of a centrally controlled system include massive
scalability, reduced need for human supervision, and resilience against individual failure.
A promising branch of inquiry into swarm robotics has been biomimicry, specically
the use of articial pheromone. The use of pheromone maps to represent the environment
a swarm occupies allows algorithms to combine the information sharing and collective
decision making elements of swarm behaviour into a single intuitive concept. The prob-
lem with pheromone as a concept, however, is that the analogy only holds so long as the
information is stored in the environment itself. With the problem of detecting and then
monitoring bushres in an area without the infrastructure needed to provide long range
97
Chapter 10: Conclusions and Final Remarks
communication there is no immediately apparent medium to store this information.
The algorithm proposed in this dissertation implements a pheromone map to represent
environmental knowledge. A unique copy of this pheromone map is stored in each
UAVs memory. Pheromone levels are a function of information age and priority level,
and a heuristic is used to generate waypoints based on this information. The use of
priority maps allows for pheromone maps to specify areas of the environment to avoid
or prioritise, and also allows this knowledge to be communicated in an intuitive manner.
As priority maps are most often arranged as a matrix, there is a level of redundancy
when searching this grid with a non-square camera. A formula for calculating the optimal
dimensions of the priority map matrix to ensure minimal redundancy while ensuring
exhaustive coverage was found which also incorporates the safeguard of an arrival radius.
An artefact of using optimal node spacings was found to be emergent edge following
behaviour.
UAVs controlled by priority maps have local autonomy, which means they are self con-
tained with regard to decision making. As a result of this, communication constraints
are not pronounced. Outside of fringe cases, the eects of communication range, broad-
cast period, and outright communication failure have a limited inuence on the ability of
UAVs to perform their mission. Due to the distributed nature of the swarm intelligence
algorithms, the swarm is able to function at near maximum capacity even with a low
degree of information sharing among individuals.
With priority maps, the algorithm presented in this dissertation achieved substantially
higher survey frequencies than a comparable eld-leading algorithm. In addition to in-
creased mission performance, there was a greatly reduced dependence on pre-mission
calibration of variables for optimal performance. In addition, fewer computational re-
sources were required to execute the algorithm.
10.2. Research Contributions
This dissertation has made the following contributions to the eld of swarm intelligence:
An algorithm has been developed for performing exhaustive, continuous surveil-
lance of an area using a decentralised UAV swarm. The algorithm uses inter-
nal pheromone maps to both represent the environment and communicate with
the swarm. Simulation results demonstrating that the proposed priority map al-
gorithm shows signicant improvements in average search times compared to a
similar eld leading algorithm.
98
A method for producing avoidance and prioritisation in UAVs using priority maps
has been established. By replacing dynamic pheromone quantities with static
timestamps and scaling the change in time by priority level, computational over-
heads are reduced and new behaviours are made possible.
A lower bound for the positive eects of communication between swarm peers
has been presented. In very low communication scenarios, the decreased diversity
resulting from communication is more detrimental than the eect of cooperation
through shared information.
Outside very low communication fringe cases, the eects of communication range,
broadcast period, and outright communication failure have a limited inuence
on a swarm which prioritises local search. Due to the distributed nature of the
algorithm, the swarms are able to function at near maximum capacity even with
a low degree of information sharing amoung individuals.
An explanation for the emergent edge following found in uniform environments
has been provided, which is applicable to all matrix based pheromone maps which
prioritise movements to adjacent nodes. Analysis of search patterns shows that
discrete decision spaces result in predictable movement patterns in the continuous
environment.
Bounding factors for swarm eciency have been identied. With decision logic
which is fully distributed, search eciency of the swarm increases near-linearly
until such point as UAV density exceeds a calculable threshold. Agent density
thresholds are applicable on both global and local scales, with local agent density
limiting maximum practical relative priority levels.
The relation between monitoring active bushre events and maintaining surveil-
lance for new ignitions was described. By using dynamic priority updates, it is
possible to use priority maps to both closely track re fronts and simultaneously
discover secondary res.
10.3. Future Work
A primary extension for this work would be real world testing using inexpensive kit-built
MAVs. While simulations suggest that priority maps will be an eective technique for
99
swarm coordination, hardware constraints and environmental dynamics will necessitate
platform specic optimisations for real world performance to be realised.
Additionally, and closely related, is the issue of safety verication. For Unmanned
Aircraft Systems (UAS) to become a practical solution to civil applications such as re
spotting, they will require access to non-segregated civil airspace. While autonomy oers
promises of improved capabilities at a reduced operational cost, there are concerns about
being able to verify such autonomous systems (Pecheur 2000).
The route to airframe certication has already been established with regard to tra-
ditional aircraft, however with regard to the decision making logic, which acts as the
aircrafts pilot, the process is still unclear. In this area regulators themselves are still to
reach consensus about what will be required, a process being aided by projects such as
ASTRAEA, ASTRAEA II, SEAS DTC, and FAA UAPO. At a minimum, certication
will likely require verication and validation of unmanned aerial system models against
the rules of the air (Alexander et al. 2007, 2009; Sirigineedi et al. 2009). Any future
work in implementing the proposed algorithm in a UAS will require breakthroughs in
non-deterministic behaviour modelling and safety verication.
A further potential extension of this work would be in extending the priority map
into a third dimension and implementing a test bed of underwater vehicles. Underwater
vehicles share many similarities with aerial vehicles, without the herculean task of safety
certication (Heidemann et al. 2006). Their use would allow for real world results as
well as the potential for new applications such as micro-habitat monitoring, structural
monitoring, and wide-area environmental monitoring.
100
Bibliography
R. D. Alexander, M. Hall-may, and T. P. Kelly. Certication of autonomous systems
under UK military safety standards. In Proceedings Of the 2nd SEAS DTC Technical
Conference, 2007.
R. D. Alexander, T. Kelly, and N. Herbert. Deriving safety requirements for autonomous
systems. In Proceedings of the 4th SEAS DTC Technical Conference, 2009.
Y. Altshuler, V. Yanovsky, I. A. Wagner, and M. Alfred. The cooperative hunters -
ecient cooperative search for smart targets using UAV swarms. In Second Inter-
national Conference on Informatics in Control, Automation and Robotics (ICINCO),
the First International Workshop on Multi-Agent Robotic Systems (MARS), pages
165170, 2005.
V. G. Ambrosia, S. S. Wegener, D. V. Sullivan, and et al. Demonstrating UAV-acquired
real-time thermal data over res. Photogrammetric Engineering & Remote Sensing,
69(4):391402, 2003.
E. M. Arkin, S. P. Fekete, and J. S. B. Mitchell. Approximation algorithms for lawn
mowing and milling. Computational Geometry, 17(1):2550, 2000.
J. Bachrach, J. D. McLurkin, and A. Grue. Protoswarm: a language for program-
ming multi-robot systems using the amorphous medium abstraction. In Proceedings
of the 7th international joint conference on Autonomous agents and multiagent sys-
tems, volume 3, pages 11751178, Estoril, Portugal, 2008. International Foundation
for Autonomous Agents and Multiagent Systems.
D. B. Barber, J. D. Redding, T. W. Mclain, R. W. Beard, and C. N. Taylor. Vision-based
target geo-location using a xed-wing miniature air vehicle. Journal of Intelligent and
Robotic Systems, 47(4):361382, 2006.
101
Bibliography
M. Batalin, M. Hattig, and G. S. Sukhatme. Mobile robot navigation using a sensor
network. In International Conference on Robotics and Automation, pages 636642.
IEEE, 2003.
L. Bayindir and E. Sahin. A review of studies in swarm robotics. Turkish Journal of
Electrical Engineering, 15(2):115147, 2007.
R. Beckers. From local actions to global tasks: Stigmergy and collective robotics. In
Articial Life IV, Proceedings of the Fourth International Workshop of the Synthesis
and Simulation of Living Systems, pages 181189. MIT Press, 1994.
G. Beni. From swarm intelligence to swarm robotics. In Swarm Robotics, volume 3342
of Lecture Notes in Computer Science, pages 19. Springer, 2005.
E. Bonabeau, M. Dorigo, and G. Theraulaz. Swarm Intelligence: From Natural to
Articial Systems. Santa Fe Institute Studies on the Sciences of Complexity. Oxford
University Press, 1999.
S. Camazine. The regulation of pollen foraging by honey bees: How foragers assess the
colonys need for pollen. Behavioral Ecology and Sociobiology, 32(4):265272, 1993.
I. Chabini. Discrete dynamic shortest path problems in transportation applications:
complexity and algorithms with optimal run time. Transportation Research Record:
Journal of the Transportation Research Board, 1645(1):170175, 1998.
X. Chen, S. Gong, N. A. Schmid, and M. C. Valenti. UAV-based distributed ATR
under realistic simulated environmental eects. In Proceedings of SPIE 6567, Signal
Processing, Sensor Fusion, and Target Recognition. International Society for Optical
Engineering, 2007.
P. Cheney and A. Sullivan. Grassres: Fuel, Weather and Fire Behaviour. CSIRO,
2008.
Civil Aviation Authority (UK). Unmanned aerial vehicle operations in UK airspace:
Guidance. In CAP 722. The Stationery Oce, 2012.
E. Sahin. Swarm robotics: From sources of inspiration to domains of application. In
Swarm Robotics, volume 3342 of Lecture Notes in Computer Science, pages 1020.
Springer, 2005.
102
Bibliography
P. Dasgupta. Distributed automatic target recognition using multi-agent UAV swarms.
In Proceedings of the fth international joint conference on Autonomous agents and
multiagent systems, pages 479481. ACM, 2006.
L. Deok-Jin and M. Richard. Decentralized control of unmanned aerial robots for wireless
airborne communication networks. The International Journal of Robotics Research, 7
(3):191200, 2010.
M. Dorigo, V. Maniezzo, and A. Colorni. Ant system: optimization by a colony of
cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics, Part B:
Cybernetics, 26(1):2941, 1996.
F. Ducatelle, A. F orster, G. Di Caro, and L. Gambardella. New task allocation methods
for robotic swarms. In Proceedings of the 9th IEEE/RAS Conference on Autonomous
Robot Systems and Competitions, 2009.
M. Ebling, M. Loreto, M. Presley, F. Wieland, and D. Jeerson. An ant foraging
model implemented on the time warp operating system. In Proceedings of the SCS
Multiconference on Distributed Simulation, pages 2126, 1989.
C. A. Erignac. An exhaustive swarming search strategy based on distributed pheromone
maps. In AIAA Infotech@Aerospace 2007 Conference and Exhibit, 2007.
C. A. Erignac. Exhaustive swarming search strategy using distributed pheromone maps,
United States Patent 7606659, Issued 20, Oct, 2009.
M. Flint, M. Polycarpou, and E. Fernandez-Gaucherand. Cooperative control for mul-
tiple autonomous UAVs searching for targets. In Proceedings of the 41st IEEE Con-
ference on Decision and Control, volume 3, pages 28232828, 2002.
E. Freund and H. Hoyer. Real-time pathnding in multirobot systems including obstacle
avoidance. The International Journal of Robotics Research, 42(7):4270, 1988.
K. Frisch. The dance language and orientation of bees. Science, 159(3817):864865,
1968.
G. Ganewatta and J. Handmer. Bushre management: Why, where and how economics
matter? Proceedings of the Royal Society of Queensland, 112(25278):1218, 2006.
Gentle, Neil; Kierce, Sharyn; Nitz, Alistair. Economic costs of natural disasters in
australia. Australian Journal of Emergency Management, The, 16(2):3843, 2001.
103
Bibliography
P. P. Grasse. La reconstruction du nid et les coordinations interindividuelles chez
bellicositermes natalensis et cubitermes sp. la theorie de la stigmergie: Essai
dinterpretation du comportement des termites constructeurs. Insectes Sociaux, 6
(1):4180, 1959.
J. Heidemann, Wei Y., J. Wills, A. Syed, and Yuan L. Research challenges and applica-
tions for underwater sensor networking. In Wireless Communications and Networking
Conference, pages 228235, 2006.
F. Heppner and U. Grenander. A stochastic nonlinear model for coordinated bird ocks.
In In The ubiquity of chaos, pages 233238, 1990.
N. Hogan. Impedance control: an approach to manipulation. In Proceedings of American
Control Conference, pages 304313. IEEE, 1984.
D. P. Horner. Use of articial potential elds for uav guidance and optimization of wlan
communications. In Autonomous Underwater Vehicles, pages 8895. IEEE/OES, 2004.
D. J. Howden. Continuous swarm surveillance via distributed priority maps. In Pro-
ceedings of the 4th Australian Conference on Articial Life: Borrowing from Biology,
pages 221231. Springer-Verlag, 2009.
D. J. Howden. Fire tracking with collective intelligence using dynamic priority maps. In
Evolutionary Computation (CEC), IEEE Congress on, pages 26102617. IEEE, 2013.
D. J. Howden and T. Hendtlass. Collective intelligence and bush re spotting. In
Proceedings of the 10th annual conference on Genetic and evolutionary computation,
pages 4148. ACM, 2008.
T. Hutchings, S. Jeryes, and S. J. Farmer. Architecting UAV sense & avoid systems
& avoid systems. In Autonomous Systems, pages 18. Institution of Engineering and
Technology, 2007.
C. Intanagonwiwat, R. Govindan, and D. Estrin. Directed diusion: A scalable and
robust communication paradigm for sensor networks. In MOBICOM, pages 5667.
ACM, 2000.
E. Izquierdo-Torres. Collective intelligence in multi-agent robotics: Stigmergy, self-
organization and evolution. In Animal and Machine Intelligence, 2004.
104
Bibliography
M. Jasiunas. Mobility of recongurable computing tasks with applications to swarms of
small UAVs. Masters thesis, University of South Australia, 2009.
Y. Kant, V. K. Prasad, and Badarinath K. V. S. Algorithm for detection of active re
zones using NOAA AVHRR data. Infrared Physics & Technology, 41(1):2934, 2000.
D. Kearney and M. Jasiunas. Using simulated partial dynamic run-time reconguration
to share embedded FPGA compute and power resources across a swarm of unpiloted
airborne vehicles. EURASIP Journal on Embedded Systems, Special Issue(1):18,
2007.
J. Kennedy. Swarm intelligence. In Handbook of Nature-Inspired and Innovative Com-
puting, pages 187219. Springer US, 2006.
J. Kennedy and R. Eberhart. Particle swarm optimization. In Proceedings of the IEEE
International Conference on Neural Networks, volume 4, pages 19421948, 1995.
O. Khatib. Real-time obstacle avoidance for manipulators and mobile robots. The
International Journal of Robotics Research, 5(1):9098, 1986.
B. H. Krogh. A generalized potential eld approach to obstacle avoidance control. In
Proceedings of SME Conference on Robotics Research. Robotics International of SME,
1984.
Y. Kuwana, I. Shimoyama, Y. Sayama, and H. Miura. Synthesis of pheromone-oriented
emergent behavior of a silkworm moth. In Proceedings of the IEEE/RSJ International
Conference on Intelligent Robots and Systems, volume 3, pages 17221729, 1996.
Y. Kuwana, S. Nagasawa, I. Shimoyama, and R. Kanzaki. Synthesis of the pheromone-
oriented behaviour of silkworm moths by a mobile robot with moth antennae as phero-
mone sensors. Biosensors and Bioelectronics, 14(2):195202, 1999.
C. Martin and K. S. Barber. Multiple, simultaneous autonomy levels for agent-based sys-
tems. In Proceedings of the Fourth International Conference on Control, Automation,
Robotics, and Vision, pages 13181322, 1996.
J. D. McLurkin. Algorithms for distributed sensor networks. Masters thesis, University
of California, 1999.
105
Bibliography
J. D. McLurkin. Stupid robot tricks : a behavior-based distributed algorithm library for
programming swarms of robots. Masters thesis, Massachusetts Institute of Technol-
ogy, 2004.
J. D. McLurkin. Analysis and implementation of distributed algorithms for multi-robot
systems. PhD thesis, Massachusetts Institute of Technology, 2008.
J. D. McLurkin and E. D. Demaine. A distributed boundary detection algorithm for
multi-robot systems. In Intelligent Robots and Systems, International Conference on,
pages 47914798. IEEE/RSJ, 2009.
J. D. McLurkin and J. Smith. Distributed algorithms for dispersion in indoor envi-
ronments using a swarm of autonomous mobile robots. In Distributed Autonomous
Robotic Systems Conference, pages 399409, 2004.
J. D. McLurkin and D. Yamins. Dynamic task assignment in robot swarms. In Proceed-
ings of Robotics: Science and Systems, 2005.
H. Mei, Y. Tian, and L. Zu. A hybrid ant colony optimization algorithm or path planning
of robot in dynamic environment. International Journal of Information Technology,
12(3):7888, 2006.
P. M. Miller. Mini, Micro, and Swarming Unmanned Aerial Vehicles: A Baseline Study.
Federal Research Division, Library of Congress, 2006.
I. Millington. Articial Intelligence for Games. Morgan Kaufmann Publishers Inc., 2006.
N. Minar, R. Burkhart, C. Langton, and M. A. Askenazi. The swarm simulation system:
A toolkit for building multi-agent simulations. Technical report, Santa Fe Institute,
1996.
M. E. J. Newman. The structure and function of complex networks. SIAM review, 45
(2):167256, 2003.
P. Olsson, J. Kvarnstrom, P. Doherty, O. Burdakov, and K. Holmberg. Generating
UAV communication networks for monitoring and surveillance. In Proceedings of the
International Conference on Control, Automation, Robotics and Vision, 2010.
K. Osborn. Aerial layer expands Army network during evaluation [press re-
lease]. http://www.army.mil/article/61171/_Aerial_layer__expands_Army_
network_during_evaluation/, 2011. Accessed: 06/01/2012.
106
Bibliography
H. V. D. Parunak. Go to the Ant: Engineering principles from natural multi-agent
systems. Annals of Operations Research, 75(0):69101, 1997.
H. V. D. Parunak. Making swarming happen. In Proceedings of Swarming and Network-
Enabled C4ISR, 2003.
H. V. D. Parunak, S. A. Brueckner, and J. A. Sauter. Digital pheromone mechanisms
for coordination of unmanned vehicles. In Proceedings of the rst international joint
conference on Autonomous agents and multiagent systems: part 1, pages 449450.
ACM, 2002a.
H. V. D. Parunak, M. Purcell, and R. OConnell. Digital pheromones for autonomous
coordination of swarming UAVs. In Proceedings of the AIAAs First Technical Con-
ference and Workshop on Unmanned Aerospace Vehicles, Systems, and Operations,
2002b.
D. Payton, M. Daily, R. Estowski, M. Howard, and C. Lee. Pheromone robotics. Au-
tonomous Robots, 11(3):319324, 2001.
D. Payton, R. Estkowski, and M. Howard. Compound behaviors in pheromone robotics.
Robotics and Autonomous Systems, 44(3):229240, 2003.
D. Payton, R. Estkowski, and M. Howard. Pheromone robotics and the logic of virtual
pheromones. In Proceedings of the 2004 international conference on Swarm Robotics,
pages 4557. Springer, 2005.
C. Pecheur. Verication and validation of autonomy software at NASA. In NASA
Technical Reports Server. Ames Research Center, 2000.
A. H. Purnamadjaja and R. A. Russell. Pheromone communication in a robot swarm:
necrophoric bee behaviour and its replication. Robotica, 23(6):731742, 2005a.
A. H. Purnamadjaja and R. A. Russell. Congregation behaviour in a robot swarm using
pheromone communication. In Proceedings of the Australasian Conference on Robotics
and Automation, pages 17. Australian Robotics and Automaton Association, 2005b.
A. H. Purnamadjaja and R. A. Russell. Guiding robots behaviors using pheromone
communication. Autonomous Robots, 23(2):113130, 2007.
107
Bibliography
A. Restas. Forest re management supporting by UAV based air reconnaissance results of
szendro re department, hungary. In First international Symposium on Environment
Identities and Mediterranean Area, pages 7383, 2006a.
A. Restas. Wildre management at aggtelek national park, hungary: Integrated vegeta-
tion re management. Forest Ecology and Management, 234(Suppl.):257267, 2006b.
C. W. Reynolds. Flocks, herds, and schools: A distributed behavioral model. Computer
Graphics, 21(4):2534, 1987.
R. A. Russell. Robot pheromone communication using vortex ring transmission. Journal
of Bionic Engineering, 6(2):153160, 2009.
C. Sanderson. FPGA computing provides superior performance density for UAV appli-
cations. In COTS Journal, pages 8285, 2003.
J. A. Sauter, R. Matthews, H. V. D. Parunak, and S. A. Brueckner. Performance of dig-
ital pheromones for swarming vehicle control. In Proceedings of Fourth International
Joint Conference on Autonomous Agents and Multi-Agent Systems, pages 903910.
ACM Press, 2005.
F. S. Schill. Distributed Communication in Swarms of Autonomous Underwater Vehicles.
PhD thesis, Australian National University, 2007.
T. Schmickl and K. Crailsheim. Inner nest homeostasis in a changing environment with
special emphasis on honey bee brood nursing and pollen supply. Apidologie, 35(3):
249263, 2004.
T. Schmickl and K. Crailsheim. Trophallaxis among swarm-robots: A biologically in-
spired strategy for swarm robotics. In The First IEEE/RAS-EMBS International
Conference on Biomedical Robotics and Biomechatronics, pages 377382, 2006.
T. Schmickl, C. M oslinger, and K. Crailsheim. Collective perception in a robot swarm. In
Swarm Robotics, volume 4433 of Lecture Notes in Computer Science, pages 144157.
Springer Berlin / Heidelberg, 2007.
M. Schmidt and H. Lipson. Distilling free-form natural laws from experimental data.
Science, 324(5923):8185, 2009.
108
Bibliography
J. Seyfried, M. Szymanski, N. Bender, R. Esta na, M. Thiel, and H. W orn. The I-
SWARM project: Intelligent small world autonomous robots for micro-manipulation.
In Swarm Robotics, volume 3342 of Lecture Notes in Computer Science, pages 7083.
Springer Berlin / Heidelberg, 2005.
F. W. Shaw, A. Chiu, and J. D. McLurkin. Agreement on stochastic multi-robot sys-
tems with communication failures. In Intelligent Robots and Systems, International
Conference on, pages 60956100. IEEE, 2010.
G. Sirigineedi, A. Tsourdos, B. White, and R. Zbikowski. Towards veriable approach to
mission planning for multiple uavs. In PROCEEDINGS of AIAA Infotech Aerospace
and Unmanned Unlimited Conference, pages 600611, 2009.
L. Steels. The Articial Life Roots of Articial Intelligence. MIT Press, Cambridge,
Mass., 1993.
T. Stirling, S. Wischmann, and D. Floreano. Energy-ecient indoor search by swarms of
simulated ying robots without global information. Swarm Intelligence, 4(2):117143,
2010.
K. Sugawara, T. Kazama, and T. Watanabe. Foraging behavior of interacting robots
with virtual pheromone. In Proceedings of the IEEE/RSJ International Conference
on Intelligent Robots and Systems, volume 3, pages 30743079. IEEE, 2004.
U.S. Government Accountability Oce. Defense acquisitions: Restructured JTRS pro-
gram reduces risk, but signicant challenges remain, 2005.
P. Valdastri, P. Corradi, A. Menciassi, T. Schmickl, K. Crailsheim, J. Seyfried, and
P. Dario. Micromanipulation, communication and swarm intelligence issues in a swarm
microrobotic platform. Robotics and Autonomous Systems, 54(10):789804, 2006.
R. Varga. The UAV reloaded: Placing video intelligence on-board. In COTS Journal,
pages 7276, 2003.
C. E. V. Wagner. A simple re-growth model. The Forestry Chronicle, 45(2):103104,
1969.
B. Walter, A. Sannier, D. Reiners, and J. Oliver. UAV swarm control: Calculating
digital pheromone elds with the GPU. Journal of Defense Modeling and Simulation,
3(3):167176, 2006.
109
Bibliography
H. Woern, M. Szymanski, and J. Seyfried. The I-SWARM project. In The 15th IEEE
International Symposium on Robot and Human Interactive Communication, pages
492496, 2006.
H. Yamada, T. Tominaga, and M. Ichikawa. An autonomous ying object navigated
by real-time optical ow and visual target detection. In Proceedings of International
Conference on Field-Programmable Technology (FPT), pages 222227, 2003.
H. Yamaguchi and T. Arai. Distributed and autonomous control method for generating
shape of multiple mobile robot group. In Proceedings of the IEEE/RSJ/GI Interna-
tional Conference on Intelligent Robots and Systems Advanced Robotic Systems and
the Real World, volume 2, pages 800807, 1994.
110
Appendices
111
A. Glossary
ACO : Ant colony optimisation is a metaheuristic optimisation al-
gorithm inspired by foraging behaviour in ant colonies.
Dead reckoning : Dead reckoning is a process of navigation where current loca-
tion is calculated based on a previous determined position.
FPGA : A eld programmable gate array is an integrated circuit de-
signed to be customisable post-manufacturing.
FSM : A nite state machine is a model of computation which re-
quires that the abstract machine be in one of a number of states.
GPS : The global positioning system is a publicly accessible satel-
lite network which provides time and location information to
anyone with a receiver.
INS : An inertial navigation system uses motion and rotation sen-
sors to calculate position, velocity and orientation based on
dead reckoning.
ISR : Acronym for intelligence, surveillance and reconnaissance,
military terminology for battleeld information gathering.
MAV : A miniature aerial vehicle is a subclass of UAVs informally
dened by having dimensions under 1m square.
PSO : Particle swarm optimisation is a metaheuristic optimisation
algorithm inspired by ocking behaviour in birds.
Survey Period : Survey period is the average time between surveys of a given
node. Usually taken as an average for all nodes in a subset
of the environment, this is the primary measure of algorithm
performance used in this dissertation.
Stigmergy : Stigmergy is the indirect communication between members of
a colony through modication of a shared environment.
113
Appendix A: Glossary
UAS : An unmanned aircraft system is a UAV with its attendant
support hardware, such as ground terminals and communica-
tion stations. While still relatively uncommon, this term is
becoming increasingly preferred in regulatory literature.
UAV : An unmanned aerial vehicle is any aircraft without a human
pilot on board.
114
B. Comparison of Currently Deployed
Unmanned Aircraft Systems
Table B.1.: Existing Unmanned Aerial Vehicles of 3m wingspan or less. Speeds shown
are, where known, cruising speed.
Name Weight
(g)
Wingspan
(cm)
Endurance
(m)
Speed
(kn)
Take-o Recovery
WASP 430 72 45 35 Hand Unassisted
Dragon Eye 2700 110 60 19 Catapult Unassisted
RAVEN 1900 140 100 44 Hand Unassisted
Buster 4500 126 240 35 Catapult Net
Sender 4500 120 120 50 Hand Unassisted
PUMA 5900 140 120 45 Hand Unassisted
Silver Fox 12000 239 600 43 Hand Net
Aerosonde 16800 345 1440 50 Catapult Unassisted
ScanEagle 18000 205 1200 65 Catapult Net
Fulmar 19000 310 480 54 Catapult Net
115
C. Publications Arising from this Study
Publications by the Candidate Relevant to the
Dissertation
D. J. Howden and T. Hendtlass. Collective intelligence and bush re spotting. In Pro-
ceedings of the 10th annual conference on Genetic and evolutionary computation, pages
4148. ACM, 2008.
D. J. Howden. Continuous swarm surveillance via distributed priority maps. In Pro-
ceedings of the 4th Australian Conference on Articial Life: Borrowing from Biology,
pages 221231. Springer-Verlag, 2009.
D. J. Howden. Fire tracking with collective intelligence using dynamic priority maps.
In Evolutionary Computation (CEC), IEEE Congress on, pages 26102617. IEEE, 2013.
Additional Publications by the Candidate Relevant to
the Dissertation but not Forming Part of it
K. Bogdanov, D. J. Howden, T. Dodd. Automated Model Inference for Autonomous
UAV Systems (manuscript in preparation for journal publication)
117

Bushfire Surveillance Using Dynamic Priority Maps and Swarming Unmanned Aerial Vehicles

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bushfire Surveillance Using Dynamic Priority Maps and Swarming Unmanned Aerial Vehicles

Uploaded by

Copyright:

Available Formats

Bushfire Surveillance Using Dynamic

Priority Maps and Swarming Unmanned

and optical 10zoom. Using these specications,

2)(CR AR) (4.1)

E. b) A corresponding priority map, where each pixel

You might also like