Sense and Vision Based Obstacle Avoidance

Algorithm for Autonomous UAVs

B.M.T.S. Balasooriya#1, D.P.Liyanage#2, H.W.T.C. Yasarathna#3 ,K.D.I. Udayasanka #4, P.G.S.R Wijethunga#5
Department of Electrical and Information Engineering, University of Ruhuna
Hapugala, Galle, Sri Lanka
1 bmtanushka@gmail.com
2 l.dilini@yahoo.com
3tharindu.y.lk@ieee.org
4 kdiudayasanaka@gmail.com
5ranjan@eie.ruh.ac.lk

Abstract In recent years there has been an increase of interest

related to obstacle avoidance techniques for unmanned aerial
vehicles (UAVs). This paper describes a sense and vision-based
algorithm for autonomous UAVs for collision avoidance with
unforeseen obstacles using a vision sensor. The design platform
used, is a common type of UAV, called quad copter. Our
approach uses fuzzy logic to determine an appropriate avoidance
manoeuvre after a possible collision is detected. The image from
the camera is processed on-board and the information is send to
the fuzzy logic controller which then sends commands to modify
the orientation of the UAV.

because the necessary hardware was available and that the

concept of fuzzy logic suits the controlling task well.
Quad copters has to have some sort of sensors to keep
track of how its moving and we use a 3-axis accelerometer, a
3-axis gyroscope and a 3-axis barometer. A quad copter is
shown in Fig 1.

Keywords unmanned aerial vehicle, collision avoidance, quad

copter, fuzzy logic, vision-based

I. INTRODUCTION
UAV is an aircraft which can operate without a human pilot.
UAVs can be remote controlled or can be flew autonomously
based on pre-defined flight plans or more complex dynamic
automation systems [3]. Last decades, due to their potential
use on military and civil applications, such as missions search
and rescue, surveillance of disaster stricken areas, and
battlefield assessment, UAVs became considerably popular.
The main inhibitor of general acceptance of UAVs for
commercial applications is the danger of collisions with other
aircrafts and terrestrial structures because there is not enough
advancement in autonomous sense and avoids technologies
for UAVs. This paper presents the development and
implementation of a sense and vision based algorithm to
detect and avoid obstacles for UAVs in order to make UAVs
more commercially viable [2].
A. Quad Copter
Quad copters usually fall under the category UAV. It is a
rotorcraft machine that has four rotors to generate lift and
propulsion. In a typical quad copter design the propellers are
located at the same distances from each other on a cross [1].
For this project the decision to build a quad copter was made

Fig. 1 Quad copter

B. Fuzzy Logic
Fuzzy logic is a form of logic that aims to resemble human
heuristic knowledge about how to control a system. In
contradiction to ordinary two-value logic, fuzzy logic uses
reasoning where logic variables have a truth value in the range
between 0 and 1. A system that uses fuzzy logic is called a
fuzzy controller. A typical Fuzzy controller design is shown in
fig 2.

Fig. 2 Fuzzy controller system

Fuzzification is the process of converting the input values

to the controller to fuzzy values that can be used by the
inference mechanism. Every input (linguistic variable) has
several membership functions that each represents a linguistic
term. This set of functions is called the fuzzy set of the
linguistic variable. The membership degrees of every
linguistic variable is calculated with respect to the linguistic
terms.
The rule base is the set of rules that are used to calculate
the firing strength of each individual membership function, i.e.
how much the condition part of each rule is satisfied in the
current situation.
The inference mechanism is the part of the fuzzy system
that emulates human decision making in applying the rules
from the rule base. The recommendations from the individual
rules are combined to get the aggregated values by using
different equations.
Finally, the defuzzification process converts the
aggregated values from the inference mechanism to crisp
values usable outside of the fuzzy controller.
II. LITERATURE REVIEW
The main restriction for the general acceptance of UAVs
for commercial applications is the danger of collisions with
other aircrafts and terrestrial structures because there is not
enough advancement in autonomous sense and avoids
technologies for UAVs. Due to the heavy, expensive, and
energy consuming nature of the sensors necessary for
detection and analysis of obstacles, it is difficult to implement
an optimum technology which balances both performance and
restrictions in payload, energy consumption and cost [4].
Due to importance of the topic, many organizations and
universities are conducting research on obstacle avoidance
capability for the UAVs. Recently, the research community
has shown increased interest in developing the obstacle
avoidance capability for quad copters, a type of UAV, as the
quad copter has the ability to hover and is capable of
instantaneously changing direction, eliminating the need to
incorporate minimum radius turning into the algorithm [1].
III. SOFTWARE DESIGN
The flow chart of the proposed obstacle avoidance algorithm
is shown in Fig 3.
A. Image Processing
1) Image Acquisition and Noise Filtering: The sensing array
continuously measures the distance to the obstacle. When the
distance exceed the safe distance of 1m image acquisition
using a camera mounted on the quad copter, is started. Then
the image is processed frame by frame until the obstacle is
avoided.
Images are corrupted by random variations in intensity
called noise due to non-perfect camera acquisition or
environmental conditions. As salt and pepper noise is one of
the form of noise present in the image, median filter is used to
remove these noises in the acquired image.

Fig. 3 Flow chart of the implemented algorithm

2) Calculating Minimum Pixel Distance to the Obstacle

from the Left and Right Borders of the Image: Calculate the
pixel distance to the nearest edge of the obstacle from each
pixel in the currently processing image frame. QuasiEuclidean distance method is used to find the pixel distance.
Quasi-Euclidean distance between ((x1,y1) and (x2,y2) is,

(1)

Otherwise
(2)

The original binary image and the plot of the Euclidean

distance against the intensity are shown in Fig 4 and 5
respectively.

r = the minimum distance to the right border of the image

from the closest edge of the object
f = 320-(maximum among 1 and r) [i.e the shortest distance
to avoid the obstacle]
4) Pseudo Code of the Image Processing Procedure:

Fig. 4 Original binary image

Fig. 5 Each pixels distance to the closest object mapped to intensity

Among above calculated distances, the minimum distance to

the left border and the right border from the object is chosen
for the further processing.
3) Determine the Shortest Path to Avoid the Obstacle:
Maximum distance from the left distance and right distance is
chosen. The direction to which the quad copter should be
turned can be determined according to the above result.
Fig 6 describes how the measurements are taken for
processing purposes.

f
l

Fig. 6 Measurements taken from an image

Pixel width of the image = 320

The measurements to right are taken as positive
l = the minimum distance to the left border of the image from
the closest edge of the object

d= get distance to object

while d <= 1m
while output!= 0
I = read the image
Convert the image (I) in to binary
image using threshold value
Filter the image (I) using Median
filter
Get the distance to the closest
edge of the object from each pixel,
in the binary smoothed image (I)
l = the minimum distance to the
left border of the image from
the
closest edge of the object
r = the minimum distance to the
right border of the image from the
closest edge of the object
l = frame width l
r = frame width r
if l < r
output = -l
else
output = r
end if
end while
end while
Further to calculate accurate pitch angle, other than the
distance, current speed and the direction of the quad copter is
also needed. The time taken to process an image and the
difference in calculated pixel distances is used to determine
the speed.
B. Fuzzy Logic
1) Fuzzification: Three membership functions for both the
inputs, distance and speed, are defined to generate a total of 9
Fuzzy rules. The linguistic terms used are:
P-Positive
Z-Zero
N-Negative
Three membership functions of input variable distance is
shown in Fig 7 and three membership functions of input
variable speed are shown in Fig 8.

2) Inference: Once the rule matrix is filled with all the

Degree of Memberships from the fuzzifier, the aggregated
value is calculated in the inference. The MAX-MIN method
used to aggregate. It tests the magnitudes of each rule and
selects the highest one. The horizontal coordinate of the
"fuzzy centroid" of the area under that function is taken as the
output.

Fig. 7 Membership functions of input variable distance

3) Defuzzification: The defuzzification of the data into a

crisp output is accomplished by combining the results of the
inference process and then computing the "fuzzy centroid" of
the area. The weighted strengths of each output member
function are multiplied by their respective output membership
function center points and summed. Finally, this area is
divided by the sum of the weighted member function strengths
and the result is taken as the crisp output [5].
()+()+()
++

Fig. 8 Membership functions of input variable speed

Membership functions defined for the output of the Fuzzy

controller, pitch angle, is shown in Fig 9.

(3)

NC-negative center
NS- negative strength
ZC-zero center
ZS-zero strength
PC-positive center
PS-positive strength
Fig 10 shows an example of how the Fuzzy crisp output of
the pitch angle is determined for some input values of distance
and speed.

Fig. 9 Membership functions of output pitch angle

TABLE I
FUZZY LOGIC RULE SET

Fig. 10 Example of Fuzzy inputs and corresponding Fuzzy output

The plot of pitch angle corresponding to the variation of

speed and distance is shown in Fig 11.

l=8
r = 165
f = 155
Output pitch angle = 8.94
Above pitch angles are calculated for the zero speed input.
Table II shows some testing results obtained for various
distance and speed combinations.
TABLE II
TESTING RESULTS

Test
No.

1
2
3
4
5
6
7
8

Fig. 11 Control surface for the fuzzy logic controller

IV. TESTING AND RESULTS

Tests were conducted using the quad copter test bench. A
typical cube shaped object was selected as the object to avoid.
As mentioned earlier only the pitch angle was controlled to
avoid the object. The information used by the pitch-controller
is the visual information from the camera and ultra sonic
sensors.
Fuzzy controller system was implemented in Atmega 2560
controller board. Image processing was done using Matlab
R2010a software. Fig 12 and 13 show two instances of images
captured by the camera and further processed.

Fig.12 Object near the right most border of the image

l = 141
r = 32
f = -179
Output pitch angle = -9.24

Fuzzy Input
Distance

Speed

-246.60
-199.60
-152.70
-41.10
-35.23
140.90
205.50
229.00

-140.90
22.73
280.70
-313.60
22.73
-313.00
-40.45
131.80

Fuzzy
Output
Pitch
Angle
-4.19
-9.12
-9.18
6.85
-2.26
9.09
9.01
4.50

V. CONCLUSION
This paper presented a UAV visual and sensor based
approach for sense-see-and-avoid. A Fuzzy Logic controller
has been developed to automate the collision avoidance. This
controller acts changing the pitch angle of the aircraft, until
the object can be overtaken. Excellent results have been
obtained in real tests with a quick response and low error
estimation.
In our paper we have present an optimized algorithm
rather than other algorithms. In this procedure it can be used
to build in on-board UAVs. Current developed algorithms are
developed for off-board because of the energy limited
problems. But in our algorithm it uses less energy and can be
use on-board UAVs. If there are signal crashes, off-board
vision based collision avoidance systems can be crashed due
to communication loss. But in our procedure those things will
not happen although there are signal crashes.
We are in the process to extending this approach by adding
altitude control. Also, we plan to study different optimization
algorithms to improve the controller.
REFERENCES
[1]

[2]
Fig. 13 Object near the left most border of the image

[3]

Rose M., Wagster J., and Yaralian H.: Obstacle Avoidance System for
a Quadrotor UAV: http://www.csupomona.edu/~cppsrc/abstracts/
wagster-rose-yaralian-summary.pdf
M. stberg A.Widenius P. Whlin M. Norgren, Quadrocopter
balancing in one axis using fuzzy logic, Mlardalen University, 2011
Steven D. Kaehler (2006) Fuzzy Logic Tutorial. [Online]. Available:
http://www.seattlerobotics.org/encoder/mar98/fuz/fl_part6.html#INTR
ODUCTION

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

M. A. Olivares-Mendez, L.Mejias, P.Campoy, L.Mellado-Bataller,

Quadcopter See and Avoid Using a Fuzzy Controller, in
Proceedings of the 10th International FLINS Conference on
Uncertainty Modeling in Knowledge Engineering and Decision
Making (FLINS 2012), World Scientific, Istanbul, Turkey
Ortiz, A.E. & Neogi, N., 2006. Color Optic Flow: A Computer Vision
Approach for Object Detection on UAVs. In: Twenty fifth Digital
Avionics Systems Conference, 2006 IEEE/AIAA, October 15-19, 2006,
Portland, OR.
Kwang, Y.K., & Kwang, Y., 2005. Performance Simulation of Radar
Sensor Based Obstacle Detection and Collision Avoidance for Smart
UAV. In: Digital Avionics Systems Conference, October 30
November 3, 2005, South Korea.
Shim, D., & Sastry, S., 2007. An Evasive Maneuvering Algorithm for
UAVs in See-and-Avoid Situations. In: American Control Conference,
July 9-13, 2007, New York.
Crescenzi, T., et al., 2011. The Problem of Collision Avoidance in
Unmanned Aerial Vehicles. In: IEEE International Instrumentation
and Measurement Technology Conference, June 21-26, 2011, Brisbane,
Australia.
Nagy, Z., et al., 2012. Volume and power optimized high-performance
system for UAV collision avoidance. In: Circuits and Systems (ISCAS),
2012 IEEE International Symposium, May 20-23, 2012, Seoul, Korea.
Mejias, L., et al., 2010. Vision-based detection and tracking of aerial
targets for UAV collision avoidance. In: Intelligent Robots and
Systems (IROS), 2010 IEEE/RSJ International Conference, October
18-22, 2010, Taipei.
Zarandy, A., et al., 2001. Visual sense-and-avoid system for UAVs. In:
Cellular Nanoscale Networks and Their Applications (CNNA), 2012
13th International Workshop, August 29-31, 2012, Turin.
Zsedrovits, T., et al., 2011. Collision avoidance for UAV using visual
detection. In: Circuits and Systems (ISCAS), 2011 IEEE International
Symposium, May 15-18, 2011, Rio de Janeiro.