Proceedings of Intelligence

Computational the 2nd IFAC Conference

and oninEmbedded
Telematics Control Systems,
Computational
Proceedings
June of Intelligence
of
22-24, 2015.
Proceedings the 2nd
2nd IFAC
theMaribor,
IFAC and Telematics
Conference
Slovenia
Conference oninEmbedded
on Control Systems,
Embedded Systems,
June 22-24, 2015.
Computational
Computational Maribor, Slovenia
Intelligence
Intelligence and
and Telematics
Telematics in
in Control
Available online at www.sciencedirect.com
Control
June
June 22-24,
22-24, 2015.
2015. Maribor,
Maribor, Slovenia
Slovenia

ScienceDirect
IFAC-PapersOnLine 48-10 (2015) 016–021
Embedded Image Processing Systems
Embedded Image Processing Systems
for Automatic
Embedded
Embedded Recognition
Image Processing
Image of Cracks
Processing using UAVs
Systems
Systems
for Automatic Recognition of Cracks using UAVs
for
for Automatic
Automatic Recognition
Recognition of of Cracks
Cracks using
using UAVs
UAVs
Fábio Celestino Pereira, Carlos Eduardo Pereira
Fábio Celestino Pereira, Carlos Eduardo Pereira
Fábio
Federal University
Fábio Celestino
of Rio Grande Pereira,
Celestino Carlos
Carlos Eduardo
do Sul - UFRGS/
Pereira, Pereira
DELET -Av.
Eduardo Osvaldo Aranha, 103
Pereira
Federal University of Rio Grande do Sul - UFRGS/ DELET -Av. Osvaldo Aranha, 103
Federal
Federal University
CEP: 90035-190 of
of Rio
- Porto Alegre,
University Grande
RioRS do
– Brazil
Grande Sul
Sul -- UFRGS/
UFRGS/ DELET
DELET -Av.
-Av. Osvaldo
do (e-mail:fabio.celestino@ufrgs.br, Aranha,
Aranha, 103
cpereira@ece.ufrgs.br)
Osvaldo 103
CEP: 90035-190 - Porto Alegre, RS – Brazil (e-mail:fabio.celestino@ufrgs.br, cpereira@ece.ufrgs.br)
CEP:
CEP: 90035-190
90035-190 -- Porto
Porto Alegre,
Alegre, RS
RS –– Brazil
Brazil (e-mail:fabio.celestino@ufrgs.br,
(e-mail:fabio.celestino@ufrgs.br, cpereira@ece.ufrgs.br)
cpereira@ece.ufrgs.br)
Abstract: Aiming at the use of Unmanned Aerial Vehicle (UAV) in civil construction for autonomous
Abstract:
inspection Aiming of building at thepathologies,
use of Unmanned this paper Aerial Vehicle some
discusses (UAV)implementation
in civil construction for autonomous
alternatives of image
Abstract:
inspection
Abstract: Aiming
of
Aiming at
building
at the
the use
use of
pathologies,
of Unmanned
Unmannedthis Aerial
paper
Aerial Vehicle
discusses
Vehicle (UAV)
some
(UAV)
processing algorithms for the detection of cracks in building facades. These algorithms should in civil
implementation
in civil construction for
for autonomous
alternatives
construction of
runimage
autonomous
in an
inspection
processing
inspection of
of building
algorithms
building pathologies,
for the detection
pathologies, this
thisof paper
cracks
paper discusses
in building
discusses some
facades.
some
embedded computing platform installed on UAVs. Two image processing algorithms for crack detection implementation
These
implementation alternatives
algorithms should
alternatives of
run
of image
in an
image
processing
embedded
processing
and algorithms
computing
algorithms
classification were for the
the detection
platform
forselected installed
detection of
ofoncracks
and different UAVs.
cracks in building
inTwo image
building
embedded facades.
processing
facades.
system These
These
implementationsalgorithms
algorithms
algorithms forshould
crack
should
were run in
in an
detection
run
evaluated. an
A
embedded
and
embedded
version computing
classification
computing
of the were
algorithms platform
selected
platform
running installed
anda Matlab
installed
in on
differentUAVs.
on UAVs. Two
embedded image
Two image
environment system
on processing
aprocessing
desktop algorithms
implementations
algorithms
computer, for
were
which crack detection
evaluated.
for crack
can bedetection
used in A
and
versionclassification
andapproach
an of theonalgorithms
classification whichwere
were theselected
running
selected and
in
image processing different
anda Matlab
different embedded
is environment
embedded
done system
on a desktop
system
in the ground and implementations
thecomputer,
implementations
UAV is only were
which
were
appliedevaluated.
can be
forused
evaluated.
image A
in
A
version
an of
approach
version of the
acquisition, theon
was algorithms
which
algorithms the running
image
used as arunning in
baselineinfor a Matlab
processing
a Matlab is environment
done in
environment
the comparison. the on
grounda desktop
and
Thisonbaseline
a desktop the computer,
UAV is
computer, with
is compared which
only
which can
applied be
forused
image
can be used in
implementations in
an
an approach
acquisition,
approach
running on an on
was which
used as
on embedded
which the
thea image
baseline
image
processor processing
for
(a the
processing is
is done
comparison.
Raspberry donePI, in
in the ground
This
the
running and
and the
baseline
ground UAV
is compared
the
a distribution UAV of is only
withapplied
isDebian
only ARM for image
implementations
applied for image
operating
acquisition,
running
acquisition,
system) on was
an
and was used
embedded
withused as a baseline
processor
as a baselineon
implementations for
(a the
fora the comparison.
Raspberry
comparison.
Xilinx PI,
FPGA-board This
running baseline
a
Thisinstalled is
distribution
baselineinis the compared
comparedof with
Debian implementations
ARM operating
with implementations
UAV. Different scenarios for
running
system)
running
the on
on an
and
execution an ofembedded
withembedded
inspection processor
aimplementations
processor
task of (a
(aa Raspberry
onbuilding
Xilinxfacades
Raspberry PI,
FPGA-board running
PI, were
running aa distribution
installed
defined in the
distribution
and UAV.of
of Debian
the obtained Different
Debian
resultsARM
ARM operating
scenarios
are operating
presentedfor
system)
the
system)
in the paper.and
execution
and with
of
with aimplementations
inspection
implementations task ofon
on a
a Xilinx
building
Xilinx FPGA-board
facades
FPGA-boardwere installed
defined
installed andin
in the
the
the UAV.
obtained
UAV. Different
results
Different scenarios
are presented
scenarios for
for
the
in
thetheexecution
paper. of
execution of aa inspection
inspection task task ofof building
building facades
facades werewere defined
defined and and thethe obtained
obtained results
results areare presented
presented
in
in the
the paper.
Keywords: paper.Unmanned Aerial Vehicle, Image Processing , Embedded System, Cracks detection.
© 2015, IFAC
Keywords: (International
Unmanned Aerial Federation
Vehicle, of ImageAutomatic Control)
Processing Hosting bySystem,
, Embedded ElsevierCracks
Ltd. All rights reserved.
detection.
Keywords: Unmanned Aerial Vehicle, Image Processing
Keywords: Unmanned Aerial Vehicle, Image Processing , Embedded System, Cracks detection., Embedded System, Cracks detection.

1. INTRODUCTION structure. These UAVs have the disadvantage of requiring

1. INTRODUCTION structure.
runway or Theselauncher UAVs have the disadvantage
for takeoff-landing and not beingof requiring
able to
1. INTRODUCTION structure.
runway
structure.
hover. or
On These
launcher
These
the UAVs
UAVs
otherfor have
have
hand, the
takeoff-landing
the
rotary disadvantage
and
disadvantage
wing not
UAVs of
being
of requiring
ablethe
to
requiring
have
Monitoring and maintenance
1. INTRODUCTION of civil engineering artifacts,
Monitoring and maintenance of civil engineering artifacts, runway
hover. or
On launcher
the otherfor takeoff-landing
hand, rotary and
wing not
UAVs being able
able to
have the
such as buildings and bridges, is an expensive yet essential runway advantage or oflauncher
being ablefor takeoff-landing
to hover, takeoff andandnotland
being to
vertically
suchMonitoring
as buildings and maintenance
and bridges, of
is civil
an engineering
expensive yet artifacts,
essential hover.
advantage
hover. On
On of the
being
the other
otherable hand,
to
hand, rotary
hover,
rotary wing
takeoff
wing UAVs
and
UAVsland have the
vertically
have the
taskMonitoring
in maintaining and maintenance
a safe operation. of civil engineering
Traditional artifacts, with agile maneuvering capability at the expense of high
monitoring
such as buildings advantage
agile of being
being ableable to hover, takeoff and land vertically
task
such in
methods use visualand
as maintaining
buildings and bridges,
ainspection
safe
bridges, is
is an
operation.
on an expensive
Traditional
expensive
a regular basisyet
yet essential
monitoringoften with
essential
and advantage
mechanical ofmaneuvering
complexity, to capability
lowhover,
speed at
andtheshort
takeoff expense
and land ofrange.
high
vertically
flight
task
task in
methods
in maintaining
use visual
maintaining a
a safe
inspection
safe operation.
on
operation. a Traditional
regular
Traditionalbasis
require inspectors to travel to the artifact to be investigated [ZHAO 2007]monitoring
and
monitoringoften with
with agile
mechanical
agile maneuvering
complexity,
maneuvering capability
low speed
capability at
and
at the
the expense
short flight
expense of
of high
range.
high
methods
require
methods use
use visual
inspectors
visual to inspection
travel
inspection to on
the
on a regular
artifact
a regular to basis
be
basis and often
investigated
and often mechanical
[ZHAO
mechanical2007] complexity,
complexity, low
low speed
speed and
and short
short flight
flight range.
range.
and determine its deterioration level. Automation of this
require
and inspectors
determine its to travel to
to the
deterioration artifact
thelevel. to be
be investigated
Automation [ZHAOFig. 2007]
1 presents one of RUAV available on the lab of the
require
process inspectors
may result to
in travel
great monetary artifact
savings to and can of leadthis
investigated to [ZHAO Fig.
2007]
1 Automation,
presents one of RUAV available on the lab of the
and determine
process may result
and determine
more frequent its
its indeterioration
inspection great monetary
deterioration
cycles level.
level.
[ABDEL Automation
savings and
Automation
et ali. can of
2003]. of this
leadthis
to Control, and Robotics Group (GCAR) at
process may result in great monetary savings and can lead to Fig.
Control,
Fig.
UFRGS, 11which
presents
Automation,
presents is one in
one
used of
and
of RUAV
RUAV
this work. available
Robotics
availableGroupon the
on the lab of
(GCAR)
lab of the
the
at
more frequent inspection cycles [ABDEL et ali. 2003].
process may result in great monetary savings and can lead to Control, Automation, and Robotics Group (GCAR) at
more frequent UFRGS, which is used in this
increasing Control, Automation, and Robotics Group (GCAR) at work.
more frequent inspection
Motivated by recentcycles
inspection cycles [ABDEL
[ABDEL et
developments and
et ali. 2003].
ali.an
2003]. UFRGS, which is used in this work.
Motivated
availability by recent
of UAVs developments
(Unmanned and an aincreasing
Aerial Vehicle) growing UFRGS, which is used in this work.
Motivated
availability
Motivated
number of by
UAVs recent
by recenthave
of applications developments
(Unmanned
developmentsAerial
been developed and
Vehicle)
and an anover aincreasing
growing
increasing
the last
availability
number of of
availability
years, ranging of UAVs
UAVsfrom(Unmanned
applications have beenAerial
(Unmanned
inspections developed
Aerial Vehicle)
Vehicle)
of transmission over aa growing
thelines,
growing last
number
years,
number of
of applications
ranging from
applications have
have been
inspections
been developed
of over
transmission
developed
monitoring of fractionation towers in refineries, among others over the
the last
lines,
last
years,
years, ranging
monitoring of
ranging from
fractionation
from inspections
towers
inspections
[DONG et ali. 2012], [DENG et ali 2014]. In civil in of transmission
refineries,
of among
transmission lines,
others
lines,
monitoring
[DONG
monitoring
engineering etof fractionation
ali. 2012], [DENG
ofconstructions
fractionation towers
several in
towers etrefineries,
inrecent ali works
refineries, among
2014].
amonghaveIn others
civil
others
been
[DONG
engineering
[DONG
developed, et ali.
et some
ali. of 2012],
constructions
2012], [DENG
several recent
[DENG
them focusing et
et the
on ali 2014].
ali works
2014].
use of have In
In been
UAVs civil
civil
for
engineering
developed, constructions
some of
engineeringofconstructions
inspection them
bridges, viaducts several
focusing
severaland recent
on
recent the works
use
roadsworks of
[METNI have
UAVs
haveetbeenbeen
for
ali.
developed,
inspection
developed,
2006],[ some
of of
bridges,
some of them
RATHINAM them focusing
2006]. on the use of UAVs for
viaducts
et ali.focusing and on the
roads use of
[METNI UAVs et ali.
for
inspection
2006],[ of
RATHINAM bridges,
inspection of bridges, viaductset viaducts
ali. 2006].andand roads
roads [METNI
[METNI et et ali.
ali.
2006],[ RATHINAM et Fig. 1 - RUAV available at GCAR-UFRGS.
The abbreviation
2006],[ RATHINAM UAVs et ali.
is 2006].
ali. a generic term that identifies an
2006]. Fig. 1 - RUAV available at GCAR-UFRGS.
The abbreviation UAVs is a generic
aircraft that can fly without a crew, usually designed toterm that identifies an
The abbreviation
abbreviation
aircraft that can fly UAVs is aarepetitive
without generic
a crew,term term that identifies
usually identifies
designed an
to One Fig.
of the
Fig. 11 --major
RUAV available
available at
RUAVintegrated GCAR-UFRGS.
equipment
at GCAR-UFRGS. in RUAVs are
The
operate in dangerous UAVs andis generic that
situations in regions an One of the major integrated equipment inmayRUAVs
aircraft
operate that can
in hostile
dangerous fly without a
and repetitive crew, usually designed to cameras to capture images and video, which assist are
the
aircraft
consideredthat can fly without
or difficult crew, situations
toaaccess. usually in can
UAVsdesigned regions to cameras
be One
One of
to
of the
capture
the major
major integrated
images and
integrated equipment
video, which
equipment in
inmayRUAVs
assist
RUAVs are
thea
are
operate
consideredin dangerous
hostile and repetitive situations in regions operator during a controlled flight, for instance to track
operate ininto
classified twoormain
dangerous difficult to access.fixed
andcategories:
repetitive UAVs
situations
wing in and can
regions
rotarybe cameras
particular target object, or also to allow a later analysis of the
operator
cameras to capture
during
to capture a images
controlled
images and
and video,
flight,
video, forwhich may
instance
which may toassist
track
assist thea
considered
classified
considered hostile
into
hostiletwoor difficult
main
orwing
difficult to
to access.
categories:
access. fixed UAVs
wing and
UAVs can
of rotary
can be
be operator during aa at
controlled flight, for
wing UAVs. Fixed UAVs have the advantage being particular
operator
images, target object,
during
usually or also
controlled
the to allow
flight,
ground. for
By a instance
later to
analysis
instance
embedding to track
of theaa
track
image
classified
wing
classifiedflyinto
UAVs. into two
atFixed
two main
highwing
main categories:
UAVs for have
long the
categories: fixed
fixed wing
advantage
wing and
and rotary
ofsimpler
being particular
rotary target object, or
able
wing
to speeds duration with images,
particular usually
processing target at the
object,
algorithms ortoalsotheto
also to allow
ground. allow
UAV,Byaa later analysis
embedding
later
the analysis
possibilityof
of the
image
the
of
wing UAVs. Fixed wing UAVs have the advantage of being
able UAVs.
to fly atFixed
high wing
speeds UAVs for have
long the advantage
duration with of
simpler
being images,processing
images, usually
algorithms
usually at
at the
the to ground.
the
ground. UAV,By
By embedding
the possibility
embedding image
of
image
able
able to
to fly
fly atat high
high speeds
speeds for for long
long duration
duration with with simpler
simpler processing algorithms to the UAV, the possibility of
Copyright © 2015 IFAC 16 processing algorithms to the UAV, the possibility of
Copyright © 2015 IFAC 16
2405-8963
Copyright ©©2015,
2015 IFAC
IFAC (International Federation of Automatic Control)
16 Hosting by Elsevier Ltd. All rights reserved.
Copyright
Peer review©under
2015responsibility
IFAC 16 Control.
of International Federation of Automatic
10.1016/j.ifacol.2015.08.101
 CESCIT 2015
June 22-24, 2015. Maribor, Slovenia Fábio Celestino Pereira et al. / IFAC-PapersOnLine 48-10 (2015) 016–021 17
performing the image analysis during the flight may
introduce greater intelligence to the system in order to
automate the interpretation of these images producing
important data for certain application.
Among the aspects that impact the use of RUAVs in
particular applications one can mention their capacity of
flying autonomously, the maximum flight duration (which
restricts the distance to be traveled), maximum carried load
(which directly impacts the maximum flight duration),
between other factors.
To allow their autonomous operation in open areas UAVs
embedded electronics usually include a navigation system
with Global Positioning System (GPS) as the primary means
of positioning and reference. For indoor operation where the
reference of the GPS signal is not possible, studies have been
done for the use of other positioning strategies, such as the
use of images captured by the camera as an aid in navigation
path, landing and take-off and to avoid collision with
obstacles [WANG 2007] [LANGE 2009][JENSENA 2010].
In civil engineering, studies of building pathologies are of
great importance, because the occurrence of pathological
manifestations may have severe consequences and even lead
to the collapse of buildings [HARRIS 2001]. Cracking may
impair the durability of concrete by allowing immigration of
external aggressive agents; therefore, crack monitoring is
always a vital part in building pathology [SHAM 2008]. Such
applications usually require the analysis of areas of difficult
access, located at high altitude. Therefore, the observation of
these manifestations can become arduous, and the
professional needs to use scaffolding or platforms to be able
to access these regions. These are things that make the
analysis more time-consuming, not to mention the safety
requirement that should also be taken into consideration.
The use of UAVs to capture images in the civil
construction for further processing via software has been the
subject of some research papers lately [ESCHMANN 2012].
However, most of these studies don’t address the
development of embedded systems to process these images
during the flight. The purpose of this paper is to investigate
different alternatives for image processing systems to be
embedded on UAVs for automatic detection of cracks in
building facades, specifically the cracks in coatings mortars.
This paper is divided as follows: section 2 gives an
overview on pathological manifestation in buildings and
section 3 presents some image processing algorithms used in
inspection of civil engineering artifacts. Section 4 discusses
some possible alternatives for embedded image processing to
be installed on UAVs and compares the results of the
implementation of some of these architectures. Section 5
draws the conclusions and signals future work directions.
2. PATHOLOGICAL MANIFESTATIONS
The pathological manifestations are caused by many
reasons, and most of these problems arise during the lifetime
of the building resulting both from wrong decisions during
project design or during construction. One of the most
17
common pathological manifestations found in facades with
coatings mortars are the cracks.
Cracks in mortar lining are related to several factors,
among them retraction, thermal expansion and variation can
be highlighted. Cracks can be caused by excessive loads,
restrained expansion, unaccommodated contraction,
excessive deflection of the supporting structure, weathering,
incompatibility of materials, poor workmanship, vibration,
frame movement, foundation movements, and chemical
change [GRIMM 1997]. The study of these causes is very
important to understand the reason of cracks in buildings.
These cracks allow the penetration of external aggressive
agents, including water, which can lead to the appearance of
new pathological manifestations [RICHTER at ali. 2008].
Continuous walls (walls without openings) are not only the
most sensitive part of the façade, but also the place where the
highest levels of degradation are more easily found.
Statistically, cracking (due to differential movements or due
to mortar retraction) and damp patches (namely revealing the
pattern of the brickwork under the lining) represent almost
two thirds of the problems on continuous walls. [GASPAR el
ali. 2005]
Fig.2 illustrates some of the mortar cracks that may be
found in facades. In the proposed work, the analysis is
performed on algorithms for specific detection of cracks in
masonry walls, composed of mortar between bricks or blocks
used for structural function or not, and coated with mortar. It
is assumed that is possible to collect images with good
lighting conditions as required by the correct processing by
image processing algorithms. That means, it is considered
that it’s not necessary that the UAV carry any lighting
equipment, which would impact negatively on the autonomy
time of the UAV flight, either by increased load or by the
consumption of battery power for the lighting actuation.
Fig. 2 – Examples of cracks found in facades
As depicted in Fig. 2 – different kinds of cracks can be
identified in facades: horizontal cracks, vertical cracks,
cracks at 45 degrees, map-like cracks, stair-like crack, wider
crack, among others. Each one represents an effect caused by
problems during execution or during and subsequent climatic
effects. There are also cracks located near the frames with
very specific characteristics, which are not considered in this
work.
 CESCIT 2015
June 22-24, 2015. Maribor, Slovenia Fábio Celestino Pereira et al. / IFAC-PapersOnLine 48-10 (2015) 016–021
18

3. IMAGE PROCESSING ALGORITHMS After the linear transformation, the morphological

There are several publications discussing image
processing algorithms to be used in the detection of cracks in
buildings and other artifacts designed by civil engineers.
Some examples can be seen in [Abdel-Qader et ali., 2003]
[NIEMUELLER 2006][HOLMES 2010][FURTADO
2012][ELLENBERG et ali. 2014]. From the existing
algorithms two were selected and possible embedded
implementations were evaluated. The algorithms were tested
with simulated pictures of mortar facades with different
cracks. These images were also used for calibrating the basic
parameters of algorithms to facilitate the detection of the
proposed object.
In the following subsections the selected algorithms are
described into more details.
3.1 Algorithm-Based Segmentation by Edge Detection
The first algorithm uses edge detection based on the Sobel
operator or Sobel filter [SOBEL 1990]. It is a discrete
differentiation operator, computing an approximation of
the gradient of the image intensity function. At each point in
the image, the result of the Sobel operator is either the
corresponding gradient vector or the norm of this vector..
During the simulation tests with other operators such as
Robert operator , Prewitt operator, Kirsch operator and Marr
operator, the Sobel operator presented the best performance
especially for detection of inclined cracks.
Image is initially acquired in RGB color format. In the
sequence, it is converted to a grayscale, by applying equation
1, where c (x, y) is the gray scale comprising a pixel level
intensity values between 0 and 255, and R, G, B represent the
red, green and the blue components respectively.
mathematical operations are performed: the first operation
involves image dilation, to perform the connection of the
pixels in the image edges thus avoiding discontinuity of a
crack after edge detection. Morphological mathematical
operations use morphological operators called structuring
elements. For the dilation procedure a 7x7 with its center
represented by a cross was used (see 3a).
a) b)
Fig. 3 - Structuring element. a) (7x7) for dilation and filling
of the image; b) (3x3) for filling and erosion of the image.
After connecting the pixels that represent an edge, it is
necessary to fill the pixels that represent holes in the image.
For that the morphological fill operations are executed, where
the structural element (3 x 3) depicted in 3b is adopted.
Finally, the erosion of the image is performed, in order to
eliminate the pixels off the edge of the image, using the
diamond structuring element (3x3) represented in 3b. After
these image processing steps, the separation of all objects is
concluded, and object´s attributes such as area, major axis,
and the angle along the thickness of the object are obtained.
Storing details of detected objects, such as orientation,
thickness, length, center is important to future classification
of the cracks. Fig.4 presents the sequence of images after the
use of methods for the detection of the crack in the image.

Then the edge detection algorithm is performed using the

Sobel edge operator, applying a threshold reference factor
comprised between 40% and 20% of the gradient magnitude
comprising the edges limits of a crack.
The magnitude grad is basically obtained by partial
derivatives calculated separately in the direction of two
spatial coordinates x and y with subsequent combination of
results. Equation (2) defines the operator that approximates
the gradient magnitude as the difference of weighted values
of the gray levels of the image in both directions.

As a next step, image pixels are converted to logic levels

0 and 1. For this, a threshold is applied, using the operation Fig. 4 - Sequence of images after the methods used for crack
described in equation (3) [GONZALEZ 2002]. detection.
1, if fx, y >  Due to textures commonly found in mortar facades, it is
gx, y =  (3)
0, if fx, y <  possible to identify the presence of low-intensity points,

18
 CESCIT 2015
June 22-24, 2015. Maribor, Slovenia Fábio Celestino Pereira et al. / IFAC-PapersOnLine 48-10 (2015) 016–021 19
which don’t represent a crack. Through the characteristics of
the crack, conducted by the description of the object, it is
possible to discard these points.
3.2 Algorithm Based on Particle Filter
The second selected algorithm is a non-parametric filter
based on Bayes algorithm, the Particle Filter [THRUN 2006].
The particle filter seeks to relate the probability of an image
segment to be characterized or not by a crack, based on pixel
intensity and on the number of pixels in its neighborhood.
The Particle Filter algorithm proposed in [Furtado 2012],
which was conceived to the detection of underwater cracks,
was adapted to enable the automatic conversion of code
without changing its fundamental characteristics.
This algorithm delivered around 74% accuracy in
detection of the cracks in the analyzed samples for the
considered parameters, but the method is susceptible to false
positives because it does not consider the pattern
characteristics of a crack. Fig. 5 presents the image with the
detection points more likely region of the crack.
Fig. 5– Crack detection using a Particulate Filter.
4. DESIGN SPACE EXPLORATION
Design space exploration refers to the activity of
exploring design alternatives in order to identify the
alternative that better fulfill the application requirements. In
this work, following implementation alternatives were
compared: i) coding the image processing algorithms in
Matlab and execute them in a Matlab runtime environment
for a desktop computer (which will be located in a ground
station, that means, it is not installed on the UAV because of
its high energy consumption and weight); ii) to generate a C /
C++ program from the Matlab code and execute it on a
runtime environment with Linux support; iii) to generate a
VHDL version of the most time consuming parts of the
algorithms in order to assess the speed-up that can be
obtained running this code on a FPGA-Board. These
implementations could be used in the following application
scenarios. Scenario A: the UAV is only used to perform data
acquisition and storage and image processing is performed by
a desktop computer in the ground station. This scenario can
be sub-divided into different sub-scenarios. One on which the
UAV stores all images and fly back to the ground station to
download them for further processing (scenario A1) and
another on which the UAV may transfer the acquired images
via a wireless link to the ground station (scenario A2) .
19
The desktop in the ground station can either be executing
the image processing algorithms over a Matlab runtime or as
C++ code running on top of a Linux-based OS.
In Scenario B the UAV is equipped with an onboard
Raspberry-PI processor with the embedded image processing
algorithms running on it and in this case the crack detection
can be performed during the flight.
In the last scenario C, the UAV is equipped with a FPGA-
board on which the generated C / C++ program is ported to a
FPGA-Board with a Microblaze soft processor and OS
support (scenario C).
In Scenario A the desktop can process larger images (for
instance 5 megapixel images with the size of 2592 x 1944
pixels), however in case the images are to be transferred via a
wireless link to the ground station (scenario A2) larger image
sizes can become a problem due to the required transmission
time from the UAV to the ground station. Therefore in this
case two different image sizes were considered: 5 and 1
megapixels images.
The initial idea was to generate the C / C++ and the
VHDL code using the corresponding MATLAB toolboxes –
CODEGEN C and CODEGEN HDL, in order to ensure the
same base code. However, in the CODEGEN version used in
this work most of the Matlab functions used within the
Matlab Image Processing Toolbox were not available in the
code generation tool. Therefore, those functions were
manually coded to the target programming languages (C++).
Additionally, to ensure an appropriate code generation for
each language some changes in the original code had to be
introduced. A comparison of the generated code in terms of
lines of code and size of the generated executable code can be
seen in Table I.
Table I – Algorithms comparison (LOC and size)
Lines of Executable
Environments code (LOC) file (bytes)
Matlab Code on Desktop Computer With
Image Processing Toolbox of MATLAB - 118 3.977
Algorithm Particle Filter
Matlab Code on Desktop Computer Without
Image Processing Toolbox of MATLAB - 392 21.882
Algorithm Sobel
C-C++ Code on Desktop Computer with
Linux-based OS - Algorithm Sobel 2607 152.963
C-C++ code on Raspberry-PI with Linux-
Based OS - Algorithm Sobel 2607 153.885
In implementing the program in the proposed device, the
behaviors of the two algorithms were similar to the
simulation environment. In the temporal analysis were used a
set of 48 images of 1 Megapixel with dimensions 1024 x 768
and 58 images of 5 Megapixels with dimension of 2592 x
1944, both with facade cracks images. Table II presents the
average time and standard deviation in the execution time of
the image processing algorithms for 1 Megapixels images in
both plataforms.
 CESCIT 2015
June 22-24, 2015. Maribor, Slovenia Fábio Celestino Pereira et al. / IFAC-PapersOnLine 48-10 (2015) 016–021
20

Table II – Execution times autonomous crack detection in building facades was

calculated. Following time measurements were considered:
Time in Seconds for images of 1Mpixels t1 – UAV travel time from take-off to building location
Code Plataform Average Standard deviation
PC (Linux) 3,987012 3,203406 t2 – image acquisition time, assuming a façade with 200
Sobel Operator Raspberry 80,597374 65,641125 m² and that the UAV can move with a maximum speed of 0,5
Microblaze 5058,043518 4788,524014 m/s and is at a distance of 1,05m of the building façade so
PC (Linux) 0,136843 0,006654
that each image corresponds to an area of 1,05x0,79 m² in a
Particle Filter Raspberry 0,897121 0,074693
Microblaze 233,966558 1,329718
distance of acquisition of 1,00m of the building façade.
t3 – time to transfer the images from the UAV to ground
As it can be observed in Table II, the Particle Filter
station via wireless link (assuming a data throughput of 110
alternatives presented much shorter execution times than the kbps and a distance of 400 m)
version with Sobel operator. In Raspberry and PC platforms
the execution time of the particle filter algorithms was below t4 – image processing time, which is dependent on the
1s, while for the Sobel operator this execution time could be selected algorithm and on the target hw platform.
100 times slower (80s per image for the Raspberry-PI
version). In Microblaze, execution times were higher, with Table III presents the required times for the image
respect the other two platform, Analyzing the compiled processing algorithm using the Sobel operator and Table IV
for the particle filter version.
instructions of each platform, we find that the number of
instructions to perform calculations involving floating-point Table III – Sobel Operator Code
and 64 bits data are considerably lower in Raspberry PI
(RISC) and PC platform than in Microblaze environment, UAV Image - Image
Image
Take-off Wireless Processing Total Time
even with the option available to optimize this calculation in Situations to Façade Aquisition Transfer Time
the FPGA. t1 t2 t3 t4 t1/t2/t3/t4
Scenario A1 00:05:00 00:08:48 - 00:17:32 00:46:20
Scenario A2 00:05:00 00:08:48 02:20:24 00:17:32 02:39:16
Scenario B 00:05:00 00:08:48 - 05:54:05 06:07:54
Scenario C 00:05:00 00:08:48 - 15 days 10 hr. 15 days 10 hr
55 min. 55 min.

Table IV – Particle Filter Code

UAV Transfer Image
Image Total
Take-off Image Processing
Situations to Façade Aquisition Time
Wireless Time
t1 t2 t3 t4 t1/ t2/t3/t4
Scenario A1 00:05:00 00:08:48 - 00:03:31 00:29:24
Scenario A2 00:05:00 00:08:48 02:20:24 00:00:36 02:39:12
Scenario B 00:05:00 00:08:48 - 00:03:56 00:22:44
Scenario C 00:05:00 00:08:48 - 17:07:57 17:17:57

As it can be observed in the previous tables, the time to

Fig. 6– Execution time for the sub-functions within the
algorithm using the Sobel operator
An analysis of the execution times for each sub-function
called with the algorithm using the Sobel operator indicated
that basically two functions consumed most of this execution
time (see 6). This motivated the idea of implementing those
functions directly in hardware by implementing them in
VHDL language and running them on a FPGA board.
However, in an attempt to synthesize a code in HDL from the
HDL Coder ™ tool Matlab, it was found that a large amount
of memory would be necessary to process the image, which
made it impossible to synthesize these functions.
5. PERFORMANCE ANALYSIS USING THE DEFINED
APPLICATION SCENARIOS
As previously mentioned, the alternative implementations
were compared using the application scenarios described in
the previous section and the total time to complete the task of
complete the tasks may vary from around 30 minutes to
several days. The fastest response was achieved using the
particle filter executing on the Raspberry Processor in the
UAV. The execution of the Sobel operator algorithm in the
Raspberry PI takes considerably more time (almost 6 hours).
The same algorithm, when executed on the desktop computer
takes around 18 minutes, indicating that in this case it is
better to use the UAV only to perform image acquisition and
execute the image processing algorithms in a ground station.
The alternative of performing the image transmission, via a
wireless link, to the ground station in parallel with the image
acquisition has shown to be not very effective, because in this
case the transmission process becomes the bottleneck.
Transmission time for a 1 megapixel image takes around 32
seconds while a 5 megapixel image may take up to 3 minutes
and 30 seconds, even considering no packet losses, what is
not very realistic for wireless transmission. The execution
times of the Sobel operator algorithm when running on the
Raspberry-PI board makes this alternative unfeasible to the

20
 CESCIT 2015
June 22-24, 2015. Maribor, Slovenia Fábio Celestino Pereira et al. / IFAC-PapersOnLine 48-10 (2015) 016–021 21
target application (total required execution time greater than
UAV´s battery duration). Due to the amount of memory
required by the complete set of images to be analysed, the
alternative of transmitting the images from the UAV to the
ground station via a wireless link also took much longer time
than allowed by the application. An interesting alternative
seems to be is the use of particle filter embedded to a
preliminary detection of crack delivering the image to the
algorithm sobel operator to confirm the presence of the crack
and obtaining the description of data.
6. CONCLUSIONS
By means of the obtained times observed that the
algorithm execution time with a particulate filter and
considerably less on all platforms than the algorithm that uses
segmentation with Sobel operator. Entretando, the
implementation of Particulate Filter on Raspberry PI platform
enabling the timely detection of cracks during a UAV flight
is feasible to allow a system vessel directly into the UAV.. In
image processing, the choice of images with 1-megapixel size
is satisfactory, without loss of detail in capturing the image
that will be very close to the facade. When using the
description algorithm, we note that scenario 2 is the best
choice, but at the cost of higher consumption of the system
due to data transmission. However, the use of segmentation by
Edge Detection is important for obtain characterizing data of
the crack as angle, size and width. Based on the obtained
results, the idea of combining the proposed scenarios seems
very promising: to embedded the particle filter on the
Raspberry-Pi for preliminary detection of the crack and in
case the particle filter algorithm indicates a high possibility of
having a crack then that particular image is stored for further
processing by a Sobel operator filter running on the ground
station. This is part of an ongoing work, whose results will be
presented at the conference in case the paper is accepted.
REFERENCES
ABDEL-QADER I. ABUDAYYEH O., KELLY M. "Analysis of edge-
detection techniques for crack identification in bridges." Journal of
Computing in Civil Engineering 17.4 (2003): 255-263.
DENG C; WANG S; HUANG Z.;TAN Z.;LIU J. Unmanned Aerial Vehicles
for Power Line Inspection: A Cooperative Way in Platforms and
Communications. Journal of Communications Vol.9, No.9, September 2014
http://www.jocm.us/uploadfile/2014/0918/20140918110959273.pdf
DONG G.; CHEN X.; WANG B., ZHANG J.; LIU L., WANG Q., WEI C.
Inspecting Transmission Lines with an Unmanned Fixed - Wings Aircraft.
2012. 2nd International Conference on Applied Robotics for the Power
Industry (CARPI), ETH Zurich, Switzerland. September 11-13, 2012
DRAPER, M.; CALHOUN, G.; NELSON J.; RUFF, H. Evaluation of
Synthetic Vision Overlay Concepts for UAV Sensor Operations: Landmark
Cues and Picture-in-Picture. 2006. Proceedings of the Human Factors &
Ergonomics Society Annual Meeting, 28 February 2006.
ELLENBERG, A., KONTSOS, A., BARTOLI, I., and PRADHAN, A..
2014. Masonry Crack Detection Application of an Unmanned Aerial
Vehicle. Computing in Civil and Building Engineering (2014)1788-1795.
FURTADO, F. Recognition by Computer Vision Cracks Using a Particle
Filter. 2012. 41p. Monograph (Graduation Control Engineering and
Automation) – Federal University of Ouro Preto, Ouro Preto, 2012. acess
may 5, 2014: available in
<http://www.em.ufop.br/cecau/monografia_2012.php>
21
LANGE, S.; S¨UNDERHAUF, N.; PROTZEL, P. A Vision Based Onboard,
Approach for Landing and Position Control of an Autonomous Multirotor
UAV in GPS-Denied Environments. 2009. Advanced Robotics, 2009. ICAR
2009. 22-26 June 2009 Page(s): 1 - 6.
ESCHMANN, C. ; KUO, C.-M.; KUO C.-H.; BOLLER, C. Unmanned
Aircraft Systems for Remote Building Inspection and Monitoring. 6th
European Workshop on Structural Health Monitoring - Th.2.B.1, July 3 -6
2012 – Germany
GASPAR P.; BRITO J. Mapping defect sensitivity in external mortar
renders. Elsevier. 2005. Construction and Building Materials. Volume 19,
Issue 8, October 2005, Pages 571–57
GRIMM,C.T. Masonry Cracks: Cause, Prevention and Repair . Masonry
International,BMB, vol.10, n.3, 1997, p.66-76.
JENSENA, A. M.; WILDMANND, N.; Y. Q. CHEN, H. VOOS, In-Situ
Unmanned Aerial Vehicle (UAV) Sensor Calibration To Improve Automatic
Image Orthorectification, Geoscience and Remote Sensing Symposium
(IGARSS), 2010 IEEE International, 25-30 July 2010, pages:596–599.
HARRIS S.Y. Building Pathology: Deterioriation, Diagnostics, and
Intervention. John Wiley & Sons. 2001. 654 p. ISBN 0-471-33172-4
HENDRY, A.W.; KHALAF,F.M. Masonry Walls Construction . London:
Spon Press, 2001. ISBN 0-203-46772-8HOLMES J. Development of an
Automated Pavement Crack Sealing System. 2010. 125p. Georgia Dot
Research Project 2047 Final Report. Department of Transportation os State
of Georgia, Georgia, 2010. available in: <
http://trid.trb.org/view.aspx?id=1106013> acess may 22, 2014.
METNI N.; HAMEL T. - A UAV for bridge inspection: Visual servoing
control law with orientation limits - Elsevier - Automation in Construction
Volume 17, Issue 1, November 2007, Pages 3–10 22 December 2006 -
Available in < http://www.sciencedirect.com/science/article/pii/S0926580507000052 >
NIEMUELLER T. Automatic Detection and Segmentation of Cracks in
Underground Pipelines Images. 2006. 27p. Seminar: Medical Image
Processing Summer Semester 2006. MN 236106. Institut f¨ur Medizinische
Informatik, RWTH Aachen, Alemanha, 2006.
RATHINAM S.; KIM Z.;SENGUPTA R. Vision-Based Following of
Structures Using an Unmanned Aerial Vehicle (UAV) .March 2006. Institute
of Transportation Studies University of California at Berkeley ISSN 0192
4095. < http://its.berkeley.edu/publications/UCB/2006/RR/UCB-ITS-RR-2006-1.pdf >
ROBERTS, L.G. Machine Perception of Three Dimensional Solids, Optical
and Electro-optical Information Processing, MIT Press, Cambridge, MA,
United States, pp. 159-197
SOBEL, I. Na Isotropic 3 x 3 Image Gradient Operator, Machine Vision for
Three-Dimensional Scenes, Academic Press, p. 376-379., 1990.
GONZALEZ, R. C.; WOODS, R. E. Digital image processing. Digital
Image. Processing, Second Edition, Pearson-Prentice Hall,. Inc., 2002. 780p.
ISBN 81-7758-168-6
Raspberry Pi Foundation. About us. [Internet]. [acess November 5,
2014].http://www.raspberrypi.org/about.
RICHTER C.; FORMOSO C.T.; MASUERO A.B. – Analysis of
pathological Defects of Load-Bearing Mansonry Technology in Low Income
Projects. 2008. UFRGS. Avaliable in: <
http://www.ceramicapalmadeouro.com.br/downloads/richter2008.pdf>
SHAM, F. C.; CHEN N.; LONG, L. Surface crack detection by flash
thermography on concrete surface. Insight - Non-Destructive Testing and
Condition Monitoring, Volume 50, Number 5, May 2008, pp. 240-243(4)
THRUN, S., Probabilistic robotics, Cambridge, Mass. : MIT Press, c2006,
647 p. ISBN:0262201623 9780262201629
ZHAO, J. QI, X.; JIANG, Z.; HAN, J. An Adaptive Threshold Neural-
Network Scheme for Rotorcraft UAV Sensor Failure Diagnosis. Advances in
Neural Networks, Volume 4493, 2007, pp 589-596.