2154
5, OCTOBER 2014
A Vision-Based Broken Strand Detection Method
for a Power-Line Maintenance Robot
Yifeng Song, Hongguang Wang, Member, IEEE, and Jianwei Zhang, Member, IEEE
AbstractThe broken strand of overhead ground wire (OGW),
which is mainly caused by lightning strikes or the vibration of
OGW, can lead to serious damage to the power grid system.
Power-line maintenance work is generally carried out by special-
ized workers under extra-high voltage live-line conditions which
involve great risks and high labor intensity. In this paper, we
present a broken strand detection method which can be prac-
tically applied by maintenance robots. This method is mainly
implemented in three steps. First, we obtain the region of interest
(ROI) from the image acquired by the robot. Second, a histogram
of an oriented gradients descriptor vector is calculated to obtain
the image gradient feature in ROI. In the third step, we apply a
multiclassifier which consists of two support vector machines to
classify the wires into normal wire, broken strand malfunction,
and obstacles on OGW. Experiment results successfully demon-
strate the effectiveness of the proposed method.
Index TermsBroken strand, power-line maintenance robot, vi-
sual detection.
I. INTRODUCTION
R EGULAR power-line inspection and maintenance play
an important role in the normal running of power grid
systems. Traditionally, power-line inspection and maintenance
tasks are performed by specialized workers. To ensure that the
entire power grid system runs smoothly, the workers have to
work under extra-high voltage live conditions which involve
great risks [1], [2]. Power-line inspection and maintenance
robots are thus strongly practically required for the normal
running of power grid systems.
Research on power-line inspection and maintenance robots
began about two decades ago [3], [4]. So far, robotic tech-
nology has been successfully applied to power-line inspection
and maintenance tasks. (See [5], [6], and the references therein.)
References [7][9] introduced three different robots focusing
Manuscript received July 23, 2013; revised March 07, 2014 and March 27,
2014; accepted May 28, 2014. Date of publication June 25, 2014; date of current
version September 19, 2014. Paper no. TPWRD-00828-2013.
Y. Song is with the State Key Laboratory of Robotics, Shenyang Institute of
Automation, Chinese Academy of Sciences, Shenyang 110016, China, and also
with the University of Chinese Academy of Sciences, Beijing 100039, China
(e-mail: songyifeng@ sia.cn).
H. Wang is with the State Key Laboratory of Robotics, Shenyang Institute of
Automation, Chinese Academy of Sciences, Shenyang 110016, China (e-mail:
hgwang@ sia.cn).
J. Zhang is with the Department of Informatics, the Institute of Technical As-
pects of Multimodal Systems, University of Hamburg, Hamburg 22527, Ger-
many (e-mail: zhang@informatik.uni-hamburg.de).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TPWRD.2014.2328572
0885-8977 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
IEEE TRANSACTIONS ON POWER DELIVERY, VOL. 29, NO.
on the power-line inspection task. In [10], a robot named
LineROVer was developed for the deicing maintenance task.
Reference [11] introduced a robot that could install and remove
the aircraft warning spheres on OGW. LineScout robot [12],
[13], which was equipped with a variety of special tools, could
perform not only the inspection task, but also several mainte-
nance tasks (e.g., tightening screws and temporally repairing a
broken strand).
The successful detection of obstacles and power-line mal-
functions is an important prerequisite for power-line mainte-
nance. Based on the sensory techniques used for detection, we
categorize the relevant literature into two kinds, namely, the de-
tection using nonvision-based techniques [14][21] and the de-
tection using vision-based techniques [22][26].
For nonvision-based detection methods, a high-temperature
superconductor (HTS) superconducting quantum interference
device (SQUID) was used to detect single-wire breakage in
aluminum transmission lines [14]. A periodic pattern was de-
tected with wire breakage, while this pattern was not observed
in the normal wire. Using electromagnetic-acoustic transducers
(EMAT), [15] presented a nondestructive inspection system to
diagnose the mechanical integrity of conductors. The system
was trained to identify four conditions: 1) normal; 2) minor ab-
normal; 3) major normal; and 4) corrosion. Local and global
wave-based methods using ultrasonic waves were presented for
wire breakage detection in overhead transmission lines (OTLs)
[16]. Both methods used a sending/receiving transducer to gen-
erate an ultrasonic wave in the cable. Defects could be detected
through the variation of reflected ultrasonic waves. In [17], the
eddy current transducer (ECT) was developed to provide in-
formation to evaluate serious deterioration due to forest fires
in aluminum-conductor steel-reinforced conductors. Reference
[18] designed a variant of ECT for the transmission-line broken
strand detection. The developed ECT had good performance
on reliability and sensitivity. There were also nonvision-based
detection methods based on guided waves technology [19], in-
frared technology [20], and other technologies [21].
For vision-based methods, [22] proposed an algorithm based
on straight line and circular extraction for obstacle detection.
The proposed method first removed transmission lines and
towers that were straight lines in the image. By recognizing the
different features of circular or elliptic parts left in the image,
insulators and counterweights could be detected. In [23], a robot
system was set up to detect a spacer in quad-bundled conduc-
tors. The detailed process could be described in the following
steps: digital image acquisition, conductor localization, and
spacer detection. One or more disruptions of conductors were
sufficient proof that a spacer was present. In [24], a geometric
SONG et al.: A VISION-BASED BROKEN STRAND DETECTION METHOD FOR A POWER-LINE MAINTENANCE ROBOT
model, which included the position relationship between the
camera and the obstacle location center, was built by using the
Hough transform for the spacer and counterweight detection
in a dual-split conductor. In [25], a multisensor system was
constructed for conductor strand inspection. With two cameras
and a mirror, all strands of the conductor could be inspected
simultaneously. Furthermore, the laser sensors were used to
detect obstacles by measuring the variance of the conductor
diameter. In [26], an obstacle detection approach based on
contour view synthesis for the inspection robot was presented.
A virtual contour image library was first built offline through
the reconstruction of multiview contour images with regard
to several types of obstacles. The detection could be then
accomplished by extracting obstacle contour and matching the
extracted contour with virtual contour images in the library.
Among all power-line malfunctions, the OGW broken strand
is one of the most serious malfunctions, which can result in
flashover or even the crash of a power grid system. The de-
tection of the OGW broken strand is essential for the safety
of power grid systems. Each of the nonvision-based and vi-
sion-based methods has its advantages in transmission-line mal-
function detection. However, as the sensing unit of the inspec-
tion or maintenance robot, the vision sensor has advantages in
broken strand detection due to the following reasons: 1) with
small volume and light weight, the vision sensor can be well-in-
tegrated with the robot system, while some nonvision sensors,
for example, HTS-SQUID and EMAT, are beyond the robots
load capability; 2) the vision sensor is not sensitive with the am-
bient temperature and variation of the magnetic field, which may
interfere with some nonvision sensors, for example, infrared
sensor and ECT; and 3) the vision sensor facilitates the me-
chanical integration with robots. Most nonvision sensors (ECT,
EMAT, ultrasonic sensors, etc.) used for detection have an in-
stallation requirement on the distance between the probe and the
wire. It causes difficulties with the robot motion, because the
sensors may interfere with robot moving parts or the transmis-
sion line. In addition, the vision sensor can provide information
on power-line fittings and the transmission corridor for inspec-
tion work.
In this paper, we present a vision-based OGW broken strand
detection method for the application of a power-line inspection
and maintenance robot. The proposed vision-based method for
the OGW broken strand detection consists of three steps: first,
we obtain the region of interest (ROI) from the acquired image;
second, we utilize the histogram of oriented gradients (HOG)
method to extract the image gradient feature; a hybrid classi-
fier consisting of support vector machines (SVMs) is applied to
the classification of the normal wire, broken wire, and obstacles
in the third step. It should be noticed that the proposed method
can detect the OGW broken strand effectively, even though the
pattern of broken wires with a broken strand greatly varies. The
contribution of this paper is the proposal of a vision-based detec-
tion method that targets the broken strand malfunction occurring
on OGW, which has not been effectively solved yet in the ex-
isting literature. The proposed detection method is highly accu-
rate and with invariance photometric transformations. We have
demonstrated the effectiveness of the proposed method through
numerous experimental studies.
2155
Fig. 1. Broken strand maintenance carried out by workers and the robot.
II. BACKGROUND OF BROKEN STRAND DETECTION
Extra-high voltage transmission lines mainly consist of
OGW, towers, single, double, or quad-bundled conductors and
fittings. Conductors are installed for power transmission, while
OGW installed above the conductors is mainly used for the
power grid system protection from lightning strikes. According
to facility installation requirements, several kinds of fittings
(e.g., counterweights) are mounted on OGW and conductors.
However, the huge current transferred to the ground by OGW
during lightning strikes may result in local overheating and,
thus, causes the strands to melt and break. Typically, the broken
strands are located in the middle of the span where they are far
away from the tower. Since the dangling broken strand could be
very close to the conductors, the huge potential difference be-
tween the broken strand and the conductors can form a voltage
breakdown and flashover, which results in severe damage to the
power grid system. Furthermore, the broken strand reduces the
strength of OGW and causes other strands to break due to in-
creased cable tension, which may even result in the rupture of
OGW and communication optical cables inside OGW. This will
cause communication signal interruption, damage of power-line
equipment, and huge economic losses.
Due to aforementioned reasons, regular inspection of the
strands is quite important for the smooth running of a power
grid system. Once a broken strand is detected, maintenance
work needs to be carried out immediately. Traditionally, the
broken strand maintenance work is performed by specialized
workers, as shown in Fig. 1(a). However, most power grid
systems are built in complex geographical environments (e.g.,
forests, marshes, and mountains) and OGWs are tens of meters
over the ground in extra-high voltage transmission systems.
These poor working conditions make the maintenance work
hard to implement. Furthermore, the maintenance work usually
needs to be performed in the middle of the span and under
extra-high voltage live conditions, and these factors also in-
crease the safety risk to the workers.
For these reasons, robots are of great importance to replace
workers in performing the power-line inspection and mainte-
nance work. Fig. 1(b) schematically shows the procedures of a
robot performing inspection and maintenance work. After in-
stallation, the robot runs along OGW to perform the inspec-
tion and maintenance work, which consists of four parts: 1)
crossing over the tower; 2) crossing the obstacles such as coun-
terweights; 3) detecting and 4) fixing the broken strand. In the
2156
Fig. 2. Framework of the robot maintenance procedure.
following part, we present a vision-based method for the de-
tection of OGW broken strand malfunction. Interested readers
should refer to our previous work for the development of a se-
ries of power-line inspection and maintenance robots, including
robot mechanical design based on the tasks of towers and coun-
terweights crossing [27], broken strand repair maintenance tools
design [28], and motion control strategy [5].
III. BROKEN STRAND DETECTION METHOD
Fig. 2 gives the framework of broken strand detection,
which is implemented by a maintenance robot for broken
strand repair. The detection method is vision based and can
be described as follows. We prepared three classes of images,
that is: 1) obstacles (as a common obstacle, counterweight is
selected); 2) broken strand (the wire with broken strand); and
3) normal wire. For these images, a special region of the image
is selected as ROI and an image gradient feature is extracted
in ROI as training data for classifiers. After a multiclassifier
is constructed by training, the robot is installed on OGW to
implement the maintenance task. Images of OGW captured by
the robot are processed in the same way to obtain the image
gradient feature. The extracted feature is put into the classifier
as testing data and, thus, the class label of the captured image
is finally assigned by the classifier.
A. ROI Selection
In the image captured by the robot, the targeting object
that is useful for further processing is in a certain region. The
remaining regions are not as important for further processing
and will bring complexity to feature extraction. Therefore, ROI
should be selected to remove the unnecessary part of the image
and reduce the amount of computation.
Here, ROI is picked in a rectangular region by three steps.
First, we use Hough transform to extract the straight lines of
OGW boundaries as the horizontal reference and choose pixels
at the top of the image as the vertical reference. Second, a point
with a fixed distance to the horizontal and vertical reference is
marked as point A. At last, the rectangular region with a prede-
fined length and width and point A being the top left vertex, is
picked as ROI, as shown in Fig. 3. In the ROI, there is OGW
with a clear strands contour that can be used for image gradient
extraction.
The position of the ROI is chosen based on the following
reasons. 1) due to the robot installation position with respect to
OGW, OGW only appears in the center strip of the image and the
IEEE TRANSACTIONS ON POWER DELIVERY, VOL. 29, NO. 5, OCTOBER 2014
Fig. 3. ROI selection process.
Fig. 4. (a) Gradients of a normal wire are mainly in the vertical and spiral lead-
angle orientation, the principal orientations, as shown by the arrows, (b) The
gradients of the counterweight obstacle distribute irregularly as shown by the
arrows. (c) The gradients of a broken wire are mainly in the vertical and spiral
lead-angle orientation as shown by the arrows, but the gradient intensity in these
two orientations is lower than in normal wire due to the broken strand, whose
gradient orientation is shown by the square.
outside of the center strip can be removed; 2) the upper part of
image is closer to the camera and exhibits high resolution, so it
is well-suited to obtain a clear contour of the strands for further
image gradient feature extraction; and 3) the background in ROI
is the robots driving wheel component, which does not change
with the motion of the robot.
B. Image Features Extracted by HOG
A broken wire possesses many similar features to a normal
wire, for example, gray intensity distribution. However, the
broken strand affects the strand configuration of OGW. For
a normal wire, all strands wind the OGW axis as a helix. In
this case, the image gradients extracted from ROI are mainly
distributed in two principal orientations: 1) the vertical orien-
tation (OGW edges) and 2) the spiral-lead angle orientation
of the helix (normal strands), as shown in Fig. 4(a). On the
other hand, the broken strand has a different configuration from
the normal strands, and that leads to the increase of image
gradients in other orientations, as shown in Fig. 4(c). Although
the image gradient of the broken wire is also mainly in the ver-
tical orientation and spiral lead-angle orientation, the gradient
intensity in these two principal orientations is lower than that of
a normal wire due to the broken strand. The image gradient of
the obstacle is not distributed in the aforementioned principal
orientations, as shown in Fig. 4(b), because OGW is blocked
by the counterweight obstacle. Based on the aforementioned
analysis, the normal wire, the broken wire, and obstacles have
different image features on gradient distribution that can be
extracted for classification.
Based on the aforementioned analysis, the gradient feature
can be extracted for the broken strand, normal wire, and obstacle
classification. Here, we choose HOG as the image gradient fea-
ture descriptor for the classification. HOG is a common feature
descriptor used in computer vision for object detection. Given
the hypothesis that local object appearance and shape can be
characterized well by the distribution of local intensity gradient
SONG et al.: A VISION-BASED BROKEN STRAND DETECTION METHOD FOR A POWER-LINE MAINTENANCE ROBOT
Fig. 5. ROI before and after histogram equalization.
or edge directions, the object can be described even without pre-
cise knowledge of the corresponding gradient or edge positions
[29]. Edges or gradient structures captured by an HOG are in-
trinsic characteristics of the local object shape, and the HOG
descriptor is invariant to geometric and photometric transfor-
mations. These are the key advantages of the HOG descriptor
compared with other descriptors. Typically, there are five steps
to obtain the HOG descriptor of an image, and we describe these
steps as follows.
1) Histogram Equalization: To reduce the computation of
image feature extraction, color images are first converted to
grayscale images. After that, histogram equalization is applied
on the converted grayscale images to regulate the image gray
values into a prescribed intensity scope, which can reduce the
negative impact of the environment lighting condition. Fig. 5
shows the ROI before and after histogram equalization. Before
histogram equalization, the image background owns similar
gray values as the strands. It can be found that the gray intensity
distribution of the ROI is regulated by effectively spreading
out the most frequent intensity values through histogram
equalization.
2) Gradient Computation: For each pixel on the
image, we calculate the pixel gradient by
(1)
where is the image gray value at this pixel, and
are, respectively, the horizontal and
vertical gradient magnitudes. We then formulate the orientation
angle and norm of the pixel gradient as follows:
(2)
(3)
3) Orientation Binning: Hence, we divide the image into
many pieces called cells that contain several pixels. Cells that
cover a small spatial region are defined as a standard unit for a
local HOG orientations calculation. For each cell, a local 1-D
histogram of gradient directions is calculated by accumulation
of gradient magnitudes in each orientation. Here, the cell is di-
vided into a square shape and each cell contains 8 8 pixels.
Since the entire image gradients have been calculated in the last
step, the orientation and magnitude are known. The number of
orientation bins is set to 9 in the histogram, that is, for every 20 ,
the gradient magnitude is accumulated for a bin. With this set-
ting of orientation bins, gradients in vertical orientation, spiral
lead-angle orientation, and other orientations, respectively, con-
tribute to different orientation bins so that different image fea-
tures can be extracted among normal wire, broken wire, and ob-
2157
Fig. 6. Orientation binning in a 8 8 pixel cell.
Fig. 7. Descriptor block normalization.
stacles. Fig. 6 shows the division of cells and the local 1-D his-
togram.
4) Descriptor Block Normalization: To increase the robust-
ness of the HOG descriptor against local variations in illumina-
tion and foreground-background contrast, the gradient of each
cell must be normalized locally within a spatially connected cell
group, which is denoted as a block. A vector is then used to
represent the normalized gradient information of all the cells
within each block. Typically, the blocks overlap, so the local 1-D
histogram of each cell is normalized with respect to different
blocks and contributes several components to the final HOG
descriptor vector. In this way, the HOG descriptor has a fine ef-
fect of normalization and has good performance on detection.
So far, there are four different block normalization methods,
namely: 1) the L2-norm method; 2) the L2-hys method; 3) the
L1-norm method; and 4) the L1-sqrt method, and the first three
methods have equally good performance on local normalization
[29], [30]. In this paper, we set 2 2 cells for each block and use
the L2-norm method for block normalization.
Fig. 7 shows the process of descriptor block normalization: At
first, the process of orientation binning is accomplished for each
cell in the block. Hence, by combining all of the histograms, we
can obtain a vector containing gradient information of all the
cells within the block. Finally, the normalization is carried out
to obtain the normalized block vector.
5) Image Features Extracted by an HOG: Finally, by con-
necting all normalized block vectors, we can extract the HOG
descriptor as a 1-D combined vector. The dimension of the HOG
descriptor vector, which affects image-processing speed, is re-
lated to the block number in the image, the cell number in a
block, and the number of orientation bins. Fig. 8 shows the HOG
descriptor vector of a normal wire image.
C. Classification by the SVM
1) Maximum Margin Classifier: SVM is a supervised
learning method that is usually used as a binary data classifier
[31]. For a classification task, two data sets are required, that is,
2158 IEEE TRANSACTIONS ON POWER DELIVERY, VOL. 29, NO. 5, OCTOBER 2014

Fig. 8. HOG descriptor vector for a normal wire image.

Fig. 9. Maximum margin separating hyperplane.

the training data set and the testing data set. The training data
Through equivalent transformation and simplification, the
set consists of enormous instances, and each instance is rep-
problem can be further formulated as
resented by several attributes and an associated label showing
the class that the instance belongs to. It should be noted that
the instance class labels are predefined in the training data set,
while the class label of each instance is unknown for the testing Subject to (6)
data set. The SVM classifier aims at assigning the respective
It should be noted that only a few training data sets are right
class label to each instance of the testing data set.
with a distance of to the separating hyperplane, which are then
Given the training data instance is
called support vectors. As shown in Fig. 9, the support vectors
the instance index and is the total number of instances in the
of different classes are, respectively, located in two hyperplanes,
training data set. is a p-dimensional real-value vector
and set off by a gap from the separating hyperplane.
expressing the instance attributes, and is the in-
2) Slack Variables: It is obvious that the support vectors
stance class label. The object of the linear classifier is to find
have an important influence on forming separating hyperplanes.
the separating hyperplane that divides the training data instance
There may be some abnormal instances out of the normal range
into two classes according to their class labels. However, there
in the training data, which are denoted as outliers. If outliers
is probably more than one separating hyperplane that can di-
become support vectors of a separating hyperplane, the sepa-
vide the training data. The SVM method is proposed to find
rating hyperplane has then very poor performance on classifi-
a separating hyperplane with maximum margin. Classification
cation. To reduce the influence of outliers, slack variable is
through a maximum margin hyperplane can maximize the sta-
introduced to the constraint, and the constraint is changed into
bility and confidence of classification and benefit the extension
. It means the support vector is allowed
application of the classifiers [32].
to have an exclusion with respect to the separating hyperplane
In the linear classifier, the separating hyperplane can be ex-
and the exclusion distance is limited to . It should be noted
pressed by function , where and are the normal
that the exclusion distance should be as low as possible; other-
vector and the intercept of the hyperplane. The classification
wise, any hyperplane can be treated as a separating hyperplane.
function can be formulated as follows:
Therefore, the optimization problem can be formulated as
(4)

Putting the instance attributes of training data into the classi-

fication function, the function value should be positive for the Subject to
data with class label , while the function value should be
(7)
negative for the data with class label .
For each , the formulation can be established as where is the penalty parameter and is the slack variable
, where is the corresponding point of on the of the th instance, which is introduced to measure the degree
hyperplane and is the distance from point to the hyper- of misclassification of the data.
plane. It should be noted that is a point on the hyperplane, Since the optimization problem is a convex
which means exists. Therefore, we can obtain problem, the following equations should be sat-
. Here, the geomet- isfied based on KarushKuhnTucker conditions
rical margin is defined as (5) to show the minimum distance .
from the data set to the hyperplane By solving the optimization problem, we can obtain the
maximum margin hyperplane parameters , and find the
hyperplane. Furthermore, the optimized classification function
(5) for testing data can be described as
As previously discussed, SVM is a special classifier to find
the maximum margin hyperplane. After the definition of the ge- (8)
ometrical margin, the problem can be described as .
Besides, the constraint should be satis- 3) Nonlinearity Classification: Since the SVM is established
fied based on the definition of the geometrical margin. based on the linear classification model, it cannot directly be
SONG et al.: A VISION-BASED BROKEN STRAND DETECTION METHOD FOR A POWER-LINE MAINTENANCE ROBOT
Fig. 10. Multiclassification for normal wire, broken wire, and counterweight.
used for nonlinear classification. In this case, the data need to
be mapped into feature space from the input space and then the
SVM can be applied to classification in feature space. The clas-
sification function can be formulated due to the dual form of the
SVM
where is the mapping from the input space to feature space.
With a kernel trick [33], the inner product in feature space
can directly be calculated through the kernel
function
Here, the widely used Gaussian radial basis is chosen as the
kernel function as shown
(9)
The application of the kernel trick effectively avoids the cal-
culation complexity in feature space and makes nonlinear clas-
sification with an SVM practical.
4) Multiclassifier: Before the robot is applied to this spe-
cific maintenance task, numerous images of three classesthe
normal wire, the broken wire, and the obstacleshave been
taken in the laboratory, and HOG descriptor vectors of these im-
ages have been calculated as the training data for SVMs. Since
SVM is a binary classifier, a multiclassifier needs to be con-
structed with several SVMs. Fig. 10 shows the structure of the
constructed multiclassifier. Two SVMs are established and put
into two layers, respectively. Since the HOG descriptor of the
obstacles is quite different from the normal wire and the broken
wire, we treat the obstacles alone as a class and the normal wire
and the broken wire as the other class to train an SVM, which
is inserted into the first layer in the multiclassifier. The second
SVM is trained to classify the normal wire and broken wire and
is inserted into the second layer in the multiclassifier. After the
two-layers classification, the testing data can be assigned with
proper labels.
IV. EXPERIMENT
A. Experimental Platform
The inspection and maintenance robots that we have de-
veloped [8] can be used for the experiments and we selected
2159
Fig. 11. AApe-D robot and broken strand detection experiment.
Fig. 12. ROI edge orientation extraction. (a) ROI. (b) Image edge. (c) Hori-
zontal gradient magnitude. (d) Vertical gradient magnitude.
AApe-D as the experimental platform. AApe-D, which consists
of a mobile platform and specialized maintenance tools, is a
maintenance robot for broken strand repair, as shown in Fig. 11.
With the wheel structure, the robot can move continuously
along OGW. The gripper wheels installed under the driving
wheels can increase the adhesion capability of the driving wheel
and guarantee the safety of the robot during the maintenance
task. A passive revolute joint is installed on each arm to im-
prove the obstacle crossing capability. Through these mechan-
ical designs, the mobile platform can run along OGW; cross ob-
stacles, for example, counterweights and splicing sleeves; and
carry repair tools to the broken strand. A pan-tilt-zoom camera
is installed on the electrical box for image acquisition.
The specialized tools installed on the platform, Looper and
Clamper, play different roles during the repair process. At first,
Looper is used to put the broken strand back to the original lo-
cation of OGW, and a clamp is then installed at the malfunction
location by Clamper to prevent the broken strand from unrav-
eling once again.
The robot operator can operate AApe-D through a
humancomputer interface at the ground control station,
where the information of the robot state and the environment
state can be transferred by wireless data and image modems.
B. Experimental Result Analysis
Before the robot is installed on OGW for the detection, the
multiclassifier is trained by three sets of data with known labels.
After that, the HOG descrptor is calculated from the image cap-
tured by the robot on OGW. The horizontal and vertical gradient
magnitudes are first calculated as shown in Fig. 12(c) and (d),
where the color bar represents the gradient magnitude. Based on
the calculated image gradient, the image feature of HOG can be
2160
Fig. 13. HOG descriptor vector extracted from normal wire, broken wire, and obstacle images. (a) Normal wire. (b) Broken wire. (c) Obstacles.
TABLE I
DETECTION ACCURACY OF THE PROPOSED METHOD
obtained by the method mentioned in Section III, and the HOG
descriptor vectors are as shown in Fig. 13.
HOG features of obstacles are apparently different from
the other two classes, because OGW is partially or totally
blocked by obstacles. Although the gradients of both broken
wire images and normal wire images mainly distribute in the
vertical orientation and the spiral lead-angle orientation due to
the configuration of OGW, the normalized gradient intensity in
these two orientations of the normal wire images is higher than
that of the broken wire images. The reason is that the broken
strand, whose image gradient is not in the vertical direction or
the strands spiral lead-angle direction, affects the image gra-
dient distribution. Based on the aforementioned analysis, the
three classes of images own different image gradient features
for classification. The source code of the proposed method is
available at http://www.mathworks.cn/matlabcentral/fileex-
change/45784-broken-strand-detection [34], [35].
Through our laboratory experiments, we demonstrate the
effectiveness of the proposed method. By testing 100 images
of each environmental state, it shows that images of different
classes have characteristic gradient features, and the detection
accuracy for each kind is above 96%, as shown in Table I.
According to the detection result, the robot can apply a proper
control strategy on OGW. If the detection result is a broken wire
or obstacle, the corresponding location can be estimated through
the position of the ROI for robot motion planning. Regarding
the detection result of normal wire and obstacle, the robot can
run along OGW and cross the obstacle to approach the broken
strand malfunction. The broken strand repair maintenance is im-
plemented with specialized tools of the robot once the broken
strand is detected. The detection of the broken strand and ob-
stacle lays a foundation for autonomous robot control.
IEEE TRANSACTIONS ON POWER DELIVERY, VOL. 29, NO. 5, OCTOBER 2014
V. CONCLUSION
This paper presents a vision-based method for the OGW
broken strand detection, which facilitates the practical ap-
plication of power-line maintenance robots. The proposed
vision-based detection method is mainly implemented in three
steps, namely, the 1) ROI selection; 2) the acquisition of image
features of an HOG; and 3) the application of a multiclas-
sifier for classification. The effectiveness of the proposed
vision-based broken strand detection method is verified and
demonstrated by numerous experimental studies.
In our future work, we will work on enhancing the robustness
of the proposed method and design a specialized mechanism to
reduce the effect of external factors, for example, illumination
for the experiments in the real environment. In addition, we will
focus on autonomous robot control based on the detection result
and enlarging the detection range of other obstacles and mal-
functions on OGW.
REFERENCES
[1] W. Li, Risk Assessment of Power Systems, Models, Methods, and Ap-
plications. Piscataway, NJ, USA: IEEE, 2005.
[2] T. A. Short, Electric Power Distribution Handbook. Boca Raton, FL,
USA: CRC, 2004.
[3] S. Aoshima, T. Tsujimura, and T. Yabuta, A wire mobile robot with
multi-unit structure, in Proc. IEEE/RSJ Int. Workshop Intell. Robots
Syst., 1989, pp. 414421.
[4] J. Sawada, K. Kusumoto, Y. Maikawa, T. Munakata, and Y. Ishikawa,
A mobile robot for inspection of power transmission lines, IEEE
Trans. Power Del., vol. 6, no. 1, pp. 309315, Jan. 1991.
[5] K. Toussaint, S. Montambault, and N. Pouliot, Transmission line
maintenance robots capable of crossing obstacles: State-of-the-art
review and challenges ahead, J. Field Robot., vol. 26, no. 5, pp.
477499, 2009.
[6] J. Katrasnik, F. Pernus, and B. Likar, A survey of mobile robots for
distribution power line inspection, IEEE Trans. Power Del., vol. 25,
no. 1, pp. 485493, Jan. 2010.
[7] P. Debenest, M. Guarnieri, K. Takita, E. F. Fukushima, S. Hirose,
K. Tamura, A. Kimura, H. Kubokawa, N. Iwama, and F. Shiga,
ExplinerRobot for inspection of transmission lines, robotics and
automation, in Proc. IEEE Int. Conf. Robot. Autom., 2008, pp.
39783984.
[8] H. G. Wang, F. Zhang, Y. Jiang, G. J. Liu, and X. J. Peng, Develop-
ment of an inspection robot for 500 kV EHV power transmission lines,
in Proc. IEEE Int. Conf. Intell. Robots Syst., 2010, pp. 51075112.
SONG et al.: A VISION-BASED BROKEN STRAND DETECTION METHOD FOR A POWER-LINE MAINTENANCE ROBOT
[9] L. Tang, S. Fu, L. Fang, and H. Wang, Obstacle-navigation strategy
of a wire-suspend robot for power transmission lines, in Proc. IEEE
Int. Conf. Robot. Biomet., 2004, pp. 8287.
[10] S. Montambault and N. Pouliot, The HQ LineROVer: Contributing to
innovation in transmission line maintenance, in Proc. IEEE Int. Conf.
Transm. Distrib. Construct., Oper. Live-Line Maint., 2003, pp. 3340.
[11] M. F. M. Campos, G. A. S. Pereira, S. R. C. Vale, A. Q. Bracarense, G.
A. Pinheiro, and M. P. Oliveira, A mobile manipulator for installation
and removal of aircraft warning spheres on aerial power transmission
lines, in Proc. IEEE Int. Conf. Robot. Autom., 2002, pp. 35593564.
[12] N. Pouliot and S. Montambault, Geometric design of the linescout, a
teleoperated robot for power line inspection and maintenance, in Proc.
IEEE Int. Conf. Robot. Autom., 2008, pp. 39703977.
[13] N. Pouliot and S. Montambault, LineScout technology: From inspec-
tion to robotic maintenance on live transmission power lines, in Proc.
IEEE Int. Conf. Robot. Autom., 2009, pp. 10341040.
[14] A. Miyazaki, Y. Hatsukade, H. Matsuura, T. Maeda, A. Suzuki, and
S. Tanaka, Detection of wire element breakage in power transmission
line using HTSSQUID, Phys. C, vol. 469, no. 1520, pp. 16431648,
2009.
[15] S. W. Lim and R. A. Shoureshi, Advanced monitoring system for in-
tegrity assessment of electric power transmission lines, in Proc. Amer.
Control Conf., 2006, pp. 1416.
[16] H. Thomas, B. M. Beadle, H. Sprenger, and L. Gaul, Wave-based
defect detection and interwire friction modeling for overhead trans-
mission lines, Archive Appl. Mechan., vol. 79, no. 67, pp. 517528,
2009.
[17] S. D. Kim and M. M. Morcos, Power engineering lettersAn applica-
tion of solenoid sensor for inspecting deterioration of acsr conductors
due to forest fires, IEEE Power Eng. Rev., vol. 21, no. 10, pp. 5052,
2001.
[18] X. L. Jiang, Y. F. Xia, J. L. Hu, Z. J. Zhang, L. C. Shu, and C. S.
Sun, An S-transform and support vector machine (SVM)-based online
method for diagnosing broken strands in transmission lines, Energies,
vol. 4, no. 9, pp. 12781300, Aug. 2011.
[19] C. D. Hernandez-Salazar, A. Baltazar, R. Mijarez, and L. Solis,
Structural damage monitoring on overhead transmission lines using
guided waves and signal processing, in Proc. AIP Conf., 2010, pp.
17211728.
[20] G. Wu, H. Cao, X. Xu, H. Xiao, S. Li, Q. Xu, B. Liu, Q. Wang, Z.
Wang, and Y. Ma, Design and application of inspection system in a
self-governing mobile robot system for high voltage transmission line
inspection, in Proc. IEEE Int. Conf. Power Energy Eng., 2009, pp.
14.
[21] F. J. F. Peters, S. Ramanna, and M. S. Szczuka, Towards a
line-crawling robot obstacle classification system: A rough set
approach, Rough Sets, Fuzzy Sets, Data Mining, and Granular
Computing, LNAI 2639, pp. 303307, 2003.
[22] S. Y. Fu, W. M. Li, Y. C. Zhang, Z. Z. Liang, Z. G. Hou, M. Tan, W. B.
Ye, B. Lian, and Q. Zuo, Structure-constrained obstacles recognition
for power transmission line inspection robot, in Proc. IEEE Int. Conf.
Intell. Robots Syst., 2006, pp. 33633368.
[23] W. H. Li, A. Tajbakhsh, C. Rathbone, and Y. Vashishtha, Image
processing to automate condition assessment of overhead line com-
ponents, in Proc. IEEE Int Conf. Appl. Robot. Power Ind., 2010, pp.
16.
[24] C. S. Hu, G. P. Wu, H. Cao, and X. H. Xiao, Obstacle recognition
and localization based on the monocular vision for double split trans-
mission lines inspection robot, in Proc. IEEE Int. Conf. Image Signal
Process., 2009, pp. 15.
[25] P. Debenest, M. Guarnieri, K. Takita, E. F. Fukushima, S. Hirose,
K. Tamura, A. Kimura, H. Kubokawa, N. Iwama, and F. Shiga,
Sensor-arm robotic manipulator for preventive maintenance and
inspection of high-voltage transmission lines, in Proc. IEEE Int.
Conf. Intell. Robots Syst., 2008, pp. 17371744.
[26] G. Yao, Y. Liu, F. Dong, and B. Lei, An obstacle detection approach
of transmission lines based on contour view synthesis, in Proc. IEEE
Int. Conf. Autom. Logist., 2010, pp. 1924.
[27] H. G. Wang, Y. Jiang, A. H. Liu, L. J. Fang, and L. Ling, Research
of power transmission line maintenance robots in SIACAS, in Proc.
IEEE Int. Conf. Intell. Robots Syst., 2010, pp. 17.
2161
[28] Y. F. Song, H. G. Wang, Y. Jiang, and L. Ling, AApe-D: A novel
power transmission line maintenance robot for broken strand repair,
in Proc. IEEE Int. Conf. Appl. Robot. Power Ind., 2012, pp. 117122.
[29] N. Dalal and B. Triggs, Histograms of oriented gradients for human
detection, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recogn.,
2005, vol. 2, pp. 886893.
[30] D. G. Lowe, Distinctive image features from scale-invariant key-
points, Int. J. Comput. Vis., vol. 2, no. 60, pp. 91110, 2004.
[31] C. Cortes and V. Vapnik, Support-vector networks, Mach. Learning,
vol. 20, no. 3, pp. 273297, 1995.
[32] M. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector
Machines. Cambridge, U.K.: Cambridge Univ. Press, 2000.
[33] B. Schoeumlkopf and A. Smola, Learning With KernelsSupport
Vector Machines, Regularization, Optimization and Beyond. Cam-
bridge, MA, USA: MIT Press, 2001.
[34] Leo, Histograms of oriented gradients. Aug. 2012. [Online]. Available:
http://www.mathworks.com/matlabcentral/fileexchange/33863-his-
tograms-of-oriented-gradients
[35] C. C. Chang and C. J. Lin, LIBSVM : A library for support vector
machines, ACM Trans. Intell. Syst. Technol. vol. 2, pp. 27:127:27,
2011. [Online]. Available: http://www.csie.ntu.edu.tw/cjlin/libsvm
Yifeng Song received the B.Eng. degree in mechan-
ical engineering from Huazhong University of Sci-
ence and Technology, Wuhan, China, in 2007 and is
currently pursuing the Ph.D. degree in mechatronic
engineering at the State Key Laboratory of Robotics,
Shenyang Institute of Automation and University of
Chinese Academy of Sciences, Beijing, China.
His current research interests are power-line
inspection and maintenance robot.
Hongguang Wang (M11) received the B.Eng.
degree in mechanical engineering from Zhengzhou
Institute of Technology, Zhenzhou, China, in 1986,
the M.Eng. degree in mechanical engineering
from the Northeastern University in Mechanics,
Shenyang, China, in 1993, and the Ph.D. degree in
mechatronic engineering from the Graduate School
of the Chinese Academy of Sciences, Beijing, China,
in 2008.
Currently, he is a Professor at the State Key Lab-
oratory of Robotics, Shenyang Institute of Automa-
tion, Chinese Academy of Sciences, Shenyang, China. His current research in-
terests include the analysis and synthesis of robot mechanisms, the mechanics
of serial and parallel manipulators, as well as modular reconfigurable robots and
autonomous mobile robots.
Jianwei Zhang (M92) received the B.Eng. (Hons.)
and M.Eng. degrees in computer science from
the Department of Computer Science of Tsinghua
University, Beijing, China, in 1986 and 1989, re-
spectively, and the Ph.D. degree in computer science
at the Institute of Real-Time Computer Systems
and Robotics, Department of Computer Science,
University of Karlsruhe, Karlsruhe, Germany, in
1994.
Currently, he is a Professor and Director of the In-
stitute of Technical Aspects of Multimodal Systems,
Department of Informatics, University of Hamburg, Hamburg, Germany. His re-
search interests are multimodal information systems, machine learning, service
and cognitive robots, human-computer communication, and networked audio-
video systems.