sensors

Article
Automated Landslides Detection for Mountain Cities
Using Multi-Temporal Remote Sensing Imagery
Zhong Chen, Yifei Zhang *, Chao Ouyang, Feng Zhang and Jie Ma
National Key Laboratory of Science & Technology on Multi-Spectral Information Processing,
Huazhong University of Science and Technology, Wuhan 430074, China; henpacked@163.com (Z.C.);
ouyangchao16@hust.edu.cn (C.O.); M201572329@hust.edu.cn (F.Z.); majie@mail.hust.edu.cn (J.M.)
* Correspondence: zhang_yifei@hust.edu.cn; Tel.: +86-156-0710-9516

Received: 26 January 2018; Accepted: 6 March 2018; Published: 9 March 2018

Abstract: Landslides that take place in mountain cities tend to cause huge casualties and economic
losses, and a precise survey of landslide areas is a critical task for disaster emergency. However, because
of the complicated appearance of the nature, it is difficult to find a spatial regularity that only relates
to landslides, thus landslides detection based on only spatial information or artificial features usually
performs poorly. In this paper, an automated landslides detection approach that is aiming at mountain
cities has been proposed based on pre- and post-event remote sensing images, it mainly utilizes
the knowledge of landslide-related surface covering changes, and makes full use of the temporal
and spatial information. A change detection method using Deep Convolution Neural Network
(DCNN) was introduced to extract the areas where drastic alterations have taken place; then, focusing
on the changed areas, the Spatial Temporal Context Learning (STCL) was conducted to identify
the landslides areas; finally, we use slope degree which is derived from digital elevation model (DEM)
to make the result more reliable, and the change of DEM is used for making the detected areas more
complete. The approach was applied to detecting the landslides in Shenzhen, Zhouqu County and
Beichuan County in China, and a quantitative accuracy assessment has been taken. The assessment
indicates that this approach can guarantee less commission error of landslide areal extent which is below
17.6% and achieves a quality percentage above 61.1%, and for landslide areas, the detection percentage is
also competitive, the experimental results proves the feasibility and accuracy of the proposed approach
for the detection landslides in mountain cities.

Keywords: landslides detection; remote sensing images; change detection; Deep Convolution Neural
Network; Spatial Temporal Context Learning

1. Introduction
Landslides are a major hazard in lots of countries, causing huge property losses and many
casualties every year [1]. Especially in the areas that are close to both cities and mountains, landslides
that are triggered by earthquakes or heavy rain happen frequently and cause even more damage, just
like those that happened in Wenchuan [2]. Against this background, quick landslides hazard emergency
response and accurate risk assessment based on landslide inventories are of great importance and
the key of them is to detect critical areas [3,4]. If a landslide area is detected, its specific analysis can be
carried out for emergency response and the precautions can be done.
Remote sensing is an ideal tool for landslides detection because it offers a wide area of territorial
data with various degrees of both spatial and temporal resolution. It has been widely used in landslides
detection with the help of image analysis technology, which can save human resources costs and make
the process more automatic. In the past, there are two main approaches that use remote sensing images
in landslides detection, one of them only makes use of the post-event data and the landslides areas
are identified by classification, such as scene classification method [5], object-oriented analysis and

Sensors 2018, 18, 821; doi:10.3390/s18030821 www.mdpi.com/journal/sensors

Sensors 2018, 18, 821 2 of 18

classification method [6,7], pixel-based classifying method using Bayesian framework [8], generalized
positive Boolean function [9], and neural network [10]. The methods using post-event image for
classification assume that all the landslides are triggered by the same event like earthquake, rainstorm,
and so on, and have similar spectral and texture characteristics, without indicating the time period
when the landslides happened.
The other one uses data of pixels or features that are calculated by rule to conduct multi-temporary
change detection between pre-event and post-event data, including the method based on image difference
and landslide related factor [11], method based on change detection [12], method using change detection
coupled with false alarm removal [13] or texture analysis [14], and so on. Binary temporal methods
can use temporal spatial information to reduce the search area and specify the time period that is
determined by the dates of experimental image pair. Both of the two approaches use classification
to confirm the final landslide areas, the classification is conducted based on the model trained
by landslide samples [5,9,10,15] or the rules made of features selected by experts [6–8,11–14,16],
the parameters of model are determined through trial and error, and the threshold of each feature
is determined through the knowledge of landslide areas that have been found. Actually, the use
of classification inevitably leads to omission/commission errors, and applying artificial features to
recognition landslides would mistake other land covering types for landslides because the spatial
regularity that only relates to landslides have not be found. Additionally, most methods using training
samples are semi-automated because samples should be extracted by experts and it is difficult to
acquire appropriate landslide samples in many areas where there is no landslides that have recorded
in the past. For the above reasons, an automated method, which uses landslide-related knowledge for
landslides detection without information of landslides inventories, is needed. Behling et al. [17] have
proposed a multi-temporal landslide detection method based on NDVI-trajectories and relief-oriented
analysis because the occurrence of landslides may cause an abrupt decrease of vegetation covering
in mountainous regions. However, the method using only the change of vegetation covering can not
identify the whole landslide areas in mountain cities where the occurrence of landslides may cause
more casualties and serious economic losses, because landslides in mountains cities may cause not
only the decrease of vegetation covering, but also the vanishing of buildings.
Deep Convolutional Neural Network (DCNN) have attracted widespread is the element-wise
product. Then, the spatial context model can be calculated byattention since Krizhevsky et al. [18]
revealed its outstanding capacity for image classification in 2012, and then it was successfully applied
to object detection [19], natural language processing [20], and speech recognition [21]. The multi-layer
feature extraction framework of DCNN contributes to abstracting the features that can more effectively
describe the nature of images than artificial features, because artificial features usually only concentrate
on one aspect of spatial regularity, such as texture or spectrum. Spatial Temporal Context Learning
(STCL) was proposed by Zhang et al. [22] for accurately real time visual tracking. The method
mainly builds spatial context model of object and uses the model for detection in time-series images.
When compared with the classification method, STCL makes full use of the temporal and spatial
information, and avoids omission/commission errors, thus it was chosen for landslides detection.
In this article, a novel automated method based on the knowledge about landslide-related surface
covering change in mountain cities is proposed. The approach introduces a new change detection
method based on DCNN to find changed area. The STCL is used to identify the specific landslide
areas based on the change of vegetation covering and built-up areas. The vegetation covering areas are
represented by Normalized Difference Vegetation Index (NDVI), and the built-up areas are identified by
calculation of the texture-derived built-up presence index, which is called PANTEX [23]. The feasibility
and accuracy of the proposed method is validated using remote sensing imagery of mountain cities.

2. Study Area and Data

Landslides in Shenzhen, Zhouqu Country, and Beichuan County are chosen to validate the proposed
method. These three Chinese cities are mountainous and they suffered significant losses because of
Sensors 2018, 18, x FOR PEER REVIEW 3 of 18

2. Study Area and Data

Sensors 2018, 18, 821 3 of 18
Landslides in Shenzhen, Zhouqu Country, and Beichuan County are chosen to validate the
proposed method. These three Chinese cities are mountainous and they suffered significant losses
landslides.
because of The landslides
landslides. of Shenzhen
The landslidesdated on 20 December
of Shenzhen dated on 2015
20led to the loss
December of 69led
2015 livesto and
the dozens
loss of
of
69 buildings
lives and in ruins,of
dozens and the massive
buildings landslides
in ruins, and the coupled with
massive debris flow
landslides in Zhouqu
coupled with County where
debris flow in
earthquakes,
Zhouqu County landslides, and debris flow
where earthquakes, often occur,
landslides, andhappened
debris flowonoften
7 August
occur,2010, leavingon
happened 1765 people
7 August
dead or missing,
2010, leaving 1765and smashed
people dead or thousands
missing, andof houses.
smashed Beichuan
thousands County was devastated
of houses. Beichuan County because of
was
the Wenchuan earthquake happened on 12 May 2008, the earthquake
devastated because of the Wenchuan earthquake happened on 12 May 2008, the earthquake triggered lots of landslides that
destroyed
triggered lotsthe city and buried
of landslides thousands
that destroyed of the
people.
city and buried thousands of people.
Figure 11showsshowsthethe experimental
experimental images for landslides
images detection.
for landslides The three
detection. The image
threesets are processed
image sets are
with registration
processed [24], and then
with registration histogram
[24], and thenmatching
histogram is applied
matching to is
removing
applied the influence the
to removing of discrepant
influence
radiation
of discrepantintensity [25]. intensity
radiation Landslides that
[25]. had taken
Landslides place
that hadintaken
the three
placesites ledthree
in the to drastic
sites led alteration in
to drastic
land surface
alteration in and
landobvious
surface sediment
and obvious transport.
sedimentIn Figure 2a–d,Inwe
transport. can see
Figure thewe
2a–d, sediment
can seetransport bury
the sediment
vegetation
transport bury and buildings.
vegetation Additionally,
and buildings. the landslides
Additionally,detection
the method
landslidesusesdetection
a digital elevation
method uses modela
(DEM) of 5 m spatial
digital elevation modelresolution
(DEM) of to5derive
m spatialslope information,
resolution whichslope
to derive can make the landslides
information, which can detection
make
more reliable [17],
the landslides and themore
detection change of DEM
reliable is also
[17], andused for recognizing
the change of DEMthe residual
is also used part
forofrecognizing
landslide areas
the
where
residual there
partare ofno change areas
landslide of the where
cover type.
there are no change of the cover type.

a c e

0 0.75 1.5 0 2.5 5 0 0.75 1.5

km km km

b d f

0 0.75 1.5 0 2.5 5 0 0.75 1.5

km km km

Figure 1.1. Pre-

Pre- and
and post-event
post-event satellite
satellite imagery. (a) (a) GF-1
GF-1 data
data taken
taken inin Shenzhen, China on
24 December 2013; (b) GF-1 data taken in Shenzhen, China on 29 December 2015; (c) Landsat-8 data
Zhouqu County,
taken in Zhouqu County,China
Chinaon
on1313July
July2009;
2009;(d)
(d)Landsat-8
Landsat-8 data
data taken
taken in in Zhouqu
Zhouqu County,
County, China
China on
on September
10 10 September 2010;
2010; (e) Formosat-2
(e) Formosat-2 datadata taken
taken in Beichuan
in Beichuan County
County on 5 on 5 April
April 2006;2006; (f) Formosat-2
(f) Formosat-2 data
data taken
taken in Beichuan
in Beichuan CountyCounty on 162008.
on 16 May MayAll
2008. All ofimages
of above above have
images have 4(blue,
4 bands bands (blue,
green, redgreen, red
and near
infrared). The landslides
and near infrared). in Beichuan
The landslides County are
in Beichuan different
County are from the rest
different frombecause
the restofbecause
the wideof coverage
the wide
and the large
coverage and sum.
the large sum.
 Sensors 2018, 18, 821 4 of 18
Sensors 2018, 18, x FOR PEER REVIEW 4 of 18

a b

c d

Figure 2.Figure 2. (a) Aerial

(a) Aerial images images of landslide
of landslide regionsin
regions in Shenzhen;
Shenzhen; (b)(b)
Aerial images
Aerial of landslide
images regions in
of landslide regions in
Zhouqu County; (c,d) Aerial images of landslide regions in Beichuan County.
Zhouqu County; (c,d) Aerial images of landslide regions in Beichuan County.
3. Methodology
3. Methodology
The proposed method aims at the automated landslides detection in mountain cities using
Themulti-temporal
proposed method aims data.
remote sensing at the automated
Thus, the changeslandslides detection
of the earth surface duringin the
mountain cities using
analyzed time
span should be taken in consideration and the landslide-related phenomena in the
multi-temporal remote sensing data. Thus, the changes of the earth surface during the analyzed study area should be time
used for extracting the landslide areas. To meet these requirements, the method uses a patch-oriented
span should be taken in consideration and the landslide-related phenomena in the study area should
change detection and conducts a spatial-temporal analysis of the NDVI and PANTEX in the changed
be usedpatches
for extracting the landslide areas. To meet these requirements, the method uses a
of the image. The Integral process of the proposed method is showed in Figure 3.
patch-oriented changeofdetection
Procedures the proposedandapproach
conducts for alandslides
spatial-temporal analysis
detection are of the NDVI and PANTEX
as following:
in the changed patches of the image. The Integral process of the proposed method is
1. Cloud-covered regions ought to be abandoned in all images. Unconcerned regions with
showed in Figure 3.
vegetation, water and building are also supposed to be discarded in the post-event image,
Proceduresthen of
thethe proposed
possible approach
landslide areas are for landslides
obtained (Sectiondetection
3.1). are as following:
2. Extract changed areas by applying change detection with Deep Convolution Neural Network to
1. Cloud-covered regions ought to be abandoned in all images. Unconcerned regions with
pre- and post-event images (Section 3.2).
vegetation,
3.
water and building are also supposed to be discarded in the post-event image, then
STCL aiming at binary temporal NDVI and PANTEX in changed areas is used and the suspicious
the possible landslide
landslide areas is areas are(Section
extracted obtained
3.3).(Section 3.1).
2. Extract
4. changed areas is
The post process byadded
applying
to get change detection
more reliable with Deep
and complete Convolution
landslide Neural
areas (Section 3.4). Network to
pre- and post-event images (Section 3.2).
3. STCL aiming at binary temporal NDVI and PANTEX in changed areas is used and the
suspicious landslide areas is extracted (Section 3.3).
4. The post process is added to get more reliable and complete landslide areas (Section 3.4).
Sensors 2018, 18, 821 5 of 18
Sensors 2018, 18, x FOR PEER REVIEW 5 of 18

Input Remove the irrelevant area(Sect 3.1)

Pre-
landslide Binary
Removing cloud Binary Image of
Bi-temporal image a
images Post- possible landslide
landslide Removing cloud、water、 Binary area
vegetation and built-up area image b

Change detection (Sect 3.2)

Change detection Binary Image of

CNN model
using CNN changed area

Spatio-temporary context analysis (Sect 3.3)

Conducting
Calculation
STC using
of NDVI Binary image of
Bi-temporal NDVI
images of suspicious
Conducting landslide area
changed area Calculation
STC using
of PANTEX
PANTEX

Post processing (Sect 3.4)

Average slope Filtering Checking nearby areas Result binary

DEM
> 7° small area by the change of DEM image

Figure
Figure 3. Integral process
3. Integral process of
of proposed
proposed landslides
landslides detection
detection approach
approach for
for mountain
mountain cities
cities using
using
multi-temporal
multi-temporal remote
remote sensing
sensing imagery.
imagery.

3.1. Remove the Irrelevant Areas

According
According toto the
theterrain
terraintransformation
transformationanalysis
analysis[26] that
[26] is related
that to landslides,
is related regions
to landslides, thatthat
regions are
covered by vegetation,
are covered water,
by vegetation, and and
water, buildings in post-event
buildings images
in post-event can be
images canignored to reduce
be ignored the area
to reduce the
of terrain
area to be to
of terrain detected, and the
be detected, computational
and cost ofcost
the computational the of
whole method.
the whole Besides,
method. areas with
Besides, areascloud
with
in pre-inand
cloud post-landslide
pre- imagesimages
and post-landslide should should
be abandoned to guarantee
be abandoned finding finding
to guarantee real changed area in
real changed
change detection.
area in change detection.
The built-up areas are extracted by calculating PANTEX, the index is based on fuzzy rule-based
composition of anisotropic textural co-occurrence measures that are derived from the gray-level gray-level
co-occurrence matrix (GLCM) [23]. Calculating
Calculating NDVI,
NDVI, Normalized
Normalized Difference
Difference Water
Water Index (NDWI),
for each pixel is a simple
simple and
and quick
quick method
method to obtain
obtain vegetation
vegetation covering and water covering from
remote sensing data [27,28]. The two indexes are defined
defined asas

NNIR−RR
NDVI
NDVI =  IR ,, (1)
(1)
IRIR+RR
NN

G − NIR
NDWI = GN , (2)
NDWI G + NIRIR , (2)
G  NIR
where NIR stands for near infrared band, R stands for red band and G is the green band.
whereIn the
N IRexperiment, the threshold
stands for near of NDVI
infrared band, and NDWI
R stands for redisband
0.32 and
and 0,G respectively, the pixel whose
is the green band.
valueIn
is the
higher than the threshold means that there is vegetation or water, and the pixel
experiment, the threshold of NDVI and NDWI is 0.32 and 0, respectively, the pixel whose PANTEX
whoseis
higher than
value is 0.3 would
higher than be
thedetermined
threshold as a built-up
means that area.
thereFor
is masking
vegetationclouds, a threshold
or water, (threshold
and the = 180)
pixel whose
PANTEX is higher than 0.3 would be determined as a built-up area. For masking clouds, a threshold
(threshold = 180) has been applied to the average value of four bands, the threshold is selected
Sensors 2018, 18, 821 6 of 18
Sensors 2018, 18, x FOR PEER REVIEW 6 of 18

thinking
has of separating
been applied temporalvalue
to the average bright of objects (e.g.,the
four bands, clouds) fromispermanent
threshold ones (e.g.,
selected thinking buildings
of separating
and sand).
temporal bright objects (e.g., clouds) from permanent ones (e.g., buildings and sand).
Aftermarking
After markingthethe vegetation,
vegetation, water,water, and built-up
and built-up areas inareas in post-event
post-event images and images
clouds and clouds
covering in
covering
all in all ofbinary
of the images, the images,
image binary
a and bimage a and
in Figure b in
3 can beFigure 3 canThe
generated. be generated. The in
values of pixels values of image
binary pixels
ainwould
binarybeimage
0 if theapixel
would be 0same
in the if the pixel in pre-landslide
position the same positionimageinhave
pre-landslide
been marked; image
otherhave been
pixels in
marked; other pixels in binary image a would take 1 as their value. For binary image
binary image a would take 1 as their value. For binary image b, the values of pixels which have been b, the values of
pixels which
marked have been marked
in post-landslide in post-landslide
images would be 0, othersimages
would would be 0, image
be 1. Binary others that
would showsbe 1. Binary
possible
image that shows possible landslide regions is made of above images a and b, it is
landslide regions is made of above images a and b, it is the result of “and” operation from two images,the result of “and”
operation
so its pixelfrom
whose two images,
value so its pixelthe
is 1 represents whose value
possible is 1 represents
landslide region.the possible landslide region.

3.2.
3.2. Change
Change Detection
Detection Using
Using DCNN
DCNN
In
In this
thissection,
section,change
changedetection
detectionisisapplied
appliedtotothe possible
the possiblelandslide areas
landslide that
areas have
that been
have found
been in
found
Section 3.1. As
in Section 3.1.shown in Figure
As shown in 4, the key
Figure 4, ideas are leveraging
the key ideas are the high capacity
leveraging of Deep
the high Convolutional
capacity of Deep
Neural NetworkNeural
Convolutional for feature extraction
Network for [29,30]
featuretoextraction
learn robust representations
[29,30] of images,
to learn robust and calculating
representations of
the Manhattan
images, distance between
and calculating representations
the Manhattan distanceof between
corresponding possible landslide
representations area in twopossible
of corresponding images,
the Manhattan
landslide area indistance is used
two images, thefor measuring
Manhattan the change
distance degree,
is used and then the
for measuring a threshold is usedand
change degree, to
determine whether
then a threshold is the
usedchange have occurred
to determine whether in the
thischange
area. have occurred in this area.

Resize to
28x28
DCNN
Input
Distance(D)
Manhattan

Output
Pre-landslide image Feature 1 D>=Threshold: Change
D<Threshold: No Change
Resize to
28x28
DCNN
Input
Output
Post-landslide image Feature 2

Figure 4.
Figure 4. Change
Change detection
detection based
based on
on Deep
Deep Convolution
Convolution Neural
Neural Network.
Network.

3.2.1. Feature
3.2.1. Feature Extraction
Extraction
The architecture
The architecture of of our
our DCNN
DCNN model
model isis summarized
summarized ininFigureFigure5.5. It
It contains
contains five
five learned
learned
layers—two convolutional, two pooling (average pooling and max pooling),
layers—two convolutional, two pooling (average pooling and max pooling), and a fully connected. and a fully connected.
In convolutional
In convolutional process,
process,thethesize
sizeofofkernels
kernelsisis55×× 55 and
and thethe stride
stride parameter
parameter is is 1;
1; in pooling process,
in pooling process,
the size
the size of
of kernels
kernels isis33×× 33 and
and the
the stride parameter is
stride parameter is 2.
2. The
The DCNN
DCNN model
model isis trained
trained by SAT-4 [31]
by SAT-4 [31]
dataset, and an extra fully connected layer is added for classification in the training
dataset, and an extra fully connected layer is added for classification in the training process, this layer process, this
is
layer is not shown in Figure 5 because it would be
not shown in Figure 5 because it would be abandoned in the feature extraction process. abandoned in the feature
extraction process.
After the DCNN model has been learnt, the possible landslide pixels with their 100 × 100 neighbor
would be resizedDCNN
After the to 28 ×model
28 image haspatches
been learnt,
that arethe
the possible landslide
input of the pixels
model, and thewith
64-D their
output100 × 100
vector is
neighbor
the featurewould be resizedimage
of corresponding to 28 patch.
× 28 image
Due topatches that aredeeply
the multilayer the input of the model,
abstraction of DCNN, andthethe 64-D
feature
output vector
consists is the texture,
of spectrum, feature and
of corresponding image
shape information thatpatch. Due tothe
can describe thenature
multilayer
of thedeeply abstraction
input image patch.
of DCNN, the feature consists of spectrum, texture, and shape information that can describe the
nature of the input image patch.
 Sensors
Sensors 2018, 18, x2018,
FOR 18,PEER
821 REVIEW 7 of 18 7 of 18

Figure 5. Structure of the Deep Convolution Neural Network.

Figure 5. Structure of the Deep Convolution Neural Network.
3.2.2. Change Detection
3.2.2. Change Detection
Since the feature extracted by DCNN can describe the nature of the image, if changes happen in
Since
an the
area,feature extracted
the feature by DCNN part
of the corresponding can ofdescribe
the imagethe nature
would of the
change image, Therefore,
accordingly. if changes happen in
after
extracting the two features of the corresponding 100 × 100 neighbor that center
an area, the feature of the corresponding part of the image would change accordingly. Therefore, on the same possible
landslide pixel in pre- and post-landslide image, the Manhattan distance is calculated between these
after extracting the two features of the corresponding 100 × 100 neighbor that center on the same
two features. Manhattan distance between feature (x1 , x2 , . . . , xn ) and feature (y1 , y2 , . . . , yn ) can be
possible expressed
landslide pixel in pre- and post-landslide image, the Manhattan distance is calculated
as follows:
between these two features. Manhattan distance
D = | x1 − y1 | + between
| x2 − y2 | + . . . + | xn −feature
y n |, ( 𝑥1 , 𝑥2 , … , 𝑥𝑛 ) and
(3) feature
(𝑦1 , 𝑦2 , … , 𝑦𝑛 )Ifcan be expressed as follows:
the Manhattan distance is bigger than the threshold, the pixel and its 100 × 100 neighbor would
be determined as a changedDpossible
= |𝑥1landslide
− 𝑦1 | + area,
|𝑥2 −and
𝑦2the
| +value
⋯ + of
|𝑥𝑛pixels
− 𝑦𝑛in|,this area can be retained (3)
based on the binary image that shows possible landslide regions in Section 3.1; but when the Manhattan
If the Manhattan
distance does notdistance is biggerthese
reach the threshold, thanpixels
the threshold, the pixel
would be modified to 0. and its 100
Though × 100 neighbor
the threshold
should be modified when the experimental area or data is changed, it is not
would be determined as a changed possible landslide area, and the value of pixels in this sensitive and it can be setarea
to can be
1.5 times the average Manhattan distance between features of all the corresponding patches in pre- and
retained based on the binary image that shows possible landslide regions in Section 3.1; but when
post-landslide image. The process is visualized in Figure 6. It obviously shows that the landslide-related
the Manhattan distance does not reach the threshold, these pixels would be modified to 0. Though
surface covering change causes larger variation of features than that caused by common artificial or
the threshold
naturalshould be modified
surface covering change.when the experimental area or data is changed, it is not sensitive
and it can After
be setthistoprocedure,
1.5 times the average
a binary image that Manhattan distanceregions
indicates changed betweenhave features
been formed. of all the
Because the change detection is only applied to possible landslide pixels and their
corresponding patches in pre- and post-landslide image. The process is visualized in Figure 6. It neighbor, the pixels
whose values are still 1 represent the areas where the landslide may occur and the striking changes of
obviously shows that the landslide-related surface covering change causes larger variation of
the earth surface have happened between the date of pre- and post-landslide image.
features than that caused by common artificial or natural surface covering change.
After this procedure, a binary image that indicates changed regions have been formed. Because
the change detection is only applied to possible landslide pixels and their neighbor, the pixels whose
values are still 1 represent the areas where the landslide may occur and the striking changes of the
earth surface have happened between the date of pre- and post-landslide image.
 Sensors 2018, 18, 821 8 of 18
Sensors 2018, 18, x FOR PEER REVIEW 8 of 18

Post-landslide Pre-landslide No-landside area No-landside area No-landside area

Input image

Output feature

Subtraction image
(L1-norm)

Sum = 1229 Sum = 315 Sum = 472 Sum = 253

Landslides in Shenzhen (average Manhattan distance = 350 Threshold = 525)

Post-landslide Pre-landslide No-landside area No-landside area No-landside area

Input image

Output feature

Subtraction image
(L1-norm)

Sum = 232 Sum = 95 Sum = 126 Sum = 103

Landslides in Zhouqu County (average Manhattan distance = 114 Threshold = 171)
Post-landslide Pre-landslide No-landside area No-landside area No-landside area

Input image

Output feature

Subtraction image
(L1-norm)

Sum = 1134 Sum = 447 Sum = 392 Sum = 501

Landslides in Beichuan County (average Manhattan distance = 568 Threshold = 852)

Figure6.
Figure Visualizing the
6. Visualizing the change
change detection
detectionprocess
processofofthree
threelandslides.
landslides.ForFor
each landslides,
each the the
landslides, firstfirst
row
images
row are 4are
images pairs of image
4 pairs patches
of image chosenchosen
patches for visualization, each pair
for visualization, image
each pairpatches
imageare the same
patches are area
the
in pre- and post-landslide image, the first pair is landslide area, others are no-landslide
same area in pre- and post-landslide image, the first pair is landslide area, others are no-landslide area; The second
row are
area; Theoutput
secondfeatures
row of areDCNN,
outputthe 64-D output
features of DCNN,featurestheare64-D
reshaped
output × 8 feature
to 8features are images
reshaped for
visualization; Subtraction operator is conduct on each pair of feature images with
to 8 × 8 feature images for visualization; Subtraction operator is conduct on each pair of feature L-1 norm, the results
are shown
images withinL-1
the third
norm,row.the The sumare
results of pixels
shownvalues
in theinthird
everyrow.
subtraction
The sumimage is calculated,
of pixels values it
inisevery
equal
to Manhattan
subtraction distance
image between it
is calculated, the
is two
equal features of the pair
to Manhattan of image
distance patches.
between the two features of the pair
of image patches.
3.3. Spatial Temporal Context Analysis
3.3. Spatial
In thisTemporal
section, Context
the exactAnalysis
landslide areas are detected based on spatial and temporary information,
and In thethis
process is pixel-oriented
section, for higherareas
the exact landslide extraction accuracy. based
are detected The STCL on algorithm
spatial and is used in this
temporary
process, it is and
information, basedtheon formulating
process the spatialfor
is pixel-oriented andhigher
temporal relationships
extraction accuracy.between
The STCLthe target and its
algorithm is
local context. A Bayesian framework is used to express the statistical correlation
used in this process, it is based on formulating the spatial and temporal relationships between theof low-level features
(i.e., intensity
target and its and
localposition).
context. InA our research,
Bayesian changed pixels
framework is usedin to
Section
express3.2 are
the regarded
statisticalascorrelation
the target and
of
the two type spatial context models of these pixels are built based on NVDI and PANTEX
low-level features (i.e., intensity and position). In our research, changed pixels in Section 3.2 are information,
respectively,
regarded in pre-landslide
as the target and the image, then the
two type NDVI
spatial models
context and PANTEX
models of thesemodels
pixels are used
built to identify
based on
the same
NVDI andposition
PANTEX in information,
the post-landslide image to
respectively, indetermine
pre-landslideweather
image, thethen
vegetation
the NDVI or built-up
models andarea
PANTEX models are used to identify the same position in the post-landslide image to determine
Sensors 2018, 18, 821 9 of 18

have disappeared, in consideration of the landslide-related disappearance of vegetation and buildings

in mountain cities.

3.3.1. Spatial Context Modeling

In STCL, a confidence map is computed as a following equation to estimate the target location
likelihood:
c( x ) = P( x |o )
(4)
= ∑c(z)∈X c P( x |c(z), o ) P(c(z)|o ) ,

where x ∈ R2 is the location of target and o indicates the target present in the scene. The spatial
context set is defined as X c = {c(z) = ( I (z), z)|z ∈ Ωc ( x ∗ )} and I (z) denotes the intensity of location
z in image, Ωc ( x ∗ ) denotes the 21 × 21 neighborhood of the object location x ∗ . Location likelihood is
based on condition probability P( x |c(z), o ) and prior probability P(c(z)|o ) . The condition probability
is spatial context model that describes the spatial relationship between the target and its context
information, it is defined as:
P( x |c(z), o ) = hsc ( x − z), (5)

where hsc ( x − z) is related to the distance and direction between the location x of the target and its
neighbor context location z. P(c(z)|o ) denotes a context prior probability, it is computed as follows:

P(c(z)|o ) = I (z)wσ (z − x ∗ ), (6)

with a weighted function wσ ( x ) which is defined by

 
wσ ( x ) = a × exp[−x 2 / (σ2 )], (7)
 

where parameter a is used for normalization and σ is a scale parameter.

If the target location x ∗ is known, the confidence map can be calculated as the following equation:
  
c( x ) = P( x o ) = b × exp[−( x − x ∗ )/ α β ], (8)
  

where b is a normalization constant, β and α is, respectively, scale parameter and shape parameter.
Based on (5)–(8), the likelihood (4) can be written as:

= b × exp[−( x − x ∗ )/ α β ]
 
c( x )
= ∑z∈Ωc (x∗ ) hsc ( x − z) I (z)wσ (z − x ∗ ) , (9)
= hsc ( x ) ⊗ ( I ( x )wσ ( x − x ∗ ))

where ⊗ represents the operator of convolution.

In consideration of that convolution can be replaced by Fast Fourier Transform (FFT), (8) can be
expressed as:
   
F b × exp[−( x − x ∗ )/α β ] = F (hsc ( x ))  F ( I ( x )wσ ( x − x ∗ )), (10)
 

where F denotes FFT and  is the element-wise product. Then, the spatial context model can be
calculated by (  ∗
β
)
−1 F b × exp[− ( x − x ) /α ]
 
sc
h (x) = F ∗
, (11)
F ( I ( x )wσ ( x − x ))

where F −1 is the inverse FFT.

In our experiment, every changed pixel in possible landslide area is regarded as target for which
two models (hsc sc sc
NDV I ( x ) and h PANTEX ( x )) are built, h NDV I ( x ) is based on NVDI image computed as
(1), hsc
PANTEX ( x ) is based on PANTEX image computed as [23], the parameters of likelihood function
Sensors 2018, 18, 821 10 of 18

are set to α = 2.25 and β = 1, the scale parameter σ = 1 and the normalization constant a and b is
respectively 0.075 and 0.5 according to [22].

3.3.2. Temporary Confidence Learning

After getting the spatial context model of the changed possible landslide pixel in pre-landslide
image, a confidence is computed for the same location in post-landslide image. At first, the NDVI
image and PANTEX image are calculated based on post-landslide image, and then for every changed
possible landslide pixel, convolution process using NDVI spatial model and PANTEX spatial model is
conducted to compute the confidence maps, as follows:

c NDV I ( x ) = F −1 (F (hsc ∗
NDV I ( x ))  F ( I NDV I ( x ) wσ ( x − x ))), (12)

c PANTEX ( x ) = F −1 (F (hsc ∗
PANTEX ( x ))  F ( IPANTEX ( x ) wσ ( x − x ))), (13)

where INDV I ( x ) and IPANTEX ( x ) denotes the intensity of locating x in NDVI image and PANTEX
image calculated based on post-landslide image, x ∈ Ωc ( x ∗ ), where x ∗ is the location of the changed
possible landslide pixel. The confidence map is a 21 × 21 matrix, and the likelihood L of the location
in post-landslide image is the value of matrix center, L is represented as L NDV I = c NDV I (11, 11),
and L PANTEX = c PANTEX (11, 11). if L NDV I or L PANTEX is smaller than 0, it seems that changes
have happened in the vegetation or built-up area, and the corresponding pixel is determined as
suspicious landslide pixel because landslides can lead to disappearing of vegetation and buildings in
mountain cities.

3.4. Post Processing

Suspicious landslide areas are further evaluated with relief-based plausibility using the slope
parameter which is derived from DEM and has been generally used for landslide validation [32–34],
it considers the fact that the occurrence of landslides needs an initial relief contrast that enable
the downward flow of material. In the experiments, the suspicious landslide areas whose average
slope is greater than 7◦ are confirmed to be landslide areas. Then, minute and scattered detected regions
are supposed to be filtered out according to their size, because of the spatial concentration of landslides,
and the size threshold is 50 (pixels) in the experiments. Finally, because there are some areas without
cover type changes but with topography changes should also be part of landslide areas, and they are
usually connected to landslide areas that are with cover type changes, thus for all of the pixels that are
connected to the detected landslide areas, the change of DEM data in corresponding position should be
taken into account. If the change of DEM data is higher than 4m, then the corresponding pixel would
be taken as part of landslide areas, and the pixels that are connected to this new part of landslide areas
should also be checked in the same way.

4. Result and Discussion

4.1. Experiment Result

Figure 7 shows the results of Shenzhen’s landslide areas being detected by the proposed method.
The binary image of possible landslide areas is shown in Figure 7a, from which we can see the possible
landslide areas, which are represented by white pixels, occupy a small number of pixels and have
very high concentration in landslide area because the main landslide-unrelated land covering types
(i.e., vegetation, water, and built-up area) have been extracted and removed. Figure 7b shows changed
possible landslide areas that are marked as white pixels, when compared to Figure 7a, this process only
retains the white pixels that are in changed area, and removes others to prevent the confusing causing
by land covering types that are visually similar to landslides in remote sensing images (e.g., bare land).
In Figure 7c, we can see some white regions were abandoned by STCL because in the changed area
there are still many areas that are visually similar to landslide. With the help of STCL that is based
Sensors 2018, 18, 821 11 of 18
Sensors 2018, 18, x FOR PEER REVIEW 11 of 18

on NDVI and
vegetation or PANTEX,
buildings onlyhavethe areas whereinvegetation
disappeared the givenorperiod
buildings
canhave disappeared
be left, in keepingin the
withgiven
the
period can be left, in keeping with the knowledge about landslide-related surface covering
knowledge about landslide-related surface covering changes in mountain cities. In consideration of changes in
mountain
the cities. Inand
concentrative consideration of theofconcentrative
spatial pattern and spatial
landslides, regions thatpattern of landslides,
are smaller regions
than 50 are thatout,
filtered are
smaller than 50 are filtered out, and some pixels which are connected to the detected landslide
and some pixels which are connected to the detected landslide areas are included as part of landslide areas
are included
areas becauseasof parttheof great
landslide areasofbecause
change DEM in of the great changeareas,
corresponding of DEM theinfinal
corresponding areas,
result is shown
theFigure
in final result
7d. is shown in Figure 7d.

(a) (b)

(c) (d)
Figure 7. Experimental results of Shenzhen, China. (a) Black-white image of possible landslide
Figure 7. Experimental results of Shenzhen, China. (a) Black-white image of possible landslide regions
regions (Section 3.1); (b) Black-white image of changed regions (Section 3.2); (c) Black-white image of
(Section 3.1); (b) Black-white image of changed regions (Section 3.2); (c) Black-white image of suspicious
suspicious landslide regions (Section 3.3); and, (d) Final result image (Section 3.4).
landslide regions (Section 3.3); and, (d) Final result image (Section 3.4).

Results
Results of the landslides
of the landslides detection
detection in Zhouqu County
in Zhouqu County are shown in
are shown in Figure
Figure 8.
8. The
The binary
binary image
image
of possible landslide
of possible landslide areas is shown
areas is shown inin Figure
Figure 8a.
8a. The
The Binary
Binary image
image ofof changed
changed areas
areas is shown in
is shown in
Figure 8b, we can find the river was incorrectly extracted, but it was removed by STCL whose
Figure 8b, we can find the river was incorrectly extracted, but it was removed by STCL whose result is result
is shown
shown in Figure
in Figure 8c. Figure
8c. Figure 8d result
8d is the is theimage
resultand
image andthe
it shows it location
shows the
andlocation
shape of and shape of
the landslides.
the landslides.
Sensors 2018, 18, 821 12 of 18
Sensors 2018, 18, x FOR PEER REVIEW 12 of 18
Sensors 2018, 18, x FOR PEER REVIEW 12 of 18

(a) (b)
(a) (b)

(c) (d)
(c) (d)
Figure 8. Experimental results of Zhouqu County, China. (a) Black-white image of possible landslide
Figure 8.
Figure Experimental results
8. Experimental results of
of Zhouqu
Zhouqu County,
County, China. (a) Black-white
China. (a) Black-white image
image ofof possible
possible landslide
landslide
regions (Section 3.1); (b) Black-white image of changed regions (Section 3.2); (c) Black-white image of
regions (Section 3.1); (b) Black-white image of changed regions (Section 3.2); (c) Black-white
regions (Section 3.1); (b) Black-white image of changed regions (Section 3.2); (c) Black-white image image of
of
suspicious landslide regions (Section 3.3); (d) Final result image (Section 3.4).
suspicious landslide regions (Section 3.3); (d) Final result image (Section 3.4).
suspicious landslide regions (Section 3.3); (d) Final result image (Section 3.4).

Results of
Results of the
the landslides
landslides detection
detection in Beichuan County
in Beichuan
Beichuan are
County are shown
are shown
shown inin Figure 9.
in Figure
Figure 9. There are
There are many
are many
many
Results of the landslides detection in County 9. There
landslides that
landslides that are
that are widely
are widely distributed
widely distributed and
distributed and part
and part of
part of them
of them are
them are close
are close to
close to others,
to others, and the changed regions
landslides others, and
and the
the changed
changed regions
regions
(Figure 9b)
(Figure 9b) make
make up
up most
mostofofthe
theimage.
image. With thethe
With help of of
help STCLSTCL(Figure 9c) 9c)
(Figure andand
the the
postpost
process, the
process,
(Figure 9b) make up most of the image. With the help of STCL (Figure 9c) and the post process, the
irrelevant
the regions
irrelevant were
regionswere removed
were removed
removed and and the outline and independence of landslides were
irrelevant regions andthe outline
the and and
outline independence of landslides
independence were retained
of landslides were
retained
(Figure (Figure 9d).
retained9d).
(Figure 9d).

(a) (b)
(a) (b)

Figure 9. Cont.
Sensors 2018, 18, 821 13 of 18
Sensors 2018, 18, x FOR PEER REVIEW 13 of 18

(c) (d)
Figure
Figure 9.9.Experimental
Experimentalresults
results of Beichuan
of Beichuan County,
County, China.China. (a) Black-white
(a) Black-white image ofimage
possibleof landslide
possible
landslide regions (Section 3.1); (b) Black-white image of changed regions (Section
regions (Section 3.1); (b) Black-white image of changed regions (Section 3.2); (c) Black-white image 3.2);
of
(c) Black-white image of suspicious landslide regions (Section 3.3); and,
suspicious landslide regions (Section 3.3); and, (d) Final result image (Section 3.4). (d) Final result
image (Section 3.4).
4.2. Accuracy Assessment
4.2. Accuracy Assessment
To acquire a synthetical assessment of the landslides detection method, consistency analysis
To acquire a synthetical assessment of the landslides detection method, consistency analysis
between the automated detected landslides and the real ones should be done. The comparison puts
between the automated detected landslides and the real ones should be done. The comparison puts
the experimental areas into three classes: true positive (TP), false positive (FP) and false negative
the experimental areas into three classes: true positive (TP), false positive (FP) and false negative
(FN). TPs contains the real landslide pixels which have been found by the method. FNs and FPs
(FN). TPs contains the real landslide pixels which have been found by the method. FNs and FPs
denotes two different error, the former represents the missing landslide pixels, and the latter represents
denotes two different error, the former represents the missing landslide pixels, and the latter
the wrongly extracted pixels, which are not landslides.
represents the wrongly extracted pixels, which are not landslides.
Further, several evaluation indicators, DP, QP, and CE [6,14] are calculated using the following
Further, several evaluation indicators, DP, QP, and CE [6,14] are calculated using the following
equations to implement quantitative evaluation:
equations to implement quantitative evaluation:
TP
DP = 100% × TP , (14)
DP  100% TP + FN , (14)
TP  FN
TP
QP = 100% × , (15)
TP + FP
TP+ FN
QP  100%  , (15)
CE = 100% ×
TP FP
FP  ,FN (16)
TP + FP
DP denotes the proportion of correctly detected FP
CE  100%  landslides,
, with respect to real landslides.
(16)
TP  FP
QP takes FN into account and measures the result synthetically. CE is error rate of detected landslides,
it is the proportion of falsely detected.
DP denotes the proportion of correctly detected landslides, with respect to real landslides. QP
The experimental results of proposed method are concluded in Table 1 based on landslide areal
takes FN into account and measures the result synthetically. CE is error rate of detected landslides, it
extent. For the case of Shenzhen, DP and CE are, respectively, 79.9% and 9.9%. Furthermore, the QP
is the proportion of falsely detected.
reaches 73.4%. For the landslides in Zhouqu County, DP and CE take the value of 70.3% and 17.6%,
The experimental results of proposed method are concluded in Table 1 based on landslide areal
and the QP is 61.1%, less than that in Shenzhen’s landslide because of the varied topography, disperse
extent. For the case of Shenzhen, DP and CE are, respectively, 79.9% and 9.9%. Furthermore, the QP
coverage and the wispy shape of the landslide in Zhouqu County. For the landslides in Beichuan
reaches 73.4%. For the landslides in Zhouqu County, DP and CE take the value of 70.3% and 17.6%,
County, DP and QP reach, respectively, 88.5% and 80.1%, while CE is 9.6%. For the number of landslide
and the QP is 61.1%, less than that in Shenzhen’s landslide because of the varied topography,
areas, the result of accuracy assessment is shown in Table 2.
disperse coverage and the wispy shape of the landslide in Zhouqu County. For the landslides in
Beichuan County, DP and QP reach, respectively, 88.5% and 80.1%, while CE is 9.6%. For the
number of landslide areas, the result of accuracy assessment is shown in Table 2.

Table 1. Accuracy statistics for landslide areal extent in three validation sites.

Validation Site Number of TPs Number of FPs Number of FNs DP (%) QP (%) CE (%)
Shenzhen 7543 832 1898 79.9 73.4 9.9
Zhouqu 2953 630 1247 70.3 61.1 17.6
Beichuan 40466 4315 5237 88.5 80.1 9.6
Sensors 2018, 18, 821 14 of 18

Table 1. Accuracy statistics for landslide areal extent in three validation sites.

Validation Site Number of TPs Number of FPs Number of FNs DP (%) QP (%) CE (%)
Shenzhen 7543 832 1898 79.9 73.4 9.9
Zhouqu 2953 630 1247 70.3 61.1 17.6
Beichuan 40466 4315 5237 88.5 80.1 9.6
Sensors 2018, 18, x FOR PEER REVIEW 14 of 18
Table 2. Accuracy statistics for landslide areas in three validation sites.
Table 2. Accuracy statistics for landslide areas in three validation sites.
Number
Number of
of Number
Number of
of Number
Number
of
of
Validation Site
Validation Site
DP (%) QP (%) CE (%)
TP Areas
TP Areas
FP Areas FN Areas
FP Areas
FN Areas DP (%) QP (%) CE (%)

Shenzhen
Shenzhen 55 2 2 0 0 100100 71.4
71.4 28.6
28.6
Zhouqu
Zhouqu 44 2 2 0 0 100100 66.7
66.7 33.3
33.3
Beichuan
Beichuan 48
48 3 3 11 11 81 81 77.4
77.4 5.9
5.9

To make the experimental results more intuitive, TPs, FNs, and FPs of Shenzhen, Zhouqu
To make the experimental results more intuitive, TPs, FNs, and FPs of Shenzhen, Zhouqu County,
County, and Beichuan County are showed in Figure 10c, 11c, and 12crespectively, they are
and Beichuan County are showed in Figure 10c, Figure 11c, and Figure 12c respectively, they are generated
generated through comparison between the detected landslides and the real ones. As shown in
through
Figure comparison
10a,b, five between
landslide the detected
regions landslides
in Shenzhen haveand theextracted
been real ones. by As shownmethod
proposed in Figurewith10a,b,
no
five one
landslide regions in Shenzhen have been extracted by proposed method with
missing, achieving a recall rate of 100%. QP in region 1 to 5 reach 76.1%, 77.6%, 65.2%, 45.7% and no one missing,
achieving
82.1% aseverally,
recall rate of 100%.
region QPhave
4 only in region
a QP of1 to 5 reach
45.7% 76.1%,
because of77.6%, 65.2%,
the cloud 45.7% For
coverage. andthe 82.1% severally,
landslide in
region 4 only have a QP of 45.7% because of the cloud coverage. For the landslide
Zhouqu County, as Figure 11a,b show, the recall rate of landslide areas reaches 100%, despite some in Zhouqu County,
as Figure
areas 11a,b
beingshow,
wispytheandrecall
tiny,rate
andofQPlandslide
of areasareas reaches
1–4 are, 100%, despite
respectively, 70.6%, some areas
60.0%, beingand
58.5%, wispy and
55.2%.
and QPthere
tiny,Besides, of areas 1–4 are,wrongly
are several respectively, 70.6%,
detected 60.0%, areas
landslides 58.5%, and 55.2%.
depicted Besides,
by isolated redthere arejust
region several
like
wrongly
area detected landslides
6–7 in Figure 10c andareas depicted
area by isolated
5–6 in Figure 11c, inredthese
region just the
areas, like earth
area 6–7 in Figure
surface 10c that
changes and area
are
similar to landslide have happened artificially or naturally. For the landslides
5–6 in Figure 11c, in these areas, the earth surface changes that are similar to landslide have happened in Beichuan County,
as Figure
artificially 12 shows,For
or naturally. thethe
method can identify
landslides in Beichuan largeCounty,
landslides well, just
as Figure like landslide
12 shows, the method 1–5 can
in
Figure
identify 12clandslides
large whose QPwell,are, respectively, 95.5%,1–5
just like landslide 94.0%, 92.9%,12c
in Figure 85.3%,
whose andQP 84.5%. But, there are95.5%,
are, respectively, still
isolated
94.0%, 92.9%, and tiny landslides,
85.3%, and 84.5%.like landslide
But, there are 6–10,
stillwhich haveand
isolated nottiny
beenlandslides,
identified because the change
like landslide 6–10,
degree of the surface is small.
which have not been identified because the change degree of the surface is small.

a c TP
4 5
FN
FP

b 2 6
3

Figure 10. Consistency

Figure analysis
10. Consistency of landslides
analysis in Shenzhen,
of landslides China. (a)China.
in Shenzhen, Real landslide
(a) Realregions; (b) Extracted
landslide regions;
Landslide regions;Landslide
(b) Extracted (c) Location of true
regions; positive of
(c) Location (TP),
truefalse negative
positive (TP), (FNs), and false
false negative positive
(FNs), (FPs)
and false
generated through
positive comparison
(FPs) generated between
through the extracted
comparison landslides
between and the
the extracted real ones.and the real ones.
landslides

a c TP
FN
2 FP

b 3
6

4
 7

Figure 10. Consistency analysis of landslides in Shenzhen, China. (a) Real landslide regions;
Sensors 2018,
(b)18, 821
Extracted Landslide regions; (c) Location of true positive (TP), false negative (FNs), and false15 of 18
positive (FPs) generated through comparison between the extracted landslides and the real ones.

a c TP
FN
2 FP

b 3
6

4
5
Sensors 2018, 18, x FOR PEER REVIEW 15 of 18

Figure 11. Consistency

Figure 11. Consistencyanalysis of of
analysis landslides
landslidesininZhouqu
ZhouquCounty,
County,China.
China. (a) Real landslide
(a) Real landslideregions;
regions;
(b) Extracted Landslide regions; (c) Location of TPs, FNs, and FPs generated through comparison
(b) Extracted Landslide regions; (c) Location of TPs, FNs, and FPs generated through comparison
between the the
between extracted landslides
extracted and
landslides andthethe
real ones.
real ones.

a c TP
10
6 7 8
FN
9 FP

5
1
b 3

Figure 12. Consistency analysis of landslides in Beichuan County, China. (a) Real landslide regions;
Figure 12. Consistency analysis of landslides in Beichuan County, China. (a) Real landslide regions;
(b) Extracted Landslide regions; (c) Location of TPs, FNs, and FPs generated through comparison
(b) Extracted Landslide regions; (c) Location of TPs, FNs, and FPs generated through comparison
between the extracted landslides and the real ones.
between the extracted landslides and the real ones.

4.3. Discussion
4.3. Discussion
The proposed method automatically detects landslides in mountain cities based on the
The proposed
knowledge method
of the earth automatically
surface change,detects
withoutlandslides in mountain
other artificial spatialcities based features,
or spectral on the knowledge
just like
of the
texture. To obtain robust and precise results in areas that cover various land surfacetexture.
earth surface change, without other artificial spatial or spectral features, just like changesTo obtain
caused
robust
by and
human precise resultsbehaviors,
or natural in areas that coverthat
regions various
containland surface land
irrelevant changescovercaused
types by human or natural
in post-event image
behaviors, regions
are filtered out that contain
in first, thenirrelevant land cover
change detection types
using DCNNin post-event image are
is implemented for filtered out in first,
merely extracting
thenregions
changewheredetection using DCNN
the change degreeisofimplemented
land surface for merely
is high, extracting
therefore regions
those where
scattered theregions
little change
where
degree the land
of land surface
surface change
is high, are similar
therefore those to landslide
scattered areregions
little removed, and the
where the land
CE for landslides
surface change areal
are
extent
similar is effectively
to landslide are reduced.
removed, and the CE for landslides areal extent is effectively reduced.
Next,
Next, TheTheSTCLSTCL based
based onon NDVI
NDVI andPANTEX
and PANTEXachieves
achievesfullyfully utilizing
utilizing the temporal
temporaland andspatial
spatial
information and the landslides-related surface change knowledge—disappearing of vegetationand
information and the landslides-related surface change knowledge—disappearing of vegetation and
built-up
built-up areas. areas. Because
Because the first
the first two twostepssteps
do notdotake
not account
take accountof theofland
the covering
land covering type changing
type changing that is
thatby
caused is landslides,
caused bythe landslides,
result imagethe ofresult
themimage of them
indicates indicates
the areas wherethe areas have
changes where changes have
intensively taken
place on large scale, the changes may be triggered by landslides or other events. With theother
intensively taken place on large scale, the changes may be triggered by landslides or help ofevents.
STCL,
With the help of STCL, regions of landslides are identified and other areas can be ruled out. Besides,
regions of landslides are identified and other areas can be ruled out. Besides, the land covering models
the land covering models learned by STCL is based on spatial context instead of single pixel, making
the changing measurement robust and stable, and can overcome the disturbance of illumination
variation and salt & pepper noise.
Finally, slope parameter which is derived from post-landslide DEM is used to evaluate the
detected areas and makes the results more reliable. Meanwhile, taking the size of landslides in
account, tiny areas are filtered out for reducing the false alarm rate, which can also be depicted by
Sensors 2018, 18, 821 16 of 18

learned by STCL is based on spatial context instead of single pixel, making the changing measurement
robust and stable, and can overcome the disturbance of illumination variation and salt & pepper noise.
Finally, slope parameter which is derived from post-landslide DEM is used to evaluate the detected
areas and makes the results more reliable. Meanwhile, taking the size of landslides in account, tiny areas
are filtered out for reducing the false alarm rate, which can also be depicted by commission error.
In addition, in view of there are part of landslide areas which are without cover type changes, the change
of DEM is used for detecting these landslide areas, making the shapes and outlines of the detected
landslides more consistent with the ground truth.
In order to compare the proposed approach with other landslides detection approaches, two recent
studies which employ the same accuracy assessment metrics have been selected. Behling et al. [17]
proposed a method based on NDVI-trajectories and achieved QP of 50–89% and CE of 10–21% for
landslide areal extent. Rau et al. [6] achieved QP of 58–81.7% and CE of 11–32%. In our experiment
in three sites, the QP for landslide areal extent are between 61.1–80.1%, the CE for landslide areal
extent is below 17.6%. For the accuracy assessment on the number of landslide areas, the QP and CE of
Behling’s method is, respectively, 34.6–44.1% and 30–70%, and our approach, whose QP and CE are
between 66.7–77.4% and 5.9–33.3%, performs better.
Overall, in terms of the detection accuracy, the proposed method reaches a lower commission
error due to filtering the unnecessary areas based on land covering type, change degree, knowledge
of land covering change, and size. The quality percentages of landslide areal extent are lower than
the maximum quality percentages of Behling’s and Rau’s studies, but higher than their minimum
quality percentages. For the assessment against the number of landslide areas, our approach performs
better with higher quality percentages. In terms of practicality, the proposed automated approach can
detect landslides in mountain cities and Behling’s method cannot identify landslides where there used
to be built-up areas. Rau’s method is semiautomatic and need landslides samples which should be
extracted by experts in target zone, if the detection is conduct in a new area, the proposed automated
approach which need not samples and saves labor cost is more appropriate.

5. Conclusions
In this paper we presented an automated approach of landslides detection using multi-temporal
remote sensing imagery, aiming at mountain cities. The patch-oriented change detection strategy
based on DCNN model was introduced for extracting area with high change degree, and the usage of
STCL makes the most of temporal and spatial information of remote sensing images for the accurate
identification of landslide areas. This multi-level extraction and filtering frame achieves removing
the unrelated areas and identifying the landslides gradually, and ensuring reliable detection percentage
while reducing the false alarm rate.
The approach was conducted on the areas of landslide in Shenzhen, Zhouqu County and Beichuan
County where both the cities and mountains existed, and the results showed that more than 70.3%
landslide areal extent was extracted, meanwhile, the CE for landslide areal extent was below 17.6%.
The experiment demonstrates that the proposed approach has a high capacity of landslide detection
in mountain cities. Besides, the detection is absolutely based on the knowledge of land covering
change related to landslide without extracting samples or other manual operation, thus the approach is
automated and saves a lot of labor resources and time cost. However, there are still landslide areas that
may not conform to the landslides-related surface change knowledge of the proposed method, like the
source area of debris flow where erosion occurred, and rivers buried by landslides. Although these areas
usually occupy small part of the landslides in mountain cities, more comprehensive models would be
studied to solve the problem and enhance the robustness and accuracy in the next step. Since the spatial
resolutions of remote sensing images in this experiment are 8 m and 30 m, in future work, the approach
will be developed for images that cover larger area and has more complex natural environment.

Acknowledgments: The paper was supported by the Space Support Technology Funds from China Aerospace
Science & Industry Corp.
Sensors 2018, 18, 821 17 of 18

Author Contributions: Zhong Chen and Yifei Zhang conceived and designed the experiments; Yifei Zhang and
Chao Ouyang performed the experiments; Zhong Chen and Feng Zhang analyzed the data; Jie Ma contributed
reagents/materials/analysis tools; Yifei Zhang wrote the paper.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Keefer, D.K.; Larsen, M.C. Assessing landslide hazards. Science 2007, 316, 1136–1138. [CrossRef] [PubMed]
2. Yin, Y.; Wang, F.; Sun, P. Landslide hazards triggered by the 2008 Wenchuan earthquake, Sichuan, China.
Landslides 2009, 6, 139–152. [CrossRef]
3. Westen, C.J.V.; Castellanos, E.; Kuriakose, S.L. Spatial data for landslide susceptibility, hazard, and
vulnerability assessment: An overview. Eng. Geol. 2008, 102, 112–131. [CrossRef]
4. KhairunnizaBejo, S.; Petrou, M.; Ganas, A. Local similarity measure for landslide detection and identification
in comparison with the image differencing method. Int. J. Remote Sens. 2010, 31, 6033–6045. [CrossRef]
5. Cheng, G.; Guo, L.; Zhao, T.; Han, J.; Li, H.; Fang, J. Automatic landslide detection from remote-sensing imagery
using a scene classification method based on BoVW and pLSA. Int. J. Remote Sens. 2013, 34, 45–59. [CrossRef]
6. Rau, J.Y.; Jhan, J.P.; Rau, R.J. Semiautomatic object-oriented landslide recognition scheme from multisensor
optical imagery and DEM. IEEE Trans. Geosci. Remote Sens. 2014, 52, 1336–1349. [CrossRef]
7. Aksoy, B.; Ercanoglu, M. Landslide identification and classification by object-based image analysis and fuzzy logic:
An example from the Azdavay region (Kastamonu, Turkey). Comput. Geosci. 2012, 38, 87–98. [CrossRef]
8. Mondini, A.C.; Marchesini, I.; Rossi, M.; Chang, K.T.; Pasquariello, G.; Guzzeti, F. Bayesian framework
for mapping and classifying shallow landslides exploiting remote sensing and topographic data.
Geomorphology 2013, 201, 135–147. [CrossRef]
9. Chang, Y.L.; Liang, L.S.; Han, C.C.; Fang, J.P.; Liang, W.Y.; Chen, K.S. Multisource data fusion for landslide
classification using generalized positive Boolean functions. IEEE Trans. Geosci. Remote Sens. 2007, 45, 1697–1708.
[CrossRef]
10. Pradhan, B.; Lee, S. Delineation of landslide hazard areas on Penang Island, Malaysia, by using frequency ratio,
logistic regression, and artificial neural network models. Environ. Earth Sci. 2010, 60, 1037–1054. [CrossRef]
11. Lee, S.; Lee, M.J. Detecting landslide location using KOMPSAT 1 and its application to landslide-susceptibility
mapping at the Gangneung area, Korea. Adv. Space Res. 2006, 38, 2261–2271. [CrossRef]
12. Hervás, J.; Barredo, J.I.; Rosin, P.L.; Pasuto, A.; Mantovani, F.; Silvano, S. Monitoring landslides from optical
remotely sensed imagery: The case history of Tessina landslide, Italy. Geomorphology 2003, 54, 63–75. [CrossRef]
13. Lacroix, P.; Zavala, B.; Berthier, E.; Audin, L. Supervised method of landslide inventory using panchromatic SPOT5
images and application to the earthquake-triggered landslides of Pisco (Peru, 2007, Mw8. 0). Remote Sens. 2013,
5, 2590–2616. [CrossRef]
14. Pascale, S.; Pastore, V.; Sdao, F.; Sole, A.; Roubis, D.; Lorenzo, P. Use of remote sensing data for landslide change
detection: Montescaglioso large landslide (Basilicata, southern Italy). Int. J. Agric. Environ. Inf. Syst. 2012, 3, 14–25.
[CrossRef]
15. Mondini, A.C.; Guzzetti, F.; Reichenbach, P.; Rossi, M.; Cardinali, M.; Ardizzone, F. Semi-automatic recognition
and mapping of rainfall induced shallow landslides using opticalsatellite images. Remote Sens. Environ. 2011,
115, 1743–1757. [CrossRef]
16. Martha, T.R.; Kerle, N.; Westen, C.J.V.; Jetten, V.; Kumar, K.V. Object-oriented analysis of multi-temporal panchromatic
images for creation of historical landslide inventories. ISPRS J. Photogramm. Remote Sens. 2012, 67, 105–119. [CrossRef]
17. Behling, R.; Roessner, S.; Kaufmann, H.; Kleinschmit, B. Automated spatiotemporal landslide mapping over
large areas using rapideye time series data. Remote Sens. 2014, 6, 8026–8055. [CrossRef]
18. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks.
Adv. Neural Inf. Process. Syst. 2012, 25, 1097–1105. [CrossRef]
19. Sa, I.; Ge, Z.; Dayoub, F.; Upcroft, B.; Perez, T.; McCool, C. DeepFruits: A Fruit Detection System Using Deep
Neural Networks. Sensors 2016, 16, 1222. [CrossRef] [PubMed]
20. Collobert, R.; Weston, J. A unified architecture for natural language processing: Deep neural networks with
multitask learning. In Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland,
5–9 July 2008; pp. 160–167.
Sensors 2018, 18, 821 18 of 18

21. Deng, L.; Hinton, G.; Kingsbury, B. New types of deep neural network learning for speech recognition and
related applications: An overview. In Proceedings of the 38th IEEE International Conference on Acoustics,
Speech, and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013; pp. 8599–8603.
22. Zhang, K.; Zhang, L.; Yang, M.H.; Zhang, D. Fast tracking via spatio-temporal context learning. In Proceedings
of the 13rd European Conference on Computer Vision, Zurich, Switzerland, 6–12 September 2014; pp. 127–141.
23. Pesaresi, M.; Gerhardinger, A.; Kayitakire, F. A robust built-up area presence index by anisotropic
rotation-invariant textural measure. IEEE J. Sel. Top. Appl. Earth Obs. 2008, 1, 180–192. [CrossRef]
24. Behling, R.; Roessner, S.; Segl, K.; Kleinschmit, B.; Kaufmann, H. Robust automated image co-registration of optical
multi-sensor time series data: Database generation for multi-temporal landslide detection. Remote Sens. 2014,
6, 2572–2600. [CrossRef]
25. Cheng, K.S.; Wei, C.; Chang, S.C. Locating landslides using multi-temporal satellite images. Adv. Space Res. 2004,
33, 296–301. [CrossRef]
26. Cruden, D.M.; Varnes, D.J. Landslide types and processes. In Landslides Investigation and Mitigation;
Transportation Research Board, Special Report 247; Turner, A.K., Schuster, R.L., Eds.; National Academy
Press: Washington, DC, USA, 1996; pp. 36–75. ISBN 030906208X.
27. Rouse, J.W.; Haas, R.H.; Schnell, J.A.; Deering, D.W. Monitoring the Vernal Advancement and Retrogradation of
Natural Vegetation; NASA/GSFC Type II Progress Report; NASA: Greenbelt, MD, USA, 1973.
28. McFeeters, S.K. The use of the normalized difference water index (NDWI) in the delineation of open water features.
Int. J. Remote Sens. 1996, 17, 1425–1432. [CrossRef]
29. Bluche, T.; Ney, H.; Kermorvant, C. Feature Extraction with Convolutional Neural Networks for Handwritten
Word Recognition. In Proceedings of the 20th international Conference on Document Analysis and Recognition,
Washington, DC, USA, 25–28 August 2013; pp. 285–289.
30. Zhao, W.; Du, S. Learning multiscale and deep representations for classifying remotely sensed imagery.
ISPRS J. Photogramm. Remote Sens. 2016, 113, 155–165. [CrossRef]
31. Basu, S.; Ganguly, S.; Mukhopadhyay, S.; Dibiano, R.; Karki, M.; Nemani, R. Deepsat—A learning framework
for satellite imagery. In Proceedings of the 23rd ACM SIGSPATIAL International Conference on Advances in
Geographic Information Systems, New York, NY, USA, 3–6 November 2015.
32. Hölbling, D.; Füreder, P.; Antolini, F.; Cigna, F.; Casagli, N.; Lang, S. A semi-automated object-based approach
for landslide detection validated by persistent scatterer interferometry measures and landslide inventories.
Remote Sens. 2012, 4, 1310–1336. [CrossRef]
33. Barlow, J.; Franklin, S. High spatial resolution satellite imagery, dem derivatives, and image segmentation
for the detection of mass wasting processes. Photogramm. Eng. Remote Sens. 2006, 72, 687–692. [CrossRef]
34. Lu, P.; Stumpf, A.; Kerle, N.; Casagli, N. Object-oriented change detection for landslide rapid mapping. IEEE Geosci.
Remote Sens. Lett. 2011, 8, 701–705. [CrossRef]

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).