Urban Remote Sensing _ Final Paper _ NKOMEJE FELICIEN

HOHAI UNIVERSITY
School of Earth Sciences and Engineering

Course: Urban Remote Sensing

Support Vector Machine (SVM) and Artificial Neural Network (ANN) theory and
applications

Done by:
NKOMEJE Felicien () majoring in Cartography and
GIS

Registration number: M2014047

Lecturer: SU Hongjun

Nanjing, 08 July 2015

Urban Remote Sensing _ Final Paper _ NKOMEJE FELICIEN

Support Vector Machine (SVM) and Artificial Neural Network (ANN) theory and applications
a) Support Vector Machine (SVM)
What? SVM is a supervised non parametrical statistical learning algorithm which can be used for
classification or regression problems. It uses a technique called the kernel trick or method (The kernel
trick takes the data you give it and transforms it. In goes some great features which you think are going to
make a great classifier and outcome some data that you don't recognize anymore.) to transform your data
and then based on these transformations it finds an optimal boundary between the possible outputs.
Simply, it does some extremely complex data transformations, and then figures out how to separate your
data based on the labels or outputs you've defined [Lipo W., 2005; Roland N. et al, 2006; Jason W., no
date].
Why? The solid mathematical in statistical learning theory and its capability in numerous real world
applications like image processing make SVM greatly as one of the art tools for machine learning and
data mining, along with other soft computing techniques, e.g., neural networks and fuzzy systems. It can
perfectly do both classification and regression [Giorgos M., 2010]. Focusing on SVM use for
classification and particularly non-linear kernel SVM (means the boundary that the algorithm calculates
doesn't have to be a straight line), the benefit is that you can capture much more complex relationships
between your data points without having to perform difficult transformations on your own [Jason w., no
date].
Attention: Unfortunately the magic of SVM is also the biggest drawback. The complex data
transformations and resulting boundary plane are very difficult to interpret. This is why it's often called a
black box [Roland N. et al, 2006]. Support Vector Machines (SVMs) have proven to be highly effective
for learning many real world datasets but have failed to establish themselves as common machine
learning tools[Colin S. et al no date].
Examples:
1. SVM, as the nonlinear transfer function in inverse problem can be used in ocean color remote sensing.
Here I must use a field data set to evaluate the performance of the proposed approach. Many studies
showed that SVM performs as well as the optimal multi-layer perception (MLP) and can be a promising
alternative to the conventional MLPs for the retrieval of oceanic chlorophyll concentration from marine
reflectance.
2. The SVM classification can be used with hyper-spectral images for urban land use/cover mapping.
Many researchers found that hyper-spectral data in combination with the SVM method can be quite useful

Urban Remote Sensing _ Final Paper _ NKOMEJE FELICIEN

for improving the mapping of spectrally complex classes in the urban remote sensing analysis [Dee Shi et
al., Sept 2012]. The SVM has shown the robustness of geospatial technique for mapping heterogeneous
landscapes in urban remote sensing. The big idea behind SVM is to construct separating hyper-planes
between classes in feature space which are lying at the edges of class domains. SVM seek the optimal
hyper-plane that can separate classes from each other with the maximum margin (Vapnik, 1995).

Example of a separable problem in a 2 dimensional spaces

In this example, the support Vectors marked with Grey Square, define the margin of largest separation
between the 2 classes.
b) Artificial Neural Network (ANN)
Definition: ANN is simply a network of many very simple processors (based on input, Hidden layer
(neurons or networks) and finally output) and each possibly having a local memory.
How it works? The units are connected by unidirectional communication channels, which carry numeric
data and they operate only on their local data and on the inputs they receive via the connections. The
ANN design motivation makes it differ from other mathematical learning techniques and its functioning
of human brains and components.

Urban Remote Sensing _ Final Paper _ NKOMEJE FELICIEN

ANN structure
The process is implemented step by step. Lets take a simple example of perceptron which is an
algorithm for supervised learning of binary classifiers: functions that can decide whether an input
(represented by a vector of numbers) belongs to one class or another.
Perceptron structure (It consists of one or more inputs, a processor and a single output)

Step1 is receiving inputs: Lets say we have two inputs x1 and x2.
Input 0: x1 = 12
Input 1: x2 = 4
Step2 is weighting inputs: Each input that is sent into the network (neuron) must first be weighted (often
a number between -1 and 1).
Weight 0: 0.5
Weight 1: -1
We take each input and multiply it by its weight.
Input 0 * Weight 0 12 * 0.5 = 6
Input 1 * Weight 1 4 * -1 = -4
Step3 is Sum inputs: The weighted inputs are then summed. Sum = 6 + -4 = 2
Step4 is Generating output: The output of a perceptron is computed by passing that sum through an
activation function. In the case of a simple binary output, the activation function is what tells the
perceptron whether to fire or not. Finally we can make the activation function the sign of the sum which
means that if the sum is a positive number, the output is 1; if it is negative, the output is -1.
Output = sign (sum) sign (2) +1

Consider a line in two-dimensional space and points in that space can be classified as living on
either one side of the line or the other. We can determine on which side a point lies with some
simple algebra), it shows how a perceptron can be trained to recognize points on one side versus
another.

Urban Remote Sensing _ Final Paper _ NKOMEJE FELICIEN

A simple pattern recognition using spectron

ANN rationale: The foundation of neural network approaches doesnt rely on parametrical classification
or knowledge of the statistical data distribution.
Application examples
Watershed land cover mapping: 1. Watersheds are ecologically rationale because of their components
importance and its relationship with water quality, estuarine productivity, and wildlife system. This study
can be conducted using remotely sensed data. In a local scale watershed application, high spatial
resolution is often necessary for feature extraction with an acceptable degree of accuracy. Neural
network-based classifiers have been found to be robust and well suited for a wide variety of remotely
sensed data.
2. ANN in image processing and CAD based: ANN can be used in automatic road extraction and
vectorization from high resolution satellite images like AKONOS. This technique combines image
processing algorithms and exploiting CAD-based facilities. Firstly, artificial neural networks are used to
discriminate between road and non-road pixels. Then road centerlines are extracted using image
processing algorithms such as morphological operators, and a road raster map is produced.
After normalizing the RGB values between 0 and 1, the spectral information for each pixel is simply
entered to the network as its input parameters. After this, neurons are designed in input layer in charge of
receiving spectral values for each pixel in entire AKNOS image.

a) RGB IKONOS Image from Kish Island in Iran b) Road extracted results

Urban Remote Sensing _ Final Paper _ NKOMEJE FELICIEN

Network structure when neighbor pixels with their normalized distances form input parameters

ANN was designed and implemented to extract road centerlines from high resolution satellite images.
Neural networks and digital image processing algorithms such as morphological operators were used to
extract road centerline from satellite images and present it as an edited road raster map. Then obtained
raster map was prepared and vectorized using the facilities provided by CAD-based environments. The
final result is the structured vector based road network presented in CAD environment where it can be
easily transformed to GIS. The comparison between obtained results and road network reference map
proved acceptable geometric accuracy of designed procedure
Conclusion
SVMs have developed a new approach to the problem of pattern recognition with clear connections to the
underlying statistical learning theory and they differ wholly from other comparable approaches used in
classification. SVM training always finds a global minimum, and their simple geometric interpretation
gives the fertile position for further investigation. An SVM is mainly based on choice of its kernel.
The future of Neural Networks is wide open, and may lead to many answers and questions in different
domain (Sciences, medicine, Engineering...). The capability of networks is related to their structure,
dynamics and learning methods. According to what ANN does, we can ask ourselves if it is possible to
create a conscious machine (Computer) and what rights do these computers have? Also we can this about
how the human mind work and what does it mean to be human?

References
A.K. Skidimore et al, Performance of a Neural Network: Mapping Forests Using GIS and Remotely
Sensed Data may 1997.
Colin Campbell et al, Simple Learning Algorithms for Training Support Vector Machines, no date.
Dick de Ridder, Nonlinear image processing using artificial neural Networks, No date.
F. Qiu et al, Opening the black box of neural networks for remote sensing image Classification, May
2004.
H. Zhang, Application of Support Vector Machines in Inverse Problems in Ocean Color Remote Sensing,
22 April 2005.

Urban Remote Sensing _ Final Paper _ NKOMEJE FELICIEN

Kim, K. In et al, Fast color texture-based object detection in images: application to license plate
localization, July 2014 and modified 05 Jan 2016.
Mathieu FAUVEL, Spectral and Spatial Methods for the Classification of Urban Remote Sensing Data,
November 2007.
A.K. Skidimore et al, Performance of a Neural Network: Mapping Forests Using GIS and Remotely
Sensed Data, may 1997.
P. Timon McPhearson et al, Remote Sensing Applications to Biodiversity Conservation, 2004.
Peijun Duet al,Remote Sensing Image Interpretation for Urban Environment Analysis: Methods, System
and Examples, 2014.
Xiaoli Tao et al, Novel Artificial Neural Networks For Remote-Sensing Data Classification no date.
Centralized website: www.kernel-machines.org