Level Set Segmentation of Hyperspectral Images Using

Joint Spectral Edge and Signature Information

Radford R. Juang+ Philippe Burlina+* Amit Banerjee+
Johns Hopkins University Applied Physics Lab+ & Dept. of Computer Science*
Laurel, MD 20723
radford.juang@jhuapl.edu

Abstract - This paper describes a new method for
segmenting hyperspectral imagery (HSI) using dynamic
curves. We are concerned about challenging HSI target
segmentation/detection use cases where the scene
includes confusers exhibiting a spectral return similar
to the desired signature and in close proximity of the
object of interest. Our method is based on a level sets
approach. It fuses all available spectral bands and
incorporates spectral as well as spatial information to
obtain a finer target segmentation. The proposed
method applies level set segmentation to HSI by defining
an expansion force field that combines both
hyperspectral gradient information as well as the
desired spectral signature. We carry out experiments on
HSI datacubes obtained from a sensor spanning visible
and near IR and show improved results when compared
to direct spectral matching in challenging close range
scenes including significant level of nearby confusers.

Keywords: Level set segmentation, spectral signature,
spectral gradient, hyperspectral imagery.

1 Introduction
Recent advances in hyperspectral sensors with high
spectral and spatial resolution have led to increased
interest in exploiting spectral imagery for target detection,
image segmentation, and object classification [1]
[4][6][7][13]. HSI provides spectral information for each
pixel in the image, resulting in a hyperspectral datacube.
This spectrum often spans frequencies beyond the visible
range including near-, mid-wave, and long-wave infrared.
One advantage of the added spectral diversity is that it
provides rich information that may be exploited to yield a
better segmentation of the scene.
For our segmentation purposes, the image is
modeled as being composed of distinct, contiguous
regions, each of which is described by constant or
homogeneous attributes. Due to blurring and
additive/multiplicative noise, an image is a corrupted
version of an underlying piecewise smooth scene [11][17].
A natural approach to delineate the regions of an image is
to statistically estimate the attributes of the regions and
use the descriptions to differentiate between them. Many
such approaches to segmentation have been studied.
These methods can be broadly classified as either
Bayesian or data clustering approaches. Given an image,
these methods extract d-dimensional feature vectors from
the data and map them into a d-dimensional feature space.
Feature vectors from pixels of similar objects or materials
will form dense local clusters in the feature space.
Segmentation is then achieved by detecting and
segregating these clusters.
Bayesian methods generally proceed by formulating
statistical model assumptions for the region generation
and image formation processes. Maximum likelihood
(ML) or maximum a posteriori (MAP) estimation is then
used for segmentation [9][16]. Techniques that model
images as Markov random fields (MRFs) have been
extensively investigated [7][8].
A recent approach for segmenting hyperspectral
images using level sets was proposed by Ball and Bruce
[1][2][3]. Given a set of training signatures, they first
reduce the dimensionality of the datacube by selecting
best bands for classification. Next, they explore the use of
spectral angle, spectral information divergence, and
maximum likelihood as measures for classifying
signatures into the corresponding classes. For each class,
they automatically determine threshold values by utilizing
the other classes as non-member data. Dimensionality
reduction may sometime compromise segmentation
results. That approach also requires that the training
signatures belong to at least two classes. Because they
have signatures from other classes, they can utilize these
signatures as non-membership information to determine
the most optimal hyperplane that divides between those
classes and the target class signature(s).
In this paper, we describe a method which
incorporates spatial gradient information with spectral
measurements to achieve more accurate segmentation
with level sets. Specifically, we are interested in
obtaining a fine segmentation of an object given
information from the target signature only. Such an
approach obviates the need for knowledge about the
background classes. Hence, the proposed technique can
be considered as a single-class, instead of multi-class,
segmentation algorithm.
A common obstacle that emerges in single-class
problems is when the target signature is closely related to
that of the background, or when nearby/intersecting
objects in the scene have similar signatures. Our approach
leverages both the target spectral signature as well as edge
1927
information in a level set formulation to overcome this
obstacle.
The paper is organized as follows. In the next
section, level set methods are explained. We describe how
level set methods can be introduced to resolve the one
class problem of hyperspectral segmentation in Section 3.
Finally, results of the approach are presented in Section 4.
2 Level Set Segmentation
The level set method is a mathematical formulation for
tracking moving fronts. Let ( ) C t represent an evolving
contour at time t , and ( , , ) F x y t represent a function
describing the force that drives the evolution of the curve
over time. One could naively pick random points along
( ) C t and track their evolution independently
using ( , , ) F x y t . However, there are a number of issues in
doing so including numerical stability problems. Osher
and Sethian devised a method to resolve this numerical
instability [15]. Instead of tracking the whereabouts of
points along ( ) C t directly, they used a function ( , , ) x y t
where ( , , ) x y t represents the signed distance of a point
( , ) x y from the curve ( ) C t . Points inside (resp. outside)
the curve would be assigned a negative (resp. positive)
distance. At any time, the curve ( ) C t is represented by:

( ) {( , ) | ( , , ) 0} C t x y x y t = = (1)

Using , one can evolve the curve using the evolution
equation:
| | 0
d
F
dt

+ = (2)

where F represents the speed function. This is called the
level set equation [15].
The level set equation can be applied to the problem
of image segmentation, as demonstrated by Malladi et al.
[14], by defining F as a speed function dependent on the
curvature of the current curve, and/or on features from the
image. For example, define F as [14]:

( , ) ( )
I A G
F x y k F F = + (3)


1
1 | ( , ) |
I
k
G I x y

=
+
(4)


2 2
2 2 3/ 2
( 2 )
( )
yy x x y xy xx y
G
x y
F


+
=
+
(5)

where ( , ) I x y is the image, and | ( , ) | G I x y

is the
magnitude of the gradient of the image after it is
convolved with a Gaussian kernel with variance ,
G
F is
the curvature, and
A
F is a constant force term that causes
the curve to evolve in regions of little or no gradient. The
term
I
k causes the curve to slow down and stop at edges,
while the curvature term
A
F causes the curve to preserve
its smoothness.
The level set method in the traditional form has its
own computational inefficiencies in maintaining
numerical accuracy. As is updated over time according
to (2), the function ( , , ) x y t slowly departs from the
actual signed distance function. Osher and Sethians
original formulation required that be reinitialized
every so often, leading to a computationally expensive
formulation. Recent work by Li et al. in [12] pushed to
resolve these issues. They reformulated the level-set
equation for segmentation to include a penalty term to
evolve so that it is close to the signed distance function.
Taking advantage of the signed distance property| | 1 = ,
Li et al. proposed the following functional as a metric to
characterize how close is to the signed distance
function [12]:

( )
2 1
( ) 1
2
P dxdy

(6)

2
\ and represents the domain of . Using this
metric, they proposed the following as the evolution
equation:


t

=

(7)

( ) ( ) ( )
m
P = + (8)

where 0 > is a weighting term that controls how
closely must follow the signed distance function, and
( )
m
is the function that drives the curves evolution to
the desired goals. They defined ( )
m
as follows:

( ) ( ) ( )
m g g
L A = + (9)

where 0 > and are constants that control the penalty
related to the length and area of the evolved curve. The
length and area of the evolved curve are controlled by:

( ) ( )
g
L g dxdy

(10)

( ) ( )
g
A gH dxdy

(11)
where ( ) is the Dirac delta function and ( ) H is the
Heaviside function. g is the force field that is derived
from the image data and characterizes how the curve will
1928
evolve. Putting all of these together, Lis formulation can
be summarized as the following:


| |
( ) ( )
| |
d
div
dt
div g g

( | |
= +
( |

\ .
| |
+
|

\ .
(12)


3 Force Field
We use the Li et al. [12] formulation to apply level set
segmentation to hyperspectral data. Their method not only
maintains a stable curve evolution without re-
initialization, it also eliminates the requirement that be
initialized as a signed distance function. These two factors
simplify and reduce the computational complexity of the
algorithm. We define g in equation (12) by fusing data
from all of the bands in the image. The design of ( ) g is
critical in the convergence to the desirable boundaries.
First, we assume that an initial rectangular region of
samples, R , will be provided containing the samples that
we want the segmentation to match to. We define g as
follows:


1 2 1
1
( , ) ( , ) ( , )
( , )
d x y d x y h x y
g x y
M
= (13)

where
1
M is the maximum value of the numerator across
all ( , ) x y . We combine information from several metrics
and normalize the result. The terms in the product above
are defined as follows:


1 ,
2
1
( , ) 1 ( , , )
mahal x y R R
d x y D S
M
= (14)

1
1 1
( ) ( )
( , , )
T
mahal
T T
u v u v
D u v
u u v v

=

(15)

1
( , ) d x y incorporates the normalized Mahalanobis
distance measure between the vector at ( , ) x y and the
mean vector of the samples in R .
, x y
S is the spectrum at pixel ( , ) x y .
R
is the mean vector of the samples in R .
R
is the covariance matrix for the samples in R .
2
M is the maximum value of
,
( , , )
mahal x y R R
D S across
all pixels.
Equation (15) is the normalized Mahalanobis distance
between vector u and v , where is the sample
covariance matrix. Next we have:


2 ,
3
1
( , ) ( , , )
wcos x y R
d x y D S v
M
= (16)

1
1 1
( , , )
T
wcos
T T
u v
D u v
u u v v

=

(17)


R R
v = (18)

2
( , ) d x y includes the whitened cosine of the angle
between the image vector at ( , ) x y and the re-centered
mean of the samples in R .
R
is a scalar containing the
average (across the bands) of the vector elements in
R
.
This makes v the zero-centered version of
R
.
3
M is the
maximum value of
,
( , , )
wcos x y R
D S v across all ( , ) x y .
1
( , ) h x y is defined by:


1
4
| ( , ) |
( , ) 1
H x y
h x y
M
| |
=
|
\ .
(19)

This is a map of the edges where the edge values are
small/close to zero. The operation H is a gradient as
defined by Stokman and Gevers [18].
4
M is the
maximum value of | ( , ) | H x y across all ( , ) x y .
To improve the results of g , we can also threshold
each of the terms in g so as to place emphasis on the
higher values. We define a function ( , ) x t as follows:

( , )
0
x t x t
x t
x t

=

<

(20)

And redefine each term as follows:

1 1
1
2
( ( , ), )
( , )
d x y t
d x y
M

(21)


2 2
2
3
( ( , ), )
( , )
d x y t
d x y
M

(22)

[ ]
1 3
( , ) 1 (| ( , ) |, ) 0 h x y H x y t = > (23)

Where
1
t ,
2
t ,
3
t are threshold parameters that can be
used to tune the algorithm,
2
M is the maximum value of
1 1
( ( , ), ) d x y t across all ( , ) x y , and
3
M is the maximum
value of
2 2
( ( , ), ) d x y t across all ( , ) x y . The greater than
operator x y > returns 1 for x y > and 0 otherwise.

1929
3.1 Automatic Threshold Selection
Given , the maximum ratio of the number of pixels in
the target to segment to the number of pixels in the image
(i.e. the maximum size ratio of the target image to
segment), we describe here how to automatically tune the
threshold value(s) for a field ( , ) F x y . We apply the
algorithm in Table 1 to
1
( , ) d x y ,
2
( , ) d x y ,
1
( , ) h x y , and
| ( , ) | H x y to determine
1
t ,
2
t , and
3
t . To express this in
written form, we replace ( , ) x t in equations (21)-(23)
with '( , ) x :

( , ) ( , ( , )) GetThresholdValue x x x = (24)


1 1
1
2
( ( , ), )
( , )
d x y
d x y
M

=

(25)

2 2
2
3
( ( , ), )
( , )
d x y
d x y
M

=

(26)
[ ]
1 3
( , ) 1 (| ( , ) |, ) 0 h x y H x y = > (27)

Our overall algorithm is listed in Table 2:

LevelSetSegment( I , R ,
1
,
2
,
3
):
1 Construct a force field g :
a. Compute metrics
1 2 1
, , d d h as
described in equations (24)-(27).
b. Compute:


1 2 1
1
( , ) ( , ) ( , ) d x y d x y h x y
g
M

=
Where
1
M is the maximum of the
numerator in g

2 Apply the level set evolution
algorithm as described in [12] using the
field g . We keep evolving until growth
of the enclosed area in the curve is
trivial. We use the rectangle R as the
initial contour and evolve our curve
outwards.

Table 2 - Overall algorithm for level set segmentation of
a one class problem
4 Results
We provide results of applying our level set algorithm
with and without gradient information and compare these
with direct spectral matching (see Figure 1 to Figure 3).
These datacubes were taken with a hyperspectral camera
that captures a scene with 22 spectral bands across the
wavelengths 400.00 nm - 1000.00 nm. Each image has a
resolution of 512x400 pixels.
As we argued earlier, one of the challenges we are
addressing in this paper lies in trying to differentiate
between the target and neighboring objects with similar
spectra. For example, the spectral signature for the shirt
and pants of the person in Figure 1 are quite similar as
seen in the plot in Figure 4.
If we were to rely solely on spectral matching, such
as the whitened cosine distance between spectra, the
segmentation result would be highly sensitive to the
selected threshold. Incorrect selection of the thresholds, as
shown in the matching and thresholding results in figures
1c and 2c, leads to poor segmentation as objects of similar
spectra act as confusers.
The advantage of using level set methods is that we
can be more permissive in the selection of the matching
threshold. Consider Figures 1c and 2c for which straight
matching and thresholding yields significant errors.
However, for similar threshold values, we can apply the
level set method described above and dynamically grow
curves over the region of interest. The curves growth is
varied according to the thresholded whitened cosine
distance to the target signature. We see in figures 1a,b and
2a,b that the results are significantly improved over the
direct matching and thresholding results.


GetTresholdValue( F , ):
1 Construct a normalized histogram of
F across all( , ) x y .
2 Construct a normalized cumulative
distribution function (CDF) by
cumulatively adding each bin of
the histogram (running sum).
3 Find the first value in the CDF
that is greater than 1 and return
the value of the bin corresponding
to the value.

Table 1 Algorithm to automatically compute the
thresholds from a field ( , ) F x y
1930

We show that in some circumstances it is beneficial to
incorporate gradient information (as described in Section
3) to differentiate the region of interest from neighboring
confusers with similar spectral signatures. The scenario in
Figure 1 includes two people crossing with shirts that are
spectrally similar. Furthermore, the shirt and pants of the
left person in Figure 1 also have similar spectra (as
illustrated in Figure 4). Without spectral edge information,
as shown in Figure 1b, the level set methods evolve the
curve into the neighboring regions of the pants and the
crossing persons shirt. If we incorporate gradient
information (as described in Section 3), it can be seen
from Figure 1a that the problem is avoided. In figure 2a,
the spectral edges have a lesser impact since bordering
objects do not share the same spectral characteristics as
the target object.


5 Conclusion
In this paper, we have shown how to incorporate the level
set method into the problem of hyperspectral datacube
segmentation where only one class of signatures (the
target class) is known. We provide an algorithm which
relies on a force field that incorporates both spectral
signature and spectral edge information and we show how
it improves the segmentation results when compared to
direct matching.



Figure 1 - (a) Left: Segmentation on HSI image using level sets with spectral edge information. (b) Middle:
Segmentation on HSI image using level set without spectral edge information. (c) Right: Segmentation on HSI image via
direct spectral matching and thresholding. All results are obtained for same thresholding value.


Figure 2- (a) Left: Segmentation on HSI image using level sets with spectral edge information. (b) Middle:
Segmentation on HSI image using level set without spectral edge information. (c) Right: Segmentation on HSI image via
direct spectral matching and thresholding. All results are obtained for same thresholding value.

1931


Figure 4 - Plot of the mean spectra signature for shirts and
pants of person in Figure 1















References
[1] J.E. Ball and L.M. Bruce, Level Set Segmentation
of Remotely Sensed Hyperspectral Images. Proceedings
of the 2005 IEEE International Geoscience and Remote
Sensing Symposium, vol. 8, pp. 5638--5642, July, 2005.
[2] J. E. Ball and L.M. Bruce, Level Set Hyperspectral
Segmentation: Near-Optimal Speed Functions using Best
Band Analysis and Scaled Spectral Angle Mapper, in
Geoscience and Remote Sensing Symposium, 2006, pp.
2596--2600, July, 2006.
[3] J.E. Ball and L.M. Bruce, Level Set Hyperspectral
Image Classification Using Best Band Analysis, in IEEE
Transactions on Geoscience and Remote Sensing, vol.45,
no.10, pp.3022--3027, October, 2007.
[4] A. Banerjee, P. Burlina, C. Diehl, Support vector
methods for anomaly detection in hyperspectral imagery,
in IEEE Transactions on Geoscience and Remote Sensing,
vol. 44, no. 8, pp. 2282--2291, August, 2006.
[5] A. Banerjee, P. Burlina, J. Broadwater, A machine
learning approach for finding hyperspectral End-
members, in Trans. IGARS, Barcelona, 2007.
[6] S.G. Beaven, D. Stein, and L.E. Hoff, Comparison
of Gaussian mixture and linear mixture models for
classification of hyperspectral data, in Proc. IGARSS
2000, Honolulu, HI, pp. 1597--1599, July, 2000.
[7] C.A. Bouman, and M. Shapiro, A Multiscale
Random Field Model for Bayesian Image Segmentation,
IEEE Trans. Image Processing, Vol. 3, pp. 162-177,
1994.
[8] S. Geman and D. Geman, Stochastic Relaxation,
Gibbs Distribution, and Bayesian Restoration of Images,
IEEE Trans. Pattern Anal. Machine Intell., Vol. 6, pp.
721-741, 1984.
[9] P.A. Kelly, H. Derin, and K.D. Hartt, Adaptive
Segmentation of Speckled SAR Images Using a
Hierarchical Random Field Model, IEEE Trans.
Acoustics, Speech, and Signal Processing, 1988.


Figure 3 Result from different iterations of the curve evolution.
1932
[10] H. Kwon and N. Nasrabadi, Kernel RX-algorithm:
A nonlinear anomaly detector for hyperspectral imagery,
in IEEE Transactions on Geoscence and Remote Sensing,
vol. 43, no. 2, pp. 388--397, February, 2005.
[11] Y. LeClerc, Constructing Simple Stable
Descriptions for Image Partitioning, International
Journal of Computer Vision, Vol. 3, pp. 73-102, 1989.
[12] C. Li et al., Level Set Evolution without Re-
Initialization: A New Variational Formulation, CVPR,
2005, vol. 1, pp. 430--436, June, 2005.
[13] X. Lu, L.E. Hoff, I.S. Reed, M. Chen, and L.B.
Stotts, Automatic target detection and recognition in
multiband imagery: A unified ML detection and
estimation approach, IEEE Trans. Image Processing, vol.
6, pp. 143--156, January, 1997.
[14] R. Malladi, J.A. Sethian, B.C. Vemuri, Shape
modeling with front propagation: a level set approach, in
IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 17, no. 2, pp. 158--175, February, 1995.
[15] S. Osher and J. Sethian, Fronts propagating with
curvature-dependent speed: Algorithms based on
Hamilton-Jacobi equations, Journal of Computational
Physics, vol. 79, pp. 12--49, 1988.
[16] E. Rignot and R. Chellappa, Segmentation of
Polarimetric Synthetic Aperture Radar Data, IEEE Trans.
Image Processing, Vol. 1, pp. 281-300, 1992.
[17] A. Rosenfeld, R. Hummel, and S. Zucker, Scene
Labeling by Relaxation Operations, IEEE Trans.
Systems, Man, and Cybernetics, Vol. 6, pp. 420-433,
1976.
[18] H. Stokman and T. Gevers, Detection and
Classification of Hyper-Spectral Edges, Proc. 10
th
British
Machine Vision Conf., pp. 643--651, 1999.
1933