Short Papers___________________________________________________________________________________________________

Encoding Visual Information

Using Anisotropic Transformations
Giuseppe Boccignone, Member, IEEE,
Mario Ferraro, and Terry Caelli
AbstractThe evolution of information in images undergoing fine-to-coarse
anisotropic transformations is analyzed by using an approach based on the theory
of irreversible transformations. In particular, we show that, when an anisotropic
diffusion model is used, local variation of entropy production over space and scale
provides the basis for a general method to extract relevant image features.
Index TermsScale space, anisotropic diffusion, entropy production, feature
encoding.

1 INTRODUCTION AND PRELIMINARIES

INa previous paper [3], we proposed a method based on the theory
of thermodynamics of irreversible transformations to compute
variations of local entropy in images undergoing fine-to-coarse
transformations. We have applied this method using an isotropic
diffusion model to produce image segmentation where region
boundaries were depicted by different rates of local entropy
variation [3]. Here, we combine the method with an anisotropic
diffusion model. It will be shown that the measure of local entropy
variation, which gauges image structure dissipation, contains a
term accounting for the bias or a priori information introduced by
anisotropy. As a result, more precise and more stable feature
encoding is obtained than by using isotropic diffusion [3].
Let be a subset of
P
, (r. y) denote a point in , and the scalar
field 1 X (r. y) t 1(r. y. t) represent a gray-level image. The
nonnegative parameter t defines the scale of resolution at which
the image is observed; small values of t correspond to fine scales,
while large values correspond to coarse scales. A scale transforma-
tion is assumed here to be a noninvertible or irreversible transforma-
tion of 1, which preserves the total intensity, i.e.,

1(r. y. t)drdy = coi:t.

Let 1
+
denote the fixed point of the transformation, that is,
1
+
(. H) = 1
+
(. t) for all t and let 1
+
be stable, that is,
lim
t
1(. t) = 1
+
(). Let ) be the normalized version of 1, that
is, )(r. y. t) = 1(r. y. t)

1(r. y. t)drdy

I
, then )(r. y) can be
interpreted as an estimate of the probability that a photon, from
the 3D scene, impinge on the image domain at the point
(r. y). In [3], the conditional entropy H() [ )
+
) was introduced
as the negative of the Kullback-Leibler distance. That is,
H() [ )
+
) =

)(r. y. t) ln
)(r. y. t)
)
+
(r. y)
drdy.
and
0H() [ )
+
)
0t
=
0H())
0t

0
0t

)(r. y. t) ln)
+
(r. y)drdy. (I)
where H()) =

)(r. y. t) ln )(r. y. t)drdy is the Boltzmann-

Gibbs entropy measure. In this case, the fine-to-coarse transforma-
tion can be modeled by a diffusion equation [4], namely
01(r. y. t),0t = \
P
1(r. y. t), and the evolution of H() [ )
+
) is
determined solely by T = 0H()),0t =

)(r. y. t)o(r. y. t)drdy,

where
o(r. y. t) =
\)(r. y. t) \)(r. y. t)
)(r. y. t)
P
(P)
is the density of entropy production in the thermodynamical sense.
Since T _ H is nonnegative, H()) is an increasing function of t, and
lim
t
o = lim
t
T = H.
2 INFORMATION FROM ANISOTROPIC
TRANSFORMATIONS
Anisotropic fine-to-coarse transformations can be modeled by
generalizing the isotropic diffusion equation as
0
0t
)(r. y. t) = di.((r. y)\)(r. y. t)). (Q)
where the function is used to force convergence of the diffusion
process toward some desired image representation. For instance,
Perona and Malik [7] assume to be a nonnegative, monotonically
decreasing function of the magnitude of local image gradient. In
this way, the diffusion mainly takes place in areas where intensity
is constant or varies slowly, whereas it does not affect areas with
large intensity transitions. As a result, small variations in ), such as
noise, can be smoothed while edges are retained. Unfortunately,
this diffusion process does not incorporate a convergence criterion.
Nordstrom [6] conjectured that, at the limit t , the image
would converge to a piecewise constant image and developed an
algorithm, called biased anisotropic diffusion, closely related to
Perona and Malik's, which converges to a steady state solution
without prespecifying the number of iterations. In discrete
simulations, the diffusion process may diverge depending on the
different schemes and grid sizes [5], [8], [10]; a method was
presented by Black et al. [1], which is well-posed and robust, in the
sense that, given a piecewise constant image, diffusion will leave
the image unchanged.
In this note, we assume Nordstrom's conjecture in order to
avoid the use of specific well-behaved diffusion, such as the one
detailed in [1], which may not apply to the many different types of
images of interest here. That is, we assume that, under a suitable
choice of and, for t , the stationary point )
+
of the
transformation corresponds to a piecewise constant function with
sharp boundaries between regions of constant intensity. In
practice, such ideal fixed points can be obtained either by using
a well-behaved diffusion such as the ones proposed in [1], [11], or
by defining )
+
() = )(. t
+
), with t
+
H (as it is actually done in the
simulations).
Consider now the general form of 0H() [ )
+
),0t (see (1)); by
taking into account that
0
0t

)(r. y.t)drdy = H, after some algebra

one obtains, by making use of (3):
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 23, NO. 2, FEBRUARY 2001 207
. G. Boccignone is with the Dipartimento di Ingegneria dell' Informazione e
Ingegneria Elettrica, Universita di Salerno, via Ponte Don Melillo, 1 84084
Fisciano, Salerno Italy. E-mail: boccig@diiie.unisa.it.
. M. Ferraro is with the Dipartimento di Fisica Sperimentale, Universita di
Torino, via Pietro Giuria, 1 10125 Torino, Italy.
E-mail: ferraro@ph.unito.it.
. T. Caelli is with the Department of Computer Science, General Services
Building, The University of Alberta, Edmonton, Alberta, Canada T6G
2H1. E-mail: tcaelli@ualberta.ca.
Manuscript received 12 July 1999; revised 25 July 2000; accepted 5 Sept.
2000.
Recommended for acceptance by Y.-F. Wang.
For information on obtaining reprints of this article, please send e-mail to:
tpami@computer.org, and reference IEEECS Log Number 110216.
0162-8828/01/$10.00 2001 IEEE
0
0t
H() [ )
+
) =

[ln )
+
(r. y) ln )(r. y. t)[di.((r. y)\)(r. y.t))drdy.
(R)
Applying the Gauss divergence theorem and making use of
Neumann's boundary conditions, we eventually obtain the
formula for the evolution of the conditional entropy H() [ )
+
) in
the anisotropic case as:
0
0t
H() [ )
+
) =

)(r. y. t)(r. y)
\)(r. y. t) \)(r. y. t)
)(r. y. t)
P
drdy

)(r. y. t)(r. y)
\)(r. y. t) \)
+
(r. y)
)(r. y. t))
+
(r. y)
drdy.
(S)
thus, during anisotropic diffusion, the evolution of the conditional
entropy H() [ )
+
) is not determined solely by the entropy
production T = 0H()),0t.
Let
T =

)(r. y. t)o
/
oi
(r. y. t)drdy
and
o =

)(r. y. t)o
//
oi
(r. y. t)drdy.
A local measure o
oi
of the variation rate of conditional entropy
H() [ )
+
) can be defined by setting
0
0t
H() [ )
+
) =
T o =

)(r. y. t)o
oi
(r. y. t)drdy.
(T)
and o
oi
can be written as the sum of two terms, denoted by o
/
oi
and
o
//
oi
, respectively,
208 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 23, NO. 2, FEBRUARY 2001
Fig. 1. Left image: The original Aerial1 image. Right image: The fixed point of Aerial1 obtained through 2,000 iterations of anisotropic diffusion. The graph below the
images plots 0H() [ )
+
),0t, T, and o as a function of scale, represented by iterations of the anisotropic diffusion process. Units are iot:,t.
o
oi
(r. y. t) = o
/
oi
(r. y. t) o
//
oi
(r. y. t)
= (r. y)
\)(r. y. t) \)(r. y. t)
)(r. y. t)
P
(r. y)
\)(r. y. t) \)
+
(r. y)
)(r. y. t))
+
(r. y)
.
(U)
It is worth noting that o
oi
, as defined in (7), is not necessarily
always positive; however, numerical calculations of 0H() [ )
+
),0t
and T, o, performed on a data set of IPH natural images provide
evidence that, if is a nonnegative decreasing function of [ \) [ ,
then H < o < T, so that T o H.
We have used different kinds of images (animals, flowers,
landscapes, objects, submarine, aerial) of various sizes and
resolutions to evaluate the model. An example is provided by
the image Aerial1 presented in Fig. 1 (left image).
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 23, NO. 2, FEBRUARY 2001 209
Fig. 2. Results of region identification by activity for image Aerial1 shown in Fig. 1. The left column displays results obtained experimenting with the isotropic process [3]:
The top image presents h-type regions, whereas the bottom image presents m-type regions. The two images in the right column are the corresponding results obtained
exploiting the anisotropic process. In both experiments, the activity has been computed by integrating over 100 iterations; the anisotropic process used the fixed point
achieved after 2,000 iterations.
The term
o
//
oi
= (r. y)
\)(r. y. t) \)
+
(r. y)
)(r. y. t))
+
(r. y)
is calculated by using an approximated fixed point image
e
)
+
(r. y) )
+
(r. y) obtained by letting the anisotropic diffusion
run for large t. As previously discussed, for simulation purposes,
large values of t provide a suitable fixed point approximation.
The right image in Fig. 1 shows the fixed point for the Aerial1
image obtained for t = PHHH iterations and using a balanced
backward sharpening diffusion, as detailed in [11]. In the same
figure, the graphs show the plots of 0H() [ )
+
),0t and T. o, as
functions of t.
By definition, it is clear that o
oi
is a function of (r. y) and
depends on the local characteristics of the image. This property
holds also for the entropy production o (see (2)), and it has been
the basis for a method of region identification [3], which exploits
the measurement of the activity of o
o
across scales, defined as
o
o
(r. y) =

H
)(r. y. t)
\)(r. y. t) \)(r. y. t)
)(r. y. t)
P
dt.
Consider a generalization of the activity concept to the case of
anisotropic diffusion. Since the anisotropic process performs
information selection (see (7)), the region identification algorithm
is, to some extent, simplified while increasing precision in
identifying different information content structures. Define the
activity of o
oi
as o
ooi
(r. y) =

H
)(r. y. t)o
oi
(r. y. t)dt. and likewise
define
o
o
/
oi
=

H
)(r. y. t)o
/
oi
(r. y. t)dt.
and
o
o
//
oi
=

H
)(r. y. t)o
//
oi
(r. y. t)dtY
then, o
ooi
= o
o
/
oi
o
o
//
oi
.
First note that, ideally, by definition of o
/
oi
, the activity o
o
/
oi
is
relatively large at points where different kinds of structures are
present: textures, edges, etc. On the other hand, o
o
//
oi
is approxi-
mately different from zero only where [ \)
+
[,= H and this happens
only along strong edges. Thus, the total activity o
ooi
is different
from zero only along weak edges. In other terms, o
ooi
and o
o
//
oi
naturally encode medium information content (m-type) and high
information content (h-type) regions, respectively. Clearly, this is
an ideal model. When dealing with real images, various sources of
noise and degradation must be taken into account; however, the
information provided by o
ooi
and o
o
/
oi
can be recovered by means
of a statistical technique, as follows.
Consider the probability distributions

1(o
ooi
),

1(o
o
//
oi
) estimated
bymeans of the histograms of o
ooi
ando
o
//
oi
, respectively. Note that, in
contrast with the isotropic case, anisotropic diffusion allows for the
definition of two probability distributions which contain contribu-
tions from high and medium activity, respectivelymaking the
separation between the activity types clearer.
Here, an iterative thresholding [3] is not needed (in the
sense that preliminary h-type / m-type identification is
incorporated in the process itself) but, rather, a one-pass
thresholding, applied to

1(o
ooi
) and to

1(o
o
//
oi
). Results using
the anisotropic process are shown, for the Aerial1 image, in
the right column of Fig. 2. They demonstrate that the use of
anisotropic activity makes localization of different features in
the image more precise; in particular, localization of edges is
improved since anisotropic diffusion avoids edge blurring and
displacement. Moreover, the identification of h-type and m-type
regions is much more biased to detect proper edge and texture
regions, respectively.
Clearly, the procedure discussed above encompasses more
general aspects than just edge detection. However, since by using
anisotropic activity differently from isotropic activity proposed in
[3], h-type regions should represent reliably edges, it may be of
some interest to compare such results with those achieved by other
methods. Fig. 3 provides an example where edge detection has
been performed on the fixed point image of Fig.1 by adopting the
same procedure used in [7]. By comparing Fig. 3 with the top right
image of Fig. 2, it can be noted a slight improvement, as regards
the saliency of edge regions encoded.
A further example of results obtained with the procedure
proposed here is presented in Fig. 4.
3 DISCUSSION AND CONCLUSION
In this note, we have considered the case of anisotropic diffusion
for the extraction of features over space and scale. It has been
shown that, in this case, the rate of change of information across
scales does not depend solely on entropy production, but that, due
to the characteristics of the process, the loss of information is, at
least, partially prevented by a term that depends on the degree of
parallelism between the gradient of the image at scale t and that of
the image representing the fixed point of the anisotropic diffusion
process. It has also been shown that different types of regions and
features, can be derived from local measures of variation of
conditional entropy and that this new model is an improvement
over the isotropic case. The reason for this is that the anisotropic
diffusion imposes constraints that limit the destruction of
structures in the image, which represent strongly oriented features.
Then the term o, which tends to prevent entropy growth, can be
interpreted as accounting for a priori knowledge or information
introduced by the anisotropic process as opposed to the isotropic
process.
210 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 23, NO. 2, FEBRUARY 2001
Fig. 3. Edge detection results obtained on the fixed point of image Aerial1 shown
in Fig. 1 using the procedure reported in [7].
The idea of applying information theory to scale-space
representations has been discussed in a recent and interesting
paper by Sporring and Weickert [9], who also present a
generalization to previous methods by studying the behavior of
Renyi's entropy throughout linear and nonlinear scale-spaces.
However, a basic limitation of such approach is that only global
measures can be defined for an image. Although this can be useful
when general characteristics (e.g., finding the fingerprint of a
texture [9]) are of interest, it cannot encode structures localized in
space and spatial scale.
An issue that remains to be investigated is the extension to
vector-valued images of the framework developed here and in [3].
In this case, the transformation from fine-to-coarse representations
can be modeled by a generalization of the diffusion equation,
namely
01i
0t
= di.

J
i
, where

J
i
is a flow and the index i labels the
bands of the image

1 = 1
I
. F F F . 1
i
, and by considering flows as
functions of the thermodynamical forces expressed by the
phenomenological laws of irreversible processes. Preliminary
results for color images have been reported in [2].
REFERENCES
[1] M.J. Black, G. Sapiro, D.H. Marimont, and D. Heeger, Robust Anisotropic
Diffusion, IEEE Trans. Image Processing, vol. 7, no. 3, pp. 421-432, 1998.
[2] G. Boccignone, M. Ferraro, and T. Caelli, Entropy Production in Colour
Images, Proc. 15th Int'l Conf. Pattern Recognition, vol. 1, pp. 202-205, 2000.
[3] M. Ferraro, G. Boccignone, and T. Caelli, On the Representation of Image
Structures via Scale Space Entropy Conditions IEEE Trans. Pattern Analysis
and Machine Intelligence, vol. 21, no. 11, pp. 1199-1203, Nov. 1999.
[4] T. Lindeberg, Scale-Space Theory in Computer Vision. Dordrecht, The
Netherlands: Kluwer Academic, 1994.
[5] M. Nitzberg and T. Shiota, Nonlinear Image Filtering with Edge and
Corner Enhancement, IEEE Trans. Pattern Analysis and Machine Intelligence,
vol. 14, pp. 826-833, 1992.
[6] N. Nordstrom, Biased Anisotropic Diffusion: A Unified Regularization
and Diffusion Approach to Edge Detection, Image Vision Computing , vol. 8,
pp. 318-327, 1990.
[7] P. Perona and J. Malik, Scale-Space and Edge Detection Using Anisotropic
Diffusion, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12,
pp. 629-639, 1990.
[8] L.I. Rudin, S. Osher, and E. Fatemi, Nonlinear Total Variation Based Noise
Removal Algorithms, Physica D, vol. 60, pp. 259-268, 1992.
[9] J. Sporring and J. Weickert, Information Measures in Scale-Spaces, IEEE
Trans. Information Theory, vol. 45, no. 3, pp. 1051-1058, 1999.
[10] R.T. Whitaker and S.M. Pizer, A Multiscale Approach to Nonuniform
Diffusion, CVIGP: Image Understanding, vol. 57, no 1, pp. 99-110, 1993.
[11] Y.L. You, X. Wenyuan, A. Tannenbaum, and M. Kaveh, Behavioral
Analysis of Anisotropic Diffusion in Image Processing, IEEE Trans. Image
Processing, vol. 5, no. 11, pp. 1539-1552, 1996.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 23, NO. 2, FEBRUARY 2001 211
Fig. 4. Results of region identification by means of anisotropic activity on the
natural image Old Lady characterized for varying details, low resolution, and
limited dynamic range. Diffusion iterations (100) have been used and a fixed point
of 500 iterations. The top image is the original Old Lady, images in the middle
and on the bottom show, respectively, h-type regions and m-type regions.