Automatic Texture Segmentation for Texture-based Image Retrieval
Ying Liu, Xiaofang Zhou
School of ITEE, The University of Queensland, Queensland, 4072, Australia
liuy@itee.uq.edu.au, zxf@itee.uq.edu.au
Abstract
Texture-segmentation is the crucial initial step for
texture-based image retrieval. Texture is the main
difficulty faced to a segmentation method. Many image
segmentation algorithms either can’t handle texture
properly or can’t obtain texture features directly during
segmentation which can be used for retrieval purpose.
This paper describes an automatic texture segmentation
algorithm based on a set of features derived from wavelet
domain, which are effective in texture description for
retrieval purpose. Simulation results show that the
proposed algorithm can efficiently capture the textured
regions in arbitrary images, with the features of each
region extracted as well. The features of each textured
region can be directly used to index image database with
applications as texture-based image retrieval.
1. Introduction
Region-based image retrieval (RBIR) systems retrieve
images on the basis of automatically-derived features
such as color, texture and shape from interested regions in
an image. Automatic image segmentation is a crucial
initial step before performing RBIR. A variety of
techniques have been proposed in the past, including
stochastic model based approaches [1], curve evolution
[2], energy diffusion[3], region growing[4], and graph
partitioning[5]. Quantitative evaluation methods have also
been suggested [6].
Among contents based features, texture is a
fundamental feature which provides significant
information for scene interpretation and image
classification [7], hence texture-based image retrieval is
an important topic. Textures can be defined as
‘homogeneous patterns or spatial arrangements of pixels
that regional intensity or color alone does not sufficiently
describe’ [8].
Texture is the main difficulty faced to a segmentation
method [9]. There are many existing techniques for the
segmentation of images that contains only homogeneous
color regions, such as direct clustering methods in color
space [10]. But natural scenes are rich in both color and
texture. Most texture segmentation algorithms [1][3]
requires the estimation of texture model parameters,
which is proved to be difficult [11]. To solve this
problem, a region growing method based on image color
space quantization, named JSEG, is presented in [12].
Experiments show that JSEG provides good segmentation
results on a variety of images. In [11], a method based on
local homogeneity analysis is presented. This method is
similar to JSEG but with simpler criterion used.
The above algorithms though can catch textured
regions, the segmentation is not based on texture features.
Instead, they make use of other information such as color
and spatial arrangement to handle texture. For texture-
based image retrieval purpose, we need texture features
which can be used for image retrieval. For this, one way
is to use the above mentioned segmentation algorithm,
and extract texture feature of the segmented regions after
segmentation. Alternatively, we can implement texture
segmentation in such a way that the features used for
segmentation can be directly used for retrieval purpose.
The latter is simpler as feature extraction after
segmentation is not necessary, and more efficient as
segmentation using salient texture features could handle
texture more properly.
In this paper, we proposed an automatic texture
segmentation method based on Discrete Wavelet
Transform (DWT). This method can well catch textured
regions of arbitrary images. In addition, feature of each
segmented region are obtained during segmentation and
can be used directly for the purpose of texture-based
image retrieval. The remainder of this paper is organized
as follows. Section 2 describes the related work. Section 3
introduces the proposed algorithm. Section 4 gives the
experimental results. Finally, section 5 concludes this
paper.
2. Related work
 A wide range of natural images can be considered as a
mosaic of different textured regions, each textured region
of the images in the database can be characterized and
added to a texture-based index of the database for
retrieval purpose. To efficiently handle texture, salient
texture features can be used in texture segmentation.
Gabor filters are designed to respond to different spatial
frequencies and have been applied to texture
segmentation [13]. However, it is computationally
expensive as a large combination of parameters are used.
Wavelet has been proved to be a promising alternative of
Gabor filters for texture segmentation purpose [14]. In
[15], a quad-tree decomposition based on wavelet features
is proposed. The approach is ‘top-down’ in a way that it
starts from the whole image, which is the root node of the
decomposition tree. Using a quad-tree decomposition, the
algorithm extracts texture feature from spatial blocks at a
hierarchy of scales in each image. Finally, homogeneous
blocks of texture are extracted which can be used in a
database index.
In the algorithm in [15], at each level of the tree,
texture content of the parent block is compared with the
children block, to judge if it is a texture. The problem is
that before the algorithm stops, the parent block is not a
texture, and the children blocks might not be texture
either. As we know, description of non-textured images
is more difficult than that of textured images. Hence, to
properly measure the similarity between the parent block
and the children blocks, more complicated feature
extraction algorithm is required. In addition, this
algorithm stops at low resolution textures, and can’t
obtain high-resolution textures which users might be
interested in. For example, it can catch an arrangement of
oranges which is a texture. But what if we are interested
in the skin of an orange?
To solve these problems, in this paper, we proposed a
‘bottom-up’ texture segmentation algorithm, which starts
from small homogeneous blocks and grow up gradually to
find textured regions.
‘Bottom-up’ texture segmentation can be pixel-wise
or block-wise. Pixel-wise segmentation schemes evaluate
the texture features in a neighbourhood surrounding each
pixel of the image. The advantage of pixel-wise
segmentation lies in the removal of blockyness at region
boundaries. However, the computation load is heavier. As
image retrieval system does not require exact boundary of
the segmented regions [15], block-wised segmentation is
often chosen since it is much faster [16].
The proposed block-wise texture segmentation
algorithm is based on a set of features derived from
wavelet domain. This set of features has been proved to
be efficient for texture description. The algorithm can
well capture textured regions of arbitrary images.
Moreover, the features of each region can be directly used
to index image database for retrieval purpose.
3. The proposed algorithm
In the proposed system, an image is first partitioned
into blocks of 4*4 pixels, then a 2-level wavelet transform
using 4-tap Daubechies filter is applied to each block.
This produces 7 sub-bands as shown in Figure 1.
Features in wavelet domain have been shown to be
effective in texture representation for retrieval purpose
[17]. In our experiments, mean, and variance of each of
the 7 sub-bands are extracted to form feature vector of 14
terms.
Let Sw and Sh be the width and height of a certain sub-
band in wavelet domain, mean (m) and variance (var) of
the coefficients in this sub-band can be calculated as:
( S −1) ( S −1 )
∑ ∑
h w
m =[ C ( i , j )] /( S * S ) (1)
f w h
i=0 j=0
( S h −1 ) ( S w −1 )
var = ( ∑ ∑ [C
i=0 j=0
f (i, j ) − m ] 2 (2)
where C f ( i , j ) is the transform coefficient.
With features of each block available, we can classify
all blocks into different classes. The block at the top-left
corner belongs to the 1st class, and its feature is the initial
feature of this class. We then scan all blocks one by one.
For block Bc, suppose the blocks previously scanned
belong to m different classes, if Bc belongs to any of the
m classes, we update the feature of this class (each term
of the updated feature is simply calculated as the average
of the corresponding term of all feature vectors in the
class so far); if not, add a new class as the (m+1)th class,
and feature of Bc is the initial feature of the new class.
In order to determine which class Bc belongs to, we
compute its Euclidean distance d i ( i = 1, 2 ,..., m ) to each
of the m classes obtained so far. Euclidean distance
between two l-dimensional vectors X and Y is defined as:
l −1
d = ∑i= 0
(x i − yi) 2 (3)
If the smallest distance d j falls below the given
threshold Thr, then Bc belongs to class j. If all distances
are above Thr, Bc belongs to a new class. Note that
heavier weight is assigned to var when we compute
Euclidean distance of two feature vectors. Experiments
show that a weights ratio of 9:1(for var and m) is proper.
Finally we obtain N classes each containing Mi (i=1,…,N)
blocks, and we have the features of each classes obtained.
Note that one class might contain more than one regions
which are disconnected.
 Results show that this texture segmentation algorithm
is effective in capturing the textured regions of an image.
Examples are given in Figure 2.
There are some tiny isolated regions existing after
segmentation. In most cases, these regions are
meaningless. We have not implemented post-processing
of these regions yet since these errors are often not
significant. They can be merged with larger visually
close regions if any, or ignored.
0 1
4
2 3
5 6
Figure 1. 7 sub-bands in a 2-level DWT
4. Adjusting parameters
4.1. Thr
Obviously, the higher the Thr, the smaller the number
of classes we can get. If Thr is too low, one textured
region we are interested in might be segmented into
different parts. On the other hand, if Thr is too high,
different textured regions might be classified into same
part.
Experiments show that Thr between 10 and 20 is
proper for many images.
4.2 Eng_High
With segmentation above, we found that regions with
higher variance are easy to be segmented into different
parts, as in Figure 3(a). Increasing Thr can to certain
degree relieve this problem. But if Thr is too high, some
different regions might be classified into same class, as
shown in Figure 3(b). Hence, we consider using various
Thr instead of uniform Thr for different blocks.
According to their texture property, regions of an
image can be roughly classified as ‘flat’ regions which
contain little intensity changes, ‘textured’ regions which
are composed of repetitive patterns, and ‘non-textured’
regions.
High frequency coefficients in wavelet domain
correspond to sharp intensity changes in the image, like
edges and boundaries. Sub-bands 4, 5, 6 in Figure 1 are
the three high frequency sub-bands. We define a
parameter Eng_High, which is the percentage of energy
of coefficients in sub-bands 4,5,6, relative to the total
coefficient energy across all 7 sub-bands.
To study the relationship between Eng_High and the
texture property of images, we randomly chose 100
textured images and 100 non-textured images of different
sizes, and calculated their Eng_High values. Figure 4
shows the histograms of Eng_High for the two types of
images.
It is shown that the two histograms separate around
the decision threshold 0.03. For ‘flat’ images which
contain very little high frequency information, their
Eng_High values are very small. We found that a
threshold around 0.005 is proper to separate ‘flat’ images
from textured and non-textured images. Figure 5 shows
a few images with their Eng_High values given. We
found that the above property is invariant to image sizes,
and the decision threshold varies with the DWT
decomposition level we chose.
Previous work proved that larger threshold should be
set for blocks with larger variance during the
segmentation procedure [15]. Textured regions contain
more intensity changes than non-textured regions and
hence higher Thr should be used. In our algorithm,
experimentally, different Thr are set to blocks with
different Eng_High as below:
if (Eng_High<0.005) Thr=Thr0; //flat regions
else if(Eng_High<0.03) Thr=Thr0*1.7;
//non-textured regions
else Thr=Thr0*2.5; //textured regions
20
Number of Images
16
12
8
4
0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
Eng_High
t extured non-textured
Figure 4. The histogram of Eng_High over 100
textured and 100 non-textured images
With higher threshold for blocks with more high
frequency information, textured regions which were
segmented into different parts before can be recognized as
one region. In addition, the number of classes is reduced
and hence computational load is less. Figure 3(c) shows
an example.
5. Image retrieval based on texture content
With the proposed texture segmentation algorithm, we
can capture the textured regions of arbitrary images. In
addition, texture features of the segmented regions are
obtained. These features can be used directly to index the
image database with applications as texture-based image
retrieval.
Our testing database is a collection of 250 arbitrary
images with different sizes and contents. Texture
segmentation is applied to each image in the database,
and features of each region are stored. The feature of the
query region is calculated using the same way as
described in Section 2. The images containing region(s)
similar to the query region are selected as retrieval results.
Figure 6 shows a retrieval example with the top 8 selected
images listed.
6. Conclusions
This paper presents a block-wise texture segmentation
algorithm, which automatically segment an image into
textured regions. Meanwhile, texture features of each
region are extracted which can be used as indexing of
image databases. We expect the proposed algorithm to
allow efficient image retrieval on the bases of texture
content.
In our algorithm, the feature of the segmented region is
calculated as the average of the features of all the blocks
it contains. To describe a region more efficiently,
information of the entire region could be used. We will
work on this later. In addition, we will extend the system
to large real-world image database and apply index to
facilitate fast retrieval in our future work.
Acknowledgement
This work is partly completed when the first author was
attached to the School of Computing, National University
of Singapore, under project R-252-000-015-112/303.
7. References
[1] S.Belongie, C. Carson, H. Greenspan, and J. Malik, “Color-
and texture-based image segmentation using EM and its
application to content-based image retrieval”, Proc. of ICCV,
1998, pp675-682.
[2] H.Feng, D.A. Castanon, and W.C. Karl, “A curve evolution
approach for image segmentation using adaptive flows”, Proc.
of ICCV, 2001.
[3] W.Y.Ma and B.S. majunath, “Edge flow: a framework of
boundary detection and image segmentation”, Proc. of CVPR,
1997, pp744-749.
[4] R.Adams and L.Bischof, “Seeded region growing”, IEEE
Trans. On Pattern Analysis and Machine Intelligence (PAMI),
16(6), 1994, pp641-647.
[5] J. Shi and J. Malik, “Normalized cuts and image
segmentation”, IEEE Trans. On Pattern Analysis and
Machine Intelligence (PAMI), 22(8), Aug. 2000, pp888-905.
[6] M. Borsotti, P.Campadelli, and R. Schettini, “Quantitative
evaluation of color image segmentation results”, Pattern
Recognition letters, vol.19, no.8, 1998, pp741-748.
[7] Yong Man Ro, “Matching pursuit: contents featuring and
image indexing”, SPIE, Vol. 3527, pp89-92, 1998.
[8] J. R. Smith, “Integrated Spatial Feature Image Systems:
Retrieval, Analysis and Compression”, Ph.D. thesis, Graduate
School of Arts and Sciences, Columbia University, 1997.
[9] Yining Deng, B.S. Manjunath, “Unsupervised segmentation
of color-texture regions in images and video”, Unsupervised
segmentation of color-texture regions in images and video
(PAMI), vol 23, No.8, Aug., 2001, pp800-810.
[10] D. Comaniciu and P.Meer, “Robust analysis of feature
spaces:color image segmentation”, Proc. of IEEE Conf. on
Computer Vision and Pattern Recognition,1997, pp750-755.
[11] Feng Jing, Mingling Li, Hongjiang Zhang, Bo Zhang,
“ Unsupervised image segmentation using local homogeneity
analysis”, Proc. IEEE Inter. Symposium on Circuits and
Systems, 2003.
[12] Yining Deng, B.S.Manjunath and Hyundoo Shin, “Color
Image Segmentation”, Proc. of IEEE Computer Society Conf.
on Computer Vision and Pattern Recognition, CVPR’99, Fort
Collins, CO, vol.2, June 1999, pp446-451.
[13] Dunn, D. and Higgins, W.E., “Optimal Gabor filters for
texture segmentation”, IEEE Trans. on Image Processing,
Vol4., No.7,1995, pp947-964.
[14] Mausumi Acharyya and Malay K. Kundu, “Adaptive basis
selection for multi texture segmentation by M-band wavelet
packet frames”, Inter. Conf. on Image Processing, 2001.
pp.622-625.
[15] J.R.Smith and Shih-Fu Chang, “Quad-tree segmentation for
texture-based image query”, Proc. of the 2nd Annual ACM
Multimedia Conference, San Francisco, Ca., Oct. 1994.
[16] Jia Li, James Ze Wang, and Gio Wiederhold,
“Classification of textured and non-textured images using
region segmentation”, Inter. Conf. on Image Processing,
Sep.2001, pp 754-757.
[17] J.R.Smith, S.-F. Chang, “Transform features for texture
classification and discrimination in large image databases”,
Inter. Conference on Image Processing, vol.3, 1994, pp407-
411.
 Figure 2. Examples of texture segmentation
(Left: Original images, Right: Segmentation results)

Uniform, Thr=10, N=54 Uniform Thr=20, N=9 Various Thr (Thr0=10), N=6

(a) (b) (c)

Figure 3. Segmentation results with uniform Thr and various Thr
 Eng_High=0.260 Eng_High=0.025 Eng_High=0.001
Figure 5. Textured and non-textured images with Eng_High

Query region query image

Figure 6. Texture-based image retrieval