A Novel Unsupervised Salient Region Segmentation for Color Images

Yu-Hsin Kuan, Shih-Ting Chen, Chung Ming Kuo, and Chaur-Heh Hsieh Department of Information Engineering, I-Shou University Tahsu, 840, Kaohsiung, Taiwan

Abstract
In this paper, we propose a novel unsupervised algorithm for the segmentation of salient regions in color images. There are two phases in this algorithm. In the first phase, we use nonparametric density estimation to extract dominant colors in an image, which are then used for the quantization of the image. The label map of the quantized image forms initial regions of segmentation. In the second phase, a region merging approach is performed. It merges the initial regions using a novel region attraction rule to form salient regions. Experimental results show that the proposed method achieves excellent segmentation performance for most of our test images. In addition, the computation is very efficient.

segment every single object in an image but to find the salient regions that are relatively meaningful to human perception. In the past few years, some methods have been proposed for finding salient regions in images [3, 4], but the computational complexity and the segmentation results are not satisfactory. The proposed method will effectively address these drawbacks. The remainder of this paper is organized as follows. Section 2 explains the basic idea of our model. Section 3 describes the experiments and demonstrates the experimental results. Section 4 concludes the paper.

2. The proposed method

Our method consists of two major portions: the first is dominant color extraction and image quantization and the second is spatial segmentation. The flow charts of the proposed method are illustrated in Figure 1.

1. Introduction
Nowadays, color images are extensively used in multimedia applications. The use of low-level visual features to retrieve relevant information from image and video databases has received much attention in recent years. For last two decades, many content-based image retrieval systems have been established [1, 2]. Usually, low-level feature descriptors (color, shape, texture, etc.) retrieve too many unrelated images and consequently their performances are unsatisfactory. Using high-level semantic descriptors such as object, scene, place, etc. should be more consistent with human perception. Nevertheless, the semantic image segmentation is still a challenging problem. A trade off solution to narrow down the gap between low-level features and human perception is to use spatial local features instead of global features of images. This means that we need the perceptually relevant regions in an image and the extracted features for matching are not from the entire image but from segmented regions. Therefore, a suitable image segmentation technique, which effectively partitions image into salient regions, is an important issue. The main purpose of this paper is not to precisely

2.1. Dominant color extraction and image quantization

Because the luminance and chrominance are mixed together in RGB color space, we adopt YUV color space to take advantage of decorrelating the luminance and chrominance. The dominant colors are extracted based on nonparametric density estimation [3, 5]. Given an n-dimensional dataset {xi Rn ; i =1N} , the nonparametric density f (x) is obtained by convolving the dataset with a unimodal density kernel K (x),

f ( x) =

1 N

N i =1

K ( x xi )

(1)

is the bandwidth for the kernel. In our work, where we selected a Gaussian kernel as (2) 2 where 2 is its variance. The density of each channel (Y, U, and V) is estimated by (1). To speed up the computation of estimation, the convolution of Gaussian
2

K (x) =

1 2

/ 2

Proceedings of the First International Conference on Innovative Computing, Information and Control (ICICIC'06) 0-7695-2616-0/06 $20.00 2006

kernel does not directly apply to the source image but to the histogram of each channel. This tremendously increases the processing speed and achieves equivalent results. After the convolution, the local maxima of each channel are obtained by using the gradient ascent scheme. The candidates of dominant colors are the combination of the local maxima of the 3 channels. The number of candidates depends on the bandwidth of the Gaussian kernel The number of the candidates will decrease when a large bandwidth is selected and vice versa. Figure 2 gives an example of the dominant color extraction. To avoid over-smoothing of the densities, a smaller bandwidth is preferred. We assign the image pixels to one of the candidates according to the color distance between them. Because it may cause too many candidates of dominant colors, we eliminate the candidates that the image pixels assignment is lower than a pre-defined threshold. Then we merge the similar colors to obtain the final dominant colors. After the dominant color extraction, each pixel of the image has already been replaced by the nearest dominant color. Consequently, a quantized color image is obtained and a label map is created as well.

If the important index of a region is less than merge threshold Tm then it should be merged into an adjacent neighbor with greatest attraction. Moreover, Tm is proportional to the important index of the second important region. 2.2.2. Attraction computation. The idea of attraction computation is analog to Newtons law of universal gravitation. For any two objects with mass m1 and m2, separated by a distance D, attract each other with a force
F =G m1m 2 D2

(4 )

For any two connected regions R1, R2 in an image, D in (4) is replaced by the color distance between R1 and R2, and m1, m2 are replaced by the region size of R1, R2 respectively and the universal gravitation constant G is set to one. Hence, the attraction can be expressed as follows:
Attraction( R1, R2 ) = RegionSizeR RegionSizeR
1 2

ColorDistance2 ( R1, R2 )

(5)

The Euclidean distance is used to compute the distance between two colors.
d (R1, R 2 ) = y
R1

2.2. Region merging strategy

The initial regions are obtained by region growing [6] according to the quantization label map. Some of them may be very small and less important. Therefore, not all the initial regions are salient. In the following, we will define the property of salient region, and then a new region merging strategy will be proposed. 2.2.1. Important index computation. Our goal is to find the salient regions in an image. Therefore, we should define saliency first. A salient region should be compact, complete and significant enough. Based on the definition, neither a small region nor a fragmentary region can be important. Thus, the important index of region is defined as follows:

2
R2

+ u

R1

2
R2

+ v

R1

2
R2

(6)

Assume a is a region to be merged and b, c, d are its neighboring regions. We should compute the attractions between a and each of b, c, d to decide where a should be merged into. Since a is common to all attraction computations the region size of a can be neglected. Thus (5) can be rewritten as
Attraction(a, k ) = RegionSizek ColorDistance2 (a, k )

(7)

where k {b, c, d}. To make it more reasonable for the computation of attraction, the region size is quantized into ten levels, from one to 10, to decrease the influence of the variation of region size. Furthermore, the ColorDistance in (7) is defined as
ColorDistance(a,k ) = 2 max(d ( a, k )) , d (a, k ) > Td d ( a, k ) , d ( a, k ) Td

Imp
Ri j
N

(R ) =
i j

N
mi

i j

R
i j

N
n i =1 m
i

i j

(3)
R
i j

(8)

N
j =1

N
j =1

: a region with color label i , region index j.

R ij

Imp(Ri j) : Important index of Ri j. : The number of pixels of Ri j. : Total number of pixels of all regions with N R ij j=1 color label i. m
mi

where T d = max (d ( a , k ) ) , and k {b, c, d}. 2 When a region is merged into another region, just simply change its color label and region index and add its region size to the target region.

3. Experiments and results

Three parameters in our algorithm need to be preset. The first is the bandwidth of the convolution kernel.

N i=1 j=1

R ij

:Total number of pixels of an image.

Proceedings of the First International Conference on Innovative Computing, Information and Control (ICICIC'06) 0-7695-2616-0/06 $20.00 2006

This is set proportional to the standard deviation of each color component histogram. The second is the allowed maximal number of dominant colors. Based on our experiments, 10 to 20 colors are suitable for most images. The third is the merge threshold Tm, which is proportional to the important index of the second important region. These parameter values are decided ahead of our experiments and identical setting is used for all images. There is no human intervention during the process. Therefore, our approach is fully unsupervised. We have implemented the proposed algorithm on a Pentium 4 PC, 2.66 GHz CPU with 512 MB RAM. The computational efficiency of the algorithm is very good. For CIF format images, the average speed is around 0.6 second for each image. Here, we present some of our test images and segmentation results to demonstrate the power and potential of our work. Figure 3 shows the experimental results. Because we are interested in salient regions in image, the region boundaries, which do not precisely match the contours of objects, are acceptable.

segmentation results satisfied our definition of saliency, and the proposed method effectively addressed the over-segmentation problem in traditional segmentation algorithms.

5. References
[1] Y. Rui, T.S. Huang, S.F. Chang (1999), Image Retrieval: Current Techniques, Promising, Directions, and Open Issues. Journal of Visual Communication and Image Representation, 10(1): 39-62. [2] B. Johansson (2000), A Survey on: Contents Based Search in Image Databases. Technical Report LiTH-ISY-R-2215. [3] E. J. Pauwels, G. Frederix. Finding salient regions in images: Nonparametric clustering for image segmentation and grouping. Computer Vision and Image Understanding, 75(1/2):73-85, 1999. [4] A. Dimai, Unsupervised Extraction of Salient Region-Descriptors for Content Based Image Retrieval. Proceedings of the 10th International Conference on Image Analysis and Processing, pages 686-691, September 1999. [5] A. Elgammal, R. Duraiswami, D. Harwood, L. S. Davis, Background and Foreground Modeling Using Nonparametric Kernel Density Estimation for Visual Surveillance. Proceedings of the IEEE, vol. 90, Issue 7, PP. 1151 1163, July 2002. [6] R. Adams, and L. Bischof, Seeded Region Growing. IEEE Transactions on Pattern Analysis and Machine Intelligence, June 1994, vol. 16, no. 6, pp. 641-647.

4. Conclusions
We presented a new salient region segmentation approach for color images based on dominant color extraction and region merging. A nonparametric density estimation was first employed to extract dominant colors and quantize images in an efficient way. A region attraction rule was then developed to merge the initial regions generated in the quantization step. The proposed approach effectively extracts salient regions in color images. Experiments show that the
Digital im ag e C olor space transform atio n (R G B to Y U V ) U histog ram K ern el den sity estim ator local m ax Finding C olor com bination (C olo r candidates) C olo r selectio n (Q u antized colors) Im ag e lab eling u sing quantized colo rs Q uantized im ag e

Quantized image

Region growing

Y histogram K ernel den sity estim ator local m ax Find ing

V h istog ram Kernel d ensity estim ato r

Important index computation

Important index < Threshold

no

local m ax Fin din g

yes Compute attraction between current region and its adjacent regions

Segmentation result

Merge current region into the region with maximum attraction

(a)

(b)

Figure 1. Proposed segmentation method (a) Dominant color extraction and image quantization (b) Spatial segmentation.

Proceedings of the First International Conference on Innovative Computing, Information and Control (ICICIC'06) 0-7695-2616-0/06 $20.00 2006

(a)

(b)
2 local maxima 2 local maxima Candidates = 2 x 2 x 1 = 4 colors 1 local maximum

Figure 2. (a) Original densities (b) Nonparametric densities

(a)

(b)

(a)

(b)

(a)

(b)

(a)

(b)

(a)

(b)

(a)

(b)

(a)

(b)

(a)

(b)

Figure 3. (a) Source images (b) Segmentation results

Proceedings of the First International Conference on Innovative Computing, Information and Control (ICICIC'06) 0-7695-2616-0/06 $20.00 2006