2017 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW)
Automatic Recognition of Clothes Pattern and Motifs
Empowering Online Fashion Shopping
K.S. Loke, Swinburne University of Technology Sarawak Campus, Member, IEEE
AbstractAutomatic recognition of clothes pattern and styles
have broad application in consumer online fashion shopping. In
this research, we propose a fast and accurate method of
recognizing a clothes textile design and pattern. We used a
modified 6-channel co-occurrence matrix with a random forest
classifier. We tested the accuracy of recognizing clothing of
fashion models and obtained results of 93%.
I. INTRODUCTION
In recent years, there have been renewed interests in clothes
recognition, classification and retrieval. Recent research has
begun to address specifically clothes-based recognition and
detection, in parallel with a surge in startup investments on
fashion-based apps. We are interested in the consumer
application of clothes recognition in particular related to
content-based retrieval of similar clothes based on textile
patterns and motifs. The general idea is to facilitate the
consumer to search and purchase similar textile patterns. For
example, a consumer may view a dress with a particular
pattern on the television or on the street; he or she may capture
it by camera and use it to search for a similar dress. This is not
a new idea but better precision is needed than general content-
based retrieval. Our work currently does not address dress
style yet.
Our work is broadly related to two broad categories of
works. One is in texture recognition and the other in clothes
recognition. Our work is a combination of both but not strictly
classified as texture recognition though it is quite related. In
texture recognition, the research is mostly interested in
identifying natural material surfaces like marble, wood grains,
tree barks and so on. Natural textures tend to be uniform and
finer in structure. The other works involve classification of
clothes and their styles. A sub-category of that is classification
by the textile pattern and motifs such as floral, plaids,
geometric and so on. Many of these works attempt to classify
texture and motifs into a broad themed category such as floral
and so on. However, such broad classifications have
indeterminate boundaries and can be subjective. In our view,
linguistic classification is not very useful. For example
retrieving all floral patterns is not a useful task we are looking
for a dress with similar patterns. Therefore, we are interested
in accurate recognition of clothes patterns and motifs.
II. RELATED WORKS
We briefly review some of the recent works. Bossard et al
[1] used a random forest [2] for classifying type of clothes and
support vector machines for the style of the apparel. They used
multiple features including Scale-Invariant Feature Transform
SIFT [3] to create a spatial pyramid of code books. Fu et al [4]
uses the human pose to extract bag of words features
978-1-5090-4017-9/17/$31.00 2017 IEEE 375
embedded in a vocabulary tree. Yamaguchi [5] also used the
human pose to extract clothing segments and used a variety
features including color histogram and Gabor features. A
couple of recent works also used convolutional neural
networks to match clothes style and patterns [6] [7] [8] but
none involved direct textile patterns.
There are some recent works that addressed texture
recognition using features from filter banks [9] and Fisher
Vectors [10]. These local feature descriptors have been
coupled with classifiers like random forest, support vector
machines and convolutional neural networks (CNN) [11].
However, textures are not like clothes patterns.
III. APPROACH
We selected a range of images from the fashion website
Forever21.com by using these search terms: floral, patterns,
plaids, paisley, geometric and tribal. From these images, we
collected 100 different classes of patterns with 10-11 samples
for each class, each 100 by 100 pixels, sampled from pictures
of models. These image patches represent the clothes patterns.
The patches are extracted from images of fashion models
(Fig.1). From the above, we assembled three datasets. The first
dataset, D1, consist of 100 classes with 1080 training samples,
each image sampled from the front, side and back views of
models. Training and test are 100x100 patches.
The second dataset D2 consists of training image patches
from D1 but uses the fashion models front, side and back
view as test. The size of the dataset is 98 classes.
The third dataset, D3, consists of training and test image
taken from mutually exclusive model images (Fig. 2). The
total number of classes is reduced to 84 due to lack of suitable
patches in some images.
Our approach centers on calculating the second order
statistics of the image grey levels. Grey Level Co-occurrence
Matrix (GLCM) [12] counts the grey level occurrence
between all pairs of nearby pixels. If we consider the pixels
red, green and blue channels, there would be six-paired
GLCM such as red-red, blue-blue, green-green, red-blue, red-
green and blue-green. The matrix Mab for channels a-b, can be
mathematically formulated as in (1):
(, ) = [(, ) ] [( + , (1)
, ,
+ ) ]
where []is the Kronecker delta, I(x,y) is the pixel intensity
at (x,y) and (tx, ty) is the translation of the paired pixel from
(x,y). U is the set of pixels that is one pixel from (x,y) in all
directions.
Cheong et al [13] used orthogonal polynomial (O.P.) to
encode the GLCM matrix. The calculation of O.P. is slow,
therefore we use the raw numbers in the matrix reduced to
32x32 by averaging the neighborhood (Fig.1). This method
reduces overfitting and is fast. Cheong et al tested only 30
classes using 150 samples based on Batik patches; here we
tested on 87-100 classes using fashion model images.
We implemented the training and tests in Java using Weka
Java libraries. For the convolutional neural network test, we
used the Keras Python libraries.
conjunction with object detection such as convolutional neural
networks to obtain full classification of the clothes. However,
there is still an unresolved issue on how to deal with different
scale. This is because the scale will affect the statistical
properties of the pixel distribution. This is the topic of our
future research.

Fig.2.D3 dataset example - training: back view (left) and testing: front
view (right). Testing is done by calculating mean score for each patches,
resulting in accuracy of 93.4% overall for 87 classes.

Fig.1. (Left) Image patch with the 6 GLCM . (Right) Source of the REFERENCE
patches. The GLCM are used as input features.

[1] L. M. Bossard, Dantone, C. Leistner, C. Wengert, T. Quack and L. V.

IV. RESULTS & DISCUSSION Gool, "Apparel classification with style," in ACCV, 2012.
The results are presented in Table I. [2] L. Breiman, "Random forests," Machine Learning, vol. 45, pp. 5-32,
2001.
TABLE I [3] D. G. Lowe, "Distinctive image features from scale-invariant keypoints,"
Test-Type Test Data Algorithm Accuracy (%) International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110,
10-fold D1: 100 Random Forest, 96.9 2004.
classes,11 100 trees, 13 [4] J. Fu, J. Wang, Z. Li, M. Xu. and H. Lu, "Efficient clothing retrieval with
samples each features semantic-preserving visual phrases," in ACCV, 2012.
10-fold D1: as above Multi-layer neural 97.0
[5] K. Yamaguchi, M. H. Kiapour, L. E. Ortiz and T. L. Berg, "Parsing
network, 3 hidden
clothing in fashion photographs.," 2012.
layers
Single run D1: 10 training, Convolutional 28.0 [6] M. H. Kiapour, X. Han, S. Lazebnik, A. C. Berg and T. L. Berg, "Where
1 as test NN, 2 layers of 32 to buy it: Matching street clothing photos in online shops," in ICCV,
conv filters. 2015.
Single run D2: 98 classes, Random Forest, 97.0 [7] J. Huang, R. S. Feris, Q. Chen and S. Yan, "Cross-domain image
training & test 100 trees, 13 retrieval with a dual attribute-aware ranking network.," in ICCV, 2015.
data overlap; 1- features
[8] Z. Liu, P. Luo, S. Qiu, X. Wang and X. Tang, "DeepFashion: Powering
3 test images
Robust Clothes Recognition and Retrieval with Rich Annotations," in
Single run D3: 84 classes, Random Forest, 93.4
CVPR, 2016.
training data 100 trees, 13
different from features [9] V. Andrearczyk and P. F. Whelan, "Using Filter Banks in Convolutional
test. Neural Networks for Texture Classification," Pattern Recognition
Letters, vol. 84, pp. 63-69, 2016.
The results obtained are quite respectable, and the random [10] M. Cimpoi, S. Maji, J. Kokkinos and A. Vedaldi, "Deep Filter Banks for
forest training and test is very fast. The CNN we used is Texture Recognition, Description, and Segmentation," International
Journal of Computer Vision, vol. 118, no. 1, pp. 65-94, 2016.
essentially the same used in by LeCun et al [14]. CNN are
[11] F. H. C. Tivive and A. Bouzerdoum, "Texture classification using
useful for extracting invariant image features but in the case of convolutional neural networks," in 2006 IEEE Region 10 Conference
textile patterns the statistical relationship between pixels are (TENCON 2006), Hong Kong.
more important. Experiments with D2 and D3 used fashion [12] M. Haralick, K. Shanmugam and I. Dinstein, "Textural features forimage
model images for testing (Fig. 2) but patches for training. The classification," IEEE Trans. Syst., Man, Cybernetics, vol. 3, no. 6, pp.
610-621, 1973.
winning classification is determined by the largest average
[13] M. Cheong and K.-S. Loke, "Textile Recognition Using Tchebichef
score after subdividing the image into 100x100 patches. Moments of Co-occurrence Matrices," Lecture Notes in Computer
Science, vol. 5226, pp. 1017-1024, 2008.
V. CONCLUSION [14] Y. LeCun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based
learning applied to document recognition," in Proc. IEEE, 1998.
We demonstrated a fast and accurate method to distinguish
patterns on clothes easily. This method can be used in

376