Convolutional Neural Network based Face detection
Subham Mukherjee
Institute of Engineering & Management, Kolkata
E-mail: subhammukherjee61196@gmail.com
Sumalya Saha
Institute of Engineering & Management, Kolkata
E-mail: saha.papu.sumalya@gmail.com
Sounak Lahiri
Institute of Engineering & Management, Kolkata
E-mail: sounaklahiri01@gmail.com
Abstract—In this paper, we discuss and analyze two different
paradigm of techniques for face detection in images containing
human. We discuss the formulation for both the methods,
i.e., using hand-crafted features followed by training a simple
classifier and an entirely modern approach of learning features
from data using neural network. We discussed the theoretical
advantages of the special kind of neural network we used, i.e.,
Convolutional Neural Network. Lastly we ran both the methods
to FDDB face detection dataset and provided some qualitative
results.
Keywords—Hand-crafted feature; Neural Network; Face de-
tection; Convolutional
I. I NTRODUCTION
From the very moment when digitization of data became
possible, the need of processing digitized data increased as
well. The amount of data acquired by various devices kept
on increasing as time progresses. One of the most important
category of data is images and videos which became quite
popular because of their expressive power. But, being one of
the most complex type of data, processing of images became
a huge field of research. The application which it got most
of the attention from is ‘security’. For security application,
researchers started to investigate methods by which human
presence in images can be determined with efficiency. ‘Face
detection’ is one of the components of a rigorous security
system which is presented in this paper. In the early days
of ‘Image Processing’, manual techniques were used to detect
faces from a given image. Hand-crafted features were extracted
and classified with a so-called ‘sliding window classifier’
which SVM, DTree as classifiers. As the ‘Computer Vision’
field started to emerge, more modern statistical techniques
with ‘Advanced Machine Learning’ took over the old method-
ologies for face detection. In this paper, we presented a
qualitative analysis of both the old and modern methods for
face detection.
978-1-5386-1703-8/17/$31.00 2017
c IEEE
Ayan Das
Institute of Engineering & Management, Kolkata
E-mail: das.ayan.iem@gmail.com
Ayan Kumar Bhunia
Institute of Engineering & Management, Kolkata
E-mail: ayanbhunia007@gmail.com
Aishik Konwer
Institute of Engineering & Management, Kolkata
E-mail: konweraishik@gmail.com
Arindam Chakraborty
Institute of Engineering & Management, Kolkata
E-mail: arindam.chakraborty@iemcal.com
II. R ELATED W ORK
In classical computer vision, detection or localization was
a challenging task. In [1] geometric and colour invariance has
been implemented for detection of both regular and irregular
objects with the help of C-SIFT descriptors.Here,points of
interest are detected at the extreme of a difference of Gaussian
pyramid of the input image to construct the feature descriptors.
In [5] and [11] normalized histogram of oriented features(R-
HOG & C-HOG) along with linear SVM as a filter has been
executed to detect human beings. In [6] deep learning has been
utilised for generic visual recognition using a python network
to train the models easily.Here a open-source convolutional
model is implemented for extracting feature visualisations like
LLC and GIST. In [8] convolutional networks are used with
a different approach where non-saturated neutrons are used to
decrease training time .In order to reduce over-fitting of data
,a special regularization method is used by data augmentation
and dropout methodology. In [4] joint cascade method is
used for face detection where face alignment is combined
with detection in the same cascade network. In [10] used an
AdaBoost based algorithm which selects critical features from
large sets by combining increasingly complex classifiers in a
cascade.
The paper is organized as follows. Section I introduces
the problem and corresponding issues. Section II discusses
an important portion of the literature regarding face detection.
Section III elaborates the two paradigms in details, i.e., section
III-A depicts the old method where III-B depicts the modern
method of solving the problem. Section IV shows the used
dataset and qualitative results and lastly conclusion has been
drawn.
III. M ETHODOLOGIES
Detection of objects, humans or parts of human body
was considered to be a challenging problems in the field of
Computer Vision (CV) in its initial stage. But with the help
of modern Computer vision algorithms and Deep learning

Fig. 1. Sliding-window approach in the context of Face detection/localization
models, significant improvements in the performances have
been reported.
Classical computer vision (CV) algorithms used to follow a
relatively restricted pipeline. In classical CV, fixed features
used to be extracted from images and fed them into a
relatively simpler (mostly linear) classifier like linear-SVM,
Decision-Tree etc. The tasks like localization or detection
was carried out similarly with an additional sliding-window
concept, where an image is divided into smaller regions and
then regular pipeline is followed on every such region. Usually
these sliding-windows are square or rectangular in nature
and a stride is considered between each such windows. An
illustration of the sliding window approach is shown in figure
1.
In modern CV, a more flexible pipeline is followed for
face detection. The modern pipeline is not only used for face
detection but for any localization task. For processing a raw
image, neural networks are used. Neural networks are complex
signal processing device which can be tuned to behave as
per application. Neural networks are designed by a loose
inspiration from human brain. Neural networks, if properly
structured, can handle complex high-dimensional signals. In
this paper, we discussed one architecture of neural network
suitable for face localization tasks.
A. Using Hand-crafted features
Features are mathematical representations of objects to be
considered for the use of Machine learning algorithms (clas-
sifiers/regressors). Features are high dimensional vectors (col-
umn matrix) which represent an object as a high-dimensional
point on a complex high dimensional space. They are usually
denoted as X ∈ Rd where, R is the set of real numbers and
the vector is d-dimensional. The choice of feature is upto the
application in hand. Usually, to select proper feature, we need
domain knowledge of the application.
The entire framework of hand-crafted features is as follows:
An image I is divided into small fixed-size (usually 32x32
or 64x64) overlapping patches (Ii |N i=1 ) with fixed stride. A
previously chosen feature extractor function (say f ()) is
applied to extract feature from a single image patch Ii , i.e.,
the extracted feature fi = f (Ii ). The next step is to train a
classifier to take decision on whether a given feature fi is
a face or not. A Dataset with proper labels is to be used
for training purpose. Formally, the dataset has to be a set of
two-tuples, image patch (Ii ) and corresponding binary label
(Li ) indicating whether its a face or not. The labels are not
produced as it seems, rather a binary mask (Ib ) is created out
of an image with white color on the positions where face is
present and black elsewhere. Then the labels Li is obtained as

1, if I(position(Ibi )) contains a face
Li =
0, otherwise
There’s a practical problem with the sliding window ap-
proach which we will discuss below:
In natural images with humans and faces, the size of the
humans/faces can vary largely. The size of the object creates
a problem in possibly all major CV detection/localization
problems. A standard way of handling the problem is to use
image pyramids [3] to form pyramids which are basically
different zoom-levels of an image. We obtain k pyramids and
treat each of them separately in our described framework. This
simple method provides scale-invariance.
A crucial point of the entire framework is the choice of the
feature extractor. There are some features which have shown
performance benefits on benchmark datasets and algorithms.
Some of them are discussed below:
1) HOG [5]: : The main idea behind the Histogram of
Oriented Gradients descriptor is that local object appearance
and shape within an image can be described by the distribution
of intensity gradients or edge directions. The image is divided
into small connected regions called cells, and for the pixels
within each cell, a histogram of gradient directions is com-
piled. The descriptor is the concatenation of these histograms.
For improved accuracy, the local histograms can be contrast-
normalized by calculating a measure of the intensity across a
larger region of the image, called a block, and then using this
value to normalize all cells within the block. This normaliza-
tion results in better invariance to changes in illumination and
shadowing.
2) LBP [2]: : In the LBP descriptor for texture classifi-
cation, the occurrences of the LBP codes in an image are
collected into a histogram. The classification is then performed
by computing simple histogram similarities. However, consid-
ering a similar approach for facial image representation results
in a loss of spatial information and therefore one should codify
the texture information while retaining also their locations.

Fig. 2. General neural network
One way to achieve this goal is to use the LBP texture descrip-
tors to build several local descriptions of the face and combine
them into a global description. Such local descriptions have
been gaining interest lately which is understandable given the
limitations of the holistic representations. These local feature
based methods are more robust against variations in pose or
illumination than holistic methods.
B. Using Deep Learning
In the setup of modern computer vision, machine learning
is used as an integral part of the framework. Neural net-
work, described earlier, is the most vital component of Deep
learning based systems. The modern era of deep learning
revolves entirely around Neural Networks. Neural networks,
in a rough sense, are mathematical devices to represent data in
hierarchical organization. In deep learning terminology, these
hierarchies are called ‘Layers’. The number of such layers
is called the depth of a network, and so the name ‘Deep’
learning. Literature have proven the fact that more number of
layers increases representation power but comes at a cost of
difficulty in training phase. Researches are going on in this
field to understand this trade-off.
1) Neural Network: : Let’s assume the data to be classified
is X ∈ Rd , where R is the set of real numbers and d is
dimension. Let’s also assume the dataset comprised of data
and label( y ∈ C ) is
D  {(X(i) , y (i) )}|N
i=1
where, N is the number of total samples in dataset. The
basic principle of neural network is to transform the input
X linearly and apply proper non-linearity to produce desired
output. Mathematically, the non-linear transform of input is as
follows:
y(i) = fl3 (fl2 (fl1 (X(i) ; Wl1 ); Wl2 ); Wl3 )
where, fli is a transfer function for layer li which can be
further decomposed as the most general form as
T
fli (X(i) ; Wli ) = fnon−lin (X(i) Wli )
where, fnon−lin is a non-linear function like sigmoid(),
tanh() etc. The number of layers is a hyper-parameter which
can be chosen by guess or by cross-validation. For training
the network with our dataset D, we use the famous gradient-
descent(GD) algorithm. It is the most simple algorithm in
terms of theoretical analysis. The idea behind GD is to reduce
a predefined loss function in the direction which is steepest
w.r.t the model weights/parameters. The loss is defined as
1 
N
loss(D)  l(X(i) , y (i) )
N i=1
where,
l(X(i) , y (i) ) = ||y (i) − y(i) ||22
usually, squared-error loss is considered as per-sample loss
l(X(i) , y (i) ).
2) Conv-Net for Face detection:: Convolutional neural net-
work is a special family of neural network designed [9]
specially to handle image or 2D spatial data. The architecture
(Refer to figure 4) of convolutional network is inspired from
the human visual perception system. ConvNet is composed
mainly of 3 different kind of layers namely the most important
‘Convolutional layers’, an optional and configurable ‘Pooling
layers’ and a ‘Fully Connected layer’. The fully connected
layer can potentially be an entire multi-layer perceptron
(MLP).
But the architecture described above is not the only archi-
tecture to be used. As per application requirement, a different
structure is used for face detection or any object localization
purposes. As described in section III-A, the network composed
only of convolutional layers and optional pooling layers. The
target of the network should be to produce the output shown in
figure 3. As the target output of the network has to be a binary
mask, the last layer is proposed to be consisting of sigmoid
non-linearity because
1
Sigmoid(x) = ∈ (0, 1)
1 + exp(−x)
In theory, if the stated architecture is trained on data like figure
3, the different layers (mainly higher convolutional layers) will
be trained as face detectors.
IV. DATASET AND R ESULTS
The comparative studies depicted in this paper has
been evaluated qualitatively. Below is the details of
the dataset and some results of our experimentation.
A. Dataset
The dataset used is “Face Detection Dataset and Benchmark
(FDDB)” [7] which is comprised of 5171 faces in a set of
2845 images taken from the Faces in the Wild (FTW)
data set. FDDB images have accurate annotation by human
 Fig. 3. Input-Output of the ConvNet

Fig. 4. Convolutional Neural Network

experts which plays a great role in supervised learning.

We used the annotations to produce binary face masks (as
show in figure 3) which have been used in the training
of the specific architecture described in section III-B2.

B. Results
In the testing phase, test data from the FDDB dataset have
been used against the trained model. As the convolutional
network architecture has sigmoid layer as the terminating layer
(Refer to section III-B2), it can produce a grayscale image as
output to a specific input. But to produce a deterministic face
boundary, some post-processing has to be done. The gray-
scale output of the network is used as a density plot and fit
a gaussian distribution on it. Then the boundary ellipse has
been drawn upto one standard deviation of the gaussian. The
process is depicted in figure 6.
Fig. 5. Sample data from FDDB dataset
C ONCLUSION
In this paper, we analyzed both old and modern techniques
for face detection. We described the method of extracting
hand-crafted features and compared with a more advanced
neural network based methodology. We presented some quali-
tative results to prove the efficiency of the modern method over
the old one. In future works we may present some quantitative
descriptions and a more rigorous analysis of the work.
 R EFERENCES
[1] A. E. Abdel-Hakim and A. A. Farag. Csift: A sift
descriptor with color invariant characteristics. In 2006
IEEE Computer Society Conference on Computer Vision
and Pattern Recognition (CVPR’06), volume 2, pages
1978–1983, 2006. doi: 10.1109/CVPR.2006.95.
[2] T. Ahonen, A. Hadid, and M. Pietikainen. Face de-
scription with local binary patterns: Application to face
recognition. IEEE Transactions on Pattern Analysis and
Machine Intelligence, 28(12):2037–2041, 2006.
[3] C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden.
Pyramid methods in image processing, 1984.
[4] D. Chen, S. Ren, Y. Wei, X. Cao, and J. Sun. Joint
Cascade Face Detection and Alignment. Springer Inter-
national Publishing, 2014.
[5] N. Dalal and B. Triggs. Histograms of oriented gradients
for human detection. In 2005 IEEE Computer Society
Conference on Computer Vision and Pattern Recognition
(CVPR’05), pages 886–893 vol. 1, 2005.
[6] J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang,
E. Tzeng, and T. Darrell. Decaf: A deep convolutional
activation feature for generic visual recognition. CoRR,
2013.
[7] V. Jain and E. Learned-Miller. Fddb: A benchmark
for face detection in unconstrained settings. Technical
Report UM-CS-2010-009, University of Massachusetts,
Amherst, 2010.
[8] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet
classification with deep convolutional neural networks.
In Advances in Neural Information Processing Systems,
page 2012.
[9] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-
based learning applied to document recognition. Pro-
ceedings of the IEEE, 86(11):2278–2324, 1998.
[10] P. Viola and M. Jones. Rapid object detection using a
boosted cascade of simple features. In Proceedings of the
2001 IEEE Computer Society Conference on Computer
Vision and Pattern Recognition. CVPR 2001, volume 1,
pages I–511–I–518 vol.1, 2001.
[11] C. Vondrick, A. Khosla, T. Malisiewicz, and A. Tor-
ralba. Hoggles: Visualizing object detection features. In
The IEEE International Conference on Computer Vision
(ICCV), 2013, 2013.

Fig. 6. The output of the network is the second image when input was the
first one. The detection density is fitted by gaussian and boundary has been
draw upto one s.d. of the distribution