Fine-grained Bird Classification

Group members:Rahul Kumar(B12089)

Ajay Kumar(B12039)
Mentor: Arnav Bhavsar

Problem Statement

We are given a set of images of the different birds , where each

bird image is labelled with class name which it belongs to , and
we are going to build a model which will first detect the bird
location and then , will classify into the corresponding classes.

Data Set

We are training our model on Caltech-UCSD Birds image dataset.

There are total 200 classes, with 15 features of each image

Each image have:1. part-location labels

2. Class belongingness labels.
3. Visibility attribute for that part location

Model Process
Our objective is to detect bird in a given image and classify it into
respective category. So our model process can be divide into two parts
1) Bird Detection
First we process an image into our model which detect the bird
location in given image.
2) Bird Classification
After the bird detection, we classify bird image which we got from
detection into their respective classes.

Model Process

Bird Classification

Accuracy:

For 2-Class:- 73.23%

Multiclass :- 80.4%.

Using R-CNN :- 82%

Bird Detection

Detecting objects in images is a challenging task owing to their

variable appearance and the wide range of position that they can
adopt.

So, first we need a robust feature set that allows the bird form
to be discriminated cleanly, even in cluttered backgrounds under
difficult illumination.

For any bird image given ,image can be divided into two part
One that contains bird and other as Background.

Bird detection become two class problem(Bird , Background)

Template matching Based bird Detection

technique for finding small part of an image which can

match the template image. So if take bird part and use as
template we can find bird in image.

In

our dataset, we have different color birds so , we have

to make a template which will not be affected by
different color.

So we use HOG feature which give shape information and

is not effected by different color.

For template matching, we use sum of absolute

difference.

Sum of absolute Difference (SAD)

The sum of absolute differences (SAD) is a measure of the

similarity between image blocks. It is calculated by taking the
absolute difference between each pixel in the original block and
the corresponding pixel in the block being used for comparison.

Eg.

Image block

Search image

2 5 5

0 7 2 5 5

2 7 5

2 7 2 7 5

8 6 4

8 4 8 6 4

After calculation
16 25 0

Clustering

Cluster analysis or clustering is the task of grouping a set of

objects in such a way that objects in the same group (called a
cluster) are more similar (in some sense or another) to each
other than to those in other groups.

Ex. K-means clustering

Training process(Template matching

based Bird Detection)
a) Divide the dataset into train and test.
b) Now take an image and extract the image parts which contain
bird and rescale it and using histogram of oriented gradient
(HOG) extract feature.
c) Using step (b) for all training images extract features and
concatenate them so will have a matrix which contain all
training images bird feature.
d) Now for creating a template we can use these methods1)

Average all bird feature and use average feature as

template.

2)

But every bird not has same pose so we mean as template

will not work. So do clustering and divide the bird feature
into different group and use there center feature as
template.

Detection process(Template matching

based Bird Detection)
a) Take a test image and calculate hog feature.
b) Now using SAD apply template matching between test hog image and
template (which we got from training).
c) Now from step b) we will get scores where minimum score represent best
matching.
b) Now downscaled the image and calculate hog features and do same as
step b) and step c).
c) Now from all scales, find minimum score and window size.
d) Choose that window as detection window.

Object Detection using HOG Feature and Support Vector Machine

#2. HOG Feature and Support Vector Machine

For bird detection in scene image, we can divide the image into two
parts bird part and background part.

We treat bird Detection as 2 class problem. In this we use histogram

of oriented gradient to extract the feature which give the shape,
edges information.

Training process(HOG and SVM)

a) Divide the dataset into train and test.
b) Now take an image and extract the image parts which contain bird
and background put it into bird class and background class. Do this for
all training images.
c) Now we extract hog features from all bird and background images
and create training dataset and label vector which contain their
respective class label.
d) Train a linear SVM model.

Detection Process (HOG and SVM)

a) Take test image and move a sliding window of a particular size on it
and extract size window feature and their position.
b) Now downscaled the image and move sliding window of a particular
size on it. Do this till image size is greater than sliding window size.
c) Now extract features using hog and calculate weight for each sliding
window.
d) Now apply thresholding using NMS and find detection box for bird in
image

Result
Bird Detection Accuracy

Using Template matching (template mean) 69%

Using Template matching (template cluster) 78%

Using HOG + SVM 88.5%

Future Work

We are working on the R-CNN model for classification and

detection. Classification work we have done (pre used network)
with an accuracy of 82%. And now we are working on detection
part.

Thank You