Abstract

1. Library Used
All the image processing in this project will be carried out using the OpenCV image
processing library. The language used is C++.
2. Strategy
Our basic strategy is outlined in the block diagram shown below. Most of the
processing will be done on a laptop. The laptop will process the video stream of the arena
and send commands to the robot. Feedback from the camera will be used to accurately
control the position of the bot.

The navigation algorithm takes the video feed of the arena and the output of the division
algorithm as the inputs. It uses simple thresholding operation to segment out the required
colors and finds the centroid of the thresholded images to obtain the position of the
colored circles. The robot itself will have colored markers on its top surface so that the
program can identify the position and the orientation of the robot. A simplified version of
1

the A* search algorithm is used to help the bot navigate the arena without running over
the colored circles. An outline of the algorithm used for navigation is outlined below.
Convert the denomination counts (N1, N2, N3) into a sequence of colors to
which the robot has to navigate.
Insert the black color after every two colors
Get next video frame
Threshold for red, green, yellow, black and for the markers on the bot
Calculate the positions of the red, green, yellow and black parking spots by
finding the centroids of the thresholded images
Calculate the position and orientation of the robot from the thresholded
image of the robots markers
Use a simplified version of the A* graph search algorithm to make the bot
navigate to each color while avoiding the other colors (by considering them
as obstacles.
3. Code and Screenshots
This section contains code to identify the coordinates of the green and red circles in the
image below.

3.1 Code

#include <iostream>
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>

using namespace std;
using namespace cv;

//Thresholds
int Blue_lower_thresh = 61;
int max_thresh = 255; int Blue_upper_thresh = 91;
int Green_lower_thresh = 88; int Green_upper_thresh = 204;
int Red_lower_thresh = 179; int Red_upper_thresh = 255;

//Function to get centroid of thresholded binary image
void getCentroid(Mat &thresholded_image, Point &Centroid, int &Area);

int main(int argc, char** argv)
{
//Variable declarations
Mat red, green, temp;
Mat arena = imread("BankJob.png");
char keypress;
Point centroid; int Area;

//Named Window for the trackbar
namedWindow("find_threshold");

//Creating Trackbar for finding thresholds
createTrackbar("Blue upper", "find_threshold", &Blue_upper_thresh, max_thresh, NULL);
createTrackbar("Blue lower", "find_threshold", &Blue_lower_thresh, max_thresh, NULL);
createTrackbar("Green upper", "find_threshold", &Green_upper_thresh, max_thresh,
NULL);
createTrackbar("Green lower", "find_threshold", &Green_lower_thresh, max_thresh,
NULL);
createTrackbar("Red upper", "find_threshold", &Red_upper_thresh, max_thresh, NULL);
createTrackbar("Red lower", "find_threshold", &Red_lower_thresh, max_thresh, NULL);

//Convert to grayscale for thresholding
cvtColor(arena, red, CV_RGB2GRAY);
cvtColor(arena, green, CV_RGB2GRAY);
cvtColor(arena, temp, CV_RGB2GRAY);
//Show arena
imshow("Arena", arena);

3

while(keypress !=27)
{

inRange(arena, Scalar(Blue_lower_thresh, Green_lower_thresh, Red_lower_thresh),
Scalar(Blue_upper_thresh, Green_upper_thresh, Red_upper_thresh), temp);
//Low pass filter removes noise and makes centroid calculation more accurate
medianBlur(temp, temp, 5);
imshow("find_threshold", temp);
imshow("Red Circle", red);
imshow("Green Circle", green);
keypress = waitKey(100);
switch(keypress)
{
case 'r': //Activate red thresholding
inRange(arena, Scalar(Blue_lower_thresh, Green_lower_thresh,
Red_lower_thresh),
Scalar(Blue_upper_thresh, Green_upper_thresh, Red_upper_thresh), red);
//Low pass filter removes noise and makes centroid calculation more accurate
medianBlur(red, red, 5);
getCentroid(red, centroid, Area);
cout<<"Red Circle is at X: "<<centroid.x<<" Y: "<<centroid.y<<endl;
break;
case 'g':
inRange(arena, Scalar(Blue_lower_thresh, Green_lower_thresh,
Red_lower_thresh),
Scalar(Blue_upper_thresh, Green_upper_thresh, Red_upper_thresh), green);
//Low pass filter removes noise and makes centroid calculation more accurate
medianBlur(green, green, 5);
getCentroid(green, centroid, Area);
cout<<"Green Circle is at X: "<<centroid.x<<" Y: "<<centroid.y<<endl;
break;
}
}
}

void getCentroid(Mat &thresholded_image, Point &Centroid, int &Area)
{
///The object that holds all the centroids.
///Pass in the image. The boolean true tells the function that the image is binary
Moments m = moments(thresholded_image, true);
///Moment along x axis
double M10 = m.m10;
///Moment along y-axis;
double M01 = m.m01;
///Area
4

double M00 = m.m00;

Centroid.x = int(M10/M00);
Centroid.y = int(M01/M00);
Area = int(M00);
}

3.2 Screenshot

ScreenshotofProgramrunning.TheredcircleisatX:109Y:116andthegreencircleisatX:93Y:256

On a system running Ubuntu 12.04 with opencv installed and pkg-config configured, the
code can be compiled with:

g++ abstract_code.cpp -o abstract_code -g -Wall `pkg-config opencv --cflags --libs`

In the image above, the program was run on the terminal (Numbered 5). The
final_threshold window has trackbars to control the upper and lower thresholds of red,
green and blue. Once the thresholds are found, the windows to the right numbered 2 and 3
will show the thresholded position of the red and the green circles respectively. The
window numbered 4 shows the original image from the Abstract.odt file downloaded
from the Pragyan website. Once thresholded, the program finds the centroid of the
5

thresholded image to find the position of the colored circles which are then displayed as
pixel coordinates.
6