11 09 OpenCV Tutorial

OpenCV, MATLAB or AForge ??
Image processing is the process of manipulating image data in

order to make it suitable for computer vision applications or to
make it suitable to present it to humans. For example, changing
brightness or contrast is a image processing task which make
the image visually pleasing for humans or suitable for further
processing for a certain computer vision application.
Computer vision which go beyond image processing, helps to
obtain relevant information from images and make decisions
based on that information. In other words, computer vision is
making the computer see as humans do. Basic steps for a
typical computer vision application as follows.
1. Image acquisition
2. Image manipulation
3. Obtaining relevant information
4. Decision making
If you are new to computer vision, you may be wondering where to
start. First you have to understand the basic principles of image
processing and computer vision. Then you have to choose a suitable
language to develop your computer vision application. Some of the
most popular methods are using OpenCV with
C/C++, MATLAB and AForge. If you don't really know why you
would choose one over the other, here is my explanation.
MATLAB is the most easiest and the inefficient way to process
images and OpenCV is the most efficient and hardest way to process
images. AForge has qualities in between OpenCV and MATLAB.
OpenCV has become hardest only because there is no proper
documentation and error handling codes. But OpenCV has lots
of basic inbuilt image processing functions so that those who want to
learn computer vision can develop their applications through proper
understanding about what they do.
So, I think that it is worthy to learn computer vision with OpenCV.

Therefore in this blog, it is presented basic image processing
functions and computer vision applications with line by
line explanations.
What is OpenCV?
OpenCV is an open source C++ library for image processing and

computer vision, originally developed by Intel and now supported by
Willow Garage.
It is free for both commercial and non-commercial use.
Therefore it is not mandatory for your OpenCV applications to
be open or free.
It is a library of many inbuilt functions mainly aimed at real time
image processing. Now it has several hundreds of image processing
and computer vision algorithms which make developing advanced
computer vision applications easy and efficient.
If you are having any troubles with installing OpenCV or configure
your Visual Studio IDE for OpenCV, please refer to Installing and
Configuring with Visual Studio.
Key Features
 Optimized for real time image processing & computer
vision applications
 Primary interface of OpenCV is in C++
 There are also C, Python and JAVA full interfaces
 OpenCV applications run on Windows, Android, Linux,
Mac and iOS
 Optimized for Intel processors
OpenCV Modules
OpenCV has a modular structure. The main modules of OpenCV are

listed below. I have provided some links which are pointing to some
example lessons under each module.
 core
This is the basic module of OpenCV. It includes basic data
structures (e.g.- Mat data structure) and basic image
processing functions. This module is also extensively used by
other modules like highgui, etc.
 highgui
This module provides simple user interface capabilities, several image
and video codecs, image and video capturing
capabilities, manipulating image windows, handling track bars and
mouse events and etc. If you want more advanced UI capabilities,
you have to use UI frameworks like Qt, WinForms, etc.
e.g. - Load & Display Image, Capture Video from File or
Camera, Write Image & Video to File
 imgproc
This module includes basic image processing algorithms
including image filtering, image transformations, color space
conversions and etc.
 video
This is a video analysis module which includes object tracking
algorithms, background subtraction algorithms and etc.
 objdetect
This includes object detection and recognition algorithms for
standard objects.
OpenCV is now extensively used for developing advanced image

processing and computer vision applications. It has been a tool for
students, engineers and researchers in every nook and corner of the
world.
Basics of OpenCV API
Header files
These are some of important OpenCV header files for C++ interface.
As a beginner, you will need few of these header files for your
application. In my following lessons, I will include only necessary
header files to my example programs. If you are not sure what to
include, include them all. No any penalty incurred for including all
this header file except for a fact that the length of your source code
will increased by few lines than necessary.
o #include "opencv2/core/core.hpp"
o #include "opencv2/flann/miniflann.hpp"
o #include "opencv2/imgproc/imgproc.hpp"
o #include "opencv2/photo/photo.hpp"
o #include "opencv2/video/video.hpp"
o #include "opencv2/features2d/features2d.hpp"
o #include "opencv2/objdetect/objdetect.hpp"
o #include "opencv2/calib3d/calib3d.hpp"
o #include "opencv2/ml/ml.hpp"
o #include "opencv2/highgui/highgui.hpp"
o #include "opencv2/contrib/contrib.hpp"
o #include "opencv2/core/core_c.h"
o #include "opencv2/highgui/highgui_c.h"
o #include "opencv2/imgproc/imgproc_c.h"
Namespace
All OpenCV classes and functions are in cv namespace. So, you have
to do one of following
 Add the 'using namespace cv' line just after including your
header files (I have used this method in all my sample programs)
e.g.
#include "opencv2/core/core.hpp"
using namespace cv;
int main()
{
Mat frame = cvQueryFrame(
capture );
imshow( "Video", frame );
}
 append the cv:: specifier at the beginning of every OpenCV

classes, functions and data structures in your source code
e.g.
#include "opencv2/core/core.hpp"
int main()
{
cv::Mat frame = cvQueryFrame(
capture );
cv::imshow( "Video", frame );
}
Data Types for Arrays
Data type of an array defines the number of bits allocated for each
element of array (pixels in an image) and how the value is represented
using those bits. Any array elements should have one of following
data types.
For single channel arrays :

 CV_8U (8 bit unsigned integer)
 CV_8S (8 bit signed integer)
 CV_16U (16 bit unsigned integer)
 CV_32F (32 bit floating point number)
 CV_64F (64 bit float floating point number)
e.g. : Here I have illustrated a single channel array with 8 bit
unsigned integers. As the datatype of this array is 8 bit
unsigned integers, each element should have a value from 0 to
255.
Single Channel Array
For multi channel arrays :
We can define all of above data types for multi channel arrays
(supports up to 512 channels). Here I am going to show you
how to define CV_8U data type for multi channel arrays.
 CV_8UC1 (single channel array with 8 bit unsigned
integers)
 CV_8UC2 (2 channel array with 8 bit unsigned integers)
 CV_8UC(n) (n channel array with 8 bit unsigned integers
(n can be from 1 to 512) )
e.g. 1 : Here I have illustrated a 3 channel array with 8 bit
unsigned integers. As the datatype is 8 bit unsigned integers,
each element should have a value from 0 to 255. Because this
is a 3 channel array, array consists of tuples with 3 elements.
The first tuple is {54, 0, 34}, second tuple is {58, 78, 185} and
so on.
3 Channel Arrays
e.g. 2 : Here I have illustrated a 2 channel array with 8 bit

signed integers. As the datatype is 8 bit signed integers, each
element should have a value from -128 to 127. Because this is
a 2 channel array, array consists of tuples with 2 elements. The
first tuple is {-85, -127}, second tuple is {25, 23} and so on.
2 Channel Array
Note : CV_8U = CV_8UC1 = CV_8UC(1)
Example Usage :
o Mat img1(3, 5, CV_32F ); //3 x 5 single-channel array

with 32 bit floating point numbers
o Mat img2(23, 53, CV_64FC(5) ); //23 x 53 5-channel
array with 64 bit floating point numbers
o Mat img3(Size(100, 200), CV_16UC2 ); //100 x 200
2-channel array with 16 bit unsigned integers
Remember :
Some OpenCV functions can handle only a subset of above
data types. So, be careful, when using OpenCV functions.
Bit Depths for IplImage (C style)

o IPL_DEPTH_<bit_depth>(S|U|F)
 Here possible values for <bit_depth> are 1,8,16,32
and 64
 S = Signed
 U = Unsigned
 F = Float
 1 bit depth images should be unsigned
 8 bit depth images should be signed or unsigned
 16 bit depth images should be signed or unsigned
 32 bit depth images should be signed or float
 64 bit depth images should be float
o E.g.:
 IPL_DEPTH_1U (1 bit depth and unsigned)
 IPL_DEPTH_8U (8 bit depth and unsigned)
 IPL_DEPTH_16U
 IPL_DEPTH_32F ( 32 bit depth and float )
 IPL_DEPTH_8S
 IPL_DEPTH_16S ( 16 bit depth and signed )
 IPL_DEPTH_32S
 IPL_DEPTH_64F
Bit depth means the number of bits allocated for a pixel. For example,
IplImage with IPL_DEPTH_8U uses 8 bit unsigned integer per each
pixel. That means each pixel can hold 0 to 255 integer numbers.
IPL_DEPTH_8U, IPL_DEPTH_8S,
IPL_DEPTH_16S, IPL_DEPTH_32S, IPL_DEPTH_32F and
IPL_DEPTH_64F are currently supported by IplImage data structure.
Read & Display Image
Read Image from File and Display
Here I am going to explain how to read an image from a file and

display the content using OpenCV library functions. First of all, open
your C++ IDE and create a new project. You have to configure your
new project in order to use OpenCV library functions. If you have
not configured the project for OpenCV yet, please refer to Installing
& Configuring with Visual Studio.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////
#include "opencv2/highgui/highgui.hpp"
#include <iostream>
using namespace cv;

using namespace std;
int main( int argc, const char** argv )

{
Mat img = imread("MyPic.JPG",
CV_LOAD_IMAGE_UNCHANGED); //read the image data in
the file "MyPic.JPG" and store it in 'img'
if (img.empty()) //check whether the image is loaded or

not
{
cout << "Error : Image cannot be loaded..!!" << endl;
//system("pause"); //wait for a key press
return -1;
}
namedWindow("MyWindow",
CV_WINDOW_AUTOSIZE); //create a window with the name
"MyWindow"
imshow("MyWindow", img); //display the image which is
stored in the 'img' in the "MyWindow" window
waitKey(0); //wait infinite time for a keypress
destroyWindow("MyWindow"); //destroy the window

with the name, "MyWindow"
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////
Before you run this program, put any image file (MyPic.JPG) into the
folder where your c++ file is.Otherwise you have to change the first
argument of imread() function and give the absolute path to your
image file.
You can download this OpenCV visual c++ project from here. (The
downloaded file is a compressed .rar folder. So, you have to extract it
using Winrar or other suitable software)
Load & Diplay Image from File

Explanation
Let's review the above OpenCV code line by line.

 #include "stdafx.h"
If you are developing your application in Visual Studio, don't forget
to put this line above the code.
 #include "opencv2/highgui/highgui.hpp"
imread(), namedWindow(), imshow() and waitKey() functions
are declared in the above header file. So you must include it.
We are using Mat data structure in the above program. It is

declared in "opencv2/core/core.hpp" header file. Then why don't
we include this? It's
because "opencv2/highgui/highgui.hpp" header file include that
header file inside it. So, we don't need to include it again in our
program.
 using namespace cv;

All the data structures and functions
in "opencv2/core/core.hpp" and "opencv2/highgui/highgui.
hpp" are declared inside cv namespace. So, we have to add
the above line in the top of our program. Otherwise we have to
append 'cv::' specifier before each OpenCV functions and data
structures. (e.g - cv::Mat, cv::imread() , etc). If you are not
familiar with namespaces, please refer this article.
'#include <iostream>' and 'using namespace std' are added

because we are using 'cout' to display some strings in the
console. This is basic C++ and you should be familiar with this.
 Mat img = imread(const string& filename,

int flags=CV_LOAD_IMAGE_COLOR)
Mat is a data structure to store images in a matrix. It is declared
in "opencv2/core/core.hpp" header file.
imread() is a function declared
in "opencv2/highgui/highgui.hpp" header file. It loads an image
from a file and stores it in Mat data structure.
Arguments of imread() function

 filename - location of the file. If you just give the filename
only, that image should be in the same folder as your C++ file.
Otherwise you have to give the full path to your image.
 flags - There are four possible inputs
o CV_LOAD_IMAGE_UNCHANGED - image-depth=8
bits per pixel in each channel, no. of channels=unchanged
o CV_LOAD_IMAGE_GRAYSCALE - image depth=8
bits, no. of channels=1
o CV_LOAD_IMAGE_COLOR - image-depth=?, no. of
channels=3
o CV_LOAD_IMAGE_ANYDEPTH - image-
depth=unchanged , no. of channels=?
o CV_LOAD_IMAGE_ANYCOLOR - image-depth=?, no.
of channels=unchanged
You can combine these above parameters to get desired image

output.
e.g -
CV_LOAD_IMAGE_ANYDEPTH |
CV_LOAD_IMAGE_ANYCOLOR - image-
depth=unchanged, no. of channels=unchanged
CV_LOAD_IMAGE_COLOR | CV_LOAD_IMAGE_ANYDEPTH
- image-depth=unchanged, no. of channels=3
If you are not sure what to do, use CV_LOAD_IMAGE_COLOR

as the 2nd parameter of imread() function.
To understand image-depth and concept of channels, you

should be familiar with theory of image processing. So, let's
discuss little bit of theory of image processing.
Any digital image consists of pixels. Any pixel should have
some value. The minimum value for a pixel is 0 and it
represents black.When the value of the pixel is increased, the
intensity of that pixel is also increased. In a computer memory,
). In decimal, it is 255. fixed number of bits are allocated for
every pixel. Say the number of allocated bits per pixel is 8.
Then the maximum number that a pixel can have is 255
(11111111 in binary)
Now what is image-depth? The image-depth means the

number of bits allocated for each pixel. If it is 8, each pixel can
have a value between 0 and 255. If it is 4, each pixel can have
a value between 0 to 15 (1111 in binary).
Here is a simple model of a image with image-depth of 8 bits.

Each small box represents a pixel. So, each box may contain a
value between 0 to 255.
Here is some properties of the following image.

o Image-depth 8 bit
o 1 channel ( So, this is a grayscale image )
o The height is 4 pixel
o The width is 5 pixels
o The resolution of this image is 4x5.
This is a grayscale image (black and white image) because this
image has no color content. If the value of this pixel is higher, it
will be shown more brighter. If the value is low, it will be shown
more darker.
Grayscale Image with image depth of 8
Following image is a simple model of a color image. Color

image should consist of at least 3 planes; Red, Green and
Blue. Any color can be created using a particular combination
of these 3 colors. Any pixel is a combination of three 3 values.
(255, 0, 0) represent pure red. (0, 255, 0) represent pure green.
(255, 0, 255) represents pure violate. In the same way, you can
create many color. Image-depth is 24 because each pixel is
represented with 8 x 3 bits (8 bits from each channel).
Here is some properties of the following image.
o Image-depth 24 bit
o 3 channels ( So, this is a color image )
o The height is 4 pixel
o The width is 5 pixels
o The resolution of this image is 4x5.
R, G, B planes of a color image
In the above model, top left pixel is (23, 231, 46). It will be
shown as a greenish color because the green value(231) of
that pixel is larger than the red(23) and blue(46) value.
 if (img.empty())
If imread() function fails to load the image, 'img' will not be
loaded any data. Therefore 'img.empty()' should return true.
It's a good practice to check whether the image is loaded
successfully and if not exit the program. Otherwise your
program will crash when executing imshow()function.
 bool Mat::empty()
This function returns true, if Mat::data==NULL or Mat::total() == 0
 //system("pause");
If you are using Visual Studio, it's better to uncomment this line
because it will pause the program until user press any key. If
we don't uncomment it, the program will exit immediately so
that user will not see the error message.
 void namedWindow(const string& winname, int flags
= WINDOW_AUTOSIZE);
This function creates a window.
Parameters -
o winname - Title of the window. That name will
display in the title bar of the newly created window
o flags - determine the size of the window. There are
two options
 WINDOW_AUTOSIZE - User cannot resize the
image. Image will be displayed in its original size
 CV_WINDOW_NORMAL - Image will resized if
you resize the the window
 void imshow(const string& winname, InputArray mat);

This function shows the image which is stored in the 'mat' in a
window specified by winname. If the window is created
with WINDOW_AUTOSIZE flag, image will be displayed in its
original size. Otherwise image may be scaled to the size of the
window.
Parameters -
o winname - Title of the window. This name is used to
identify the window created by namedWindow() function.
o mat - hold the image data
 int waitKey(int delay = 0)

waitKey() function wait for keypress for certain time, specified
by delay (in milliseconds). If delay is zero or negative, it will wait
for infinite time. If any key is pressed, this function returns the ASCII
value of the key and your program will continue. If there is no key
press for the specified time, it will return -1 and program will
continue.
 void destroyWindow(const string& winname)
This function closes the opened window, with the title
of winname and deallocate any associated memory usage. This
function is not essential for this application because when the
program exits, operating system usually close all the opened windows
and deallocate any associated memory usage.
Summary
When running this program, the image of 'MyPic.JPG' is loaded into

the variable, 'img' of type Mat. Then a window named 'MyWindow'
is opened. After that 'img' is loaded to that window. The window
with the image will be displayed until any key is pressed.
Create a Blank Image & Display

This program is also very much similar to the previous application.
The only difference is that this program creates a blank image instead
of loading an existing image from a file.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////
#include <iostream>
using namespace cv;


{
Mat img(500, 1000, CV_8UC3, Scalar(0,0, 100)); //create
an image ( 3 channels, 8 bit image depth, 500 high, 1000
wide, (0, 0, 100) assigned for Blue, Green and Red plane
respectively. )

not
{
cout << "Error : Image cannot be loaded..!!" << endl;
return -1;
}
"MyWindow"
waitKey(0); //wait infinite time for a keypress
destroyWindow("MyWindow"); //destroy the window

with the name, "MyWindow"
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////
Before you run this program, put any image file (MyPic.JPG) into the
folder where your c++ file is.Otherwise you have to change the first
argument of imread() function and give the absolute path to your
image file.
Create & Display Image
New OpenCV functions

 Mat::Mat(int rows, int cols, int type, const Scalar& s);
This is the one of the many constructor available in Mat class. It
initialize the Mat object with the value given by the Scalar object
Parameters :
o rows - Number of rows in the 2D array ( height of the
image in pixels)
o cols - Number of columns in the 2D array ( width of the
image in pixels)
o type - specify the bit depth, data type and number of
channels of the image. I gave CV_8UC3 and it specify 8 bit
unsigned integers with 3 channels. Here are some of possible inputs
for this parameter
 CV_8UC1 - 8 bit unsigned integers with single
channel
 CV_8UC3 - 8 bit unsigned integers with 3 channels
 CV_64FC1 - 64 bit floating point value with 1
channels
If you want more details about this, please refer to Data Types for
Arrays in the Basics of OpenCV API
o s - Initialize each array element with the value given by s.
In the above application, I gave Scalar(0,0,100). So, it initialize my
first channel (Blue plane) with 0, 2nd channel (Green plane) with 0
and 3rd channel (Red Plane) with 100. So, my final image is red. You
can try different combinations of these three and see the output image.
Summary
In this program, I created a 3 channel image with 500 height and 1000
width. 8 bit unsigned integer is allocated for each pixel in each
channel. (8x3 = 24 bits per each pixel) And each pixel is assigned
with (0,0,100) scalar value. That means 1st channel is all zero, 2nd
channel is also all zero and the 3rd channel is all 100. Therefore we
can see a red image as the output of the program.
Capture Video from File or Camera
Capture Video From File
In this lesson, I am going to show you how to read a video file.

It's fairly simple. You just need to initialize a VideoCapture object
which will open a video file and read frame by frame from that
opened video. Here is the sample OpenCV code. If you are using
Visual Studio, you will need to include "stdafx.h" header file.
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////
#include <iostream>
using namespace cv;

int main(int argc, char* argv[])

{
VideoCapture
cap("C:/Users/SHERMAL/Desktop/SampleVideo.avi"); //
open the video file for reading
if ( !cap.isOpened() ) // if not success, exit program

{
cout << "Cannot open the video file" << endl;
return -1;
}
//cap.set(CV_CAP_PROP_POS_MSEC, 300); //start the

video at 300ms
double fps = cap.get(CV_CAP_PROP_FPS); //get the

frames per seconds of the video
cout << "Frame per seconds : " << fps << endl;
namedWindow("MyVideo",CV_WINDOW_AUTOSIZE); //c
reate a window called "MyVideo"
while(1)
{
Mat frame;
bool bSuccess = cap.read(frame); // read a new frame

from video
if (!bSuccess) //if not success, break loop

{
cout << "Cannot read the frame from video
file" << endl;
break;
}
imshow("MyVideo", frame); //show the frame in

"MyVideo" window
if(waitKey(30) == 27) //wait for 'esc' key press for 30

ms. If 'esc' key is pressed, break loop
{
cout << "esc key is pressed by user" << endl;
break;
}
}
return 0;
New OpenCV functions which are not found earlier are explained
here
 VideoCapture::VideoCapture(int device)
This is one of constructors available in VideoCapture class. This
constructor open the camera indexed by the argument of this
constructor and initializes the VideoCapture object for reading
the video stream from the specified camera.
Here the '0' means the index of the camera to be used. You can use
1,2,3.. instead of 0, if your computer is attached to more than 1 camera.
The destructor of this class will deallocated any associated memory
with this object. Therefore you don't need to deallocate memory
explicitly in your program.
 if (!cap.isOpened())
If the VideoCapture object initialization unsuccessful, the
expression inside the 'if 'parentheses will evaluate to true and the
statements inside the 'if' block will be executed.
It is a good practice to check this scenario and handle it
accordingly because otherwise it may cause the program to
crash.
 double dWidth =
cap.get(CV_CAP_PROP_FRAME_WIDTH)
This function obtain the width (in pixels) of frames of the camera
output.
 double dHeight =
cap.get(CV_CAP_PROP_FRAME_HEIGHT)
This function obtain the height (in pixels) of frames of the camera
output.
All other OpenCV functions and their arguments are exactly

same as the 1st program.
Write Image & Video to File
In this lesson, I am going to show you how to write images and videos
to a file using OpenCV.
If you have not install and configure OpenCV yet, please refer
to Installing & Configuring with Visual Studio.
Write Image to File

In the following example, a yellow image is created and written
into a file. Let's see how to do it with OpenCV.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////
#include <iostream>
using namespace cv;


{
Mat img(650, 600, CV_16UC3, Scalar(0,50000, 50000)); //create
an image ( 3 channels, 16 bit image depth, 650 high, 600 wide, (0,
50000, 50000) assigned for Blue, Green and Red plane respectively.
)

not
{
cout << "ERROR : Image cannot be loaded..!!" << endl;
return -1;
}
vector<int> compression_params; //vector that stores the
compression parameters of the image
compression_params.push_back(CV_IMWRITE_JPEG_QUA
LITY); //specify the compression technique
compression_params.push_back(98); //specify
the compression quality
bool bSuccess = imwrite("D:/TestImage.jpg", img,

compression_params); //write the image to file
if ( !bSuccess )
cout << "ERROR : Failed to save the image" << endl;
}
"MyWindow"
waitKey(0); //wait for a keypress
destroyWindow("MyWindow"); //destroy the window with

the name, "MyWindow"
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////
The above program is very much similar to the program under 'Create
a Blank Image & Display' section in the lesson of Read & Display
Image. If you need further clarifications of any OpenCV functions
which are not explained here, please refer to Read & Display
Image lesson.
New OpenCV functions which are not found earlier

 bool imwrite( const string& filename, InputArray
img, const vector<int>& params=vector<int>())
The function saves the image in the variable 'img' to a file, specified
by 'filename' . If this function fails to save the image, it will return
false. On success of writing the file to the harddisk, it will return true.
Parameters -
o filename - specify the location and name of the file to

be saved
o img - hold the image which is going to save
o params - This is a int vector to which you have to
insert some int parameters specifying the format of the image
 JPEG format - You have to
puch_back CV_IMWRITE_JPEG_QUALITY first and then a
number between 0 and 100 (higher is the better). If you want the best
quality output, use 100. I have used 98 in the above sample program.
But higher the value, it will take longer time to write the image
 PNG format - You have to
puch_back CV_IMWRITE_PNG_COMPRESSION first and then
a number between 0 and 9 (higher is the better compression, but
slower).
The image format is chosen depending on the file name extension. Only
images with 8 bit or 16 bit unsigned single channel or 3 channel (
CV_8UC1, CV_8UC3, CV_8SC1, CV_8SC3, CV_16UC1,
CV_16UC3) with 'BGR' channel order, can be saved. If the depth or
channel order of the image is different,
use 'Mat::convertTo()' or 'cvtColor' functions to convert the image
to supporting format before using imwrite function.
The above program is very much similar to the program under 'Create
a Blank Image & Display' section in the lesson of Read & Display
Image. If you need further clarifications of any OpenCV functions
which are not explained here, please refer to Read & Display
Image lesson.
Summary
This program creates an yellowish image ( 3 channels, 16 bit image
depth, 650 high, 600 wide, (0, 50,000, 50,000) assigned for BGR
channels). Because the image has 16 bit depth, you can use values
between 0 and 65535 for each element in each channel. I have used
50,000 for each element in the green and red planes which gives the
yellowish color. You can try different values.
Then it specifies the compressing technique. I have used JPEG as the
compression technique in the above example. Then it saves the image
in the "D:/TestImage.jpg" location. Then it displays the newly
created image in a window and wait indefinitely until an user
presses a key.
Write Video to File

In the following example, a video is captured from the webcam and
written into a file. Let's see how to do it with OpenCV.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////
#include <iostream>
using namespace cv;


{
VideoCapture cap(0); // open the video camera no. 0
if (!cap.isOpened())
// if not success, exit program
{
cout << "ERROR: Cannot open the video file" << endl;
return -1;
}
namedWindow("MyVideo",CV_WINDOW_AUTOSIZE); //crea
te a window called "MyVideo"
double dWidth =
cap.get(CV_CAP_PROP_FRAME_WIDTH); //get the width of
frames of the video
double dHeight =
cap.get(CV_CAP_PROP_FRAME_HEIGHT); //get the height of
frames of the video
cout << "Frame Size = " << dWidth << "x" << dHeight << endl;
Size
frameSize(static_cast<int>(dWidth), static_cast<int>(dHeight));
VideoWriter oVideoWriter ("D:/MyVideo.avi",

CV_FOURCC('P','I','M','1'), 20, frameSize, true); //initialize the
VideoWriter object
if ( !oVideoWriter.isOpened() ) //if not initialize the VideoWriter

successfully, exit the program
{
cout << "ERROR: Failed to write the video" << endl;
return -1;
}
while (1)
{
Mat frame;
bool bSuccess = cap.read(frame); // read a new frame from

video

{
cout << "ERROR: Cannot read a frame from video file" <<
endl;
break;
}
oVideoWriter.write(frame); //writer the frame into the file
imshow("MyVideo", frame); //show the frame in "MyVideo"

window
if (waitKey(10) == 27) //wait for 'esc' key press for 30ms. If

'esc' key is pressed, break loop
{
break;
}
}
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////
Here is how I record my video from my webcam and save to a file

in D drive
here
 Size
frameSize(static_cast<int>(dWidth), static_cast<int>(dHeight))
Create a Size object with a given width and height. Here I have cast the
width and height to integers because they are originally double values
and the Size constructor does not accept double values as its
parameters.
 VideoWriter::VideoWriter(const string&
filename, int fourcc, double fps, Size frameSize, bool isColor=true)
This is the constructor of the VideoWriter class. It initializes the object
with following parameters
o const string& filename - Specify the name and the location
of the output file. The video stream is written into this file
o int fourcc - specify the 4 character code for the codec which
is used to compress the video. Your computer may not be supported
some codecs. So, if you fail to save the video, please try other
codecs. Here are some popular codecs.
 CV_FOURCC('D', 'I', 'V', '3') for DivX MPEG-4
codec
 CV_FOURCC('M', 'P', '4', '2') for MPEG-4 codec
 CV_FOURCC('D', 'I', 'V', 'X') for DivX codec
 CV_FOURCC('P','I','M','1') for MPEG-1 codec
 CV_FOURCC('I', '2', '6', '3') for ITU H.263 codec
 CV_FOURCC('M', 'P', 'E', 'G') for MPEG-1 codec
Complete list of codec can be found here
Here I have used CV_FOURCC('P', 'I', 'M', '1') as the four character
code of codec to compress the video. If the output file cannot be
opened, you can try different four character code of codecs.
For Windows users, it is possible to use -1 instead of the above codecs
in order to choose compression method and additional compression
parameters from a dialog box. It is a best method for Microsoft
Windows users.
o double fps - frames per seconds of the video stream. I have

used 20. You can try different values. But the codec should support the
fps value. So, use an appropriate value.
o Size frameSize - Size object which specify the width and
the height of each frame of the video stream.
o bool isColor - If you want to save a color video, pass the
value as true. Otherwise false. Remember codec should support
whatever value, you pass. In the above example, you have to give true
as the 5th argument. Otherwise it will not work.
 if ( !oVideoWriter.isOpened() )
Check whether the VideoWriter object initialize successfully. If not
exit the program immediately.
 void write(const Mat& image)

Write a frame to the video stream. The size of the frame should be same
as the size you specified when initializing the VideoWriter object.
All other OpenCV functions have been discussed in earlier lessons. So,
if you are not familiar with those OpenCV functions yet, please go
through Capture Video from File or Camera
Filtering Images
Image filtering is an important part of computer vision. For most of
computer vision applications, filtering should be done before anything
else. OpenCV supports lots of in-build filtering methods for images.
Here is the list of filtering methods that I am going to discuss with
you in the following posts (with OpenCV 2.4.5 and C++ )
 Change Brightness of Image or Video

 Change Contrast of Image or Video
 Histogram Equalization of Grayscale or Color Image
 Smooth / Blur Images
Here is the list of image filtering methods which are explained using
examples with OpenCV 2.1 in C style (not C++)
 Eroding
 Dilating
 Inverting
Here is the original image which I am going to filter using above
methods.
Original Image
If you have not install and configure OpenCV yet, please refer
to Installing & Configuring with Visual Studio.
Eroding
Eroding is a simple way of filtering images. Here is how it can be
done with OpenCV.
///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>
#include <highgui.h>
int main()
{
//display the original image
IplImage* img = cvLoadImage("C:/MyPic.jpg");
cvNamedWindow("MyWindow");
cvShowImage("MyWindow", img);
//erode and display the eroded image

cvErode(img, img, 0, 2);
cvNamedWindow("Eroded");
cvShowImage("Eroded", img);
cvWaitKey(0);
//cleaning up
cvDestroyWindow("MyWindow");
cvDestroyWindow("Eroded");
cvReleaseImage(&img);
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////
You can download this OpenCV visual c++ project from here.
Eroded Image
here
 cvErode(img, img, 0, 2)
The 1st parameter is the source image.
The 2nd parameter is the destination image which is to be the eroded
image.
Here the 3rd parameter is the structuring element used for erosion. If
it is 0, a 3×3 rectangular structuring element is used.
The 4th parameter is the number of times, erosion is applied.
This function can process images in place. That means same variable
can be used for the 1st and 2nd parameters.
If you want more explanation about various methods in the above

computer application , please refer to Capturing Images & Videos.
Dilating
Dilating is something like opposite of the eroding an image. Here is
the OpenCV code.
///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>
int main()
{
//dilate and display the dilated image

cvDilate(img, img, 0, 2);
cvNamedWindow("Dilated");
cvShowImage("Dilated", img);
cvWaitKey(0);
//cleaning up
cvDestroyWindow("Dilated");
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////
Dilated Image
here
 cvDilate(img, img, 0, 2)
The 1st parameter is the source image.
The 2nd parameter is the destination image which is to be the dilated
image.
Here the 3rd parameter is the structuring element used for dilation. If
it is 0, a 3×3 rectangular structuring element is used.
The 4th parameter is the number of times, dilation is applied.
Inverting
Inverting an image is like taking the negative of an image.
///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>
int main()
{
//invert and display the inverted image

cvNot(img, img);
cvNamedWindow("Inverted");
cvShowImage("Inverted", img);
cvWaitKey(0);
//cleaning up
cvDestroyWindow("Inverted");
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////
Inverted Image
New OpenCV functions which are not found earlier are

explained here
 cvNot(img, img)
This function inverts every bit in every element of the image in the 1st
parameter and places the result in the image in the 2nd parameter.
e.g - For a 8 bit image, the value 0 will be mapped to (255-0)=255

the value 46 will be mapped to (255-46)=209
For a 16 bit image, the value 0 will be mapped to (65535-

0)=65535
the value 46 will be mapped to (65535-
46)=65489
Change Brightness of Image or Video
Changing brightness is a point operation on each pixel. If you
want to increase the brightness, you have to add some
constant value to each and every pixel.
new_img (i, j) = img(i, j) + c
If you want to decrease the brightness, you have to subtract

some constant value from each and every pixel.
new_img (i, j) = img(i, j) - c
e.g- Say, this is your original image
Original Image
Say, you want to increase the brightness of the image by 20

units. Here is the output image of which the brightness is
increased by 20 units.
Image of which brightness is increased
Say, you want to decrease the brightness of the image by 20

units. Here is the output image of which the brightness is
decreased by 20 units.
Image of which brightness is decreased
Note :
You may already notice that although the 1st pixel of the above
image should have (12 - 20) = -8, I have put 0. It is because
pixels never have negative values. Any pixel value is bounded
below by 0 and bounded above by 2^(bit depth).
Change the Brightness of an Image
Now I am going to show you how to increase or decrease

brightness of an image using an OpenCV C++ example.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////
#include <iostream>
using namespace cv;


{
CV_LOAD_IMAGE_COLOR);
if (img.empty())
{
cout << "Image cannot be loaded..!!" << endl;
return -1;
}
Mat imgH = img + Scalar(75, 75, 75); //increase the

brightness by 75 units
//img.convertTo(imgH, -1, 1, 75);
Mat imgL = img + Scalar(-75, -75, -75); //decrease the

//img.convertTo(imgL, -1, 1, -75);
namedWindow("Original Image",
CV_WINDOW_AUTOSIZE);
namedWindow("High Brightness",
namedWindow("Low Brightness",
imshow("Original Image", img);

imshow("High Brightness", imgH);
imshow("Low Brightness", imgL);
waitKey(0);
destroyAllWindows(); //destroy all open windows
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////
Here is the original image.
original image
Here is the image of which brightness is increased by the

OpenCV program.
Brightness is increased with OpenCV
Here is the image of which brightness is decreased by the

OpenCV program.
Brightness is decreased with OpenCV
 Mat imgH = img + Scalar(75, 75, 75);

This line of code adds 75 to each and every pixel in the 3
channels (B, G, R channels) of 'img'. Then it assigns this new
image to 'imgH'.
Instead you can use this function also.
img.convertTo(imgH, -1, 1, 75);
 Mat imgL = img + Scalar(-75, -75, -75);

This line of code subtracts 75 from each and every pixel in the
3 channels (B, G, R channels) of 'img'. Then it assigns this
new image to 'imgL'.
Instead you can use this function also.
img.convertTo(imgL, -1, 1, -75);
Change the Brightness of a Video
Now I am going to show you how to increase or decrease the

brightness of a video using an OpenCV C++ example. This is
pretty much similar to the previous example.
/////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////
#include <iostream>
using namespace cv;


{
VideoCapture
cap("C:/Users/SHERMAL/Desktop/SampleVideo.wmv"); //
open the video file for reading

{
return -1;
}
namedWindow("Original
Video",CV_WINDOW_AUTOSIZE); //create a window called
"Original Video"
namedWindow("Brightness
Increased",CV_WINDOW_AUTOSIZE); //create a window
called "Brightness Increased"
namedWindow("Brightness
Decreased",CV_WINDOW_AUTOSIZE); //create a window
called "Brightness Decreased"
while(1)
{
Mat frame;
bool bSuccess = cap.read(frame); // read a new

frame from video

{
file" << endl;
break;
}
Mat imgH = frame + Scalar(50, 50, 50); //increase the

Mat imgL = frame + Scalar(-50, -50, -50); //decrease

the brightness by 75 units
imshow("Original Video", frame); //show the frame in

"Original Video" window
imshow("Brightness Increased", imgH); //show the
frame of which brightness increased
imshow("Brightness Decreased", imgL); //show the
frame of which brightness decreased
if (waitKey(30) == 27) //wait for 'esc' key press for 30

ms. If 'esc' key is pressed, break loop
{
break;
}
}
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////
Change Contrast of Image or Video
Changing the contrast is also a point operation on each pixel. The

easiest way to increase the contrast of an image is, multiplying each
pixel value by a number larger than 1.
new_img (i, j) = img(i, j) * c c>1
The easiest way to decrease the contrast is, multiplying each pixel value
by a number smaller than 1.
new_img (i, j) = img(i, j) * c c<1
There are more advance methods to adjust contrast of an image such as

histogram equalization. Such method adjust the contrast of an image
such that color distribution is balanced equally. We will discuss the
histogram equalization in the next lesson.
e.g- Say, this is your original image

Example Image
By multiplying each pixel value by 2, you can effectively double the

contrast of an image. Here is the image of which the contrast is
increased.
I have considered this image as a 8 bit unsigned image. So, any pixel
value should be from 0 to 255. If the resulting image has values more
than 255, it should be rounded off to 255.
Contrast Increased
By multiplying each pixel value by 0.5, you can effectively halve

the contrast of an image. Here is the image of which contrast is
decreased.
Contrast Decreased
Change Contrast of an Image

How to increase or decrease the contrast of an image is demonstrated
in the following OpenCV C++ example. Keep in mind that this is the
very basic way of changing contrast. In the next lesson, I'll show you
how to change the contrast using histogram equalization.
/////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////
#include <iostream>
using namespace cv;

CV_LOAD_IMAGE_COLOR); //open and read the image
if (img.empty())
{
return -1;
}
Mat imgH;
img.convertTo(imgH, -1, 2, 0); //increase the contrast (double)
Mat imgL;
img.convertTo(imgL, -1, 0.5, 0); //decrease the contrast (halve)
//create windows
namedWindow("High Contrast",
namedWindow("Low Contrast",
//show the image

imshow("High Contrast", imgH);
imshow("Low Contrast", imgL);
waitKey(0); //wait for key press

return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////
Here is the original image.
Original Image
Here is the image of which contrast is increased by the OpenCV

program.
Contrast is increased with OpenCV
Here is the image of which contrast is decreased by the OpenCV

program.
Contrast is decreased with OpenCV

New OpenCV methods
 void convertTo( OutputArray
m, int rtype, double alpha=1, double beta=0 )
This OpenCV function converts image into another format with
scaling. Scaling is done according to the following formula.
m[i,j] = alfa * img[i,j] + beta
Here is the parameters of this function

o OutputArray m - Store the converted image
o int rtype - Depth of the output image. If the value
of rtype is negative, output type is same as the input image. I
have used a negative value in the above program because I don't
want to change the depth of the original image. Possible inputs
to this parameter
 CV_8U
 CV_32S
 CV_64F
 -1
Complete list of depths can be found in Basics of OpenCV API
o double alpha - Multiplication factor; Every pixel will be
multiplied by this value
o double beta - This value will be added to very pixels
after multiplying with the above value.
Here is the formula again. Here m[i, j] means a pixel at ith row
and jth column.
m[i,j] = alfa * img[i,j] + beta
Change the Contrast of a Video

It is similar to the above program except that you have to change the
contrast for each and every frame of the video. Here is the example
OpenCV program.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////
#include <iostream>
using namespace cv;


{
VideoCapture
cap("C:/Users/SHERMAL/Desktop/SampleVideo.wmv"); // open
the video file for reading

{
return -1;
}
//create windows
namedWindow("Original
Video",CV_WINDOW_AUTOSIZE);
namedWindow("Contrast
Increased",CV_WINDOW_AUTOSIZE);
namedWindow("Contrast
Decreased",CV_WINDOW_AUTOSIZE);
while (1)
{
Mat frame;
bool bSuccess = cap.read(frame); // read a new frame from

video

{
cout << "Cannot read the frame from video file" <<
endl;
break;
}
Mat imgH;
frame.convertTo(imgH, -1, 2, 0); //increase the contrast
(double)
Mat imgL;
frame.convertTo(imgL, -1, 0.5, 0); //decrease the contrast
(halve)
//show the image

imshow("Original Video", frame);
imshow("Contrast Increased", imgH);
imshow("Contrast Decreased", imgL);
if (waitKey(30) == 27) //wait for 'esc' key press for 30 ms. If

{
break;
}
}
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////
downloaded file is a compressed .rar folder. So, you have to
extract it using Winrar or other suitable software)
All the OpenCV methods in the above example program have been
discussed in previous lessons. So, I am not going to repeat them again.
Histogram Equalization of Grayscale or Color Image
Histogram
Histogram is the intensity distribution of an image.
E.G -
Consider the following image. Say, depth of the image is 2 bits.
Therefore the value range for each and every pixel is from 0 to 3.
Sample Image (Depth = 2 bits)
Histogram of the a image shows how the pixel values are distributed.
As you can see in the above image there are 5 pixels with value 0, 7
pixels with value 1, 9 pixels with value 2 and 4 pixels with value 3.
These information is tabulated as follows.
Intensity Distribution of above image
Histogram of a image usually presented as a graph. The following

graph represents the histogram of the above image.
Image Histogram
Histogram Equalization
Histogram Equalization is defined as equalizing the intensity
distribution of an image or flattening the intensity distribution curve.
Histogram equalization is used to improve the contrast of an image.
The equalized histogram of the above image should be ideally like the
following graph.
Equalized Histogram
But practically, you cannot achieve this kind of perfect histogram
equalization. But there are various techniques to achieve histogram
equalization close to the perfect one. In OpenCV, there is an in-built
OpenCV function to equalize histogram.
Histogram Equalization of Grayscale image

Here is the sample program demonstrating how to equalize the
histogram of a grayscale image (black and white image) using a
OpenCV in-built function.
/////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
using namespace cv;


{
if (img.empty())
{
return -1;
}
cvtColor(img, img, CV_BGR2GRAY); //change the color

image to grayscale image
Mat img_hist_equalized;
equalizeHist(img, img_hist_equalized); //equalize the
histogram
//create windows
namedWindow("Histogram Equalized",
//show the image

imshow("Histogram Equalized", img_hist_equalized);
return 0;
}
/////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////
Original Image
Image with Equalized Histogram

 void cvtColor( InputArray src, OutputArray dst, int code,
int dstCn=0 )
This function converts image from one color space to another color
space.
OpenCV usually loads an image in BGR color space. In the above
example, I want to change the image to grayscale color space. So, I
use the CV_BGR2GRAY as the 3rd parameter. If you want to convert
to HSV color space, you should use CV_BGR2HSV.
This is an explanation of each parameters of the above function.
o InputArray src- Input image ( it should be 8 bit unsigned

or 16 bit unsigned or 32 bit floating point image)
o OutputArray dst - Output image ( It should have a same
size and depth as the source image )
o int code- Should specify the color space conversion. There
are many codes available. Here are some of them.
 CV_BGR2HSV
 CV_HSV2BGR
 CV_RGB2HLS
 CV_HLS2RGB
 CV_BGR2GRAY
 CV_GRAY2BGR
o int dstCn - Number of channels in the destination image.
If it is 0, number of channels of the destination image is
automatically derived from source image and color space conversion
code. For a beginner, it is recommended to use 0 for this parameter.
 void equalizeHist( InputArray src, OutputArray dst )

This function equalizes the histogram of a single channel image (
Grayscale image is a single channel image )
By equalizing the histogram, the brightness is normalized. As a result,
the contrast is improved.
Here is the description of each parameters of the above OpenCV

function.
o InputArray src - 8 bit single channel image
o OutputArray dst - Destination image of which histogram
is equalized ( It should have the same size and depth as the source
image. )
Histogram Equalization of Color image

In the above example, I have shown how to equalize the histogram of
a grayscale image. Now I am going to show you how to equalize
histogram of a color image using sample OpenCV program.
/////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////
#include <iostream>
using namespace cv;


{
if (img.empty()) //if unsuccessful, exit the program

{
return -1;
}
vector<Mat> channels;
Mat img_hist_equalized;
cvtColor(img, img_hist_equalized, CV_BGR2YCrCb); //change

the color image from BGR to YCrCb format
split(img_hist_equalized,channels); //split the image into

channels
equalizeHist(channels[0], channels[0]); //equalize histogram on

the 1st channel (Y)
merge(channels,img_hist_equalized); //merge 3 channels including

the modified 1st channel into one image
cvtColor(img_hist_equalized, img_hist_equalized,
CV_YCrCb2BGR); //change the color image from YCrCb to BGR
format (to display image properly)
//create windows
namedWindow("Original Image", CV_WINDOW_AUTOSIZE);
namedWindow("Histogram Equalized",
//show the image

imshow("Histogram Equalized", img_hist_equalized);
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////
You can download this OpenCV visual c++ project
from here. (The downloaded file is a compressed .rar folder. So,
you have to extract it using Winrar or other suitable software)
Original Color Image
Color Image with Equalized Histogram

 cvtColor(img, img_hist_equalized, CV_BGR2YCrCb)

This line converts the color space of BGR in 'img' to YCrCb color
space and stores the resulting image in 'img_hist_equalized'.
In the above example, I am going to equalize the histogram of

color images. In this scenario, I have to equalize the histogram
of the intensity component only, not the color components. So,
BGR format cannot be used because its all three planes
represent color components blue, green and red. So, I have to
convert the original BGR color space to YCrCb color space
because its 1st plane represents the intensity of the image where
as other planes represent the color components.
 void split(const Mat& m, vector<Mat>& mv )

This function splits each channel of the 'm' multi-channel array
into separate channels and stores them in a vector, referenced
by 'mv'.
Argument list
o const Mat& m - Input multi-channel array
o vector<Mat>& mv - vector that stores the each
channel of the input array
 equalizeHist(channels[0], channels[0]);
Here we are only interested in the 1st channel (Y) because
it represents the intensity information whereas other two
channels (Cr and Cb) represent color components. So, we
equalize the histogram of the 1st channel using OpenCV in-built
function, 'equalizeHist(..)' and other two channels remain
unchanged.
 void merge(const vector<Mat>& mv, OutputArray dst )

This function does the reverse operation of the split function. It
takes the vector of channels and create a single multi-channel
array.
Argument list
o const vector<Mat>& mv - vector that holds several
channels. All channels should have same size and same depths
o OutputArray dst - stores the destination multi-
channel array
 cvtColor(img_hist_equalized, img_hist_equalized,
CV_YCrCb2BGR)
This line converts the image from YCrCb color space to BGR
color space. It is essential to convert to BGR color space
because 'imshow(..)' OpenCV function can only show images
with that color space.
This is the end of the explanation of new OpenCV functions,

found in the above sample code. If you are not familiar with other
OpenCV functions, please refer to the previous lessons.
Smooth / Blur Images
In this lesson, I am going to explain how to smooth an image.
Sometimes it is also called blurring. The main objective of
smoothing is to reduce noise. Such noise reduction is a typical
image pre-processing method which will improve the final
result.
There are various ways to smooth or blur an image. I will
explain you of most common smoothing techniques with
OpenCV C++ examples.
1. Homogeneous Smoothing
2. Gaussian Smoothing
3. Median Smoothing
4. Bilateral Smoothing
Smoothing is done by sliding a window (kernel or filter) across
the whole image and calculating each pixel a value based on
the value of the kernel and the value of overlapping pixels of
original image. This process is mathematically called as
convolving an image with some kernel. The kernel is the only
difference in all of the above types of smoothing (Blurring)
methods.
For example, 5 x 5 kernel used in homogeneous smoothing

(blurring) is as below. This kernel is known as "Normalized
box filter".
whereas 5 x 5 kernel used in Gaussian smoothing (blurring) is
as below. This kernel is known as "Gaussian kernel"
I will give you some important facts about smoothing kernels

(filters)
 Number of rows and number of columns of a kernel should
be odd (e.g. - 3x3, 11x5, 7x7, etc)
 When the size of the kernel is getting larger, processing
time also becomes larger
Homogeneous Smoothing
"Homogeneous Smoothing" is also called as "Homogeneous
Blurring", "Homogeneous Filtering" or "Box Blurring". This is the
most simplest method of smoothing an image. It takes simply
the average of the neighbourhood of a pixel and assign that
value to itself.
You have to choose right size of the kernel. If it is too large,
small features of the image may be disappeared and image will
look blurred. If it is too small, you cannot eliminate noises of the
image.
OpenCV code
In the following OpenCV code, kernel size is increasing from

1x1 to 29x29. Observe the smoothed image when increasing
the kernel size. The size of the kernel by which the image is
smoothed, is displayed in the middle of the image.
Note that in the following OpenCV code, kernel size is

incremented by two in each steps to get odd numbers.
/////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////
using namespace cv;
int main( int argc, char** argv )

{
//create 2 empty windows
namedWindow( "Original Image" ,
CV_WINDOW_AUTOSIZE );
namedWindow( "Smoothed Image" ,
// Load an image from file
Mat src = imread( "MyPic.JPG", 1 );
//show the loaded image

imshow( "Original Image", src );
Mat dst;
char zBuffer[35];
for ( int i = 1; i < 31; i = i + 2 )

{
//copy the text to the "zBuffer"
_snprintf_s(zBuffer, 35,"Kernel Size : %d x %d", i, i);
//smooth the image in the "src" and save it to "dst"

blur( src, dst, Size( i, i ) );
//put the text in the "zBuffer" to the "dst" image

putText( dst, zBuffer, Point( src.cols/4, src.rows/8),
CV_FONT_HERSHEY_COMPLEX, 1, Scalar(255, 255, 255) );
//show the blurred image with the text

imshow( "Smoothed Image", dst );
//wait for 2 seconds

int c = waitKey(2000);
//if the "esc" key is pressed during the wait, return

if (c == 27)
{
return 0;
}
}
//make the "dst" image, black

dst = Mat::zeros( src.size(), src.type() );

_snprintf_s(zBuffer, 35,"Press Any Key to Exit");

putText( dst, zBuffer, Point( src.cols/4, src.rows / 2),
//show the black image with the text

//wait for a key press infinitely

waitKey(0);
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////
You can download this OpenCV visual C++ project from here.
Original Image
Smoothed Image - Because of the kernel size is too large, the image
looks blurry
Explanation of New OpenCV Functions
 void blur( InputArray src, OutputArray dst, Size ksize, Point

anchor=Point(-1,-1), int borderType = BORDER_DEFAULT )
This OpenCV function will smooth an image using normalized
box filter (Homogeneous Blur). The normalized box filter
(kernel) is explained in the beginning of this post. Every
channel of the input image is processed independently.
o src - Source image. (The depth of the image should
be one of the following; CV_8U, CV_16S, CV_16U, CV_32F or
CV_64F)
o dst - Output image (It will have the same size and
same depth as the input image)
o ksize - Size of the kernel
o anchor - Point(-1,-1) value means that the anchor is
at the middle of the kernel. If you want, you can define your
own point
o borderType - You can define various border
interpolation methods. This value only affects the pixels at the
border. ( You can use one of the following;
BORDER_DEFAULT, BORDER_REFLECT,
BORDER_REPLICATE, BORDER_TRANSPARENT,
BORDER_REFLECT_101 )
 void putText( Mat& img, const string& text, Point org,

int fontFace, double fontScale, Scalar color, int
thickness=1, int lineType=8, bool bottomLeftOrigin=false )
This OpenCV function renders text string in the image.
o img - The image that you want to put your text
o text - The text that you want to put into the image
o org - Bottom-left corner of the text string in the image
o fontFace - Font type (You can use one of the
following; FONT_HERSHEY_SIMPLEX, FONT_HERSHEY_PLAI
N,
FONT_HERSHEY_DUPLEX, FONT_HERSHEY_COMPLEX, FON
T_HERSHEY_TRIPLEX, FONT_HERSHEY_COMPLEX_SMALL,
FONT_HERSHEY_SCRIPT_SIMPLEX or
FONT_HERSHEY_SCRIPT_COMPLEX. Any one of them can be
do "bitwise or" with FONT_ITALIC to italicize the font )
o fontScale - Scaling factor ( If you use 1, default font
size will be drawn)
o color - BGR color of the text
o thickness - Thickness of the lines of the font
o lineType - Line type
o bottomLeftOrigin - If this is true, the origin is set at
the bottom-left corner. Otherwise the origin is set at the top-left
corner
 static MatExpr zeros(Size size, int type)

This OpenCV function will return an array of specified size and
type with zero values
o size - size of array ( e.g - Size(no. of columns, no. of
rows) )
o type - type of the array elements
Gaussian Smoothing
"Gaussian Smoothing" is also called as "Gaussian Blurring" or

"Gaussian Filtering". This is the most commonly used
smoothing method. This is also used to eliminate noises in an
image. Gaussian kernel is slided across the image to produce
the smoothed image.
Size of the kernel and the standard deviation of the Gaussian
distribution in X and Y direction should be chosen carefully. For
more information about the Gaussian smoothing, please
refer here
OpenCV Code
This OpenCV/C++ example is very much similar to the previous

code except for a function.
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////
using namespace cv;

{
namedWindow( "Original Image" , CV_WINDOW_AUTOSIZE
);

Mat src = imread( "MyPic.JPG",
CV_LOAD_IMAGE_UNCHANGED );

Mat dst;
char zBuffer[35];
for ( int i = 1; i < 31; i = i + 2 )
{
//smooth the image using Gaussian kernel in the "src" and save
it to "dst"
GaussianBlur( src, dst, Size( i, i ), 0, 0 );

CV_FONT_HERSHEY_COMPLEX, 1, Scalar(255, 255, 255), 2 );



if (c == 27)
{
return 0;
}
}





waitKey(0);
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////
Smooth Image using Gaussian Filter
New OpenCV Function

There is only a new OpenCV function that we can find in the
above example.
 void GaussianBlur( InputArray src, OutputArray dst,
Size ksize, double sigmaX, double sigmaY=0, int
borderType=BORDER_DEFAULT )
This OpenCV function smooths an image using Gaussian
kernel. All channels of the image are processed independently.
o src - Source image. (The depth of the image should

be one of the following; CV_8U, CV_16S, CV_16U, CV_32F or
CV_64F)
same depth as the input image)
o ksize - Size of the Gaussian kernel (Both dimension
of the kernel should be positive and odd)
o sigmaX - Standard deviation in X direction. If 0 is
used, it will automatically calculated from the kernel size
o sigmaY - Standard deviation in Y direction. If 0 is
used, it will take the same value as the sigmaX.
Median Smoothing
"Median Smoothing" is also called as "Median Blurring" or

"Median Filtering". This is also a common smoothing technique.
The input image is convolved with a Median kernel. Median
filtering is widely used in edge detection algorithms because
under certain conditions, it preserves edges while removing
noise. For more information about median smoothing, please
refer to here.
OpenCV Code
Following OpenCV example is also as same as the previous

except for one line.
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////
using namespace cv;

{


Mat dst;
char zBuffer[35];
for ( int i = 1; i < 31; i = i + 2 )

{
//smooth the image using Median kernel in the "src" and

save it to "dst"
medianBlur( src, dst, i );

CV_FONT_HERSHEY_COMPLEX, 1, Scalar(255, 255, 255), 2
);



if (c == 27)
{
return 0;
}
}





waitKey(0);
return 0;
}
/////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////
Smooth Image using Median Filter (17 x 17)
New OpenCV Functions

 void medianBlur( InputArray src, OutputArray dst, int
ksize )
This OpenCV function smooth the input image using a Median
filter. All channels of the input image is processed
independently. This method works in-place.
o src - Input image ( images with 1, 3 or 4 channels /
Image depth should be CV_8U for any value of "ksize". If
"ksize" equals 3 or 5, image depths of CV_16U and CV_32F
are also supported.
o dst - Output image (It will be of same size and depth
as the input image)
o ksize - size of the filter ( It should be odd and greater
than 1 ) (Note - The resulting filter has the size
of ksize x ksize )
Bilateral Smoothing
"Bilateral Smoothing" is also called as "Bilateral Blurring" or

"Bilateral Filtering". This is the most advanced filter to smooth
an image and reduce noise. All of the above filters will smooth
away the edges while removing noises. But this filter is able to
reduce noise of the image while preserving the edges. The
drawback of this type of filter is that it takes longer time to
process. For more information about bilateral smoothing,
please refer to here.
OpenCV Code
In the following OpenCV/C++ example code is same as the

above examples except for one function.
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////
using namespace cv;

{


Mat dst;
char zBuffer[35];
for ( int i = 1; i < 31; i = i + 2 )

{
//smooth the image using Bilateral filter in the "src" and

save it to "dst"
bilateralFilter( src, dst, i, i, i);
CV_FONT_HERSHEY_COMPLEX, 1, Scalar(255, 255, 255), 2
);



if (c == 27)
{
return 0;
}
}





waitKey(0);
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////
Smooth Image using Bilateral Filter ( 9 x 9 )
New OpenCV Functions

 void bilateralFilter( InputArray src, OutputArray dst,
int d, double sigmaColor, double sigmaSpace, int
borderType=BORDER_DEFAULT )
This OpenCV function filters an image using a Bilateral kernel.
Each channel is processed independently. This method does
not work in place. For real time applications, it is advised to use
a value smaller than 5 for "d".
o src - Input image (Image with 1 or 3 channels)
depth as the input image)
o d - Diameter of each pixel neighbourhood
o sigmaColor - sigma in the color space
o sigmaSpace - sigma in the coordinate space
I have discussed with you about most commonly used
smoothing methods with OpenCV/C++ examples. Try all of the
above smoothing techniques using different filter sizes and
observe the output image and the processing time.
How to Add Trackbar
Trackbars are very useful in lots of occasions. It enables users to
change various parameters while the OpenCV application is running.
( If you have not install and configure OpenCV yet, please refer
to Installing & Configuring with Visual Studio. )
Simple Use of Trackbars

Whenever you change the position of a trackbar, the value of an
integer variable is changed. Using that value, we can change a
property of an image or a video. The following example will show
you how to do it with OpenCV.
OpenCV Example of How to Change Brightness and Contrast of

an Image with Trackbars
In the following example, I have added two trackbars to change the

brightness and contrast of an image. It is iterating in a infinite while
loop and applying the brightness and contrast to the image
periodically because I want to apply the changes to the image
whenever the user changes the position of the trackbar.
/////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////
#include <iostream>

using namespace cv;

{
// Read original image
Mat src = imread("MyPic.JPG");
//if fail to read the image

if (!src.data)
{
cout << "Error loading the image" << endl;
return -1;
}
// Create a window
namedWindow("My Window", 1);
//Create trackbar to change brightness

int iSliderValue1 = 50;
createTrackbar("Brightness", "My Window",
&iSliderValue1, 100);
//Create trackbar to change contrast

int iSliderValue2 = 50;
createTrackbar("Contrast", "My Window", &iSliderValue2,
100);
while (true)
{
//Change the brightness and contrast of the image (For
more infomation http://opencv-srf.blogspot.com/2013/07/change-
contrast-of-image-or-video.html)
Mat dst;
int iBrightness = iSliderValue1 - 50;
double dContrast = iSliderValue2 / 50.0;
src.convertTo(dst, -1, dContrast, iBrightness);
//show the brightness and contrast adjusted image

imshow("My Window", dst);
// Wait until user press some key for 50ms
int iKey = waitKey(50);
//if user press 'ESC' key

if (iKey == 27)
{
break;
}
}
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////
You can download this OpenCV visual c++ example from here.
Explanation of New OpenCV Functions
 int createTrackbar(const string&

trackbarname, const string& winname, int* value, int count,
TrackbarCallback onChange = 0, void* userdata = 0)
This OpenCV function creates a trackbar and attached that trackbar to
a specified window
o trackbarname - The name of the trackbar

o winname - The name of the window to which the trackbar
is attached
o value - This integer, pointed by this pointer, holds the
value associated with the position of the trackbar
o count - The maximum value of the trackbar. The
minimum value is always zero.
o onChange - This function will be called everytime the
position of the trackbar is changed. The prototype of this function
should be "FunctionName(int, void*)". The "int" value is the value
associate with the position of the trackbar. And "void*" is any pointer
value which you pass as the "userdata" (See the next parameter).
o userdata - This pointer variable will be passed as the
second parameter of the above function
All other functions have been discussed in the previous lessons. If you
have not followed them yet, please visit
 Read & Display Image
 Change Contrast of Image or Video
which have the all the other OpenCV functions in the above example
code.
Trackbar with Callback Function
In the above example, I have used only 4 parameters for the

"createTrackbar" function. But there are 2 more parameters. Here I
am going to explain, how to use a callback function using the 5th and
6th parameters of "createTrackbar". The advantage of using the
callback function is that it is not required to iterate in a while loop
periodically as in the above example.
In the following OpenCV example, I have added two trackbars to

change the brightness and contrast of an image. And a callback
function is implemented for each trackbar.
///////////////////////////////////////////////////////////////////////////////////////
#include <iostream>

using namespace cv;
Mat src;
void MyCallbackForBrightness(int iValueForBrightness, void *us

erData)
{
Mat dst;
int iValueForContrast = *( static_cast<int*>(userData) );
//Calculating brightness and contrast value

int iBrightness = iValueForBrightness - 50;
double dContrast = iValueForContrast / 50.0;
//Calculated contrast and brightness value

cout << "MyCallbackForBrightness : Contrast=" <<
dContrast << ", Brightness=" << iBrightness << endl;
//adjust the brightness and contrast


}
void MyCallbackForContrast(int iValueForContrast, void *userD

ata)
{
Mat dst;
int iValueForBrightness = *( static_cast<int*>(userData) );
//Calculating brightness and contrast value
int iBrightness = iValueForBrightness - 50;
double dContrast = iValueForContrast / 50.0;
//Calculated contrast and brightness value

cout << "MyCallbackForContrast : Contrast=" << dContrast
<< ", Brightness=" << iBrightness << endl;
//adjust the brightness and contrast


}
int main(int argc, char** argv)

{
// Read original image
src = imread("MyPic.JPG");

if (src.data == false)
{
return -1;
}
// Create a window
int iValueForBrightness = 50;

int iValueForContrast = 50;
//Create track bar to change brightness
createTrackbar("Brightness", "My Window",
&iValueForBrightness, 100, MyCallbackForBrightness,
&iValueForContrast);
//Create track bar to change contrast

createTrackbar("Contrast", "My Window",
&iValueForContrast, 100, MyCallbackForContrast,
&iValueForBrightness);
imshow("My Window", src);
// Wait until user press some key

waitKey(0);
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////
Trackbar with Callback Function
Explanation
I have used 2 callback functions; "MyCallbackForBrightness(int,

void*)" for the "Brightness" trackbar
and "MyCallbackForContrast(int, void*)"for
the "Contrast" trackbar.
I have used a global variables, src because it should be accessed from

all the 3 methods.
Examine closely the 5th and 6th parameters

of "createTrackbar" method in the "main" method.
Whenever the position of the "Brightness" trackbar is

changed, "MyCallbackForBrightness(int, void*)" will be called.
The 1st integer argument holds the value of the position of
the "Brightness" trackbar. The position of the "Contrast" trackbar is
passed as the 2nd argument. (Observe the 5th and 6th
parameter; createTrackbar("Brightness", "My Window",
&iValueForBrightness, 100, MyCallbackForBrightness,
&iValueForContrast);)
Whenever the position of the "Contrast" trackbar is

changed, "MyCallbackForContrast(int, void*)" will be called. The
1st integer argument holds the value of the position of
the "Contrast" trackbar. The position of the "Brightness" trackbar is
passed as the 2nd argument. (Observe the 5th and 6th
parameter; createTrackbar("Contrast", "My Window",
&iValueForContrast, 100, MyCallbackForContrast,
&iValueForBrightness))
How to Detect Mouse Clicks and Moves

OpenCV supports for detecting mouse events. Mouse events include
mouse clicks and movements over an attached OpenCV window.
OpenCV Example Code
It is very simple to do that. All you have do is to define a callback

function in the OpenCV C++ code attaching to the OpenCV window.
That callback function will be called every time, mouse events occur.
That callback function will also give the coordinates of the mouse
events. (e.g - (x, y) coordinate of a mouse click).
Here is the simple OpenCV code to detect left, right and middle mouse
clicks and mouse movements with its coordinates
/////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////
#include <iostream>

using namespace cv;
void CallBackFunc(int event, int x, int y, int flags, void*

userdata)
{
if ( event == EVENT_LBUTTONDOWN )
{
cout << "Left button of the mouse is clicked - position
(" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_RBUTTONDOWN )
{
cout << "Right button of the mouse is clicked -
position (" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_MBUTTONDOWN )
{
cout << "Middle button of the mouse is clicked -
position (" << x << ", " << y << ")" << endl;
}
else if ( event == EVENT_MOUSEMOVE )
{
cout << "Mouse move over the window - position (" <<
x << ", " << y << ")" << endl;
}
}

{
// Read image from file
Mat img = imread("MyPic.JPG");
if ( img.empty() )
{
return -1;
}
//Create a window
//set the callback function for any mouse event

setMouseCallback("My Window", CallBackFunc, NULL);
//show the image

imshow("My Window", img);

waitKey(0);
return 0;
/////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////
You can download this OpenCV visual C++ project from here
Detect Mouse Clicks and Moves Over the OpenCV Window
Summary
In the above OpenCV sample code, "CallbackFunc" function

will be called on any mouse event (Moving a mouse over the
attached OpenCV window is also a mouse event). By using
suitable if - else blocks, I printed only left, right and middle
mouse clicks and mouse movements over the window.
Here are new OpenCV functions, found in the above example

code. If you are not familiar with the other OpenCV functions as
well, please go through the other lessons in this tutorial.
 void setMouseCallback(const string& winname,

MouseCallback onMouse, void* userdata = 0)
This function sets a callback function to be called every time
any mouse events occurs in the specified window. Here is the
detailed explanation of the each parameters of the above
OpenCV function.
o winname - Name of the OpenCV window. All mouse

events related to this window will be registered
o onMouse - Name of the callback function. Whenever
mouse events related to the above window occur, this callback
function will be called. This function should have the signature
like the following
 void FunctionName(int event, int x, int y, int
flags, void* userdata)
 event - Type of the mouse event. These
are the entire list of mouse events
 EVENT_MOUSEMOVE
 EVENT_LBUTTONDOWN
 EVENT_RBUTTONDOWN
 EVENT_MBUTTONDOWN
 EVENT_LBUTTONUP
 EVENT_RBUTTONUP
 EVENT_MBUTTONUP
 EVENT_LBUTTONDBLCLK
 EVENT_RBUTTONDBLCLK
 EVENT_MBUTTONDBLCLK
 x - x coordinate of the mouse event
 y - y coordinate of the mouse event
 flags - Specific condition whenever a
mouse event occurs. See the next OpenCV example code for
the usage of this parameter. Here is the entire list of enum
values which will be possesed by "flags"
 EVENT_FLAG_LBUTTON
 EVENT_FLAG_RBUTTON
 EVENT_FLAG_MBUTTON
 EVENT_FLAG_CTRLKEY
 EVENT_FLAG_SHIFTKEY
 EVENT_FLAG_ALTKEY
 userdata - Any pointer passes to the
"setMouseCallback" function as the 3rd parameter (see
below)
o userdata - This pointer will be passed to the callback
function
OpenCV Example to Detect Mouse Clicks While Pressing a

Key
I am going to explain you how to detect a mouse event while

pressing a key of the keyboard.
The following OpenCV example code will detect left mouse

clicks while pressing the "CTRL" key , right mouse clicks while
pressing the "SHIFT" key and movements of the mouse over
the OpenCV window while pressing the "ALT" key.
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////
#include <iostream>

using namespace cv;
void CallBackFunc(int event, int x, int y, int flags, void*

userdata)
{
if ( flags == (EVENT_FLAG_CTRLKEY +
EVENT_FLAG_LBUTTON) )
{
cout << "Left mouse button is clicked while pressing
CTRL key - position (" << x << ", " << y << ")" << endl;
}
else if ( flags == (EVENT_FLAG_RBUTTON +
EVENT_FLAG_SHIFTKEY) )
{
cout << "Right mouse button is clicked while
pressing SHIFT key - position (" << x << ", " << y << ")" <<
endl;
}
else if ( event == EVENT_MOUSEMOVE && flags ==
EVENT_FLAG_ALTKEY)
{
cout << "Mouse is moved over the window while
pressing ALT key - position (" << x << ", " << y << ")" <<
endl;
}
}

{
// Read image from file
Mat img = imread("MyPic.JPG");

if ( img.empty() )
{
return -1;
}
//Create a window
//set the callback function for any mouse event

setMouseCallback("My Window", CallBackFunc, NULL);
//show the image

imshow("My Window", img);

waitKey(0);
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////
You can download this visual C++ OpenCV project from here.
No new OpenCV functions in the above example. If you have

some doubt about any OpenCV functions in the above
example, please refer to previous lessons.
Rotate Image & Video

Rotate Image
Rotating images by a given angle is a common image
processing task. Although it seems little bit complicated,
OpenCV provides some built-in functions making it easy to do
it. Here is a simple OpenCV C++ example code to rotate an
image. Here I use a track bar to change the rotating angle
dynamically.
/////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////
using namespace cv;

{
// Load the image
Mat imgOriginal = imread( "MyPic.JPG", 1 );
//show the original image

const char* pzOriginalImage = "Original Image";
namedWindow( pzOriginalImage,
imshow( pzOriginalImage, imgOriginal );
const char* pzRotatedImage = "Rotated Image";

namedWindow( pzRotatedImage,
int iAngle = 180;

createTrackbar("Angle", pzRotatedImage, &iAngle, 360);
int iImageHieght = imgOriginal.rows / 2;

int iImageWidth = imgOriginal.cols / 2;
while (true)
{
Mat matRotation = getRotationMatrix2D(
Point(iImageWidth, iImageHieght), (iAngle - 180), 1 );
// Rotate the image

Mat imgRotated;
warpAffine( imgOriginal, imgRotated, matRotation,
imgOriginal.size() );
imshow( pzRotatedImage, imgRotated );

int iRet = waitKey(30);
if ( iRet == 27 )
{
break;
}
}
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////
You can download this visual C++ project from here.
Original Image
Rotated Image
Explanation
Here is the explanation for new OpenCV functions which are not
found in previous lessons.
 Mat getRotationMatrix2D( Point2f center, double angle,
double scale )
This function returns 2x3 affine transformation matrix for the 2D

rotation.
Arguments -
o center - The center of the rotation of the the source

image.
o angle - Angle of rotation in degrees (Positive values for
counter-clockwise direction and negative values for clockwise
rotation)
o scale - The scaling factor of the image. (Scaling factor of 1
means its original size)
Try different values for center, angle and scale and observe
the output image.
 void warpAffine( InputArray src, OutputArray dst,

InputArray M, Size dsize, int flags = INTER_LINEAR, int
bordreMode=BORDER_CONSTANT, const Scalar&
borderValue=Scalar() )
This OpenCV function applies affine transformation to an
image.
Arguments -
o src - Source Image
o dst - Destination image which should have the same
type as the source image(The transformed image is stored in
this location)
o M - 2x3 affine transformation matrix
o dsize - Size of the destination image
o flags - Interpolation methods
o borderMode - pixel extrapolation method. (Try these
values; BORDER_REPLICATE, BORDER_CONSTANT, BORD
ER_REFLECT, BORDER_WRAP, BORDER_REFLECT_101,
BORDER_TRANSPARENT and BORDER_ISOLATED)
o borderValue - If you use BORDER_CONSTANT
for borderMode, this argument define the value used for the
border
Another OpenCV Example to Rotate an Image

Here is another way to rotate an image. Here I use a callback
function to apply the rotation instead of using a infinite while
loop. Other than the rotation, you can change the scale of the
image and border extrapolation method dynamically.
Here is the OpenCV C++ code.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////
using namespace cv;
int iAngle = 180;

int iScale = 50;
int iBorderMode = 0;
Mat imgOriginal ;
int iImageCenterY = 0;
int iImageCenterX = 0;
const char* pzRotatedImage = "Rotated Image";
void CallbackForTrackBar(int, void*)

{
Mat matRotation = getRotationMatrix2D( Point(
iImageCenterX, iImageCenterY ), (iAngle - 180), iScale /
50.0 );
// Rotate the image

Mat imgRotated;
warpAffine( imgOriginal, imgRotated, matRotation,
imgOriginal.size(), INTER_LINEAR, iBorderMode, Scalar() );
imshow( pzRotatedImage, imgRotated );

}

{
// Load the image
imgOriginal = imread( "MyPic.JPG", 1 );
iImageCenterY = imgOriginal.rows / 2;
iImageCenterX = imgOriginal.cols / 2;

const char* pzOriginalImage = "Original Image";
namedWindow( pzOriginalImage,
imshow( pzOriginalImage, imgOriginal );
//create the "Rotated Image" window and 3 trackbars in it

namedWindow( pzRotatedImage, CV_WINDOW_AUTOSIZE
);
createTrackbar("Angle", pzRotatedImage, &iAngle, 360,
CallbackForTrackBar);
createTrackbar("Scale", pzRotatedImage, &iScale, 100,
CallbackForTrackBar);
createTrackbar("Border Mode", pzRotatedImage,
&iBorderMode, 5, CallbackForTrackBar);
int iDummy = 0;
CallbackForTrackBar(iDummy, &iDummy);
waitKey(0);
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////
Rotated Image which is down-scaled and whose border values have

been extrapolated
All the OpenCV functions have been discussed previously.
Rotate a Video
Rotating a video is also simple. The code is just like the 1st
example in this lesson. Here is the OpenCV C++ code.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////
#include <iostream>
using namespace cv;


{
// open the video file for reading
VideoCapture
cap("C:/Users/SHERMAL/Desktop/MyVideo.mp4");
// if not success, exit program

if ( !cap.isOpened() )
{
return -1;
}
const char* pzOriginalWindowName = "Original Video";

namedWindow(pzOriginalWindowName,
const char* pzRotatingWindowName = "Rotated Video";

namedWindow( pzRotatingWindowName,
int iAngle = 180;

createTrackbar("Angle", pzRotatingWindowName,
&iAngle, 360);
while (true)
{
Mat matOriginalFrame;
// read a new frame from video
bool bSuccess = cap.read(matOriginalFrame);
//if not success, break loop

if (!bSuccess)
{
file" << endl;
break;
}
imshow(pzOriginalWindowName, matOriginalFrame);
//get the affine transformation matrix

Mat matRotation = getRotationMatrix2D(
Point(matOriginalFrame.cols / 2, matOriginalFrame.rows /
2), (iAngle - 180), 1 );
// Rotate the image

Mat matRotatedFrame;
warpAffine( matOriginalFrame, matRotatedFrame,
matRotation, matOriginalFrame.size() );
imshow( pzRotatingWindowName, matRotatedFrame );
//wait for 'esc' key press for 30 ms. If 'esc' key is pressed,
break loop
if (waitKey(30) == 27)
{
break;
}
}
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////
All the OpenCV functions, found in the above example have

been discussed earlier.
Color Detection & Object Tracking
Object detection and segmentation is the most important and

challenging fundamental task of computer vision. It is a critical part
in many applications such as image search, scene understanding, etc.
However it is still an open problem due to the variety and complexity
of object classes and backgrounds.
The easiest way to detect and segment an object from an image is the
color based methods . The object and the background should have a
significant color difference in order to successfully segment
objects using color based methods.
Simple Example of Detecting a Red Object
In this example, I am going to process a video with a red color object

and create a binary video by thresholding the red color. (Red color
area of the video is assigned to '1' and other area is assigned to '0' in
the binary image so that you will see a white patch wherever the red
object is in the original video)
/////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////
#include <iostream>
using namespace cv;


{
VideoCapture cap(0); //capture the video from web cam

{
cout << "Cannot open the web cam" << endl;
return -1;
}
namedWindow("Control",
CV_WINDOW_AUTOSIZE); //create a window called
"Control"
int iLowH = 0;
int iHighH = 179;
int iLowS = 0;
int iHighS = 255;
int iLowV = 0;
int iHighV = 255;
//Create trackbars in "Control" window

cvCreateTrackbar("LowH", "Control", &iLowH, 179); //Hue
(0 - 179)
cvCreateTrackbar("HighH", "Control", &iHighH, 179);
cvCreateTrackbar("LowS", "Control", &iLowS,
255); //Saturation (0 - 255)
cvCreateTrackbar("HighS", "Control", &iHighS, 255);
cvCreateTrackbar("LowV", "Control", &iLowV,

255); //Value (0 - 255)
cvCreateTrackbar("HighV", "Control", &iHighV, 255);
while (true)
{
Mat imgOriginal;
bool bSuccess = cap.read(imgOriginal); // read a new

frame from video

{
cout << "Cannot read a frame from video
stream" << endl;
break;
}
Mat imgHSV;
cvtColor(imgOriginal, imgHSV,
COLOR_BGR2HSV); //Convert the captured frame from
BGR to HSV
Mat imgThresholded;
inRange(imgHSV, Scalar(iLowH, iLowS, iLowV),

Scalar(iHighH, iHighS, iHighV),
imgThresholded); //Threshold the image
//morphological opening (remove small objects from the

foreground)
erode(imgThresholded, imgThresholded,
getStructuringElement(MORPH_ELLIPSE, Size(5, 5)) );
dilate( imgThresholded, imgThresholded,
//morphological closing (fill small holes in the foreground)

imshow("Thresholded Image", imgThresholded); //show

the thresholded image
imshow("Original", imgOriginal); //show the original
image
if (waitKey(30) == 27) //wait for 'esc' key press for

30ms. If 'esc' key is pressed, break loop
{
break;
}
}
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////
Explanation
OpenCV usually captures images and videos in 8-bit, unsigned

integer, BGR format. In other words, captured images can be
considered as 3 matrices; BLUE, GREEN and RED (hence the name
BGR) with integer values ranges from 0 to 255.
The following image shows how a color image is represented using 3

matrices.
How BGR image is formed
In the above image, each small box represents a pixel of the image. In
real images, these pixels are so small that human eye cannot
differentiate.
Usually, one can think that BGR color space is more suitable for color
based segmentation. But HSV color space is the most suitable color
space for color based image segmentation. So, in the above
application, I have converted the color space of original image of the
video from BGR to HSV image.
HSV color space is also consists of 3 matrices, HUE, SATURATION

and VALUE. In OpenCV, value range
for HUE, SATURATION and VALUE are respectively 0-179, 0-
255 and 0-255. HUE represents the color, SATURATION represents
the amount to which that respective color is mixed with white
and VALUE represents the amount to which that respective color is
mixed with black.
In the above application, I have considered that the red object

has HUE, SATURATION and VALUE in between 170-180, 160-255,
60-255 respectively. Here the HUE is unique for that specific
color distribution of that object.
But SATURATION and VALUE may be vary according to the
lighting condition of that environment.
Hue values of basic colors

o Orange 0-22
o Yellow 22- 38
o Green 38-75
o Blue 75-130
o Violet 130-160
o Red 160-179
These are approximate values. You have to find the exact range of
HUE values according to the color of the object. I found that the
range of 170-179 is perfect for the range of hue values of my
object. The SATURATION and VALUE is depend on the lighting
condition of the environment as well as the surface of the object.
How to find the exact range of HUE, SATURATION and VALUE for
a object is discussed later in this post.
After thresholding the image, you'll see small white isolated objects
here and there. It may be because of noises in the image or the actual
small objects which have the same color as our main object.
These unnecessary small white patches can be eliminated by
applying morphological opening. Morphological opening can be
achieved by a erosion, followed by the dilation with the same
structuring element.
Thresholded image may also have small white holes in the

main objects here and there. It may be because of noises in the
image. These unnecessary small holes in the main object can
be eliminated by
applying morphological closing. Morphological closing can
be achieved by a dilation, followed by the erosion with the
same structuring element.
Now let's discuss new OpenCV methods in the above application.

 void inRange(InputArray src, InputArray
lowerb, InputArray upperb, OutputArray dst);
Checks that each element of 'src' lies between 'lowerb' and 'upperb'. If
so, that respective location of 'dst' is assigned '255' , otherwise '0'.
(Pixels with value 255 is shown as white whereas pixels with value 0
is shown as black)
Arguments -
o InputArray src - Source image
o InputArray lowerb - Inclusive lower boundary
(If lowerb=Scalar(x, y, z), pixels which have values lower than x, y
and z for HUE, SATURATION and VALUE respectively is
considered as black pixels in dst image)
o InputArray upperb - Exclusive upper boundary (If it
is upperb=Scalar(x, y, z), pixels which have values greater or
equal than x, y and z for HUE, SATURATION and VALUE
respectively is considered as black pixels in dst image)
o OutputArray dst - Destination image (should have the
same size as the src image and should be 8-bit unsigned integer,
CV_8U)
 void erode( InputArray src, OutputArray dst,
InputArray kernel, Point anchor=Point(-1,-1), int
iterations=1, int borderType=BORDER_CONSTANT, const
Scalar& borderValue=morphologyDefaultBorderValue() )
This function erode the source image and stores the result in
the destination image. In-place processing is supported. (which
means you can use the same variable for the source and
destination image). If the source image is multi-channel, all
channels are processed independently and the result is stored
in the destination image as separate channels.
Arguments -
o OutputArray dst - Destination image (should have
the same size and type as the source image)
o InputArray kernel - Structuring element which is
used to erode the source image
o Point anchor - Position of the anchor within the
kernel. If it is Point(-1, -1), the center of the kernel is taken as
the position of anchor
o int iterations - Number of times erosion is applied
o int borderType - Pixel extrapolation method in a
boundary condition
o const Scalar& borderValue - Value of the pixels in
a boundary condition if borderType = BORDER_CONSTANT
 void dilate( InputArray src, OutputArray dst,

InputArray kernel,
 Point anchor=Point(-1,-1), int iterations=1,
 int borderType=BORDER_CONSTANT,
 const Scalar&
borderValue=morphologyDefaultBorderValue() );

This function dilate the source image and stores the result in
the destination image. In-place processing is supported. (which
means you can use the same variable for the source and
destination image). If the source image is multi-channel, all
channels are processed independently and the result is stored
in the destination image as separate channels.

the same size and the type as the source image)
o InputArray kernel - Structuring element which is
used to dilate the source image
o Point anchor - Position of the anchor within the
kernel. If it is Point(-1, -1), the center of the kernel is taken as
the position of anchor
o int iterations - Number of times dilation is applied
o int borderType - Pixel extrapolation method in a
boundary condition
o const Scalar& borderValue - Value of the pixels in
a boundary condition if borderType = BORDER_CONSTANT
o
 void cvtColor( InputArray src, OutputArray dst, int

code, int dstCn=0 )
This function convert a source image from one color space to
another. In-place processing is supported. (which means you
can use the same variable for the source and destination
image)
the same size and the depth as the source image)
o int code - Color space conversion code (e.g
- COLOR_BGR2HSV, COLOR_RGB2HSV, COLOR_BGR2GR
AY, COLOR_BGR2YCrCb, COLOR_BGR2BGRA, etc)
o int dstCn - Number of channels in the destination
image. If it is 0, number of channels is derived automatically
from the source image and the color conversion code.
All other OpenCV methods in the above application have been

discussed in early OpenCV tutorials.
Simple Example of Tracking Red objects
In the previous example, I showed you how to detect a color

object. In the following example, I'll show you how to track a
color object. There are 3 steps involving to achieve this task.
1. Detect the object
2. Find the exact position (x, y coordinates) of the object
3. Draw a line along the trajectory of the object
Here is how it is done with OpenCV / C++.
/////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////
#include <iostream>
using namespace cv;

{
VideoCapture cap(0); //capture the video from webcam

{
cout << "Cannot open the web cam" << endl;
return -1;
}
namedWindow("Control",
CV_WINDOW_AUTOSIZE); //create a window called "Control"
int iLowH = 170;

int iHighH = 179;
int iLowS = 150;

int iHighS = 255;
int iLowV = 60;

int iHighV = 255;
//Create trackbars in "Control" window

createTrackbar("LowH", "Control", &iLowH, 179); //Hue (0 -
179)
createTrackbar("HighH", "Control", &iHighH, 179);
createTrackbar("LowS", "Control", &iLowS, 255); //Saturation

(0 - 255)
createTrackbar("HighS", "Control", &iHighS, 255);
createTrackbar("LowV", "Control", &iLowV, 255);//Value (0 -

255)
createTrackbar("HighV", "Control", &iHighV, 255);
int iLastX = -1;
int iLastY = -1;
//Capture a temporary image from the camera

Mat imgTmp;
cap.read(imgTmp);
//Create a black image with the size as the camera output

Mat imgLines = Mat::zeros( imgTmp.size(), CV_8UC3 );;
while (true)
{
Mat imgOriginal;
bool bSuccess = cap.read(imgOriginal); // read a new frame

from video

{
cout << "Cannot read a frame from video stream" <<
endl;
break;
}
Mat imgHSV;
cvtColor(imgOriginal, imgHSV,
COLOR_BGR2HSV); //Convert the captured frame from BGR to
HSV
Mat imgThresholded;
inRange(imgHSV, Scalar(iLowH, iLowS, iLowV),
Scalar(iHighH, iHighS, iHighV), imgThresholded); //Threshold
the image
//morphological opening (removes small objects from the

foreground)
//morphological closing (removes small holes from the

foreground)
//Calculate the moments of the thresholded image

Moments oMoments = moments(imgThresholded);
double dM01 = oMoments.m01;

double dM10 = oMoments.m10;
double dArea = oMoments.m00;
// if the area <= 10000, I consider that the there are no object in
the image and it's because of the noise, the area is not zero
if (dArea > 10000)
{
//calculate the position of the ball
int posX = dM10 / dArea;
int posY = dM01 / dArea;
if (iLastX >= 0 && iLastY >= 0 && posX >= 0 && posY >= 0)
{
//Draw a red line from the previous point to the current point
line(imgLines, Point(posX, posY), Point(iLastX, iLastY),
Scalar(0,0,255), 2);
}
iLastX = posX;
iLastY = posY;
}
imshow("Thresholded Image", imgThresholded); //show the

thresholded image
imgOriginal = imgOriginal + imgLines;

imshow("Original", imgOriginal); //show the original image
if (waitKey(30) == 27) //wait for 'esc' key press for 30ms. If

{
break;
}
}
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////
Object Tracking
Explanation
In this application, I use moments to calculate the position of the

center of the object. We have to calculate 1st order spatial moments
around x-axis and y-axis and the 0th order central moments of the
binary image.
0th order central moments of the binary image is equal to the white
area of the image in pixels.
 X coordinate of the position of the center of the object = 1st

order spatial moment around x-axis / 0th order central moment
 Y coordinate of the position of the center of the object = 1st
order spatial moment around y-axis / 0th order central moment
If there are 2 or more objects in the image, we cannot use this method.
And noise of the binary image is also should be at minimum level to
get accurate results.
In the above application, I considered that if the white area of the

binary image is less than or equal to 10000 pixels, there are no objects
in the image because my object is expected to have an area more than
10000 pixels.
Now, let's discuss new OpenCV methods that can be found in the
above application.
 Moments moments( InputArray array, bool

binaryImage=false )
This OpenCV function calculates all of the spatial moments up
to the third order and returns a Moments object with the
results.
o InputArray array - Single channel image
o bool binaryImage - If this is true, all non zero pixels
are considered as ones when calculating moments.
 void line(Mat& img, Point pt1, Point pt2, const Scalar&

color, int thickness=1, int lineType=8, int shift=0)
This function draws a line between two points on a given image
o Mat& img - image which you want to draw the line
o Point pt1 - First point of the line segment
o Point pt2 - Other point of the line segment
o const Scalar& color - Color of the line (values of
Blue, Green and Red colors respectively)
o int thickness - Thickness of the line in pixels
 static MatExpr zeros(Size size, int type)

This function returns a black image (with pixels with zero
values) with a given size and type.
o Size size - Size of the required image ( Size(No of
columns, No of rows) )
o int type - Type of the image (e.g
- CV_8UC1, CV_32FC4, CV_8UC3, etc)
How to Find Exact Range for 'Hue', 'Saturation' and 'Value' for a
Given Object
Finding the optimum HUE, SATURATION and VALUE ranges for

an object is a 4 step process.
1. Track bars should be placed in a separate window so
that ranges for HUE, SATURATION and VALUE can be
adjusted. And set the initial ranges for HUE, SATURATION
and VALUE as 0-179, 0-255 and 0-255 respectively. So, we
will see a complete white image in the 'Control' window.
2. First, adjust 'LowH' and 'HighH' track bars so that the gap
between 'LowH' and 'HighH' is minimized. Here you have to be
careful that white area in 'Ball' window that represents the object
should not be affected, while you are trying to minimize the gap.
3. Repeat the step 2 for 'LowS' and 'HighS' trackbars
4. Repeat the step2 for 'LowV' and 'HighV' trackbars
Now you can find the optimum HUE, SATURATION and VALUE
ranges for the object. It is 163-179, 126-217 and 68-127 in my case as
you can see in the below picture.
Shape Detection & Tracking using Contours
In the previous tutorial, we could detect and track an
object using color separation. But we could not identify
the shape of the object there. In this tutorial, let's see
how to identify a shape and position of an object using
contours with OpenCV.
Using contours with OpenCV, you can get a sequence

of points of vertices of each white patch (White patches
are considered as polygons). As example, you will get 3
points (vertices) for a triangle, and 4 points
for quadrilaterals. So, you can identify any polygon by
the number of vertices of that polygon. You can even
identify features of polygons such as convexity,
concavity, equilateral and etc by calculating and
comparing distances between vertices.
Let's see how this can be done with OpenCV. All you
need, is a binary image in which your objects should be
white and the background should be black.
Now I am going to identify triangles and quadrilaterals
and heptagon in the above image using a
C++ application with OpenCV. I'll draw a line along
the perimeter of every identified polygon with colors
blue for triangle, green for quadrilaterals and red for
heptagons. Here is the code.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>
int main()
{
IplImage* img
= cvLoadImage("C:/Users/SHERMAL/Desktop/FindingCont
ours.png");

cvNamedWindow("Raw");
cvShowImage("Raw",img);
//converting the original image into grayscale

IplImage* imgGrayScale = cvCreateImage(cvGetSize(img),
8, 1);
cvCvtColor(img,imgGrayScale,CV_BGR2GRAY);
//thresholding the grayscale image to get better results
cvThreshold(imgGrayScale,imgGrayScale,128,255,CV_THR
ESH_BINARY);
CvSeq* contours; //hold the pointer to a contour in the

memory block
CvSeq* result; //hold sequence of points of a contour
CvMemStorage *storage =
cvCreateMemStorage(0); //storage area for all contours
//finding all contours in the image

cvFindContours(imgGrayScale, storage,
&contours, sizeof(CvContour), CV_RETR_LIST,
CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));
//iterating through each contour

while(contours)
{
//obtain a sequence of points of contour, pointed by the
variable 'contour'
result = cvApproxPoly(contours, sizeof(CvContour),
storage, CV_POLY_APPROX_DP,
cvContourPerimeter(contours)*0.02, 0);
//if there are 3 vertices in the contour(It should be a

triangle)
if(result->total==3 )
{
//iterating through each point
CvPoint *pt[3];
for(int i=0;i<3;i++){
pt[i] = (CvPoint*)cvGetSeqElem(result, i);
}
//drawing lines around the triangle

cvLine(img, *pt[0], *pt[1], cvScalar(255,0,0),4);

quadrilateral)
else if(result->total==4 )
{
CvPoint *pt[4];
}
//drawing lines around the quadrilateral

}

heptagon)
else if(result->total ==7 )
{
CvPoint *pt[7];
}
//drawing lines around the heptagon

}
//obtain the next contour

contours = contours->h_next;
}
//show the image in which identified shapes are marked

cvNamedWindow("Tracked");
cvShowImage("Tracked",img);
cvWaitKey(0); //wait for a key press
//cleaning up
cvDestroyAllWindows();
cvReleaseMemStorage(&storage);
cvReleaseImage(&imgGrayScale);
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////
As you can see, triangles are marked with

blue, quadrilaterals are marked with green and
heptagons are marked with red. So, now it is obvious
that this method is capable of identifying shapes.
Explanation
Here I have converted the original image in to gray
scale. It is because this method works only with gray
scale image with single channel. To get better results, I
threshold the gray-scale image using 'cvThreshold'
function. You can use your own way to threshold the
image. Then I find all contours in the thresholded
image and identify and track all triangles,
quadrilaterals and heptagons.
Let's discuss new OpenCV functions, found in this

application.
 cvThreshold( const Mat& src, Mat& dst,

double threshVal, double max, int thresholdType )
applies a fix level threshold to the each element of 'src'
array write a value to corresponding array element of
'dst'
Arguements -
o const Mat& src - Source array (This should be
single channel)
o Mat& dst - Destination array which has the
same size and same type as the 'src'
o double threshVal - Threshold value
o double max - Maximum value to use
with 'THRESH_BINARY' and 'THRESH_BINARY_INV'
which are thresholding types
o int thresholdType - You can use one of the
following for this arguement
 THRESH_BINARY
dst(x,y)=max, if src(x,y) >
ThreshVal
dst(x,y)=0, if src(x,y) <
ThreshVal
 THRESH_BINARY_INV
dst(x,y)=0, if src(x,y) >
ThreshVal
dst(x,y)=max, if src(x,y) <
ThreshVal
 THRESH_TOZERO
dst(x,y)=src(x,y), if src(x,y) > ThreshVal
dst(x,y)=0, if src(x,y) < ThreshVal
 THRESH_TOZERO_INV
dst(x,y)=0, if src(x,y) >
ThreshVal
dst(x,y)=src(x,y), if src(x,y) <
ThreshVal
 THRESH_TRUNC
dst(x,y)=threshVal, if src(x,y) >
ThreshVal
dst(x,y)=src(x,y), if src(x,y) <
ThreshVal
In the above application, I have used 'THRESH_BINARY',

because I want to assign 255 (white) where the objects
are located and 0 (black) elsewhere.
 cvCreateMemStorage(int byteSize)
Creates memory storage which has the capacity
specified by the parameter 'byteSize'. But if byteSize=0,
the allocated capacity is the default value(usually 64 Kb)
 cvFindContours( CvArr* img,

CvMemStorage* str, CvSeq** first_contour,
int header_size, int mode, int method,
CvPoint offset )
Find all contours in a binary image
Arguments -
o CvArr* img - Source image (This should be 8
bit single channel). All non-zero pixels are considered as
1 and all zero remain zero.
o CvMemStorage* str - Memory blocks to store
all obtained contours
o CvSeq** first_contour - store a pointer to the
first contour in the memory block, 'str'
o int header_size - size of the sequence header
o int mode - mode of retrieval of contours from
the image
You have to choose one of the following
 CV_RETR_LIST - Retrieves all of the
contours and put them in a list
 CV_RETR_EXTERNAL - Retrieves only
the extreme outer contours
 CV_RETR_CCOMP - Retrieves all of the
contours and organizes them into a two-level hierarchy:
on the top level are the external boundaries of the
components, on the second level are the boundaries of
the holes
 CV_RETR_TREE - Retrieves all of the
contours and reconstructs the full hierarchy of nested
contours
o int method - Approximation method

You have to choose one of the following
 CV_CHAIN_CODE - Outputs contours

in the Freeman chain code
 CV_CHAIN_APPROX_NONE -
Translates all of the points from the chain code into
points
 CV_CHAIN_APPROX_SIMPLE -
Compresses horizontal, vertical, and diagonal segments
and leaves only their end points
 CV_CHAIN_APPROX_TC89_L1,CV_CH
AIN_APPROX_TC89_KCOS - Applies one of the flavors of
the Teh-Chin chain approximation algorithm.
 CV_LINK_RUNS - uses a completely
different contour retrieval algorithm by linking horizontal
segments of 1’s. Only the 'CV_RETR_LIST' retrieval
mode can be used with this method.
o CvPoint offset - Offset by which every contour

point should be shifted. This is useful when we have set
ROI (Region Of Interest) in the image. Normally we set
the offset to 'cvPoint(0,0)'
 cvApproxPoly( const void* src,

int header_size, CvMemStorage* storage,
int method, double para1, int para2 )
Approximate polygonal curves with specified precision
arguments -
o const void* src - Sequence of points
o int header_size - size of the sequence header
o CvMemStorage* storage - memory block that
contains all contours
o int method - Approximation method. (The only
method, available to use for this argument is
'CV_POLY_APPROX_DP')
o double para1 - approximation accuracy
o int para2 - Determines whether the single
sequence should be approximated or all sequences in
the same level or below
 cvGetSeqElem( const CvSeq* seq, int index )

Returns a pointer to the element of 'seq' at 'index'

cvReleaseMemStorage( CvMemStorage** storage
)
Deallocate memory blocks which have been allocated
by 'cvCreateMemStorage()' function
Real World Example
The above example is not really useful

in practical situation. Usually, there are lots of noises in
an image such as irregular lighting, shadows, camera
irregularities and etc. So, above application as it is,
cannot be used to identify shapes in a real image. It
should be modified to cope with these noises. And
images usually have 3 channels (BGR color). So,
it should be converted into grey-scale which has only
one channel.
Here is a real world image of an arena of a robot

soccer, taken from a camera.
Here, we are going to detect and mark the perimeter of

each triangle in the image with a blue line. Let's see
the modified OpenCV c++ application which accomplish
the above task.
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>
int main()
{
IplImage* img
= cvLoadImage("C:/Users/SHERMAL/Desktop/DetectingCo
ntours.jpg");

cvNamedWindow("Original");
cvShowImage("Original",img);
//smooth the original image using Gaussian kernel to

remove noise
cvSmooth(img, img, CV_GAUSSIAN,3,3);
//converting the original image into grayscale

IplImage* imgGrayScale = cvCreateImage(cvGetSize(img),
8, 1);
cvCvtColor(img,imgGrayScale,CV_BGR2GRAY);
cvNamedWindow("GrayScale Image");
cvShowImage("GrayScale Image",imgGrayScale);
//thresholding the grayscale image to get better results
cvThreshold(imgGrayScale,imgGrayScale,100,255,CV_THR
ESH_BINARY_INV);
cvNamedWindow("Thresholded Image");
cvShowImage("Thresholded Image",imgGrayScale);
CvSeq* contour; //hold the pointer to a contour

CvSeq* result; //hold sequence of points of a contour
cvCreateMemStorage(0); //storage area for all contours

cvFindContours(imgGrayScale, storage,
&contour, sizeof(CvContour), CV_RETR_LIST,

while(contour)
{
//obtain a sequence of points of the countour, pointed by
the variable 'countour'
result = cvApproxPoly(contour, sizeof(CvContour),
cvContourPerimeter(contour)*0.02, 0);
//if there are 3 vertices in the contour and the area of the
triangle is more than 100 pixels
if(result->total==3 && fabs(cvContourArea(result,
CV_WHOLE_SEQ))>100 )
{
CvPoint *pt[3];
}
//drawing lines around the triangle


contour = contour->h_next;
}
//show the image in which identified shapes are marked

cvNamedWindow("Tracked");
cvShowImage("Tracked",img);
cvWaitKey(0); //wait for a key press
//cleaning up
return 0;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////
from here. (The downloaded file is a compressed .rar folder.
So, you have to extract it using Winrar or other suitable
software)
Gray scale Image

Thresholded Image
Triangles Detected
In the same way, any shapes with any sizes can be

detected with OpenCV.
Explanation
To reduce the noise level of the original image, I have

smoothed the original image with a Gaussian kernel.
Further you can change the
5th argument of cvApproxPoly() function to cope with
the noise. In the above example, I have
used cvContourPerimeter(contour)*0.02 as the 5th
argument of cvApproxPoly(). You can
try cvContourPerimeter(contour)*0.01 or cvContourPerime
ter(contour)*0.04 or any other value and see the
difference of the output yourselves.
Still there may be very small triangles, formed due to
the noise. Therefore all triangles with areas less than
100 pixels are filtered out.
Here are the new OpenCV functions, found in the above

example.
 cvContourArea(const CvArr* contour, CvSlice
slice)
Calculate the area enclosed by sequence of contour
points.
o const CvArr* contour - array of vertices of the
contour
o CvSlice slice - starting and ending point of the
contour. 'CV_WHOLE_SEQ' will take the whole contour
to calculate the area
The orientation of contour affects the area sign. So,
this function may return a negative value. So, it should
be used fabs() function to get the absolute value.
 fabs(double x)
This function returns the absolute value of any floating

point number. ( This is a C function, not a OpenCV
function)
Tracking two Triangles in a Video
Here I am going to track the two triangles in a video.

The blue triangle is marked with red and the green
triangle is marked with blue.
/////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>
IplImage* imgTracking=0;
int lastX1 = -1;

int lastY1 = -1;
int lastX2 = -1;

int lastY2 = -1;
void trackObject(IplImage* imgThresh){

CvSeq* contour; //hold the pointer to a
contour
CvSeq* result; //hold sequence of points
of a contour
cvCreateMemStorage(0); //storage area for all
contours

cvFindContours(imgThresh, storage,
&contour, sizeof(CvContour), CV_RETR_LIST,

while(contour)
{
//obtain a sequence of points of the
countour, pointed by the variable 'countour'
result =
cvApproxPoly(contour, sizeof(CvContour),
cvContourPerimeter(contour)*0.02, 0);
//if there are 3 vertices in the contour and

the area of the triangle is more than 100 pixels
if(result->total==3 &&
fabs(cvContourArea(result,
CV_WHOLE_SEQ))>100 )
{
CvPoint *pt[3];
pt[i] =
(CvPoint*)cvGetSeqElem(result, i);
}
int posX=( pt[0]->x + pt[1]->x + pt[2]-

>x )/3;
int posY=( pt[0]->y + pt[1]->y + pt[2]-
>y )/3;
if(posX > 360 ){

if(lastX1>=0 && lastY1>=0 &&
posX>=0 && posY>=0){
// Draw a red line from the
previous point to the current point
cvLine(imgTracking, cvPoint(posX,
posY), cvPoint(lastX1, lastY1), cvScalar(0,0,255),
4);
}
lastX1 = posX;
lastY1 = posY;
}
else{
if(lastX2>=0 && lastY2>=0 &&
posX>=0 && posY>=0){
// Draw a blue line from the
previous point to the current point
cvLine(imgTracking, cvPoint(posX,
posY), cvPoint(lastX2, lastY2), cvScalar(255,0,0),
4);
}
lastX2 = posX;
lastY2 = posY;
}
}

contour = contour->h_next;
}
}
int main(){
//load the video file to the memory
CvCapture *capture
= cvCaptureFromAVI("E:/Projects/Robot/IESL
Robot/robot/a.avi");
if(!capture){
printf("Capture failure\n");
return -1;
}
IplImage* frame=0;
frame = cvQueryFrame(capture);
if(!frame) return -1;
//create a blank image and assigned to

'imgTracking' which has the same size of original
video
imgTracking=cvCreateImage(cvGetSize(frame)
,IPL_DEPTH_8U, 3);
cvZero(imgTracking); //covert the image,
'imgTracking' to black
cvNamedWindow("Video");
//iterate through each frames of the video

while(true){
frame = cvQueryFrame(capture);
if(!frame) break;
frame=cvCloneImage(frame);
//smooth the original image using Gaussian

kernel
cvSmooth(frame, frame,
CV_GAUSSIAN,3,3);
//converting the original image into
grayscale
IplImage* imgGrayScale =
cvCreateImage(cvGetSize(frame), 8, 1);
cvCvtColor(frame,imgGrayScale,CV_BGR2GRAY);
//thresholding the grayscale image to get

better results
cvThreshold(imgGrayScale,imgGrayScale,100,255
,CV_THRESH_BINARY_INV);
//track the possition of the ball

trackObject(imgGrayScale);
// Add the tracking image and the frame

cvAdd(frame, imgTracking, frame);
cvShowImage("Video", frame);
//Clean up used images

cvReleaseImage(&frame);
//Wait 10mS
int c = cvWaitKey(10);
//If 'ESC' is pressed, break the loop
if((char)c==27 ) break;
}
cvReleaseImage(&imgTracking);
cvReleaseCapture(&capture);
return 0;
}
/////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////
from here. (The downloaded file is a compressed .rar folder.
So, you have to extract it using Winrar or other suitable
software)
Explanation
You already know how to obtain 3 vertices of a triangle

with OpenCV. Averaging those 3 vertices gives you the
center point of the triangle. So, it is easy to track
triangles in a video.
Then, how do you identify two similar
triangles separately? Here I have used a simple trick. I
know that the green triangle always is in the left side of
the video and the blue triangle is in the right side of
the video. So, if the x coordinate of a triangle is more
than (frame width)/2 = 360, then it is the blue triangle,
otherwise it is the green triangle.

11 09 OpenCV Tutorial

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

11 09 OpenCV Tutorial

Uploaded by

Copyright:

Available Formats

OpenCV, MATLAB or AForge ??

Image processing is the process of manipulating image data in

So, I think that it is worthy to learn computer vision with OpenCV.

OpenCV is an open source C++ library for image processing and

OpenCV has a modular structure. The main modules of OpenCV are

OpenCV is now extensively used for developing advanced image

 append the cv:: specifier at the beginning of every OpenCV

Data Types for Arrays

For single channel arrays :

Single Channel Array

For multi channel arrays :

e.g. 2 : Here I have illustrated a 2 channel array with 8 bit

o Mat img1(3, 5, CV_32F ); //3 x 5 single-channel array

Bit Depths for IplImage (C style)

Read Image from File and Display

Here I am going to explain how to read an image from a file and

using namespace cv;

int main( int argc, const char** argv )

if (img.empty()) //check whether the image is loaded or

waitKey(0); //wait infinite time for a keypress

destroyWindow("MyWindow"); //destroy the window

Load & Diplay Image from File

Let's review the above OpenCV code line by line.

We are using Mat data structure in the above program. It is

 using namespace cv;

'#include <iostream>' and 'using namespace std' are added

 Mat img = imread(const string& filename,

Arguments of imread() function

You can combine these above parameters to get desired image

If you are not sure what to do, use CV_LOAD_IMAGE_COLOR

To understand image-depth and concept of channels, you

Now what is image-depth? The image-depth means the

Here is a simple model of a image with image-depth of 8 bits.

Here is some properties of the following image.

Following image is a simple model of a color image. Color

R, G, B planes of a color image

 void imshow(const string& winname, InputArray mat);

 int waitKey(int delay = 0)

When running this program, the image of 'MyPic.JPG' is loaded into

Create a Blank Image & Display

using namespace cv;

int main( int argc, const char** argv )

if (img.empty()) //check whether the image is loaded or

waitKey(0); //wait infinite time for a keypress

destroyWindow("MyWindow"); //destroy the window

Create & Display Image

New OpenCV functions

In this lesson, I am going to show you how to read a video file.

using namespace cv;

int main(int argc, char* argv[])

if ( !cap.isOpened() ) // if not success, exit program

//cap.set(CV_CAP_PROP_POS_MSEC, 300); //start the

double fps = cap.get(CV_CAP_PROP_FPS); //get the

bool bSuccess = cap.read(frame); // read a new frame

if (!bSuccess) //if not success, break loop

imshow("MyVideo", frame); //show the frame in

if(waitKey(30) == 27) //wait for 'esc' key press for 30

All other OpenCV functions and their arguments are exactly

Write Image to File

using namespace cv;

int main( int argc, const char** argv )

if (img.empty()) //check whether the image is loaded or

bool bSuccess = imwrite("D:/TestImage.jpg", img,

cout << "ERROR : Failed to save the image" << endl;