Introduction to Machine vision

Machine vision in robotics is nothing but, giving an eye

(generally in the form of a camera) to the robot.

Its main application area is in the industries to describe, the

understanding and interpretation of technically obtained images for
controlling production processes.

It is also mostly & very widely used in the field of medical &
space research works.

SENSORS
Analogous to human sensory organs: Eyes,
ears, nose, tongue, skin
Help robot to know its surroundings better
Improves its actions and decision making
ability
Provides feedback control.

SENSORS USED IN MACHINE VISION

Generally the optical/light sensors are used in machine vision
system.
From the list of the optical/light sensors, camera is widely used as
a sensor in machine vision system.

CAMERA VISION

CAMERA VISION CONT.

TYPES OF CAMERA VISION USED IN ROBOTICS:
Onboard camera vision

Overhead camera vision

DIFFERENT TYPES OF CAMERAS

CCD
Higher quality images with little noise
Typically more pixels
Higher sensitivity to light

CMOS
Lower cost
Easier to build
Lower power consumption

BASIC CONSTRUCTION & WORKING OF A CAMERA

The image sensor employed by most digital cameras

is a charge coupled device (CCD). Some cameras use
complementary metal oxide semiconductor (CMOS)
technology instead.
Both CCD and CMOS image sensors convert light into
electrons.

BASIC CONSTRUCTION & WORKING OF A CAMERA

Serial - The earliest digital cameras had a serial interface, but no current
cameras use this since it is so slow.

USB 1.1 - USB was the first widespread high speed method of data transfer
from cameras. It is theoretically capable of transfer speeds up to 11
megabits/second (note megabits not megabytes)
USB 2.0 - A development of USB but much faster - up to 480
megabits/second.
IEEE 1394 (Firewire) - Though this is an older interface than USB, it was
originally only really used much on Apple computers. It's capable of high
speed transfer (400 megabits/second) and it's now found on some PCs or
it can be added to them via a plug-in card. More common on digital video
cameras than still digital cameras.

ADVANTAGES OF CAMERA VISION

It is a combination of multiple sensors.
Much wider view angle.
Can be employed from a far distance to
keep/record the information.

IMAGE PROCESSING TOOLS

EXPLORING MATLAB
MATLAB is a program for doing numerical
computation. It was originally designed for
solving linear algebra type problems using
matrices.
Its name is derived from MATrix LABoratory.
MATLAB is an interpreter -> not as fast as
compilers.
File extension for MATLAB is .m

INTRODUCTION TO MATLAB

INTRODUCTION TO COMMANDS & FUNCTIONS

Type the commands by terminating with a semicolon(;) in the
command window & see the o/p or result on the same
window.
MATLAB is case sensitive i.e. b and B are different variables.
% sign in the beginning of any sentence is used to give
comment line. Multiline comment is not possible.

To get help (when keyword is known) type help XXX (where

XXX is the keyword).
Type lookfor (when keyword is unknown).)

INTRODUCTION TO COMMANDS & FUNCTIONS CONT.

VARIABLES:
Variable names can contain up to 63 characters
Variable names must start with a letter followed by letters, digits,
and underscores.
Variable names are case sensitive
Special variables: ans
Default variable name for results
pi
Value of
eps
Smallest incremental number
inf
Infinity
NaN
Not a number e.g. 0/0
Realmin
The smallest usable positive real number
realmax
The largest usable positive real number
Disp
displays the contents of a variable

INTRODUCTION TO COMMANDS & FUNCTIONS CONT.

FUNCTIONS:

round(X): round to nearest integer, trailing 5 rounds to the nearest

integer away from zero. For example, round(2.5) returns 3; round(-2.5)
returns -3.

fix(X): round to nearest integer toward zero (truncate). For example,

fix(2.7) returns 2; fix(-2.7) returns -2

floor(X): round to the nearest integer toward minus infinity (round to

the nearest integer less than or equal to X). For example, floor(2.7) returns
2; floor(-2.3) returns -3.

ceil(X): round to the nearest integer toward positive infinity (round to

the nearest integer greater than or equal to X); for example, ceil(2.3)
returns 3; ceil(-2.7) returns -2

CONDITIONAL STATEMENTS & LOOPING

if..elseif..else..end

CONDITIONAL STATEMENTS & LOOPING CONT.

for..end

CONDITIONAL STATEMENTS & LOOPING CONT.

while..end

MATHEMATICAL OPERATIONS

MATHEMATICAL OPERATIONS CONT.

MATHEMATICAL OPERATIONS CONT.

LOGICAL & CONDITIONAL OPERATIONS

& : and
| : or
~ : not
< : less than
<= : less than or equal to
> : greater than
>= : greater than or equal to
== : equal
~= : not equal

SIGNAL REPRESENTATION

MATLAB has two inbuilt functions to represent a

signal on the display:
PLOT function is used for continuous domain functions.
For discrete domain functions, you should use STEM
function.

EXAMPLE:

> n=-5:0.5:5;
> x(n)=exp(j*(0.4*pi*n-0.5*pi));
> y(n)=x(n)-x(n-1);
> stem(y,n)

SIGNAL REPRESENTATION CONT.

Signal representation example:

fs=100;
t=0:1/fs:1;
f=5;
x=sin(2*pi*f*t);
subplot(211);plot(t,x);
title('Continuous-time');
subplot(212);stem(t,x);
title('Discrete-time');

% sampling frequency
% setup time from 0 to 1 second
% input frequency
% input signal
% plot the signal
% plot the signal using stem

ARRAY OPERATION
The Matlab array arithmetic operations are:
addition (+)

subtraction (-)

array multiplication (.*)

array power (.^)

array division (./)

These operations act element-wise on the arrays, for example if A is an n by m matrix

and B is an p by q matrix then A.*B is defined only if n=p and m=q, and the (i,j)
element of A.*B is the (i,j) element of A multiplied by the (i,j) element of B.
Example:

>> A = [ 1 2; 3 4], B = [5 6; 7 8]
>> A.*B
>> A.^2
>> A./B

ARRAY OPERATION CONT.

>> size(A); The result gives the size of A, i.e., the number of
rows and columns.
>> sum(A) ; This function calculates the column sums in a
rectangular array A and gives the sums in a row vector. (A column
sum is the sum of the elements of a column.) If A is a row vector,
then the sum of the elements of A is calculated. The function
>> prod(A); does the same for the product.
Other Array operations are:

transpose()
> greater than
>= greater than or equal to
<= lower than or equal to

< lower than

== equal to
~= not equal to

STRING OPERATION
A string matrix is like any other, except the
elements in it are interpreted as ASCII
numbers.
To create a string variable, enclose a string of
characters in single quotation marks
Ex: stg = 'This is a string';
x = ['ab' ; 'cd']
x = ['ab' 'cd']

MATRIX OPERATION
Matlab treats all variables as matrices. For our purposes a
matrix can be thought of as an array, in fact, that is how it is
stored.
Vectors are special forms of matrices and contain only one
row OR one column.
Scalars are matrices with only one row AND one column.
A matrix with only one row is called a row vector. A row
vector can be created in Matlab as follows (note the
commas):
rowvec = [12 , 14 , 63]
rowvec =
12 14 63

MATRIX OPERATION CONT.

A matrix with only one column is called a column vector.
A column vector can be created in MATLAB as follows
(note the semicolons):
colvec = [13 ; 45 ; -2]
colvec =
13
45
-2

MATRIX OPERATION CONT.

A matrix can be created in Matlab as follows (note the
commas AND semicolons):
matrix = [1 , 2 , 3 ; 4 , 5 ,6 ; 7 , 8 , 9]
matrix =
1 2 3
4 5 6
7 8 9

MATRIX OPERATION CONT.

EXTRACTING A SUB-MATRIX:
A portion of a matrix can be extracted and stored in a
smaller matrix by specifying the names of both matrices
and the rows and columns to extract. The syntax is:
sub_matrix = matrix ( r1 : r2 , c1 : c2 ) ;
where r1 and r2 specify the beginning and ending
rows and c1 and c2 specify the beginning and
ending columns to be extracted to make the new
matrix.

MATRIX OPERATION CONT.

EXTRACTING A SUB-MATRIX:

MATRIX OPERATION CONT.

Accessing Single Elements of a Matrix

A(i,j)
Accessing Multiple Elements of a Matrix

A(1,4) + A(2,4) + A(3,4) + A(4,4)

sum(A(1:4,4)) or

sum(A(:,end))

The keyword end refers to the last row or column.

Deleting Rows and Columns

to delete the second column of X, use

X(:,2) = []
Concatenating Matrices A and B

C=[A;B]

MATRIX OPERATION CONT.

Colon Operator

MATRIX OPERATION CONT.

MATRIX FUNCTIONS:
X = ones(r,c)
% Creates matrix full with ones
X = zeros(r,c)
% Creates matrix full with zeros
A = diag(x)
% Creates squared matrix with
vector x in diagonal
[r,c] = size(A)
% Return dimensions of matrix A
+-*/
% Standard operations
.+ .- .* ./
% Wise addition, substraction,
v = sum(A)
% Vector with sum of columns

MATRIX OPERATION CONT.

MATRIX FUNCTIONS:

X = A
X = inv(A)
d = det(A)
[X,D] = eig(A)

% Transposed matrix
% Inverse matrix squared matrix
% Determinant
% Eigenvalues and eigenvectors

MATRIX OPERATION CONT.

MATRIX FUNCTIONS:

ones
ones matrix
zeros
zeros matrix
size
size of a matrix
diag
diagonal elements of a matrix
inv
matrix inverse
rand
uniformly distributed random numbers
randn
normally distributed random numbers
cumprod
cumulative product of elements

max
largest component
min
smallest component
sum
sum of elements
mean
average or mean value
median
median value
std
standard deviation
sort
sort in ascending order
find
find indices of nonzero entries
corrcoef
correlation coefficients
cov
covariance matrix

M- FILES & FUNCTIONS IN MATLAB

Text files containing MATLAB programs. Can be called form
the command line or from other M-files.
Present .m extension.
Two kind of M-files:
The main difference between a script and a function is
1. Scripts
that a function accepts input from and returns output
to its caller, whereas scripts do not.
2. Functions
A function file starts with a line declaring the function, its arguments
and its outputs. There follow the statements required to produce the
outputs from the inputs (arguments). That's it.
Here is a simple example:
function y = port_val(holdings,prices) y =
holdings*prices;
Note that your function file name should be same as your function
name with .m extension.

FAMILIARIZATION WITH IMAGE PROCESSING

TERMINOLOGIES
Definitions of an image:
It is a combination of pixel values.
0 - means Black & 1- means white.
Mathematically it is a 2-D function.

FAMILIARIZATION WITH IMAGE PROCESSING

TERMINOLOGIES
Dizitization
capturing an analog signal in digital form.
Sampling
converting a continuous signal into a
discrete signal.
Quantization
rounding process of samples to a fixed set of numbers (such as
integers).

TYPES OF IMAGE
Monochrome
An image displayed in a single color or shades of a single color.
Least memory requirement
Also known as 1-bit image, each pixel is stored as a bit.

Gray Scale
A grayscale (or graylevel) image is simply one in which the only colors are shades of gray.
Processing is faster & easier.
Also known as 8-bit image, each pixel is usually stored as a byte (value between 0 to 255) .

Colored
A (digital) color image is a digital image that includes color information for each pixel.
It is the combination of the 3- primary colors (RGB).
Also known as 24- bit image, each pixel contains 3 values i.e. R, G & B-value.

DIFFERENT PARAMETERS OF AN IMAGE

PIXEL
Short for Picture Element,
A single point in a graphic image.

RESOLUTION
900 pixels 300ppi
= 3 inches wide
600 pixels 300ppi
= 2 inches high

TEXTURE
The feel of a surface or a fabric
Texture mapping is a method for adding details
or color to a computer generated graphic or 3-D model.

DIFFERENT CONCEPTS

Luminance
The amount of brightness.

Chrominance

Same chrominance ,
different luminance

The color information.

Hue
The property of colors by which they can be perceived as ranging
from red through yellow, green, and blue, as determined by the
dominant wavelength of the light.

Same luminance,
different chrominance

Intensity/Saturation
The strength of a color, especially the degree to which it lacks its complementary color.

DIFFERENT COLOUR SPACES

Color Space:
RGB - Red Green Blue
CMYK - Cyan Magenta Yellow Black
HSV - Hue Saturation Value

YIQ - Luminance Hue Saturation

YCbCr - Luminance and Chrominance

DIFFERENT COLOUR SPACES CONT.

RGB Color Space:

Additive Model
Three primary Colors
Design Simplification
Computer Graphics

DIFFERENT COLOUR SPACES CONT.

CMYK Color Space:
Subtractive Model

Printing applications

DIFFERENT COLOUR SPACES CONT.

YIQ Color Space:

Y luminance
I Hue (Orange Blue range)
Q Saturation (Purple Green range)

Grayscale information is separated from color data

Histogram Equalization (can be applied on Y)

Defined by NTSC

Used in United States Televisions

DIFFERENT COLOUR SPACES CONT.

YCbCr Color Space:

Y

- luminance

Cb & Cr

- Chrominance Information

Widely Used in Digital Video

Cb

Cr

HANDLING IMAGES IN MATLAB

There are two possible ways of handling image:

Using the Image Acquisition Tool box
It is faster & flexible
Available on MATLAB7.0 and above
Example: >>imaqhwinfo
>> vid = videoinput(winvideo, 1);
>> preview(vid);
Using VFM (Video for MATLAB)
Add-on for all versions of MATLAB
Once started you have to stop it along with MATLAB
First of all you have to keep the vfm.m & vfm.dll file in your
current directory (work), otherwise it wont be work.

IMPORTING IMAGE
There are two possible ways of importing image:
Using webcam with Image Acquisition Tool box
Example:

>> vid = videoinput(winvideo, 1);

>> image=getsnapshot(vid);
>> imshow(image);

Using webcam with VFM (Video for MATLAB)

>> image=vfm(grab);
>> imshow(image);
Using the stored image from the hard disk
>> image=imread(ball.jpg);
>> imshow (image);
Example:

N.B.: Before using this you have to store ball.jpg in the current directory.

IMPORTING IMAGE CONT.

Graphical Representation of Importing an Image

IMAGE REPRESENTATION
You can store/save an image by using imwrite();

An image is stored as a matrix using standard Matlab matrix

conventions.
There are five basic types of images supported by Matlab:

Grayscale
True color RGB
Indexed
Binary
8-bit images/Unit8

IMAGE REPRESENTATION CONT.

Functions for conversion between different color spaces:

FINDING PIXEL VALUE

Use pixval to find out the information about pixels.
Use the command imview(xyz.jpg) & move the cursor over
the image which will show you the pixel value.
Use the following commands:
>> i=imread('balls.bmp');
>> p=impixel(i);
>> disp(p);
Write click on image

OPERATIONS USING PIXEL VALUE

The value of a single pixel in an image is indexed as:
p(m,n)
Multiple pixels can be indexed with a vector as:
p([1 3],1)
which selects pixels p(1,1) and p(3,1).
Multiple pixels also can be indexed using a range. The first
row is all pixels with row index 1 and any column index:
p(1,1:N)
The notation 1:N indicates all indexes from 1 to N.

The notation can contain a step value, e.g., 1:2:5 indicates

indexes 1, 3, 5.

OPERATIONS USING PIXEL VALUE CONT.

To change the color value of a pixel use:
>> y=imread('balls.bmp');
>> y(100,155)=5;
>> y(100,154)=5;
>> imview(y);

Filling an ROI
Filling is a process that fills a region of interest
(ROI) by interpolating the pixel values from the
borders of the region.
>> I = imread('ballgray.bmp');
>> imshow(I);
>> I2 = roifill;
>> imshow(I2);

IMAGE MATRIX
Any image (supported format) can be stored in MATLAB in
the form of a matrix.
To show how an image is being stored in a matrix:
>> I = imread('balls.bmp');
>> size(I)
>> I(1:10,1:10)

To resize the image, rotate to 45 & crop to original size:

>> I = imread('balls.bmp');
>> I2 = imresize(I, 0.5, 'bil');
>> imshow(I2);
>> I3 = imrotate(I2, 45,'bil', 'crop');
>> imshow(I3);

IMAGE PROCESSING APPROACHES

Image Processing generally involves extraction of useful
information from an image.
The basic block diagram of image processing system:
Database of images

Pre-processing
Decision
Making

Image acquisition

Feature extraction

Data
communication

FUNCTIONS TO ENHANCE IMAGES

imadjust - Adjust image intensity values
>> RGB1 = imread('balls.bmp');
>> RGB2 = imadjust(RGB1,[.2 .3 0; .6 .7 1],[]);
>> imshow(RGB1), figure, imshow(RGB2)

histeq - Enhance contrast using histogram equalization

>> I = imread('ballgray.bmp');
>> J = histeq(I);
>> imshow(I)
>> figure, imshow(J)
Some other functions:

imnoise Add noise to an image.

medfilt2 Perform 2D median filtering.
ordfilt2 Perform 2D orderstatistic filtering.
stretchlim Find limits to contrast stretch an image.
wiener2 Perform 2D adaptive noiseremoval filtering.

FUNCTIONS TO ENHANCE IMAGES CONT.

Filtering the image:
Filtering is a technique for modifying or enhancing an image.
Image processing operations implemented with filtering include
smoothing, sharpening, and edge enhancement.
>> I = imread('balls.bmp');
>> h = fspecial('unsharp');
>> I2 = imfilter(I,h);
>> imshow(I), title('Original Image')
>> figure, imshow(I2),title(Filtered Image)

Other Filters are:

average
'disk
'gaussian
'laplacian
'unsharp

'log
'motion
'prewitt
'sobel

THRESHOLDING
Thresholding is a non-linear operation that converts a gray-scale image
into a binary image where the two levels are assigned to pixels that are
below or above the specified threshold value.
You can apply a threshold to data directly from the command line,
e.g., BW = im2bw(I, level)
Automatically calculate a threshold value using an iterative method. >>
>>x=imread('balls.bmp');
>> x=imread('balls.bmp');
>> BW = im2bw(x,0.4);
>> Y = zeros(194,250);
>> imshow(x), figure, imshow(BW)
>> for(i=1:194)
>>for(j=1:250)
>>if(x(i,j) > 50)
>>Y(i,j) = 1;
>>end
>>end
>>end
>> imshow(Y);

IMAGE ARITHMATIC

NOISE & NOISE REDUCTION

Where does noise come from?

Scanner resolution
Film grain (granularity)
Hardware (interference patterns)
Other

Common type of noise:

Whitenoise (Gaussian)
Local variance (Gaussian with intensity-dependent
variance)
Salt and pepper
Speckle

NOISE & NOISE REDUCTION CONT.

Modeling of noise:
The function imnoise allows different types of noise
to be modeled.
Syntax:
J = imnoise(I,type,parameters)
I Image
type gaussian, localvar, poisson, salt & pepper, speckle
parameters additional parameters needed
given the type of noise

NOISE & NOISE REDUCTION CONT.

The Image Processing Toolbox provides three main methods
of filtering:
Linear filtering
A linear filter computes each output pixel value according to a linear combination of
the input pixel's neighborhood. The basics of linear filtering are done through correlation
and convolution.

Median filtering
When noise causes the pixels to vary greatly from the original value (salt and
pepper), a median filter is more effective in reducing the noise.

Adaptive filtering
This type of filter is effective in reducing the effects of Gaussian whitenoise.

NOISE & NOISE REDUCTION CONT.

Program to add & remove noise in an image:
Adding noise to an image:
>> I = imread('ballgray.bmp');
J = imnoise(I,'salt & pepper',0.05);
figure, imshow(I);
figure, imshow(J);

Removing noise from an image:

>> I = imread(noisyball.bmp');
B = medfilt2(J,[2 3]);
figure, imshow(I);
figure, imshow(B);

EDGE DETECTION
Edge detection is nothing but a filtering process.
The edge function of the Image Processing Toolbox makes it easy to
perform edge detection.
Followings are the different edge detection functions:
sobel
prewitt

roberts
log (laplacian)

Canny (The perfect one)

zerocross

EDGE DETECTION CONT.

Edge information in an image is found by looking at the relationship a pixel
has with its neighborhoods.
If a pixels gray-level value is similar to those around it, there is probably
not an edge at that point.
If a pixels has neighbors with widely varying gray levels, it may present an
edge point.
Many are implemented with convolution mask and based on discrete
approximations to differential operators.

Roberts Mask

Prewitts Mask

Sobel Mask

HARDWARE DETAILS
CMOS webcam:

Video Format : 24bit RGB

30 Frames/sec
Focus Range : 3 centimeters To Limitless

DATA COMMUNICATION USING MATLAB

MATLAB provides support to access serial port (also called as COM port)
and parallel port (also called as printer port or LPT port) of a PC.
Serial port:
If you have to transmit one byte of data, the serial port will transmit 8 bits as one bit
at a time. The advantage is that a serial port needs only one wire to transmit the 8 bits
(while a parallel port needs 8).

Parallel port:
Parallel port has 25 pins Pins 2-9 are bi-directional data pins (pin 9 gives the most
significant bit (MSB)), pins 10-13 and 15 are output pins (status pins), pins
1,14,16,17 are input pins (control pins), while pins 18-25 are Ground pins.

ACCESSING THE PCs SERIAL PORT

MATLAB code for accessing the serial port is as
follows:
>> ser= serial('COM1','BaudRate',9600,'DataBits',8);
>> fopen(ser);
To send data through the serial port, the available
commands
>> fwrite (ser,1); % for left motion
>> fwrite (ser,2); % for right motion
You can close the port in case there are other
applications using this port using the fclose command.
>> fclose(ser);

ACCESSING THE PCs PARALLEL PORT

Parallel port interfacing:

To access the parallel port in MATLAB, define an
object:
>> parport= digitalio('parallel','LPT1');
You may obtain the port address using,
>> get(parport,'PortAddress')
>> daqhwinfo('parallel'); % To get data acquisition
hardware information

ACCESSING THE PCs PARALLEL PORT CONT.

You have to define the pins 2-9 as output pins, by
using addline function
>> addline(parport, 0:7, 'out')
Now put the data which you want to output to the
parallel port into a matrix; e.g.
>> dataout = logical([1 0 1 0 1 0 1 1]);
Now to output these values, use the putvalue function
>> putvalue(parport,dataout);
Alternatively, you can write the decimal equivalent of
the binary data and output it.
>> data = 23;

DEVELOPMENT OF IMAGE PROCESSING ROBOTS

Amount of data to be processed:

If we have each frame to be of size 320x240 pixels for an RGB image the
amount of data produced by each bit will be,
Each color (R,G,B) will be represented by 8 bits.
Hence, each pixel will be of 8x3=24 bits
Now, total number of pixels are 320x240=76,800 pixels
Hence, each frame will be of 76800x24=18,43,200 bits
Hence, in terms of bytes the volume of data generated will be
18,43,200/8=230KB

If we need to capture around 15 frames per second, the amount of data

that will be generated per second will be approximately 4MB.

DEVELOPMENT OF IMAGE PROCESSING ROBOTS CONT.

Why conventional microcontrollers cant be

used for image processing?
Limited throughput
Low clock frequency (up to around 32Mhz)
Lack of powerful Instruction set required for
handling arrays
Smaller word length handling capability
Limited on-chip memory (RAM)

DEVELOPMENT OF IMAGE PROCESSING ROBOTS CONT.

Analyse the problem statement:

Overhead
Onboard

Finalize the hardware required:

Processor for image processing & navigation

Camera/Image acquisition device
Machine with motors and motor drivers
Processor to/from machine communication

COLOR DETECTION ROBOT

Capture the image.

Select which colour to detect.
Set the threshold value accordingly, thats it.

How to set the threshold value?

Capture few live images of the ball
Using MATLAB (imview) analyze the pixel values
Take a mean value
Observe variation in pixel values with changing light conditions
Light conditions keep on varying with every captured frame
Try using a color space where color information (chrominance) and light
(luminance) can be separated.
So which color space to use?

COLOR DETECTION ROBOT CONT.

MATLAB code:
clc
clear
while(1)
RGB=vfm('grab');
mode=input('Enter which color to detect: 1-Red, 2-Green, 3-Blue: ');
thres=input('Enter the threshold value: ');
[M,N,t] = size(RGB);
switch (mode)
case 1
I1 = zeros(M,N); I2 = zeros(M,N);
I1( find(RGB(:,:,1) > thres * RGB(:,:,2)) ) = 1;
I2( find(RGB(:,:,1) > thres * RGB(:,:,3)) ) = 1;
strTitle = 'Color RED detected (white areas)';
I = I1 .* I2;

CONT.

subplot(2,1,1),imshow(RGB); title('Original Image');

subplot(2,1,2),imshow(I,[]); title(strTitle);
end

COLOR DETECTION ROBOT

case 2
I1 = zeros(M,N); I2 = zeros(M,N);
I1( find(RGB(:,:,2) > thres * RGB(:,:,1)) ) = 1;
I2( find(RGB(:,:,2) > thres * RGB(:,:,3)) ) = 1;
strTitle = 'Color GREEN detected (white areas)';
I = I1 .* I2;
case 3
I1 = zeros(M,N); I2 = zeros(M,N);
I1( find(RGB(:,:,3) > thres * RGB(:,:,1)) ) = 1;
I2( find(RGB(:,:,3) > thres * RGB(:,:,2)) ) = 1;
strTitle = 'Color BLUE detected (white areas)';
I = I1 .* I2;
otherwise
fprintf('Enter a valid choice for color'\n);
return;
end

BALL FOLLOWER ROBOT

Place the ball in such a distance that centre of

ball and centre of frame should coincide.
According to the color of the ball set the
threshold value which will separate the
background from the ball.

Set the values for forward/backward movement

through parallel port communication.

BALL FOLLOWER ROBOT CONT.

Ideal position of the ball

Mark all pink pixels as
1 and other pixels as
0

Distinguishing between
BG color & ball color by
setting proper threshold value

The diameter of the ball when it is at the ideal position is

always fixed

BALL FOLLOWER ROBOT CONT.

Problematic situation:

Can
be tackled by selecting appropriate threshold

Can be solved by selecting appropriate threshold value.

BALL FOLLOWER ROBOT CONT.

Deciding forward/backward & left/right movement:

Dia.

Diameter will decide either to go forward/backward.

The distance between frame center & ball CG will decide
right/left turn.

BALL FOLLOWER ROBOT CONT.

How to prevent the machine from oscillation?

Eg:

Select a no-action range so that the machine will not

oscillate.

BALL FOLLOWER ROBOT CONT.

MATLAB code: Flie name: main.m
clc
clear
%cb=110;%orange ball
%cr=160;%orange ball
%cb=125;%pink ball
%cr=160;%pink ball
cb=90;%yellow ball
cr=115;%yellow ball
while(1)
image=vfm('grab');
image_ycbcr=rgb2ycbcr(image);
image_cb=image_ycbcr(:,:,2);
image_cr=image_ycbcr(:,:,3);
[r c d]=size(image_ycbcr);
output_image=zeros(r,c);

for i1=1:r
for i2=1:c
if (image_cr(i1,i2)>cr &&
image_cb(i1,i2)<cb)
output_image(i1,i2)=1;
else
output_image(i1,i2)=0;
end
end
end
imshow(output_image)

BALL FOLLOWER ROBOT CONT.

[r_cent c_cent]=centroid1(output_image);
total_pix=sum(sum(output_image));
if(total_pix<700)
stop();
disp('No Ball');
else
if (c_cent<120)
left();
disp('left()');
elseif (c_cent>200)
right();
disp('right()');
else
stop();
disp('stop()');
end

if (r_cent<90)
disp('forward()');
forward();
stop();
forward();
stop();
elseif (r_cent>150)
disp('back()');
back();
stop();
back();
stop();
else
stop();
disp('stop');
end
end
end

BALL FOLLOWER ROBOT CONT.

File name: centroid1.m:
function [r_cent c_cent]=centroid1(arg)
[r_index c_index]=find(arg);
File name: delay.m
r_cent=mean(r_index);
function[ ]=delay(x)
c_cent=mean(c_index);
%tic
File name: forward.m
function [ ]=forward()
parport=digitalio('parallel','LPT1');
line=addline(parport,0:7,0,'out');
pval=[1 0 1 0 0 0 0 0];
putvalue(parport,pval);

for i=1:500
for j=1:500
for k=1:x %if u give x=420 u will get a
delay of 1 second.
end
end
end
%toc

LINE FOLLOWER ROBOT

All the steps discussed above for ball follower is applicable
here.
But here the only change you have to make is for
forward/backward movement, you have to take care of
the changes in the x- direction.
You have to control the speed of the robot by generating
PWM signal.
Use right spin, left spin instead of right turn, left turn.

LINE FOLLOWER ROBOT CONT.

MATLAB code:
clc
clear
cb=127;
cr=0;
while(1)
image=vfm('grab');
image_ycbcr=rgb2ycbcr(image);
image_cb=image_ycbcr(:,:,2);
image_cr=image_ycbcr(:,:,3);
[r c d]=size(image_ycbcr);
output_image=zeros(r,c);
for i1=1:r
for i2=1:c
if (image_cr(i1,i2)>cr &&
image_cb(i1,i2)<cb)
output_image(i1,i2)=1;
else
output_image(i1,i2)=0;
end
end
end

imshow(output_image)
[r_cent c_cent]=centroid1(output_image);
total_pix=sum(sum(output_image));
disp(total_pix);
if(total_pix<7000)
% stop();
disp('No Line');
else
if (c_cent<120)
left();
left();
disp(left()');
disp(c_cent);
disp(r_cent);

LINE FOLLOWER ROBOT CONT.

MATLAB code:
elseif (c_cent>200)
right();
right();
disp(right()');
disp(c_cent);
disp(r_cent);
else
end

if (r_cent<90)
back();
disp('back()');
disp(c_cent);
disp(r_cent);

elseif (r_cent>120)
forward();
stop();
forward();
stop();
disp('forward()');
disp(c_cent);
disp(r_cent);
else
disp('stop()');
disp(c_cent);
disp(r_cent);
end
end
end