Image Capture and Processing in MATLAB

Dr Chris Mellor
Introduction
In the past image capture from the webcams in the third year laboratory was done via
consumer software. We have now added the Image Acquisition toolbox to the Schools
MATLAB teaching licence so images can be acquired from within MATLAB. This has
several advantages for scientific use. Firstly the exact time at which a frame is taken can
be stored along with the image. Secondly MATLABs extensive data manipulation
commands are immediately available so that one image can be subtracted from another,
colour images can be split into their different colour values or converted to monochrome
and fast Fourier transforms performed. The downside of this method of image capture is
that you will need to learn the basics of MATLAB.
Getting Started with MATLAB
The PC must be connected to the campus network. Login as normal and start MATLAB
from the NAL. It is important to take a few minutes to look at the MATLAB windows.
On the right of the main window is the command window into which you can enter
MATLAB commands or the names of files containing MATLAB scripts (lists of
MATLAB commands put together to form a program). At the top right there is a window
into which you can type the current directory where MATLAB will look for files and
save files to by default. At the top left there is a window displaying the names of
variables as well as their size and format. At the bottom left there is a window that
displays the history of commands that you have entered.
If you enter a command that creates graphical output (a graph or an image) on the screen,
a window will open with the requested image. I recommend that you close this window
as soon as you have finished with it otherwise you can end up with so many windows
open that you lose track of what you are doing!
The website of the third year MSci module in which MATLAB is taught can be found at:
http://www.nottingham.ac.uk/~ppzfrp/matlab/
This page also contains a reference section of common commands:
http://www.nottingham.ac.uk/~ppzfrp/matlab/#commands
There is also an extensive help system built into MATLAB which often has examples
that can be copied into your own programs. Finally, the MATLAB website,
www.mathworks.com has many example programs available on it.

Writing Programs
The basic way of writing programs in MATLAB is to write a list of MATLAB
commands as a script file. To open a new editing window go to FILE > NEW >M-File.
An m-file needs to have the extension .m e.g. myfirst.m so that it is recognized as a
MATLAB file. The file can be run by typing the name of the file into the MATLAB
command window.
Image capture
The image capture commands allow you to acquire and store images as .avi files. This is
a common file format for video data and the files can be viewed in the Microsoft media
player and other programs. The first step is tell MATLAB about the camera you are
using. The following commands can be found in the file imagecontrol.m on the lab
website.
imagecontrol.m
%
%
%
%

Basic control of USB2 webcam others should be similar but parameter

values may vary
11th September 2003
Chris Mellor

% Define video object from which MATLAB can obtain data

vidobj = videoinput('winvideo', 1);
% Open the preview window so you can see what the webcam is looking at.
preview(vidobj)
% to close preview window use: closepreview
pause % the pause command waits until a key is pressed
closepreview
% to see the various parameters you can set and to see current values
% use get(objectname)
get(vidobj)
% to control frame rate from the camera you have to use the following
% commands
src = getselectedsource(vidobj);
actualRate = 30; % USB2 camera should be able to use 15fps and 30fps
src.FrameRate = num2str(actualRate);
% Always release video objects after use
delete(vidobj)
clear vidobj

So with the program listed above you can see the image from the webcam and control
basic parameters such as frame rate. More examples of setting parameters can be found
under image acquisition toolbox in the MATLAB help files. I recommend that you stay
with RGB files where possible as these seem easier to acquire and handle in MATLAB.
Frame rates are hardware dependent and so only certain rates are allowed. The size of the
image (in pixels) is also hardware dependent.
To look at the control parameters and settings of the camera in more detail you can use
the following program (imagediag.m)
imagediag.m
% USB web camera diagnostics
% 11th September 2003
% Chris Mellor
% show attached hardware devices (should be winvideo, 1)
imaqInfo = imaqhwinfo
% Access an image acquisition device.
vidobj = videoinput('winvideo', 1, 'RGB24_320x240');
% note I have also specified a video format
% Display properties of video object
get(vidobj)
%
dev_info = imaqhwinfo('winvideo',1)
celldisp(dev_info.SupportedFormats)
pause
% Open the preview window.
preview(vidobj)
pause
% take snapshot
closepreview
snapshot = getsnapshot(vidobj);
imagesc(snapshot)
pause
delete(vidobj)
clear vidobj

Storing Images
Storing images using MATLAB is possible in several ways. The first important point is
that video images contain a lot of data. For example a 24 bit colour picture 640x480
pixels in size will be just under 1Mbyte. Now store 30 frames a second and you can see
that you rapidly run out of memory space. This is why digital video is usually
compressed to make a smaller file with most of the useful information kept hopefully.
However if the compression is too great you can lose valuable information.
Storing the images directly into memory is the recommended route unless you are
acquiring images at a rate below 4 fps when you can save it directly to the hard disc. For

higher frame rates the PC simply cannot keep up and so you need to store into memory
and then store to disc afterwards. Keep the length of the video captured as short as
possible. Remember you are going to have to analyze the images afterwards so you need
to be careful to take the right data and not too much of it. If you record too much video
the PC may well crash or slow down drastically because it has run out of memory.
On the Viglen PCs there are three partitions on the hard disc. The first is c: which is
where the programs are kept. The second d: is an image of c: and allows the technical
staff to completely overwrite c: if it gets damaged and restore it to its original state. Drive
e: is spare hard disc space where you can store your video images. Please make a new
folder under your username so that everyone knows that the files in the folder are for
your project. The Viglen PCs also have CD burners on them so that the image files can
periodically be written to CD-R so that you can take the data away with you to analyze.
Writing to CD-R is also important as it backs up your files so you do not lose the data if
the hard disk crashes.
The following program captures images (imagestore.m) and stores to disc and memory
(only suitable for slow acquisition) :
imagestore.m
% Basic acquisition to disc of USB2 webcam others should be similar but
parameter
% values may vary
% 11th September 2003
% Chris Mellor
% Define video object from which MATLAB can obtain data and sets the
% resolution in this case 320x240 RGB format 24 bit colour
% To set it to maximum resolution use RGB24_640x480
vid = videoinput('winvideo', 1,'RGB24_320x240');
%
%
%
%

Prepare a
stream to
to stream
Note this

file for the image to be stored in if you are going to

disk - commented out as it is better for fast acquisition
to memory and then compress later
filename should not already exist

logfile = avifile('logfile2.avi')
% set frame rate parameter in avi file - this allows slow motion replay
in media player for example
logfile.fps = 1; %Media player will playback at 1 fps
% Choose a compression algorithm for making the avi file
logfile.Compression = 'Indeo3';
% Tell the video object to use the assigned file
vid.DiskLogger = logfile;
% to control frame rate from the camera you have to use the following
% commands

src = getselectedsource(vid);
actualRate = 30; %number is in frames per second fps
% for Orange ibot usb2 camera the allowed values are 15 fps and 30 fps.
src.FrameRate = num2str(actualRate);
% To check that the image is what you want you can use preview
preview(vid)
% To control the object use set command as below
% for example to have MATLAB grab every tenth frame of the video stream
% NB these parameters do not affect preview only the stored stream
% number of frames that are ignored after start command
set(vid,'TriggerFrameDelay',30);
% take every 10th frame ( to take every 3rd frame set parameter to 3)
set(vid,'FrameGrabInterval',10);
% set logging mode
set(vid,'LoggingMode','disk&memory');
% can also be 'disk' or 'disk&memory'
%Note the 'disk&memory' option only seems to work reliably at 4 fps or
%slowerat full resolution and 20fps at 320x240 I did not try 'disk' but
%I bet it is the same. Therefore I recommend for fast acquisitions that
%you grab the image to memory and then store into a avi file
%afterwards.
% number of frames recorded after each start command
set(vid,'FramesPerTrigger',60);
% To check that the image is what you want you can use preview
preview(vid)
%press any key to continue
pause
closepreview
start(vid)
wait(vid) % waits until video object has finished recording the images
% Once the file has been recorded close the avi file if you were
% streaming to disk
aviobj = vid.Disklogger;
file = close(aviobj);
% pull image data and timestamps from image buffer memory into MATLAB
workspace
[frames timeStamp] = getdata(vid);
% Graph frames vs time.
plot(timeStamp,'x')
xlabel('Frame Index')

ylabel('Time(s)')
% Find the time difference between frames.
diffFrameTime = diff(timeStamp);% Graph the time differences.
plot(diffFrameTime, 'x');
xlabel('Frame Index')
ylabel('Time Difference(s)')
set(gca, 'ylim', [0 0.4]) % manually sets limit on yaxis
avgTime = mean(diffFrameTime)
exptRate = 1/avgTime
delete(vid)
clear vid

To store data to memory and then save compressed video later you can modify the
following program (imagestore2.m)
%
%
%
%

Basic acquisition to disc of USB2 webcam others should be similar but

parameter values may vary
11th September 2003
Chris Mellor

%
%
%
%
%
%
%
%

Define video object from which MATLAB can obtain data and specify
video format possible values are:
RGB24_640x480, RGB24_320x240, RGB_160x120 and with I420 and YUY2
replacing RGB24
I420 has little to recommend it
YUY2 stores an intensity and two chrominance signals
RGB stores red, green and blue signals and is what MATLAB likes best!
to convert RGB to black and white use RGB2gray function

vid = videoinput('winvideo', 1,'RGB24_640x480');

% to control frame rate from the camera you have to use the following
% commands
src = getselectedsource(vid);
actualRate = 30; % number is in frames per second (fps)
% for Orange ibot usb2 camera the allowed values are 15 fps and 30 fps.
src.FrameRate = num2str(actualRate);
% To control the object use set command as below
% for example to have MATLAB grab every tenth frame of the video stream
% NB these parameters do not affect preview only the stored stream
set(vid,'TriggerFrameDelay',10); % number of frames that are ignored
after start command
set(vid,'FrameGrabInterval',1);
set(vid,'LoggingMode','memory');% can also be 'disk' or 'disk&memory'
%Note the 'disk&memory' option only seems to work reliably at 4 fps or
%slowerat full resolution and 20fps at 320x240 I did not try 'disk' but
%I bet it is the same. Therefore I recommend for fast acquisitions that
%you grab the image to memory and then store into a avi file
%afterwards.
fpt=60;

set(vid,'FramesPerTrigger',fpt); % number of frames recorded after each

start command
% To check that the image is what you want you can use preview
preview(vid)
%press return to start capture
pause
%closepreview
start(vid)
wait(vid) % waits until video object has finished recording the images
% pull image data and timestamps from image buffer memory into MATLAB
workspace
[frames timeStamp] = getdata(vid);
% Graph frames vs time.
plot(timeStamp,'x')
xlabel('Frame Index')
ylabel('Time(s)')
% Find the time difference between frames.
diffFrameTime = diff(timeStamp);% Graph the time differences.
plot(diffFrameTime, 'x');
xlabel('Frame Index')
ylabel('Time Difference(s)')
set(gca, 'ylim', [0 0.4]) % manually sets limit on yaxis
avgTime = mean(diffFrameTime)
exptRate = 1/avgTime
% Always delete and clear video object after use to free memory
delete(vid)
clear vid
% now need to save video images to disk as a compressed file.
% as it is not realtime we can use high quality settings
fig=figure;
set(fig,'DoubleBuffer','on'); % allows smooth image update - no flashes
mov = avifile('example.avi');
% set frame rate parameter in avi file - this allows slow motion replay
% in media player for example
mov.fps = 1; % Media player will playback at 1 fps
% Choose a compression algorithm for making the avi file
mov.Compression = 'none';
%Indeo5 not working correctly as colour is messed up
% and MSVC or RLE do not work with truecolor images, Indeo3 is default
mov.Quality = 75;
for k=1:fpt
imagesc(frames(:,:,:,k))
F = getframe(gca);
mov = addframe(mov,F);
end
mov = close(mov); %25MB without compression for 60 full size frames!

% Can also save data to a mat file using the save command

Reading images back into MATLAB

We have now looked at acquiring images and storing images. What we want to do now is
to read a video file stored in avi format (note the file does not have to have been created
by MATLAB) into MATLAB so we can look at it and carry out basic operations on the
data.
The printout that follows shows how to handle the video data in MATLAB. Important
points which need to be remembered include:
1. The video data in an avi file has been compressed. This can change the size of the
image so do not assume that a 640x480 image will be the same size when reading back
from an avi file.
2. The video data is stored in a particular MATLAB data format called a cell structure.
The program below shows you how to extract the data and convert it into more
conventional data types, which you can then carry out mathematical operations on.
3. Always delete the variables with video data in them when you have finished with them
using the clear command. This will stop you running out of memory.
4. imview is a very useful command which will display the image in an interactive
window so that you can move a cursor and look at the properties and locations of any
pixel. This can be a great help when you are trying to find out where the important parts
of your image are.
What is shown here is very basic image analysis. It will not run as the video file ball.avi
no longer exists. In the video a white ball was dropped against a black background. The
program allows the user to look at a selected portion of the video and then subtract two
images to show how far the ball has moved in the image. If the distance scale was then
calibrated the acceleration and velocity of the ball could be calculated. More examples
can be found on the Mathworks website (www.mathworks.com) and in the help files for
both the image acquisition and analysis toolboxes. If you are careful it is sometimes
possible to carry out some image processing in real time (i.e. whilst acquiring data) as
long the frame rates are sufficiently low.
imageanalysis.m
% Simple file showing what can be done with MATLAB
% Sept 11th 2003
% Chris Mellor
% read and display part of an image file called ball.avi stored in
% current directory
mov=aviread('ball.avi', 15:30);

% interesting frame range had previous been found using media player
% Play selected part in matlab
movie(mov);
% Mov is a cell structure that needs to be separated into the frames
% for analysis.
for k = 1:16
m(:,:,:,k) = mov(k).cdata;
end
%Once this is done delete mov to save space
clear mov
% Note the avi conversion has lost resolution even without compression
% so for highest resolution it is better to store to a mat file.
% To load in data from a MAT file use load command
% For interactive data analysis use
imview(m(:,:,:,3)); % etc
% As object of interest is a white object against a dark background
% convert colour image to grey scale remembering matlab was written in
% the US so colour is color
B=rgb2gray(m(:,:,:,16)); % background image
F=rgb2gray(m(:,:,:,9));
G=rgb2gray(m(:,:,:,8));
%
%
%
%

RGB data stored as unsigned integers between 0 and 255 which cannot
be used for mathematical operations
To make this useable for math convert to double precision and divide
by 255 to get range from 0 to 1

B=double(B)./255;
F=double(F)./255;
G=double(G)./255;
% Now subtract a line of the images from background to see what has
% altered
% The interesting line was identified using imview
a=F(:,234)-B(:,234);
b=G(:,234)-B(:,234);
figure
hold on
plot(a,'r')
plot(b,'b')