ABSTRACT This project present a desktop base technique to represent contour in navigation with contour tracking for a general

purpose mobile robot in given global environment with image processing technique using open Source Computer Vision ( open CV ) library on Visual C++ or Mat Lab. Vision Sensor has very important role in Mobile Robot application. It is now almost rare to find a Mobile Robot that does not employ some form of vision sensor to inspect, locate, measure or tracking for navigation purpose. The contour representation technique a part of vision sensor system and will be use by fuzzy controller or neural controller to complete the vision sensor system. This represent contour is used to guide the navigation of the Mobile Robot toward a goal by changing Mobile Robot direction of movement. The contour of line is taken into consideration and an overhead camera (above the robots environment) is used to gather this information. The images captured are processed using different techniques to get desire contour. The contour needed was captured using a camera which is situated on top of the desired environment in which the mobile robot will be moving. The contour will represent by the line of tape. It is important to get the edges of the tape within the environment. There are many noise of the image must be filtered in image processing.

CHAPTER 1 INTRODUCTION 1.1 Introduction Computer vision is the study and application of methods which allow computers to "understand" image content or content of multidimensional data in general. The term "understand" means here that specific information is being extracted from the image data for a specific purpose: either for presenting it to a human operator (e. g., if cancerous cells have been detected in a microscopy image), or for controlling some process (e. g., an industry robot or an autonomous vehicle). The image data that is fed into a computer vision system is often a digital gray-scale or colour image, but can also be in the form of two or more such images (e. g., from a stereo camera pair), a video sequence, or a 3D volume (e. g., from a tomography device). In most practical computer vision applications, the computers are preprogrammed to solve a particular task, but methods based on learning are now becoming increasingly common. Computer vision can also be described as the complement (but not necessary the opposite) of biological vision. In biological vision and visual perception real vision systems of humans and various animals are studied, resulting in models of how these systems are implemented in terms of neural processing at various levels. Computer vision, on the other hand, studies and describes technical vision system which is implemented in software or hardware, in computers or in digital signal processors. There is some interdisciplinary work between biological and computer vision but, in general, the field of computer vision studies processing of visual data as a purely technical problem.

Figure 1.1 shown Relation between Computer Vision and Various other fields

The field of computer vision can be characterized as immature and diverse. Even though earlier work exists, it was not until the late 1970's that a more focused study of the field started when computers could manage the processing of large data sets such as images. However, these studies usually originated from various other fields, and consequently there is no standard formulation of the "computer vision problem". Also, and to an even larger extent, there is no standard formulation of how computer vision problems should be solved. Instead, there exists an abundance of methods for solving various well-defined computer vision tasks, where the methods often are very task specific and seldom can be generalized over a wide range of applications. Many of the methods and applications are still in the state of basic research, but more and more methods have found their way into commercial products, where they often constitute a part of a larger system which can solve complex tasks (e.g., in the area of medical images, or quality control and measurements in industrial processes). Computer vision is by some seen as a subfield of artificial intelligence where image data is being fed into a system as an alternative to text based input for controlling the behaviour of a
3

system. Some of the learning methods which are used in computer vision are based on learning techniques developed within artificial intelligence. Since a camera can be seen as a light sensor, there are various methods in computer vision based on correspondences between a physical phenomenon related to light and images of that phenomenon. For example, it is possible to extract information about motion in fluids and about waves by analyzing images of these phenomena. Also, a subfield within computer vision deals with the physical process which given a scene of objects, light sources, and camera lenses forms the image in a camera. Consequently, computer vision can also be seen as an extension of physics. A third field which plays an important role is neurobiology, specifically the study of the biological vision system. Over the last century, there has been an extensive study of eyes, neurons, and the brain structures devoted to processing of visual stimuli in both humans and various animals. This has led to a coarse, yet complicated, description of how "real" vision systems operate in order to solve certain vision related tasks. These results have led to a subfield within computer vision where artificial systems are designed to mimic the processing and behaviour of biological systems, at different levels of complexity. Also, some of the learning-based methods developed within computer vision have their background in biology. Yet another field related to computer vision is signal processing. Many existing methods for processing of one-variable signals, typically temporal signals, can be extended in a natural way to processing of two-variable signals or multi-variable signals in computer vision. However, because of the specific nature of images there are many methods developed within computer vision which have no counterpart in the processing of one-variable signals. A distinct character of these methods is the fact that they are non-linear which, together with the multi-dimensionality of the signal, defines a subfield in signal processing as a part of computer vision. Beside the above mentioned views on computer vision, many of the related research topics can also be studied from a purely mathematical point of view. For example, many methods in computer vision are based on statistics, optimization or geometry. Finally, a significant part of the field is devoted to the implementation aspect of computer vision; how existing methods can be realized in various combinations of software and hardware, or how these methods can be modified in order to gain processing speed without losing too much performance.
4

1.2 Problem Statements The major problems

i.

associated with vision system are:

For better image acquisition, high quality camera is needed. If not, the image capture by the camera produced high noise.

ii.

When computer vision technique implement in vision sensor system, it need an appropriate algorithm for image processing which does not required high precision but easy to implement in term of simple and fast processing.

iii.

The distance of the camera and lighting influence the quality of image capture by the camera. So that, these factors are seriously need to be concern.

1.3 Objectives
i.

To develop computer vision technique for image acquisition using low CMOS camera to view video of the camera and capture the image that can be implementing in vision system.

ii.

To develop image processing algorithm to detect and represent the contour of any image capture by the CMOS camera.

1.4 Project Scope The scope of this project is mainly concentrated on the operation of vision system or computer vision system to detect and represent contour used in mobile robot contour tracking. Hence, all the activities carried out are based on these assumptions, such as view the video of the camera, capture the image from the video frame, save the capture image to computer memory and process image to represent contour. In Oder to avoid perplexity to the

image processing algorithm, identification of the environment shall be held under these factors:
i.

Surrounding environment

The plan is to capture the environment images at a minimum noise with adequate lighting source around.
ii.

Optimal illumination

The initial stage of capturing the image is executed under optimal illumination to provide bright daylight environment.
iii.

Distance of camera

Images of robot environment are grabbed from a prefixed distance of the camera which mounted on the mobile robot.
iv.

Contour orientation

The preliminary intension is to capture the image with minimum distortion and the contour only to the left, right and straight. v. Contour representative

The contours are represented by the line of tape at the fixed form of the size and colour. The colour recognitions are not under consideration.

1.5 Literature Review

6

The Vision Sensor development methodology including the algorithm, software and hardware development. The existing image acquisition device used and proposed in literature is reviewed. The contour tracking algorithm development related to image processing that is used for the Vision Sensor is described in detail. The contour needed was captured using a web camera which is situated on the top of the desired workplace in which arm robot will be moving. It is important to get the edge of the work piece within the workplace environments. This edge detection serves as an initial contour detection to determine the number of contour within the workplace environment. There are many type of algorithms for edge detection , but using Canny edge detection is more suitable for a low camera quality image resulting a sharp and detailed edge image. A threshold value is needed for Canny edge detection and this is determined using Otsus method in which the threshold value is found by minimise the within class weighted variance. Inokuchi et. al. (1986) mentioned that detection of lines and edges is the most fundamental problems in image processing and various methods have been proposed for it such as a Hough transform method which can detect a line and edge and it is widely used because the method is elegant and not sensitive to noise or to missing edge elements of the objects. There have been a number of application studies, surveys, explanations, and program libraries regarding the use of Hough Transform (Illingworth J and Kittler J, 1988). Anyway, Hough transforms demands for a tremendous amount of computational power and large memory storage. Next, it also cannot detect both ends of one line and cannot recognize any collinear lines separately. Adolfo et. al. (2001) used predictive look ahead sensor guided technique using CCD camera to capture contour line and approximate them with a polynomial of certain degree to smooth out the curve and neglecting any experimental error. It was shown how a smart tool integrated with sensor that can look ahead and plan the trajectory can be attached and reattached in a flexible manner to Robotic manipulators. This solution can automate industrial programming processes in an intelligent manner. Those are objects of intense research efforts in the field of artificial intelligence (AI): to build machines that consider the information captured from the surrounding environment in a proper (intelligent) manner. With the support of sensors the working trajectory of the robot can be obtained within a certain sensor field which will be used here as the minimization of the tracking error. The
7

main contribution of the work was an on-line tracking optimization scheme for sensor guided robotic manipulators by associating sensor information, manipulator dynamics and a path generator model. Feedback linearization-decoupling permit the use of linear sensor prediction models for the dynamics of each robot joint. Scene interpretation of CCD-camera images generates spline fitted segments of future trajectory that will be tracked. In the sensor vision field the proposed optimization criteria minimizes the error between state variables of the prediction model and the state variables of the spline trajectory generator. These techniques, allied with separation of disturbance rejection and path-tracking performance by the proposed feed-forward following model predictive (FMP) servo-controller design, permits very high path tracking dynamics (and consequently small errors). Vision Sensor The first stage of vision system is image acquisition. This stage requires camera are connected into computer. However, computer does not see an image in the same way as human beings. Humans see an image as a complete picture with lots of background knowledge, while computer see images as an array or matrix which consists of integer or floating point number. For example, a standard image consists of red, green, and blue channel while each channel consists of matrix as shown in Figure 2.5.

After the image is obtained and transferred into the computer, various methods of image processing can be applied to the image. Image processing technique aims to perform an image operator to the image to obtain useful information that is contained within the image.

Various types of the image acquisition device (including references) which can be used for vision system is shown in below. Most of the typical cameras are very sensitive to environment and sometimes it is necessary to control the lighting condition. However, the lighting condition can be controlled by using dimmer control based on the surface reflectance of robot environment being observed. Furthermore, several comparisons of both are shown in Table 2.5 (Dalsa, 2010). It can be seen that CMOS devices have high noise immunity, low static power consumption and also very cheap in comparison to CCD.

Software Tools for Vision System The other commercial library package offers rapid development for vision system such as Matlab and National Instrument (NI) Vision Development Module. Besides, the free open source library such as OpenCV, Mimas, ImageLib AForge.Net, and FreeImage is also available for vision system development. The other popular image processing tools are given in Table 2.3 (Malamas et. al., 2003).

Although there are certain of image processing libraries available today, the issue of the adoption for certain software development methodology is the ignorance of the time constraint. In fact, vision system development does not consider this issue offers understanding for software development. Image processing review Camera As with all computer vision systems, the vision camera must be calibrated before it can be used for an application. Calibration determines the relationship between the camera and the work cell. There are two aspects to this relationship: Camera calibration better

This describes the relationship between what the camera sees (described using units of pixels) and the real world (in millimetres). The camera calibration accounts for both the difference
10

in scale and for any perspective distortion created when the camera imaging element is not parallel with the work surface. The Camera Calibration program allows performing the following functions: Set the camera focus and the gap. Adjust the camera video gain and offset. Adjust the black/white threshold. Adjust the image border parameters. Determine the camera calibration. Determine the location of the camera relative to the robot. Test the performance achieved using the calibration data. Load or store calibration data on a disk.

Lighting There are various type of lighting that can be used as an Illumination source to provide good image capturing. Table 1: Types of Lighting Type Incandescent Fluorescent TungstenHalogen Strobes IR & Diodes IR Laser He-Ne Laser Advantages Inexpensive, can be cycled Efficient, cool High output Gas High power, freezes Laser Not regulated Good CCD sensitivity Highly collimated, Disadvantages Short of life ( for AC lamps ), heat Cant be cycled, flickers Heat, cant be cycled Expensive, poor CCD sensitivity Low power, highly divergent Highly divergent, federally regulated Fragile, expensive, bulky

Depth of field Lighting Strategies Diffuse

11

Light is reflected from a shiny surface at the opposite angle from which it struck the surface. If most of the light that is reflected from a surface comes from the same angle, the surface will be sharp shadows and great deal of surface glare. Diffuse lighting illuminates a surface with light that strikes the surface from a many different angles as possible, thus minimizing shadows, reflections, and the need for critically placed light sources. Fluorescent lighting is the most diffuse of the lighting types listed in Table 1. Applications with high contrast, complicated objects, spherical objects, shiny objects, or objects that require multiple inspections of interior features are candidates for diffuse lighting. Pixel The basic unit of a vision image is a pixel. It is the smallest unit of information a vision system can return to us. The number of pixels the system can process determines the systems resolution and affects the computer processing time needed to analyze an image. A pixel can be thought of a single cell in a matrix that the camera overlays on the field of view (FOV). The value that is placed in a cell (pixel) will be a shade of gray that represents the intensity of the light reflected from the corresponding area in the field of view. This ability to analyze an image based on intensity values is referred to as grayscale vision. Figure 1 below shows how a 22 x 16 pixel camera would see the sample object. In addition to grayscale processing, Vision Ware can process image data in binary mode. In binary mode, all the cells with a value above a certain value will be seen as white and those below that value will be seen as black. Figure 2 below shows how the sample object would be seen in binary mode.

Resolution The camera representation for this theoretical part use 22 x 16 pixels shows very poor resolution due to the low density of pixels. In addition to the number of pixels in the camera array, the lens focal length combined with the distance of an object from the camera will have the greatest effect on the final
12

resolution of whatever we are viewing. The relationship between focal length and viewing distance. In general, the best resolution will come when the object of interest fills as much of the field of view as possible while still being in focus. An important concept is the relationship between the pixel and the physical size of an object. A pixel will always have a relative relationship to the size of an object. It will have an absolute relationship only when we fix our viewing distance and lens focal length, and then calibrate the vision system. The calibration process establishes an absolute relationship between a pixel and the actual dimensions of the field of view.

Grayscale vs. Binary Processing Depending on the tool operation we select from the tool record, Mat Lab will process either in binary mode or grayscale mode. The processing mode is selected by Mat Lab and in most cases; we will not have to be concerned with which type of processing is taking place. In many cases, however, we will have to set threshold values for binary processing or the edge strength value for grayscale processing. The information below will give an idea on of how to set those values. A magnified section of the array of pixels that might be returned by a camera. In each pixel of the matrix is the grayscale value the camera has registered from the field of view. When the software processes image data in binary mode, each value in the matrix is compared with the threshold value. All the pixels with a value above the threshold value are considered white and all the pixels below this value are considered black. In binary mode, edges of objects within the field of view are found by looking for a change from a white section of pixels to a dark section of pixels. The system would process the data into a binary image if the threshold can be set automatically from within any picture record. There are actually two threshold values that can be used together to isolate a range of intensity values. When 2nd threshold and 1st threshold will be white. The sample array with the threshold set to 34 and 2nd threshold set to 89. When the software processes the data in grayscale mode, it looks at a three-by-three section of pixels to the value of edge strength. If the difference found exceeds edge strength,

13

the system considers the three-by-three area to be part of an edge. The edge system would find in the data from the array if the edge strength were set to 20. This algorithm uses when looking at the three-by-three neighbourhood of pixels allows edges to be located with sub-pixel accuracy. Thus, grayscale processing is generally more accurate than binary processing.

CHAPTER II METHODOLOGY In the vision base for real time mobile robot application, a webcam is mounted at the top of mobile robot. The webcam directly interface with a computer.
14

2.1 Element of Computer Vision System

It is apparent from figure that the computer vision system integrates the following items:

Camera is to capture image from robot environment to track contour Light source for illumination Computer is to provide high speed application in image processing and image acquisition

2.2 Software Frameworks A software framework for video processing in contour tracking of real time computer vision system will be developed. The frame work will start from image acquisition and will go through a series of process before the result can be output .figure 4 has shown the whole process in general. The process start with images acquisition where image will be capture,
15

following by image enhancement of the images captured to reduced noise in the image and make in grayscale.Images are then segmented to detect contour and change to binary. Finally according to the contour edge parameter, the images characteristic to represent contour are shown in term of image histogram.

START

IMAGE ACQUISITION
Accessing device Grab image from camera Save in database

IMAGE ENHANCEMENT
Image Filtering Grayscale/Blur Crop the interest

IMAGE SEGMENTATION
Contour detection Erode and Dilate Multilevel threshold

IMAGE CHARACTERISTIC
Histogram to represent Contour

END

2.2.1 Image Acquisition The purpose of image acquisition is to grab image from camera and generate the digital image data. Before being processed, the camera should be installed with the lighting system
16

at the mobile robot. When all was ready, the camera can capture images for all regions of robot environment. 2.2.2 Image Enhancement There is a limitation when first grabbed image has been done. The characteristic of the environment, which is non-uniform texture of surface, makes then feature in image covered with a lot of noise. Because of that, further process could not directly do. For preliminary process, adjust contrast value of the image to make the defect brighten from the whole of image. Gaussian filter is used to remove the noise. So that it can differentiate the contour from the noises in order to get the feature of the contour. Applying a Gaussian blur to an image is same as convolving the image with a Gaussian function.

2.2.3 Image Segmentation Segmentation is an essential part in a wide range of image processing tasks and building block of many visualization environments. One of segmentation techniques is a multilevel threshold. This technique segments a gray-level image into several distinct regions. This process is often designed using the gray level histogram of the image. The multi level threshold technique determines more than one threshold value for the given image. And then it segments the image into certain brightness regions, which correspond to one background. 2.2.4 Image Characterization This method is used to find the relationship between contour images with its histogram. From here, the different of the contour image will represent the different of the histogram.

CHAPTER III EXPECTED RESULT Image Acquisition

17

Camera capture in difference illumination condition.

Image Enhancement
18

19

Image Segmentation

20

Image Characteristic

21

CHAPTER IV CONCLUSSION The state of the art shows that the vision sensor development requires cost effective vision sensor, contour represents algorithm and relevant software development method. The CMOS based camera shows as more feasible to use as vision sensor even quality image. Mat Lab software incorporate development though it has low method to methodology is relevant

the time constraint requirement in the vision sensor development.

22

CHAPTER V

23

CHAPTER VI REFERENCE Dalsa.(2010).CCD vs. .CMOS Available at:http://www.dalsa.com/corp/markets/CCD_vs_CMOS. Accessed January, 2010. Gomaa, H.(2008).Advances in Software Design Methods for Concurrent, Real-Time and Distributed Applications. Proceeding of the 3rd International Conference on Software Engineering Advances, pp. 451-456 Habibullah A. and Prabuwono, A. S., (2008). Webcam Based System for Press Part Industrial Inspection, International Journal on Computer Science and Network Security, Vol. 8, pp. 170-177. Holt, J. D. (1996). Practical Issues in the Design of AI Software. Engineering Application Artificial Intelligence, Vol. 9, pp. 429-437. Laplante P.A.(2002).A Retrospective on Real-Time Imaging, a Roadmap for the Future. Real-Time Imaging, Vol. 8, pp. 413-425. Zunic, J. Hirota, K. & Rosin, P. L. (2010). A Hu Moment Invariant as Shape Circularity Measure. Pattern Recognition, Vol. 43, pp. 47-57. Castilho, H. P., Goncalves, P. S. S., Pinto, J. R. C. & Serafim, A. L. (2007). Intelligent RealTime Fabric Defect Detection. Image Analysis and Recognition, Vol. 4633/2007, pp. 12971307. Chapman, S. (2008). MATLAB Programming for Engineers. 3rd ed., Canada: Thomson Learning. Chen, S., Hu, T., Liu, G., Pu, Z., Li, M., & Du, L. (2008). Defect Classification Algorithm for IC Photomask Based on PCA and SVM. Proceeding of the International Congress on Image and Signal Processing, pp. 491-496. Malamas E.N., Petrakis E.G.M, Zervakis M., Petit L. & Legat J.-D. (2003). A Survey on Industrial Vision Systems, Applications and Tools. Image and Vision Computing, Vol. 21, pp.171-188. Matlab. (2009). Matlab and Simulink Student Version Help v. 7.8.0.347
24

New Taxonom and a

F. Y. Shih And O. R. Mitchell, ``Threshold Decomposition Of Grayscale Morphology Into Binary Morphology,'' IEEE Trans. Pattern Analysis And Machine Intelligence, Vol. 11, No. 1, pp. 31-42, Jan. 1989. Adolfo M. Bronstein, Influence Of Action and Expectation on Visual Control of Posture, Vol.11, pp 259-266 Inokuchi,Photoemission from an amorphous pentacene film, Vol.109, pp 157-162

25