Professional Documents
Culture Documents
Nadu, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - The process deals with product recognition products they buy and use in their day to day life. The focus
for the blind, using label text as well as barcode of the revolves around designing a portable camera based assistive
same. The implementation is done using MATLAB. First, text and product label reading along with barcode
recognition from hand held objects for blind person. In the
the Maximally Stable Extremal Region (MSER) algorithm
currently available systems, portable bar code readers are
is used to identify the Region of Interest (ROI) from the designed to help blind people to identify different products
given input image, which would consist of the product in an extensive product database, which helps the blind
label part. Using the background subtraction technique, users to access information about these products through
the foreground is alone extracted and the remaining speech. This system has a limitation that it is very hard for
parts are neglected. After extracting the foreground, the blind users to find the position of the bar code and to
canny edge detection algorithm is used to find the correctly point the bar code reader at the bar code. Some
boundaries of objects in ROI and those boundaries are reading assistive systems such as pen scanners might be
enhanced using edge grow up algorithm. Optical employed in these and similar situations. Such systems
Character Recognition (OCR) is the key algorithm which integrate OCR software to offer the function of scanning and
recognition of text and some have integrated voice output.
is used for extracting the label text from the given input
However, these systems can be performed only for the
image, irrespective of the font style of the label and no document images with simple backgrounds, standard fonts, a
matter however complex the background maybe. Once small range of font sizes, and well-organized characters
the product name is extracted, the output is given in the rather than with multiple decorative patterns.
form of speech using voice board and is also displayed A number of portable reading assistants have been designed
using LCD by interfacing both with a microcontroller. specifically for the visually impaired K-Reader Mobile runs
Using the histogram equalization technique, the given on a cell phone and allows the user to read mail, receipts,
barcode image is analyzed and the barcode number is fliers, and many other documents. However, the document to
alone extracted from the entire input. Once the number be read; must be nearly flat, placed on a clear, dark surface
is extracted, it is searched through the database and the and contain mostly text. In addition, K-Reader Mobile
accurately reads black print on a white background, but has
corresponding product details are identified and
problems recognizing colored text or text on a colored and
announced. complex background. Furthermore, these systems require a
blind user to manually localize areas of interest and text
regions on the objects in most cases.
Key Words: Maximally Stable Extremal Region (MSER),
Even though a number of reading assistants have been
Region of Interest, Canny edge detection, Background
designed specifically for the visually impaired, no existing
subtraction, Optical Character Recognition (OCR), Histogram
reading assistant can read text from the kinds of challenging
equalization.
patterns and backgrounds found on many everyday
commercial products such as text information can appear in
various scales, fonts, colors, and orientations. There are
1. INTRODUCTION various drawbacks of these existing systems like, the texts
from non-uniform or complex backgrounds cannot be
There are 314 million visually impaired people worldwide, identified, the algorithms cannot process text strings fewer
of those 45 million are blind according to a report released than three characters and for the barcode recognition, only
by World Health Organization in 10 facts regarding the barcode number is read and the product details cannot
blindness. Reading is obviously essential in todays society. be identified.
Developments in computer vision technologies are focusing
on assisting these individuals. So the basic idea is to develop
a camera based device which combines computer vision
technology, to help the blind identify and understand the
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2576
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072
Reading is obviously essential in todays society. Printed A barcode is an optical machine-readable representation of
texts are everywhere in the form of reports, receipts, bank data relating to the object to which it is attached. Originally
statements, restaurant menus, classroom handouts, product barcodes systematically represents data by varying the
packages, instructions on medicine bottles, etc. And while widths and spacings of parallel lines, and may be referred to
optical aids, video magnifiers, and screen readers can help as linear or one-dimensional (1D). Later they evolved into
blind users and those with low vision to access documents, rectangles, dots, hexagons and other geometric patterns in
there are a few devices that can provide good access to two dimensions (2D). A barcode has several distinctive
common hand-held objects such as product packages and characteristics that can be used to distinguish it from the
objects printed with text such as prescription medication rest of an image, the first of which is that it contains black
bottles. The ability of people who are blind or having bars against a white background. Occasionally, a barcode
significant visual impairments to read printed labels and may be printed in other colors than black and white, but the
product packages will enhance independent living and foster pattern of dark bars on a light background is roughly
economic and social self-sufficiency. So a system is proposed equivalent after the image has been converted to grayscale.
that will be useful to blind people for the aforementioned. Barcodes initially were scanned by special optical scanners
The system framework consists of three functional called barcode readers. Generally, a simple barcode has five
components they are: sections. They are-number system character, three guard
bars, manufacturer code, product code, check digit. Number
Scene Capture system character specifies the barcode type. Three guard
bars indicate the start and end of the barcode, difference
Data Processing between manufacturer code and product code. Manufacturer
code has a five digit number which is assigned to the
Audio Output manufacturer of the product. These codes are assigned and
maintained by the Uniform Code Council (UCC). The product
The scene capture component collects scenes code is a five digit number that the manufacturer assigns for
a particular product. Check Digit is also called as "self-check"
containing objects of interest in the form of images; it
digits. The check digit is used to recognize whether the other
corresponds to a camera attached to a pair of digits are read correctly.
sunglasses or anywhere as per the requirement. Barcodes provide a convenient, cost-efficient, and
accurate medium for data representation and are
The data processing component is used for deploying the ubiquitous in modern society. However, barcodes are
proposed algorithms, including following processes not human-readable and traditional devices for reading
barcodes have not been widely adopted for personal
Object-of-interest detection to carefully extract the
image of the object held by the blind user from the
use. This prohibits individuals from taking advantage
clustered background or other neutral objects in the of the information stored in barcodes. Image
camera view. processing provides possibilities for resolving the
discrepancy between the availability of barcodes and
Text localization is to obtain text containing image the capability to read them by enabling barcodes to be
regions, then text recognition to transform image- identified and read by software on general imaging
based text information into readable codes. devices. Barcodes may appear in an image at any
orientation or size and multiple barcodes may be
The audio output component is to inform the blind user of present. Additionally, complex backgrounds make
the recognized text codes in the form of speech or audio. A
picking out true barcodes from false positives a
Bluetooth earpiece with mini microphone or earpiece is
employed for speech output.
challenging process.
Thus blind persons will be well navigated if a system can tell
them what the nearby text signage says. Blind persons will
2. PROPOSED SYSTEM
also encounter trouble in distinguishing objects when
shopping. They can receive limited hints of an object from its
shape and material by touch and smell, but miss descriptive To overcome the problems in the existing system and also to
labels printed on the object. Some reading-assistive systems, assist blind persons to read text from those kinds of
such as voice pen, might be employed in this situation. challenging patterns and backgrounds found on many
everyday commercial products of Hand-held objects, a
camera based assistive text reading technology is used in
this project. In the proposed system the product can be
identified by label reading and it can also be identified
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2577
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072
through barcode recognition. The input image is captured 3.1 BLOCK DIAGRAM FOR LABEL READING
using the web camera and processed using MATLAB for both
label reading and bar code recognition.
A region of interest is taken and a mixture of Gaussian based
background subtraction techniques is used. Here in the
region of interest, a novel text localization method is used to
identify the texts. To get over the problems described in
explanations and also to assist sightless individuals to read
written text from those kinds of complicated styles and
background scenes found on many day to day professional
products of hand-held things, to draw out written text
information and brand from complicated background scenes
with several and varying written text styles, a written text
localization criteria is suggested. In the proposed system
adaptive threshold and histogram equalization are used for
the recognition of barcode of the product. The recognizable
barcode will be conformed, once it is conformed then the
particular barcode details will be displayed.
The extracted output component is used to inform the
blind user of recognized text codes in the form of
speech or audio.
3. BLOCK DIAGRAMS
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2578
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072
Background subtraction is a computational vision process of microcontroller using DB9, which in turn acts as a
extracting foreground objects in a particular scene. A connection establisher. Here a level shifter is used in
foreground object can be described as an object of attention addition to all these; since the PC and the wires have
which helps in reducing the amount of data to be processed different operating voltages (The PC works at a voltage of 0
as well as provide important information to the task under to +5v and the wires work at -12 to +12v).
consideration. Here the background is removed or The required data is thus sent to the microcontroller
subtracted in which the label text is present. For this, the (PIC 16F877A) which displays the data through the
indices value or the value obtained using the MSER corresponding output units like LCD, voice board etc.
algorithm is taken. With the obtained indices value, the which are interfaced to the microcontroller using the
intensity values of the entire region are compared. If the
respective codes. The LCD output is directly viewed
values match with the indices value, then it is considered as
true value, else it is taken as false value and the search while the voice output is received with an additional
resumes. Thus the foreground portion is alone extracted speaker unit.
from the given ROI. Edge detection is the name for a set of
mathematical methods which aim at identifying points in a
digital image at which the image brightness changes sharply 3.2 BLOCK DIAGRAM FOR BARCODE RECOGNITION
or, more formally, has discontinuities. The points at which
image brightness changes sharply are typically organized
into a set of curved line segments termed edges. Using this
method, only the boundary of the label text or the region to
be extracted is obtained. There are 3 types of edge detection
techniques that are preferred in practice namely-Canny edge
detection, Sobel edge detection and Prewitt. Of the three,
canny edge detection is commonly used because here only
the boundary region has to be processed unlike the other
techniques where the inner details have to be processed in
addition to the boundary. Once the skeleton of the ROI is
obtained, edge grow up algorithm is used to check the
correctness of the boundaries. In the edge grow up algorithm
we use gradient method. Gradient is defined as the slope Figure2. Block diagram of barcode section
between the high and low contrast. By studying the growth
of the gradient curves, the boundary perfection is verified. 3.2.1 BLOCK DESCRIPTION FOR BARCODE
Filtering is used to remove any kind of unwanted RECOGNITION
information or noise from the given image. Here median
filter is used for that purpose. The output from this section is The block diagram for the product recognition using barcode
a text box with the required label text in it. Morphological is given in the figure2. Once the product is brought in front of
processes such as smoothening is done on the filtered image. the input capturing unit or the camera, a picture of the
The obtained text region is taken to apply OCR or optimal product is taken and it is identified using the predefined
character recognition algorithm. It is an algorithm to read pattern. A generalized pattern of the bar code is fed into the
out printed texts using any photoelectric device or camera. system which acts as the predefined pattern.
The OCR comprises of many kinds of fonts and styles in its Preprocessing algorithms frequently form the first
function which makes the identification of text strings easier. processing step after capturing the image. Image
The text box is segmented and each segment is compared in preprocessing is the technique of enhancing data images
the function database and the matched alphabet is prior to computational processing. Preprocessing images
considered as the output. Thus the required text is extracted commonly involves removing low-frequency background
using OCR. The result of OCR is a text string, which consists noise, normalizing the intensity of the individual particles
of the name of the product label as separate alphabets. The images, removing reflections, and masking portions of
obtained output from the MATLAB section is to be sent to images, conversion of RGB into gray scale.
the hardware part or the Microcontroller part, for which Histogram is constructed for the preprocessed image to
serial data transfer mechanism, is made use of. obtain the foreground region. A histogram is a graphical
For the serial data transfer, the obtained text string is first representation of the distribution of numerical data. It plots
sent to the SBUF or the Serial Buffer which is used as a the number of pixels for each intensity value. Using the
temporary storage for the data. The data in the SBUF is the histogram equalization algorithm, the maximum value of
transmitted in the form of frames using the data-frame histogram is obtained. And the maximum size of the
section. In the data-frame part, the frame consists of 8 bits obtained histogram value is determined using the size of
each with a corresponding start bit and a stop bit for each operator. Also, a threshold value is set with reference to the
frame. Then the frames are transferred to the obtained maximum histogram value, which is used for
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2579
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2580
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072
From the figure6, the first part of the image depicts the
canny edge detection that is performed on the obtained
image where the boundary of all the objects of the
image are sketched and thus a rough outline of the Figure8. Original MSER and segmented MSER regions
differentiation between the background and the
objects are obtained. The second part of the image The original MSER region consists of the boundary of
shows the intersection of the canny edge detected the ROI along with a few unwanted portions. In order
boundaries along with that of MSER. By this, we obtain to remove those, segmentation is done by negating the
the boundary of the ROI alone. Thus the image depicts gradient grown edges and performing the logical AND
the differentiation of the outputs obtained from the operation with that of the MSER mask. By this, the
two consecutive steps. Also this image portrays the
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2581
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2582
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2583
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056
Volume: 04 Issue: 03 | Mar -2017 www.irjet.net p-ISSN: 2395-0072
All the product details such as barcode number, name [7] Rajkumar, N., Anand, M.G., Barathiraja, N. (2014)
Portable Camera-Based Product Label Reading For
of the product, weight, price, manufacturing date and Blind People, IJETT Vol.10, No. 11.
the expiry date are displayed from the database as [8] Sree Dhurga, K. and Logisvary, V. (2015) An Image Blur
shown in the figure18 and audio output is also given. Estimation Scheme For Severe Out Of Focus Images
While Scanning Linear Barcodes, IEEE.
6. CONCLUSIONS
REFERENCES
2017, IRJET | Impact Factor value: 5.181 | ISO 9001:2008 Certified Journal | Page 2584