Efficient Decoding of Blurred, Pitched, and Scratched Barcode Images

J. P. Liyanage
Zone24x7 Inc janakal@zone24x7.com

Abstract - While there are number of domain transformation based image processing algorithms for decoding barcode images, these algorithms are computationally intensive and thus not suitable for embedded or online applications. These algorithms are better implemented on desktop or server systems for decoding images offline. In this paper, we discuss a camera independent barcode image decoding algorithm suited for embedded real-time applications like barcode scanning in Point-Of-Sales terminals. While the algorithm is ideal for a charge-coupled device (CCD) type scanner with a camera module, it can also be used to decode images taken from a distance (as in laser scanners) and is robust against most of the common distortions caused by low cost cameras/lenses.

I. INTRODUCTION With the advent of the microprocessors and digital cameras, using the digital images captured from cameras for many non-traditional applications has seen a sharp rise. One such application is using smart cameras for barcode decoding, where a digital image is processed by a microprocessor to obtain the data coded in the barcode. Since prices of both digital cameras and microprocessors are going down in a rapid pace, such an apparatus can effectively replace high cost traditional barcode scanners. However, processing barcode images introduces lot of challenges since the images can be blurred, rotated, or distorted. A trivial scan line based approach may not work for most of the images and number of people has proposed approaches that are more sophisticated. Given an image containing a barcode, finding the data string encoded in the barcode is usually two fold: Searching the image to find the barcodes called barcode localization Decoding the barcode and extracting the information contained called decoding (reading) In this paper, our focus would be on the barcode decoding or reading phase, where we have introduced a simple yet powerful algorithm to decode most types of 1D barcodes.

II. BACKGROUND There are two fundamental types of one-dimensional barcodes; width code and delta code [8]. In delta codes, we divide the available interval into modules and assign either

1s (where the modules are painted to form bars) or 0s (forming spaces). In width codes, we assign 1 to a narrow bar or space and 0 to a wide bar or space. We can have either just two different widths or number of different widths (usually a power of two). If we had four different widths we can assign two bits (e.g. 00, 01 etc) to each bar or space, encoding more information. The barcode decoding is made complicated by various distortions found on captured barcode images. The distortions may be either present on the original barcode (on paper) or introduced in the capturing stage by the digital camera. The distortions in the original barcode include ink spread, scratches, fading, folded papers, and curved surfaces. The distortions introduced by the capturing phase include reflections, skewing, pitching, blurring, noise, and rotations. There are number of barcode decoding algorithms developed my many scientists ranging from those that uses basic thresholding to advanced edge detection in the spatial domain and Fourier transformation in frequency domain. Rotations are one common problem that affects most of the decoding algorithms negatively. Most of the barcode algorithms work only on horizontally oriented barcodes, requiring them to use an additional algorithm to correct the rotations in the image. Detecting the barcode orientation has been implemented using number of methods. A simple approach is discussed by Mikio, where the relative position of corners of the barcode is used to determine the orientation [4]. Another approach proposed by Eisaku is to use the spiral method to find a bar and then use edge detection to find the orientation of the edge [7]. Noise and scratches are some other major causes, which make decoding complicated. Donald explains a method, which uses Fourier transformation and putting together number of images to reduce the effect of noise and increase the resolution of the barcode images [2]. Super-resolution is a method based on the idea of using the information contained in the whole 2D image to create a single high resolution 1D signal. A barcode reader based on this principal of projecting all the pixels in the barcode image to a single high resolution line is presented by Todd [10]. Some decoding algorithms like the one presented by Rusen are forced to find a predetermined number of bars in the image [9]. While this would substantially simplify the

algorithm and will work very well on symbologies like EAN13, it will not work on many popular symbologies like CODE39 or CODE128 as they support any number of characters and thus contain variable number of bars. Most of the best decoding algorithms are based on transforming the original image information to a frequency domain, such as approaches based on the Fourier transformation or the Hough transformation as proposed by Muniz [5]. These approaches are often used in professional image recognition software running on desktop computers, as they offer very good recognition rates. However, their requirements in terms of system resources can be too demanding for typical mobile devices and thus not suitable for real time barcode decoding in mobile devices [1]. An alternative to costly domain transformation is proposed by Robert [1]. It is a much simpler approach based on scanlines, which try to detect the bar code along a particular line through the image. Though such algorithms need much fewer computing resources making them suitable even for mobile camera phones, they suffer from poor recognition rates specially when dealing with dirty surfaces, reflections, shadows, or slight misalignments. Though some of these problems can be alleviated by extensive use of multiple scanlines, it substantially increases the complexity of the algorithm. In addition, they also need barcode localization to be carried out prior to decoding, in order to properly align the scanline. Because of uncertainties about the contrast and problems with ambient illumination, it is advisable to look for changes in the intensity of the reflected light rather than the absolute level of the light (despite of automatic gain controls). Most bar code systems use edge detection or highly adaptive thresholding techniques that look at the slope of the waveform produced when a bar code is scanned. While the ideal signal would be a set of rectangular pulses, the real signal has a rounded form because of convolution distortion [8].

when used with non-maximum suppression is somewhat robust in identifying bar positions in blurred barcode images as well. Normand proposes a method based on second derivative edge detection and blur modeling [6]. Eugene has given an effective method for blurred images using waveform peak location detection [3] as opposed to the point of greatest slope (NMS). This method significantly reduces misclassification for blurred images. Narrow bars are more susceptible to severe distortions since lesser number of pixels is used to represent them in the captured image. For this reason, it is always advisable to magnify the image using zoom lenses or macro lenses when the barcode is of high density. III. METHODOLOGY Our approach to barcode decoding is primarily based on edge detection. The requirement for the algorithm to be simple and efficient prevented us from going for a complex algorithm based on domain transformations. However, it was required that the algorithm is robust against most of the common distortions occurring when the barcode images are captured from a low cost camera in a mobile device. The predominant distortion in images captured from mobile devices is blurring, which occurs due to focusing problems in the camera. Blurring can be very severe that it merges two bars together or severely changes the widths of bars in a barcode; virtually making is impossible for the decoding algorithm to extract the information. In general, our recognition algorithm is also scanline based, in the sense that it applies edge detection mask along horizontal scanlines. In order to improve robustness, we are using a set of multiple scanlines. If multiple scanlines cross the bar code, each at a different position in the barcode, we can increase the robustness by combining the results from multiple scanlines in a majority-voting fashion. This way an inaccuracy due to dirt, noise or reflections on one line can be compensated by two or more correct identifications on other lines. Finally yet importantly, by using a variable amount of scanlines, we have a simple mechanism to adapt our algorithm to the processing power of the individual device it is running on. The algorithm combines information from other scan lines corresponding to the missing information, and computes the best fit. Even if a bar code is scratched diagonally from one end to the other, this algorithm is able to find the correct barcode, where a scan line bar code reader (like a laser scanner) will fail. The algorithm works in four stages: 1. Identify the alternating black and white bars in the image using edge detection 2. Calculate the widths of these bars 3. Identify the barcode Symbology from bar widths 4. Decode the bar widths resulting in text data

Fig. 1 The thin lines show the ideal waveform from a scan, while the dotted line shows a possible output from a sensor [8]

Edge detection is a popular method as it can identify the orientations of individual bars as well. Edge detection

The major workload is done in the first and last stages of the algorithm. The second and third stages were added to make it easier to understand. The Fig. 2 below elaborates on the inner workings of the first stage.
Manually localized barcode image Edge Detection on Horizontal Scan Lines

Calculate Mid point of thick edges

Determine angle of the first bar

Remove Noisy edge points

Find the next best bar Determine the angle of this bar

Fig. 2: Sub modules of stage 1 - determining the positions of individual bars

Since the algorithm does not implement the barcode localization part, the input image should contain only the barcode. A simple charge-coupled device (CCD) type barcode scanner, which requires the scanner to be in nearcontact with the printed barcode while capturing the image, results in images suitable for this algorithm (without any localization). A simple localization algorithm can be used to obtain a cropped image in all other cases, where the barcode will not fill the entire image area. As for the edge detection, since we are only interested in near vertical black lines, we are using a mask, which detects vertical edges. This mask was derived from firstderivative based Sobel operator. The masks, which are convoluted with the images, are given in Fig. 3.

in thick edges in the edge detection phase, which are problematic in identifying the width of the bars. While finding the point of greatest change (NMS) can eliminate thick edges in a normal edge detection applications it may not be suitable for the bar width identification. Therefore we are calculating the mid point of thick edges by taking the position where the intensity is halfway between the highest and lowest. The efficacy of this method over NMS in identifying widths of blurred bars is illustrated in Fig. 4. Next step is to remove the edge points created due to various noise characteristics of the camera sensors. The magnitude of an edge is calculated to be the difference between highest and lowest intensity values. Any edge whose magnitude falls below an empirical value (which depends on the camera sensor), would be regarded as a noisy edge and discarded. The algorithm assumes that individual bars would be straight lines and tries to match a set of edge points along a near-vertical straight line to an edge of a bar. Due to this assumption, the algorithm fails if the bars are severely curved. However, preliminary test results show that the algorithm decodes barcodes with moderately curved bars successfully. In order to determine the equation of a potential straight line to match with a bar, we first need an approximate value for the gradient of the line. The orientations of consecutive bars in a barcode do not change by much, making the gradient of the previously matched straight line a good candidate for this. However, for the first bar (since we do not have a previous bar) we need a different method for calculating the orientation. A simple method of determining the orientation would be to use an edge detection operator. However, this method is not robust in case of distorted barcode images, and thus we find the orientation by matching a regression line to the set of first edge points from each of the scanlines. After finding an approximate gradient for a straightline to match with the edge points, straightlines at various offsets are matched with the set of closest edge points to that line. The matched straightline is the one that is closest to its closest edge points along each scanline. This is illustrated in Fig. 5.
NMS identified point Mid point of intensity

-1 -2 -1

1 2 1

-1 -2 -1

0 0 0

1 2 1

Fig. 3 2x3 and 3x3 Sobel masks used to detect vertical edges

Blurred bars results in gradual changes in intensity values opposed to sudden changes as in sharp images. This results

Fig. 4 NMS identifies the point of greatest slope, which is highly dependent on the shape of the waveform

Edge point closest to the straight line

Matched line

Calculate the widths of the bars

Horizontal scanlines

Another line with the same gradient but different offset

Determine the type of the barcode

Fig. 5 Matching straighlines with the same gradient but different offsets to edge points

Perform CODE39 decoding

Perform EAN13 decoding

After finding the matched straightline, the associated edge points are matched again with a regression line to find out the exact gradient of the line created by this set of edge points. This is required since orientations of bars may change gradually from one end of the barcode to the other. This new gradient is used to match a straightline to the set of edge points constituting the next bar. This process continues from the left end of the barcode to the right end of the barcode and stopped when sufficient number of edge points can not be found to be matched with a line. The set of matched straightlines are sent as output from the first stage to the second stage as input. The stages after the first stage are illustrated in Fig. 6. The second stage of calculating the widths of the bars is straightforward. The widths are derived from the horizontal offset difference of two consecutive straightlines. The third stage of identifying the symbology of the barcode is simple since only two different types of symbologies are supported at this time. The program currently supports only Code39 (a width code) and EAN13 (a delta code). The number of bars identified in the first two stages can be used to determine the type of the barcode without any ambiguity in this case. However as support for more symbologies are added ambiguities will occur with this approach and in such cases we will have to decode the bar widths according to the rules of two for more symbologies to determine the correct type.

Check Digit calculation

Fig. 6 Stages in the decoding the barcode after the first stage of identifying the bars

The fourth stage of decoding the bar widths is non trivial as widths can change substantially due to distortions such as blurring. It is observed that blurring has the most negative effects in changing the widths from the original values. A Decoding CODE39 CODE39 symbology, as a width code of just two different widths is relatively simple to decode, since even with moderate width changes it is easier distinguish between the two categories, narrow and wide. Since each character is represented by nine bars (6 narrow, 3 wide), a set of nine consecutive bar widths are selected and sorted. In the sorted list, if the difference between the 6th and 7th widths is more than a specified threshold value, the bar widths are matched with the CODE39 decoding table to find out the representing character. Otherwise, the algorithm returns as unsuccessful. This is illustrated in Fig. 7.

x>T

x<T
Fig. 7 Sorted bar widths in a line; can be decoded (top); cannot be decoded (bottom)

This method prevents incorrect decoding which can occur due to substantial blurring. Since the decoding is carried

out character at a time without defining a global threshold to categorize the narrow and wide bars, local changes in the widths due to curved (horizontal) barcodes or lens distortions do not affect the decoding. B Decoding EAN13 Decoding EAN13 bar widths is manly concerned about categorizing bar widths into one if four possible classes. Since we have four different widths as opposed to just two in CODE39, blurring and other distortions, which affect the bar widths, reduce the decoding rate substantially. The three thresholds required to classify the bar widths into four classes, are calculated as follows: Sort the bar widths in ascending order Let the ordered list be W, where Calculate a new

Wi , i [1, N ]
list

(W Wi ) Z i = i +1 , i [1, N 1] Wi where
Calculate

Z,

(W + Wi ) AvgWi = i +1 , i [1, N 1] 2

new

list

AvgW,

where

Sort the AvgW list taking Z as the keys to sort it in the descending order Now the first three elements in the AvgW list are taken as the global threshold to classify the bar widths, such that

Fig. 8 The main window of the prototype barcode decoder decoding a skewed barcode image

For each width w in W if (w AvgW1 ) then w C 4 else if (w AvgW2 ) then w C 3 else if (w AvgW3 ) then w C 2 else w C1
where C1, C2, C3 and C4 are the narrowest to widest classes in that order. Since global thresholds are used to categorize the bars, curved barcodes or lens distortions will make barcode hard to decode. A different method which does adaptive thresholding should be found in order to improve the robustness for these cases. IV. RESULTS The prototype algorithm was written in C# using .NET framework 2.0. The programs main window is shown in Fig. 8.

The algorithm was tested with various barcodes imaged under various distortions and the results surpassed the original expectations of the algorithms decoding rate. Following are some of the barcode images that were decoded successfully.

Fig. 9 faded EAN13 code, the algorithm compensates the faded regions from good ones

Fig. 10 scaled image of a CODE39 with reflections

Fig. 11 CODE39 on crumpled paper

Fig. 12 CODE39 on vertically curved surface - like on a marker pen

Since our algorithm assumes that each individual bar in the barcode have different yet associated orientations,

skewed and pitched barcodes can be decoded successfully. The algorithm is also robust against barcodes with scratches and reflections as long as these distortions do not cover a complete bar. Following are some of the images, which failed to decode.

distortions. Replacing empirical constants with dynamic values derived from the attributes of the image is one. Another is to replace the global thresholding with adaptive thresholding so that various local distortions will not have an effect on the decoding. A Fuzzy based algorithm can be used for finding a matching character for a set of bar widths. Currently this matching is implemented as a simple table lookup. ACKNOWLEDGMENT

Fig. 13 A blurred image of EAN13, taken without properly focusing the camera

I want to thank the management of the Zone24x7 Inc, who provided the necessary guidance and resources to start and continue the project from start to end. Special thanks go to Manjula Dissanayake and Llavan Fernando for the encouragement provided in the same. REFERENCES
[1] R. Adelmann, M. Langheinrich, C. Flrkemeier, Toolkit for Bar Code Recognition and Resolving on Camera Phones Jump Starting the Internet of Things. Institute for Pervasive Computing, ETH Zurich, 2006 D. G. Bailey, Super-resolution of bar codes, Institute of Information Sciences and Technology, Massey University, New Zealand. Journal of Electronic Imaging / January 2001 / Vol. 10(1)/213. E. Joseph, T. Pavlidis, Bar Code Waveform Recognition Using Peak Locations. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 6, June 1994. M. Kuroki, T. Yoneoka, T. Satou, Y. Takagi, T. Kitamura, N. Kayaniori, Bar-code Recognition System Using Image Processing, Hitachi Process Computer Engineering, Inc, 1990. R. Muniz, L. Junco, A. Otero, A robust software barcode reader uses the Hough transform, in Proceedings, International Conference on Information Intelligence and Systems, pages 313 319, 1999. N. Normand, C. Viard-Gaudin, A two-dimensional barcode reader,12th International Conference on Pattern Recognition, vol. 3, pp. 201-203, October 9-13, 1994. E. Ohbuchi, H. Hanaizumi, L. A. Hock, Barcode Readers using the Camera Device in Mobile Phones, in Proceedings, International Conference on Cyberworlds,2004. T. Pavlidis, J. Swartz, Y. P. Wang, Fundamentals of Bar Code Information Theory, Symbol Technologies, 1990. O. Rusen, O. Levent, A Super-resolution Approach for Bar Code Reading, Electrical and Electronics Engineering Department, Atilim University, Turkey,2002.

Fig. 14 EAN13 on a horizontally curved surface

In Fig. 13 the image is blurred severely such that consecutive narrow bars have merged to form a wide bar. Barcodes like this cannot be decoded without a technique like blur modeling which is computationally intensive and not implemented in our algorithm. When the barcode is printed on a horizontally curved surface, the bars in the middle of the barcode become wider than those in the edges like in Fig. 14, making it impossible to classify the bars using a global threshold. A Focus Requirements The algorithm works optimally when distance between the barcode and the camera is 5 to 10 centimeters and that the camera is properly focused onto the barcode. If the barcode X dimension is small, camera can be brought closer to the barcode and change the focus accordingly to obtain an adequate sized image. B Resolution Requirements The resolution requirement is such that each bar should be at least one pixel wide. However due to camera noise and focus issues, an ideal bar with one pixel wide is not practically obtainable. It is observed that a barcode with its narrowest bar being approximately two pixels wide is easily decoded by our algorithm. Depending on the length of the barcode, the resolution required by the camera to capture an adequate sized barcode image changes. V. CONCLUSION The simple yet powerful algorithm for barcode decoding presented in this paper can be used to decode barcode images in devices with limited processing power. A simple barcode scanner can be implemented by porting the algorithm into an Advanced RISC Machine (ARM) processor and interfacing a low-cost camera module to the same. It would take less than one second for a barcode image in full VGA resolution (640x480) to be decoded in a 266Mhz ARM processor (e.g. FreeScale MX21). Number of improvements can be done in order to increase the decoding rate of the algorithm, under various

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10] T. Wittman, M. Trajkovi, Super-Resolution of 1D Barcode Images, University of Minnesota, Twin Cities, 2004.