Professional Documents
Culture Documents
Team Members
Max Guise
Electrical Engineering
Technical Staff, Sandia National Labs
Stephan Goupille
MS&E, Electrical Engineering Growth Analyst, Facebook
Project Aims
Image
Balancing
First, the several color transformations and balancing schemes will be examined, including adaptive histogram equalization and using the different color spaces (CIELAB).
Plate Extraction
Image patches that contain license plates will be extracted, based on morphological operations, oriented gradients, edge detection, filtering, and erosion.
Finally, character recognition will be performed on these regions of interest and decision criteria will be devised to choose when to Character output a result. Recognition
Tools
Motorola Droid Phone
Great for snapping a quick picture Originally thought RGB values could be exported directly (function hasnt been implemented)
MATLAB
One of the best, fastest mathematical analysis tools Many built-in image processing functions
Our goal to accomplish task using basic properties of license plates no machine learning necessary
Methods
Pre-Processing
KMeans
Image Transformation
Histogram of Oriented Gradients Harris Corners Hough Transform Edge Detection Filtering
Decision-Making
Thresholding Erosion
KMeans
Unsupervised Nearest Centroids in 5-Space (X,Y,R,G,B)
Variable number of centers can be chosen Can weight X,Y differently from R, G, B
KMeans
Issues:
Optimal number of centers to use varies with size of license plate
As a result, cant guarantee good separation of plate from bumper, or might split license plate region into multiple parts
Histogram Equalization
Image pixels may be far from uniformly spread across the display range
Changing the distribution can add contrast
Original
6 5
x 10
Equalized
10
x 10
Adaptive
4 6 3 4 2 2 2 4 6
100
200
300
100
200
300
100
200
300
Histogram Equalization
200 400 600 800 1000 1200 1400 1600 1800 500 1000 1500 2000 2500
10
Histogram Equalization
200 400 600 800 1000 1200 1400 1600 1800 500 1000 1500 2000 2500
11
Histogram Equalization
200 400 600 800 1000 1200 1400 1600 1800 500 1000 1500 2000 2500
12
13
Since we have data in two directions, can calculate a direction for each pixel
Biggest problem here is that edges of plates themselves not always well defined
Illumination changes, license plate holders, glare, etc.
Bottom line: need machine learning technique here to make this work
14
Our idea was to search the detected Harris corners for rectangles of approximately the right aspect ratio
Turned out to be an incredible computational burden with the detector sensitivity high enough to reliably detect license plate corners
100 200 300 400 500 600 700
10 20 30 40 50 60 70
800
80
900 200 400 600 800 1000 1200
20
40
60
80
100
120
140
Hough/Radon Transform
Canny edge detection to get edge map For each edge pixel, record the set of all lines that could pass through it
Parameterized by distance from origin and angle
15
Resulting image has peaks at (,) pairs representing prominent lines in the image
Idea was to search for peaks corresponding to the approximate aspect ratio of license plates
Hough/Radon Transform
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
16
Simplest is best
Underlying insight is that license plates have a lot of vertical edges (letters) and a specific aspect ratio
Take horizontal gradient Filter with matched filter Designed with aspect ratio, approximate license plate size in mind Threshold Bit of an art ties in closely with the erosion
17
Erode
Region label Throw out regions that are too small Cascaded false positive rejection
Rely on context of license plate (car bumper, license plate holder etc.)
Horizontal Gradient
100
18
200
300
400
500
600
700
800
900
200
400
600
800
1000
1200
200 400 600 800 200 1000 400 1200 600 1400 800 1600 1000 1800 1200
1400 1600 1800
Matched Filtering
19
1 0.5 0 -0.5 -1 40 30 20 10 0 0 20 40 60
500
1000
1500
2000
2500
500
1000
1500
2000
2500
100 200 300 400 100 500 200 600 300 700 400 800 500 900 600
700 800
200
400
600
800
1000
1200
20
21
Conclusions
Detection without learning = hard
Lots of issues: background clutter, contrast, scale/rotation invariance, computation, etc.
22
Next steps would be window refinement, exploration of scale and rotation capabilities (and, of course, character segmentation and recognition) Thanks!
References
Dalal, N. and Triggs, B. 2005. Histograms of Oriented Gradients for Human Detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cvpr'05) - Volume 1 - Volume 01 (June 20 - 26, 2005). CVPR. IEEE Computer Society, Washington, DC, 886-893. Vahid Abolghasemi, Alireza Ahmadyfard. An edge-based color-aided method for license plate detection. Image and Vision Computing, Volume 27, Issue 8, 2 July 2009, pp. 1134-1142. Stokman, Harro and Gevers, Theo. Selection and Fusion of Color Models for Image Feature Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 29.3 (2007): 371-381.
23
Questions?
24