Car License Plate Recognition

Use Neutral Network

Diep N. Phan, Dai V.Tran

Tuan V. Pham

Electronic & Telecommunication Engineering Department

University of Technology, University of Danang
Danang, Vietnam
ngocdiep10dt2@gmail.com, daitv73@gmail.com

Center of Excellence
University of Technology, University of Danang
Danang, Vietnam
pvtuan@dut.udn.vn

Abstract Car license plate recognition is the extraction of

car license plate information from an image or a sequence of
images. In this paper, we present an improved approach four
steps for Car License Plate Recognition: Preprocessing, Plate
localization, Character segmentation and Character recognition.
Preprocessing stage will improve the quality of the acquired
image, which is a major factor in the success of the Car license
plate recognition. Plate localization stage uses the projection
method. Character segmentation Algorithm uses the Labeling
method. In character type classification, the number of closedarea is used to separate characters into three types and then in
recognition stage three different feed-forward neural networks
are trained to identify characters in each type. Experimental
results show that the proposed approach is robust to a variety of
illumination, view angle, size, and plate type under complex
environments. The performance of the character recognition
stage achieved 99.43% for image high quality.
Keywords Car license plate recognition; Plate localization;
Character segmentation; Character recognition; Neural network;
Image projection; Image label.

I. INTRODUCTION
Image processing techniques have been applied widely
from civil device to specialized equipment. The use of image
processing for license plate recognition will contribute to
solving a part of the problem traffic congestion and automate
some tasks related to the management of cars. Now,
management of transport and general manager automobiles and
motorcycles in particular is extremely complex, as well as the
work of detecting and sanctioning traffic violations, theft... will
spend a lot of time and effort. And then, the demand of
building automation system identification and manage
motorized means of traffic are also born. This system will
reduce the pressure on human resources in management and
control these transport. Through this theme, also create the
premise for develop solutions such as: license plate
recognition, documents recognition.
License Plate Recognition (LPR) is a technology to extract
license number from vehicle image capturing by a single or
multiple cameras. It has various applications in traffic control,
vehicle theft prevention, vehicle surveillance, parking lot
access control, etc. A LPR system typically consists of three
steps: plate extraction, character segmentation and character
recognition.

The LPR system that extracts a license plate number from a

given image can be composed of four stages (Fig 1). The first
stage is to applied plates image, remove noise and improve
images quality [1-4]. The second stage is to extract the license
plate region from the image based on some feature, such as
boundary, the color, or the existence of the characters [3, 8].
The next, individual characters are segmented using connected
component analysis which is simple and straightforward [4, 5].
However, it has been seen that the connected component
analysis method may fail to extract all the characters when they
are joined or broken. Hence, several morphological operations
have been utilized to improve the robustness of character
segmentation. Lastly, a four-stage classifier is employed to
recognize characters. This classifier categorizes plate
characters into three types. The next stage consists of three
different feed-forward neural networks trained to recognize
characters in correspondence with each type above [6-10]. The
reason under the hood of this two-stage structure is to
overcome difficulties stemming from training a single large
neural network. Two training models are also proposed to
improve robustness of the neutral networks: clean model and
noisy model. In addition, three test scenarios are presented to
evaluate performance of the recognition stage.
The remainder of this paper is organized as follows. In
Section II, improves quality of input image. Section III
demonstrates how to extract the license plate region. Section
IV demonstrates character segmentation methods and Section
V discusses character recognition methods. In Section VI, we
summarize the paper and discuss the future research.
Input

Preprocessing
Plate
Localization
Character
Segmentation
Recognition

43A 03246

Fig 1: Block diagram of License Plate Recognition

II. PREPROCESSING
A. Convert RGB image or colormap to grayscale
In Number Plate Detection, the image of a car plate may
not always contain the same brightness and shades. Therefore,
the given image has to be converted from RGB to Gray form.
Converts RGB values to Gray scale values by forming a
weighted sum of the R, G, and B components.
B. Noise filter and dilate an image
However, during this conversion from RGB to Gray form,
certain important parameters like difference in color, lighter
edges of object, etc. may get lost [6]. The process of dilation
will help to nullify such losses. Dilation is a process of
improvising given image by filling holes in an image, sharpen
the edges of objects in an image, and join the broken lines and
increase the brightness of an image. Using dilation, the noise
within an image can also be removed. By making the edges
sharper, the difference of gray value between neighboring
pixels at the edge of an object can be increased. The process of
dilation will help to nullify such losses.
We can use a periodical convolution of the function f with
specific types of matrices m to noise filter and dilate an image:
(

)
)

(
,

)-

Preprocessing

Fig 3: Result after preprocessing.

III. LICENSE PLATE LOCALIZATION

After the series of convolution operations, we can detect an
area of the number plate according to a statistics of the
snapshot. There are various methods of statistical analysis.
One of them is a horizontal and vertical projection of an image
into the axes x and y. In the present work, we use a projection
approach based on gray level computed from vehicle images
for localization of significant license plate regions [6-8].
Advantage of this method is very simple implementation.

Vertical image projection

Let an input image be defined by a discrete function
(
). Then, a vertical projection py of the function at a
point y is a summary of all pixel magnitudes in the yth row of
the input image. We can mathematically define the vertical
projection as:

(1)

As: w, h are width and height of the image represent by the

function .
-represents the element in xi
Note: The expression ,
column and yi row of matrix m.

( )

As:

( )

(3)

w and h are dimensions of the image

Each image operation, filter is defined by a convolution

matrix. The convolution matrix defines how the specific pixel
is affected by neighboring pixels in the process of convolution.
Individual cells in the matrix represent the neighbors related to
the pixel situated in the center of the matrix. The pixel
represented by the cell y in the destination image (fig. 3) is
affected by the pixels x0x8 according to the formula:
y=
(2)
Image input Matrix

Convolution
matrix

x1

x2

x3

m1 m2 m3

x4

x5

m4

x6

x7

x8

m6 m7 m8

m5

Fig 4: Vertical image projection.

Image output Matrix

Fig 5: Detected license plate.
y

Fig 2: The pixel is affected by its neighbors according to the convolution

matrix.

IV. PLATE CHARACTER SEGMENTATION

In this section, we describe our proposed character
segmentation approach which based on binary connected
components detection and a processing chain for character
identification using geometrical constraints of Vietnam license
plates.
Character Segmentation separates each letter or number
where it is subsequently processed by Optical character
recognition (OCR).

A. Preprocessing

Fig.6 Block diagram of processing stage

Input image as depicted in Fig.7a is initially processed to
improve it is quality. In the beginning, the input color images
are transformed into grayscale images using the NTSC
standard method.
It then is converter into binary image as presented in Fig.7b
using Local Adaptive Threshold Method.
(

* (
(

(
(

)+
)
)

b.

First, all connected regions are found and labeled using

connected component analysis [12]. Then all regions, that the
heights and the areas of their bounding box are outliers, will
be removed. Fig.8 shows the intermediate results of this
process.

(4)
}

(5)

The tilted plate image has bad effects on segmentation and

recognition stages as reported in [9]. Therefore, in our
approach, the tilted plate is corrected by rotating an angle
which is estimated by the Hough Transform [10]. The
corrected tilted plate is shown as in Fig.7c.

a.

B. Segmentation
The purpose of this step is to find the individual characters
on the plate. Pixel connectivity and projection profiles are two
popular features for segmenting license plate characters [11].
In the proposed approach, pixel connectivity feature was used
because it is more robust to rotation than projection profiles.
Fig. 7 shows the algorithm of segmentation stage.

a.

All regions are found are labeled using connected

component analysis

b.

All label after coarse segmentation

c.

All character in plate are segmented

The color tilted plate image

The resulted plate image after begin binarized

Fig. 8 Character segmentation

c.

Plate image in Hough diagram.

The remaining objects which are mostly characters will be

resized into 64x32 pixel image.
V. PLATE CHARACTER RECOGNOTION

d.

The tilted plate image after being corrected

Fig. 7 Preprocessing plate

After the segmentation of elements, the final module in the

license plate recognition process is character recognition. For
recognition problem, Multi-layer perceptron (MLP) neural
network is an important in the recognition by [13][14]. In the
paper, we proposed an improved method based on MLP neural
network and back-propagation algorithm for training to
recognize character and number in Vietnam license plates.

A. Character type classification

For recognition problems, (MLP) neural networks are one
of the most common used. In spite of their advantages, neural
networks have some limitations. One crucial limitation is the
difficulty in training a large neural network. For training such
network, it requires a large dataset which may not easy to
collect. Thus, to overcome this issue, the proposed
classification process divides characters into three categories.
Each category will then be recognized using small separated
neural network as shown in Fig. 9.

Fig.9 Block diagram of plate characters recognition stage

In character recognition, feature points are one of the most
useful features. To extract the feature points, character image
is first skeletonized as shown as Fig. 10-a,b. Then the number
of its intersection(s), end-point(s) and closed-areas will be
counted (Fig.11). In our experiment the number of
closed-areas is robust for character type classification while
the other feature points are not good for this task. By using
this feature, characters will be pre-classified into three type
which are type 1 (one closed-area), type 2 (two closed-areas),
type 3 (three closed-areas) as listed in Fig. 12.

B. Feature Extraction
Each character image has large dimension preventing it
from being use as the input feature. However, we can
use dimension reduction techniques to map data to a
lower dimensional space such that uninformative
variance in the data is discarded. There are many
dimension reduction techniques such as principal
components analysis (PCA), projection pursuit (PP),
principal curves (PC), self-organizing maps (SM). In the paper
proposed a hybrid method of principal components analysis
and local binary pattern (LBP). Firstly, PCA extracted the
global grayscale feature of the whole facial expression image
and reduced the data size at the same time. And LBP extracted
local neighbor texture feature of the mouth area, which
contributes most to facial expression recognition. Fusing the
global and local feature will be more effective for facial
expression recognition.

PCA algorithm follows 6 steps:

Step 1: Give

vector representing a set of

sampled images:

Step 2: Compute the average vector

Step 3: Stack the data into n-by-m matrix

are

where the rows

Step 4: Compute SVD of

Step 5: Keep the first 26 rows of
with largest singular
values:
as principal component
Step 6: Project images on this principal component to get 26Dimensional representations:

a.

Segment character
b. Skeletonized character
Fig.10 Segmented character is skeletonized

) (
)
.(
Local Binary Pattern

The LBP is non-parametric operator which describes the

local spatial structure of an image. At a given pixel position
(
), LBP is defined as an ordered set of binary
comparisons of pixel. The resulting LBP pattern at the pixel
can be expressed as follows:
(
a.

Intersection
b. end-point c. closed-area
Fig. 11 Three feature points

corresponds to gray value of the center pixel (

Where

to the gray values of the 8 surrounding pixels, and function

as:

( ) defined

( )

(
(

)
}
)

Using LBP operator the whole image is transformed to LBP

map.
Fig. 12 All characters is divided into three type based on
closed-area feature

LBP Histogram Sequence

We use local feature histogram to present the region

property of the LBP patterns by the following processing: first
each LBP map is spatially divided into multiple non-

overlapping regions, and histogram h is computed from each

region. Then histogram sequence H.
Here the histogram h of image f(x,y) with grey levels in
range [0,L-1] is defined as:
, (

)
(

( )||

||

Step 3: For
set
()

(
is the number of pixels in

Assume the whole is divided into m regions, then the

histogram of the r-th region could be expressed as
(
) and the concatenated histogram sequence
(
).
as
C. Neural network with back-propagation algorithm
In order to assign each digit signature to its corresponding
ASCII representation, two feed forward back propagation
neural network (NNET) were designed, the one assigned to
Type1, one other to Type2 and one other to Type3. The
networks consist of one hidden and one output layer with LogSigmoid transfer function, as show in Fig13

for each node i in layer l ,

Where i is the i-th gray level,

the image with gray level I and
( )

Step 4: Compute the desired partial derivatives, which are

given as:
For i=1 to m
a. Use
backpropagation
to
compute
) and
)
() (
() (
With:
)
() (
)
() (
(
)
)
() (
()
()
b. Set
)
() (
()
c. Set ( )
(
)
()
Step 5: Update the parameters:
()

()

()

()

VI.

[(
[(

()

()

()

)]

EVALUATION

A. Classification measure
In this paper, performance of the proposed recognition
algorithm is assessed via the true positive rate (TPR) and false
positive rate (FPR) which are defined as follow:
,
Fig13. Type1: 62-50-25 NNET Type2: 62-50-8 NNET
Type3: 60-50-2 NNET.
The Type 1 NNET receives a 62x1 vector in the form of
Fig13. It passes through the first log-sigmoid hidden layer
with contains 50 neurons and finally it enter the output logsigmoid layer which contains 25 neurons, leading to vector .
The output 25x1 vector values correspond to the sequence of
Type1 and are between 0 and 1(due to the log-sigmoid
function). It can be said that each value represent the
probability that the input signature is classified to a specific
Type1. The final result is provided through a competitive
transfer function which returns the index with the optimum
value.
The Type2 and Type3 NNET work identically to the
Type1 network, except that it use 8, 2 neurons in the output
layer.
Back-propagation algorithm.
Five steps of training MLP network use back-gropagation
algorithm as following:
Step 1: Perform a feedforward pass, computing the activation
for layers
, and so on up to the output layer
:
(

)
(

() ()
)

where TP, FP, FN and TN are determined as follows:

True positives (TP): The amount of character A

which is correctly recognized as character A.
False positives (FP): The amount of non-A character
which is wrongly recognized as character A.
False negatives (FN): The amount of character A
which is wrongly recognized as non-A character.
True negatives (TN): The amount of non-A character
which is correctly recognized as non-A character.

B. License plate localization evaluation

Database of plate image input

To evaluate performances of the proposed method, a
database containing 127 plates has been built based on
collection of plate images of Viet Nam on Internet. The
database is divided into 6 sets according to the illumination,
view, quantity, contrast, weather, position/angle of orientation
and the images quality condition. The details of this result are
described in Table I.
TABLE I.

DETAILS OF RESULT

()

( ( ))
Step 2: For each output unit i in layer (the output layer), set:

Test
database

Amount

Segmentation

TPR

FPR

Total
Contrast
Low
High
Weather
Rainy
Cloudy
Sunny
Position/ Angle
Straight
Rotation
Projection
Quantity
High
Low
View
Front
Back
Background
Simple
Complex

127

107

84.25

15.75

35
95

27
79

77.14
85.87

22.86
14.13

32
93
25

31
78
23

96.88
83.87
92

3.13
16.13
8

60
39
17

55
32
11

91.67
82.05
64.71

8.33
17.95
35.29

101
15

83
12

82.18
64.52

17.82
20

95
31

86
20

90.53
64.52

9.47
35.48

40
28

34
20

85
71.43

15
28.57

TABLE III.

Set1
Set2
Set3

TABLE II.

Test
database

Set1

Set2

Set3

DATABASE OF PLATES DESCRIPTION

Quantity

Description

183

Clear data with good conditions:

normal lighting condition, nice
weather, no angle of orientation,
black
characters
on
white
background, high quality.

67

Rotated plates with various angles

of orientation and good lighting
conditions.

91

Rotated and projected plates with

large angle of orientation, plates
with bad lighting conditions (too
bright, too dark, night light), bad
quality, blurred, noise, small size.

Evaluation

The result of segmentation step is shown as Table II. The

performance of three scenarios are 96.16%, 76.12% and
61.54% respectively.

Total plate
183
67
91

Segmentation
176
51
56

Ratio
96.17%
76.12%
61.54%

D. Character type classification

1) Character database
Due to the varieties of image's capturing condition which
lead to the differences in size, brightness, and contract of
segmented characters; two training models are proposed as
clean model and noisy model to improve robustness of the
classifier. The clean model is trained on a dataset consists of
characters with good illumination condition and no rotation.
Each character in training set has 20 samples. The noisy model
is trained on a dataset which has 20 good samples and 30
noisy sample per character.
The test data is divided into 3 scenarios:

The Well Matched scenario (WM): The tested samples

of each character are similar to the ones for training.

The Medium Mismatched scenario (MM): This test set

consisting of samples that have relative differences in
lighting condition, fonts and angle of orientation.

C. Segmentation evaluation

Database of plate region

A database containing 341 plates has been built based on
collection of plate images on Internet and plates capturing by
our research team. The database is divided into 3 sets
according to the illumination, weather, angle of orientation
and the images quality condition. The details of this database
are described in Table II.

THE RESULT OF SEGMENTATION STEP

The Highly Mismatched scenario (HM): There are

completely differences in fonts, angle of orientation and
illumination conditions between the training and test samples
of each character.
In this database, the clean training characters and WM test
characters were extracted from Set 1 of plate database
mentioned. MM test characters were extracted from Set 2 and
HM test characters were extracted from Set 3.
Detailed description of the database with two training
models is depicted in Table IV.
TABLE IV.

CHARACTER DATABASE DESCRIPTION

Train
(samples/character)
Clear
Noisy
Clean
model
Noisy
model

Test
(samples/character)
WM MM HM ALL

50

20

20

20

60

50

30

10

10

10

30

2) Evaluation
The test sets consist of WM, MM, HM and ALL
(WM+MM+HM) are used for testing in both training models.
At first, closed-area feature is used for categorizing the type of
characters. The categorizing results of three subsets are shown
in Table 4 (clean model) and Table 5 (noisy model).

a) Clean model
The test set contains of test WM (700 chars), test MM (700
chars), test HM (700 chars) and test ALL (2100 chars) are
used for evaluation the clean model.
TABLE V.

Clean
model

WM

THE RESULT OF TESTING CLEAN MODEL

MM

HM

ALL

T1 T2 T3 T1 T2 T3 T1 T2 T3 T1 T2 T3

Type1
498 2 0 499 1 0 499 1 0 1496 4 0
(T1)
Type2
2 158 0 4 155 1 6 150 4 12 463 5
(T2)
Type3
0 0 40 0 0 40 1 0 39 1
0 119
(T3)
b) Noisy model
The test set contains of test WM (350 chars), test MM (350
chars), test HM (350 chars) and test ALL (1050 chars) are
used for evaluation the noisy model.
TABLE VI.

THE RESULT OF TESTING NOISY MODEL

WM
MM
HM
ALL
Noisy
model T1 T2 T3 T1 T2 T3 T1 T2 T3 T1 T2 T3
Type1
250 0 0 249 1 0 249 1 0 748 2 0
(T1)
Type2
2 78 0 1 79 0 1 78 1 4 235 1
(T2)
Type3
0 0 20 0 0 20 0 0 20 0 0 60
(T3)

HM
ALL

VII. CONCLUSION

ACKNOWLEDGMENT
We would like to give a special thank to Mr. Tuan M.
Nguyen
and
Mr
Anh
Nguyen,
Electronic
&
Telecommunication Engineering Department, Danang
University of Technology, The University of Danang.
REFERENCES

1) Clean model

[3]

[2]

RESULT OF TESTING CHARACTER RECOGNITION FOR CLEAN

MODEL

Clean model

Total

TPR

FPR

[4]

WM
MM
HM
ALL

700
700
700
2100

99.43%
93.98%
85.12%
92.84%

0.018%
0.187%
0.458%
0.221%

[5]

2) Noisy model
RESULT OF TESTING CHARACTER RECOGNITION FOR NOISY
MODEL

Noisy
model
WM
MM

Total

TPR

FPR

350
350

99.43%
98.29%

0.018%
0.054%

0.249%
0.107%

In this paper, we presented about four step of Car License

Plate recognition. We had seen, performance of License plate
localization is not good, which only more 84%, because we
using the projection method to detection, that is very simply,
but efficiency not high. And remaining, performance of
License plate segmentation and Character recognition is very
good, which more 98%. In future, we will try to complete it
better.

[1]

TABLE VIII.

92.00%
96.57%

According to the testing result, we can see that:

When training with clean model, the TPR is high and
FPR is quite low for three scenarios. The TPRs of
WM, MM, HM are 99.43%, 93.98%, 85.12%
respectively and the average TPR of three scenarios
is 92.84%.
When training with noisy model, the classification
performance improves reasonably. The TPRs of MM, HM
scenarios increase approximately 4%.

E. Character Recognition evaluation

After categorizing characters into three types, characters are
recognized by three different neural networks. The average
TPR and FPR of all characters are shown in Table VI (clean
model) Table VII (noisy model).
TABLE VII.

350
1050

[6]

[7]

Badaw, "Automatic License Plate Recognition (ALPR):

A State of the Art Review" Circuits and Systems for
Video Technology, IEEE Transactions on , vol.PP,
no.99, pp.1, 0.
Fodor I. K.;, "A survey of dimension reduction
techniques," LLNL technical report, june 2002.
Zhenhua Guo; Lei Zhang; Zhang, D.; , "A Completed
Modeling of Local Binary Pattern Operator for Texture
Classification," Image Processing, IEEE Transactions
on , vol.19, no.6, pp.1657-1663, June 2010.
Ondrej Martinsky; Algorithmic And Mathematical
Principles Of Automatic Number Plate Recognition
Systems, Faculty Of Information Technology
Department Of Intelligent Systems, Brno University Of
technology, 2007.
Peter Tarbek;, Morphology Image Pre-Processing For
Thinning Algorithms, Journal of Information, Control
and Management Systems, vol. 5, no. 1, pp. 131-138,
2007.
Naikur Bharatkumar Gohil; Car License Plate
Detection; B.E., Dharmsinh Desai University, India,
2006.
Lu Liu; Hongjiang Yu; Kehe Cai; Jia Wang, "License
plate
recognition
using
topology
structure
features" Computing, Control and Industrial Engineering
(CCIE), 2011 IEEE 2nd International Conference on ,

vol.2, no., pp.251-254, 20-21 Aug. 2011.

Trng Quc Bo, V Vn Phc; Gii thut mi cho
bi ton nh v v nhn dng bin s xe t; Tp ch
khoa hc Trng i hc Cn Th; Phn A: Khoa hc
T nhin, Cng ngh v Mi trng: 27(2013):44-45.
[9] Zhenhui Zhang; Shaohong Yin, Hough Transform and
Its Application in Vehicle License Plate Tilt Correction
Computer and Information Science, vol. 1, pp. 116-119,
Aug. 2008.
[10] Xiaodan Jia; Xinnian Wang; Wenju Li; Haijiao Wang; ,
"A Novel Algorithm for Character Segmentation of
Degraded License Plate Based on Prior Knowledge"
Automation
and
Logistics,
2007
IEEE
International Conference on , vol., no., pp.249-253, 1821 Aug. 2007.
[11] Badawy, W, "Automatic License Plate Recognition
(ALPR): A Stateof the Art Review" Circuits and Systems
for Video Technology, IEEE Transactions on , vol.PP,
[8]

no.99, pp.1, 0. [4] Ni Ma; Bailey, D.G.; Johnston, C.T.; ,

"Optimised single pass connected components analysis,"
ICECE Technology, 2008. FPT 2008. International
Conference on , vol., no., pp.185-192, 8-10 Dec. 2008.
[12] B. Kroese. An Introduction to Neural Networks,
Amsterdam, University of Amsterdam, 1996, 120 p.
[13] FEITOSA, R. Q., VELLASCO, M. M. B., MAFFRA, D.
V., ANDRADE, S. S. R. S., OLIVEIRA, D. T., Facial
Expression Classification Using RBF and Back
Propagation
Neural
Networks,In:4th
World
Multiconference
onSystemics,
Cybernetics
and
Informatics (SCI'2000) and the 6th International
Conference on Information Systems Analysis and
Synthesis (ISAS'2000), 2000, pp.73 77.
[14] Chun Wang, Bo Liu, Xinzhi Zhou, Research on vehicle
plate character recognition based on BP neural network,
China Measurement Technology, 2005, vol.31, pp. 26-28
(inChinese).