Number plate recognition using enhanced segmentation and boosting algorithm
Hello all!
It’s been quite a long time since I’ve written anything in
the blog so I thought it’s about time I should share a project I recently made
with a friend of mine from college.
First of all we should see what this project does and why I
made it.
The project is used to change an image of a car with the
number plate visible into a string of characters. There are similar projects
that have already been made and implemented in certain countries but it is
especially difficult to implement in India due to a large variety of plates and
due to fact that the number plate rules are rarely followed in India.
Motivation for this project stemmed when a friend of mine
lost the receipt for parking in a mall and had to pay more than 4 times for
parking.
So me and my friend thought to make the project with special
measures to make the recognition of numbers easier for Indian plates.
We made this project in MATLAB with the help of image processing
and image acquisition toolbox
The
project can be thought of as six blocks:
- Image retrieval block that consists of a camera
- Pre-processing
- Number plate localization
- Number plate segmentation
- Optical Character Recognition(OCR)
- Text output
Image Retrieval Block
This block consists of a camera. This
camera can be a CCTV camera, a dedicated camera for NPR or any other camera
system. If CCTV cameras are used the current surveillance system can be used to
perform the recognition with no extra cost of having to install extra cameras.
Dedicated NPR cameras also sometimes use illumination (Infrared Radiation) to
counter the effects of low lighting conditions and reflections.
The ideal camera for this system should
have a high shutter speed and high resolution so that various restrictions to
the system such as motion blur can be removed altogether. Although our system
attempts to reduce the dependence of the camera on the resolution and the
blurring of the pictures by pre-processing the image so as to remove these.
For testing purposes we used an image acquired from the internet
PRE-PROCESSING
Pre-processing of the image is done so
as to make the image ready for further processing. In our system the
pre-processing block reduces noise in the retrieved image by applying a median
filter and it also adjusts the contrast of the image so as to have better
results in conditions where there is less than adequate light for the image to
be of good quality.
The image is also re-sized to a
resolution so as to reduce the processing time for larger images, The
resolution is chosen to reduce processing time without compromising the ability
of the system to recognize the characters.
The image is first converted to a
grayscale image. Then a median filter in applied so as to reduce the salt and
pepper noise in the image which is the random introduction of black and white
pixels in the image. The image is then adjusted for contrast and converted to
black and white.
LOCALIZATION OF IMAGE
The localization of the image is a
process whereby the image is processed so that only the number plate of the
vehicle is visible in the image. It is an important phase in the detection of
the numbers.
There are various methods that can be
applied for finding the number plate. Some of the methods include the
recognition of the rectangular boundary of the number plate. The method used by
us is based on the fact that the alphanumeric characters in the number plate
will have a large number of vertical edges and thus the same can be used to
localize the number plate.
The vertical edges are detected using
‘sobel’ edge detection. There are other edge detection techniques like ‘canny’ but
they are not suitable for our purposes.
The vertical edges in the image are
detected and the image is dilated to make the edges more prominent.
We assume that the number plate will
have larger white region on account of the numbers having a large number of
vertical edges. We make two projections of image, horizontal and vertical.
These projections show the row-wise and column-wise mean of the pixel values in
the corresponding row or column.
We use an averaging filter to make the histogram
smooth so as to reduce peaks caused by little areas of white and to normalize
the values. The value for averaging was set using trial and error. A threshold
value was then selected based on the normalized values and all the values below
this threshold were made zero. This process was performed for both horizontal
and vertical projections.
Two masks are created, one horizontal
and one vertical for the image. These masks are created such that the values
that are non-zero in the projection remain white and other are zero (indicating
black). The size of the image is same as the original image.
These masks are then applied to the
image. After the masks are applied, only the number plate remains in the image.
Since the threshold values vary from image to image some images may contain
some other areas may be visible in the image. To remove these areas, any areas
in the image less than a certain size are filled with black. This removes small
objects that represent noise for our purposes.
As a result of all these processes the image now
contains a region that has the number plate and the rest of the area is black.
Hence the plate is localized and ready for segmentation.
Segmentation
Segmentation is the process by which we
separate each number or letter in the number plate so as to process them
further one by one. There are different methods that can be used for the
achieving the segmentation process. Our approach is to create bounding boxes
such that the areas containing connected objects are separated, since a letter
or a number would represent a connected object, a bounding box is created
around each object.
To prevent small or too large connected
objects to be part of a bounding box a condition regarding the minimum size and
the maximum size of a bounding box are defined. These conditions were found by
trial and error to select the optimum values so that only the numbers and
letters are selected and nothing else.
Once these bounding boxes are created
sub images are created from the black and white image corresponding to each
number or letters. These sub images are used in the further processing of the
images to find out the text equivalent of the image.
OPTICAL CHARACTER RECOGNITION
Optical Character Recognition is the
process of converting the images into corresponding ASCII characters. Various
methods can be used for applying OCR. Our approach uses correlation to find the
probability of the character it likely is.
Correlation is a process that tells us
how similar two signals are, these images in matrix form can also be used to
perform correlation and determine what the image represents in alphanumeric
form.
A previously created set of images for
all the alphanumeric characters (A-Z, 0-9) is used and the results of the
bounding box analysis are correlated with these images. The maximum correlation
is likely to be with the letter that resembles the one in the image. Hence the
character is selected and this process is repeated for all the characters in
the number plate and one by one all the characters are recognized.
The result of the OCR is a string of
characters and is to be stored in a format that can easily be viewed and is
also usable by other applications that would require using the results. For the
purposes of this project the results are stored in a text file that is named
“text.txt” In the working directory of the project and is opened once the
computation is finished by the main program for the viewing of the user.
The conversion of image to text provides
a very high compression ratio. An image file containing the same information
would be of the size of a few hundred kilo bytes but the text file containing
the same information will be only of a few bytes and hence can be much easily
transferred to a remote location.
The text file is appended every time
such that the record of the previous transactions is maintained.
You can find all the files and code used in this post here.Don't get scared seeing the matlab ".m" file most of the code in the starting is from the GUI so you won't have to write any of that yourself. There are comments included in the file but if you do not understand a command or what it does you can see the help in MATLAB 2011b or you can refer to the online MATLAB documentation.
