Number Plate Recognition

Number plate recognition using enhanced segmentation and boosting algorithm

Hello all!

It’s been quite a long time since I’ve written anything in the blog so I thought it’s about time I should share a project I recently made with a friend of mine from college.

First of all we should see what this project does and why I made it.

The project is used to change an image of a car with the number plate visible into a string of characters. There are similar projects that have already been made and implemented in certain countries but it is especially difficult to implement in India due to a large variety of plates and due to fact that the number plate rules are rarely followed in India.

Motivation for this project stemmed when a friend of mine lost the receipt for parking in a mall and had to pay more than 4 times for parking.

So me and my friend thought to make the project with special measures to make the recognition of numbers easier for Indian plates.

We made this project in MATLAB with the help of image processing and image acquisition toolbox

The project can be thought of as six blocks:

1.   Image retrieval block that consists of a camera
2.  Pre-processing 
3.  Number plate localization 
4.  Number plate segmentation 
5.  Optical Character Recognition(OCR) 
6.  Text output

 

Image Retrieval Block 

This block consists of a camera. This camera can be a CCTV camera, a dedicated camera for NPR or any other camera system. If CCTV cameras are used the current surveillance system can be used to perform the recognition with no extra cost of having to install extra cameras. Dedicated NPR cameras also sometimes use illumination (Infrared Radiation) to counter the effects of low lighting conditions and reflections.

 The ideal camera for this system should have a high shutter speed and high resolution so that various restrictions to the system such as motion blur can be removed altogether. Although our system attempts to reduce the dependence of the camera on the resolution and the blurring of the pictures by pre-processing the image so as to remove these.

For testing purposes we used an image acquired from the internet

 

PRE-PROCESSING

Pre-processing of the image is done so as to make the image ready for further processing. In our system the pre-processing block reduces noise in the retrieved image by applying a median filter and it also adjusts the contrast of the image so as to have better results in conditions where there is less than adequate light for the image to be of good quality.

The image is also re-sized to a resolution so as to reduce the processing time for larger images, The resolution is chosen to reduce processing time without compromising the ability of the system to recognize the characters.

The image is first converted to a grayscale image. Then a median filter in applied so as to reduce the salt and pepper noise in the image which is the random introduction of black and white pixels in the image. The image is then adjusted for contrast and converted to black and white.

LOCALIZATION OF IMAGE

The localization of the image is a process whereby the image is processed so that only the number plate of the vehicle is visible in the image. It is an important phase in the detection of the numbers.

There are various methods that can be applied for finding the number plate. Some of the methods include the recognition of the rectangular boundary of the number plate. The method used by us is based on the fact that the alphanumeric characters in the number plate will have a large number of vertical edges and thus the same can be used to localize the number plate.

The vertical edges are detected using ‘sobel’ edge detection. There are other edge detection techniques like ‘canny’ but they are not suitable for our purposes.

The vertical edges in the image are detected and the image is dilated to make the edges more prominent.

We assume that the number plate will have larger white region on account of the numbers having a large number of vertical edges. We make two projections of image, horizontal and vertical. These projections show the row-wise and column-wise mean of the pixel values in the corresponding row or column.

We use an averaging filter to make the histogram smooth so as to reduce peaks caused by little areas of white and to normalize the values. The value for averaging was set using trial and error. A threshold value was then selected based on the normalized values and all the values below this threshold were made zero. This process was performed for both horizontal and vertical projections.

Two masks are created, one horizontal and one vertical for the image. These masks are created such that the values that are non-zero in the projection remain white and other are zero (indicating black). The size of the image is same as the original image.

 

These masks are then applied to the image. After the masks are applied, only the number plate remains in the image. Since the threshold values vary from image to image some images may contain some other areas may be visible in the image. To remove these areas, any areas in the image less than a certain size are filled with black. This removes small objects that represent noise for our purposes.

 

As a result of all these processes the image now contains a region that has the number plate and the rest of the area is black. Hence the plate is localized and ready for segmentation.

  Segmentation

Segmentation is the process by which we separate each number or letter in the number plate so as to process them further one by one. There are different methods that can be used for the achieving the segmentation process. Our approach is to create bounding boxes such that the areas containing connected objects are separated, since a letter or a number would represent a connected object, a bounding box is created around each object.

To prevent small or too large connected objects to be part of a bounding box a condition regarding the minimum size and the maximum size of a bounding box are defined. These conditions were found by trial and error to select the optimum values so that only the numbers and letters are selected and nothing else.

Once these bounding boxes are created sub images are created from the black and white image corresponding to each number or letters. These sub images are used in the further processing of the images to find out the text equivalent of the image.

  OPTICAL CHARACTER RECOGNITION

Optical Character Recognition is the process of converting the images into corresponding ASCII characters. Various methods can be used for applying OCR. Our approach uses correlation to find the probability of the character it likely is.

Correlation is a process that tells us how similar two signals are, these images in matrix form can also be used to perform correlation and determine what the image represents in alphanumeric form.

A previously created set of images for all the alphanumeric characters (A-Z, 0-9) is used and the results of the bounding box analysis are correlated with these images. The maximum correlation is likely to be with the letter that resembles the one in the image. Hence the character is selected and this process is repeated for all the characters in the number plate and one by one all the characters are recognized.

The result of the OCR is a string of characters and is to be stored in a format that can easily be viewed and is also usable by other applications that would require using the results. For the purposes of this project the results are stored in a text file that is named “text.txt” In the working directory of the project and is opened once the computation is finished by the main program for the viewing of the user.

The conversion of image to text provides a very high compression ratio. An image file containing the same information would be of the size of a few hundred kilo bytes but the text file containing the same information will be only of a few bytes and hence can be much easily transferred to a remote location.

The text file is appended every time such that the record of the previous transactions is maintained.

   

You can find all the files and code used in this post here.Don't get scared seeing the matlab ".m" file most of the code in the starting is from the GUI so you won't have to write any of that yourself. There are comments included in the file but if you do not understand a command or what it does you can see the help in MATLAB 2011b or you can refer to the online MATLAB documentation.