A robust tracking algorithm for super-resolution reconstruction of vehicle license plates

In the installation of video surveillance systems it is quite common to look for a compromise for what concerns the focal length of the optics of the cameras: if on one hand the choice of wide angle lens, i.e. lenses with a large angle of view, permits a global inspection of the area to be monitored, so considerably limiting the "dead zones", on the other hand this approach often jeopardizes the readability of important details in the image. This problem is particularly significant in the identification of the license plates of vehicles, since most often they are represented in a very small area of the image, so that only a quite low resolution version of the license plate is available for identification. Alternatively, the use of narrow angle cameras facilitates the recognition but can be taken only in very limited and specific cases, i.e. only if the spatial location of the license plate is known a priori. Consequently, the police personnel often needs to extract, from low resolution and noisy sequences of images, essential information for the recognition of the targets. In order to solve the problem, we can observe that although the single image is not detailed enough to allow a proper identification, on the other hand the availability of an entire sequence, composed by several images of the same target, can lead, through super-resolution techniques[1-3], to the reconstruction of an image with a resolution higher than the original one, in a process which aims at reversing the process that from the actual scene generated the low resolution image. However, an essential point, in order to obtain a good reconstruction, is the ability to identify in an extremely precise way and with sub-pixel resolution the exact position of the license plate area in each single frame. In this paper we have optimized each step of the entire process that from a low resolution, real world sequence, leads to a super-resolution image. The procedure must identify the target, track its trajectory along the sequence with great precision, extract its position in each frame and eventually combine all the low resolution images in a higher resolution version of the target. The procedure we propose follows a semi automatic approach and consists in several steps. Firstly the user identifies, in the first frame of the sequence, several points of interest (POIs) of the vehicle located on the plane which contains the license plate, including the corners of the license plate itself. Than the system automatically estimates, frame by frame, the new positions of all these POIs. This phase of the process makes use of genetic algorithms in order to solve a constrained optimization problem which aims at identifying the most likely location of each POI constrained by the fact that, since all the points belong to the same rigid body, their position in the different frames must be described by an appropriate perspective transformation [4,5]. The proposed system is able to achieve excellent performance in the tracking of the target with sub-pixel resolution. The final step of the process is the reconstruction phase, where each frame is perspective transformed, aligned, cropped, de-convolved and interpolated to higher resolution. Eventually, all these data are combined together in a super-resolution image.