\section{Introduction}
\label{sec-intro}
Automatic License Plate Recognition (ALPR) systems have received much attention to modern transportation services mainly for  automatic management of traffic, parking, toll\hyp station, and other road operations including surveillance and recognition of potential threats~\cite{du2012automatic}.
An ALPR system consists of three main phases: \textit{frame\hyp selection}, \textit{character\hyp segmentation}, and\textit{ optical character recognition} (OCR).
The first phase verifies the existence of any Vehicle License Plate (VLP) in the input frame by extracting any possible character's features from the frame. The second phase separates the characters from the background followed by their recognition in the last phase.
Such a system has more potential for the developing countries or states
requiring higher road\hyp safety measures and better traffic management.\looseness -1

One such potential group of the developing regions is the Bengali speaking  countries and states that still requires a promising ALPR system for the Bengali VLP application.
Unlike English characters, Bengali has more complex features leaving its accurate recognition from a VLP more challenging. Despite being the six largest population  around the world~\cite{ ethnologue2021}, no significant progress can be tracked in the Bengali language countries or states for the ALPR system addressing their inadequate road\hyp safety measures and poor traffic management. Besides, the performance of the prominent ALPR models developed for the English VLP is also unknown for the Bengali VLP recognition application.\looseness -1

Recent ALPR systems for English VLP captures the employment of promising machine learning models. Such models mainly detects the VLP in the given image (or a video frame) followed by the recognition of the text information on the plate.
A variety of models are stemmed from the need for improving the classification accuracy, robustness to environmental artifacts, and computational efficiency. 
For example, the Convolutional Neural Network (CNN) based bounding box detectors were developed with regression algorithms~\cite{bulan2017segmentation, Xueccv2018, laroca2018, wang2019, silva2021}, manual annotation~\cite{Zhuang_2018_ECCV}, and transfer learning~\cite{c12}. 
Other development with  recurrent neural network (RNN) based architectures include the  Bidirectional Long short-term memory (BiLSTM) based models~\cite{zou2020robust}, and real\hyp time object detection algorithm, YOLO (You Only Look Once) and its variants based models~\cite{chen2019automatic, al2019efficient, yolo}. The end-to-end cascaded and unified architectures of the neural networks were also studied for the ALPR systems~\cite{Hsu2013, Montazzolli2017, Li2019, onim2020traffic}.

However, Unlike the English VLP, the effort in developing the ALPR systems for the Bengali VLP is particularly limited.
Despite much interest in developing Bengali handwriting, scripts and character recognition in general, it has not captured the ALPR system yet. 
A few notable developments of the ALPR systems for the Bengali VLP include the feature extraction based on the digital curvelet transform~\cite{majumdar2007bangla}, 
Tesseract OCR~\cite{hasnat2009integrating}, and CNN with  Adam optimizer~\cite{rabby2018bornonet}.


In summary, no Deep Neural Networks (DNN) model can be tracked in the literature that can detect the Bengali VLP and recognize its characters simultaneously.
A few models only focused on the Bengali characters recognition~\cite{ hasnat2009integrating, rabby2018bornonet}, but their performance is unknown for ALPR system. 
Although Tesseract and BornoNet have relatively high character recognition accuracy, they are not suitable for real\hyp time applications like ALPR due to higher processing time.
Unlike the above models, Onim~\emph{et al}\onedot\cite{onim2020traffic} recently combined YoloV4 for detection of VLP, and Tesseract as OCR engine.
However, the model requires reasonably higher time for number plate detection and character recognition.
All these mean that the models  developed for either Bengali VLP detection or Bengali characters recognition are generally limited with \textit{low detection and recognition accuracy}, and \textit{high computational complexity} making them unsuitable for a real~\hyp time application.

In this paper, we, therefore, report an ALPR system with a new DNN model that we call Bengali License Plate Network (BLPnet) (Sec.~\ref{sec-our ALPR}).
BLPnet is constructed to have three primary phases.
Particularly, the contributions with the proposed three\hyp phase ALPR system can be summarized as follows: 

\begin{itemize}
    \item Building on the NASNet\hyp Mobile backbone architecture, the first phase employs more \textit{dense} and \textit{pulling} layers on the network\hyp head to identify the vehicles more efficiently with a region of interest bounding\hyp box (Sec.~\ref{subsec-boundingbox}). 
    
    \item The second phase includes an InceptionV3 architecture customized with several \textit{dense} and \textit{pulling} layers to detect the VLP in the bounding\hyp box region (Sec.~\ref{subsec-VLPdetect}). 
   
   
   
    \item The third phase introduces a new Bengali\hyp OCR engine (Sec.~\ref{subsec-OCR}).
   
   
   
    Unlike the existing ALPR systems that cannot adequately tackle the artifacts like motion\hyp blur and non\hyp uniform shadow, the proposed engine  employs de\hyp blurring and Region Scalable Fitting (RSF) based segmentation, which are optionally invoked (\emph{i.e}\onedot} \def\Ie{\emph{I.e}\onedot, when characters are not recognized) to better tackle the intensity inhomogeneity.
    \item Additionally, detecting vehicle regions (first phase) prior to VLP (second phase), as considered in the proposed ALPR system, would significantly reduce computational cost and false\hyp positives making the system faster and more accurate. 
\end{itemize}






\section{Related ALPR Systems}
Development of several prominent ALPR systems for English VLP can be tracked in the literature so far. 
Getting the VLP information requires the localization and detection of the VLP. In CNN based approach, features are extracted for VLP and then object localization is done with bounding box detectors and regression algorithms~\cite{Xueccv2018, wang2019}.
Silva~\emph{et al}\onedot~\cite{silva2021} and Laroca~\emph{et al}\onedot~\cite{laroca2018} used bounding box regression to localize the object of interest with image coordinates using YOLO. These are One-stage detection networks and relatively faster than other detectors. But these approach is computationally inefficient as it requires training \textit{darknet} backbone of over $27M$ parameters. 

Alternatively, VLP can also be semantically segmented using a either CNN or deep segmentation network. 
Bulan~\emph{et al}\onedot\cite{bulan2017segmentation} proposed a real\hyp time complete CNN model  having higher adaptability to tackle the scene\hyp variations  requiring minimum possible human\hyp intervention.
An image\hyp based classification was proposed with weak\hyp Snow and strong\hyp CNN classifiers. 
The first classifier was initially used for the region\hyp tagging of the possible number\hyp plates, and the strong\hyp CNN refined the classification outputs for the actual number\hyp plates
Zhuang~\emph{et al}\onedot~\cite{Zhuang_2018_ECCV} proposed such method where they segmented the VLP and the characters in it. This approach also needs a manual annotation of a characters and LP. Besides these segmentation based networks, segmentation free Networks have also been proposed. 
Wang~\emph{et al}\onedot~\cite{wang2019} proposed a convolutional Recurrent Neural Network (CRNN) and proposed a multi-task license plate detection and recognition (MTLPR) model.
Zou~\emph{et al}\onedot~\cite{zou2020robust} used Bi-LSTM and directly localized characters without segmentation.\looseness -1

Both end-to-end cascaded and unified architectures have been used for ALPR system. Li~\emph{et al}\onedot~\cite{Li2019}
used end-to-end trainable LP detection model that remains error-prone to the challenging conditions and also depends heavily on architectural design.
On the other hand, cascaded architectures have the flexibility of tuning and testing. Hsu~\emph{et al}\onedot~\cite{Hsu2013} and Montazzolli~\emph{et al}\onedot~\cite{Montazzolli2017} proposed cascaded architecture for real time VLP detection and recognition.

Huang~\emph{et al}\onedot~\cite{c12} introduced Transfer learning into their CNN model to tackle the limited labeled data for a high detection accuracy of VLP of  93\%.
Later, with a modified YOLOv3 model with 10 CNNs, Chen~\emph{et al}\onedot\cite{chen2019automatic} attempted to detect and recognize the VLP  with an accuracy of about 98\% and  78\%, respectively in a variety of conditions (\emph{i.e}\onedot} \def\Ie{\emph{I.e}\onedot, rainy backgrounds, darkness and dimness, and varied colors and saturation of photos). 
In addressing several other environmental artifacts like
poor\hyp contrast and noisy images resulting from foggy, distortion and dusty conditions,
Al-Shemarry~\emph{et al}\onedot\cite{al2019efficient} improved the accuracy of VLP detection using a  binary descriptor with contrast enhancement, leaving their proposed system limited to real\hyp time application with poor frame\hyp rate and higher computational cost.
To tackle the motion artifacts (\emph{i.e}\onedot} \def\Ie{\emph{I.e}\onedot, blurry input images),
Zou~\emph{et al}\onedot\cite{zou2020robust} recently proposed a Bi-LSTM model with  robust blur\hyp kernel estimation that offered a reasonably lower accuracy of 79.55\%.
Another recent development with object bounding\hyp box detection for ALPR system is YOLOv4 model~\cite{yolo}, which is
a deep convolutional network that can localize VLPs with immense frame rate and high accuracy.\looseness -1 



Besides, on Bengali character detection and recognition, 
Majumdar~\emph{et al}\onedot~\cite{majumdar2007bangla} proposed a feature extraction based on the digital curvelet transform.
Separate $k$\hyp nearest neighbor classifiers were trained using the curvelet coefficients of an original image and its morphologically changed copies. 
The overall recognition accuracy of 96.8\% was reported 
while trained for twenty popular Bengali fonts and tested for different font sizes.
Later, Hasnat~\emph{et al}\onedot~\cite{hasnat2009integrating} attempted to successfully introduce Bengali script in Tesseract OCR. 
Tesseract can learn new languages or alphabets with training along with existing languages for character recognition. 
Recently, Rabby~\emph{et al}\onedot~\cite{rabby2018bornonet} presented a new 13-layer CNN model called BornoNet for Bengali characters recognition. The model was designed with two sub-layers optimized using adam optimizer
Despite having relatively higher accuracy in the models for Bengali characters recognition, their applications to an ALPR system is not yet studied.
In addressing the potential gap of having a promising ALPR system for Bengali VLP, we therefore develop and present real\hyp time End\hyp to\hyp End  DNN based model that we call BLPnet.



















\section{A New ALPR System with BLPnet}
\label{sec-our ALPR}
This section presents a newly developed BLPnet model for ALPR system.
As illustrated in Fig.  \ref{Overall block diaram}, the model consists of three main phases that are cascaded together to detect vehicle, VLP and recognize the characters in it.
It takes real-time video frames as input consisting of vehicles and its surroundings.
In the first phase, the vehicle region is detected and distinguished using a bounding\hyp box 
as the smallest possible rectangle with vertical and horizontal sides that entirely surround the vehicle captured in a frame. 
The frames containing the bounding boxes are fed to the VLP detection model along with the coordinates of the bounding\hyp box. 
Later, the second phase detects and extracts the VLP contour 
followed by the character  localization and recognition in the final phase.
Consideration of intensity\hyp inhomogeneity resilient segmentation and effective deployment of these cascaded modules would reduce the computational cost and false\hyp positive predictions improving the accuracy and reliability of the system for a real\hyp time application. 
These phases are now discussed below with more technical details.

\begin{figure*}[htp]
\centering
\includegraphics[width=0.95\columnwidth]{Fig-1}
\caption{Key processes of the proposed ALPR system}
\label{Overall block diaram}
\end{figure*}
 


\subsection{An Extended Vehicle Bounding\hyp Box Detector}
\label{subsec-boundingbox}
A NASNet\hyp Mobile is extended and used as the backbone of our DNN followed by six hidden layers to improve the vehicle detection accuracy (see Fig.~\ref{fig-BBox}).
The NASNet\hyp Mobile is more computationally efficient than its counterpart like the ResNet-based architecture that needs higher hardware configuration, making it unsuitable for large\hyp scale deployment~\cite{c13}. 
Thus, our model can be implemented on Field-Programmable Gate Array (FPGA) and other embedded devices to balance between hardware resources and processing speed. The TensorFlow (TF) v2.4 framework was used for the overall construction of our CNN model.
The backbone was pre-trained with the Imagenet dataset \cite{imagenet}. 
Three hidden layers added as network head were fully connected layers followed by a dropout layer. A few important processing and considerations are briefly discussed below.\looseness -1 

\begin{figure*}[htb]
\centering
\includegraphics[width=0.6\columnwidth]{Fig-2}
\caption{Vehicle Bounding\hyp box detector of the proposed ALPR system}
\label{fig-BBox}
\end{figure*}

\paragraph{Dataset Collection}
For the training of vehicle bounding\hyp box detection model, we used the Cars dataset~\cite{stanford} of Stanford AI Lab.
There are 16,185 images in the dataset, representing 196 different car classifications. The data is divided into 8,144 training images and 8,041 testing images, with about a 50-50 ratio between the two classes. Typically, classes are organized by Make, Model, and Year, such as 2012\hyp Tesla\hyp Model\hyp S or 2012\hyp BMW\hyp M3\hyp coupe. Each image in the dataset has also  a  bounding\hyp box output value as an ideal reference that suits our requirement.

\paragraph{Pre-Processing}
We considered a total of  8,041 training images and passed them through two preprocessing stages. At first image augmentation was done to simulate challenging conditions which the real\hyp time traffic video might face. The applied augmentations include: rotate, shift, flip, varying contrast, blur, and salt and pepper noise.
After augmentation, label-encoding and  training-validation split are done. In this case, we made an 80-20 split. After the splitting, the model is trained and tuned.

\paragraph{Transfer Learning (TL)}
A pre\hyp trained model (NASNet-Mobile) as the backbone architecture of the proposed model ensures optimised training.
Unlike the isolated learning paradigm, Transfer learning allows extending the pre\hyp learned features to adequately address the new related challenges. 
Particularly, an average pooling layer usually can extract the pretrained models edge detection capability.
From the final layer of shape $(10\times10\times1056)$, an average pooling layer is thus added followed by a group of fully connected dense layers and dropout layers. 
 Finally the output layers identifies the object and generates the bounding box coordinates.
Thus, our model was pre\hyp trained with over fourteen  million images from the ImageNet dataset, which is now capable of categorising images into over 1000 different classes with transfer learning ability.
As a result, it has accumulated a library of rich feature sets for a wide variety of classes.


\paragraph{Hyper-parameters and Tuning}
The training hyper-parameters are given in Table~\ref{tab:training_params}. 
Finally, the parameters with the best accuracy have been chosen after several trials and errors.
The optimizer that suits best for the VLP detection is Stochastic gradient descent(SGD). 
We have also tweaked several parameters of the optimiser 
for our dataset and defined early stopping to reduce the training time. 
Additionally, Reduced Learning Rate functionality has been used to further expedite the training process. 




\begin{table}[htb]
    \centering
    \caption{Comparison of the training hyper\hyp parameters}
    \setlength{\tabcolsep}{4pt}
    \resizebox{\linewidth}{!}{
    \begin{tabular}{l|cc|cc|c|c|c}
    \toprule
       \bf Model	& \multicolumn{2}{c|}{\bf Ours (BLPnet)}	& \multicolumn{2}{c|}{\bf Onim~\emph{et al}\onedot~\cite{onim2020traffic}}	& \bf Zou~\emph{et al}\onedot~\cite{zou2020robust}	& \bf \makecell{Hendry \&\\ Chen~\cite{chen2019automatic}}	& \bf Li~\emph{et al}\onedot~\cite{LI201814}\\
        \midrule
        \bf Parameters &	Vehicle	& VLP	& Vehicle &	VLP	& VLP	& VLP	& VLP\\
        \midrule
        \bf Backbone	& \makecell[l]{NASNet\\Mobile}	& InceptionV3	& Darknet	& Darknet	& MobileNetV3	& Darknet & - \\
        \midrule
        \bf Classes &	1 &	1 &	4 &	1 &	37 &	60 &	37\\
        \midrule
        \bf Batch size	& 32 &	64 &	32 &	64 &	- &	- &	32\\
        \midrule
        \bf Image Shape &	$300\times300$ &	$200\times200$ &	$224\times224$ &	$128\times128$ &	$152\times56$ &	$256\times256$ &	$24\times24$ \\
        \midrule
        \bf Optimizer &	SGD &	Adam &	Adam &	Adam &	
        SGD &	Adam &	Adam\\
        \midrule
        \bf Epochs &	300 &	300 &	100 &	3500 & - &		600 & - \\
    \bottomrule
    \end{tabular}
    \label{tab:training_params}
    }
\end{table}






\subsection{VLP Detection with Inception-v3}
\label{subsec-VLPdetect}
We have also extended the
Inception-v3 DNN architecture and employed it as the backbone of the second phase process of BLPnet for VLP detection as illustrated in Fig.~\ref{lp-BBox}.
Inception-v3 is a CNN designed from the Inception family that transports label\hyp  information to a lower level of the network using label\hyp smoothing, factorised $7\times 7$ convolutions, an auxiliary classifier, and batch\hyp  normalization for layers in the side head. 
Inception architecture being 42 layers deep has 2.5 times lower  computational cost than that of GoogLeNet and can function well even when memory and computational budgets were limited~\cite{szegedy2016rethinking}.

We have used an annotated dataset of 1500 training and 300 validation images to train our model. 
From real\hyp world video footage taken on roadways in Dhaka, Bangladesh, with varying road conditions, including high traffic congestion, the VLP were detected with reasonably higher accuracy (see Sec.~\ref{sec-result}).
The training began with a number of iterations set at 6000. 
The average loss did not diminish significantly after 2000 iterations,
and after each 1000 cycles, the training was capable of taking weight backup. 
To reduce training duration, we thus used early stopping.

\begin{figure*}[htb]
\centering
\includegraphics[width=0.6\columnwidth]{Fig-3}
\caption{License Plate Bounding\hyp box detector of the proposed ALPR system}
\label{lp-BBox}
\end{figure*}


\subsection{A New Bengali\hyp OCR Engine}
\label{subsec-OCR}
This step recognised characters in the extracted VLP from the previous step. 
Similar to~\cite{9003211}, we addressed the character recognition as an object recognition problem. 
By treating characters as objects, the character segmentation and recognition steps are integrated together that consists of a total 5 convolutional 2D layers with 16, 32, 64, 128 and 512 nodes.
Each of them includes a preceding max-pooling layer with a pool size of 2 and a dropout layer with a drop rate of 20\%. 
The dropout layers were used to prevent the model from over-fitting. 
The kernel size of 2 is considered with necessary padding. 
Relu activation function is used for all the backbone layers.
Finally, a global\hyp average pooling layer is included before the output layer followed by two dense layers. 
Two important considerations of this model are the conditional use of de\hyp blurring filter and intensity inhomogeneity invariant segmentation of the character.

Blurring is known to be a key challenge in VLP recognition resulting from wrong focusing or vehicle's motion that requires
restoration of the blurred images
for a higher accuracy in character recognition. 
A set of filters like  Wienner filter, Total Variation (TV) deblurring, Haar deblurring and a combination of both is considered  for restoring any blurred image to its estimated original version. 
For an input image,  $Im[n]$, let the output of the filter is  $x[n]$ as in~\eqref{eqn_xn}, where $G(z)$ is the filter function, $N$ is the number of past taps with coefficient $a$, $s[n]$  is the reference signal, and $e[n]$ is the residual error.
The criteria to minimize the mean squared error (MSE) for the coefficient is defined in~\eqref{eq-a}, where $E[.]$ is the expectation operator.
On the other hand, Haar deblurring~\cite{DBLP:journals/corr/abs-1905-01003} uses a deconvolution filter and reduce the MSE as in~\eqref{eq-har}, where  $I(i,j)$ is the expected and $G(i,j)$ is the desired deblurred images. 
In our model, we have designed a conditional deblurring process invoked when not enough number of characters are detected in the OCR. 
An iterative approach is employed for this process with a varying threshold
based on the Fast-Iterative Shrinkage Thresholding Algorithm (FISTA)~\cite{doi:10.1137/080716542} in the VLP image.\looseness -1


\begin{subequations}
\begin{equation}
\label{eqn_xn}
x[n]=\sum_{i=0}^{N} a_{i} \times \operatorname{Im}[n-1]
\end{equation}
\begin{equation}\label{eq-a}
a_{i}=\operatorname{argmin}\left(E\left[e^{2}[n]\right]\right)\\
\end{equation}
\end{subequations}\vspace{2pt}




\begin{equation}\label{eq-har}
M S E=\frac{1}{m \times n} \sum_{i=0}^{m-1} \sum_{j=0}^{n-1}[I(I, J)-G(i, j)]^{2}
\end{equation}

Additionally, we have considered the intensity inhomogeinity invariant segmentation of the characters. Thus, we extracted the characters from the VLP with two segmentation models: Chan~\&~Vese (CV) model and Region-Scalable Fitting (RSF) model (see Table~\ref{Table for Performance Analysis of OCR} in Sec.~\ref{sec-result}). As most of the images for VLP have very high contrast, CV model performed well in approximating the intensities of the foreground or object and background. 
In case of intensity inhomogeneity, CV model failed to segment the characters, where RSF model was used.



The complete architecture is illustrated in Fig.~\ref{OCR}.  
The output layer contains 60 classes to capture the total number of Bengali alphabets and digits. 
The output layer uses \textit{Softmax} as an activation function. 
We have also used CMATERdb version 3.1.2 for training the alphabets for the digits. A total alphabet\hyp images of  15000 and digits of 500 have been split as follows: 9900 images for training,   2600 images for validation, and 3000 images for testing.

\begin{figure*}
    \centering
    \includegraphics[width=0.8\linewidth]{Fig-4}
    \caption{The newly developed OCR engine for BLPnet}\vspace{-3pt}
    \label{OCR} 
\end{figure*}


The images are then augmented so that the model can identify images on non-ideal conditions. The following augmentations are considered: (\textit{i})~10\% shift in left and 10\% shift in right; (\textit{ii})~10\% shift in vertical and 10\% shift in horizontal; (\textit{iii})~7.5$\degree$ rotation in left and 7.5$\degree$ in right; (\textit{iv})~zoom range 20\%; and, (\textit{v})~Shear range 20\%.
Besides, each character illustrated in Fig.~\ref{Character Class} is resized to $16\times 16$ after segmentation before passing through the network. 
As a result,  this network can work with variable size characters.

\begin{figure}[thb]
\centerline{\includegraphics[width=0.55\linewidth]{Fig-5}}
\caption{Bengali character classes for the OCR training}\vspace{-3pt}
\label{Character Class}
\end{figure}



\section{Results and Analysis}
\label{sec-result}
We now present the performance of the proposed BLPnet model for the ALPR system, and compare its performance with some related models: 
Li~\emph{et al}\onedot~\cite{LI201814}, Hendry \& Chen~\cite{chen2019automatic}, Zou~\emph{et al}\onedot~\cite{zou2020robust}, and Onim~\emph{et al}\onedot~\cite{onim2020traffic}. 
This section starts with the  analysis of total learnable parameters of the model followed by the
 results discussed in three main parts, namely, for vehicle detection, VLP detection and finally, for the OCR and word\hyp mapping.


\subsection{Total Learnable Parameters}
A dense layer with \textit{m} input nodes and \textit{n} output nodes will have a total of $(n+1)\times m$ learnable parameters.
The pooling layers and dropout layers does not learn anything. For convolutional layers with \textit{p} feature maps in input and \textit{q} feature maps as output having a filter size of $i\times j$ will have a total learnable parameters of $(i\times j\times p + 1)\times q$.
 The distribution of trainable parameters for bounding box detector across the network is noted in Table.~\ref{tab:bbox_param}. 
 The total number of trainable parameters is calculated as shown in Equation.~\eqref{param}. 
 Similarly, Table.~\ref{tab:ocr_params} shows the trainable parameters for our OCR engine. Here \textit{Layers} denotes the name and operation of added layers, \textit{Shape} denotes the input shape of tensors for that particular layer and finally the trainable parameter for that layer.

 \begin{table}[!th]
     \centering
     \caption{Trainable parameters for bounding box detector}
     \renewcommand{\arraystretch}{1.05}
	\setlength{\tabcolsep}{10pt}
	\resizebox{0.7\linewidth}{!}{
     \begin{tabular}{llc}
\toprule
\bf Layers &\bf Shape &\bf \makecell[c]{Trainable\\ Parameters} \\
\midrule
Global\_AveragePooling2D & (None, 1056) & 0 \\
Dense & (None, 256) & 270592 \\
Dense & (None, 256) & 270592 \\
Dense & (None, 128) & 32896 \\
Dropout & (None, 256) & 0 \\
Dropout & (None, 128) & 0 \\
Dense & (None, 128) & 32896 \\
Dense & (None, 64) & 8256 \\
Dense & (None, 64) & 8256 \\
Dropout & (None, 64) & 0 \\
Dropout & (None, 64) & 0 \\
Dense & (None, 32) & 2080 \\
Dense & (None, 32) & 2080 \\
Dense & (None, 2) & 66 \\
Dense & (None, 4) & 132 \\
\midrule
\bf Total & & 627k\\
\bottomrule
\end{tabular}
}
     \label{tab:bbox_param}
 \end{table}


\begin{equation}
\begin{aligned}
    N_{detector} &=2\times((1056+1) \times 256+(256+1) \times 128+(128+1) \times 64 \\
    & +(64+1) \times 32+(32+1) \times 2) \\
    &=627k
\end{aligned}
\label{param}
\end{equation}


\begin{table}[!t]
    \centering
    \caption{Trainable parameters for OCR engine}
\renewcommand{\arraystretch}{1.05}
	\setlength{\tabcolsep}{8pt}
	\resizebox{0.6\linewidth}{!}{
\begin{tabular}{llc}
\toprule  
\bf Layers &  \bf Shape & \bf \makecell[c]{Trainable\\ Parameters}  \\
\midrule
Conv2D  & (None, 63, 63, 16) & 80 \\
MaxPooling2D & (None, 31, 31, 16) & 0 \\
Dropout & (None, 31, 31, 16) & 0 \\
Conv2D  & (None, 30, 30, 32) & 2080 \\
MaxPooling2D & (None, 15, 15, 32) & 0 \\
Dropout & (None, 15, 15, 32) & 0 \\
Conv2D  & (None, 14, 14, 64) & 8256 \\
MaxPooling2D & (None, 7, 7, 64) & 0 \\
Dropout & (None, 7, 7,64)  & 0 \\
Conv2D  & (None, 6, 6, 128) & 32896 \\
MaxPooling 2D & (None, 3, 3, 128)  & 0 \\
Dropout  & (None, 3, 3, 128)  & 0 \\
Conv2D  & (None, 2, 2, 256)  & 131328 \\
MaxPooling 2D  & (None, 1, 1, 256)  & 0 \\
Dropout  & (None, 1, 1, 256)  & 0 \\
Flatten  & (None, 256) & 0 \\
Dense & (None, 256)  & 65792 \\
Dense & (None, 512)  & 131584 \\
Dropout & (None, 512) & 0 \\
Dense  & (None, 60) & 25650 \\
\midrule
\bf Total &  & 397K\\
\bottomrule
\end{tabular}
}
    \label{tab:ocr_params}
\end{table} 






\subsection{Vehicle Detection}
We trained the vehicle bounding\hyp box model for vehicle detection for 1000 epochs with early stopping employed to stop the model training if the accuracy improvement is negligible. 
The hardware specification to train the model is: Tesla K80 2496 CUDA cores GPU with 12~GB GDDR5 VRAM, Processor @2.3Ghz (4 core, 8 threads), and RAM of 48~GB.

\begin{figure}[!th]
\centerline{\includegraphics[width=.8\linewidth]{Fig-6}}
\caption{Loss vs Epoch curve for the first 50 epochs}
\label{Loss vs Epoch curve}
\end{figure}


\begin{figure*}[htbp]
\centering
    \subfloat[Detected vehicles (marked with green bounding\hyp boxes) ]{%
    \includegraphics[width=0.9\columnwidth, height = 3 in]{Fig-7a}\label{Bounding box output}}
    \\
    \subfloat[VLP detection (marked with blue\hyp colour bounding box)]{%
    \includegraphics[width=0.9\columnwidth, height = 3in]{Fig-7b}\label{cars}}
\caption{Example of detected vehicles and VLP from real\hyp time captured video footage with different orientations of the vehicles}
\end{figure*}







We chose the criteria for our evaluation matrices of bounding\hyp box coordinates to be MSE and accuracy.
From the loss vs epoch curve in Fig.~\ref{Loss vs Epoch curve}, we observe that the model can quickly converge to a lower loss value. After a while, the improvement gets very slow.
The best training loss and MSE of bounding\hyp box were 0.0130 and 0.0155, respectively and the validation loss and MSE were 0.0148 and 0.0152, respectively. 
Overall training parameters with their respective epochs is presented in Table~\ref{table of vb training}. 
We observe in this table that after 300 epochs, the rate of change in loss and accuracy is becoming stable.
Compared to the YOLO model, the hyper-parameters were kept similar as shown earlier in Table~\ref{tab:training_params}.
All these observations suggest that the our proposed model would effectively detect the vehicles in the input video clips. The final predicted output of the model applied in random vehicles is shown in Fig.~\ref{Bounding box output} with successful demarcation of green boxes.



\begin{table*}[!th]
\caption{Performance of vehicle detection with bounding\hyp box}
\centering
\renewcommand{\arraystretch}{1.05}
	\setlength{\tabcolsep}{3pt}
	\resizebox{\linewidth}{!}{
\begin{tabular}{cccccccccc}
\toprule
\multirow{3}{*}{\textbf{Epochs}} & 
\multirow{3}{*}{\textbf{Loss}} & \textbf{Box} & \textbf{Class} & \textbf{Box} & \textbf{Class} & \multirow{3}{*}{\makecell{\bf Validation \\ \bf Loss}} & \makecell{\bf Validation\\ \bf box} & \makecell{\bf Validation\\\bf  class} &  \makecell{\bf Validation\\ \bf box} \\
 &  & \textbf{Output} & \textbf{Output} & \textbf{Output} & \textbf{Output} &  & \textbf{Output} & \textbf{Output} & \textbf{Output} \\
  &  & \textbf{Loss} & \textbf{Loss} & \textbf{MSE} & \textbf{Accuracy} &  & \textbf{Loss} & \textbf{Loss} & \textbf{MSE} \\
    \midrule

50&
0.0056 &
0.0176 &
0.0059 &
0.0176 &
75.59 &
0.0057 &
0.0175 &
0.0057 &
0.0175 \\

100&
0.0056 &
0.0171 &
0.0058 &
0.0171&
85.89 &
0.0056 &
0.0140 &
0.0057 &
0.0140 \\


150&
0.0056 &
0.0169 &
0.0058 &
0.0169 &
89.19 &
0.0055 &
0.0138 &
0.0056 &
0.0138 \\


200&
0.00569 &
0.0173 &
0.0057 &
0.0173 &
90.55 &
0.0055 &
0.0175 &
0.0055 &
0.0175 \\


250&
0.00569 &
0.0176 &
0.0056 &
0.0176 &
94.74 &
0.0054 &
0.0175 &
0.0055 &
0.0175 \\


300&
0.00569 &
0.0176 &
0.0053 &
0.0176 &
96.99 &
0.0055 &
0.0175 &
0.0054 &
0.0175 \\


350&
0.00569 &
0.0176 &
0.0052 &
0.0176 &
97.02 &
0.0055 &
0.0175 &
0.0054 &
0.0175 \\

400&

0.00582 &
0.0130 &
0.0050 &
0.0155 &
97.05 &
0.0056 &
0.0148 &
0.0058 &
0.0152 \\
\bottomrule
\end{tabular}
}
\label{table of vb training}
\end{table*}









  
    








\subsection{VLP Detection}

Our model was evaluated using both real\hyp time and pre-recorded video clips. Table \ref{tab:training_params} shows the hyper\hyp parameters used to train the network. Our model did not over\hyp fit due to the use of adequate \textit{dropout} and \textit{pooling} layers, and thus, it took little time to train and converge to an optimum detection level. During the evaluation, the performance of the model was monitored in real\hyp time. 

Our algorithm successfully detected VLP as a few examples are illustrated in Fig.~\ref{cars}, where VLP is marked with blue bounding box. 
A VLP was detected from varying orientation of the detected vehicle and at that time, 17 frames per second processing speed was maintained on average.
Such accurate detection was observed for all the testing video clips with resolution $1920\times  1080$ and  frames per second (\textit{fps}) is of 15 and 20.\looseness -1 

Similar to the vehicle detection with  bounding box (Sec.~\ref{subsec-boundingbox}), the VLP detection training loss and MSE of bounding\hyp box were 0.013 and 0.016, respectively and the validation loss and MSE were 0.015 and 0.015, respectively.
No false\hyp positive detection was recorded during the process which justifies the effectiveness of previous phase.  \looseness-1



\subsection{Character Recognition}
The overall procedure can be broken down into three steps. First the preprocessing step, where the image is filtered based on the sharpness. 
Later in character segmentation, the fast active contour model segments the characters as objects. Finally, the character recognition followed by a word mapping. Here, for each  group of key characters recognised, a predefined word is mapped for them. Our model's generated character output is shown in Fig.~\ref{Example results OCR}.


\begin{figure}[!t] 
    \centering
 \vspace{-0.3cm}
    \includegraphics[width=0.985\linewidth]{Fig-8}
    \caption{Example of input and outputs of the newly developed OCR engine (From \textit{top}, input images in \textit{first row}, CV\hyp segmented images in \textit{second row},  LSF\hyp segmented images in \textit{third row}, and OCRs and word\hyp mapping in \textit{fourth row})}
    \label{Example results OCR} 
\end{figure}

      
        
        

    



\begin{table}[!t]
\caption{OCR performance for the CV and RSF based segmented characters}
\centering
\renewcommand{\arraystretch}{1}
	\setlength{\tabcolsep}{12pt}
	\resizebox{0.95\columnwidth}{!}{
    \begin{tabular}{ccccc}
   
\toprule
\textbf{Segmentation} & \textbf{No of} &
\textbf{Accuracy} &
\textbf{Time taken} &
\textbf{Time taken}\\
\textbf{Model} & \textbf{characters} &
\textbf{of OCR} &
\textbf{for OCR} &
\textbf{for Tesseract}\\
& \textbf{extracted} & \textbf{(in \%)} & \textbf{(in seconds)} & \textbf{(in seconds)}
\\
    \midrule
\textbf{CV model} & 4 & 90 & 0.302 & 0.402\\
& 5 & 83 & 0.395 & 0.548\\
& 6 & 81 & 0.432 & 0.701\\
& 8 & 80 & 0.502 & 0.705\\ \midrule
\textbf{RSF model} & 4 & 95 & 0.256 & 0.402\\ 
& 5 & 93 & 0.312 & 0.548\\
& 6 & 90 & 0.333 & 0.701\\
& 8 & 89 & 0.398 & 0.705\\\bottomrule
   
    \end{tabular}
    }
   
    \label{Table for Performance Analysis of OCR}
\end{table}

\begin{table}[!t]
    \caption{Average testing accuracy in characters recognition applications}
    \centering
    \renewcommand{\arraystretch}{1.2}
    \setlength{\tabcolsep}{10pt}
    \resizebox{0.95\linewidth}{!}{%
    \centering
    \begin{tabular}{lll}
        \toprule \multicolumn{1}{l}{ \bf Model } & \bf Accuracy & \bf Recognising object \\
        \midrule
       
        Hendry \& Chen~\cite{chen2019automatic} & $78.2\%$ &  VLP \& characters (English)\\
        Zou~\emph{et al}\onedot~\cite{zou2020robust} & $79.5\%$ & Blurry text recognition (English)\\
        Onim~\emph{et al}\onedot~\cite{onim2020traffic} & $90.51\%$ & VLP (Bengali)\\
       
       
        \textbf{BLPnet (proposed)} & $\mathbf{95.0\%}$ & Real\hyp time VLP \& characters (Bengali)\\
        \bottomrule
    \end{tabular}
    }
    \label{tab_comp}
\end{table}

\begin{table}[!t]
    \centering
    \caption{Comparison of features of the ALPR systems}
    \setlength{\tabcolsep}{8pt}
	\resizebox{\linewidth}{!}{
    \begin{tabular}{lp{3cm}cccp{3cm}}
\toprule          
\bf Model  &
\bf  \makecell[c]{VLP\\False--Positive\\Detection} & 
\bf \makecell{Character \\rotational \\ invariant?} &  
\makecell{\bf Trainable \\\bf parameters\\ \bf ({\it \textbf{Million}})} & \bf \makecell{Processing \\time\\ (\textit{s})}&
\bf Hardware\\
\midrule
\makecell[l]{Hendry \&\\ chen~\cite{chen2019automatic}} & Unknown & no & 8.8 & 0.8 -- 1.0 & \makecell[l]{Core i7\\ Nvidia GTX\\ 970 4GB GPU}\\
\midrule
Li~\emph{et al}\onedot~\cite{LI201814} & 
\makecell[l]{Heuristically \\minimized \\with CNN} & no & 1.0 &
2.0 -- 3.0 & \makecell[l]{Core i5\\ Nvidia Tesla\\ k40c 4GB GPU}\\
\midrule
Zou~\emph{et al}\onedot~\cite{zou2020robust} & Unknown & yes & 1.9 & Unknown & \makecell[l]{Core i5\\ Nvidia Titan\\ 12GB GPU}\\
\midrule
Onim~\emph{et al}\onedot~\cite{onim2020traffic} & \makecell[l]{Eliminated using\\ 2\hyp phase detection} & no & 27& 0.7 -- 1.0 &\makecell[l]{Core i5\\ Nvidia Tesla\\ T4 6GB GPU} \\
\midrule
\bf \makecell[l]{BLPnet \\(Ours) }& \bf \makecell[l]{Eliminated using\\ 2\hyp phase detection}  & \bf yes & \bf  0.97 &
\bf 0.32 -- 0.52  & \bf  \makecell[l]{Core i5\\ Nvidia Tesla \\k80 6GB GPU}\\
\bottomrule
\end{tabular}
    \label{tab:major_comp}
    }
\end{table} 
 

Our proposed OCR engine demonstrated reasonably better accuracy  as it treated the characters as objects. The accuracy varies from 80\% to 95\% based on the number of characters extracted. The time it takes to extract a character varies between 0.302 to 0.502 seconds. The model is weak against some vowel pairs. This weakness was tackled successfully with word mapping (see Fig.~\ref{Example results OCR}). Detailed performance analysis is given in Table~\ref{Table for Performance Analysis of OCR}. The accuracy decreases with the increase in number of characters. 
A comparison with related OCR engines is shown in Table~\ref{tab_comp}.
The other performance of the proposed model is also compared  with the other prominent models in Table~\ref{tab:major_comp}.
Considering the nullified False\hyp Positives, lower computational complexity, reduced trainable parameters, and reasonably higher accuracy, the proposed model has demonstrated a higher potential for a real\hyp time ALPR application.








 

 
 
 
 
 \section{Conclusion}
 The development of an ALPR system is a timely requirement for modern transportation services. 
 However, despite being the sixth largest population  around the world~\cite{ ethnologue2021}, no significant progress can be tracked in the Bengali language countries or states for the ALPR system addressing their inadequate road\hyp safety measures and poor traffic management. 
 To this end, we have presented a  computationally efficient and reasonably accurate BLPnet model for a new ALPR system.
 The model is designed to efficiently and correctly return the VLP number by considering a three\hyp phase top\hyp to\hyp bottom approach. 
 This means, the model starts from detecting the vehicle first, and then any possible VLP followed by the recognition of characters in the VLP.
This consideration is observed to be more  effective with lower-computational time and accuracy than the prominent ALPR systems.\looseness -1 

Moreover,  the model generates the actual license number of the vehicle from the recognized characters using our simple, yet effective mapping algorithm with a set of predefined cases of registration area\hyp codes.
The model also performed well without compromising accuracy to tackle challenging conditions that cause rotated,  blurry or noisy frames at the input. 
The preliminary results with the reasonably faster and accurate performance of the model suggest that it would be promising for the real\hyp time ALPR application with a continuing development in future.\looseness -1










 
 





 \bibliographystyle{elsarticle-num} 
 