



\section{Introduction \& Motivations}\label{sec:intro}
\input{intro}

\section{Related Works}\label{sec:related}
\input{related}

\section{Adaptive Test-Time Augmentation}\label{sec:opt}
\input{adaptive_tta}

\section{Experimental Setup}\label{sec:setup}
\input{setup}

\section{Results}\label{sec:results}
\input{results}

\section{Conclusions}\label{sec:conc}
\input{conclusions}

   


    
 
    


\bibliographystyle{IEEEtran}



\subsection{Hardware Platform and Software Configurations}\label{sec:hw}
The hardware testbench is the Odroid-C2 platform powered with the Amlogic S905 SoC. The CPU is a quad-core ARM Cortex-A53 running @1.5GHz nominal frequency. The board runs Ubuntu Mate 18.04, kernel version 3.16.72-46, released by Hardkernel.
The inference engine is TensorFlow Lite 1.14; it offers a collection of neural-network routines optimized to run on the ARM Cortex-A architecture. 
In our setup, TensorFlow Lite is cross-compiled using the GNU ARM Embedded Toolchain (version 6.5) \cite{linaro}.


\subsection{ConvNet Benchmarks}\label{sec:setup_convnets}
We adopted pre-trained ConvNets available on TensorFlow Hosted Models~\cite{tflitemodels} repository. Specifically, we considered a representative family of models for the mobile segment: {\em MobileNet}~\cite{howard2017mobilenets, sandler2018mobilenetv2}.
The networks are trained on the ImageNet dataset~\cite{deng2009imagenet} and quantized to 8-bit, a common choice for edge inference as it ensures lower memory footprint and faster processing speed with negligible accuracy loss with respect to floating-point. 
For MobileNet family, we investigated two different versions of ConvNets listed in Table~\ref{tab:benchmarks}.
The column {\bf Storage} collects the size of the \texttt{tflite}, which contains the data structures needed to deploy the model on-chip, i.e., the network weights and the topology description. The column {\bf Top-1} refers to the top-1 classification accuracy measured on the ImageNet validation set without TTA, i.e., with a standard pre-processing consisting of resizing the images to 256$\times$256 pixels and extracting the central crop of size 224$\times$224. The column $\bm{L_\textrm{\textbf{nom}}}$ reports the nominal latency of a single inference at maximum performance (4-threads @1.5GHz). 


The adopted benchmarks are state-of-the-art for image classification on embedded systems. 
MobileNets are compact hand-crafted networks optimized for high performance and low memory. 

\begin{table}[!t]
\centering
\caption{Memory, nominal top-1 accuracy without TTA (Top-1), and nominal latency ($L_\textrm{\textbf{nom}}$) of the selected benchmarks.}\label{tab:benchmarks}
\begin{tabular}{lcccc}
\toprule
{\bf ConvNet} & {\bf Storage} & {\bf Top-1} & $\bm{L_\textrm{\textbf{nom}}}$ \\ 
{} & {\bf {[}MB{]}} & {\bf [\%]} & {\bf {[}ms{]}} \\
\midrule
MobileNetV1 & 4.3    &  70.0   & $53.1$   \\
MobileNetV2  &3.4    &  70.8   & $44.2$   \\
\bottomrule
\end{tabular}
\vspace{-0.5cm}
\end{table}


\subsection{Augmentation Policies}\label{sec:setup_policy}
We adopted two popular TTA policies~\cite{krizhevsky2012imagenet, howard2013some, szegedy2015going}, which are described as follows:\\
{\bf 5-Crops (5C)} - starting from a $256\times256$ image, five crops of size $224\times224$ are extracted (the center crop and the four corner crops (top-left, top-right, bottom-left and bottom-right).\\
{\bf 10-Crops (10C)} - is an extension of the 5C policy; it applies the left-to-right horizontal flipping to the five crops of 5C for a total of 10 images.

These policies enable substantial improvements in the classification accuracy as reported  Table~\ref{tab:tta_accuracy}. Obviously, doubling the number of transformations (from 5C to 10C) increases the accuracy gain.  Moreover, the transformations get processed efficiently even on resource-constrained devices: on the Cortex-A53 CPU, cropping requires only \SI{0.8}{\milli\second} and horizontal flipping \SI{0.9}{\milli\second}. 


\begin{table}[!h]
    \centering
    \caption{Accuracy gain (in \%) of 5-Crops (5C) and 10-Crops (10C) TTA policies compared to the nominal values for static TTA and AdapTTA ($\tau=0.8)$.}\label{tab:tta_accuracy}
    \begin{tabular}{lcc|cc}
    \toprule
    \multirow{3}{*}{\bf ConvNet} & \multicolumn{2}{c|}{\bf Static TTA} & \multicolumn{2}{c}{\bf AdapTTA}\\
    {\bf ConvNet} & {\bf 5C} & {\bf 10C} & {\bf 5C} & {\bf 10C}\\
    \midrule
    MobileNetV1  &2.7 &3.1 &2.7 &3.1\\
    MobileNetV2  &2.2 &2.9 &2.2 &2.9\\
    \bottomrule
    \end{tabular}
\vspace{-0.5cm}
\end{table}







