\section{Introduction}
Adversarial attacks \cite{Szegedy2014, Goodfellow2015, Papernot2016limitations, Moosavi2016, CW2017, Madry2018, Athalye2018obfuscated} add small, imperceptible but well optimized noise to the clean image which can elicit an incorrect decision from the model.
These attack methods  \cite{poursaeed2018generative, dong2017discovering, xie2019improving, inkawhich2020perturbing, gao2020patch, wang2021admix} craft adversarial examples that can broadly be categorized based on how much information is available about the target model. In a \emph{white-box} attack setting, the attacker has complete knowledge of the target model and can directly optimize adversarial perturbations for the given model. In a more realistic \emph{black-box} attack setting, the attacker does not have access to the target model, its architectural details or targeted task (\emph{e.g}\bmvaOneDot, classification, segmentation or object detection). In such a case, adversarial examples are created on a surrogate model and then transferred to the black-box model.  Adversarial examples generated from surrogate models trained on a large-scale dataset in a supervised manner have better transferability \cite{naseer2019cross, dong2018boosting}. Transferability of such attacks improves further by fine-tuning the surrogate model to enhance their representation capacity \emph{e.g}\bmvaOneDot, by finding better self-ensemble from a given pretrained model \cite{naseer2021improving}. The adversarial transferability hence depends on the generalizability of the surrogate model. Such attacks are also restricted by the availability of a pretrained surrogate and information about the label space.

In this work, we call into question this assumption and consider a stronger threat model where an attack is launched from few unannotated or cross-domain samples (\emph{e.g}\bmvaOneDot, painting to ImageNet samples) without any knowledge of target networks or tasks. This threat model poses a challenge on how to learn an effective surrogate model from the limited unlabelled data and then how to generate self-supervised transferable adversarial examples.  

With limited availability of data, neural networks can easily memorize the data \cite{zhang2021understanding} even with strong augmentations \cite{li2020practical}. Therefore, Li \emph{et al}\bmvaOneDot \cite{li2020practical} propose to reconstruct transformed input images to learn a surrogate model. However, their effective surrogate training and attack approach requires supervision though annotated data samples. In order to reduce overfitting over few data samples and to find robust features, we take inspiration from adversarial training \cite{Goodfellow2015, springer2021little} and propose self-supervised Adversarial Pixel Restoration to train a surrogate model. The min-max objective of our proposed training allows to find a flatter minima with robust features which compliments our self-supervised adversarial attack to achieve higher adversarial transferability. Our main contributions are as follows:
\begin{itemize}
    \item We propose self-supervised Adversarial Pixel Restoration to find highly transferable patterns by learning over flatter loss surfaces. Our training approach allows launching cross-domain attacks without access to large-scale labeled data or pretrained models.
    \item Our adversarial attack is self-supervised in nature and  independent of any task-specific objective. For instance, our approach optimizes the robust transformed loss surface of the surrogate via fooling its reconstruction ability. This allows to generate task independent adversaries. Therefore, our approach can transfer perturbations to a variety of tasks as we demonstrate for classification, object detection, and segmentation.
    \item We provide a thorough analysis  of our proposed method to establish it's effectiveness. We observe that our approach leads to smoother loss landscapes (Fig. \ref{Loss surfaces}), helping in crafting more generalizable adversarial examples. Our method remains effective even in extreme data scarcity \emph{e.g}\bmvaOneDot when trained on two data samples only. (see Sec. \ref{subsec: Ablative Analysis}).
    
   
   
\end{itemize}
 
\section{Related Work}

Several gradient-based methods  \cite{Goodfellow2015, kurakin2018adversarial, mlatscale, ensembleadv} have been proposed for crafting adversarial examples directly on the target classifiers. However, when the access to the target model is limited to just a finite amount of queries, current methods either rely on the transferability of surrogate models \cite{DasTdatafree, Papernot2016transferability} \cite{Papernot2017} or estimate the gradients/boundary of the target model \cite{ZOOgradest, decisonbased_Brendel2018, decison_based_Narodytska2017}. Both of the above approaches require a non-trivial number of queries from the target model, making it highly impractical in real-case scenarios. A practical threat model was introduced in \cite{naseer2019cross}, where the attacker does not have access to the training distribution of the target model, as well as querying is prohibited. Authors train a generator-based surrogate model with the help of training data and a pretrained classifier obtained from a different domain than the target models. However, the pretrained classifier as well as the generator are trained on a large annotated dataset. 

In \cite{li2020practical}, a stronger threat model with access to limited data samples (order of tens) was proposed. Inspired from self-supervised learning methods, autoencoder-based surrogate models are trained with limited data. However, the transferability of the autoencoders trained in an unsupervised manner is still moderate. Most of the previous works have explored the transferability of surrogate  models trained on the target model's training set. In  \cite{springer2021adversarial}, the authors hypothesize that the features of robust classifiers are more universal than  non-robust classifiers, which lead to adversarial examples that are more transferable.  Building on this, \cite{springer2021little}  observe that having a classifier adversarially trained with a small perturbation budget, leads to highly transferable adversarial examples.
Unlike prior works, we focus on constructing robust surrogate models in a fully unsupervised manner. We consider  the case of training robust model with limited data 
to improve adversarial transferability. Furthermore, we also consider a practical scenario of availability of large unlabelled dataset for training of robust models which can be used for constructing cross-domain and cross-task  adversaries
\section{Adversarial Pixel Restoration for Transferable Perturbations}
Our goal is to learn a surrogate model, $\mathcal{F}$, from a given unlabelled data distribution, $P_s$, with a set of only few image samples ($\leq 20$). This setting is in contrast to the existing transferable adversarial attacks \cite{dong2018boosting, naseer2021improving, xie2019improving, Naseer_2021_ICCV} that assume access to a surrogate model trained on a large-scale annotated data (\emph{e.g}\bmvaOneDot, ImageNet \cite{deng2009imagenet}) in a supervised fashion. In the presence of unlabeled data, however, the surrogate model can be trained by defining a self-supervised task, $\mathcal{T}_s$, such as predicting rotation \cite{gidaris2018unsupervised}, solving jigsaw puzzle \cite{noroozi2016unsupervised} or by matching different views of the same input sample \cite{caron2021emerging}. A major challenge is that deep neural networks can easily memorise the data \cite{zhang2021understanding} and quickly overfit the few available samples even after applying strong data augmentation techniques \cite{li2020practical}. This results in a surrogate model with less generalizable representations and consequently, adversarial attack launched from such a model has weak transferability (Sec.~\ref{subsec: Transferability from Few In-Domain Samples}).

We propose adversarial pixel restoration as a prior to train the surrogate model $\mathcal{F}$ that boosts transferability of the adversarial attacks (see Fig. \ref{fig: concept_fig} ). We create adversarial examples from the original input images by attacking the source model in the pixel space using our proposed adversarial attack (Sec.~\ref{subsec: Adversarial Pixel Transformations}). Our approach also shifts the position of input pixels via transformations such as rotation or jigsaw shuffle. We then train a model to denoise and restore pixels to their original positions via our proposed adversarial training (Algo. \ref{algo: Adversarial Denoising and Restoration}).   The surrogate model $\mathcal{F}$ in our case consists of an autoencoder as explained below.


\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{images/CopyofConceptdrawiodrawio.png} \vspace{-0.7em}
\vspace{-2em}
\caption{\small Our approach trains an autoencoder based surrogate model via self-supervised adversarial pixel restoration to learn generalizable representations from a limited number of data samples ($\le 20$). Our training is based on a min-max strategy. We first generate adversarial examples by fooling model's reconstructive ability \emph{(maximization)}, followed by updating the model parameters based on restoration of the transformed adversarial and clean sample \emph{(minimization)}. Our approach allows launching transferable self-supervised adversarial attacks without any knowledge of target (black-box) model.
}\vspace{-1em}
\label{fig: concept_fig}
\end{figure}

\subsection{Surrogate Architecture} 
\label{subsec: Surrogate Architecture}
 The surrogate model is based on an autoencoder \cite{li2020practical}. The encoder consists of a stack of convolution layers. At the beginning of the architecture, convolution layers with a larger kernel size and stride help reduce the spatial resolution of the feature maps, followed by multiple residual blocks where the size of the feature maps is kept constant. The decoder is a lightweight model consisting of two transpose convolution layers to upsample the feature maps. The adversarial training of these autoencoders is based on denoising and pixel restoration as explained next in Sec.~\ref{subsec: Adversarial Training via Denoising}.
 
\subsection{Adversarial Training via Denoising }
\label{subsec: Adversarial Training via Denoising}
\vspace{-0.3em}
The feature space of a slightly robust classifier produces highly transferable adversarial examples \cite{springer2021little}. However, adversarial training of such models is computationally demanding due to iterative training and also requires a large-scale labeled dataset that might not be accessible to the attacker in real-world scenarios. We assume a more practical threat model, where the attacker has access to a data distribution $P_s$ with limited number of samples without any annotations. We adversarially learn a surrogate model on the data distribution $P_s$ to transfer adversarial perturbations to the target (black-box) models trained for different tasks (\emph{e.g}\bmvaOneDot, classifications, object detection and segmentation) on possibly different target distribution $Q_t\neq P_s$. In such challenging attack settings, we adversarially train an autoencoder $\mathcal{F}$ via min-max training strategy \cite{Goodfellow2015}. At maximization step, we create adversarial examples by fooling the model $\mathcal{F}$ via adversarial pixel transformations with a single step attack (Algo. \ref{algo: Adversarial Denoising and Restoration}). At minimization step, we denoise and restore the feature and pixel space to achieve generalizable loss surfaces (Fig. \ref{Loss surfaces}) which leads to more transferable attack.

\begin{algorithm}[t]
	\caption{
		Adversarial Denoising and Restoration
	}
	\label{algo: Adversarial Denoising and Restoration}
	\begin{algorithmic}[1]
		\Require Source data distribution $P_s$, pixel transformation $\mathcal{T}_s$, attack step size $\delta$, perceptual budget $\epsilon$, balancing paramter $\lambda$, and maximum training iterations $T$
		\Ensure Randomly initialize surrogate model $\mathcal{F}$
		\For {$t\in [1, 2, \ldots T]$}
    		\State Randomly sample from $P_s$: $\bm{x} \sim P_s$
    		\State Initialize adversary $\bm{x}' \leftarrow$ $\bm{x}$
    		\State Optimize  $\bm{x}'$ using Eq. \ref{eq: attack_loss} or \ref{eq: attack_loss_prototypical}: \algorithmiccomment{Adversarial Pixel Transformation}
    		\begin{equation*}
    		    \bm{x}' \gets \bm{x} + \delta\times\text{sign}\left(\nabla\mathcal{L}_{max}\right)
    		\end{equation*}
    		
    		\State Project adversaries within allowed perceptual budget: $\bm{x}' \gets \text{clip}\left(\bm{x}', \bm{x}-\epsilon, \bm{x}+\epsilon \right)$
    		
    		\State Forward-pass $\bm{x}$ and $\bm{x}'$ through model $\mathcal{F}$ and update its parameters $\theta$ by minimizing the loss given in Eq. \ref{eq: overall_loss}: \algorithmiccomment{Pixel Restoration}
    		\begin{equation*}
    		    \theta \gets \theta - \alpha\times\nabla\mathcal{L}_{min},
    		\end{equation*}
    		where $\alpha$ is the learning rate.
		\EndFor
	\end{algorithmic}
\end{algorithm}


\subsubsection{Adversarial Pixel Transformations}
\label{subsec: Adversarial Pixel Transformations}
For given input samples $\bm{x} \sim P_s$, we first find adversarial example, $\bm{x}' \;\; \text{subjected to:} \;\;  \Vert \bm{x} - \bm{x}' \Vert_{\infty} \leq \epsilon$, by maximizing the following objective ($\mathcal{L}_{max}$):
\begin{align}
\label{eq: attack_loss}
\underset{\bm{x}'}{\text{maximize}} \quad \mathcal{L}_{max} =  \|\mathcal{F} \left(\mathcal{T}_s(\bm{x}')\right)- \bm{x}\|^p,
\end{align}
where $\mathcal{T}_s$ represents the pixel transformation (\emph{e.g}\bmvaOneDot, rotation or jigsaw shuffle) that shifts the pixel positions. Therefore, our attack fools the model's ability to restore the transformed pixel space by maximizing the loss presented in Eq.~\ref{eq: attack_loss} and ultimately help to robustify self-supervised features. Our attack approach can also be extended to benefit from supervisory signals \emph{e.g}\bmvaOneDot, by fooling prototypes \cite{snell2017prototypical, li2020practical} as follows:
\begin{equation}
\label{eq: attack_loss_prototypical}
\underset{\bm{x}'}{\text{maximize}} \quad \sum^{C}_{c=1} \left(y_c \, \|\mathcal{F} \left(\mathcal{T}_s(\bm{x}')\right)- \bm{x}^{(c)}\|^p\right),
\end{equation}
where $C$ represents the number of categories, $y_c$ represents one-hot encoded labels, and $\bm{x}^{(c)}$ is the chosen prototype for a particular class. Our proposed attack objective in Eq.~\ref{eq: attack_loss_prototypical} helps to optimize for robust discriminative features with better adversarial transferability.


\subsubsection{Pixel Restoration}
\label{pixel_restoration}
For a given adversarial sample, $\bm{x}'$ created using Eqs. \ref{eq: attack_loss} or \ref{eq: attack_loss_prototypical},  we train the surrogate model, $\mathcal{F}$, by pixel restoration. Our loss function penalizes the model $\mathcal{F}$ by minimizing the reconstruction error between the original sample, $\bm{x}$, and the model's output for the transformed adversarial as well as the transformed original sample as follows:
\begin{equation}
\label{eq: out_loss}
    \mathcal{L}_{out} =\|\mathcal{F} \left(\mathcal{T}_s(\bm{x}')\right)-  \bm{x}\|^p + \|\mathcal{F} \left(\mathcal{T}_s(\bm{x})\right)-  \bm{x}\|^p.
\end{equation}

We further regulate the model's feature space during adversarial training by enforcing alignment between the original and adversarial feature distributions as follows:
\begin{equation}
\label{eq: feature_loss}
\mathcal{L}_{{feature}}=\|\mathcal{F}^{n} \left(\mathcal{T}_s(\bm{x}')\right)- \mathcal{F}^{n} \left(\mathcal{T}_s(\bm{x})\right)\|^p,
\end{equation}
where $\mathcal{F}^{n}$ represents the intermediate (encoder) layer output. Overall training objective is,
\begin{equation}
\label{eq: overall_loss}
    \mathcal{L}_{min} = \mathcal{L}_{out} + \lambda \mathcal{L}_{feature},
\end{equation}
where $\lambda$ is the balancing parameter.


\subsubsection{Behavior of Robust Loss Surfaces}
We visualize the loss landscapes of our trained autoencoders (Fig. \ref{Loss surfaces}). We use the filter normalization method proposed in \cite{li2018visualizing}, which shows the structure of the loss surface along with random directions near the optimal pretrained parameters. We observe that our approach leads to more flatter minima as compared to the baseline method \cite{li2020practical}. This has significant effect on finding generalizable adversarial example with better transferability (Sec.~\ref{subsec: Transferability from Few In-Domain Samples}).

\begin{figure}[t]
\begin{minipage}{0.69\linewidth}
    \centering
    
    \includegraphics[trim= 20mm 30mm 20mm 40mm, clip, height=2.9cm, width=0.49\textwidth]{images/Loss_Surface_Plots/Rotate.png}
    \includegraphics[trim= 20mm 30mm 20mm 40mm, clip,  width=0.49\textwidth]{images/Loss_Surface_Plots/Rotate_Ours.png}\\
    \vspace{-1em}
     \includegraphics[trim= 20mm 30mm 20mm 50mm, clip, width=0.49\textwidth]{images/Loss_Surface_Plots/Jigsaw.png}
    \includegraphics[trim= 20mm 30mm 20mm 50mm, clip, height=2.5cm, width=0.49\textwidth]{images/Loss_Surface_Plots/Jigsaw_Ours.png}
\end{minipage}
\hspace{0.01\linewidth}
\begin{minipage}{0.28\linewidth}
    \caption{\small Reconstruction loss landscape of autoencoders. The first column shows the loss landscapes of autoencoders trained using the baseline method on the task of rotation (\emph{top}) and jigsaw (\emph{bottom}). The second column shows the corresponding flatter loss surfaces obtained by training the autoencoders using our approach.}
    \label{Loss surfaces}
\end{minipage}
\end{figure}




 \section{Experimental Protocols}
 \label{sec: Experimental Protocols}

For a pixel range [0, 1], we create $l_\infty$ adversarial examples under perceptual budget of $\epsilon \le 0.1$ following \cite{li2020practical}. We show adversarial transferability of our approach against ImageNet trained models (Convolutional and ViTs), object detection and segmentation from in-domain and cross-domain (paintings, medical scans ) data samples. We used Adam \cite{kingma2014adam} optimizer with a learning rate of 0.001 for our proposed adversarial pixel restoration (Algo. \ref{algo: Adversarial Denoising and Restoration}) at $\epsilon \le \frac{2}{255}$, $\delta = \frac{2}{255}$, $\lambda=1$, $p=2$. We provide detailed ablative analysis on the effect of these hyper-parameters in Sec.~\ref{subsec: Ablative Analysis}.

\noindent\textbf{Surrogate Training  with Few Samples:} Similar to \cite{li2020practical}, we assume access to only few data samples (\emph{e.g}\bmvaOneDot, 20 samples to train a single autoencoder). We apply Eq. \ref{eq: attack_loss} as adversarial pixel transformation ($\mathcal{T}$) based on rotation or jigsaw in an unsupervised setting. We also incorporate our method in the supervised prototypical training of autoencoders mentioned in \cite{li2020practical}, were the reconstruction objective function used during the minimization and maximization step is formulated by Eq. \ref{eq: attack_loss_prototypical}. We compare the best prototypical setting reported in \cite{li2020practical}, comprising of multiple decoder networks. All models are trained for the same no. of iterations as \cite{li2020practical}. We provide pseudo code of Algo. \ref{algo: Adversarial Denoising and Restoration} in Appendix  \ref{appendix: pseudocode}.  


\noindent\textbf{Surrogate Training with Large Dataset:}  There is abundance of unannotated data available via online sources. Therefore, we also scale our proposed self-supervised adversarial training to large-scale datasets. Specifically, we train a surrogate model (single autoencoder) on paintings (79k samples) \cite{paintings}, CoCo (40k samples) \cite{lin2014microsoft}, and Comics (50k samples) \cite{comics} to test unsupervised cross-domain adversarial transferability of our method.   

\noindent\textbf{Target Models:}
Adversarial perturbations from our trained autoencoders (surrogate models) are transferred to classification models including convolutional: VGG-19 \cite{Simonyan2015}, Inception-v3 (Inc-V3) \cite{Szegedy2016}, ResNet-152 (Res152) \cite{He2016}, Dense161 \cite{Huang2017densely}, SeNet \cite{Hu2018}, Wide-ResNet-50 (WRN) \cite{Zagoruyko2016}, and MobileNet-V2 (MNet-V2) \cite{Sandler2018mobilenetv2}, and Vision Transformers: ViT-T and ViT-S \cite{dosovitskiy2020image}, DeiT-T and DeiT-S \cite{touvron2021training}. We also evaluated adversarial vulnerability of robust ResNet-50 models \cite{salman2020adversarially}. Further, we transfer attack to DETR \cite{carion2020end} and DINO \cite{caron2021emerging} to evaluate on object detection and video segmentation tasks. 

\noindent\textbf{Evaluation Metrics:} We report drop in Top-1 (\%) accuracy, Mean Average Precision (mAP), and Jaccard Index for classification, object detection and segmentation, respectively. 

\noindent\textbf{Datasets:} We evaluate classification models on 5k samples from ImageNet validation set in the same setting as \cite{li2020practical}. DETR and DINO are evaluated on CoCo (5k samples) and DAVIS (2k samples) validation sets respectively.

\noindent\textbf{Baseline Adversarial Attack:} For adversarial prototypical training, we use the same supervised attack objective as proposed by \cite{li2020practical} for direct comparison. Specifically, we use 200 iterations of I-FGSM \cite{kurakin2018adversarial} followed by 100 iterations of ILA \cite{huang2019enhancing}. The attack objective for the surrogate models trained in self-supervised manner is simply based on maximizing the reconstruction error as described in Eq. \ref{eq: attack_loss}.



\subsection{Results}

\subsubsection{Transferability from Few In-Domain Samples}
\label{subsec: Transferability from Few In-Domain Samples}
Our surrogate models trained on few data samples show significantly higher adversarial transferability as compared to the baseline \cite{li2020practical}. Attack generated on our self-supervised models (rotation, and jigsaw) performs even better than supervised (prototypical) models of \cite{li2020practical} (Tables \ref{tab:no-box table}, \ref{tab:no-box table_transformers}). Our approach further boosts the transferability rates when combined with supervised adversarial prototypical training. This compliments the benefits of our method in both supervised and self-supervised settings. We observe that vision transformers are more robust as compared to convolutional networks against such attacks \cite{naseer2021intriguing}, however, our approach provides non-trivial gains in fooling the Vision Transformers (Table \ref{tab:no-box table_transformers}). Similarly, adversarially robust models \cite{salman2020adversarially} are less vulnerable to such attacks (Fig. \ref{fig: transferability against robust models}).  


\input{tables/inDomain_tables_classification}
\input{figures/inDomain_fig_classification_robust_models}
\input{tables/crossDomain_table_classification}
\input{tables/crossDomain_table_detection_segmentation}



\subsubsection{Transferability from Cross-Domain Samples}
Extra unsupervised data can boost the performance of adversarial training \cite{carmon2019unlabeled}. We extend our approach to large-scale, unlabelled datasets to observe its effect on adversarial transferability of our approach as explained below. All surrogate models are trained for 50 epochs for cross-task adversarial transferability experiments.




\noindent \textbf{Classification:} In this task, we train a single autoencoder and transfer its adversarial perturbations to ImageNet trained convolutional models as described in Sec.~\ref{tab: single_surrogate_classification_results}. We observe that our proposed adversarial training significantly improves upon the baseline \cite{li2020practical} (Table \ref{tab: single_surrogate_classification_results}).  We further note that the surrogate models trained on `paintings' dataset show higher adversarial transferability while `Rotation' as pixel transformation performs better. Further analysis on adversarial transferability of our attack is provided in Appendix \ref{appendix: results_eps_0.08}.

\noindent \textbf{Object Detection and Segmentation:} Our adversarial attack based on simple transformed reconstruction error (Eq.~\ref{eq: attack_loss}) compliments our proposed adversarial training and successfully fools DETR for object detection and DINO for video segmentation (Tables \ref{tab:DETR_results} \& \ref{tab:DINO_results}). This signifies that our attack can be launched in real-world setting without any knowledge about the deployed vision system. 



\subsection{Ablative Analysis}
\label{subsec: Ablative Analysis}
We thoroughly analyze and develop better understanding about the behavior of our approach by studying the effect of its different components including, \textbf{a)} \emph{Effect of Perceptual Budget $\epsilon$ for our Single step Attack (Eq. \ref{eq: attack_loss})}, \textbf{b)} \emph{Effect of Iterative 
Attack during Training}, \textbf{c)} \emph{Effect of Training Iterations and Data size}, and \textbf{d)} \emph{Contribution of Losses}. All ablative experiments are conducted in limited data setting (Sec.~\ref{subsec: Transferability from Few In-Domain Samples}).



\noindent \textbf{Effect of Perceptual Budget ($\epsilon$):} Our adversarial training (Algo. \ref{algo: Adversarial Denoising and Restoration}) is computationally efficient as it is based on a single attack (maximization) step. The effect of attack step size on model generalizability and hence its transferability is presented in Table \ref{tab: effect of epsilon}. We observe that adversarial perturbation computed on models trained with a smaller $\epsilon$ results in significant improvements in terms of attack transferability. This behaviour is in line with \cite{springer2021little} which shows that slightly robust classifier models generate highly transferable adversarial examples. This is further evident by the shift in attention caused by our method (Fig. \ref{fig:attn_vis}).

\noindent\textbf{Effect of Iterative Attack:} Increasing the attack iterations during our adversarial training reduces adversarial transferability, \emph{e.g}\bmvaOneDot, average Top-1 (\%) accuracy of convolutional models increases for attacks generated from autoencoders trained for 4 iteration (30.9\%) vs. single iteration (26.87\%) for rotation transformation. This is because attack becomes unsuccessful against highly robust autoencoders which leads to poor performance. 

\noindent \textbf{Contribution of Losses:} We explore the effect of our proposed pixel and feature reconstruction losses (Eq. \ref{eq: overall_loss}) in Table \ref{tab: Loss_contrb}. We observe that feature reconstruction compliments the pixel reconstruction and leads to better surrogate model with more transferable adversarial space.

\noindent \textbf{Training iterations and Data Size:}  We explore the effect on transferability of surrogate models w.r.t \textbf{a)} training iterations, and \textbf{b)} the number of data samples in Fig. \ref{fig: ablation iterations and datasize}. Note that \cite{li2020practical} trains a single autoencoder on 20 samples and thus needs 250 autoencoder to attack 5k ImageNet validation samples. The number of autoencoders increases to 2.5k when only 2 samples are available for training of single model. In the same setting, the performance of our method improves with more iterations (Fig. \ref{fig: ablation iterations and datasize} \emph{left plot}) in contrast to \cite{li2020practical}. We report average Top-1 (\%) accuracy across all convolutional models (\emph{lower is better}). Similarly, as we increase the number of data samples during training and reduce the number of autoencoders, the performance of our approach significantly improves as compared to the baseline \cite{li2020practical}. This indicates that our adversarial objective successfully reduces overfitting while increases generalizability of crafted adversarial perturbations.

\input{tables/ablative_tables}

\begin{figure}[!t]
\small \centering
\begin{minipage}{0.68\textwidth}
    \begin{minipage}{.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{images/ablation/ab1.png}
    \end{minipage}
    \begin{minipage}{.49\textwidth}
        \centering
        \includegraphics[width=\linewidth]{images/ablation/ab2.png} 
	\end{minipage}
\end{minipage}
\begin{minipage}{0.30\textwidth}
\vspace{-1em}
    \caption{\small Top-1 (\%) average accuracy against convolutional networks. The performance of our method improves with more training iterations and data size in contrast to the baseline \cite{li2020practical}.}
\label{fig: ablation iterations and datasize}
\end{minipage}
\vspace{-0.5em}
\end{figure}

\input{figures/attention_maps}




\section{Conclusion}
In this work, we show the benefits of adversarial training to learn transferable adversarial perturbations. Our approach trains an effective surrogate by learning to restore adversarial pixel transformations created via our proposed attack. Our adversarial training reduces overfitting during training and can exploit very few data samples to learn meaningful adversarial features while it can also scale to large unsupervised datasets. Our attack is task independent and allows cross-domain attacks (\emph{e.g}\bmvaOneDot, learning surrogate on comics and transferring its perturbations to models trained on natural images). Our results bring attention to the use of self-supervised adversarial training for transferable adversarial attacks. 








