




















\section{Conclusion  and Discussion}
This work proposes a fast adversarial training method to overcome catastrophic overfitting and improve adversarial robustness by introducing the priori-guided adversarial initialization and the novel adversarial label perturbation. 
Specifically, after exploring three adversarial initialization strategies, we propose to use the momentum of adversarial gradients of all previous epochs as the initialization of FGSM for AE generation. 
Inspired by AE perturbation, we propose the adversarial label perturbation by adding adversarial perturbation to the ground truth label. 
The perturbed labels and AEs are used for training.
Evaluations on four databases demonstrate that the proposed method can prevent catastrophic overfitting and improve adversarial robustness without additional calculation cost. 

\noindent \textbf{Limitations.} 
Our method has a side effect of improving robustness, \textit{i.e.,} the clean accuracy decreases. 
This effect also exists in other FAT methods. 
It might limit the application of FAT. 
We will rethink the robustness and accuracy trade-off in the future. 



\section{Conclusion}
In this paper, we investigate how to improve adversarial example quality from the perspective of initialization and propose to adopt historically generated adversarial perturbations to initialize adversarial examples. 
It can generate powerful adversarial examples with no additional calculation cost. 
Moreover, we also propose a simple yet effective regularizer to further improve model robustness, which prevents the current perturbation deviating too much from the prior-guided initialization.
The regularizer adopts both historical and current adversarial perturbations to guide the model learning. Extensive experimental evaluations demonstrate that the proposed method can prevent catastrophic overfitting and outperform state-of-the-art FAT methods at a low computational cost.




\section{Experiments}
\label{Experiments}

\input{table/CIFAR10_ResNet_v2}
\subsection{Experimental Setting}
\noindent \textbf{Dataset Settings.} To evaluate the effectiveness of the proposed FGSM-MEP, extensive experiments are conducted on four benchmark datasets that are the most widely used to evaluate adversarial robustness, \textit{i.e.}, CIFAR-10~\cite{krizhevsky2009learning}, CIFAR-100~\cite{krizhevsky2009learning}, Tiny ImageNet~\cite{deng2009imagenet}, and ImageNet~\cite{deng2009imagenet}. 
Following the commonly used settings in the adversarial training, we adopt ResNet18~\cite{he2016deep} and WideResNet34-10~\cite{DBLP:conf/bmvc/ZagoruykoK16} as the backbone on CIFAR-10 and CIFAR-100,  PreActResNet18~\cite{he2016identity} on Tiny ImageNet, and ResNet50~\cite{he2016deep} on ImageNet. 
We adopt the SGD optimizer \cite{DBLP:journals/nn/Qian99} with a learning rate of 0.1,  the weight decay of 5e-4, and the momentum of 0.9.
As for CIFAR-10, CIFAR-100, and Tiny ImageNet, following the settings of \cite{rice2020overfitting,pang2020bag},
the total epoch number is set to 110. The learning rate decays with a factor of 0.1 at the 100th and 105th epoch. 
As for ImageNet, following the settings of \cite{shafahi2019adversarial,wong2020fast}, the total epoch number is set to 90. 
The learning rate decays with a factor of 0.1 at the 30th and 60th epoch.
Experiments on ImageNet are conducted with 8 Tesla V100 and other experiments are conducted with a single Tesla V100. 
For our hyper-parameters, the decay factor $\mu$ is set to 0.3 and the hyper-parameter $\lambda$ is set to 10. More details are presented in the \textbf{supplementary material}.


\noindent \textbf{Evaluation Metrics.} For adversarial robustness evaluation, we adopt several widely used attack methods, \textit{i.e.,} PGD~\cite{madry2017towards}, C\&W~\cite{carlini2017towards}, and an ensemble of diverse parameter-free attacks, AA~\cite{croce2020reliable} that includes APGD~\cite{croce2020reliable}, APGD-T~\cite{croce2020reliable}, FAB~\cite{croce2020minimally}, and Square~\cite{andriushchenko2020square}. 
We set the maximum perturbation strength $\epsilon$ to 8 for all attack methods. 
Moreover, PGD attack is conducted with 10, 20 and, 50 iterations, \textit{i.e.}, PGD-10, PGD-20, and PGD-50. We report the results of the checkpoint with the best accuracy under the attack of PGD-10 as well as the results of the last checkpoint. 
All experiments in the manuscript are performed with the multi-step learning rate strategy. 
We also conduct experiments by using a cyclic learning rate strategy \cite{smith2017cyclical} which are presented in the \textbf{supplementary material}.



\noindent \textbf{Competing Methods.}
We compare the proposed method with a series of state-of-the-art FAT methods, \textit{i.e.,} Free-AT\cite{shafahi2019adversarial}, FGSM-RS~\cite{wong2020fast}, FGSM-GA\cite{andriushchenko2020understanding}, FGSM-CKPT\cite{kim2020understanding}, GAT~\cite{sriramanan2020guided} and NuAT~\cite{sriramanan2021towards}. 
We also compare with an advanced multi-step AT method, \textit{i.e.,} PGD-AT\cite{rice2020overfitting}), which is an early stopping version of the original PGD-based AT method~\cite{madry2017towards}. 


\input{table/CIFAR100_v2}
\input{table/Tiny_imagenet_v2}
\input{table/Imagenet}
\input{table/Ablation_study}

\subsection{Results on CIFAR-10} 
On CIFAR-10, we adopt ResNet18 as the backbone.  The results are shown in Table~\ref{table:cifar10_ResNet}.
Note that the results of using WideResNet34-10 as the backbone are shown in the \textbf{supplementary material}.
The proposed FGSM-MEP prevents catastrophic overfitting and 
even achieves better robustness than PGD-AT in most cases.
It costs much less time than PGD-AT. Specifically, under the AA attack, PGD-AT achieves an accuracy of about 48\% while the proposed FGSM-MEP achieves an accuracy of about 49\%. 

Compared with other FAT methods, the proposed FGSM-MEP achieves comparable robustness to the previous most potent FAT method, NuAT.
But FGSM-MEP costs less time than NuAT (73 min VS 104 min). 
Besides, for the last checkpoint, the proposed FGSM-MEP can achieve the best robustness performance under all attack scenarios. 
And for the best checkpoint, the proposed FGSM-MEP also achieves the best robustness performance under the PGD-10, PGD-20, and PGD-50.
In terms of efficiency, our training process is about 3 times faster than Free-AT, 2.5 times faster than FGSM-GA, and 1.4 times faster than GAT and NuAT. Previous fast adversarial training variants~\cite{andriushchenko2020understanding,kim2020understanding,sriramanan2020guided,sriramanan2021towards} improve the quality of adversarial examples in different ways. Though they can prevent catastrophic overfitting and improve model robustness, they all require much time for  quality improvement. 
Differently, we improve adversarial example quality from the perspective of initialization and propose to adopt historically generated adversarial perturbation to initialize the adversarial examples without additional calculation cost.


\subsection{Results on CIFAR-100}  
On CIFAR-100, we adopt ResNet18 as the backbone. 
The results are shown in Table~\ref{table:cifar100_ResNet}.
Compared with CIFAR-10, it is hard for the classification model to obtain robustness because the CIFAR-100 covers more classes. 
The proposed FGSM-MEP achieves comparable robustness to PGD-AT and costs less time than PGD-AT. Specifically, under the AA attack,  PGD-AT achieves an
accuracy of about 25\%, while our FGSM-MEP also achieves an
accuracy of about 25\%. 
Note that our training process is about 3 times faster than PGD-AT.
Compared with other FAT methods, 
the proposed FGSM-MEP also achieves the best robustness against AEs under all attack scenarios on the best and last checkpoints.
In terms of training efficiency, we observe similar results on CIFAR-10.



\subsection{Results on Tiny ImageNet and ImageNet} 

\noindent \textbf{Results on Tiny ImageNet.}  Following the setting of \cite{kim2020understanding,rice2020overfitting}, we adopt PreActResNet18 to conduct AT. 
The results are shown in Table~\ref{table:Tiny_Imagnet}. 
Compared with competing FAT methods, our FGSM-MEP achieves higher robust accuracy.
Compared with PGD-AT, our FGSM-MEP achieves better robustness performance under all attack scenarios on the best and last checkpoints. In terms of training efficiency, we observe similar results on CIFAR-10 and CIFAR-100.





\noindent \textbf{Results on ImageNet.} 
Following the setting of \cite{shafahi2019adversarial,wong2020fast}, we adopt ResNet50 to conduct experiments. 
Specifically, ResNet50 is trained with the maximum perturbation strength $\epsilon=2$, $\epsilon=4$, and $\epsilon=8$. The proposed FGSM-EP and FGSM-MEP require memory consumption to store the adversarial perturbation of the last epoch, which limits their application on ImageNet. Fortunately, FGSM-BP does not require memory consumption to conduct AT. Hence, on the ImageNet,
we compare our FGSM-BP with FGSM-RS and Free-AT. The results are shown in Table~\ref{table:Imagnet}. 
Compared with FGSM-RS and Free-AT, our FGSM-BP achieves the highest clean and robust accuracy. 
Our FGSM-BP requires a bit more calculation cost than FGSM-RS, but much less time than Free-AT.





\subsection{Ablation Study}
In this paper, we propose a regularization loss term to enforce function smoothness, resulting in improving model robustness. 
To validate the effectiveness of the proposed regularization, we adopt ResNet18 as the classification model to conduct ablation experiments on CIFAR-10. 
The results are shown in Table~\ref{tb:ablation}. It can be observed that combined with our regularization method, FGSM-BP, FGSM-EP, and FGSM-MEP can achieve better robustness performance under all attack scenarios. 







\section{Introduction}
Deep neural networks (DNNs) \cite{lecun2015deep,bai2021improving,zou2019reinforced,gu2021capsule,wei2018transferable,wang2021enhancing,jia2019comdefend,liang2020efficient,liang2022parallel,li2022semi} are vulnerable to adversarial examples (AEs) \cite{szegedy2013intriguing,chen2018robust,finlayson2019adversarial,duan2020adversarial,lin2019nesterov,xie2019improving,jia2020adv,gu2021effective,duan2021adversarial,bai2020targeted,bai2020improving} generated by adding imperceptible perturbations to benign images.
Standard adversarial training (SAT) \cite{madry2017towards,wang2019improving,jia2022adversarial} is one of the most efficient defense methods against AEs. 
It adopts projected gradient descent (PGD) ~\cite{madry2017towards}, a multi-step attack method, to generate AEs for training.
But, SAT requires much time to calculate gradients of the network's input multiple times.


\input{fig/time_acc}
\par To improve efficiency, fast adversarial training (FAT) methods \cite{goodfellow2014explaining,park2021reliably,zhang2019you,jia2022boosting} have been proposed. Goodfellow~\emph{et al.} first  \cite{goodfellow2014explaining} adopt FGSM to generate AEs for training, \textit{i.e.,} FGSM-AT, but it encounters the catastrophic overfitting. 
Wong \emph{et al.} \cite{wong2020fast} propose to combine FGSM-AT with random initialization along with early stopping to overcome the overfitting. 
Andriushchenko \emph{et al.} \cite{andriushchenko2020understanding}
propose a method from the view of regularization to enhance the quality of AEs, \textit{i.e.,} GradAlign. 
Kim \emph{et al.} \cite{kim2020understanding} propose a simple method to determine an appropriate step size of FGSM-AT to generate stronger AEs, \textit{i.e.,} FGSM-CKPT. 
Sriramanan \emph{et al.} \cite{sriramanan2020guided} introduce a relaxation term in the training loss to improve the quality of the generated AEs, \textit{i.e.,} GAT. 
Then, Sriramanan \emph{et al.}~\cite{sriramanan2021towards} propose a Nuclear-Norm regularizer to enhance
the optimization of the AE generation and the model training. 
These methods not only prevent
catastrophic overfitting but also achieve advanced defense performance by enhancing the quality of AEs.
However, compared with FGSM-RS, these advanced methods require high additional calculation costs to generate stronger AEs.



As AEs are critical for adversarial training, we study the AEs of both FAT and SAT during their training processes to explore the reason for catastrophic overfitting. 
Surprisingly, we observe a distinct difference between the attack success rates of their AEs in the late training stage. 
The attack success rate of FAT drops sharply after a few training epochs while the robust accuracy decreases to 0 (see FGSM-AT and FGSM-RS in Fig.~\ref{fig:difference}).
This phenomenon indicates that  overfitting happens when the quality of AEs becomes worse. 
However, the overfitting does not exist in the two-step PGD (PGD-2-AT) that can be treated as FGSM-AT with an adversarial initialization. It means adversarial initialization could be a solution to overfitting.  
\input{fig/ASR}
Based on the observation, we raise a question \textit{``can we obtain an adversarial initialization for FGSM-AT to maintain the high quality of AEs for avoiding catastrophic overfitting without additional calculation cost?"}
We investigate several initialization strategies and propose a prior-guided initialization by leveraging historically generated AEs, dubbed FGSM-PGI. 
Specifically, we first use the buffered perturbations of AEs from the previous batch and the previous epoch as the initialization for FGSM, respectively, called FGSM-BP and FGSM-EP. The two strategies are demonstrated to be effective. 
 To exploit complete prior information, 
 we then propose to leverage the buffered gradients from all previous epochs via a momentum mechanism as an additional prior, dubbed FGSM-MEP, which works the best. 
Furthermore, we provide a theoretical analysis for the proposed initialization method. 




Moreover, we also propose a simple yet effective regularizer based on the above prior-guided initialization to guide model learning for better robustness. 
The current perturbation can be generated via FGSM using the prior-guided initialization. 
The regularizer prevents the current perturbation
deviating too much from the prior-guided initialization, which is implemented by using a squared $L_2$ distance between the predictions of adversarial examples generated based on the current perturbation and the prior-guided initialization.

The proposed method not only prevents catastrophic overfitting but also improves the adversarial robustness against adversarial attacks. 
Fig.~\ref{fig:time_acc} illustrates the robustness and training efficiency of different FAT methods.
Our main contributions are in three aspects:
\textbf{1)} We propose a prior-guided adversarial initialization to prevent overfitting after investigating several initialization strategies. 
\textbf{2)} We also propose a regularizer to guide the model learning for better robustness by considering both the currently generated perturbation and the prior-guided initialization.
\textbf{3)} Extensive experiments on four datasets demonstrate that the proposed method can outperform state-of-the-art FAT methods in terms of both efficiency and robustness. 

\subsection{Prior-Guided Adversarial Initialization}
\label{sec:Initialization}
The above three observations motivate us to dive into the question \textit{``how to obtain adversarial initialization without additional calculation cost"}. 
We come up with using the adversarial perturbations of historical AEs as the initialization for FGSM to conduct adversarial training, dubbed FGSM-PGI. 
Such perturbations serve as the prior knowledge that we can freely obtain without additional gradient calculation except for extra storage.  
We explore three strategies of exploiting historical perturbations, \textit{i.e.,} taking perturbations from the previous batch, from the previous epoch, and from the momentum of all epochs, \textit{i.e.,} FGSM-BP, FGSM-EP, and FGSM-MEP.  



\noindent\textbf{Prior From the Previous Batch (FGSM-BP).}
We store the perturbations of AEs from the previous batch and regard them as the initialization of FGSM to generate AEs in the current batch. 
As the batch is randomly sampled, there is no correspondence between perturbations from the previous batch and samples from the current batch.  
Specifically, for a data point $\mathbf{x}$, we first add the perturbation from the previous batch onto it and then conduct FGSM based on the perturbed example. 
The adversarial perturbation can be defined as:
\begin{equation}
\label{eq:FSGM_BP}
\mathbf{\boldsymbol{\delta}}_{B_{t+1}}=\Pi_{[-\epsilon, \epsilon]}\left[\mathbf{\boldsymbol{\delta}}_{B_{t}}+\alpha \cdot \operatorname{sign}\left(\nabla_{\mathbf{x}} \mathcal{L}(f(\mathbf{x}+\mathbf{\boldsymbol{\delta}}_{B_{t}} ; \mathbf{w}), \mathbf{y})\right)\right],
\end{equation}
where $\mathbf{\boldsymbol{\delta}}_{B_{t+1}}$ is the adversarial perturbation in the $t+1$-th batch. 
Compared with FGSM-RS (see Eq.~\ref{eq:FGSM_RS}), we replace the random initialization with the adversarial perturbation from the previous batch.


\noindent\textbf{Prior From the Previous Epoch (FGSM-EP).}
We store the perturbations of all adversarial samples from the previous epoch and use them as initialization of FGSM to generate adversarial perturbations for the samples in the current epoch. 
Note that there is a correspondence between each perturbation from the previous epoch and the sample in the current epoch.  
 The adversarial perturbation is defined as:
\begin{equation}
    \label{eq:FSGM_EP}
    \mathbf{\boldsymbol{\delta}}_{E_{t+1}}=\Pi_{[-\epsilon, \epsilon]}\left[\mathbf{\boldsymbol{\delta}}_{E_{t}}+\alpha \cdot \operatorname{sign}\left(\nabla_{\mathbf{x}} \mathcal{L}(f(\mathbf{x}+\mathbf{\boldsymbol{\delta}}_{E_{t}} ; \mathbf{w}), \mathbf{y})\right)\right],
\end{equation}
where $\mathbf{\boldsymbol{\delta}}_{E_{t+1}}$ is the adversarial perturbation in the $t+1$-th epoch.  
Compared with FGSM-RS (see Eq.~\ref{eq:FGSM_RS}), we replace the random initialization with the adversarial perturbation from the previous epoch.


\vspace{1mm}
\noindent\textbf{Prior From the Momentum of All Previous Epochs (FGSM-MEP).}
To completely leverage historical adversarial perturbations during the whole training process, we propose to compute the momentum of one sample's gradients across all previous training epochs. 
Then, the gradient momentum is used as the initialization of FGSM for AE generation in the current epoch. 
There is also a correspondence between the gradient momentum and the sample in the current epoch. 
The adversarial perturbation can be defined as: 
\begin{align}
&\mathbf{g}_{c} =\operatorname{sign}\left(\nabla_{\mathbf{x}} \mathcal{L}(f(\mathbf{x}+\mathbf{\boldsymbol{\eta}}_{E_t} ; \mathbf{w}), \mathbf{y})\right), \\
&\mathbf{g}_{E_{t+1}} =\mu \cdot \mathbf{g}_{E_{t}} + \mathbf{g}_{c}, \\
&\mathbf{\boldsymbol{\delta}}_{E_{t+1}}=\Pi_{[-\epsilon, \epsilon]}\left[\mathbf{\boldsymbol{\eta}}_{E_{t}}+\alpha \cdot \mathbf{g}_{c} \right], \label{eq:FSGM_MEP_1}\\
&\mathbf{\boldsymbol{\eta}}_{E_{t+1}} =\Pi_{[-\epsilon, \epsilon]}\left[\mathbf{\boldsymbol{\eta}}_{E_{t}}+\alpha \cdot \operatorname{sign}(\mathbf{g}_{E_{t+1}}) \right]. \label{eq:FSGM_MEP_2}
\end{align}
Similar to FGSM-AT (Eq.~\ref{Eq:FGSM_AT}), $\mathbf{g}_c$ is regarded as the signed gradient and $\mathbf{g}_{E_{t+1}}$ is the signed gradient momentum in the $t+1$-th epoch.   
$\mu$ represents the decay factor. 
${\boldsymbol{\delta}}_{E_{t+1}}$ is the adversarial perturbation, similar to that of FGSM-EP. 
${\boldsymbol{\eta}}_{E_{t+1}}$ is the projected perturbation, which is used as adversarial initialization in the next epoch.  
Compared with FGSM-EP, FGSM-MEP exploits the gradient momentum involving information of all previous epochs as initialization, instead of using only information in the previous epoch. 



\subsection{Prior-guided Initialization based Regularization }
\label{sec:regularization}
We propose an effective regularization method based on the prior-guided initialization to improve model robustness.
Specifically, given the prior-guided initialization, we can generate a current perturbation via FGSM with the initialization. 
Both the current perturbation and the initialization can be used to create adversarial examples. 
Forcing the two types of adversarial examples to have similar predictions could help improve the smoothness of function. 
The proposed regularization term can be added into the training loss to update the model
parameters $\mathbf{w}_{t+1}$, as follows:
\begin{equation}
\mathbf{w}_{t+1}=\arg\min_{\mathbf{w}} [\mathcal{L}(f(\mathbf{x}+\boldsymbol{\delta}_{adv}; \mathbf{w}), \mathbf{y}) +
\lambda \cdot\left\|f(\mathbf{x}+\boldsymbol{\delta}_{adv} ; \mathbf{w})-{f}(\mathbf{x}+\boldsymbol{\delta}_{pgi}; \mathbf{w})\right\|_{2}^{2}],
\label{eq:loss}
\end{equation} 
where $\boldsymbol{\delta}_{pgi}$ represents the prior-guided initialization generated by one of the above three initialization methods. 
$\boldsymbol{\delta}_{adv}$ represents the current adversarial perturbation generated via FGSM using $\boldsymbol{\delta}_{pgi}$ as initialization.  
$\lambda$ is a trade-off hyper-parameter. 
The first term is the cross-entropy loss on AEs generated using the current perturbation.   
The second term represents the regularization, \textit{i.e.,} the squared $L_{2}$ distance between the predictions of the two types of adversarial examples.  
This term makes the learned model not only robust to currently generated AEs but also historically generated AEs. 
In this way, the proposed regularization term explicitly enforces the function smoothness around samples to improve model robustness. 


\par Based on the proposed prior-guided adversarial initialization and the regularization, we can establish our FAT framework. We evaluate the three prior-guided adversarial initialization approaches on CIFAR-10 (see results in Table~\ref{table:FGSM_MEP}) and find that they all can prevent catastrophic overfitting and achieve advanced model robustness against adversarial attacks.  
FGSM-MEP works the best in terms of robust accuracy under all attack scenarios.
\textbf{The FGSM-EP, FGSM-BP and FGSM-MEP algorithms are presented in the supplementary material}.


\subsection{Theoretical Analysis}
\label{sec:theoretical}
\begin{proposition}
    Let $\boldsymbol{\delta}_{pgi}$ be the prior-guided adversarial initialization in \textbf{FGSM-BP}, \textbf{FSGM-EP} or \textbf{FSGM-MEP}, $\boldsymbol{\hat{\delta}}_{adv}$ represents the current adversarial perturbation generated via FGSM using $\boldsymbol{\delta}_{pgi}$ as initialization, and $\alpha$ be the step size of \eqref{eq:FSGM_BP}, \eqref{eq:FSGM_EP}, \eqref{eq:FSGM_MEP_1} and \eqref{eq:FSGM_MEP_2}. If $\boldsymbol{\Omega}$ is a bounded set like
    \begin{equation}
        \boldsymbol{\Omega} = \big\{\boldsymbol{\hat{\delta}}_{adv}\ :\ \|\boldsymbol{\hat{\delta}}_{adv}-\boldsymbol{\delta}_{pgi}\|^2_2\leq{\epsilon}^{2}\big\},
    \end{equation}
    and the step size $\alpha$ satisfies $\alpha\leq\epsilon$, it holds that
    \begin{equation}
        \label{eq:propo_1}
        \begin{aligned}
            & \mathbb{E}_{\boldsymbol{\hat{\delta}}_{adv}\sim\boldsymbol{\Omega}}\big[\|\boldsymbol{\hat{\delta}}_{adv}\|_2\big]&\leq&\ \ \sqrt{\ \mathbb{E}_{\boldsymbol{\hat{\delta}}_{adv}\sim\boldsymbol{\Omega}}\big[\|\boldsymbol{\hat{\delta}}_{adv}\|^2_2\big]}\\
            & &\leq&\ \ \sqrt{\frac{1}{d}}\cdot\epsilon,
        \end{aligned}
    \end{equation}
    where $\boldsymbol{\hat{\delta}}_{adv}$ is the adversarial perturbation generated by \textbf{FGSM-BP}, \textbf{FSGM-EP} or \textbf{FSGM-MEP}, and $d$ is the dimension of the feature space. 
\end{proposition}
The proof is deferred to the \textbf{supplementary material}. The upper bound of the proposed method is $\sqrt{\frac{1}{d}}\cdot\epsilon$ which is less than the bound $\sqrt{\frac{d}{3}}\cdot\epsilon$ of FGSM-RS  provided in \cite{andriushchenko2020understanding}. Due to the norm of perturbation (gradient) can be treated as the convergence criteria for the non-convex optimization problem, the smaller expectation represents that the proposed prior-guided adversarial initialization will be converged to a local optimal faster than the random initialization with the same number of iterations.
 















\section{The Proposed Approach}
We first present our observations of revisiting catastrophic overfitting in Sec.~\ref{sec:Overfitting} and then explore several prior-guided adversarial initialization  strategies for FGSM-AT to prevent the overfitting in Sec.~\ref{sec:Initialization}, \textit{i.e.,} FGSM-PGI. We propose a simple yet effective regularization method for the FGSM-PGI to further improve model robustness in Sec.~\ref{sec:regularization}. Moreover, we also provide a theoretical analysis for the proposed initialization method in Sec.~\ref{sec:theoretical}.

\subsection{Revisting Catastrophic Overfitting}
\label{sec:Overfitting}
Catastrophic overfitting is a phenomenon that the robust accuracy of FAT drops sharply to 0\% in the late training stage (see the left figure of Fig.~\ref{fig:difference}), which was noticed by Wong \textit{et al.}~\cite{wong2020fast}. 
They then found random initialization of FGSM can help avoid the overfitting with training for limited epochs. 
However, if the training process of FGSM-RS goes on, catastrophic overfitting still appears, as revealed in ~\cite{andriushchenko2020understanding}, which indicates that random initialization must be conducted with early stopping. 
Hence, FGSM-RS does not solve the overfitting fundamentally.
FGSM-GA~\cite{andriushchenko2020understanding} proposes a method from the perspective of regularization in the loss to generate stronger AEs. FGSM-CKPT~\cite{kim2020understanding} proposes a method for adjusting the step size of FGSM to improve the quality of AEs in different training stages. 
GAT~\cite{sriramanan2020guided} adopts a relaxation term for the classification loss to generate stronger AEs during the AEs generation. NuAT~\cite{sriramanan2021towards} uses a Nuclear Norm regularizer to improve the optimization of the AEs generation.
These methods prevent catastrophic overfitting by improving the quality of AEs during the training.
However, FGSM-GA requires calculating gradients multiple times for regularization, FGSM-CKPT requires multiple times of forwarding propagation for step size selection, and GAT and NuAT require much more time to conduct regularization for generating powerful AEs, which significantly reduces efficiency. 

We reinvestigate catastrophic overfitting by comparing the intermediate results of the training processes of FAT and SAT on the CIFAR-10 dataset~\cite{krizhevsky2009learning}.  
For FAT, we study two methods, \textit{i.e.,} FGSM-AT and FGSM-RS. 
For SAT, we study PGD-AT~\cite{madry2017towards} with the iteration time of 2, \textit{i.e.,} PGD-2-AT.
As AEs are the key that distinguishes adversarial training from conventional training, we observe the quality of adversarial examples during the whole training process, \textit{i.e.,} whether adversarial examples can successfully attack the target model. 
The quality is evaluated by attack success rate (ASR). 
Fig.~\ref{fig:difference} illustrates the curves of the robust accuracy and the ASR of the above methods as well as ours. 
We also observe similar phenomenons on other benchmark datasets that are shown in the \textbf{supplementary material}.   

We summarize the observations as follows. First, it can be observed that the ASRs of FGSM-AT and FGSM-RS drop sharply at the 20-th and 74-th epoch, respectively, leading to the dramatic decreases of the robust accuracy. 
This indicates that the model robustness would collapse against adversarial attacks if the generated adversarial examples fail in attacking against the model during training.  
Note that the investigation from the perspective of the quality of adversarial example (\textit{i.e.,} ASR) is ignored by previous works. 
Second, FGSM-RS with random initialization delays the overfitting from the 20-th epoch to the 70-th epoch. 
Enhancing the initialization could alleviate the overfitting, but cannot essentially prevent it, which is also observed in ~\cite{andriushchenko2020understanding}. 
Third, surprisingly, PGD-2-AT does not suffer from the overfitting, which draws our attention. 
PGD-2-AT can be treated as FGSM-AT with an adversarial initialization of FGSM for adversarial example generation. 
Even after 110 training epochs, there are still a portion of high-quality adversarial examples that can fool the model. 
It indicates that adversarial initialization can improve the quality of adversarial examples. 
The initialization requires additional gradient calculation, which is not desirable.  

\section{Related Work}
\subsection{Adversarial Training}
Adversarial training (AT) variants \cite{zhang2019theoretically,rice2020overfitting,DBLP:conf/nips/WuX020} are effective in defending against AEs. 
Madry \emph{et al.}~\cite{madry2017towards} first formulate AT as a minimax optimization problem formulated as follows:
\begin{equation}
\min _{\mathbf{w}} \mathbb{E}_{(\mathbf{x}, \mathbf{y}) \sim \mathcal{D}}[\max _{{\boldsymbol{\delta}} \in \Omega}\mathcal{L}(f(\mathbf{x}+{\boldsymbol{\delta}} ; \mathbf{w}), \mathbf{y})],
\label{Eq:minimax}
\end{equation}
where $f(\cdot ; \mathbf{w})$ represents a deep neural network with parameters $\mathbf{w}$, $\mathcal{D}$ represents a joint data distribution of the benign image $\mathbf{x}$ and the GT one-hot label $\mathbf{y}$.
$\mathcal{L}(f(\mathbf{x} ; \mathbf{w}), \mathbf{y})$ represents the classification loss function.  $\mathbf{\boldsymbol{\delta}}$ represents the adversarial perturbation, and $\Omega$ represents a threat bound which can be defined as $ \Omega = \{\mathbf{\boldsymbol{\delta}}:\|\mathbf{\boldsymbol{\delta}}\| \leq \epsilon\}$ with the maximum perturbation strength $\epsilon$. 
Multi-step AT methods generate AEs by a multi-step adversarial attack method, \textit{i.e.}, projected gradient ascent (PGD) \cite{madry2017towards}. It can be defined as:
\begin{equation}
\mathbf{\boldsymbol{\delta}}_{t+1}=\Pi_{[-\epsilon, \epsilon]}\left[\mathbf{\boldsymbol{\delta}}_{t}+\alpha \cdot \textrm{sign} \left(\nabla_{\mathbf{x}} \mathcal{L}\left(f\left(\mathbf{x}+\mathbf{\boldsymbol{\delta}}_{t} ; \mathbf{w}\right), \mathbf{y}\right)\right)\right],
\end{equation}
where $\mathbf{\boldsymbol{\delta}}_{t+1}$ represents the perturbation at the $t+1$-th iteration, $\alpha$ represents the attack step size, and $\Pi_{[-\epsilon, \epsilon]}$ is the projection that maps the input to the range $[-\epsilon, \epsilon]$. This is a prime PGD-based AT framework proposed in \cite{madry2017towards}. 
Following this work, 
many advanced variants \cite{DBLP:conf/icml/WangM0YZG19,DBLP:conf/iclr/0001ZY0MG20,DBLP:conf/iclr/BaiZJXM021,kannan2018adversarial,pang2020boosting,roth2019adversarial} have been proposed from different perspectives to improve model robustness. 
An early stopping version \cite{rice2020overfitting} of PGD-based AT stands out amongst them. 

\subsection{Fast Adversarial Training}
\par Although multi-step AT methods can achieve good robustness, they require lots of calculation costs to generate AEs for training. 
Fast adversarial training variants that generate AEs by the one-step fast gradient sign method (FGSM) \cite{goodfellow2014explaining} are proposed to improve the efficiency, which can be dubbed FGSM-based AT methods.
The perturbation of FGSM-AT \cite{goodfellow2014explaining} is defined as:
\begin{equation}
\mathbf{\boldsymbol{\delta}}=\epsilon \cdot \textrm{sign}\left(\nabla_{\mathbf{x}} \mathcal{L}(f(\mathbf{x} ; \mathbf{w}), \mathbf{y})\right),
\label{Eq:FGSM_AT}
\end{equation}
where $\epsilon$ is the maximal perturbation strength. 
Though FGSM-based AT accelerates the training speed of AT, it encounters catastrophic overfitting. 
Specifically, after a period of training, the trained model suddenly cannot defend against the AEs generated by PGD. 
Wong \emph{et al.}~\cite{wong2020fast} propose to conduct FGSM-based AT with a random initialization. 
Its perturbation can be defined as: 
\begin{equation}
\mathbf{\boldsymbol{\delta}}=\Pi_{[-\epsilon, \epsilon]}\left[\mathbf{\boldsymbol{\boldsymbol{\eta}}}+\alpha \cdot \textrm{sign}\left(\nabla_{\mathbf{x}} \mathcal{L}(f(\mathbf{x}+\mathbf{\boldsymbol{\boldsymbol{\eta}}} ; \mathbf{w}), \mathbf{y})\right)\right],
\label{eq:FGSM_RS}
\end{equation}
where  $\boldsymbol{\boldsymbol{\eta}} \in \mathbf{U}(-\epsilon, \epsilon)$ represents the random initialization, $\mathbf{U}$ is a uniform distribution, and $\alpha$ is set to $1.25\epsilon$. After that, a series of FAT methods enhances the quality of AEs to improve the defense performance. In detail, 
Andriushchenko \emph{et al.} \cite{andriushchenko2020understanding} demonstrate that using the random initialization cannot prevent catastrophic overfitting. 
They propose a regularizer to generate stronger AEs, \textit{i.e.,} FGSM-GA. 
Kim \emph{et al.}~\cite{kim2020understanding} reveal that FGSM-RS adopts AEs with the maximum perturbation instead of ones in the adversarial direction, resulting in the overfitting. They then propose a method to determine an appropriate step size to generate powerful AEs to improve model robustness. 
Sriramanan \emph{et al.} \cite{sriramanan2020guided} enhance the quality of the generated AEs by using a relaxation term for the classification loss, \textit{i.e.,} GAT. Besides, Sriramanan \emph{et al.}~\cite{sriramanan2021towards} design a Nuclear-Norm regularizer to enhance
optimization of the AEs generation to improve the model robustness, \textit{i.e.,} NuAT. 
Though these methods can prevent overfitting and achieve advanced defense performance, they require much additional calculation cost to conduct AT.  













