\section{Introduction}
\figconflict 

\IEEEPARstart{S}{emantic} segmentation is a fundamental research field in computer vision. It aims to predict the class of each pixel in a given image. The availability of large labeled datasets~\cite{everingham2011pascal,zhou2017scene} and the development of deep neural networks~\cite{minaee2020image} have resulted in significant advancements. The vast majority of semantic segmentation research focuses on the scenario where all data is jointly available for training.
However, for many real-world applications, this might not be the case, and it would be necessary to incrementally learn the semantic segmentation algorithm. Examples include scenarios where the learner has only limited memory and cannot store all data (common in robotics), or where privacy policies might prevent the sharing of data (common in health care)~\cite{de2019continual,parisi2019continual}.

Incremental learning aims to mitigate catastrophic forgetting~\cite{mccloskey1989catastrophic} which occurs when neural networks update their parameters for new tasks. Most work has focused on image classification~\cite{de2019continual,parisi2019continual,kemker2018measuring}, while less attention has been dedicated to other applications, such as object detection~\cite{shmelkov2017incremental} and semantic segmentation~\cite{tasar2019incremental,michieli2019incremental,cermelli2020modeling}. Recently MiB~\cite{cermelli2020modeling} achieved state-of-the-art results on incremental semantic segmentation. Its main novelty is to model the background distribution shift during each incremental training session by reformulating the conventional distillation loss. However, the forgetting of learned knowledge is still severe due to the lack of previous labeled data.

The main challenge of incremental semantic segmentation  is the inaccessibility of previous data. We here consider the scenario where no data of previous tasks can be stored, as is common in incremental semantic segmentation~\cite{michieli2019incremental,cermelli2020modeling}. Storing of data could be prohibited due to privacy concerns or government regulations, and is one of the settings studied in class-incremental learning~\cite{parisi2019continual}. To address this problem, we propose to use self-training to exploit unlabeled auxiliary data. Self-training~\cite{nigam2000analyzing,zou2018unsupervised,zou2019confidence} has been successfully applied to semi-supervised learning and domain adaptation. It aims to first predict pseudo labels of the unlabeled data and then learn from them iteratively. To the best of our knowledge, self-training has not yet been explored for incremental learning. One of the obstacles to self-training for class-incremental semantic segmentation is the fusion of the pseudo labels from previous and current models. We show some challenges in Figure~\ref{fig:v-conflict}. Note that our ‘new model’ is trained to be optimal for the labels of the current task, and while learning these is forgetting previous classes. To successfully join the knowledge of the old and new models, we aim to apply self-training on auxiliary data. However, as seen in Figure~\ref{fig:v-conflict} joining the label information from both models is not straightforward. 

Therefore, we propose self-training for incremental semantic segmentation. The idea is to introduce self-training of unlabeled data to aid incremental semantic segmentation task. Specifically, we first train a new model with new data, then pseudo labels are predicted and fused on auxiliary unlabeled data by both old and new models. We retrain the model on auxiliary data to mitigate catastrophic forgetting. Simply fusing the pseudo labels from two models causes problems due to conflicting predictions. Thus, we propose a conflict reduction mechanism. Additionally, predicted pseudo label from neural networks are often over-confident~\cite{pereyra2017regularizing, zou2019confidence}, which might mislead the training process.  We therefore propose to maximize self-entropy to smooth the predicted distribution and reduce the confidence of predictions.  Our method outperforms state-of-the-art methods by a significant relative gain up to 114\% on Pascal-VOC 2012  and 8.5\% on the challenging ADE20K.  

Our main contributions are:

\begin{itemize}
    \item We are the first to apply self-training for class-incremental semantic segmentation to mitigate forgetting by rehearsal of previous knowledge using auxiliary unlabeled data.  
    \item We propose a conflict reduction mechanism to tackle the conflict problem when fusing the pseudo labels from old and new models for auxiliary data. 
    \item We show that maximizing the self-entropy loss can smooth the overconfident predictions and further improve performance. 
    \item We demonstrate state-of-the-art results, obtaining up to 114\%  relative gain on Pascal-VOC 2012 dataset and 8.5\% on the challenging ADE20K dataset compared to MiB method.
\end{itemize}

\section{Related Work}

\subsubsection{Semantic Segmentation.}
Image segmentation has achieved significant improvements with advance of deep neural networks~\cite{minaee2020image}. Fully Convolutional Networks (FCNs)~\cite{long2015fully} is one of the first works for semantic segmentation to use only convolutional layers, and can take any arbitrary size of input images and output segmentation maps. Encoder-Decoder based architectures are popular for semantic segmentation. The deconvolutional (transposed convolutional) layer~\cite{noh2015learning} is proposed to generate accurate segmentation maps. SegNet~\cite{badrinarayanan2017segnet} proposes to use indices of encoder max-pooling to upsample the corresponding decoder. Additionally, multi-resolution information~\cite{zhao2017pyramid,he2019adaptive}, attention mechanism~\cite{chen2016attention,fu2019dual} and dilated convolution (atrous convolution)~\cite{chen2017deeplab,chen2017rethinking,chen2018encoder}  are further developed to improve performance. Mask R-CNN~\cite{he2017mask} obtains promising performance on instance segmentation for each object.

\subsubsection{Incremental Learning.}
The problem of catastrophic forgetting~\cite{mccloskey1989catastrophic} has been studied extensively in recent years when neural networks are required to adapt to new tasks. Most work has been focused on image classification. It can be ruffly divided into three categories according to~\cite{de2019continual,parisi2019continual,kemker2018measuring}: Regularization-based~\cite{li2017learning,kirkpatrick2017overcoming,zenke2017continual,jung2018less, aljundi2018memory}, rehearsal-based~\cite{rebuffi2017icarl,chaudhry2018riemannian} and architecture-based methods~\cite{rusu2016progressive,mallya2018piggyback,mallya2018packnet}. Regularization-based methods alleviate  forgetting of previously learned knowledge by introducing additional regularization term to constraint the output embeddings or parameters while training the current task. Knowledge distillation has been very popular for several methods~\cite{li2017learning,castro2018end, hou2019learning}.
Rehearsal based methods usually need to store exemplars (small amounts of data) from the previous tasks which are used to be replayed. Some approaches propose alternative ways of replay to avoid storing exemplars, including generative model to do rehearsal. Architecture based methods dynamically grows the network to increase capacity to learn new tasks, while the old part of the network can be protected from forgetting.

Due to privacy issues or memory limits, it is not always possible to access to data from previous tasks, which causes catastrophic forgetting of previous knowledge. In the paper, we consider this more difficult setting of \emph{exemplar-free class-IL} in which the storing of previous task data is prohibited. Unlabeled data is seen as an alternative to secure privacy and mitigate forgetting. There are some works applied to image classification. ~\cite{zhang2020class} propose to train a separate model with only new data and then use auxiliary data to train a student model using distillation loss with both new and old models. ~\cite{lee2019overcoming}  propose confidence-based sampling to build an external dataset.  

Recently, the attention of continual learning has also moved to other applications, such as object detection~\cite{shmelkov2017incremental}, and semantic segmentation~\cite{michieli2019incremental,cermelli2020modeling}. Shmelkov et al.~\cite{shmelkov2017incremental} propose to use distillation loss on both bounding box regression and classification outputs for object detection. A incremental few-shot detection (iFSD) setting is proposed in~\cite{perez2020incremental}, where new classes must be
learned incrementally (without revisiting base classes)
and with few examples.  Michieli et al. ~\cite{michieli2019incremental} propose to use distillation both on the output logits and intermediate features for incremental semantic segmentation. Recently,  MiB (Cermelli et al.~\cite{cermelli2020modeling}) achieves the state-of-the-art performance by considering previous classes  as background for the current task and current classes as background for distillation. It is also investigated in remote sensing~\cite{tasar2019incremental} and medical data~\cite{ozdemir2018learn} for incremental semantic segmentation.

\begin{figure*}[tb]
\begin{center}
\includegraphics[width=1\textwidth]{./figure/framework_main.pdf}
\caption{Overview of our method. It includes two models, one learned from previous sessions $\mathcal{T}^{t-1}$ (upper left) and one from the current session $\mathcal{T}^t$ (bottom left). Pseudo labels are generated and fused by leveraging unlabeled data (right). The conflict reduction mechanism is proposed to generate more accurate pseudo labels for further training. }
\label{fig:framework}
\end{center}
\end{figure*}

\subsubsection{Self-Training.}
Self-training~\cite{nigam2000analyzing,grandvalet2005semi,lee2013pseudo} is one simple but powerful alternate strategy, which aims to leverage unlabeled data with a teacher model trained on existing labeled data as “pseudo-labeled” training samples. Self-training iteratively generates one-hot pseudo-labels corresponding to prediction confidence by a teacher model, and then retrains network based on these pseudo-labels with unlabeled data. Recently,It has achieved significant success on semi-supervised learning~\cite{grandvalet2005semi,triguero2015self} and  domain adaptation~\cite{zou2018unsupervised, chen2011co}. However, the predicted pseudo labels tend to be over-confident, which might mislead the training process and hurt the learning behaviour~\cite{bagherinezhad2018label}. Different methods to solve noisy label learning such as label smoothing~\cite{pereyra2017regularizing} and confidence regularization~\cite{zou2018unsupervised,zou2019confidence} are proposed to mitigate this phenomena. In this work, we explore self-training for learning semantic segmentation sequentially with confidence regularization.

\section{Proposed Method}



\subsection{Class-Incremental Semantic Segmentation} 
Semantic segmentation aims to assign each pixel $(x_i,x_j)(0 < i\leq  h, 0  <j  \leq  w)$ of image $x$ a label 
$y_{i,j} \in \mathcal{Y}, \mathcal{Y}\in \{0, 1, ..., N-1\}$, representing the semantic class. Here,
$h$ and $w$ are the height and width of the input image $x$, 
$N$ is the number of classes, and we define class $0$ to be the background.
The setting for class incremental learning (CIL) for semantic segmentation was first defined by 
\cite{michieli2019incremental,cermelli2020modeling}.  Training is conducted for CIL along $T$ different training sessions. During each training session, we only have training data of newly available classes, while the training data of the previously learned classes are no longer accessible. Each session introduces \emph{novel categories} to be learned. 

Specifically, the $t_{th}$ training session contains data $\mathcal{T}^t= (X^t, Y^t)$, where $X^t$ contains the input images for the current task and $Y^t$ is the corresponding ground truth. The current label set $\mathcal{Y}^{t}$ is the combination of  the previous label set $\mathcal{Y}^{t-1}$ and a set of new classes $C^{t}$, such that $\mathcal{Y}^{t}=\mathcal{Y}^{t-1} \cup C^t$. Only pixels of new classes are annotated in $X^t$ and the remaining pixels are assigned as background. The loss function for the current training session is defined as follows: 
\begin{equation}
   \label{eq:obj-general}
    \mathcal{L}_{ce}(\theta^t)= \frac{1}{|\mathcal{T}^t|
    }\sum_{(x,y)\in\mathcal{T}^t}
    \ell_{ce}({\theta^t};x,y) 
\end{equation}
where $\ell_{ce}$ is the standard cross-entropy loss used for supervised semantic segmentation and $|\mathcal{T}^t|$ is total number of samples in current training session $\mathcal{T}^t$.

\subsection{Self-Training for Incremental Learning}

A naive approach to address the class-incremental learning (CIL) problem is to train a model $f_{\theta^t}$ on each set $X^{t}$ sequentially by simply fine-tuning $f_{\theta^t}$ from the previous model $f_{\theta^{t-1}}$. This approach would suffer from catastrophic forgetting, as the parameters of the model are biased to the current categories because no samples from previous data $X^{t-1}$ are replayed. As discussed in the related work, various approaches to prevent forgetting could be considered, like regularization~\cite{li2017learning,kirkpatrick2017overcoming,zenke2017continual,jung2018less, aljundi2018memory}, or rehearsal methods~\cite{rebuffi2017icarl,chaudhry2018riemannian}. Instead, we propose to use self-training. To the best of our knowledge, we are the first to investigate self-training for incremental learning. Our goal is to use unlabeled data to generate \emph{pseudo labels} from the previous model and current model. In this way, the models are able to 'revisit' the previous knowledge and avoid catastrophic forgetting of previously learned categories. 

We present an overview of our framework on incremental semantic segmentation with a self-training mechanism shown in Figure~\ref{fig:framework}. We first initialize the current model $f_{\theta^t}$ from  $f_{\theta^{t-1}}$ (trained on the input data set $X^{t-1}$ in the training session $\mathcal{T}^{t-1}$) and then update parameters using $X^{t}$ in training session $\mathcal{T}^t$, the same procedure as used for the Fine-tuning method for CIL. This new model is trained to be optimal for the new classes $C^t$, however it forgets the previous classes $Y^{t-1}$. To overcome this problem, we combine the knowledge from both the old and new models in order to predict all categories we have seen.
We generate pseudo label $\hat{P}^{t-1}$ and $\hat{P}^{t}$ by feeding the unlabeled data $A$ to the previous model $f_{\theta^{t-1}}$ and the current model $f_{\theta^t}$, respectively. The generated pseudo labels of the auxiliary data from both models have the potential to represent all categories the models have encountered. 

\begin{figure*}[tb]
\begin{center}
\includegraphics[width=0.95\textwidth]{./figure/train-2.pdf}
\caption{Illustration of fusing pseudo labels by conflict reduction mechanism. The category 'train' is not learned in previous training sessions, therefore it is most likely to be predicted as the similar category 'bus' but not well segmented (top image). After learning 'train' on current task, it is predicted as 'train' correctly (bottom image). To generate more accurate pseudo labels, conflict reduction is proposed to fuse the two predictions (right image).}
\label{fig:conflict}
\end{center}
\end{figure*}

 We require a strategy to fuse the pseudo-labels of both models for each image in the auxiliary dataset into a single pseudo-labeled image. We first propose a fusion based on the idea that the predictions from the previous model $\hat{P}^{t-1}$ should be trusted and we only change those background pixels that are considered foreground in the current model $\hat{P}^t$. In the next section, we improve this fusion of pseudo labels by considering a conflict reduction mechanism. 

When $\hat{P}^{t-1}$ and $\hat{P}^{t}$ both consider the pixel as background, we assign background label 0 to the corresponding pixel in the fused pseudo label $\hat{P}^{t}_A$ directly; when $\hat{P}^{t-1}$ consider the pixel as background (label is 0), while $\hat{P}^{t}$ classify it as foreground (label is larger than 0), $\hat{P}^{t}_{A}$ is equal to $\hat{P}^t$ since the pixel is likely  to be the category learned in the current model $f_{\theta^t}$; If $f_{\theta^{t-1}}$ considers the pixel as foreground, $\hat{P}^{t}_{A}$ is equal to $\hat{P}^{t-1}$, we assume the previous model has higher priority than the current model since the previous model is accumulating more and more knowledge. The final fusion of pseudo labels for these auxiliary data $\hat{P}^{t}_{A}$ can be presented as follows:

\begin{equation}
\hat{P}^{t}_{A}=
\begin{cases}
0 & \text{ if }  \hat{P}^t =0  \text{ and }  \hat{P}^{t-1}=0 \\
\hat{P}^t & \text{ if } \hat{P}^t>0  \text{ and }  \hat{P}^{t-1}=0 \\
\hat{P}^{t-1}& \text{ if } \hat{P}^{t-1} >0
\end{cases}
\label{eq:pseudo1}
\end{equation}


Finally, we update the joint model (initialized from $f_{\theta^{t-1}}$) using the auxiliary data $A$ and pseudo label $\hat{P}^{t}_{A}$. We repeat the above procedure until all tasks are learned. As we mention above, the auxiliary unlabeled dataset is expected to be related to the labeled datatset, where they share some similar or overlapped categories. For many practical applications, the assumption of an available auxiliary dataset is realistic. For instance, in the application of autonomous driving, there is abundant amount of unlabeled data available for training. In our experiments, we will show results where we use the COCO dataset as the auxiliary dataset for Pascal-VOC 2012, and the Places365 for ADE20K in Section~\ref{sec:exp} . We will also investigate how unrelated data affects the self-training process.


\subsection{Conflict Reduction}


In the above section, we introduce how self-training is adapted in the framework for incremental semantic segmentation to help mitigate catastrophic forgetting of old categories. The pseudo labels of the auxiliary data are generated from the previous and current model, respectively. Then the two pseudo labels are fused into the final pseudo label directly, however, there might be wrong fusions due to similar categories that are often mis-classified. Therefore, we propose conflict reduction to further improve the accuracy of pseudo label fusion. 

Assume 'bus' is the category added in the training session $\mathcal{T}^{t-s}$ $(s\geq 1)$ , 'train' is learned in the current training session $\mathcal{T}^t$ (as seen in Figure~\ref{fig:conflict}). When an image from the auxiliary data containing 'train' is fed into the model $f_{\theta^{t-1}}$, as 'train' has never been learned in the previous training sessions, the model assigns the maximum probability to the most similar category label 'bus' due to the usage of cross entropy loss. Following Eq.\ref{eq:pseudo1}, the fused pseudo labels are automatically assigned from $\hat{P}^{t-1}$ if the previous model regards the pixels as foreground with no need to check the pseudo label $\hat{P}^t$ obtained from the current model. This results in the mis-classification and a drop in performance of the overall semantic segmentation system. In fact, in this case the current model $f_{\theta^t}$ labels the train as 'train' very confidently, since it just learned to recognize trains from data $X^t$. Conflict frequently occurs when similar categories appear such as 'sheep' and 'cow', 'sofa' and 'chair', 'bus' and 'train' et al.. Therefore, a Conflict Reduction module is proposed to obtain better fusion when both the previous and current models consider the pixel as foreground ( $\hat{P}^t>0$ and $\hat{P}^{t-1}>0$). Therefore, we update the Eq.\ref{eq:pseudo1} as follows:

\begin{equation}
\label{eq:conflict}
\hat{P}^{t}_{A}=
\begin{cases}
0 & \text{ if }  \hat{P}^t =0  \text{ and }  \hat{P}^{t-1}=0 \\
\hat{P}^t & \text{ if } \hat{P}^t >0  \text{ and }  \hat{P}^{t-1}=0 \\
\hat{P}^{t-1} & \text{ if } \hat{P}^t=0  \text{ and }  \hat{P}^{t-1}>0 \\

\hat{P}^t & \text{ if } \hat{P}^t, \hat{P}^{t-1} >0  \text{ and }  \text{max} (\hat{q}^{t}) > \text{max} (\hat{q}^{t-1} ) \\

\hat{P}^{t-1} & \text{ if } \hat{P}^t, \hat{P}^{t-1} >0  \text{ and }  \text{max} (\hat{q}^{t}) <\text{max} (\hat{q}^{t-1}) \\

\end{cases}
\end{equation}
where $\hat{q}^{t-1}$ and $\hat{q}^{t}$ are output probabilities from previous and current models, respectively.

\subsection{Maximizing Self-Entropy}
Pseudo-labeling is a simple yet effective technique commonly used in self-training learning. One major concern of training with pseudo label is the lack of guarantee for label correctness~\cite{zou2019confidence}. To address this problem, maximizing the self-entropy loss is explored in this work to relax the pseudo labels and redistribute a certain amount of confidence to other classes. We soften the pseudo label by maximizing the self-entropy loss $\mathcal{L}_{se}(\theta^t)$ according to: 

\begin{equation}
\label{eq:total}
    \mathcal{L}(\theta^t) = \mathcal{L}_{ce}(\theta^t) - \lambda * \mathcal{L}_{se}(\theta^t)
\end{equation}
where,
\begin{equation}
    \mathcal{L}_{se}(\theta^t)= - \frac{1}{|\mathcal{T}^t| 
    }\sum_{}
    \hat{q}\log \hat{q}
\end{equation}

\subsection{Algorithm}
We provide a detailed algorithm of our incremental semantic segmentation procedure in Algorithm~\ref{table:algorithm}. For the first task (line 5), it is similar to standard incremental learning methods. For the remaining tasks, our method can be divided into two main parts: pseudo label fusion (lines 7-9) and model retraining (line 10).

\tablealgorithm 

\section{Experiments}
\label{sec:exp}

\tablepascal 

\subsection{Experimental setups}



In this section, we provide details for the datasets, evaluation metrics, implementations and compared methods. Code will be made available upon acceptance of this manuscript.

\subsubsection{Datasets} We evaluate all methods using Pascal-VOC 2012 and ADE20K. Pascal-VOC 2012~\cite{everingham2011pascal} has 10,582 images for training, 1,449 images for validation and 1,456 images for testing. Images contain 20 foreground object classes and one background class. ADE20K~\cite{zhou2017scene} is a large scale dataset containing 20,210 images in the training set, 2,000 images in the validation set, and 3,000 images in the testing set. It contains 150 classes of both stuff and objects. For auxiliary datasets, we choose COCO 2017 train set~\cite{lin2014microsoft} with 80 classes and 118K images for Pascal-VOC 2012 and Places365~\cite{zhou2017places} with 365 classes and 1.8M images for ADE20K.

\begin{figure*}[tb]
\begin{center}
\includegraphics[width=0.32\textwidth]{./figure/19-1_s_new.pdf}
\includegraphics[width=0.32\textwidth]{./figure/15-5_s_new.pdf}
\includegraphics[width=0.32\textwidth]{./figure/15-1_s_new.pdf}
\caption{Mean IoU on Pascal VOC 2012 for 19-1, 15-5 and 15-1  scenarios as a function of proportional of unlabeled data. The curve starts from 1\% (about 1K images) and ends at 100\%. The horizontal axis is in logarithmic scale. }
\label{fig:datasize}
\end{center}
\end{figure*}

\subsubsection{Implementation Details} We follow the same implementation as proposed in~\cite{cermelli2020modeling}. For all methods, we use the same framework in Pytorch. The Deeplab-v3 architecture~\cite{chen2017rethinking} with ResNet-101~\cite{he2016deep} backbone is used for all methods. In-place activated batch normalization is used and the backbone network is initilized with an ImageNet pretrained model~\cite{rota2018place}. We train the network with SGD and an initial learning rate of $10^{-2}$ for the first task and $10^{-3}$ for the rest of the tasks as in~\cite{cermelli2020modeling}. We train the current model on Pascal-VOC 2012 for 30 epochs and on ADE20K for 60 epochs. The self-training procedure is trained on unlabeled data for one pass, the trade-off $\lambda$ between cross entropy and self-entropy is set to 1.  20\% of training data are used as validation set and the final results are on the standard test set. 

\subsubsection{Compared Methods} We consider the Fine-tuning (FT) baseline and the Joint Training (Joint) upper bound. Additionally, we compare with several regularization-based methods adapted to incremental semantic segmentation including ILT~\cite{michieli2019incremental}, LwF~\cite{li2017learning},  LwF-MC~\cite{rebuffi2017icarl},
RW~\cite{chaudhry2018riemannian},
EWC~\cite{kirkpatrick2017overcoming} and
PI~\cite{zenke2017continual}. We also compare with state-of-the-art method MiB~\cite{cermelli2020modeling}. Additionally, MiB can be further trained with unlabeled data by generating pseudo labels with learned current model, we denote it as MiB + Aux by leveraging unlabeled data using pseudo labels.  All results are reported in mean Intersection-over-Union (mIoU) in percentage. It is averaged for all classes of each task after learning all tasks. Note that none of the methods has access to exemplars from previous tasks.

\subsection{On Pascal-VOC 2012}
We compare different methods in three different scenarios as in~\cite{cermelli2020modeling}. 19-1 means we first learn a model with first 19 classes and then learn the remaining class as the second task. For 15-5 scenario, there are 15 classes for the first task and the remaining 5 classes as the second task. For 15-1, the first task is the same as in 15-5 scenario, but the remaining 5 classes are learned one-by-one resulting in a total of six tasks. For all scenarios, we consider two different settings. \textit{Disjoint} setting assumes images of current task only contain the current or previous classes. While \textit{Overlapped} setting assumes that at each training session, it contains all images with at least one pixel of novel classes. Previous classes are labeled as background for both settings.


\subsubsection{Addition of one class (19-1)} As shown in Table ~\ref{tab:pascal}, in this scenario, FT and PI obtain the worst results, where they forget almost all of the first task, and perform poorly on the new task with 6.2\% and 5.9\% of mIoU in the \textit{Disjoint} setting, respectively. EWC and RW, both weights-based regularization methods, improve quite a lot compared to PI. Interestingly, activation-based regularization methods LwF, LwF-MC and ILT perform significant better in both \textit{Disjoint} and \textit{Overlapped} settings. MiB is superior to all previous methods but inferior to our method by a large margin. Our method surpasses MiB on overall mIoU by 8.0\% for the \textit{Disjoint} setting and 6.7\% for the \textit{Overlapped} setting. When MiB is further trained with unlabeled data (MiB + Aux) using generated pseudo labels, it improves by 2.2\% for the \textit{Disjoint} setting and 11.3\% for the \textit{Overlapped} setting on class 20, while keeping similar performance for the first 19 classes. Note that our method is very close to Joint training performance on both settings (75.4\% to 77.4\% and 74.5\% to 77.4\%).

\subsubsection{Single-step addition of five classes (15-5)} Similar conclusions can be drawn in this scenario. Our method outperforms MiB by 6.6\% in the \textit{Disjoint} setting and 2.1\% in the \textit{Overlapped} setting.
The performance gain due to additional pseudo labels for MiB+Aux is more obvious in this scenario (when compared to the 19-1 setting).
However, there is  still a big gap compared to our method. This shows that our proposed techniques are more efficient to leverage knowledge from unlabeled data.  Our method achieves similar overall results (71.3\% and 71.1\%) in both settings, showing the robustness of our proposed method. 

We report some qualitative results for different incremental methods (Ours, MiB and Fine-tuning) on Pascal-VOC 2012 with 15-5 scenario in Fig.~\ref{fig:voc}. The results demonstrate the superiority of our approach. FT totally forgets previously learned classes (first row and third row) but correctly predicts new classes (second row),  while our approach obtains sharper (e.g. person, bike), more coherent (e.g. potted plant) and finer-border (e.g. cow) predictions compared to the state-of-the-art method MiB. 

\subsubsection{Multi-step addition of five classes (15-1)} This is the most challenging scenario of the three. There are five more tasks after learning the first task, therefore, it is more difficult to prevent forgetting. From Table~\ref{tab:pascal}, we can observe that in general all methods forget more in this scenario. MiB only achieves 46.2\% and 35.1\% for the first task in two settings after learning all tasks. While our method achieves 70.1\% and 71.4\% of mIoU. Meanwhile our method outperforms MiB on new tasks by a large margin (21.4\% and 26.5\% respectively) as well. Overall, we gain 23.3\% for the \textit{Disjoint} setting and 33.9\% for the \textit{Overlapped} setting. We also see a significant improvement for MiB + Aux compared to the original MiB, from 37.9\% to 39.9\% for \textit{Disjoint} and from 29.7\% to 37.3\% for the \textit{Overlapped} setting. Again, without any more expensive annotation processes, unlabeled data is beneficial not only for our proposed method, but also very useful for the state-of-the-art method MiB to further boost performance. 



\figvoc


\subsection{Ablation study}

In this section, we perform an ablation study on several different aspects including the proposed techniques, the relationship between performance and the amount of unlabeled data used, epochs for self-training and hyper-parameter $\lambda$.

\subsubsection{Impact of different proposed components} In Table~\ref{tab:ablation} we conduct an ablation study on the Pascal-VOC 2012 \textit{Disjoint} setting. In the first row, we show the performance of FT on three scenarios. And in the second row, MiB is the current state-of-the-art method. When we use self-training (ST) with unlabeled data, the performance is largely improved in all three scenarios, which shows the effectiveness of self-training on unlabeled data in incremental semantic segmentation. Conflict reduction (+CR) further improves the results on all of these three scenarios by 0.6\%, 4.2\% and 3.2\%, respectively. Maximizing self-entropy (+MS) strategy further obtains 0.4\% better on the 19-1 scenario and  0.8\% better results on the 15-5 scenario. On more challenging 15-1 scenario, MS boosts performance significantly by 2.4\%.
Note that the gain for the 19-1 scenario is small compared to the other two scenarios because there is only one category ('TV') for the second task, it does not contribute much to the overall performance even without learning it. Therefore, for incremental learning, the other two scenarios are more relevant.  


\tableablation

\begin{table}[tb]
\begin{center}
\caption{Mean IoU on the Pascal VOC 2012 by using 1\% of unlabeled data (COCO) with different number of epochs. \textit{Disjoint} setting is used in this experiment.}
\resizebox{0.37\textwidth}{!}{%
\begin{tabular}{c||cccc}
Epochs & 1 & 5 & 10 & 20 \\ \hline
19-1 & 70.1 & 72.8 & 73.2 & 73.1 \\
15-5 & 46.4 & 66.9 & 68.3 & 68.8 \\
15-1 & 50.5 & 54.1 & 56.9 & 58.1 
\label{table:epoch}
\end{tabular}}
\end{center}
\end{table}

\subsubsection{The amount of unlabeled data} We also evaluate the relationship between mean IoU and the size of unlabeled data by randomly selecting a portion of unlabeled data (see Figure~\ref{fig:datasize}). We experiment on \textit{Disjoint} setup for 19-1, 15-5 and 15-1 scenarios. Notably, for 19-1 scenario, our method beats MiB by using only 1\% unlabeled data, and the mIoU continually increases when more unlabeled data is used. For 15-5 scenario, our method achieves similar results as MiB by only using 2\% of unlabeled data. It keeps improving by increasing unlabeled data and peaks when 70\% of unlabeled data are used. Similar conclusion can be observed for the 15-1 scenario, our method outperforms MiB by a large margin with only 1\% unlabeled data. When adding more unlabeled data, the curve goes up consistently until it reaches the best performance and then it drops a bit in the end. 

\subsubsection{Number of self-training epochs} Without any specific mention, we only pass all unlabeled data once into the network for  efficiency consideration throughout the paper in our experiments. In this section, we consider different multiple passes when only 1\% unlabeled data is available from COCO dataset as shown in Table ~\ref{table:epoch}. As expected, compared to only passing it once, training for more epochs achieves significant gain for all three scenarios. Specifically, when we increase the self-training epochs from 1 to 20, it improves from 70.1\% to 73.1 for 19-1 scenario, from 46.4 to 68.8 for 15-5 scenario and from 50.5 to 58.1 for 15-1 scenario, respectively. It is the most effective to 15-5 scenario because the difficulty of this scenario is between the other two scenarios. It also shows that by training multiple runs using pseudo labels the performance can be further boosted even in the low-data regime, when we have little unlabeled data.


\subsubsection{Impact of trade-off $\lambda$} This parameter controls the strength between the cross-entropy and self-entropy loss.  
 In this section, we report results using various $\lambda$'s on the 15-5 scenario of the \textit{Disjoint} setup. As shown in Table~\ref{tab:lambda}, by changing $\lambda$ from 0.1 to 5, the overall performance goes up first and then goes down. When 0.5 and 1 are used, it obtains the best performance. Using $\lambda=1$ has similar performance as 0.5 and slightly better on the new task, without any specific mention,  $\lambda=1$ is used throughout the paper.

\begin{table}[tb]
\begin{center}
\caption{The trade-off $\lambda$ between cross entropy and self-entropy on 15-5 scenario. Mean IoU is reported for both tasks and also overall performance is in the last row.}
\resizebox{0.4\textwidth}{!}{%
\begin{tabular}{c||cccc}
Trade-off $\lambda$ & 0.1  & 0.5   & 1   & 5            \\ \hline
1-15            & 76.8 & 77.4   &  77.1   & 72.4    \\
16-20              &  53.1 &  55.4  & 55.6    &   49.7         \\
all                  & 
70.9
    &  71.9 & 71.7   &       66.7     \\           
\end{tabular}}
\label{tab:lambda}
\end{center}
\end{table}


\subsection{On different self-training datasets}

In this section, we compare several datasets used as self-training datasets for Pascal-VOC to show how sensitive our method is with respect to the unlabeled data.
We choose a general-purpose ImageNet validation set, two fine-grained CUB\_200\_2011 (Birds) and Flowers datasets together with the COCO dataset as different unlabeled data to show how our method performs on Pascal-VOC 15-5 scenario (see Table~\ref{tab:fid}). It is surprising that using ImageNet (validation set) provides similar performance as using COCO dataset (mIoU: 70.0 vs 71.3) for all classes. It suggests that datasets with diverse categories can be a good option. As expected, it fails on CUB (mIoU: 10.7) and Flowers (mIoU: 5.1) datasets, since fine-grained classes do not contain diverse objects. 

We did a preliminary experiment on measuring the relatedness of datasets mathematically with Fréchet Inception Distance (FID) score\cite{heusel2017gans}. FID is originally proposed to measure the similarity of two distributions, such as generated fake images and real images, which is popular in measuring the quality of generative models.  We use FID here to measure the closeness between labeled datasets and unlabeled datasets. We compute FID between Pascal-VOC and other unlabeled datasets, we obtain an FID score with COCO (25.8), ImageNet (43.8), CUB (153.5) and Flowers (222.3). Smaller FID scores indicate that the distribution of two datasets are closer, which is consistent with performance we obtained for self-training. Therefore such a measure could be used to select good auxiliary data. 

\begin{table}[tb]
\begin{center}
\caption{Different auxiliary datasets to generate pseudo labels. Mean IoU is reported for both tasks on 15-5 scenario.}
\resizebox{0.45\textwidth}{!}{%
\begin{tabular}{c||c|cc|c}
                           &               & \multicolumn{3}{c}{\textbf{15-5}} \\
Candidate Dataset & FID Score  & 1-15   & 16-20   & all             \\ \hline
COCO                       & \textbf{25.8} & \textbf{76.9}   & \textbf{54.3}    & \textbf{71.3}   \\
ImageNet (Val)              & 43.8          & 75.5   & 53.5    & 70.0            \\
CUB                        & 153.5         & 4.0    & 30.9    & 10.7            \\
Flowers                    & 222.3         & 2.6    & 12.8    & 5.1\\       
\end{tabular}}
\label{tab:fid}
\end{center}
\end{table}

\tableade 

\figade

\subsection{On ADE20K}

Following~\cite{cermelli2020modeling}, we report average mIoU of two different class orders on ADE20K dataset. In this experiment, we only compare with activation-based regularization methods LwF, LwF-MC, ILT (much better than PI, EWC and RW), the state-of-the-art method MiB and MiB with auxiliary data (MiB + Aux). Here we consider three scenarios: 100-50 means 100 classes for the first task and 50 classes for the second tasks. 100-10 considers 100 classes as the first task and the rest classes are divided into 5 tasks. Lastly, in 50-50 scenario, 150 classes are equally distributed in three tasks with 50 classes each.

\subsubsection{Single-step addition of 50 classes (100-50).} As shown in Table~\ref{tab:ade}, FT forgets the first task totally because of the large number of classes. As seen from Joint, the overall mIoU is 38.9\%, which is much less compared to Pascal-VOC 2012, indicating this is a more challenging dataset. MiB achieves relatively robust results in this scenario, while our method outperforms it by 0.5\% in average for 150 classes. Notably, our method prevents the forgetting of previous task effectively but obtains worse performance on the second task compared to MiB. Interestingly, we have seen that for Pascal-VOC dataset, MiB + Aux outperforms baseline MiB in most cases, while it fails to further improve the performance in this case. The reason could be that ADE-20K is more challenging and accuracy itself is much lower than on Pascal-VOC, which could introduce more noise during training.
It further shows the importance of our proposed framework to leverage unlabeled data, which leads to superior performance. 


We report some qualitative results for different incremental methods (Ours, MiB and Fine-tuning) on this senario. On this challenging scenario with more categories, our approach is capable of segmenting more objects correctly (e.g. the wall, apparel, floor, painting, box, computer) than MiB.  

\subsubsection{Multi-step of addition 50 classes (100-10).} This is a more challenging scenario with six tasks in total. Most methods fail in this scenario, only achieving about 1.0\% mIoU (FT, LwF and ILT). ILT performs better than LwF-MC in most scenarios on Pascal-VOC but worse results on ADE20K. Our method still outperforms MiB overall by 2.2\%. More specifically, the gain is significant for the first four tasks from 1.8\% (first task) to 11.3\% (third task). And the performance of the fifth task is comparable, while ours obtains worse result for the last task. Again, using auxiliary data for MiB (MiB + Aux) leads to worse results compared to the original MiB, which shows that without specific design, the unlabeled data can be harmful.

\subsubsection{Three steps of 50 classes (50-50).} This is a more balanced scenario where each training session has the same number of classes. Similar to previous scenarios, we improve overall mIoU of MiB from 27.0\% to 29.0\%. Specifically, the gain is 4.5\% for the first task and 3.9\% for the second task. One general observation for all scenarios is that there is still a large gap between incremental learning and Joint for the ADE dataset. This suggests that incremental segmentation learning has to be further developed and improved.  



\section{Conclusions}

In this work, we improve incremental semantic segmentation with self-training. Unlabeled data is leveraged to combine the previous and current models. Importantly, conflict reduction provides more accurate pseudo labels for self-training. We have achieved state-of-the-art performance on two large datasets Pascal-VOC 2012 and ADE20K. We show that our method can obtain superior results on Pascal-VOC 2012 using only 1\% unlabeled data. Qualitative results show significant  more accurate segmentation maps are generated compared to the other methods. As future work, it is interesting to choose unlabeled data selectively, efficiently and actively. Although we have obtained promising gain for incremental semantic segmentation, the gap to Joint Training for the challenging ADE20K dataset indicates that CIL is still an open research topic.   



\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\bibliographystyle{IEEEtran}

