\section{Acknowledgement}
This work was supported by Honda, DARPA LwLL and NSF Award No. 1535797.



\section{Conclusion}
\vspace{-3mm}
In this paper, we introduce a method for open-set semi-supervised learning (OSSL), where samples of novel categories are present in unlabeled data. To approach the task, we propose a novel framework, OpenMatch\xspace, which unifies one-vs-all classifiers and FixMatch. Our proposed objective, open-set soft consistency loss, is shown to be effective to detect outliers from unlabeled data, which allows FixMatch to work well in the OSSL setting. In addition, OpenMatch\xspace sometimes detects outliers unseen in unlabeled data better than a supervised model. We believe that our framework for OSSL will make label-efficient techniques more practical.

\textbf{Limitations.}
OpenMatch\xspace can have difficulty in detecting outliers similar to inliers, though any methods for outlier detection can have the same limitation. If the outliers have similar visual characteristics, separating them from inliers samples is difficult. A potential solution is to introduce self-supervised learning~\cite{he2020momentum,chen2020simple} for unlabeled data.

\textbf{Broader Impact.}
OpenMatch is an effective tool for semi-supervised learning with noisy unlabeled data containing outliers. It would benefit projects with small budgets or hard-to-label data. At the same time, it is possible for malicious actors to exploit the advantages of learning from limited data. Broadly speaking, any progress on semi-supervised learning can have these consequences.


\section{Experimental Details}

\subsection{Implementation}
We utilize \url{https://github.com/kekmodel/FixMatch-pytorch} to implement Labeled Only, OpenMatch\xspace and FixMatch. The implementation reproduces the result of FixMatch paper well. For MTC, we employ the author's official implementation with Pytorch.

\textbf{Hyper-parameters.} Unless otherwise noted, the same value is used for all experiments.
\begin{itemize}
    \item The batch-size $B = 64$. 
    \item The relative size of batch-size for unlabeled data $\mu = 2$. 
    \item Trade-off parameters $\lambda_{em}=0.1$,  $\lambda_{fm}=1$. 
    \item Trade-off parameter $\lambda_{oc}=0.5$ for CIFAR10 and ImageNet, $\lambda_{oc}=1.0$ for CIFAR100.
    \item Epochs to begin FixMatch training, $E_{fix} = 10$. 
    \item Iterations per epoch $I_{max}=1024$.
    \item Total number of epochs $E_{max}=512$.
    \item Optimizer: SGD with nesterov momentum = 0.9.
    \item Learning rate $\eta=0.03$.
\end{itemize}
Hyper-parameters shared with FixMatch are borrowed from the implementation except that $\mu$, $E_{max}$ are set smaller to reduce computation time. To make a fair comparison with FixMatch, we utilize hyper-parameters defined above to train FixMatch model.

\textbf{Validation.}
In CIFAR10 and CIFAR100, 50 samples per each known class are used. For ImageNet-30, 10\% of the training split is used for the purpose. 

\textbf{MTC.}
Since the author provides implementation optimized for CIFAR10, we first employ the same hyper-parameters. Although we attempted tune the hyper-parameter such as a trade-off weight for out-lier detection loss,  the results did not improve with the tuning. Therefore, we use the same hyper-parameters for all experiments. 

\subsection{Dataset}

\textbf{ImageNet-30.}
We utilize the training split of ImageNet-30 for training and test one for evaluation.
To be specific, 2,600 samples are employed for labeled training data and validation respectively, 33,800 samples are employed for unlabeled data. Testing was done for 3,000 test samples.

\textbf{Novelty Detection Datasets}
We follow CSI~\cite{tack2020csi} to set up the datasets. See their implementation ~\url{https://github.com/alinlab/CSI} for more details.



\section{Experiments}
\vspace{-3mm}
\textbf{Setup.}
We evaluate the efficacy of OpenMatch on several SSL image classification benchmarks. Specifically,
we perform experiments with varying amounts of labeled data and varying  numbers of known/unknown classes on CIFAR10/100~\cite{krizhevsky2009learning} and ImageNet~\cite{imagenet}. We attempt to cover various scenarios by using these three datasets.

Note that we use an identical set of hyper-parameters except for $\lambda_{oc}$, which is tuned on each dataset. 
$\lambda_{em}$ is set 0.1 in all experiments. 
$\lambda_{fm}$ is set to 0 before $E_{fix}$ epochs and then set to 1 for all experiments. $E_{fix}$ is set to 10 in all experiments. The hyper-parameters for FixMatch, e.g., data augmentation, confidence threshold, are fixed across all experiments for simplicity. 
The hyper-parameters are set by tuning on a validation set that contains a small number of labeled samples. Note that the validation set does not contain any outliers. A complete list of hyper-parameters is reported in the appendix. Each experiment is done with a single 12-GB GPU, such as an NVIDIA TitanX. 

\textbf{Baselines.}
For the OSSL baseline, we employ MTC~\cite{yu2020multi} using the author's implementation.
In addition, we show the result of a model trained only with labeled samples (Labeled Only). 
We exclude UASD~\cite{chen2020semi} and D3SL~\cite{guo2020safe} from our baselines since their reported results underperform a model we train only with labeled samples. 
For instance, in the experiments for CIFAR10 at 400 labels per each class, their reported error rates are worse than 20\% while the rate of 
the Labeled Only baseline is 20.0\%.
A comparison with FixMatch~\cite{sohn2020fixmatch} reveals the effect of OVA-classifiers trained with the soft consistency loss.  
The hyper-parameters of both OpenMatch\xspace and baselines are tuned by maximizing the accuracy on the validation set. Since the validation set does not have any outliers, we choose the accuracy as a criterion. 



\textbf{Evaluation.}
We assume the test set contains both known (inlier) and unknown (outlier) classes. 
For known classes, classification accuracy is used to evaluate  performance.
To evaluate the separation into inliers and outliers, we use AUROC following the standard evaluation protocol of novelty detection~\cite{hendrycks2016baseline}. To compute the anomaly score, the outlier detector's score is employed for OpenMatch\xspace, Labeled Only, and FixMatch, where we add our outlier detector to the latter two models. 
We report the results averaged over three runs and their standard deviations.
\vspace{-2mm}
\subsection{CIFAR10 and CIFAR100}
\vspace{-2mm}
We compare OpenMatch with several baselines on the standard benchmark datasets of SSL, CIFAR10, and CIFAR100. Following ~\cite{yu2020multi}, we use a randomly initialized Wide ResNet-28-2~\cite{zagoruyko2016wide} with 1.5M parameters in these experiments.
For CIFAR10, we split the classes into known and unknown classes by defining animal classes as known and others as unknown, which results in 6 known classes and 4 unknown classes. For CIFAR100, we first split the classes by their super-classes provided by the dataset so that known and unknown classes do not share super-classes. Then, we split the super-classes into known and unknown. To evaluate performance with different numbers of outliers, we run experiments on two settings: 80 known classes (20 unknown classes) and 55 known classes (45 unknown classes). 

Tables~\ref{tb:main_accuracy} and ~\ref{tb:main_roc} describe the error rate on inliers and AUROC values respectively. Note that the number of labeled samples \textit{per  class} is shown in each column. 
With respect to the error rate, OpenMatch\xspace achieves state-of-the-art performance in all cases, and the gain over the baselines is remarkable in many cases. In particular, we see about a 10\% improvement in CIFAR10 error at 50 labels. In addition, the improvements in AUROC are also clear. MTC~\cite{yu2020multi} does not improve the performance very much compared to the labeled-only model on CIFAR100 because their method of identifying outliers, \textit{i}.\textit{e}., Otsu's thresholding~\cite{liu2009otsu}, is not very robust to the number of outliers as discussed in their paper.
On the other hand, OpenMatch\xspace improves the error rate by more than 3\% and AUROC by more than 6\% in all cases.
Simply applying FixMatch~\cite{sohn2020fixmatch} also does not always improve the error compared to the labeled-only model. This is because FixMatch confuses inliers and outliers and assigns pseudo-labels to outliers, as the significant decrease in AUROC indicates.


\input{tables/main_accuracy}
\input{tables/main_roc}
\vspace{-2mm}
\subsection{ImageNet}
\vspace{-2mm}
We evaluate OpenMatch\xspace on ImageNet to see its behavior on more complex and challenging datasets.
Since training on the entire ImageNet is computationally expensive, we choose ImageNet-30~\cite{hendrycks2016baseline}, which is a subset of ImageNet containing 30 classes. The subset is useful for evaluation on unseen outliers as done in Sec.~\ref{sec:noveldet}, because the classes do not include several super-classes such as a bird class. 
We pick the first 20 classes, in alphabetical order, as known classes, and the remaining 10 classes as unknown. We utilize the ResNet-18~\cite{he2016deep} network architecture. Hyper-parameters such as a batch-size, learning rate, data augmentation are the same as ones used in CIFAR10 and CIFAR100. 
As shown in the rightmost columns of Table~\ref{tb:main_accuracy} and ~\ref{tb:main_roc}, OpenMatch\xspace achieves the best performance in both accuracy and AUROC. Unlike the results on CIFAR, FixMatch improves the performance over the Labeled Only model. This is probably because outliers are easier to detect, as the AUROC score of the Labeled Only model already shows a higher score than other scenarios. Therefore, even without any outlier detection objectives, FixMatch can correctly select pseudo-inliers with confidence thresholding. 


\subsection{Ablation Study on Soft Open-set Consistency Regularization}\label{sec:ablation}
In this section, we validate our 
claim that the SOCR\xspace regularization is effective at separating inliers and outliers. 
We perform several ablation studies to see the benefit of SOCR\xspace and measure the model's ability to detect outliers using AUROC. In summary, both numerical and qualitative evaluations confirm the effectiveness of SOCR\xspace. 

\input{tables/ablation}
Table~\ref{tb:ablation} describes the numerical comparison of ablated models.
In this evaluation, we do not apply FixMatch to pseudo-inliers to measure the pure gain from the consistency loss. 
Introducing SOCR\xspace increases AUROC more than 20\% in CIFAR10, 8\% in CIFAR100, and 8\% in ImageNet. These results support the effectiveness of smoothing the outlier detector's output. 


\textbf{Histograms of Anomaly Scores.} 
Fig.~\ref{fig:histogram} illustrates the anomaly score of inlier and outlier test samples in the ablation experiment on CIFAR10. 
If SOCR\xspace is not employed (Fig.~\ref{fig:histogram}(a)), inliers and outliers are confused while  SOCR\xspace makes the scores of outliers larger and enhances the separation (Fig.~\ref{fig:histogram}(b)).
Further applying FixMatch to pseudo-inliers decreases the score of inliers (Fig.~\ref{fig:histogram}(c)), which separates inliers and outliers better. Considering the results of Table~\ref{tb:main_roc}, just applying FixMatch to all unlabeled samples is often not useful to detect outliers, but applying it to the pseudo-inliers can improve the outlier detection. 



\input{figures/histogram}


\subsection{Analysis}\label{sec:analysis}
\input{figures/tsne}


\textbf{Feature Visualization.} 
Fig.~\ref{fig:tsne} visualizes the feature distributions with T-SNE~\cite{maaten2008visualizing} in the experiment on CIFAR10 at 100 labels. OpenMatch\xspace (c) separates inliers and outliers well while a model trained only with 100 labeled samples (a) confuses them. The separation of OpenMatch\xspace appears to be  better than a model trained with all labeled inliers (b).

\textbf{Closed-set Consistency vs Open-set Consistency.} The soft consistency loss is computed for the outlier detector in OpenMatch\xspace. An alternative is to compute it for the closed-set classifier as done in UASD~\cite{chen2020semi}. We replace SOCR with the soft consistency loss with respect to the closed-set classifier and perform the same analysis as in Sec.~\ref{sec:ablation}. In CIFAR10 at 50 labeled samples, this model achieves $70.6\pm1.3$ \% AUROC, which is 10.1 \% better than a model without consistency loss and 10.7 \% worse than a model with SOCR. In CIFAR10 at 400 labeled samples, this model achieves $88.5\pm2.4$ \% AUROC, which is 12.7 \% better than a model without consistency loss and 8.3 \% worse than a model with SOCR. 
These results indicate that even using soft consistency for the closed-set classifier helps to detect outliers, as shown in UASD~\cite{chen2020semi}, but SOCR helps much more. Thus, our choice of combining both the outlier detector and soft consistency loss is validated in this analysis.

\vspace{-2mm}
\subsection{Novelty Detection}\label{sec:noveldet}
\vspace{-2mm}
We have seen that OpenMatch\xspace can learn to detect outliers from unlabeled data. In this experiment, we evaluate how well our OSSL model can separate inliers from outliers unseen in the unlabeled data. This setting is similar to general outlier or out-of-distribution sample detection. Note that the models evaluated in this section are the same as the ones in Table~\ref{tb:main_accuracy} and \ref{tb:main_roc}.
We consider the following datasets as out-of-distribution: SVHN~\cite{netzer2011reading}, resized LSUN ~\cite{yu2015lsun}, ImageNet, and CIFAR100 for CIFAR10 experiments (Table~\ref{tb:novel_det}(a)),
and LSUN, CUB-200~\cite{wah2011caltech}, Dogs~\cite{khosla2011novel}, Caltech~\cite{griffin2007caltech}, Flowers~\cite{nilsback2006visual}, and DTD~\cite{cimpoi2014describing} for ImageNet-30  (Table~\ref{tb:novel_det}(b)). See supplemental material for the details.
We utilize AUROC to measure how well inliers and outliers are separated. To see the gap from a supervised model, we train a model using all labeled samples of inlier classes and show the comparison. 

As seen in Table~\ref{tb:novel_det}, OpenMatch\xspace outperforms OSSL baselines by 5.8\% on average and even the supervised model by 3.4\% on average on CIFAR10. On ImageNet-30, OpenMatch\xspace outperforms OSSL baselines by more than 4.1\% on average.
This result verfies that OpenMatch\xspace is robust to various out-of-distribution data by virtue of being exposed to unlabeled data containing outliers. 
\input{tables/novelty_det}







\section{Introduction}
\vspace{-3mm}
Semi-supervised learning (SSL) leverages unlabeled data to improve a model’s performance~\cite{miyato2015distributional,verma2019interpolation, berthelot2019mixmatch, berthelot2019remixmatch,sohn2020fixmatch,laine2016temporal,tarvainen2017mean}. An SSL model can propagate the class information of  a small set of labeled data to a large set of unlabeled data, which significantly improves the recognition accuracy without any additional annotation cost. 
A common assumption of SSL is that the label spaces of labeled and unlabeled data are identical, but, in practice, the assumption is easily violated. Depending on how it was collected, the unlabeled data may contain novel categories unseen in the labeled training data, \textit{i}.\textit{e}., outliers.  
Since these outliers can significantly harm the performance of SSL algorithms~\cite{guo2020safe}, detecting them is necessary to make SSL more practical. 
Ideally, a model should classify samples of known categories  \textit{i}.\textit{e}., inliers, into correct classes while identifying samples of novel categories as {\it outliers}. 
This task is called {\it Open-set Semi-supervised Learning} (OSSL)~\cite{yu2020multi}.
While OSSL is a more realistic and practical scenario than standard SSL, it has not been as widely explored. 

\input{figures/consistency}

Existing strong SSL methods~\cite{sohn2020fixmatch, xie2019unsupervised} do not work well for OSSL. For example, FixMatch~\cite{sohn2020fixmatch} generates pseudo-labels using the model’s predictions on weakly augmented unlabeled images and trains the model to match its predictions on strongly augmented images with the pseudo-labels. This method exploits the advantage of pseudo-labeling as well as regularizing a model with the consistency between differently augmented images.  
But, in OSSL, this risks assigning  pseudo-labels of known categories to outliers, which degrades the recognition accuracy. 
A possible solution is to compute the SSL objective only for unlabeled samples considered to be inliers, where confidence thresholding is used to pick inliers. 
For instance, MTC~\cite{yu2020multi} regards some proportion of samples as outliers by using Otsu thresholding~\cite{liu2009otsu}. However, this is not robust to varying proportions of outliers as discussed in ~\cite{yu2020multi}.
DS3L~\cite{guo2020safe} proposes meta-optimization that attempts to pick unlabeled data useful to improve generalization performance. But, this method does not have an objective to separate inliers from outliers.


Given the limitations of the existing methods, we aim to learn representations that separate outliers from inliers in a feature space and a threshold effective for detecting outliers. The challenge is learning such representations with a small amount of labeled inliers and no supervision to find outliers. Moreover, choosing an accurate threshold value is not a trivial problem. 

We propose a new framework, {\it OpenMatch\xspace}, to address the above drawbacks of OSSL. 
First, we propose to utilize a One-Vs-All (OVA) network~\cite{saito2021ovanet} that can learn a threshold to distinguish outliers from inliers. A separate OVA-classifier is trained for each class, and a sample is labeled an outlier if all of the classifiers determine it to be one. Thus, this technique allows us to identify outliers in an unsupervised way. We call this an {\it outlier detector}.
Second, we propose a novel {\it open-set soft-consistency loss} to learn more effective representations for detecting outliers. We first transform an unlabeled input in two ways and obtain two logits from the outlier detector. Then, we minimize the distance between the two logits to encourage consistency (See Fig.~\ref{fig:consistency}). The main difference between SSL and OSSL is that unlabeled outliers do not have any neighboring labeled samples, which makes it risky to perform hard-labeling such as pseudo-labeling. The outlier detector outputs a distance from inliers given an input, and enhancing the smoothness of this function allows us to improve its ability to find outliers. Empirically, this objective provides significant improvements in detecting outliers.
Finally, to correctly classify inliers, we propose to apply FixMatch~\cite{sohn2020fixmatch} to unlabeled samples that are considered to be inliers by the outlier detector.

The resulting framework shows consistent gains over baselines on various datasets and settings of OSSL. For example, OpenMatch\xspace achieves a 10.4\% error rate on CIFAR10 with 300 labeled examples compared to the previous state-of-the-art of 20.3\%. Surprisingly, OpenMatch\xspace demonstrates good performance in detecting outliers unseen in unlabeled training data. For instance, in the experiments on CIFAR10 with 100 labels per class, OpenMatch\xspace achieves a 3.4\% higher AUROC in detecting outliers than a supervised model trained with all training samples.
To summarize, our contributions are as follows:
 
\begin{itemize}
    \item A soft open-set consistency regularization (SOCR), to improve outlier detection in OSSL.
   
    \item A new framework, OpenMatch\xspace, which combines a OVA-classifier, SOCR, and FixMatch.
     
    \item A new state-of-the-art in both correctly classifying inliers and detecting outliers,  even when the outliers are unseen in unlabeled training data.
\end{itemize}




  

\section{OpenMatch}
\vspace{-2mm}



\textbf{Problem Setting.} Our task is to learn a classifier using open-set semi-supervised learning. For a $K$-way classification problem, 
let $\mathcal{X}=\left\{\left(x_{b}, {y_{b}}\right) : b \in (1,... B)\right\}$ be a batch of $B$ labeled examples $x_b$ randomly sampled from the labeled training set, $\mathcal{S}$, where $y_b \in (1,... K)$ is the corresponding label. Let $\mathcal{U}=\left\{(u_{b}) : b \in (1,... {\mu}B)\right\}$ be a batch of ${\mu}B$ data randomly sampled from the unlabeled training set, $\mathcal{S}_u$, where $\mu$ is a hyper-parameter that determines the
relative sizes of $\mathcal{X}$ and $\mathcal{U}$. Unlike in SSL, which assumes all unlabeled data come from the $K$ known clases, in our OSSL setting they may come from unknown outlier classes. The goal of OSSL is to classify inliers into the correct classes while learning to detect the outliers. 

\textbf{Approach Overview.} 
Our model has three components: (1) a shared feature extractor $F(\cdot)$, (2) an outlier detector consisting of $K$ one-vs-all (OVA) sub-classifiers ${D}^{j}(\cdot), j \in \{1, . . . , K\}$, and (3) a closed-set classifier $C(\cdot)$ which outputs a probability vector $\in \mathbb{R}^{K}$ for $K$-class classification. 
At test time, the closed-set classifier is first applied to predict the $K$-way label, $\hat{y}$. If ${D}^{\hat{y}}$ predicts an inlier, then the output class is $\hat{y}$, otherwise, the output is "outlier".
The main technical novelty comes from the choice of OVA-classifiers for outlier detection as well as training them with soft open-set consistency regularization.
We first describe the training of the  OVA-classifiers~\cite{saito2021ovanet} before describing the remaining details of our framework. 
\vspace{-3mm}
\subsection{One-vs-All Outlier Detector}\label{sec:ova}
\vspace{-2mm}

For open-set~\cite{saito2018open} or universal domain adaptation~\cite{UDA_2019_CVPR}, Saito~\textit{et al}.\xspace~\cite{saito2021ovanet} propose to train OVA-classifiers to detect outlier samples of an unlabeled target domain. They aim to learn a boundary between inliers and outliers for each class. Each sub-classifier is trained to distinguish if the sample is an inlier for the corresponding class. For example, the sub-classifier for class $j$, $D^{j}$, outputs a 2-dimensional vector $z^{j}_{b}= D^{j}(F(x_{b})) \in \mathbb{R}^{2}$, where each dimension indicates the score of a sample being an inlier and outlier respectively. We denote ${p^{j}}(t=0|x_{b})$ and ${p^{j}}(t=1|x_{b})$ as the probability of $x_{b}$ being the inlier and outlier for the class $j$, computed by $\text{Softmax}(z_b^j)$ (Note that ${p^{j}}(t=0|x_{b}) + {p^{j}}(t=1|x_{b}) = 1$). Then, the sub-classifier $D^{j}$ is trained such that the data from class $j$ is treated as positives, and the data from all the other classes is treated as negatives. To achieve this, we minimize the following loss for the outlier detector  for a given batch $\mathcal{X}:= \{x_b, y_b\}_{b=1}^B$:
\vspace{-1mm}
\begin{equation}
\mathcal{L}_{ova}(\mathcal{X}):= \frac{1}{B}\sum_{b=1}^{B}-\log ({p^{y_{b}}}(t=0|x_{b}))-\min _{i \neq y_{b}} \log ({p^{i}}(t=1|x_{b})).
\label{eq:ova}
\end{equation}
Here we use the hard-negative sub-classifier sampling technique proposed by \cite{saito2021ovanet} to effectively learn the threshold. 
We refer the reader to~\cite{saito2021ovanet} for more details.
The key is that each sub-classifier outputs a distance representing how far the input is from the corresponding class. Therefore, the classifiers are effective to identify unlabeled outliers.  
The output of $D^{j}$ is used to detect outliers as follows. If the probability of a sample being an outlier for the class highest-scored by $C()$ is larger than that of a sample being an inlier, we consider the input as an outlier. To be specific, we identify the unlabled sample $u_{b}$ as an outlier if ${p^{\hat{y}}}(t=0|u_{b}) < 0.5$, where $ \hat{y} = \argmax_j C(F(u_b))$. 
This detection is employed in selecting pseudo-inliers from unlabeled data as explained in Sec.~\ref{sec:framework}.

\textbf{Open-set Entropy Minimization.} 
For unlabeled samples, \cite{saito2021ovanet} apply entropy minimization with respect to the OVA-classifiers, denoted by $L_{em}(\mathcal{U})$. The separation between inliers and outliers is enhanced through the minimization of this loss for a given batch $\mathcal{U}:= \{u_b\}_{b=1}^{{\mu}B}$:
\vspace{-1mm}
\begin{equation}
\mathcal{L}_{em}(\mathcal{U}) := -\frac{1}{{\mu}B}\sum_{b=1}^{{\mu}B}\sum_{j=1}^{K} \quad {p^{j}}(t=0|u_{b}) \log {p^{j}}(t=0|u_{b}) +{p^{j}}(t=1|u_{b}) \log {p^{j}}(t=1|u_{b}).
\label{eq:em}
\end{equation}

The difference between our setup and \cite{saito2021ovanet} is that they employ a model pre-trained on ImageNet~\cite{imagenet}, which extracts highly discriminative features, thus, inliers and outliers are well-separated even before fine-tuning. However, we aim to train a model to learn representations separating them from scratch. Thus the inliers and outliers are confused before training, and we need an objective to ensure the separation in addition to Eq.~\ref{eq:em}. 

\input{figures/openset}
\vspace{-2mm}
\subsection{Soft Open-set Consistency Regularization (SOCR\xspace)}
\vspace{-3mm}

To motivate our approach, we describe the difference between SSL and OSSL in Fig.~\ref{fig:comparison}.
In SSL, the hard consistency loss such as pseudo-labeling can be effective to propagate label information from labeled to unlabeled samples. 
Through the hard consistency loss, unlabeled samples  get training signals from neighboring labeled samples (Fig.~\ref{fig:comparison}(a)).
By contrast, in OSSL, since  outliers are not assigned any labels, labeled samples are far from outliers, which makes the hard labeling unreliable, and unsupervised training signals can be incorrect (Fig.~\ref{fig:comparison}(b)). 
The open-set entropy minimization above can also be considered a variant of the hard consistency since it is minimized if the predicted output class distribution is one-hot. 
To propagate useful training signals to unlabeled samples, we propose to smooth the decision boundary of the outlier detector by minimizing the distance between its predictions on two augmentations of the same image (Fig.~\ref{fig:comparison}(c)). We use unsharpened logits, keeping the training signal soft to avoid introducing incorrect pseudo-labels.
We call this loss {\it soft open-set consistency regularization (SOCR\xspace).}
Given the smoothed outlier detector, the hope is that the training signals of the open-set entropy minimization will also become more correct.

Specifically, SOCR\xspace enhances the smoothness of the outlier detector over data augmentation $\mathcal{T}$, which in our experiments is just standard random cropping. Note that the standard cropping is also used in computing Eq.~\ref{eq:ova} and~\ref{eq:em}, but we omit it for the simplicity of notation. 
We obtain two different views of $u_b$, $\mathcal{T}_1(u_b)$ and $\mathcal{T}_2(u_b)$, where $\mathcal{T}_{1}$ and $\mathcal{T}_{2}$ are data augmentation functions stochastically sampled from $\mathcal{T}$.
Let $p^{j}(t | \mathcal{T}_1(u_b))$ be the output logits of $\mathcal{T}_1(u_b)$ from the $j$th OVA-classifier. We encourage the consistency of the output logits over $\mathcal{T}$ to enhance the smoothness by minimizing the following loss for a given batch $\mathcal{U}:= \{u_b\}_{b=1}^{{\mu}B}$:
\begin{equation}
    \mathcal{L}_{oc}({\mathcal{U}}, \mathcal{T}) := \frac{1}{{\mu}B}\sum_{b=1}^{{\mu}B} \sum_{j=1}^{K}  \sum_{t\in(0,1)}|{p^{j}}(t | \mathcal{T}_{1}(u_b)) - {p^{j}}(t |\mathcal{T}_{2}(u_b))|^{2}.
    \label{eq:oc}
\end{equation}
This minimizes the distance between two logits to ensure smoothness with respect to $\mathcal{T}$.
Note that the key part of the regularization is not applying any sharpening to the output logits, keeping them soft. The major difference from the previous soft consistency regularization~\cite{ghosh2021data, tarvainen2017mean,laine2016temporal} is that we use the outlier detector to compute the regularization loss. 



\input{algo}
\vspace{-2mm}
\subsection{Overall Framework}\label{sec:framework}
\vspace{-2mm}
The entire OpenMatch algorithm is described in Alg.~\ref{alg:openmatch}. 
The unsupervised losses (Eqs.~\ref{eq:em},~\ref{eq:oc}) for the outlier detector are effective to detect outliers in unlabeled data. However, these losses are not sufficient to correctly classify unlabeled inliers. Therefore, we propose to introduce a semi-supervised learning loss for unlabeled samples classified as inliers. We adopt FixMatch~\cite{sohn2020fixmatch} since it is a simple, yet very strong SSL method. FixMatch first generates pseudo-labels using the model’s predictions on weakly augmented unlabeled images. The model is then trained to predict these pseudo-labels when fed a strongly augmented version of the same image. 
"Weak" denotes augmentations such as a simple random cropping while "strong" refers to extensive data augmentation such as RandAugment~\cite{cubuk2020randaugment} and CTAugment~\cite{berthelot2019remixmatch}.

For labeled samples, we also compute the standard cross-entropy loss to train $C(\cdot)$, $\mathcal{L}_{cls}(\mathcal{X})$, for the closed-set output, $C(F(x_b))$.
We summarize $\mathcal{L}_{sup}(\mathcal{X})$ as the sum of $\mathcal{L}_{ova}(\mathcal{X})$ and $\mathcal{L}_{cls}(\mathcal{X})$. 
After training a model for $E_{fix}$ epochs with $\mathcal{L}_{sup}$, $\mathcal{L}_{em}$, $\mathcal{L}_{oc}$, we begin to select pseudo-inliers from unlabeled data at every following epoch. 
Specifically, in  Alg.~\ref{alg:openmatch}, $\select(w, \mathcal{D}_{u})$ denotes the process of classifying unlabeled samples given model's parameters $w$ of models ($F(\cdot), D(\cdot),$ and $C(\cdot)$), and unlabeled data. The outlier-detection process is summarized in Sec.~\ref{sec:ova}.

Then, the FixMatch loss, $\mathcal{L}_{fm}(\mathcal{I})$, is computed only for the pseudo-inliers to optimize the model, where  $\mathcal{I}$ denotes a batch of pseudo-inliers.
The following is the resulting overall objective of OpenMatch:
\begin{equation}
    \mathcal{L}_{all}(\mathcal{X}, \mathcal{U}, \mathcal{T}, \mathcal{I}) := \mathcal{L}_{sup}(\mathcal{X}) + \lambda_{em} \mathcal{L}_{em}(\mathcal{X}) + \lambda_{oc} \mathcal{L}_{oc}(\mathcal{U}, \mathcal{T}) + \lambda_{fm} \mathcal{L}_{fm}(\mathcal{I}),
\end{equation}
where $\lambda_{em}$,  $\lambda_{oc}$ and  $\lambda_{fm}$ control the trade-off for each objective. 



\section{OpenMatch}
OpenMatch is a combination of three approaches:\rev{ one-vs-all classifier based open-set classification, open-set soft-consistency regularization, and FixMatch}{(1) one-vs-all classifier based open-set classification to detect outliers, (2) open-set soft-consistency regularization to improve the smoothness of the model, and (3) FixMatch for SSL learning objectives with in-liers}. Its main technical novelty comes from the soft-consistency regularization as well as \rev{the way of unifying the approaches}{unified framework for OSSL}. 
The one-vs-all classifies trained with soft-regularization effectively separate known and unknown samples, and FixMatch is applied only to pseudo-known samples. 
We first describe how to train one-vs-all classifiers~\cite{saito2021ovanet} before describing the details of our framework. 

\subsection{Problem setting and One-vs-All Network}
\textbf{Problem Setting.}
For a L-class classification problem, let $\mathcal{X}=\left\{\left(x_{b}, {y_{b}}\right) : b \in (1,... B)\right\}$ be a \rev{batch}{set} of B labeled examples, where $x_b$ are the training examples and $y_b \in (1,... L)$ are the corresponding labels. Let $\mathcal{U}=\left\{(u_{b}) : b \in (1,... \mu B)\right\}$ be a \rev{batch}{set} of $\mu B$ unlabeled examples where $\mu$ is a hyperparameter that determines the relative sizes of $\mathcal{X}$ and $\mathcal{U}$. SSL assumes the label space of $\mathcal{U}$ and $\mathcal{X}$ is the same whereas OSSL hypothesizes that of $\mathcal{U}$ includes labels absent in $\mathcal{X}$. The goal of OSSL is to classify known samples into the correct classes while learning representations effective to detect the outliers. \donghyun{Our model has three components: (1) shared feature extractor $F(\cdot)$, (2) open-set classifier consisted of one-vs-all sub-classifiers $O^{j}, j \in \{1, . . . , L\}$ for out-liers, and (3) closed-set classifier $C(\cdot)$ for in-liers.}


\textbf{One-vs-All Classifier.} For open-set~\cite{saito2018open} or universal domain adaptation~\cite{UDA_2019_CVPR} \donghyun{scenarios}, Saito~\textit{et al}.\xspace~\cite{saito2021ovanet} propose to train one-vs-all classifiers to detect out-lier samples of an unlabeled target domain. They aim to learn a boundary between in-liers and outliers for each class. The open-set classifiers consist of $L$ sub-classifiers, i.e., $O^{j}, j \in \{1, . . . , L\}$. Each \donghyun{sub-}classifier is trained to distinguish if the sample is an in-lier for the corresponding class. The sub-classifier outputs a 2-dimensional vector, where each dimension shows the probability of a sample being an in-lier and outlier respectively. Then, the \donghyun{sub-}classifier $O^{j}$ is trained such that the data from class $j$ is treated as positives, and the data from all the other classes is treated as negatives \donghyun{(\textit{i}.\textit{e}., one-vs-all classifier)}. The key is that \rev{the classifiers}{each sub-classifier} can output the distance representing how the input is far from \rev{each class}{its own class}. Therefore, the classifiers are effective to identify unlabeled unknown samples. Saito \textit{et al}.\xspace introduce hard-negative classifier sampling technique to effectively learn the threshold~\cite{saito2021ovanet}. Since the details are not the scope of this work, we explain them in the appendix. Given a shared feature extractor, the model has two heads: closed-set classifier outputs label-distribution of known classes, open-set classifier outputs anomaly score for each class. Then, for labeled samples, the objective consists of two terms: closed-set classification loss, which is a standard cross-entropy loss, and open-set classification loss. We summarize the supervised loss for the labeled samples, $\left(x_{b}, {y_{b}}\right)$ as $L_s(x_{b}, y_b)$. 



\textbf{Open-set Entropy Minimization.} 
For unlabeled samples, they apply entropy minimization with respect to the one-vs-all classifiers as denoted as $L_{em}(u_{b})$. The separation between known and unknown samples is enhanced through the minimization of the unsupervised loss. In their problem setting, they employ a model pre-trained on ImageNet~\cite{imagenet}, which extracts highly discriminative features, and the known and unknown samples are well-separated even before fine-tuning. However, we aim to train a model to learn representations separating known and unknown samples from scratch.

\input{figures/openset}
\subsection{Soft Open-set Consistency Regularization}
We describe the difference between SSL and OSSL in Fig.~\ref{fig:comparison}.
In SSL, the hard consistency loss such as pseudo-labeling can be effective to propagate label information from labeled to unlabeled samples. The open-set entropy minimization above is also considered to be a variant of the hard consistency since it is minimized if the predicted output class distribution is one-hot.
Through the hard consistency loss, unlabeled samples are supposed to get training signals from neighboring labeled samples. 
By contrast, in OSSL, since any novel samples are not assigned labelss, neighboring labeled samples are far from the novel samples, which makes the hard-labeling unreliable and unsupervised training signals can be noisy. Therefore, it is necessary to provide smoothing to the decision boundary of the open-set classifier. The smoothing will be effective to propagate less noisy training signals for the open-set entropy minimization.

Specifically, our soft consistency regularization enhances the smoothness of the open-set classifier on unlabeled data over data augmentations $\mathcal{T}$. We obtain two different views of $u_b$, $\mathcal{T}_1(u_b)$ and $\mathcal{T}_2(u_b)$, where $\mathcal{T}_{1}$ and $\mathcal{T}_{2}$ are data augmentation functions stochastically sampled from $\mathcal{T}$.
Let $p_{o}(t | \mathcal{T}_1(u_b))$ be the output logits from the open-set classifier of $\mathcal{T}_1(u_b)$. We encourage the consistency of the output logits over the different data augmnetations $\mathcal{T}$ to enhance the smoothness. Finally we minimize the following loss:

\begin{equation}
    \mathcal{L}_{oc} = \frac{1}{B}\sum_{b=1}^{B}|p_{o}(t | \mathcal{T}_{1}(u_b)) - p_{o}(t |\mathcal{T}_{2}(u_b))|^{2}.
    \label{eq:oc}
\end{equation}

Note that the key of the regularization is applying the consistency in a soft manner. 







\input{algo2}
\subsection{OpenMatch}
The unsupervised losses for open-set classifier are effective to detect outliers. However, the losses are not sufficient to correctly classify unlabeled in-liers. Therefore, we propose to introduce semi-supervised learning loss for unlabeled samples classified as in-liers. We introduce FixMatch~\cite{sohn2020fixmatch} since it is a simple, yet very strong SSL method. After training a model for $E_{fix}$ epochs with $L_{em}$, $L_{oc}$, and $L_{s}$, we classify unlabeled samples into known or unknown in every epoch. 
In  Alg.~\ref{alg:openmatch}, $\select(w, \mathcal{U})$ denotes the process of classifying unlabeled samples given model's parameter $w$ and unlabeled data.
Then, FixMatch loss, $L_{fm}$, is computed only for the pseudo-known samples to optimize a model. We call the whole framework {\it OpenMatch}. 
The following is the resulting objective:
\begin{equation}
    L_{s} + \lambda_{em} L_{em} + \lambda_{oc} L_{oc} + \lambda_{fm} L_{fm},
\end{equation}
where $\lambda_em$,  $\lambda_oc$ and  $\lambda_f$ control the trade-off for each objective.
$\lambda_em$ is set 0.1 in all experiments. 
Note that $\lambda_f$ is set 0 before $E_{fix}$ epochs and is set as 1 for all experiments. 
The whole algorithm is described in Alg.~\ref{alg:openmatch}. 


\section{Related Work}
\vspace{-3mm}
\textbf{Semi-supervised Learning.} 
Pseudo-labeling (PL) is a strong baseline in semi-supervised learning~\cite{lee2013pseudo}, and methods like FixMatch~\cite{sohn2020fixmatch} or UDA~\cite{xie2019unsupervised} combining data augmentation and PL show the highest performance on many benchmark datasets. However, even with these strong methods,  performance drops when a model is exposed to noisy unlabeled data that includes novel categories. 
On the other hand, methods based on soft consistency regularization enforce smoothness of the decision boundary with respect to stochastic transformations~\cite{sajjadi2016regularization} or models~\cite{laine2016temporal, tarvainen2017mean}. 
By "soft", we mean that no sharpening or pseudo-labeling is applied to logits to propagate training signals through soft targets.
The soft consistency regularization can be understood as approximated Jacobian regularization method~\cite{ghosh2021data, rifai2011manifold}.
We propose to apply soft consistency regularization to our outlier detector, which we call soft open-set consistency regularization. 
Training a model with self-supervised learning and SSL achieves label efficient training~\cite{chen2020big,zhai2019s4l}. Self-supervised learning and SSL seem to be complementary with each other to improve the recognition accuracy. But, unlabeled data with outliers will hurt the performance of these approaches since they rely on existing SSL objectives. Since our work focuses on the aspect of SSL, we hope it is complementary to self-supervised learning.

\textbf{Open-set SSL} methods include
MTC~\cite{yu2020multi}, D3SL~\cite{guo2020safe}, and UASD~\cite{chen2020semi}. 
MTC updates the network parameters and the anomaly score of unlabeled data alternately. In addition, it minimizes SSL loss (MixMatch~\cite{berthelot2019mixmatch} for a part of unlabeled data that is considered to be inliers. 
D3SL selectively employ unlabeled data to optimize SSL loss as well as optimizing a function that selects unlabeled data. 
UASD generates soft targets of the classifier for inliers (closed-set classifier) by averaging over many temporally ensembled networks' predictions. By contrast, OpenMatch\xspace does not have to store the temporal ensembles and applies soft consistency loss to the \textit{outlier detector}. Our empirical results show that the consistency loss for the outlier detector outperforms that for the closed-set classifier by a lot (See Sec.~\ref{sec:analysis}).

\textbf{Open-set Domain Adaptation (ODA).}
OSSL is similar to open-set domain adaptation~\cite{busto2017open, saito2018open} in that the unlabeled data contains novel categories. A key difference is that the unlabeled and labeled data follow different data distributions.
Another key difference is that we aim to train a model from scratch while the domain adaptation task assumes access to models pre-trained on ImageNet~\cite{imagenet}. Since the ImageNet contains 1,000 diverse categories, the pre-trained models have discriminative representations useful to detect outliers. In this point, our task is more challenging than ODA. To detect outliers in an unsupervised way, we employ a OVA-classifier model~\cite{saito2021ovanet} proposed for this task.

\textbf{Novelty Detection.}
Novelty detection or open-set classification aims to identify outliers that are completely unseen during training~\cite{hendrycks2016baseline, liang2017enhancing,devries2018learning,geng2020recent}. Self-supervised learning such as rotation prediction and contrastive learning is shown to be useful to separate outliers from inliers~\cite{hendrycks2019using, tack2020csi, winkens2020contrastive}. 
The task assumes plenty of labeled inliers in training data. 
Padhy~\textit{et al}.\xspace employ OVA-classifiers for this task~\cite{padhy2020revisiting}.
Hendrycks~\textit{et al}.\xspace reveal that exposing a model to outlier data allows it to effectively detect anomalies~\cite{hendrycks2018deep} and train it on out-of-distribution training data in a supervised way. But, collecting such useful out-of-distribution data is not always possible. 
In OSSL, a model cannot be trained on many labeled samples and needs to detect which samples are inlier or outliers in unlabeled data. In this sense, OSSL is more challenging and more realistic.





