
\section{Introduction}
\label{sec:intro}

Deep learning has reached human-level performance on domains with the abundance of large-scale labeled training data. However, learning on tasks with a small number of annotated examples is still an open challenge. Due to the lack of training data, models often overfit or are too simplistic to provide good generalization. On the contrary, humans can learn new tasks very quickly by drawing upon prior knowledge and experience. This ability to rapidly learn and adapt to new environments is a hallmark of human intelligence. 

Few-shot learning ~\citep{miller00, fei06, koch15} aims at addressing this fundamental challenge by designing algorithms that are able to generalize to new tasks given only a few labeled training examples. Meta-learning ~\citep{schmidhuber87, bengio92} has recently made major advances in the field by explicitly optimizing the model's ability to generalize, or learning how to learn, from many related tasks ~\citep{snell17,vinyals16,ravi17, finn17}.
Motivated by the way humans effectively use prior knowledge, meta-learning algorithms acquire prior knowledge over previous tasks so that new tasks can be efficiently learned from a small amount of data. However, recent works ~\citep{chen2019closer, raghu19} show that simple baseline methods perform comparably to existing meta-learning methods, opening the question about which components are crucial for rapid adaptation and generalization.

Here, we argue that there is an important missing piece in this puzzle. Human knowledge is structured in the form of reusable concepts. For instance, when we learn to recognize new bird species we are already equipped with the critical concepts, such as wing, beak, and feather. We then focus on these specific concepts and combine them to identify a new species. While learning to recognize new species is challenging in the complex bird space, it becomes remarkably simpler once the reasoning is structured into familiar concepts. Moreover, such a structured way of cognition gives us the ability to provide reasoning behind our decisions, such as ``ravens have thicker beaks than crows, with more of a curve to the end''. We argue that this lack of structure is limiting the generalization ability of the current meta-learners. The importance of compositionality for few-shot learning was emphasized in ~\citep{lake11, lake15} where hand-designed features of strokes were combined using Bayesian program learning.

Motivated by the structured form of human cognition, we propose COMET\xspace, a meta-learning method that discovers generalizable representations along human-interpretable concept dimensions. COMET\xspace learns a unique metric space for each {\em concept dimension} using concept-specific embedding functions, named {\em concept learners}, that are parameterized by deep neural networks. Along each high-level dimension, COMET\xspace defines {\em concept prototypes} that reflect class-level differences in the metric space of the underlying concept. To obtain final predictions, COMET\xspace effectively aggregates information from diverse concept learners and concept prototypes. Three key aspects lead to a strong generalization ability of our approach: (i) semi-structured representation learning, (ii) concept-specific metric spaces described with concept prototypes, and (iii) ensembling of many models. The latter assures that the combination of diverse and accurate concept learners improves the generalization ability of the base learner ~\citep{hansen90, dvornik19}. Remarkably, the high-level universe of concepts that are used to guide our algorithm can be discovered in a fully unsupervised way, or we can use external knowledge bases to define concepts. In particular, we can get a large universe of noisy, incomplete and redundant concepts and COMET\xspace learns which subsets of those are important by assigning local and global concept importance scores. Unlike existing methods ~\citep{snell17,vinyals16, sung2018learning, gidaris18}, COMET\xspace's predictions are interpretable---an advantage especially important in the few-shot learning setting, where predictions are based only on a handful of labeled examples making it hard to trust the model. As such, COMET\xspace is the first domain-agnostic interpretable meta-learning approach.

We demonstrate the effectiveness of our approach on tasks from extremely diverse domains, including fine-grained image classification in computer vision, document classification in natural language processing, and cell type annotation in biology. In the biological domain, we conduct the first systematic comparison of meta-learning algorithms. We develop a new meta-learning dataset and define a novel benchmark task to characterize single-cell transcriptome of all mouse organs ~\citep{tabula18, tabulamurissenis19}.
Additionally, we consider the scenario in which concepts are not given in advance, and test COMET\xspace's performance with automatically extracted visual concepts.
Our experimental results show that on all domains COMET\xspace significantly improves generalization ability, achieving $6$--$15\%$ relative improvement over state-of-the-art methods in the most challenging $1$-shot task. Furthermore, we demonstrate the ability of COMET\xspace to provide interpretations behind the model's predictions, and support our claim with quantitative and qualitative evaluations of the generated explanations.

\begin{figure}[t]
    \centering
    \includegraphics[width=\textwidth]{figs/model.pdf}
    \vspace{-1.5em}
    \caption{Along each concept dimension, COMET\xspace learns concept embeddings using independent concept learners and compares them to concept prototypes. COMET\xspace then effectively aggregates information across concept dimensions, assigning concept importance scores to each dimension.}
    \label{fig:model}
   
\end{figure}


\section{Related work}
\label{sec:related}

Our work draws motivation from a rich line of research on meta-learning, compositional representations, and concept-based interpretability. 

\xhdr{Meta-learning} Recent meta-learning methods fall broadly into two categories. 
Optimization-based methods ~\citep{finn17, rusu18, nichol18, grant18, antoniou19} aim to learn a good initialization such that network can be fine-tuned to a target task within a few gradient steps. On the other hand, metric-based methods ~\citep{snell17,vinyals16, sung2018learning, gidaris18} learn a metric space shared across tasks such that in the new space target task can be solved using nearest neighbour or simple linear classifier. DeepEMD ~\citep{zhang2020deepemd} learns optimal distance between local image representations. Prototypical networks ~\citep{snell17} learn a metric space such that data points cluster around a prototypical representation computed for each category as the mean of embedded labeled examples. It has remained one of the most competitive few-shot learning methods ~\citep{triantafillou2019meta}, resulting in many follow-up works ~\citep{sung2018learning, oreshkin18, ren18, liu19, xing19}. Two recent works ~\citep{hou2019cross, zhu20multi} proposed to learn local discriminative features with attention mechanisms in image classification tasks. Our work builds upon prototypical networks and extends the approach by introducing concept-based prototypes. Prototypical networks were extended by learning mixture prototypes in ~\citep{allen2019infinite}; however prototypes in this work share the same metric space. In contrast, COMET defines human-interpretable concept-specific metric spaces where each prototype reflects class-level differences in the metric space of the corresponding concept. 

\xhdr{Compositionality} The idea behind learning from a few examples using compositional representations originates from work on Bayesian probabilistic programs in which individual strokes were combined for the handwritten character recognition task ~\citep{lake11, lake15}.
This approach was extended in ~\citep{wong15} by replacing hand designed features with symmetry axis as object descriptors. Although these early works effectively demonstrated that compositionality is a key ingredient for adaptation in a low-data regime, it is unclear how to extend them to generalize beyond simple visual concepts. Recent work ~\citep{tokmakov19} revived the idea and showed that deep compositional representations generalize better in few-shot image classification. However, this approach requires category-level attribute annotations that are impossible to get in domains not intuitive to humans, such as biology. Moreover, even in domains in which annotations can be collected, they require tedious manual effort. On the contrary, our approach is domain-agnostic and generates human-understandable interpretations in any domain.

\xhdr{Interpretability} There has been much progress on designing interpretable methods that estimate the importance of individual features ~\citep{ selvaraju16gradcam, sundararajan17axiomatic, smilkov17smoothgrad, ribeiro16lime,lundberg17shap, melis18selfexplain}. However, individual features
are often not intuitive, or can even be misleading when interpreted by humans ~\citep{kim18tcav}. To overcome this limitation, 
recent advances have been focused on designing methods that explain predictions using high-level human understandable concepts ~\citep{kim18tcav, ghorbani19}. TCAV ~\citep{kim18tcav} defines concepts based on user-annotated set of examples in which the concept of interest appears.
On the contrary, high-level concepts in our work are defined with a set of related dimensions. As such, they are already available in many domains, or can be obtained in an unsupervised manner. Once defined, they are transferable across problems that share feature space. As opposed to the methods that base their predictions on the posthoc analysis ~\citep{ribeiro16lime,lundberg17shap, melis18selfexplain, kim18tcav}, COMET\xspace is designed as an inherently interpretable model and explains predictions by gaining insights from the reasoning process of the network. The closest to our work are prototypes-based explanation models ~\citep{li18protoexplain, chen19protopnet}. However, they require specialized convolutional architecture for feature extraction and are not applicable beyond image classification, or to a few-shot setting. 


\section{Proposed method}
\label{sec:proposed}

\xhdr{Problem formulation} In few-shot classification, we assume that we are given a labeled training set $\mathcal{D}_{tr}$, an unlabeled query set $\mathcal{D}_{qr}$, and a support set $\mathcal{S}$ consisting of a few labeled data points that share the label space with the query set. Label space between training and query set is disjoint, \textit{i.e.,} $\{{Y}_{tr}\}\cap\{{Y}_{qr}\}=\emptyset$, where $\{{Y}_{tr}\}$ denotes label space of training set and $\{{Y}_{qr}\}$ denotes label space of query set.
Each labeled data point $(\vec x, y)$ consists of
a $D$-dimensional feature vector $\vec x \in \mathbb{R}^D$ and a class label $y \in \{1, . . . ,K\}$.
Given a training set of previously labeled tasks $\mathcal{D}_{tr}$ and the support set $\mathcal{S}$ of a few labeled data points on a novel task, the goal is to train a model that can generalize to the novel task and label the query set $\mathcal{D}_{qr}$.
\subsection{Preliminaries}
\label{sec:preliminaries}
\xhdr{Episodic training}
To achieve successful generalization to a new task, training of meta-learning methods is usually performed using sampled mini-batches called episodes ~\citep{vinyals16}. Each episode is formed by first sampling classes from the training set, and then sampling data points labeled with these classes. The sampled data points are divided into disjoint sets of: (i) a support set consisting of a few labeled data points, and (ii) a query set consisting of data points whose labels are used to calculate a prediction error. Given the sampled support set, the model minimizes the loss on the sampled query set in each episode. The key idea behind this meta-learning training scheme is to improve generalization of the model by trying to mimic the low-data regime encountered during testing. Episodes with balanced training sets are usually referred to as ``N-way, k-shot'' episodes where $N$ indicates number of classes per episode (``way''), and $k$ indicates number of support points (labeled training examples) per class (``shot'').

\xhdr{Prototypical networks} Our work is inspired by prototypical networks ~\citep{snell17}, a simple but highly effective metric-based meta-learning method. Prototypical networks learn a non-linear embedding function $f_\vec\theta:\mathbb{R}^D\rightarrow \mathbb{R}^M$ parameterized by a convolutional neural network. The main idea is to learn a function $f_\vec\theta$ such that in the $M$-dimensional embedding space data points cluster around a single prototype representation $\vec p_k \in \mathbb{R}^M$ for each class $k$. Class prototype $\vec p_k$ is computed as the mean vector of the support set labeled with the class $k$:
\begin{equation}
    \vec p_k = \frac{1}{|\mathcal S_k|}\sum_{(\vec x_i, y_i)\in \mathcal S_k} f_\vec\theta(\vec x_i),
\end{equation}
where $\mathcal S_k$ denotes the subset of the support set $\mathcal S$ belonging to the class $k$. Given a query data point $\vec x_q$, prototypical networks output distribution over classes using the softmax function:
\begin{equation}
    p_\vec\theta(y=k|\vec x_q)=\frac{\exp(-d(f_\vec\theta(\vec x_q),\vec p_k))}{\sum_{k'} \exp(-d(f_\vec\theta(\vec x_q),\vec p_{k'}))},
\end{equation}
where $d:\mathbb{R}^M\rightarrow \mathbb{R}$ denotes the distance function. Query data point $\vec x_q$ is assigned to the class with the minimal distance between the class prototype and embedded query point.

\subsection{Meta-learning via concept learners}

Our main assumption is that input dimensions can be separated into subsets of related dimensions corresponding to high-level, human-interpretable concepts that guide the training. Such sets of potentially overlapping, noisy and incomplete human-interpretable dimensions exists in many real-world scenarios. For instance, in computer vision concepts can be assigned to image segments; in natural language processing to semantically related words; whereas in biology we can use external knowledge bases and ontologies. In many problems, concepts are already available as a prior domain knowledge ~\citep{GO00, scop95, wah2011caltech, mo2019partnet, miller00}, or can be automatically generated using existing techniques ~\citep{blei03, zhang18, jakab18}. Intuitively, concepts can be seen as part-based representations of the input and reflect the way humans reason about the world. Importantly, we do not assume these concepts are clean or complete. On the contrary, we show that even if there are thousands of concepts, which are noisy, incomplete, overlapping, or redundant, they still provide useful guidance to the meta-learning algorithm.

Formally, let $\mathcal{C}=\{\vec c^{(j)}\}_{j=1}^{N}$ denote a set of $N$ concepts given/extracted as a prior knowledge, where each concept $\vec c^{(j)}\in \{0,1\}^D$ is a binary vector such that $c_i^{(j)}=1$ if $i$-th dimension should be used to describe the $j$-th concept and $D$ denotes the dimensionality of the input. We do not impose any constraints on $\mathcal{C}$, meaning that the concepts can be disjoint or overlap. Instead of learning single mapping function $f_\vec\theta:\mathbb{R}^D\rightarrow \mathbb{R}^M$ across all dimensions, COMET\xspace separates original space into subspaces of predefined concepts and learns individual embedding functions $f_\vec\theta^{(j)}:\mathbb{R}^{D}\rightarrow \mathbb{R}^M$ for each concept $j$ (Figure ~\ref{fig:model}). Concept embedding functions $f_\vec\theta^{(j)}$, named {\em concept learners}, are non-linear functions parametrized by a deep neural network. 
Each concept learner $j$ produces its own {\em concept prototypes} $\vec p_k^{(j)}$ for class $k$ computed as the average of concept embeddings of data points in the support set:
\begin{equation}
     \vec p_k^{(j)} = \frac{1}{|\mathcal S_k|}\sum_{(\vec x_i, y_i)\in \mathcal S_k} f_\vec{\theta}^{(j)}(\vec x_i\circ \vec c^{(j)}),
\end{equation}
where $\circ$ denotes Hadamard product. As a result, each class $k$ is represented with a set of $N$ concept prototypes $\{\vec p_k^{(j)}\}_{j=1}^N$.

Given a query data point $\vec x_q$, we compute its concept embeddings and estimate their distances to the concept prototypes of each class. We then aggregate the information across all concepts by taking sum over distances between concept embeddings and concept prototypes. Specifically, for each concept embedding $f_\vec{\theta}^{(j)}(\vec x_q\circ \vec c^{(j)})$ we compute its distance to concept prototype $\vec p_k^{(j)}$ of a given class $k$, and sum distances across all concepts to obtain a distribution over support classes. The probability of assigning query point $\vec x_q$ to $k$-th class is then given by:
\begin{equation} \label{eq_softmaxx}
    p_\vec\theta(y=k|\vec x_q)=\frac{\exp(-\sum_j d(f_\vec\theta^{(j)}(\vec x_q\circ \vec c^{(j)}),\vec p_k^{(j)}))}{\sum_{k'} \exp(-\sum_j d(f_\vec\theta^{(j)}(\vec x_q\circ \vec c^{(j)}),\vec p_{k'}^{(j)}))}.
\end{equation}

The loss is computed as the negative log-likelihood $L_\vec\theta=-\log p_\vec\theta(y=k|\vec x_q)$ of the true class, and COMET\xspace is trained by minimizing the loss on the query samples of training set in the episodic fashion ~\citep{snell17, vinyals16}. In equation (\ref{eq_softmaxx}), we use euclidean distance as the distance function. Experimentally, we find that it outperforms cosine distance (Appendix \ref{sec:distance}), which agrees with the theory and experimental findings in ~\citep{snell17}. We note that in order for distances to be comparable, it is crucial to normalize neural network layers using batch normalization ~\citep{ioffe2015batch}. 

\subsection{Interpretability}

\xhdr{Local and global concept importance scores} In COMET\xspace, each class is represented with $N$ concept prototypes. Given a query data point $\vec x_q$, COMET\xspace assigns local concept importance scores by comparing concept embbeddings of the query to concept prototypes. Specifically, for a concept $j$ in a class $k$ the local importance score is obtained by inverted distance $d(f_\vec\theta^{(j)}(\vec x_q\circ \vec c^{(j)}),\vec p_k^{(j)})$. Higher importance score indicates higher contribution in classifying query point to the class $k$. Therefore, explanations for the query point $\vec x_q$ are given by local concept importance scores, and directly provide reasoning behind each prediction. To provide global explanations that can reveal important concepts for a set of query points of interest or an entire class, COMET\xspace computes average distance between concept prototype and concept embeddings of all query points of interest. Inverted average distance reflects global concept importance score and can be used to rank concepts, providing insights on important concepts across a set of examples.

\xhdr{Discovering locally similar examples} Given a fixed concept $j$, COMET\xspace can be used to rank data points based on the distance of their concept embeddings to the concept prototype $\vec p_k^{(j)}$ of class $k$. By ranking data points according to their similarity to the concept of interest, COMET\xspace can find examples that locally share similar patterns within the same class, or even across different classes. For instance, COMET\xspace can reveal examples that well reflect a concept prototype, or examples that are very distant from the concept prototype. 
\section{Experiments}
\label{sec:experiments}

\subsection{Experimental setup}

\xhdr{Datasets} We apply COMET\xspace to four datasets from three diverse domains: computer vision, natural language processing (NLP) and biology. In the computer vision domain, we consider fine-grained image classification tasks.
We use bird classification CUB-200-2011 ~\citep{wah2011caltech} and flower classification Flowers-102 ~\citep{nilsback2008flower} datasets, referred to as CUB and Flowers hereafter. To define concepts, CUB provides part-based annotations, such as beak, wing, and tail of a bird. Parts were annotated by pixel location and visibility in each image. The total number of $15$ parts/concepts is available; however concepts are incomplete and only a subset of them is present in an image. In case concept is not present, we rely on the prototypical concept to substitute for a missing concept. 
Based on the part coordinates, we create a surrounding bounding box with a fixed length to serve as the concept mask $\vec c^{(j)}$. On both CUB and Flowers datasets, we test automatic concept extraction. In NLP domain, we apply COMET\xspace to benchmark document classification dataset Reuters ~\citep{lewis2004rcv1} consisting of news articles. To define concepts, we use all hypernyms of a given word based on the WordNet hiearchy ~\citep{lewis2004rcv1}. On all datasets, we include a concept that captures the whole input, corresponding to a binary mask of all ones. 
 
In the biology domain, we introduce a new cross-organ cell type classification task ~\citep{brbic2020mars} together with a new dataset. We develop a novel single-cell transcriptomic dataset based on the Tabula Muris dataset ~\citep{tabula18, tabulamurissenis19} that comprises $105,960$ cells of $124$ cell types collected across $23$ organs of the mouse model organism. The features correspond to the gene expression profiles of cells. Out of the $23,341$ genes, we select $2,866$ genes with high standardized log dispersion given their mean. We define concepts using Gene Ontology ~\citep{GO00, GO19}, a resource which characterizes gene functional roles in a hierarchically structured vocabulary.
We select Gene Ontology terms at level $3$ that have at least $64$ assigned genes, resulting in the total number of $190$ terms that define our concepts.
We propose the evaluation protocol in which different organs are used for training, validation, and test splits. Therefore, a meta-learner needs to learn to generalize to unseen cell types across organs. This novel dataset along with the cross-organ evaluation splits is publicly available at \url{https://snap.stanford.edu/comet}. To our knowledge, this is the first meta-learning dataset from the biology domain. 

\xhdr{Baselines} We compare COMET\xspace's performance to seven baselines, including FineTune/Baseline++ ~\citep{chen2019closer}, Matching Networks (MatchingNet) ~\citep{vinyals16}, Model Agnostic Meta-Learning (MAML) ~\citep{finn17}, Relation Networks ~\citep{sung2018learning}, MetaOptNet ~\citep{lee2019metaoptnet}, DeepEMD ~\citep{zhang2020deepemd} and Prototypical Networks (ProtoNet) ~\citep{snell17}. DeepEMD is only applicable to image datasets.

We provide more details on evaluation and implementation in Appendix \ref{sec:impl_details}. Code is publicly available at \url{https://github.com/snap-stanford/comet}.
 
\subsection{Results} 

\xhdr{Performance comparison} 
We report results on CUB, Tabula Muris and Reuters datasets with concepts given as a prior domain knowledge in Table~\ref{tab:CUB_miniI}.
COMET\xspace outperforms all baselines by a remarkably large margin on all datasets. Specifically, COMET\xspace achieves $9.5\%$ and $9.3\%$ average improvements over the best performing baseline in the $1$-shot and $5$-shot tasks across datasets.
Notably, COMET\xspace improves the result of the ProtoNet baseline by $19$--$23\%$ in the $1$-shot tasks across datasets. COMET\xspace's substiantial improvement are retained with the deeper Conv-6 backbone (Appendix \ref{sec:backbone}). 
To confirm that the improvements indeed come from concept learners and not from additional weights, we compare COMET\xspace to ensemble of prototypical networks, and further evaluate performance of COMET\xspace with shared weights across all concepts.
Results shown in Table \ref{tab:ensemble} demonstrate that COMET\xspace achieves significantly better performance than the ensemble of ProtoNets even when the weights across concepts are shared. Of note, COMET\xspace's performance is only slightly affected with shared weights across concepts. More experimental details are provided in Appendix \ref{sec:ensemble}.

\xhdr{Effect of number of concepts} We systematically evaluate the effect of the number of concepts on COMET\xspace's performance on CUB and Tabula Muris datasets (Figure~\ref{fig:concepts}). In particular, we start from ProtoNet's result that can be seen as using a single concept in COMET\xspace that covers all dimensions of the input. We then gradually increase number of concepts and train and evaluate COMET\xspace with the selected number of concepts. For the CUB dataset, we add concepts based on their visibility frequency, whereas on the Tabula Muris we are not limited in the coverage of concepts so we randomly select them. The results demonstrate that on both domains COMET\xspace consistently improves performance when increasing the number of concepts. Strikingly, by adding just one most frequent concept corresponding to a bird's beak on top of the whole image concept, we improve ProtoNet's performance on CUB by $10\%$ and $5\%$ in $1$-shot and $5$-shot tasks, respectively.
On the Tabula Muris, with just $8$ concepts COMET\xspace significantly outperforms all baselines and achieves $7\%$ and $17\%$ improvement over ProtoNet in $1$-shot and $5$-shot tasks, respectively. To demonstrate the robustness of our method to a huge set of overlapping concepts, we extend the number of concepts to $1500$ by capturing all levels of the Gene Ontology hierarchy, therefore allowing many redundant relationships. Even in this scenario, COMET\xspace slightly improves the results compared to $190$ concepts obtained from a single level. These results demonstrate that COMET\xspace outperforms other methods even when the number of concepts is small and annotations are incomplete, as well as with many overlapping and redundant concepts.

\begin{table}[t]
\centering
\fontsize{9}{9}\selectfont
\caption{Results on $1$-shot and $5$-shot classification on the CUB and Tabula Muris datasets. We report average accuracy and standard deviation over $600$ randomly sampled episodes.}

\label{tab:CUB_miniI}
\begin{tabular}{lcccccc}
\toprule
\textbf{}             & \multicolumn{2}{c}{\textbf{CUB}}   & \multicolumn{2}{c}{\textbf{Tabula Muris}}   & \multicolumn{2}{c}{\textbf{Reuters}}  \\
\small{\textbf{Method}}             & \textbf{1-shot}    & \textbf{5-shot}    & \textbf{1-shot}    & \textbf{5-shot}  & \textbf{1-shot}    & \textbf{5-shot}  \\ \midrule
\small{\textbf{Finetune}}   & $61.4 \pm 1.0$ & $80.2 \pm 0.6$ & $65.3 \pm 1.0$ & $82.1 \pm 0.7$  & $48.2 \pm 0.7 $ &  $64.3 \pm 0.4 $ \\  
\small{\textbf{MatchingNet}}  & $61.0 \pm 0.9$ & $75.9 \pm 0.6$ & $71.0 \pm 0.9$ & $82.4 \pm 0.7$ & $55.9\pm 0.6 $ & $70.9 \pm 0.4 $ \\
\small{\textbf{MAML}}  & $52.8 \pm 1.0$ & $74.4 \pm 0.8$ & $50.4 \pm 1.1$ & $57.4 \pm 1.1$  & $ 45.0 \pm 0.8$ & $60.5 \pm 0.4 $  \\
\small{\textbf{RelationNet}}     & $62.1 \pm 1.0$ & $78.6 \pm 0.7$ & $69.3 \pm 1.0$ & $80.1 \pm 0.8$  & $53.8 \pm 0.7 $ & $68.3 \pm 0.3 $ \\ 
\small{\textbf{MetaOptNet}} & $62.2 \pm 1.0 $  & $79.6 \pm 0.6 $ & $73.6 \pm 1.1 $ & $ 85.4 \pm 0.9 $ & $62.1 \pm 0.8 $ & $77.8 \pm 0.4 $ \\
\small{\textbf{DeepEMD}} & $64.0 \pm 1.0$  &  $81.1 \pm 0.7 $ &  NA & NA & NA & NA  \\
\midrule
\small{\textbf{ProtoNet}}    & $57.1 \pm 1.0$ & $76.1 \pm 0.7$  & $64.5 \pm 1.0$ & $82.5 \pm 0.7$  & $58.3 \pm 0.7 $ & $75.1 \pm 0.4$ \\
\small{\textbf{COMET\xspace}} & {$\textbf{67.9} \pm \textbf{0.9}$} & {$\textbf{85.3} \pm \textbf{0.5}$} & {$\textbf{79.4} \pm \textbf{0.9}$} & {$\textbf{91.7} \pm \textbf{0.5}$}  & $\textbf{71.5} \pm \textbf{0.7}$ & \textbf{ $\textbf{89.8} \pm \textbf{0.3}$} \\
\bottomrule
\end{tabular}
\end{table}

\begin{table}[t]
\centering
\fontsize{9}{9}\selectfont
\caption{Comparison to the ensemble of prototypical networks and COMET\xspace with shared weights across concepts. On the CUB dataset weights are always shared.}
\label{tab:ensemble}
\begin{tabular}{lcccccc}
\toprule
\textbf{}             & \multicolumn{2}{c}{\textbf{CUB}}    & \multicolumn{2}{c}{\textbf{Tabula Muris}}  & \multicolumn{2}{c}{\textbf{Reuters}}              \\
\small{\textbf{Method}}             & \textbf{1-shot}    & \textbf{5-shot}  & \textbf{1-shot}    & \textbf{5-shot}  & \textbf{1-shot}    & \textbf{5-shot}  \\ \midrule
\small{\textbf{ProtoNetEns}} & $64.0 \pm 0.8$ & $82.3 \pm 0.5$ & $67.2 \pm 0.8$ & $83.6 \pm 0.5$ & $62.4 \pm 0.7$ & $79.3 \pm 0.4$ \\
\small{\textbf{COMET\xspace shared w}} & {$67.9\pm 0.9$} & {$85.3 \pm 0.5$}  & $78.2 \pm 1.0$ & $91.0 \pm 0.5$ & $69.8 \pm 0.8$ & $88.6 \pm 0.3$\\
\small{\textbf{COMET\xspace}} & {$67.9\pm 0.9$} & {$85.3 \pm 0.5$} &  {$79.4 \pm 0.9$} & {$91.7 \pm 0.5$} & {$71.5 \pm 0.7$} &  ${89.8 \pm 0.3}$ \\
\bottomrule
\end{tabular}
\end{table}

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{figs/concept_acc.pdf}
    \vspace{-1.5em}
    \caption{The effect of number of concepts on COMET\xspace's performance. COMET\xspace consistently improves performance when we gradually increase number of concept terms.}
    \label{fig:concepts}
\end{figure}

\xhdr{Unsupervised concept annotation} While COMET\xspace achieves remarkable results with human-validated concepts given as external knowledge, we next investigate COMET\xspace's performance on automatically inferred concepts. In addition to CUB dataset, we consider Flowers dataset for fine-grained image classification. To automatically extract visual concepts, we train the autoencoding framework for landmarks discovery proposed in ~\citep{zhang18}. The encoding module outputs landmark coordinates that we use as part coordinates.
We generate a concept mask by creating a bounding box with a fixed length around landmark coordinates. Although extracted coordinates are often noisy and capture background (Appendix \ref{sec:unsup_key_pts_example}), we find that COMET\xspace outperforms all baselines on both CUB and Flowers fine-grained classification datasets (Table~\ref{tab:CUB_landmarks}). This analysis shows that the benefits of our method are expected even with noisy concepts extracted in a fully automated and unsupervised way. 

To test unsupervised concept annotation on Tabula Muris and Reuters datasets, we randomly select subsets of features for concept definition. Since COMET is interpretable and can be used to find important concepts, we use validation set to select concepts with the highest importance scores. Even in this case, COMET significantly outperforms all baselines, achieving only $2\%$ lower accuracy on the Tabula Muris dataset and $1\%$ on the Reuters dataset on both 1-shot and 5-shot tasks compared to human-defined concepts. This additionally confirms COMET’s effectiveness with automatically extracted concepts.  We provide more results in Appendix \ref{sec:unsup_results} .

\begin{table}[h]
\centering
\fontsize{9}{8}\selectfont
\caption{Results on $1$-shot and $5$-shot classification with automatically extracted concepts. We report average accuracy and standard deviation over $600$ randomly sampled episodes. We show the average relative improvement of COMET\xspace over the best and ProtoNet baselines.}
\label{tab:CUB_landmarks}
\begin{tabular}{lcccc}
\toprule
\small{\textbf{Accuracy}}             & \textbf{CUB: 1-shot}    & \textbf{CUB: 5-shot}    & \textbf{Flowers: 1-shot}    & \textbf{Flowers: 5-shot}    \\ \midrule
\small{\textbf{COMET\xspace}} & $64.8 \pm 1.0$ & $82.0 \pm 0.5$ & $70.4 \pm 0.9$ & $86.7 \pm 0.6$ \\ \midrule
\multicolumn{2}{l}{\small{\textbf{Improvement of COMET\xspace...}}} & & & \\
\small{\textbf{over best baseline}} & {${1.3\%}$} & {${1.1\%}$} & {{$4.8\% $}} & {${4.6\%} $}\\
\small{\textbf{over ProtoNet}} & {${13.5\%}$} & {${7.8\%}$} & {{$6.0\%$}} & {${8.1\%}$}\\
\bottomrule
\end{tabular}
\end{table}

\subsection{Interpretability} 

We analyze the reasoning part of COMET\xspace by designing case studies aiming to answer the following questions: (i) Which concepts are the most important for a given query point (\textit{i.e.}, local explanation)? Which concepts are the most important for a given class (\textit{i.e.}, global explanation)?; (iii) Which examples share locally similar patterns?; (iv) Which examples reflect well concept prototype? We perform all analyses exclusively on classes from the novel task that are not seen during training.

\begin{figure}[!b]
    \centering
    \includegraphics[width=\textwidth]{figs/global_exp.pdf}
    \vspace{-1.5em}
    \caption{(Left) Quantitatively, on the Tabula Muris dataset COMET\xspace's global importance scores agree well with the ground truth important Gene Ontology terms estimated using differentially expressed genes. (Right) Qualitatively, on the CUB dataset importance scores correctly reflect the most relevant bird features.} 
    \label{fig:global_exp}
\end{figure}


\xhdr{Concept importance} Given a query point, COMET\xspace ranks concepts based on their importance scores, therefore identifying concepts highly relevant for the prediction of a single query point. We demonstrate examples of local explanations in Appendix \ref{sec:local_exp}.
To quantitatively evaluate global explanations that assign concept importance scores to the entire class, we derive ground truth explanations on the Tabula Muris dataset. Specifically, using the ground truth labels on the test set, we obtain a set of genes that are differentially expressed for each class (\textit{i.e.}, cell type). We then find Gene Ontology terms that are significantly enriched (false discovery rate corrected $p$-value$<0.1$) in the set of differentially expressed genes of a given class, and use those terms as ground-truth concepts. We consider only cell types that have at least two assigned terms. To obtain COMET\xspace's explanations, we rank global concept importance scores for each class and report the number of relevant terms that are successfully retrieved in top $20$ concepts with the highest scores in the $5$-shot setting (Figure~\ref{fig:global_exp} left). 
We find that COMET\xspace's importance scores agree extremely well with the ground truth annotations, achieving $0.71$ average recall@$20$ across all cell types. We further investigate global explanations on the CUB dataset by computing the frequency of the most relevant concepts across the species (Figure~\ref{fig:global_exp} right). Beak, belly and forehead turn out to be the most relevant features, supporting common-sense intuition. For instance, `beak' is selected as the most relevant concept for `parakeet auklet' known for its nearly circular beak; `belly' for `cape may warbler' known for its tiger stripes on the belly; while `belted kingfisher' indeed has characteristic `forehead' with its shaggy crest on the top of the head. This confirms that COMET\xspace correctly identifies important class-level concepts.

\xhdr{Locally similar patterns} Given a fixed concept of interest, we apply COMET\xspace to sort images with respect to the distance of their concept embedding to the concept prototype (Figure~\ref{fig:concept_ranks}).
COMET\xspace finds images that locally resemble the prototypical image and well express concept prototype, correctly reflecting the underlying concept of interest. On the contrary, images sorted using the whole image as a concept often reflect background similarity and can not provide intuitive explanations. Furthermore, by finding most distant examples COMET\xspace can aid in identifying misannotated or non-visible concepts (Appendix ~\ref{sec:concept_rank_incls}) which can be particularly useful when the concepts are automatically extracted. These analyses suggest that COMET\xspace can be used to discover, sort and visualize locally similar patterns, revealing insights on concept-based similarity across examples.

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{figs/concept_rank.pdf}
    \vspace{-1.5em}
    \caption{Top row shows images with beak concept embeddings most similar to the prototypical beak. Bottom row shows images ranked according the global concept that captures whole image. COMET\xspace correctly reflects local similarity in the underlying concept of interest, while global concept often reflects environmental similarity.}
    \label{fig:concept_ranks}
\end{figure}

\section{Conclusion}
\label{sec:conclusion}

We introduced COMET\xspace, a novel metric-based meta-learning algorithm that learns to generalize along human-interpretable concept dimensions. We showed that COMET\xspace learns generalizable representations with incomplete, noisy, redundant, very few or a huge set of concept dimensions, selecting only important concepts for classification and providing reasoning behind the decisions. Our experimental results showed that COMET\xspace does not make a trade-off between interpretability and accuracy and significantly outperforms existing methods on tasks from diverse domains, including a novel benchmark dataset from the biology domain developed in our work.
\section*{Acknowledgements}
The authors thank Yusuf Roohani, Michihiro Yasunaga and Marinka Zitnik for their helpful comments. We gratefully acknowledge the support of DARPA under Nos. N660011924033 (MCS);
ARO under Nos. W911NF-16-1-0342 (MURI), W911NF-16-1-0171 (DURIP);
NSF under Nos. OAC-1835598 (CINES), OAC-1934578 (HDR), CCF-1918940 (Expeditions), IIS-2030477 (RAPID);
Stanford Data Science Initiative, 
Wu Tsai Neurosciences Institute,
Chan Zuckerberg Biohub,
Amazon, JPMorgan Chase, Docomo, Hitachi, JD.com, KDDI, NVIDIA, Dell, Toshiba, and UnitedHealth Group. 
J. L. is a Chan Zuckerberg Biohub investigator.

\section{Experimental setup}
\label{sec:impl_details}

\xhdr{Evaluation} We test all methods on the most broadly used $5$-way classification setting. In each episode, we randomly sample $5$ classes where each class contains $k$ examples as the support set in the $k$-shot classification task. We construct the query set to have $16$ examples, where each unlabeled sample in the query set belongs to one of the classes in the support set. We choose the best model according to the validation accuracy, and then evaluate it on the test set with novel classes. We report the mean accuracy by randomly sampling $600$ episodes in the fine-tuning or meta-testing stage.

On the CUB dataset, we followed the evaluation protocol in ~\citep{chen2019closer} and split the dataset into $100$ base, $50$ validation, and $50$ test classes in the exactly same split. On the Tabula Muris, we use $15$ organs for training, $4$ organs for validation, and $4$ organs for test, resulting into $59$ base, $47$ validation, and $37$ test classes corresponding to cell types. The 102 classes of Flowers dataset are split into 52, 25, 25 as the training, validation and testing set, respectively. As for Reuters dataset, we leave out 5 classes for validation and 5 for test.

\xhdr{Implementation details} On the CUB dataset, we use the widely adopted four-layer convolutional backbones Conv-4 with an input size of $84 \times 84$ ~\citep{snell17}. We perform standard data augmentation, including random crop, rotation, horizontal flipping and color jittering. We use the Adam optimizer ~\citep{kingma2014adam} with an initial learning rate of $10^{-3}$ and weight decay 0. We train the $5$-shot tasks for $40,000$ episodes and $1$-shot tasks for $60,000$ episodes ~\citep{chen2019closer}. To speed up training of COMET\xspace, we share the network parameters between concept learners.
In particular, we first forward the entire image $\vec x_i$ into the convolutional network and get a spatial feature embedding $f_{\vec \theta}(\vec x_i)$, and then get the $j$-th concept embedding as $ f_\vec{\theta}(\vec x_i)\circ \vec c^{(j)}$. Since convolutional filters only operate on pixels locally, in practice we get similar performance if we apply the mask at the beginning or at the end while significantly speeding up training time. In case the part is not annotatated (\textit{i.e.}, visible), we use the prototypical concept corresponding to whole image to replace the missing concept. For the Tabula Muris dataset, we use a simple backbone network structure containing two fully-connected layers with batch normalization, ReLu activation and dropout. We use Adam optimizer ~\citep{kingma2014adam} with an initial learning rate of $10^{-3}$ and weight decay $0$. We train the network for $1,000$ episodes. For MAML, RelationNet, MatchingNet, FineTune and ProtoNet, we use implementations from  \citep{chen2019closer}. For MetaOptNet and DeepEMD we use implementations from the respective papers.

\section{Ablation study on distance function}
\label{sec:distance}

We compare the effect of distance metric on the COMET\xspace's performance. We find that Euclidean distance consistently outperforms cosine distance in fine-grained image classification and cell type annotation tasks.

\begin{table}[h]
\centering
\caption{The effect of distance metric on COMET\xspace's performance.}
\label{tab:distance}
\begin{tabular}{lcccc}
\toprule
\textbf{}             & \multicolumn{2}{c}{\textbf{CUB}}    & \multicolumn{2}{c}{\textbf{Tabula Muris}}            \\
\small{\textbf{Distance}}             & \textbf{1-shot}    & \textbf{5-shot}  & \textbf{1-shot}    & \textbf{5-shot}  \\ \midrule
\small{\text{Cosine}} &$65.7 \pm 1.0$ & $82.2 \pm 0.6$ & $77.1 \pm 0.9$ & $90.1 \pm 0.6$ \\
\small{\textbf{Euclidean}} & {$\textbf{67.9} \pm \textbf{0.9}$} & {$\textbf{85.3} \pm \textbf{0.5}$} &  {$\textbf{79.4} \pm \textbf{0.9}$} & {$\textbf{91.7} \pm \textbf{0.5}$}  \\
\bottomrule
\end{tabular}
\end{table}

\section{Ablation study on backbone network}
\label{sec:backbone}

We compare performance of COMET\xspace to baselines methods using deeper Conv-6 backbone instead of Conv-4 backbone on the CUB dataset. We use part based annotations to define concepts. The results are reported in Table \ref{tab:backbone}. COMET\xspace outperforms all baselines even with deeper backbone. Additionally, by adding just one most frequent concept corresponding to a bird's beak on top of the whole image concept, COMET\xspace improves ProtoNet’s performance by $3.8\%$ on 1-shot task and $2.2\%$ on 5-shot task. 


\begin{table}[h]
\centering
\fontsize{9}{8}\selectfont
\caption{Peformance using Conv-6 backbone on CUB and Flowers dataset. We report average accuracy and standard deviation over 600 randomly sampled episodes. }
\label{tab:backbone}
\begin{tabular}{lcc}
\toprule
\textbf{}  & \multicolumn{2}{c}{\textbf{CUB}} \\
\small{\textbf{Method}}             & \textbf{1-shot}    & \textbf{5-shot} \\ \midrule
\small{\textbf{Finetune}}  & $66.0 \pm 0.9$ & $82.0 \pm 0.6$  \\  
\small{\textbf{MatchingNet}} & $66.5 \pm 0.9$ & $77.9 \pm 0.7$ \\
\small{\textbf{MAML}}  &  $66.3 \pm 1.1$ & $78.8 \pm 0.7$  \\
\small{\textbf{RelationNet}}    & $64.4 \pm 0.9$ & $80.2 \pm 0.6$  \\
\small{\textbf{MetaOptNet}}    & $65.5 \pm 1.2 $ & $83.0 \pm 0.8$  \\
\small{\textbf{DeepEMD}}    & $66.8 \pm 0.9 $ & $83.8 \pm 0.7$  \\
\midrule
\small{\textbf{ProtoNet}}    &  $66.4 \pm 1.0$ & $82.0 \pm 0.6$  \\
\small{\textbf{COMET\xspace - 1 concept}} & $68.9 \pm 0.9$ &  $83.8 \pm 0.6$  \\
\small{\textbf{COMET\xspace}} & {{$\textbf{72.2} \pm \textbf{0.9}$}} & {$\textbf{87.6} \pm \textbf{0.5}$}  \\
\bottomrule
\end{tabular}
\vspace{-1em}
\end{table}

\section{Ablation study on ensemble methods}
\label{sec:ensemble}

We compare COMET\xspace to the ensemble of prototypical networks. We train ProtoNets in parallel and combine their outputs by majority voting as typically done in ensemble models. In particular, given a query point $\vec x_q$ and prototypes $\{\vec p_{k}^{(j)}\}_k$ , the prototypical ensemble outputs probability distribution for each ProtoNet model $j$: 

\begin{equation} \label{eq_softmax}
    p^{(j)}_\vec\theta(y=k|\vec x_q)=\frac{\exp(- d(f_\vec\theta^{(j)}(\vec x_q),\vec p_k^{(j)}))}{\sum_{k'} \exp(- d(f_\vec\theta^{(j)}(\vec x_q),\vec p_{k'}^{(j)}))}.
\end{equation}

On the CUB dataset, we use $5$ ProtoNets. We use smaller number than the number of concepts because  training an ensemble of a larger number of ProtoNets on CUB results in memory issues due to the unshared weights. On the Tabula Muris and Reuters datasets we use the same number of ProtoNets as the number of concepts, that is $190$ on Tabula Muris and $126$ on Reuters.


\begin{comment}
\begin{table}[h]
\centering
\caption{Comparison to ensemble of prototypical networks and COMET\xspace with shared weights across concepts.}
\label{tab:ensemble}
\begin{tabular}{lcccccc}
\toprule
\textbf{}             & \multicolumn{2}{c}{\textbf{CUB}}    & \multicolumn{2}{c}{\textbf{Tabula Muris}}  & \multicolumn{2}{c}{\textbf{Reuters}}              \\
\small{\textbf{Method}}             & \textbf{1-shot}    & \textbf{5-shot}  & \textbf{1-shot}    & \textbf{5-shot}  & \textbf{1-shot}    & \textbf{5-shot}  \\ \midrule
\small{\text{ProtoEns}} & $64.0 \pm 0.8$ & $82.3 \pm 0.5$ & $67.2 \pm 0.8$ & $83.6 \pm 0.5$ & $62.4 \pm 0.7$ & $79.3 \pm 0.4$ \\
\small{\text{COMET\xspace shared w}} & {$67.9\pm 0.9$} & {$85.3 \pm 0.5$}  & $78.2 \pm 1.0$ & $91.0 \pm 0.5$ & $69.8 \pm 0.8$ & $88.6 \pm 0.3$\\
\small{{COMET\xspace}} & {$67.9\pm 0.9$} & {$85.3 \pm 0.5$} &  {$79.4 \pm 0.9$} & {$91.7 \pm 0.5$} & {$71.5 \pm 0.7$} &  ${89.8 \pm 0.3}$ \\
\bottomrule
\end{tabular}
\end{table}
\end{comment}

\begin{comment}
\section{Ablation study on aggregation}
\label{sec:aggregation}

In COMET\xspace, we combine concept learners by aggregating distances across all concept prototypes in the softmax function. In this section, we compare our aggregation approach with the majority voting in which each concept learner votes on the final prediction. In particular, we design a following baseline method. Given a query point $\vec x_q$, concept prototypes $\{\vec p_{k}^{(j)}\}_k$ and concept embeddings $f_\vec\theta^{(j)}(\vec x_q\circ \vec c^{(j)})$ obtained using the COMET\xspace, the method outputs probability distribution for each concept $j$: 

\begin{equation} \label{eq_softmax}
    p^{(j)}_\vec\theta(y=k|\vec x_q)=\frac{\exp(- d(f_\vec\theta^{(j)}(\vec x_q\circ \vec c^{(j)}),\vec p_k^{(j)}))}{\sum_{k'} \exp(- d(f_\vec\theta^{(j)}(\vec x_q\circ \vec c^{(j)}),\vec p_{k'}^{(j)}))}.
\end{equation}

Each concept then outputs its own class prediction, and we combine predictions across all concepts with the majority voting. The results shown in Table~\ref{tab:aggregation} demonstrate that the aggregation used in COMET\xspace significantly outperforms majority voting on both domains.
\kaidi{Can we add Reuters here for completeness?}

\begin{table}[h]
\centering
\caption{Comparison between aggregation approach used in COMET\xspace and majority voting approach in which each concept votes on the final prediction.}
\label{tab:aggregation}
\begin{tabular}{lcccccc}
\toprule
\textbf{}             & \multicolumn{2}{c}{\textbf{CUB}}    & \multicolumn{2}{c}{\textbf{Tabula Muris}}  & \multicolumn{2}{c}{\textbf{Reuters}}              \\
\small{\textbf{Method}}             & \textbf{1-shot}    & \textbf{5-shot}  & \textbf{1-shot}    & \textbf{5-shot}& \textbf{1-shot}    & \textbf{5-shot}  \\ \midrule
\small{\text{Majority voting}} & $64.0 \pm 0.9$ & $82.1 \pm 0.6$ & $75.6 \pm 0.8$ & $88.2 \pm 0.5$  & {$65.3 \pm 0.7$} &  ${80.2 \pm 0.4}$ \\
\small{\textbf{COMET\xspace's aggre}} & {$\textbf{67.9} \pm \textbf{0.9}$} & {$\textbf{85.3} \pm \textbf{0.5}$} &  {$\textbf{79.4} \pm \textbf{0.9}$} & {$\textbf{91.7} \pm \textbf{0.5}$} & {$\textbf{71.5} \pm \textbf{0.7}$} &  ${\textbf{89.8} \pm \textbf{0.3}}$ \\
\bottomrule
\end{tabular}
\end{table}
\end{comment}

\section{Unsupervised concept annotation: additional results}
\label{sec:unsup_results}

We evaluate COMET\xspace and baseline methods on the Flowers dataset for fine-grained image classification. We automatically extract concepts using unsupervised landmarks discovery approach ~\citep{zhang18}. Results in Table ~\ref{tab:flower} show that COMET\xspace outperforms all baselines by a large margin.

\begin{table}[h]
\centering
\fontsize{9}{9}\selectfont
\caption{Results on 1-shot and 5-shot classification on the Flowers dataset. We report average accuracy and standard deviation over $600$ randomly sampled episodes. }

\label{tab:flower}
\begin{tabular}{lcc}
\toprule
\textbf{}             & \multicolumn{2}{c}{\textbf{Flowers}}  \\
\small{\textbf{Method}}             & \textbf{1-shot}    & \textbf{5-shot}    \\ \midrule
\small{\textbf{Finetune}} & $65.4 \pm 0.9$ & $81.9 \pm 0.7$ \\  
\small{\textbf{MatchingNet}} & $66.0 \pm 0.9$ & $82.0 \pm 0.8$ \\
\small{\textbf{MAML}} & $63.2 \pm 1.1$ & $76.6 \pm 0.8$   \\
\small{\textbf{RelationNet}}  & $66.4 \pm 0.9$ & $80.8 \pm 0.6$  \\ 
\small{\textbf{MetaOptNet}} & $64.8 \pm 1.0$ & $81.3 \pm 0.7$ \\
\small{\textbf{DeepEMD}} & $67.2 \pm 0.9$ & $82.9 \pm 0.7$ \\
\midrule
\small{\textbf{ProtoNet}} & $64.4 \pm 1.0$ & $80.2 \pm 0.8$  \\
\small{\textbf{COMET\xspace}} & $\textbf{70.4} \pm \textbf{0.9}$ & $ \textbf{86.7} \pm \textbf{0.6}$ \\
\bottomrule
\end{tabular}
\vspace{-1em}
\end{table}

On the Tabula Muris and Reuters datasets, we test COMET\xspace without any prior knowledge by defining concepts using selected random masks. In particular, we randomly select subsets of features as concepts and then use validation set to select the concepts with the highest importance scores as defined by COMET\xspace. We use same number of concepts used in Tabula Muris and Reuters datasets. Results are reported in Table \ref{tab:prior}.

\begin{table}[h]
\centering
\fontsize{9}{8}\selectfont
\caption{Results on 1-shot and 5-shot classification on Tabula Muris and Retuters dataset with selected random masks as concepts and human-defined concepts. We report average accuracy and standard deviation over $600$ randomly sampled episodes.}
\label{tab:prior}
\begin{tabular}{lcccc}
\toprule
\textbf{}  & \multicolumn{2}{c}{\textbf{Tabula Muris}}        &  \multicolumn{2}{c}{\textbf{Reuters}}            \\
\small{\textbf{Method}}             & \textbf{1-shot}    & \textbf{5-shot}    & \textbf{1-shot}    & \textbf{5-shot}  \\ \midrule
\small{with selected random masks}    & $77.2 \pm 1.0$ & $89.8 \pm 0.5$ & $70.1 \pm 0.9$ & $89.0 \pm 0.4$ \\
\small{with prior knowledge} & {$\textbf{79.4} \pm \textbf{0.9}$} & {$\textbf{91.7} \pm \textbf{0.5}$}  & $\textbf{71.5} \pm \textbf{0.7}$ & \textbf{ $\textbf{89.8} \pm \textbf{0.3}$} \\
\bottomrule
\end{tabular}
\vspace{-1em}
\end{table}

\section{Unsupervised concept annotation: landmarks examples}
\label{sec:unsup_key_pts_example}

To assess the performance of COMET\xspace using automatically extracted visual concepts on the CUB dataset, we applied autoencoding framework for landmarks discovery proposed in ~\citep{zhang18}. We use default parameters and implementation provided by the authors, and set the number of landmarks to $30$. The encoding module provides coordinates of the estimated landmarks. To create concept mask, we create a bounding box around discovered landmarks. We train the autoencoder using same parameters as ~\citet{zhang18}, and set the number of concepts to $30$. Examples of extracted landmarks for $20$ images from the CUB dataset are visualized in Figure~\ref{fig:unsup_key_pts}.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.9\textwidth]{figs/unsup_key_pts.pdf}
    \caption{Examples of automatically extracted landmarks using ~\citep{zhang18} on the CUB dataset. }
    \label{fig:unsup_key_pts}
\end{figure}

\section{Interpretability: local explanations}
\label{sec:local_exp}

Here, we demonstrate COMET\xspace's local explanations on the CUB dataset. Given a single query data point, COMET\xspace assigns local concept importance scores to each concept based on the distance between concept embedding of the query data point to the prototypical concept. We then rank concepts according to their local concept importance scores. Figure~\ref{fig:local_exp} shows examples of ranked concepts. Importance scores assigned by COMET\xspace visually reflect well the most relevant bird features.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.95\textwidth]{figs/local_exp.pdf}
    \caption{Examples of COMET\xspace's local explanations on the CUB dataset. Concepts are ranked according to the highest local concept similarity scores. Qualitatively, local importance scores correctly reflect the most relevant bird features.}
    \label{fig:local_exp}
\end{figure}

\section{Interpretability: local similarity}
\label{sec:concept_rank_incls}

Given fixed concept of interest, we apply COMET\xspace to sort images with respect to the distance of their concept embedding to the concept prototype. Figure~\ref{fig:concept_rank_incls} shows example of chipping sparrow images with the belly concept embedding most similar to the prototypical belly, and images with the belly concept embedding most distant to the prototypical belly. Most similar images indeed have clearly visible belly part and reflect prototypical belly well. On the contrary, most distant images have only small part of belly visible, indicating that COMET\xspace can be used to detect misannotated or non-visible concepts.

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{figs/concept_rank_incls.pdf}
    \caption{Images ranked according to the distance of their belly concept embedding to the belly concept prototype. Most similar images (top) and most distant images (bottom). Images closest to the prototype have clearly visible belly part that visually looks like prototypical belly of a chipping sparrow, whereas most distant images do not have belly part clearly visible.}
    \label{fig:concept_rank_incls}
\end{figure}



