\section{Introduction}

We propose a method for explaining classes in text classification tasks using deep learning models and feature attribution techniques, such as the Integrated Gradients (IG) method introduced by \citet{sundararajan2017axiomatic}. We focus specifically on IG as it provides a general framework for estimating feature importance in deep neural networks and has been shown to provide reliable saliency maps in text classification tasks among others \cite{bastings2020elephant,kokhlikyan2020captum}. 

Recently, explaining the predictions of deep neural networks has attracted a considerable amount of research interest in fields such as NLP and computer vision. Given the importance of this endeavour, several different techniques have been suggested in order to interpret model predictions \cite[see][for recent discussion]{montavon2018}. Nevertheless, these tend to focus on explaining individual predictions rather than how models perceive whole classes. To the best of our knowledge, we present the first method for aggregating explanations of individual examples in text classification to general descriptions of the classes. The method consists of three steps: 1) repeated model training and application of IG on random train/test splits, 2) aggregation of word scores of individual examples and extraction of keywords, and 3) filtering to remove spurious keywords.

We test this method by training Transformer-based text classifiers on a large Web register identification corpus and show that it is able to provide descriptive keywords for the classes. The class descriptions provide both linguistic insight and a means for analyzing and debugging neural classification models in text classification.




\section{Data and classifier} 

In our experiments, we focus on text classification using the Corpus of Online Registers of English (CORE) \cite{egbert:core}, a large-scale collection of Web texts annotated for their \textit{register} (genre) \cite{biber88}. The CORE registers are coded using a two-level taxonomy. In this study, we focus on the upper level which consists of eight register classes: Narrative (NA), Opinion (OP), How-to (HI), Interactive discussion (ID), Informational description (IN), Lyrical (LY), Spoken (SP) and Informational persuasion (IP). The dataset features the full range of registers found on the unrestricted open Web and consists of nearly 50,000 texts. In our experiments, we combine the train and development sets, totaling 38,760 documents. 

Web registers have been frequently studied in recent research both in linguistics and NLP \cite{tiktak2013,dayter-reddit-2021, vidulin2019,biber-egbert-registervariationonline2019}. 
The range of linguistic variation has, however, caused challenges for both fields, and, in particular, Web register identification studies have lacked robustness \cite{sharoff-etal-2010-web, Petrenz2011}. The method we propose in this study can benefit both fields as it provides insight about classification models and the corpora they are trained on, including potential biases.

As a classifier, we use the XLM-R deep language model \cite{conneau-etal-2020-unsupervised} because of its strong ability to model multiple languages, both in monolingual and cross-lingual settings. We use the base size, since it uses less resources and its predictive performance on the CORE corpus is competitive with XLM-R large \cite{repo2021zeroshot}.
The task is modeled as a multilabel classification task.





\section{Method}
The descriptions of classes are extracted through the following steps:

\textbf{Step 1: Train and explain.} We combine the training and development sets of the corpus and randomly split them into a new training and validation set according to a set ratio $r$, using stratification to keep class distributions stable \cite[cf.][]{Laippala2021}.
The pre-trained language model is loaded and the decision layer (a sequence regression head) is randomly initialized. Both are fine-tuned on the new training set. Text examples in the validation set are classified and the IG method is applied in order to obtain attribution scores for the network inputs, i.e., each dimension of each input token embedding, w.r.t. each predicted class $c$. The embedding dimensions are summed up per token to provide a token-level score and all tokens in a document $d$ are normalized by the L$_2$ norm. This provides a word attribution score $s_{w,d,c}$ directly if the word $w$ consists of a single token, otherwise it is calculated as the maximum of all sub-word token scores. 

\textbf{Step 2: Aggregate attributions.} We calculate the average attribution scores $\bar{s}_{w,c}$, for each $(w,c)$, as a means for ranking of keywords per class. In order to reduce noise, we only select the $n$ top-scoring words per document $d$, and we only consider true positive predictions. We note that the method could alternatively be used for error analysis by targeting false predictions.

\textbf{Step 3: Select stable keywords.} The above process is repeated $N$ times, each time randomly shuffling and splitting the data according to Step 1, in order to quantify the stability of the keywords.
The keyword candidates ranked by $\bar{s}_{w,c}$ are filtered based on selection frequency: a word is considered stable if the ratio by which it is selected (in Step 2) across the experiments is larger than a threshold value $t$.
We also ignore words that occur in $k$ documents or less in the corpus.

The selection frequency filtering allows us to remove keywords that are unstable across runs, likely reflecting spurious features, for instance, resulting from an unrepresentative split of the data or stochastic factors in the training of the classifier itself. \citet{mccoy2020berts} show in repeated experiments on a text inference task with random initialization of the decision layer and randomized order of training examples that, while consistent test set performance was achieved, the degree of generalization as measured on a related task varied significantly. Similarly, we test the persistence and presumed generalizability of the estimated keywords by considering the randomness both in training and in data selection.

In our experiments, we have used the parameters $r=0.67$, $n=20$, $N=100$, $t=0.6$ and $k=5$.



\section{Results}

The classifiers trained in our 100 experiments achieved a mean micro average F1-score of 65.10\% ($SD=6.72\%$) and mean class-wise F1-scores in the range 26.45\%--82.92\% for the eight main register classes (see Table~\ref{tab:pred_scores} in Appendix). The Spoken (SP) class stands out as a particularly difficult case where performance was particularly unstable ($SD=27.09\%$), partly due to its small size.

Our method was able to produce descriptive keywords that clearly reflect our understanding of all the main classes (see Table~\ref{tab:keywords} in Appendix) except for the Spoken class, where no keyword surpassed the selection frequency threshold. The keywords reflect both topical and functional features typical of the registers. For instance, the highest scoring words for Interactive discussion (ID) were \textit{question, faq, forum, answer}.
Similarly, we observe other register-specific linguistic characteristics, such as words associated with research papers in Informational description (IN) and with news in Narrative (NA). The keywords also share many similarities with keywords produced with other methods applied in previous studies \cite[e.g.,][]{biber-egbert-registervariationonline2019,Laippala2021}.

Furthermore, the estimated keywords display a strong discriminative power as indicated by their uniqueness in the respective register classes.
On average, 82 ($SD=4.6$) of the top 100 keywords for a given register were not shared with the other registers demonstrating that the method was able to identify register-specific keywords. Moreover, the selection frequency of the keywords across the 100 rounds demonstrated their stability -- they are consistently identified, often in over 90\% of the repetitions. 






\section{Conclusion}
We have proposed a method for describing classes in a text classification task based on IG attributions on predictions and shown that it produces stable and interpretable results for Web register classification with XLM-R. We see the method as generally applicable and useful for studying text classes also beyond registers.
In the future, we seek to extend the method and its evaluation, and apply the approach to other languages and cross-lingual settings. In particular, the comparison of monolingual and zero-shot models will be informative of both the linguistic characteristics of registers and what models such as XLM-R learn to recognize.


