\section{Introduction}

\begin{figure}[t!]
\centering
\includegraphics[width=0.45\textwidth]{intro_example_new.png}
\caption{
Interpretation for the bird-classification models using saliency maps generated by LRP ({\it layer-wise relevance propagation}) and our model PPI.
LRP shows that naively trained deep model makes decisions based on the background cues (habitat, {\it e.g.}, rocks, bulrushes) that are spuriously correlated with the bird species, while our causally informed PPI mostly focuses on the bird anatomy, that generalizes beyond the natural habitat.}
\label{fig:examples_intro}
\vspace{-1em}
\end{figure}

Deep neural networks hold great promise in applications requiring the analysis and comprehension of complex imagery.
Recent advances in hardware, network architectures, and model optimization, along with the increasing availability of large-scale annotated datasets~\cite{krizhevsky2009learning,deng2012mnist,deng2009imagenet}, have enabled these models to match and sometimes outperform human experts on a number of tasks, including natural image classification~\cite{krizhevsky2017imagenet}, objection recognition~\cite{girshick2014rich}, disease diagnosis~\cite{sajda2006machine}, and autonomous driving~\cite{chen2015deepdriving}, among others.

While deep learning solutions have been positively recognized for their ability to learn {\em black-box} models in a purely data driven manner, their very nature makes them less credible for their inability to communicate the reasoning for making predictions in a way that is comprehensible to humans~\cite{hooker2019benchmark, rebuffi2020there}. This denies consequential applications where the reliability and trustworthiness of a prediction are of primary concern and require expert audit, {\it e.g.}, in healthcare~\cite{sajda2006machine}.
To stimulate widespread use of deep learning models, a means of interpreting predictions is necessary. However, model interpretation techniques often reveal a concerning fact, that deep learning models tend to assimilate spurious correlations that do not necessarily capture the causal relationship between the input (image) and output (label)~\cite{wang2020score}. This issue is particularly notable in small-sample-size (weak supervision) scenarios or when the sources of non-informative variation are overwhelming, thus likely to cause severe overfitting.
These can lead to catastrophic failures on deployment~\cite{fukui2019attention, wang2019sharpen}.

A growing recognition of the issues associated with the lack of interpretable predictions is well documented in recent years \cite{adebayo2018sanity, hooker2019benchmark, rebuffi2020there}.
Such phenomenon has energized researchers to actively seek creative solutions.
Among these, two streams of work, namely {\it saliency mapping}~\cite{zhao2018uniqueness, simonyan2013deep, dabkowski2017real} and {\it causal representation learning} (CRL)~\cite{johansson2016learning, wang2020visual, arjovsky2019invariant}, stand out as some of the most promising directions.
Specifically, saliency mapping encompasses techniques for {\it post hoc} visualizations on the input (image) space to facilitate the interpretation of model predictions.
This is done by projecting the key features used in prediction back to the input space, resulting in the commonly known {\it saliency maps}.
Importantly, these maps do not directly contribute to model learning.
Alternatively, CRL solutions are built on the principles of establishing invariance from the data, and it entails teasing out sources of variation that are spuriously associated with the model output (labels).
CRL models, while emphasizing the differences between causation and correlation, are not subject to the rigor of causal inference approaches, because their goal is not to obtain accurate causal effect estimates but rather to produce robust models with better generalization ability relative to their naively learned counterparts \cite{arjovsky2019invariant}.

In this work, we present {\it Proactive Pseudo-Intervention} (PPI), a solution that accounts for the needs of causal representation identification and visual verification.
Our key insight is the derivation of causally-informed saliency maps, which facilitate visual verification of model predictions {\em and} enable learning that is robust to (non-causal) associations.
While true causation can only be established through experimental interventions, we leverage tools from contrastive representation learning to synthesize pseudo-interventions from observational data. Our procedure is motivated by the causal argument: perturbing the non-causal features will not change the target label. 

To motivate, in Figure \ref{fig:examples_intro} we present an example to illustrate the benefits of producing causally-informed saliency maps.
In this scenario, the task is to classify two bird species (A and B) in the wild.
Due to the differences in their natural habitats, A-birds are mostly seen resting on rocks, while B-birds are more commonly found among bulrushes.
A deep model, trained naively, will tend to associate the background characteristics with the labels, knowing these strongly correlate with the bird species (labels) in the training set.
This is confirmed by the saliency maps derived from the layer-wise relevance propagation (LRP) techniques \cite{bach2015pixel}: the model also attends heavily on the background features, while the difference in bird anatomy is what causally determines the label.
If we were provided with an image of a bird in an environment foreign to the images in the training set, the model will be unable to make a reliable prediction, thus causing robustness concerns.
This generalization issue worsens with a smaller training sample size.
On the other hand, saliency maps from our PPI-enhanced model successfully focus on the bird anatomy, and thus will be robust to environmental changes captured in the input images.

PPI addresses causally-informed reasoning, robust learning, and model interpretation in a unified framework. 
A new saliency mapping method, named {\it Weight Back Propagation} (WBP), is also proposed to generate more concentrated intervention mask for PPI training.
The key contributions of this paper include:
\begin{itemize}
    \vspace{-5pt}
    \item An end-to-end contrastive representation learning strategy PPI that employs proactive interventions to identify causally relevant features.
    \vspace{-5pt}

    \item A fast and architecture-agnostic saliency mapping module WBP that delivers better visualization and localization performance
    \vspace{-5pt}
    
    \item Experiments demonstrating significant performance boosts from integrating PPI and WBP relative to competing solutions, especially on  out-of-domain predictions, data integration with heterogeneous sources and model interpretation.
\end{itemize}


\section{Background}
\noindent{\bf Visual Explanations}
Saliency mapping collectively refers to a family of techniques to understand and interpret black-box image classification models, such as deep neural networks~\cite{adebayo2018sanity, hooker2019benchmark, rebuffi2020there}.
These methods project the model understanding of the targets, {\it i.e.}, labels, and their predictions back to the input space, which allows for the visual inspection of automated reasoning and for the communication of predictive visual cues to the user or human expert, aiming to shed model insights or to build trust for deep-learning-based systems. 

In this study, we focus on {\it post hoc} saliency mapping strategies, where saliency maps are constructed given an arbitrary prediction model, as opposed to relying on customized model architectures for interpretable predictions~\cite{fukui2019attention,wang2019sharpen}, or to train a separate module to explicitly produce model explanations \cite{fukui2019attention, goyal2019counterfactual, chang2018explaining,  fong2017interpretable, shrikumar2017learning}.
Popular solutions under this category include: activation mapping~\cite{zhou2016learning,selvaraju2017grad}, input sensitivity analysis~\cite{shrikumar2017learning}, and relevance propagation \cite{bach2015pixel}. 
Activation mapping based methods fail at visualizing fine-grained evidence, which is particularly important in explaining medical classification models~\cite{du2018towards, selvaraju2017grad, wagner2019interpretable}. Input sensitivity analysis based methods produce fine-grained saliency maps. However, these maps are generally less concentrated~\cite{dabkowski2017real,fong2017interpretable} and less interpretable. 
Relevance propagation based methods, like LRP and its variants, use complex rules to prioritize positive or large relevance, making the saliency maps visually appealing to human. However, our experiments demonstrate that LRP and its variants highlight spuriously correlated features (boarderlines and backgrounds). 
By contrast, our WBP backpropagates the weights through layers to compute the contributions of each input pixel, which is truly faithful to the model, and WBP tends to highlight the target objects themselves rather than the background. At the same time, the simplicity and efficiency makes WBP easily work with other advanced learning strategies for both model diagnosis and improvements during training. 

Our work is in a similar spirit to \cite{fong2017interpretable, dabkowski2017real, chang2018explaining, wagner2019interpretable}, where meaningful perturbations have been applied to the image during model training, to improve prediction and facilitate interpretation. Poineering works have relied on user supplied ``ground-truth'' explainable masks to perturb \cite{ross2017right,li2018tell, rieger2020interpretations}, however such manual annotations are costly and hence rarely available in practice. Alternatively, perturbations can be computed by solving an optimization for each image. Such strategies are costly in practice and also do not effectively block spurious features.  Very recently, exploratory effort has been made to leverage the tools from counterfactual reasoning \cite{goyal2019counterfactual} and causal analysis \cite{o2020generative} to derive visual explanations, but do not lend insights back to model training. Our work represents a fast, principled solution that overcomes the above limitations. It automatically derives explainable masks faithful to the model and data, without explicit supervision from user-generated explanations.



\vspace{2mm}
\noindent{\bf Contrastive Learning. }
There has been growing interest in exploiting contrastive learning (CL) techniques for representations learning \cite{oord2018representation,chen2020simple,he2020momentum,khosla2020supervised,tian2019contrastive}.
Originally devised for density estimation \cite{gutmann2010noise}, CL exploits the idea of {\em learning by comparison} to capture the subtle features of data, {\it i.e.}, positive examples, by contrasting them with negative examples drawn from a carefully crafted noise distribution.
These techniques aim to avoid representation collapse, or to promote representation consistency, for downstream tasks.
Recent developments, both empirical and theoretical, have connected CL to information-theoretic foundations~\cite{tian2019contrastive,grill2020bootstrap}, thus establishing them as a suite of {\it de facto} solutions for unsupervised representation learning \cite{chen2020simple,he2020momentum}. 

The basic form of CL is essentially a binary classification task specified to discriminate positive and negative examples.
In such a scenario, the binary classifier is known as the critic function.
Maximizing the discriminative power wrt the critic and the representation sharpens the feature encoder. 
Critical to the success of CL is the choice of appropriate noise distribution, where the challenging negatives, {\it i.e.}, those negatives that are more similar to positive examples, are often considered more effective contrasts. 
In its more generalized form, CL can naturally repurpose the predictor and loss functions without introducing a new critic~\cite{tian2019contrastive}.
Notably, current CL methods are not immune to spurious associations, a point we wish to improve in this work.

\vspace{2mm}

\noindent{\bf Causality and Interventions.} From a causality perspective, humans learn via actively interacting with the environment. We intervene and observe changes in the outcome to infer causal dependencies. Machines instead learn from static observations that are unable to inform the structural dependencies for causal decisions. As such, perturbations to the external factors, {\it e.g.}, surroundings, lighting, viewing angles, may drastically alter machine predictions, while human recognition is less susceptible to such nuisance variations.
Formally, such difference is best explained with the $do$-notation~\cite{pearl2009causality}: $\mathbb{P}(Y|do(x)) = \sum_z \mathbb{P}(Y|X=x, z) \mathbb{P}(z)$, where we identify $x$ as the features, {\it e.g.}, an object in the image, and $z$ as the confounders, {\it e.g.}, background in the example above.
Note that $\mathbb{P}(Y|do(x))$ is fundamentally different from the conditional likelihood $\mathbb{P}(Y|X=x) = \sum_z \mathbb{P}(Y|X=x,z) \mathbb{P}(z|X=x)$, which machine uses for associative reasoning.

Unfortunately, carrying out real interventional studies, {\it i.e.}, randomized control trials, to intentionally block non-causal associations, is oftentimes not a feasible option for practical considerations, {\it e.g.}, due to cost and ethics.
This work instead advocates the application of synthetic interventions to uncover the underlying causal features from observational data.
Specifically, we proactively edit $x$ and its corresponding label $y$ in a data-driven fashion to encourage the model to learn potential causal associations. Our proposal is in line with the growing appreciation for the significance of establishing causality in machine learning models \cite{scholkopf2019causality}. Via promoting invariance \cite{arjovsky2019invariant}, such causally inspired solutions demonstrate superior robustness to superficial features that do not generalize \cite{wang2019learning}. In particular, \cite{suter2019robustly, zhang2020causal} showed the importance and effectiveness of accounting for interventional perspectives. Our work brings these causal views to construct a simple solution that explicitly optimizes visual interpretation and model robustness. 

\begin{figure*}[h!]
    \centering
    \includegraphics[width=0.98\textwidth]{learning_framework_mar17-module.png
    \caption{Illustration of the proposed PPI learning strategy. 
    Input images are intervened by removing the saliency map based masks, which alters the input label ({\it e.g.}, negative control). For positive contrast, we use the original input as well as an input masked with a random slaiency map. We use WBP for the generation of saliency maps. 
   
    }
    \label{fig:learning_strategy}
    \vspace{-1em}
\end{figure*}

\vspace{-3pt}
\section{Proactive Pseudo-Intervention}
Below we describe the construction of {\it Proactive Pseudo-Intervention} (PPI), a causally-informed contrastive learning scheme that seeks to simultaneously improve the accuracy, robustness, generalization and interpretability of deep-learning-based computer vision models. 

The PPI learning strategy, schematically summarized in Figure ~\ref{fig:learning_strategy}, consists of three main components: ($i$) a saliency mapping module that highlights causally relevant features; ($ii$) an intervention module that synthesizes contrastive samples; and ($iii$) the prediction module, which is standard in recent vision models, {\it e.g.}, VGG~\cite{simonyan2014very}, ResNet~\cite{he2016deep}, and Inception Net~\cite{szegedy2016rethinking}. 
Motivated by the discussions from our introduction, PPI establishes a feedback loop between the saliency map module and the prediction module, which is interfaced by the synthesized contrastive examples in the intervention module.
Under this configuration, the prediction module is encouraged to modify its predictions only when provided with causally-relevant synthetic interventions.
Note that components ($i$) and ($ii$) do not involve any additional parameters or neural network modules, which makes our strategy readily applicable to the training of virtually any computer vision task without major customization.
Details of these building blocks are given below.

\vspace{-2pt}
\subsection{Synthetic causal interventions for contrasts}
Key to our formulation is the design of a synthetic intervention strategy that generates contrastive examples to reinforce causal relevance during model training.
Given a causal saliency map ${\boldsymbol s}_m({\boldsymbol x})$ for an input ${\boldsymbol x}$ wrt label $y=m$, where $m=1,\ldots,M$, and $M$ is the number of classes, the synthetic intervention consists of removing (replacing with zero) the causal information from ${\boldsymbol x}$ contained in ${\boldsymbol s}_m({\boldsymbol x})$, and then using it as the contrastive learning signal.

For now, let us assume the causal salience map ${\boldsymbol s}_m({\boldsymbol x})$ is known; the procedure to obtain the saliency map will be addressed in the next section.
For notational clarity, we use subscript $i$ to denote entities associated with the $i$-th training sample, and omit the dependency on learnable parameters.
To remove causal information from ${\boldsymbol x}_i$ and obtain a negative contrast ${\boldsymbol x}_i^{*}$, we apply the following {\it soft-masking} 
\begin{equation}\label{eq:mask}
    {\boldsymbol x}_i^{*} = {\boldsymbol x}_i - T({\boldsymbol s}_m({\boldsymbol x}_i)) \odot {\boldsymbol x}_i,
\end{equation}
where $T(\cdot)$ is a differentiable masking function and $\odot$ denotes element-wise (Hadamard) multiplication.
Specifically, we use the thresholded sigmoid for masking:
\begin{equation}\label{eq:sig}
    T({\boldsymbol s}_m({\boldsymbol x}_i)) = \frac{1}{1+\exp(-\omega ({\boldsymbol s}_m({\boldsymbol x}_i) - \mathbf{\sigma}))}, 
\end{equation}
where $\sigma$ and $\omega>0$ are the threshold and scaling parameters, respectively.
We set the scaling $\omega$ so that $T({\boldsymbol s})$ will result in a sharp transition from $0$ to $1$ near $\sigma$.
Using \eqref{eq:mask} we define the contrastive loss as
\begin{equation}\label{loss_fn1}
    L_{con}(\theta) = \sum_i \ell ({\boldsymbol x}_i^{*}, \neg y; f_{\theta}),
\end{equation}
where $f_{\theta}$ is the prediction module, $\ell({\boldsymbol x}, y; f_{\theta})$ is the loss function we wish to optimize, {\it e.g.} cross entropy, and $\neg$ is used to denote that the original class label has been flipped.
In the binary case, $\neg y= 1-y$, and in the multi-class case it can be interpreted accordingly, {\it e.g.}, using a one {\it vs}. others cross entropy loss. 
In practice, we set $\ell({\boldsymbol x}, y; f_{\theta}) = - \ell({\boldsymbol x}, y; f_{\theta})$.
We will show in the experiments 
that this simple and intuitive causal masking strategy works well in practice (see Tables  \ref{tab:exp_ga} and \ref{tab:exp_lidc}, and Figure~\ref{fig:exp_ga_1}). 
Alternatively, we also consider a {\it hard-masking} approach in which a minimal bounding box covering the thresholded saliency map is removed.
See the Appendix for details.

Note that we are making the implicit assumption that the saliency map is uniquely determined by the prediction module $f_{\theta}$.
While optimizing \eqref{loss_fn1} explicitly attempts to improve the fit of the prediction module $f_{\theta}$, it also implicitly informs the causal saliency mapping.
This is sensible because if a prediction is made using non-causal features, which implies the associated saliency map ${\boldsymbol s}_m({\boldsymbol x})$ is also non-causal, then we should expect that after applying ${\boldsymbol s}_m({\boldsymbol x})$ to ${\boldsymbol x}$ using \eqref{eq:mask}, we can still expect to make the correct prediction, {\it i.e.}, the true label, for both positive (the original) and negative (the intervened) samples.


\vspace{1mm}
\noindent {\bf Saliency map regularization.}
Note that naively optimizing \eqref{loss_fn1} can lead to degenerate solutions for which any saliency map that satisfies the causal sufficiency, {\it i.e.}, encompassing all causal features, is a valid causal saliency map.
For example, a trivial solution where the saliency map covers the entire image may be considered causal.
To protect against such degeneracy, we propose to regularize the $L_1$-norm of the saliency map to encourage succinct (sparse) representations, {\it i.e.}, $L_{reg} = \| {\boldsymbol s}_m \|_1$, for $m=1,\ldots,M$.

\vspace{1mm}
\noindent {\bf Adversarial positive contrasts.}
Another concern with solely optimizing \eqref{loss_fn1} is that models can easily overfit to the intervention, {\it i.e.}, instead of learning to capture causal relevance, the model learns to predict interventional operations.
For example, the model can learn to change its prediction when it detects that the input has been intervened, regardless of whether the image is missing causal features.
So motivated, we introduce adversarial positive contrasts:
\begin{equation}\label{eq:random}
    {\boldsymbol x}'_i = {\boldsymbol x}_i - T({\boldsymbol s}_m({\boldsymbol x}_j)) \odot {\boldsymbol x}_i, \quad i \neq j, 
\end{equation}
where we intervene with a {\em false} saliency map, {\it i.e.}, ${\boldsymbol s}_m({\boldsymbol x}_j)$ is the saliency map from a different input ${\boldsymbol x}_j$, while still encouraging the model to make the correct prediction via
\begin{equation}\label{loss_fn2}
    L_{ad}(\theta) = \sum_i \ell ({\boldsymbol x}_i', y ; f_{\theta})\,,
    \vspace{-3mm}
\end{equation}
where ${\boldsymbol x}'_i$ is the adversarial positive contrast. 
The complete loss for the proposed model, $L = L_{cls} + L_{con} + L_{reg} + L_{ad}$, consists of the contrastive loss in \eqref{loss_fn1}, the regularization loss, $L_{reg}$, and the adversarial loss in \eqref{loss_fn2}.

\subsection{Saliency Weight Backpropagation}
In order to generate saliency maps that inform decision-driving features in the (raw) pixel space, we describe Weight Back Propagation (WBP), a novel computationally efficient scheme for saliency mapping applicable to arbitrary neural architectures. WBP evaluates individual contributions from each pixel to the final class-specific prediction, and we empirically find the results to be more causally-relevant relative to competing solutions based on human judgement. 

\begin{table}[t!]
    \caption{WBP update rules for common transformations.}
    \centering
    \begin{tabular}{ll}
    \toprule 
    Transformation & $G(\cdot)$ \\ \midrule
    \textit{Activation Layer} & $\tilde{W}^l=h\circ{\tilde{W}^{l+1}}$\\
    \textit{FC Layer} & $\tilde{W}^{l}=\tilde{W}^{l+1}W^{l}$\\
    \textit{Convolutional Layer} & $\tilde{W}^{l}=\tilde{W}^{l+1}\otimes{[W^{l}]^{T_{0,1}}_{flip_{2,3}}}$\\
    \textit{BN Layer} & $\tilde{W}^l=\frac{\tilde{W}^{l+1}}{\sigma}\gamma$ \\
    \textit{Pooling Layer}  & Relocate/Distribute $\tilde{W}^{l+1}$\\ \bottomrule
    \end{tabular}
    \label{tbl:wbp_layers}
 \end{table}

To simplify our presentation, we first consider a vector input and a linear mapping. 
Let ${\boldsymbol x}^l$ be the internal representation of the data at the $l$-th layer, with $l=0$ being the input layer, {\it i.e.}, ${\boldsymbol x}^0 = {\boldsymbol x}$, and $l=L$ being the penultimate {\it logit} layer prior to the softmax transformation, {\it i.e.}, $\mathbb{P}(y|{\boldsymbol x}) = {\rm softmax}({\boldsymbol x}^L)$.
To assign the relative importance to each hidden unit in the $l$-th layer, we notationally collapse all transformations after $l$ into an operator denoted by $\tilde{W}^l$, which we call the {\it saliency matrix}, satisfying,
\begin{equation}
    {\boldsymbol x}^L = \tilde{W}^l {\boldsymbol x}^l, \quad \forall l\in [0, \ldots, L], 
\end{equation}
where ${\boldsymbol x}^L$ is an $M$-dimensional vector corresponding to the $M$ distinct classes in $y$.
Though presented in a matrix form in a slight abuse of notation, {\it i.e.}, the instantiation of the operator $\tilde{W}^l$ effectively depends on the input ${\boldsymbol x}$, thus all nonlinearities have been effectively absorbed into it.
We posit that for an object associated with a given label $y=m$, its causal features are subsumed in the interactions between the $m$-th row of $\tilde{W}^0$ and input ${\boldsymbol x}$, {\it i.e.},
\begin{equation}
    [{\boldsymbol s}_m({\boldsymbol x})]_k = [\tilde{W}^0]_{mk} [{\boldsymbol x}]_k, 
\end{equation}
where $[{\boldsymbol s}_m({\boldsymbol x})]_k$ denotes the $k$-th element of the saliency map ${\boldsymbol s}_m({\boldsymbol x})$ and $[\tilde{W}^0]_{mk}$ is a single element of $\tilde{W}^0$.
A key observation for computation of $\tilde{W}^l$ is that it can be done recursively.
Specifically, let $g_l({\boldsymbol x}^l)$ be the transformation at the $l$-th layer, {\it e.g.}, an affine transformation, convolution, activation, normalization, {\it etc}., then it holds that
\begin{equation}
    \tilde{W}^{l+1} {\boldsymbol x}^{l+1} = \tilde{W}^{l+1} g_l({\boldsymbol x}^l) = \tilde{W}^{l} {\boldsymbol x}^l. 
\end{equation}
This allows for recursive computation of $\tilde{W}^{l}$ via
\begin{equation}
    \tilde{W}^{l} = G(\tilde{W}^{l+1}, g_l), \quad \tilde{W}^{L} = 1,
\end{equation}
where $G(\cdot)$ is the update rule.
We list the update rules for common transformations in deep networks in Table~\ref{tbl:wbp_layers}, with corresponding derivations detailed below. 



\vspace{2mm}
\noindent{\bf Fully-connected (FC) layer.} 
The FC transformation is the most basic operation in deep neural networks.
Below we omit the bias term as it does not directly interact with the input.
Assuming $g_l({\boldsymbol x}^l) = W^l {\boldsymbol x}^l$, it is readily seen that
\begin{equation}
    \tilde{W}^{l+1} {\boldsymbol x}^{l+1} = \tilde{W}^{l+1} g_l({\boldsymbol x}^l) = (\tilde{W}^{l+1} W^l) {\boldsymbol x}^l, 
\end{equation}
so $\tilde{W}^{l} = \tilde{W}^{l+1} W^l$.
Graphical illustration with standard affine mapping and ReLU activation can be found in the appendix.

\begin{figure*}[t!]
\centering
\includegraphics[width=0.95\textwidth]{figure34.pdf
\caption{Visualization of the inferred saliency maps. Left: CUB dataset. Right: ImageNet dataset. }
\label{fig:exp_cub}
\vspace{-1em}
\end{figure*}

\vspace{2mm}
\noindent{\bf Nonlinear activation layer.} 
Considering that an activation layer simply {\em rescales} the saliency weight matrices, {\it i.e.}, ${\boldsymbol x}^{l+1} = g_l({\boldsymbol x}^l) = h^l \circ {\boldsymbol x}^l$, where $\circ$ is the composition operator, we obtain $\tilde{W}^{l} = h \circ \tilde{W}^{l+1}$.
Using the ReLU activation as a concrete example, we have $h({\boldsymbol x}^l) = \mathbbm{1}\{{\boldsymbol x}^l\geq 0\}$.

\vspace{2mm}
\noindent{\bf Convolutional layer.}
The convolution is a generalized form of linear mapping.
In practice, convolutions can be expressed as tensor products of the form $\tilde{W}^{l}=\tilde{W}^{l+1}\otimes{[W^{l}]^{T_{0,1}}_{flip_{2,3}}}$, where $W^l\in{\mathbb{R}^{D_2\times{D_1}\times{(2S+1)}\times{(2S+1)}}}$ is the convolution kernel, $T_{0,1}$ is the transpose in dimensions 0 and 1 and $flip_{2,3}$ is an exchange in dimensions 2 and 3.
See the Appendix for details.

\vspace{2mm}
\noindent{\bf Pooling and normalization layer.} Summarization and standardization are two other essential operations for the success of deep neural networks, achieved by pooling and batch normalization (BN) techniques, respectively.
They too can be considered as special instantiations of linear operations.
Here we summarize the two most popular operations in Table \ref{tbl:wbp_layers}. 


\section{Experiments}
To validate the utility of our approach, we consider both natural  and medical image datasets, and compare it to existing state-of-the-art solutions.
All the experiments are implemented in PyTorch.
The source code will be available at \url{https://github.com/author_name/PPI}.
Due to space limitation, details of the experimental setup and additional analyses are deferred to the Appendix.

\vspace{2mm}
\noindent{\bf Datasets.} We present our findings on five representative datasets: ($i$) \texttt{CIFAR-10}~\cite{krizhevsky2009learning}; ($ii$) \texttt{ImageNet (ILSVRC2012)}~\cite{ILSVRC15}; ($iii$) \texttt{CUB}~\cite{WahCUB_200_2011}, a natural image dataset with over $12k$ photos for classification of $200$ bird species in the wild, heavily confounded by the background characteristics; ($iv$) \texttt{GA}~\cite{leuschen2013spectral}, a new medical image dataset for the prediction of {\it geographic atrophy} (GA) using 3D {\it optical coherence tomography} (OCT) image volumes, characterized by small sample size ($275$ subjects) and highly heterogeneous (collected from $4$ different facilities); and ($v$) \texttt{LIDC-IDRI}~\cite{langlotz2019roadmap}, a public medical dataset of $1,085$ lung lesion CT images annotated by $4$ radiologists.
Detailed specifications are described in the Appendix. 

\vspace{2mm}
\noindent{\bf Baselines.} The following set of popular saliency mapping schemes are considered as comparators for the proposed approach: ($i$) Gradient: standard gradient-based salience mapping; ($ii$) GradCAM \cite{selvaraju2017grad}: gradient-weighted class activation mapping; ($iii$) LRP~\cite{bach2015pixel}: layer-wise relevance propagation and its variants. 

\vspace{2mm}
\noindent{\bf Hyperparameters.} 
The final loss of the proposed model is a weighted summation of four losses: $L = L_{cls} + w_1 L_{con} + w_2 L_{reg} + w_3 L_{ad}$.
The weights are simply balanced to match the magnitude of $L_{cls}$, \emph{i.e.}, $w_{3}=1$,  $w_{2}=0.1$ and $w_{1}=1$ (CUB), $=1$ (GA), and $=10$ (LIDC).
See Appendix Sec B for details about the masking parameters $\sigma$ and $\omega$.




\subsection{Natural Image Datasets}
\noindent \textbf{Classification Gains}
In this experiment, we investigate how the different pairings of PPI and saliency mapping schemes ({\it i.e.}, GradCAM, LRP, WBP) affect performance.
In Table \ref{tab:exp_ga}, the first row represents VGG11 model trained with only classification loss, and the following rows represent VGG11 trained with PPI with different saliency mapping schemes. We see consistent performance gains in accuracy via incorporating PPI training on both CUB and CIFAR-10 datasets. The gains are mostly significant when using our WBP for saliency mapping (improving the accuracy from $0.662$ to $0.696$ on CUB, and from $0.881$ to $0.901$ on CIFAR-10. 


\begin{table}[t!]
    \caption{Performance improvements achieved by training with PPI on CUB,  CIFAR-10, and GA dataset. We report means and standard deviations (SDs) from 5-fold cross-validation for GA prediction.}
    \centering
    \begin{tabular}{llll}
        \toprule
        \textbf{Models} & {\textbf{CUB}} & \textbf{Cifar-10}  & \textbf{GA }    \\ 
        \textbf{{}} & {\textbf{(Acc)}} & \textbf{(Acc)}  & \textbf{(AUC)}    \\ \midrule
       
        Classification & $0.662$ & $0.881$ & $0.877 \pm 0.040$\\  
        +$\text{PPI}_{Gradient}$ & ${0.673}$ & $0.885$ & $0.890 \pm 0.035$\\ 
        +$\text{PPI}_{LRP}$ & ${0.680}$ & $0.891$ & $0.895 \pm 0.037$\\ 
        +$\text{PPI}_{GradCAM}$ & ${0.683}$ & $0.895$ & $0.908 \pm 0.036$ \\ 
        +$\text{PPI}_{WBP}$ & \textbf{0.696} & \textbf{0.901} & $0.925 \pm 0.023$ \\ 
       
        \bottomrule
    \end{tabular}
    \label{tab:exp_ga}
    \vspace{-4mm}
\end{table}


\begin{figure*}[t!]
\centering
\includegraphics[width=0.97\textwidth]{examples_ga_mar17.jpg
\caption{Saliency maps on GA dataset based on models trained with PPI and without PPI. Maps of models trained with PPI are more clinically relevant by focusing on retinal layers likely to contain abnormalities or lesions, and more concentrated. }
\label{fig:exp_ga_1}
\vspace{-1em}
\end{figure*}

\noindent\textbf{Model Interpretability}
In this task, we want to qualitatively and quantitatively compare the causal relevance of saliency maps generated by our proposed model and its competitors.
In Figure \ref{fig:exp_cub}, we show the saliency maps produced by different approaches for a VGG11 model trained on CUB.
Visually, gradient-based solutions (Grad and GradCAM) tend to yield overly dispersed maps, indicating a lack of specificity.
LRP gives more appealing saliency maps. However, these maps also heavily attend to the spurious background cues that presumably help with predictions. When trained with PPI, the saliency maps attend to birds body, and with WBP, the saliency maps focus on the causal related pixels.

To quantitatively evaluate the causal relevance of competing saliency maps, we adopt the evaluation scheme proposed in \cite{hooker2019benchmark}, consisting of masking out the contributing saliency pixels and then calculating the reduction in prediction score.
A larger reduction is considered better for accurately capturing the pixels that `cause' the prediction.
Results are summarized in Figure~\ref{fig:cub_point}, where we progressively remove the top-$k$ saliency points, with $k=100, 500, 1000, 5000, 10000$ ($10000 \approx 6.6\%$ of all pixels), from the CUB test input images.
Our PPI consistently outperforms its counterparts, with its lead being most substantial in the low-$k$ regime.
Notably, for large $k$, PPI removes nearly all predictive signal.
This implies PPI specifically targets the causal features.
Quantitative evaluation with additional metrics are provided in the Appendix.

To test the performance of WBP itself (without being trained with PPI), we compare WBP with different approaches for a VGG11 model trained on ImageNet from PyTorch model zoo. Figure \ref{fig:exp_cub}(left) shows that saliency maps generated by WBP more concentrate on objects themselves. Also, thanks to the fine resolution of WBP, the model pays more attention to the patterns on the fur to identify the leopard (row 1). This is more visually consistent with human judgement. Figure~\ref{fig:imagenet_point} demonstrates WBP identifies more causal pixels on ImageNet validation images. 




\begin{figure}[!tbp]
\centering
\begin{subfigure}[b]{0.23\textwidth}
         \centering
         \includegraphics[width=\textwidth]{cub-point-game.png}
       
         \caption{CUB}
         \label{fig:cub_point}
     \end{subfigure}
\begin{subfigure}[b]{0.23\textwidth}
         \centering
         \includegraphics[width=\textwidth]{imagenet-point-game.png}
       
         \caption{ImageNet}
         \label{fig:imagenet_point}
     \end{subfigure}
     \vspace{-.5em}
\caption{Quantitative evaluations of causal relevance of competing saliency maps (higher is better).}
\vspace{-1em}
\end{figure}



\subsection{OCT-GA: Geographic Atrophy Classification} \label{sec:exp_ga}
Next we show how the proposed PPI handles the challenges of small training data and heterogeneity in medical image datasets.
In this experiment (with our new dataset, that we will make public), each OCT volume image consists of $100$ scans of a $512 \times 1000$ sized image \cite{boyer2017pathophysiology}.
We use a multi-view CNN model ~\cite{su2015multi} to process such 3D OCT inputs, and use it as our baseline solution (see the Appendix for details).
We investigate how the different saliency mapping schemes ({\it i.e.}, Grad, GradCAM, LRP, WBP) work with PPI.
For WBP, we also tested the bounding box variant, denoted as WBP (box) (see the Appendix for details).
In Table \ref{tab:exp_ga}, we see consistent performance gains in AUC score via incorporating PPI training (from $0.877$ to $0.925$, can be improve to $0.937$ by PPI with WBP(box)), accompanied by the reductions in model variation evaluated by the standard deviations of AUC from the five-fold cross-validation.
The gains are most significant when using our WBP for saliency mapping.
We further compare the saliency maps generated by these different combinations.
We see that without the additional supervision from PPI, competing solutions like Grad, GradCAM and LRP sometimes yield non-sensible saliency maps (attending to image corners).
Overall, PPI encourages more concentrated and less noisy saliency maps.
Also, different PPI-based saliency maps agree with each other to a larger extent.
Our findings are also verified by experts (co-authors, who are ophthalmologists specializing in GA) confirming that the PPI-based saliency maps are clinically relevant by focusing on retinal layers likely to contain abnormalities or lesions.
These results underscore the practical value of the proposed proactive interventions. 


\noindent{\bf Cross-domain generalization.} Common to medical image applications is that training samples are usually integrated from a number of healthcare facilities ({\it i.e.}, domains), and that predictions are sometimes to be made on subjects at other facilities.
Despite big efforts to standardize the image collection protocols, with different imaging systems operated by technicians with varying skills, apparent domain shifts are likely to compromise the cross-domain performance of these models.
We show this phenomenon on the \texttt{GA} dataset in Table \ref{tab:exp_transform}, where source samples are collected from four different hospitals in different health systems (A, B, C and D, see the Appendix for details).
Each cell contains the AUC of the model trained on site X (row) and tested on site Y (column), with same-site predictions made on hold-out samples.
A significant performance drop is observed for cross-domain predictions (off-diagonals) compared to in-domain predictions (diagonals).
With the application of PPI, the performance gaps between in-domain and cross-domain predictions are considerably reduced.
The overall accuracy gains of PPI further justify the utility of causally-inspired modeling.
Notably, site D manifests strong spurious correlation that help in-domain prediction but degrades out-of-site generalization, which is partly resolved by the proposed PPI. 

\begin{table}[]
\caption{AUC results for GA prediction with or without PPI. Models are trained on one site and cross-validated on the other sites. Darker color indicates better performance.}
\resizebox{\linewidth}{!}{
\begin{tabular}{@{}lllllll@{}}
\toprule
With $\text{PPI}$   & A     & B     & C    & D  & Mean & STD \\ \midrule
A  & \cellcolor{blue!40} 1.000 & \cellcolor{blue!25}0.906 & \cellcolor{blue!15}0.877 & \cellcolor{blue!15}0.865 & \textbf{0.912}	& \textbf{0.061}\\
B  & \cellcolor{blue!15}0.851 & \cellcolor{blue!35}0.975 & \cellcolor{blue!15}0.863 & \cellcolor{blue!25}0.910 & \textbf{0.900}	& 0.056\\
C & \cellcolor{blue!35}0.954 & \cellcolor{blue!15}0.875 & \cellcolor{blue!25}0.904 & \cellcolor{blue!25}0.931 & \textbf{0.916}	& \textbf{0.034}\\
D & \cellcolor{blue!5}0.824 & \cellcolor{blue!5}0.846 & \cellcolor{blue!15}0.853 & \cellcolor{blue!25}0.904 & \textbf{0.857}	& \textbf{0.034}\\
\hline
No PPI  & A     & B     & C    & D   & Mean & STD \\ \midrule
A  & \cellcolor{blue!40}1.000 & \cellcolor{blue!15}0.854 & \cellcolor{blue!5}0.832 & \cellcolor{blue!5} 0.827 & 0.878 & 0.082\\
B  & \cellcolor{blue!5}0.810 & \cellcolor{blue!15}0.874 & \cellcolor{blue!5}0.850 & \cellcolor{blue!25}0.906 & 0.860 & \textbf{0.040}\\
C & \cellcolor{blue!15}0.860 & 0.779 & \cellcolor{blue!15}0.873 & \cellcolor{blue!15}0.862  & 0.843 & 0.043\\
D & 0.748 & 0.792 & \cellcolor{blue!5}0.836 & \cellcolor{blue!35}0.961  & 0.834 & 0.092\\ \bottomrule
\end{tabular}}
\label{tab:exp_transform}
\vspace{-3mm}
\end{table}

\subsection{LIDC-IDRI: Lung Lesions Classification} \label{sec:exp_lidc}

\begin{wraptable}{r}{3.7cm}
  \vspace{-15pt}   
    \caption{LIDC-IDRI classification AUC results. }
    \centering
    \resizebox{\linewidth}{!}{
    \begin{tabular}{ll}
        \toprule
        \textbf{Models} & \textbf{AUC}    \\ \midrule
        Tensor Net-X~\cite{efthymiou2019tensornetwork} & 0.823 \\
        DenseNet~\cite{huang2017densely} & 0.829 \\
        LoTeNet~\cite{selvan2020tensor} & 0.874 \\
        Inception\_v3~\cite{szegedy2016rethinking} & 0.921 \\  
        +$\text{PPI}_{GradCAM}$ & {0.933}  \\ 
        +$\text{PPI}_{Gradient}$ & {0.930}  \\ 
        +$\text{PPI}_{LRP}$ & {0.931}  \\ 
        +$\text{PPI}_{WBP}$ & {0.935}  \\ 
        +$\text{PPI}_{WBP (box)}$ & \textbf{0.941} \\ 
        \bottomrule
    \end{tabular}}
    \label{tab:exp_lidc}
    \vspace{-7pt} 
\end{wraptable}

To further examine the practical advantages of the proposed PPI in real-world applications, we benchmark its utility on LIDC-IDRI; a public lung CT scan dataset \cite{armato2011lung}.
We followed the preprocessing steps outlined in \cite{kohl2018probabilistic} to prepare the data, and adopted the experimental setup from \cite{selvan2020tensor} to predict lesions.
We use Inception\_v3~\cite{szegedy2016rethinking} as our base model for both standard classification and PPI-enhanced training with various saliency mapping schemes.
See the Appendix for details.

\vspace{2mm}
\noindent{\bf Lesion classification.} We first compare PPI to other specialized SOTA network architectures.
Table~\ref{tab:exp_lidc} summarizes AUC scores of Tensor Net-X~\cite{efthymiou2019tensornetwork}, DenseNet~\cite{huang2017densely}, LoTeNet~\cite{selvan2020tensor}, Inception\_v3~\cite{szegedy2016rethinking}, as well as our Inception\_v3 trained with and without $\text{PPI}_{WBP}$.
The proposed $\text{PPI}_{WBP (box)}$ leads the performance chart by a considerable margin, improving Inception\_v3 from $0.92$ to $0.94$.

\begin{figure}[t!]
\centering
\includegraphics[width=0.43\textwidth]{examples_lidc_mar17.jpg
\caption{Saliency maps on LIDC-IDR. Saliency maps of PPI+WBP are mostly consistent with the ground truths.}
\label{fig:exp_lidc}
\vspace{-4mm}
\end{figure}


\vspace{2mm}
\noindent{\bf Weakly-supervised image segmentation.} In Figure~\ref{fig:exp_lidc}, we compare saliency maps generated by GradCAM, WBP, WBP (box) to the ground truth lesion masks from expert annotations.
Note that we have only supplied patch-label labels during training, not the pixel-level expert segmentation masks, which constitute a challenging task of weakly-supervised image segmentation.
In line with the observations from the GA experiment, our PPI-training enhanced WBP saliency maps are mostly consistent with the expert segmentations.
Together with Table \ref{tab:exp_lidc}, Figure \ref{fig:exp_lidc} confirms that the proposed PPI+WBP improves both the classification performance and model interpretability. 

\section{Conclusions}
We have presented \textit{Proactive Pseudo-Intervention} (PPI), a novel interpretable computer vision framework that organically integrates saliency mapping, causal reasoning, synthetic intervention and contrastive learning.
PPI couples saliency mapping with contrastive training by creating artificially intervened negative samples absent of causal features.
To communicate model insights and facilitate causal-informed reasoning, we derived an architecture-agnostic saliency mapping scheme called \textit{Weight Back Propagation} (WBP), which faithfully captures the causally-relevant pixels/features for model prediction.
Visual inspection of the saliency maps show that WBP, is more robust to spurious features compared to competing approaches.
Empirical results on natural and medical datasets verify the combination of PPI and WBP consistently delivers performance boosts across a wide range of tasks relative to competing solutions, and the gains are most significant where the application is complicated by small sample size, data heterogeneity, or confounded with spurious correlations.



{\small
\bibliographystyle{ieee_fullname}

