\section{Introduction}
Vision-Language Navigation (VLN)~\cite{anderson2018vision,ku2020room,Chen2019TOUCHDOWNNL,Nguyen2019HelpAV} has attracted increasing interests in robotic applications since an instruction-following navigation agent is  practical and flexible in real-world scenarios. 
For accomplishing successful navigation, a VLN agent needs to align complicated visual observations to language instructions to reach the required target point. For example, when asking to ``turn left to the bathroom'', the agent should choose the right observation which not only contains the mentioned object ``bathroom'' but also indicates the direction ``turn left''.

Most of early VLN approaches adopt the LSTM-based encoder-decoder framework~\cite{fried2018speaker,tan2019learning,wang2019reinforced,ma2019self,zhu2020vision}, which encodes both the visual observations and language instructions and then generates the action sequence. With the development of large-scale cross-modal pretraining models in vision-language tasks~\cite{li2020unicoder,li2020oscar,chen2020uniter,lu2019vilbert}, emerging works attempt to introduce them into VLN tasks~\cite{hao2020towards,hong2021vln,Chen2021HistoryAM,Moudgil2021SOATAS}. 
However, both the non-pretraining-based or pretraining-based approaches represent the visual observations by raw directional features and visual features trained using one-hot labels, which are difficult to be aligned to the linguistic instruction features due to the large semantic gap among them. This direct alignment process also leads to poor interpretability of action decision and therefore makes the VLN agents unreliable to be deployed to real environments.


In this work, we aim to {\it mitigate the semantic gap and simplify the alignment} in VLN by proposing a new framework, called Actional Atomic-Concept Learning (AACL). Since the instructions usually consist of atomic action concepts, e.g., ``turn right'', and object concepts\footnote{In this work, we also treat the scene concept, e.g., ``bathroom'', as the object concept. }, e.g., ``kitchen'', in AACL, the visual observations are mapped to actional atomic concepts, which are natural language phrases each containing an action and an object. The actions are extracted from a predefined atomic action set. These actional atomic concepts, which can be viewed as the bridge between observations and instructions,  effectively facilitate the alignment as well as provide good interpretability for action decision. 

AACL consists of three main components. Firstly, a \textbf{concept mapping module} is constructed to map each single view observation to the actional atomic concept. For deriving the object concept, we resort to the recently proposed Contrastive Language-Image Pretraining (CLIP) model~\cite{radford2021learning} rather than image classification or object detection models pretrained on a fixed category set. Benefiting from the powerful open-world object recognition ability of CLIP, AACL can better generalize to diverse navigation scenarios. And we map the sequential direction information in VLN environments during navigation to the action concept.  Secondly, to encourage more instruction-oriented object concept extraction for facilitating the multi-modal alignment, a \textbf{concept refining adapter} is further introduced  to re-rank the predicted object concepts of CLIP according to the instruction. 
Lastly, an  \textbf{observation co-embedding module} embeds each observation and its paired actional atomic concept, and then uses concept representations to regularize the observation representations through an observation contrast strategy. Figure~\ref{fig:motivation} presents an action selection comparison between existing VLN agents and our AACL. Through mapping visual observations to actional atomic concepts formed by language, AACL can simplify the modality alignment and distinguish different action candidates easier to make correct actions. 


We conduct experiments on several popular VLN benchmarks, including one with fine-grained instructions (R2R~\cite{anderson2018vision}) and two with high-level instructions (REVERIE~\cite{qi2020reverie} and R2R-Last~\cite{Chen2021HistoryAM}). Experimental results show that our AACL outperforms the state-of-the-art approaches on all benchmarks. Moreover, benefiting from these actional atomic concepts, AACL shows excellent interpretability in making action decision, which is a step closer towards developing reliable VLN agents in real-world applications.

\begin{figure*}[t]
\begin{centering}
\includegraphics[width=0.98\linewidth]{overview.pdf}
\par\end{centering}
\vspace{-0.2cm}
\caption{\label{fig:overview}Overview of our Actional Atomic-Concept Learning (AACL). At each timestep $t$, the agent receives the instruction $I$, the observation $O_{t}$, and the navigation history $H_{t}$. For each $O_{t,n}$ in $O_{t}$ containing the single-view image $B_{t,n}$ and the direction $A_{t,n}$,  object concept mapping and action concept mapping are conducted based on the concept refining adapter to obtain the actional atomic concept representations $\mathbf{\tilde{u}}_{t,n}$. Then $\mathbf{\tilde{u}}_{t,n}$ is used to regularize the visual representation $\mathbf{v}_{t,n}$ and the directional representation $\mathbf{e}_{A_{t,n}}$ through the observation co-embedding module for making action selection. For simplicity, we omit the learning process of navigation histories $H_{t}$, which is similar to that of observations $O_{t}$. 
}
\vspace{-0.6cm}
\end{figure*}


\section{Related Work}
\textbf{Vision-Language Navigation.}
Developing navigation agents which can follow natural language instructions has attracted increasingly research interests in recent years~\cite{anderson2018vision,ku2020room,Chen2019TOUCHDOWNNL,Nguyen2019HelpAV,qi2020reverie}. Most of early Vision-Language Navigation (VLN) approaches employ the LSTM-based encoder-decoder framework~\cite{fried2018speaker,tan2019learning,wang2019reinforced,ma2019self,zhu2020vision,Wang2020SoftER,Qi2020ObjectandActionAM,fu2020counterfactual}. Speaker-Follower~\cite{fried2018speaker} introduces a novel instruction augmentation strategy to mitigate the annotation burden of high-quality instructions. EnvDrop~\cite{tan2019learning} employs environmental dropout to augment limited training data by mimicking unseen environments.
Due to the success of Transformer-based cross-modal pretraining~\cite{li2020unicoder,li2020oscar,chen2020uniter,lu2019vilbert,Li2019VisualBERTAS}, recent works have explored transformer architectures into VLN tasks~\cite{hao2020towards,hong2021vln,Chen2021HistoryAM,Moudgil2021SOATAS,Qi2021TheRT,Guhur2021AirbertIP}.
HAMT~\cite{Chen2021HistoryAM} develops a history aware multimodal transformer to better encode the long-horizon navigation history. DUET~\cite{Chen2022ThinkGA} constructs a dual-scale graph transformer for joint long-term action planning and fine-grained cross-modal understanding.
HOP~\cite{Qiao2022HOPHA} introduces a new history-and-order aware pretraining paradigm for encouraging the learning of spatio-temporal multimodal correspondence. 
However, these pretraining-based methods still learn to align the raw directional features and visual features trained using one-hot labels to the linguistic instruction features, leading to limited performance due to the large semantic gap among these multi-modal inputs. 

In contrast to the above mentioned VLN approaches, in this work, we  build a bridge among multi-modal inputs for facilitating the alignment by introducing actional atomic concepts formed by language. Through these  actional atomic concepts, the alignment can be significantly simplified and good interpretability can be provided. 



\textbf{Contrastive Language-Image Pretraining (CLIP).}
CLIP~\cite{radford2021learning} is a cross-modal pretrained model using 400 million image and text pairs collected from the web. Through natural language supervision~\cite{Jia2021ScalingUV,Sariyildiz2020LearningVR,Desai2021VirTexLV} rather than one-hot labels of fixed size of object categories, CLIP has shown great potential in open-world object recognition. Recently, many works have attempted to introduce CLIP into various computer vision (CV) or vision-language (V\&L) tasks to improve the generalization ability of the downstream models~\cite{Song2022CLIPMA,Subramanian2022ReCLIPAS,Khandelwal2021SimpleBE,Shen2021HowMC,Rao2021DenseCLIPLD,Dai2022EnablingMG,Liang2022VisualLanguageNP}. DenseCLIP~\cite{Rao2021DenseCLIPLD} introduces CLIP into dense prediction tasks, e.g., semantic segmentation,  through converting the original image-text matching in CLIP to the pixel-text matching.~\cite{Dai2022EnablingMG} distills the vision-language knowledge learned in CLIP to enhance the multimodal generation models. EmbCLIP~\cite{Khandelwal2021SimpleBE} investigates the ability of CLIP's visual representations in improving embodied AI tasks. Some works have also tried to apply CLIP into VLN tasks~\cite{Liang2022VisualLanguageNP,Shen2021HowMC}. ProbES~\cite{Liang2022VisualLanguageNP} utilizes the knowledge learned from CLIP to build an in-domain dataset by self-exploration for pretraining.~\cite{Shen2021HowMC} replaces the ResNet visual encoder pretrained on ImageNet in the conventional VLN models with the pretrained CLIP visual encoder. 

In this paper, we resort to the powerful object recognition ability of CLIP to provide object concepts for each single-view observation. To encourage  instruction-oriented object concept extraction for better alignment, a concept refining adapter is further introduced beyond CLIP to re-rank its predicted object concepts according to the instruction. 



\begin{table*}[t]
	\fontsize{6}{6}\selectfont


\caption{Atomic Action Concept Mapping.}
	\vspace{-0.2cm}
	\label{tab:atomic action concept mapping}
	\resizebox{1.0\linewidth}{!}{
	{\renewcommand{\arraystretch}{1.2}
		\begin{tabular}{c||c|c|c|c|c|c|c}

			\specialrule{.1em}{.05em}{.05em}
		
			\multirow{2}{*}{Elevation $\tilde{\theta}_{t,n}$}&\multicolumn{7}{c}{Heading $\tilde{\psi}_{t,n}$ }\cr\cline{2-8}
			&(-2$\pi$, -3$\pi$/2]&(-3$\pi$/2, -$\pi$/2)&[-$\pi$/2, 0)&0&(0, $\pi$/2]&($\pi$/2, 3$\pi$/2)&[3$\pi$/2, 2$\pi$)\cr
			\hline
			$>$0&\multicolumn{7}{c}{go up}\cr\cline{2-8}
			$<$0&\multicolumn{7}{c}{go down}\cr\cline{2-8}
			0&turn right&go back&turn left&go forward&turn right&go back&turn left\\
        
 \specialrule{.1em}{.05em}{.05em}

		
			\end{tabular}}}

	\vspace{-0.4cm}
\end{table*}
\section{Preliminaries}
\subsection{VLN Problem Setup}
Given a natural language instruction $I=\{w_{1},...,w_{l}\}$ with $l$ words, a VLN agent is asked to navigate from a start viewpoint $S$ to the goal viewpoint $G$. At each timestep $t$, the agent receives a panoramic observation, containing $N_{o}$ image views  $O_{t}=\{O_{t,n}\}_{n=1}^{N_{o}}$. Each $O_{t,n}$ contains the image $B_{t,n}$ and the attached direction information $A_{t,n}$. The visual feature $\mathbf{v}_{t,n}$ for $B_{t,n}$ is obtained by a pretrained ResNet~\cite{he2016deep} or ViT~\cite{dosovitskiy2021an}. $A_{t,n}$ is usually composed of the heading $\psi_{t,n}$ and the elevation $\theta_{t,n}$. Each panoramic observation contains $d$ navigable viewpoints $C_{t}=\{C_{t,i}\}_{i=1}^{d}$ as the action candidates. At timestep $t$, the agent predicts an action $\mathbf{a}_{t}$ from $C_{t}$ based on the instruction $I$ and current visual observations $O_{t}$. 

\subsection{Baseline Agents}
Our AACL can be applied to many previous VLN models. In this work, two strong baseline agents HAMT~\cite{Chen2021HistoryAM} and DUET~\cite{Chen2022ThinkGA} are selected. In this section, we briefly describe one baseline HAMT. In HAMT, the agent receives the instruction $I$, the panoramic observation $O_{t}$, and the navigation history $H_{t}$ at each timestep $t$. $H_{t}$ is a sequence of historical visual observations. A standard BERT~\cite{Devlin2019BERTPO} is used to obtain the instruction feature $\mathbf{f}_{I}$ for $I$.
For each view $n$ with the angle information $<\psi_{t,n}, \theta_{t,n}>$ in $O_{t}$, the direction feature is defined by $\mathbf{e}_{A_{t,n}}=(\mathrm{sin}\psi_{t,n},\mathrm{cos}\psi_{t,n},\mathrm{sin}\theta_{t,n},\mathrm{cos}{\theta_{t,n}}$). With the visual feature $\mathbf{v}_{t,n}$ and the direction feature $\mathbf{e}_{A_{t,n}}$, the observation embedding $\mathbf{o}_{t,n}$ for each view $n$ is calculated by:
\begin{equation}
\label{eq:compose}
\begin{aligned}
\mathbf{o}_{t,n}&=\mathrm{Dr}(\mathrm{LN}(\mathrm{LN}(\mathbf{W}_{v}\mathbf{v}_{t,n})+\mathrm{LN}(\mathbf{W}_{a}\mathbf{e}_{A_{t,n}})\\
&+\mathbf{e}_{t,n}^{N}+\mathbf{e}_{v}^{T})),
\end{aligned}
\end{equation}
where $\mathrm{LN}(\cdot)$ and $\mathrm{Dr}(\cdot)$ denote layer normalization~\cite{Ba2016LayerN} and dropout, respectively. $\mathbf{W}_{v}$ and  $\mathbf{W}_{a}$  are learnable weights, and $\mathbf{e}_{t,n}^{N}$ and $\mathbf{e}_{v}^{T}$ denote the navigable embedding and the type embedding, respectively \cite{Chen2021HistoryAM}. The observation feature $\mathbf{o}_{t}$ is represented by $\mathbf{o}_{t}=[\mathbf{o}_{t,1};...;\mathbf{o}_{t,N_{o}}]$, where $N_{o}$ is the number of views. And a hierarchical vision transformer~\cite{Chen2021HistoryAM} is constructed to get the history feature $\mathbf{h}_{t}=[\mathbf{h}_{t,1};...;\mathbf{h}_{t,t-1}]$ for the navigation history $H_{t}$. 

Then $\mathbf{f}_{I}$, $\mathbf{o}_t$, and $\mathbf{h}_t$ are fed into a cross-modal transformer encoder $E^{c}(\cdot)$, resulting in:

\begin{equation}
\label{eq:ec}
\mathbf{\tilde{f}}_{I}^{t},\mathbf{\tilde{o}}_t,\mathbf{\tilde{h}}_t=E^{c}(\mathbf{f}_{I},[\mathbf{o}_t;\mathbf{h}_t]).
\end{equation}

The updated instruction feature $\mathbf{\tilde{f}}_{I}^{t}$ and observation feature $\mathbf{\tilde{o}}_t$ are used for action prediction:
\begin{equation}
\label{eq:ea}
\mathbf{a}_{t} = E^{a}(\mathbf{\tilde{f}}_{I}^{t}, \mathbf{\tilde{o}}_t),
\end{equation}
where $E^{a}(\cdot)$ is a two-layer fully-connected network. For more model details, refer to~\cite{Chen2021HistoryAM}.

In Eq.~\ref{eq:compose}, HAMT obtains the observation feature $\mathbf{o}_{t,n}$ directly by the pretrained visual feature $\mathbf{v}_{t,n}$ and the raw direction feature $\mathbf{e}_{A_{t,n}}$. 
In AACL, we map the observations $O_{t,n}$ to actional atomic concepts $U_{t,n}$ formed by language and use $U_{t,n}$ to obtain the new observation feature $\mathbf{o}'_{t,n}$. In this way,  the gap between $O_{t}$ and $I$ can be effectively mitigated to simplify the alignment. 


\section{Actional Atomic-Concept Learning}
In this section, we describe our AACL in detail, the overview of which is presented in Figure~\ref{fig:overview}. At timestep $t$, the agent receives multi-modal inputs $I$,  $O_{t}$, and $H_{t}$ similar to HAMT. For each $O_{t,n}$ in $O_{t}$ containing the single-view image $B_{t,n}$ and the direction $A_{t,n}$, AACL first conducts object concept mapping and atomic action concept mapping to obtain the object concept $U_{t,n}^{obj}$ and the action concept $U_{t,n}^{act}$. And a concept refining adapter is built to re-rank $U_{t,n}^{obj}$ according to the instruction $I$ for better alignment. The actional atomic concept $U_{t,n}$ is then obtained by concatenating $U_{t,n}^{act}$ and $U_{t,n}^{obj}$, and fed to the concept encoder $E^{t}(\cdot)$ to get the concept feature $\mathbf{\tilde{u}}_{t,n}$. Finally, an observation co-embedding module is constructed to use $\mathbf{\tilde{u}}_{t,n}$ for regularizing the visual feature $\mathbf{v}_{t,n}$ and the directional feature $\mathbf{e}_{A_{t,n}}$ to get new observation features $\mathbf{o}'_{t,n}$. For $H_{t}$ which contains historical visual observations, we also use AACL to get the enhanced history features $\mathbf{h}'_{t}$ like $O_{t}$. Then $\mathbf{o}'_{t}=\{\mathbf{o}'_{t,n}\}_{n=1}^{N_{o}}$, $\mathbf{h}'_{t}$, and the instruction features $\mathbf{f}_{I}$ are fed to the cross-modal Transformer encoder $E^{c}(\cdot)$ for calculating the action predictions $\mathbf{a}'_{t}$. 








\subsection{Actional Atomic-Concept Mapping}
\label{Actional} 

\textbf{Object Concept Mapping.}
For each observation $O_{t,n}$ containing the single-view image $B_{t,n}$, we map $B_{t,n}$ to get the object concept $U_{t,n}^{obj}$  based on a pre-built in-domain object concept repository. Benefiting from large-scale language supervision from 400M image-text pairs, CLIP~\cite{radford2021learning} has more powerful open-world object recognition ability than conventional image classification or object detection models pretrained on a fixed-size object category set. In this work, we resort to CLIP to conduct the object concept mapping considering its good generalizability.  
Concretely, the object concept repository $\{U_{c}^{obj}\}_{c=1}^{N_{c}}$ is constructed by extracting object words from the training dataset, where $N_{c}$ is the repository size. And we get the image feature $\mathbf{f}_{B_{t,n}}$ through the pretrained CLIP Image Encoder $E^{v}_{\mathrm{CLIP}}(\cdot)$:
\begin{equation}
\mathbf{f}_{B_{t,n}}=E^{v}_{\mathrm{CLIP}}(B_{t,n}).
\end{equation}
For object concept $U_{c}^{obj}$, we construct the text phrase $T_{c}$ formed as ``a photo of a \{$U_{c}^{obj}$\}''. Then the text feature $\mathbf{f}_{T_{c}}$ is derived through the pretrained CLIP Text Encoder $E^{t}_{\mathrm{CLIP}}(\cdot)$:
\begin{equation}
\mathbf{f}_{T_{c}} =E^{t}_{\mathrm{CLIP}}(T_{c}).
\end{equation}
Then the mapping probability of the image $B_{t,n}$ regarding the object concept $U_{c}^{obj}$ is calculated by:
\begin{equation}
\mathbf{p}(y=U_{c}^{obj}|B_{t,n}) = \frac{\mathrm{exp}(\mathrm{sim}(\mathbf{f}_{B_{t,n}},\mathbf{f}_{T_{c}})/\tau)}{\sum_{c=1}^{N_{c}}(\mathrm{exp}(\mathrm{sim}(\mathbf{f}_{B_{t,n}},\mathbf{f}_{T_{c}})/\tau))},
\end{equation}
where $\mathrm{sim}(\cdot,\cdot)$  denotes the cosine similarity, $\tau$ represents the temperature parameter.  Considering that a single-view image in the observation usually contains more than one salient object, we extract the top $k$ object concepts (text) having the maximum mapping probabilities conditioned on  $B_{t,n}$ as its corresponding object concepts, i.e., $U_{t,n}^{obj}=\{U^{obj}_{t,n,i}\}_{i=1}^{k}$.




\textbf{Atomic Action Concept Mapping.}
The atomic action concept $U_{t,n}^{act}$ for $O_{t,n}$ can be derived through its  directional information $A_{t,n}$ and the directional information $\tilde{A}_{t-1}$ of the agent's selected action at timestep $t$-1. We first use six basic actions in VLN tasks to build the predefined atomic action set, i.e., {\it go up, go down, go forward, go back,  turn right,} and {\it turn left}. Denote $A_{t,n}=<\psi_{t,n},\theta_{t,n}>$, where $\psi_{t,n} \in [0,2\pi)$ and $\theta_{t,n} \in [-\frac{\pi}{2},\frac{\pi}{2}]$ are the heading and the elevation, respectively. Similarly, $\tilde{A}_{t-1}=<\tilde{\psi}_{t-1},\tilde{\theta}_{t-1}>$, where $\tilde{\psi}_{t-1} \in [0,2\pi)$ and $\tilde{\theta}_{t-1} \in [-\frac{\pi}{2},\frac{\pi}{2}]$. We calculate the relative direction of $<\psi_{t,n},\theta_{t,n}>$ to $<\tilde{\psi}_{t-1},\tilde{\theta}_{t-1}>$ by:
\begin{comment}
\begin{equation}
\begin{aligned}
\tilde{\psi}_{t}^{n} &=\psi_{t}^{n}- \psi_{t-1},\\
\tilde{\theta}_{t}^{n} &=\theta_{t}^{n} -  \theta_{t-1}. 
\end{aligned}
\end{equation}
\end{comment}
\begin{equation}
\begin{aligned}
\tilde{\psi}_{t,n}=\psi_{t,n}- \tilde{\psi}_{t-1}, \quad \tilde{\theta}_{t,n} =\theta_{t,n} -  \tilde{\theta}_{t-1}. 
\end{aligned}
\end{equation}
Then we use $<\tilde{\psi}_{t,n}, \tilde{\theta}_{t,n}>$ to obtain $U_{t,n}^{act}.$ Following the direction judgement rule in VLN~\cite{anderson2018vision}, we use $\tilde{\theta}_{t,n}$ first to judge whether $U^{act}_{t,n}$ is ``go up'' or ``go down'' by comparing it to zero. Otherwise, $U^{act}_{t,n}$ is further determined through $\tilde{\psi}_{t,n}$. Specifically, if $\tilde{\psi}_{t,n}$ is equal to zero, $U^{act}_{t,n}$ is    ``go forward''. Otherwise, $U^{act}_{t,n}$ is further determined to be ``turn right'', ``turn left'', or ``go back''. The detailed mapping rule is listed in Table~\ref{tab:atomic action concept mapping}.

\subsection{Concept Refining Adapter}
\label{Concept Refining Adapter}
After getting $\{U^{obj}_{t,n,i}\}_{i=1}^{k}$ and $U_{t,n}^{act}$ for each $O_{t,n}$, the actional atomic concept $\{U_{t,n,i}\}_{i=1}^{k}$ can be obtained by directly concatenating $U_{t,n}^{act}$ and each $U_{t,n,i}^{obj}$. 
A direct way to obtain the actional atomic concept feature $\mathbf{u}_{t,n}$  is to feed each $U_{t,n,i}$ to the concept encoder $E^{t}(\cdot)$ and get a weighted sum based on their object prediction probability $\mathbf{p}_{i}$ by CLIP:
\vspace{-0.2cm}
\begin{equation}
\label{eq:concept}
\mathbf{u}_{t,n}=\sum_{i=1}^{k}\mathbf{p}_{i} \cdot E^{t}(U_{t,n,i}), 
\end{equation}
where $E^{t}(\cdot)$ is the BERT-like language encoder. 
However, even if CLIP can extract informative object concepts for each observation, some noisy object concepts may exist and extracting instruction-oriented object concepts would be more useful for alignment and making action decisions.
Inspired by~\cite{Gao2021CLIPAdapterBV}, we propose to construct a concept refining adapter beyond CLIP to refine the object concept under the constraint of the instruction.  Given a feature $\mathbf{f}$, the concept refining adapter is written as:
\begin{equation}
A(\mathbf{f}) = \mathrm{ReLU}(\mathbf{f}^{T}\mathbf{W}_{1})\mathbf{W}_{2},
\end{equation}
where $\mathbf{W}_{1}$ and $\mathbf{W}_{2}$ are learnable parameters, and $\mathrm{ReLU}(\cdot)$ is the rectified linear unit for activation.  Denote the instruction feature as $\mathbf{f}_{I}=\{\mathbf{f}_{I}^{cls},\mathbf{f}_{I}^{1},...\mathbf{f}_{I}^{l}\}$. For the image feature $\mathbf{f}_{B_{t,n}}$ of the single-view image $B_{t,n}$, we obtain the updated image feature $\mathbf{\tilde{f}}_{B_{t,n}}$ by feeding  $\mathbf{f}_{B_{t,n}}$ and $\mathbf{f}_{I}^{cls}$ to $A(\cdot)$:
\begin{equation}
\label{adapter}
\mathbf{\tilde{f}}_{B_{t,n}} = \alpha \cdot \mathbf{f}_{B_{t,n}} + (1-\alpha) \cdot A([\mathbf{f}_{B_{t,n}};\mathbf{f}_{I}^{cls}]),
\end{equation}
where $\alpha$ servers as the residual ratio to help adjust the degree of maintaining the original knowledge for better performance~\cite{Gao2021CLIPAdapterBV}, and $[\cdot;\cdot]$ denotes feature concatenation. Denote the top $k$ object concept features obtained by CLIP for the single-view image $B_{t,n}$ as $\{\mathbf{f}_{T_{i}}\}_{i=1}^{k}$. We use the updated image feature $\mathbf{\tilde{f}}_{B_{t,n}}$ to get the re-ranking object prediction probability $\mathbf{\tilde{p}}$ of $\{\mathbf{f}_{T_{i}}\}_{i=1}^{k}$:
\begin{equation}
\mathbf{\tilde{p}} = \mathrm{Softmax}(\mathrm{sim}(\mathbf{\tilde{f}}_{B_{t,n}},\mathbf{f}_{T_{1}}),...,\mathrm{sim}(\mathbf{\tilde{f}}_{B_{t,n}},\mathbf{f}_{T_{k}})).\\
\end{equation}
Then we get the refined concept feature $\mathbf{\tilde{u}}_{t,n}$ by replacing $\mathbf{p}$ in Eq.~\ref{eq:concept} by $\mathbf{\tilde{p}}$.


\subsection{Observation Co-Embedding}
\label{Observation Co-embedding}
After obtaining the concept feature $\mathbf{\tilde{u}}_{t,n}$ for each single-view observation $O_{t,n}$, we introduce an observation co-embedding module to use $\mathbf{\tilde{u}}_{t,n}$ for bridging multi-modal inputs and calculating the final observation feature $\mathbf{o}'_{t,n}$.
At first, we separately embed the visual feature $\mathbf{v}_{t,n}$, the direction feature $\mathbf{e}_{A_{t,n}}$, and the concept feature $\mathbf{\tilde{u}}_{t,n}$ to obtain  $\mathbf{o}_{t,n}^{v}$,  $\mathbf{o}_{t,n}^{a}$, and $\mathbf{o}_{t,n}^{u}$, respectively, by:
\begin{equation}
\label{separate}
\begin{aligned}
\mathbf{o}_{t,n}^{v}&=\mathrm{Dr}(\mathrm{LN}(\mathrm{LN}(\mathbf{\tilde{W}}_{v}\mathbf{v}_{t,n})+\mathbf{e}_{t,n}^{N}+\mathbf{e}_{v}^{T})),\\
\mathbf{o}_{t,n}^{a}&=\mathrm{Dr}(\mathrm{LN}(\mathrm{LN}(\mathbf{\tilde{W}}_{a}\mathbf{e}_{A_{t,n}})+\mathbf{e}_{t,n}^{N}+\mathbf{e}_{v}^{T})),\\
\mathbf{o}_{t,n}^{u}&=\mathrm{Dr}(\mathrm{LN}(\mathrm{LN}(\mathbf{\tilde{W}}_{u}\mathbf{\tilde{u}}_{t,n})+\mathbf{e}_{t,n}^{N}+\mathbf{e}_{v}^{T})),
\end{aligned}
\end{equation}
where $\mathbf{\tilde{W}}_{v}$, $\mathbf{\tilde{W}}_{a}$ and $\mathbf{\tilde{W}}_{u}$ are learnable weights. 

Unlike HAMT that combining different features into one embedding (Eq.~\ref{eq:compose}), we keep the separate embeddings as in Eq.~\ref{separate} such that a new observation contrast strategy can be performed.
Concretely, the view embedding $\mathbf{o}_{t,n}^{v}$ and the direction embedding $\mathbf{o}_{t,n}^{a}$ are summed as the visual embedding $\mathbf{o}_{t,n}^{V}=\mathbf{o}_{t,n}^{v}+\mathbf{o}_{t,n}^{a}$. Then $\mathbf{o}_{t,n}^{V}$ in each single-view observation $O_{t,n}$ is forced to stay close to the paired concept embedding $\mathbf{o}_{t,n}^{u}$ while staying far away from the concept embeddings $\overline{\mathbf{o}}_{t,n}^{u}$ in other single-view observations in $O_{t}$: 
\begin{equation}
\mathcal{L}_{\mathrm{c}}=-\sum_{t}\sum_{n}\mathrm{log}(\frac{\mathrm{e}^{\mathrm{sim}(\mathbf{o}_{t,n}^{V}, \mathbf{o}_{t,n}^{u})/\tau}}{\mathrm{e}^{\mathrm{sim}(\mathbf{o}_{t,n}^{V}, \mathbf{o}_{t,n}^{u})/\tau}+\sum\limits_{\overline{\mathbf{o}}_{t,n}^{u}}\mathrm{e}^{\mathrm{sim}(\mathbf{o}_{t,n}^{V}, \overline{\mathbf{o}}_{t,n}^{u})/\tau}}),
\end{equation}
where $\tau$ is the temperature parameter. By observation contrast, the discrimination of each single-view observation can be effectively enhanced and the semantic gap between observations and instructions can be largely mitigated with the help of the actional atomic concept. 
To fully merge the information for each observation, we use $\mathbf{o}'_{t,n}=\mathbf{o}_{t,n}^{V}+\mathbf{o}_{t,n}^{u}$ to obtain the final observation feature $\mathbf{o}'_{t,n}$.  


\subsection{Action Prediction}
Similar to the observation feature $\mathbf{o}'_{t}=\{\mathbf{o}'_{t,n}\}_{n=1}^{N_{o}}$ ($N_{o}$ is the number of views), the history feature $\mathbf{h}'_{t}$ is obtained for $H_{t}$ through AACL. 
With $\mathbf{o}'_{t}$, $\mathbf{h}'_t$, and the instruction feature  $\mathbf{f}_{I}$, the action $\mathbf{a}'_{t}$ can be obtained from the cross-modal transformer encoder $E^{c}(\cdot)$ and the action prediction module $E^{a}(\cdot)$ (see Eq.~\ref{eq:ec} and Eq.~\ref{eq:ea}).
Following most existing VLN works~\cite{tan2019learning,hong2021vln,Chen2021HistoryAM}, we combine Imitation Learning (IL) and Reinforcement Learning (RL) to train VLN agents. Let the imitation learning loss be $\mathcal{L}_{\mathrm{IL}}$ and the reinforcement learning loss be $\mathcal{L}_{\mathrm{RL}}$. The total training objective of AACL is:
\begin{equation}
\mathcal{L}=\mathcal{L}_{\mathrm{RL}}+\lambda_{1}\mathcal{L}_{\mathrm{IL}}+\lambda_{2}\mathcal{L}_{\mathrm{c}},
\end{equation}
where $\lambda_{1}$ and $\lambda_{2}$ are balance parameters.

\begin{table*}[!htb]
	\fontsize{7}{7}\selectfont


\caption{Comparison with the SOTA methods on R2R. }
	\vspace{-0.2cm}
	\label{tab:com with sota}
	\resizebox{1.0\linewidth}{!}{
	{\renewcommand{\arraystretch}{1.1}
		\begin{tabular}{c||c|c|c|c|c|c|c|c|c|c|c|c}

			\specialrule{.1em}{.05em}{.05em}
		
			\multirow{2}{*}{Method}&\multicolumn{4}{c|}{Val Seen }&\multicolumn{4}{c|}{Val Unseen}&\multicolumn{4}{c}{Test Unseen}\cr\cline{2-13}
			&TL&NE $\downarrow$&SR $\uparrow$&SPL $\uparrow$&TL&NE $\downarrow$&SR $\uparrow$&SPL $\uparrow$&TL&NE $\downarrow$&SR $\uparrow$&SPL $\uparrow$\cr
			\hline
			
       
        
       
		
		
            Seq2Seq~\cite{anderson2018vision}&11.33&6.01&39&-&8.39&7.81&22&-&8.13&7.85&20&18\\
           
            RCM+SIL(train) \cite{wang2019reinforced}&10.65&3.53&67&-&11.46&6.09&43&-&11.97&6.12&43&38\\
            
           
           
   
          EnvDropout~\cite{tan2019learning}&11.00&3.99&62&59&10.70&5.22&52&48&11.66&5.23&51&47\\ 
           
                         
        PREVALENT~\cite{hao2020towards} & 10.32 & 3.67 & 69 & 65 & 10.19 & 4.71 & 58 & 53 & 10.51 & 5.30 & 54 & 51 \\
          
           
            
           
          
           
		 ORIST~\cite{Qi2021TheRT}&-&-&-&-&10.90&4.72&57&51&11.31&5.10&57&52\\ VLN$\circlearrowright$BERT~\cite{hong2021vln}&11.13&2.90&72&68&12.01&3.93&63&57&12.35&4.09&63&57\\
 HOP~\cite{Qiao2022HOPHA}&11.51&2.46&76&70&12.52&3.79&64&57&13.29&3.87&64&58\\



		
		
   
		
		 \hline HAMT~\cite{Chen2021HistoryAM} (baseline)&11.15&2.51&76&72&11.46&3.62&66&61&12.27&3.93&65&60\\
   
   
   
   
     HAMT+AACL (ours)&11.31&2.53&76&72&12.09&3.41&69&\textbf{63}&12.74&3.71&66&\textbf{61}\\
   
   
    DUET~\cite{Chen2022ThinkGA} (baseline)&12.32&2.28&79&\textbf{73}&13.94&3.31&72&60&14.73&3.65&69&59\\
   
    DUET+AACL (ours)&13.32&\textbf{2.15}&\textbf{80}&72&15.01&\textbf{3.00}&\textbf{74}&61&15.47&\textbf{3.38}&\textbf{71}&59\\
 \specialrule{.1em}{.05em}{.05em}

		
		\end{tabular}}}

	\vspace{-0.2cm}
\end{table*}

\begin{comment}
\begin{table*}[!htb]
	\fontsize{11}{11}\selectfont


\caption{Navigation and object grounding performance on REVERIE val unseen and test splits.}
	\vspace{-0.2cm}
	\label{tab:com with sota on reverie}
	\resizebox{1.0\linewidth}{!}{
	{\renewcommand{\arraystretch}{1.2}
		\begin{tabular}{c||c|c|c|c|c|c|c|c|c|c|c|c}

			\specialrule{.1em}{.05em}{.05em}
		
			\multirow{2}{*}{Method}&\multicolumn{6}{c|}{Val Unseen}&\multicolumn{6}{c}{Test Unseen}\cr\cline{2-13}
			&TL&SR $\uparrow$&OSR$\uparrow$&SPL $\uparrow$&RGS$\uparrow$&RGSPL$\uparrow$&TL&SR $\uparrow$&OSR$\uparrow$&SPL $\uparrow$&RGS$\uparrow$&RGSPL$\uparrow$\cr
			\hline
			
       
        
       
		
		
           
            RCM~\cite{wang2019reinforced}&11.98&9.29&14.23&6.97&4.89&3.89&10.60&7.84&11.68&6.67&3.67&3.14\\
           
          SMNA~\cite{ma2019self}&9.07&8.15&11.28&6.44&4.54&3.61&9.23&5.80&8.39&4.53&3.10&2.39\\
         FAST-MATTN~\cite{qi2020reverie}&45.28&14.40&28.20&7.19&7.84&4.67&39.05&19.88&30.63&11.60&11.28&6.08\\ 
   
    SIA~\cite{Lin2021SceneIntuitiveAF}&41.53&31.53&44.67&16.28&22.41&11.56&48.61&30.80&44.56&14.85&19.02&9.20\\
            VLN$\circlearrowright$BERT~\cite{hong2021vln}&16.78&30.67&35.02&24.90&18.77&15.27&15.86&29.61&32.91&23.99&16.50&13.51\\
    HOP~\cite{Qiao2022HOPHA}&17.16&30.39&35.30&25.10&18.23&15.31&17.05&29.12&32.26&23.37&17.13&13.90\\
    TD-STP~\cite{Zhao2022TargetDrivenST}&-&34.88&39.48&27.32&21.16&16.56&-&35.89&40.26&27.51&19.88&15.40\\
 \hline HAMT~\cite{Chen2021HistoryAM} (baseline)&14.08&32.95&36.84&30.20&18.92&17.28&13.62&30.40&33.41&26.67&14.88&13.08\\ 
    
    
    HAMT+AACL (ours)&14.08&34.17&38.54&29.70&20.53&17.69&13.30&35.52&39.57&31.34&18.04&15.96\\
   
   
    DUET~\cite{Chen2022ThinkGA} (baseline)&22.11&46.98&51.07&\textbf{33.73}&32.15&\textbf{23.03}&21.30&52.51&56.91&36.06&31.88&22.06\\
   
    DUET+AACL (ours)&23.77&\textbf{49.42}&\textbf{53.93}&33.54&\textbf{33.31}&22.49&21.88&\textbf{55.09}&\textbf{59.92}&\textbf{37.08}&\textbf{33.17}&\textbf{22.55}\\
 \specialrule{.1em}{.05em}{.05em}

		
			\end{tabular}}}

	\vspace{-0.4cm}
\end{table*}
\end{comment}

\begin{table*}[!htb]
	\fontsize{20}{20}\selectfont


\caption{Navigation and object grounding performance on REVERIE.}
	\vspace{-0.2cm}
	\label{tab:com with sota on reverie}
	\resizebox{1.0\linewidth}{!}{
	{\renewcommand{\arraystretch}{1.2}
		\begin{tabular}{c||c|c|c|c|c|c|c|c|c|c|c|c|c|c|c|c|c|c}

			\specialrule{.1em}{.05em}{.05em}
		
			\multirow{2}{*}{Method}&\multicolumn{6}{c|}{Val Seen}&\multicolumn{6}{c|}{Val Unseen}&\multicolumn{6}{c}{Test Unseen}\cr\cline{2-19}
			&TL&SR $\uparrow$&OSR$\uparrow$&SPL $\uparrow$&RGS$\uparrow$&RGSPL$\uparrow$&TL&SR $\uparrow$&OSR$\uparrow$&SPL $\uparrow$&RGS$\uparrow$&RGSPL$\uparrow$&TL&SR $\uparrow$&OSR$\uparrow$&SPL $\uparrow$&RGS$\uparrow$&RGSPL$\uparrow$\cr
			\hline
			
       
        
       
		
		
           
            RCM~\cite{wang2019reinforced}&10.70&23.33&29.44&21.82&16.23&15.36&11.98&9.29&14.23&6.97&4.89&3.89&10.60&7.84&11.68&6.67&3.67&3.14\\
           
          SMNA~\cite{ma2019self}&7.54&41.25&43.29&39.61&30.07&28.98&9.07&8.15&11.28&6.44&4.54&3.61&9.23&5.80&8.39&4.53&3.10&2.39\\
         FAST-MATTN~\cite{qi2020reverie}&16.35&50.53&55.17&45.50&31.97&29.66&45.28&14.40&28.20&7.19&7.84&4.67&39.05&19.88&30.63&11.60&11.28&6.08\\ 
   
    SIA~\cite{Lin2021SceneIntuitiveAF}&13.61&61.91&65.85&57.08&45.96&42.65&41.53&31.53&44.67&16.28&22.41&11.56&48.61&30.80&44.56&14.85&19.02&9.20\\
            VLN$\circlearrowright$BERT~\cite{hong2021vln}&13.44&51.79&53.90&47.96&38.23&35.61&16.78&30.67&35.02&24.90&18.77&15.27&15.86&29.61&32.91&23.99&16.50&13.51\\
    HOP~\cite{Qiao2022HOPHA}&14.05&54.81&56.08&48.05&40.55&35.79&17.16&30.39&35.30&25.10&18.23&15.31&17.05&29.12&32.26&23.37&17.13&13.90\\
   
 \hline HAMT~\cite{Chen2021HistoryAM} (baseline)&12.79&43.29&47.65&40.19&27.20&25.18&14.08&32.95&36.84&30.20&18.92&17.28&13.62&30.40&33.41&26.67&14.88&13.08\\ 
    
    
    HAMT+AACL (ours)&13.01&42.52&46.66&39.48&28.39&26.48&14.08&34.17&38.54&29.70&20.53&17.69&13.30&35.52&39.57&31.34&18.04&15.96\\
   
   
    DUET~\cite{Chen2022ThinkGA} (baseline)&13.86&71.75&73.86&63.94&57.41&51.14&22.11&46.98&51.07&\textbf{33.73}&32.15&\textbf{23.03}&21.30&52.51&56.91&36.06&31.88&22.06\\
   
    DUET+AACL (ours)&14.54&\textbf{74.63}&\textbf{76.67}&\textbf{66.04}&\textbf{59.38}&\textbf{52.57}&\textbf{23.77}&\textbf{49.42}&\textbf{53.93}&33.54&\textbf{33.31}&22.49&21.88&\textbf{55.09}&\textbf{59.92}&\textbf{37.08}&\textbf{33.17}&\textbf{22.55}\\
 \specialrule{.1em}{.05em}{.05em}

		
			\end{tabular}}}

	\vspace{-0.4cm}
\end{table*}

\begin{comment}
\begin{table*}[!htb]
	\fontsize{5}{5}\selectfont


\caption{Comparison on R2R-Last dataset.}

	\label{tab:com with sota on r2rlast}
	\resizebox{1.0\linewidth}{!}{
	{\renewcommand{\arraystretch}{1.0}
		\begin{tabular}{c||c|c|c|c}

			\specialrule{.1em}{.05em}{.05em}
		
			\multirow{2}{*}{Method}&\multicolumn{2}{c|}{Val Seen}&\multicolumn{2}{c}{Val Unseen}\cr\cline{2-5}
			&SR$\uparrow$&SPL$\uparrow$&SR$\uparrow$&SPL$\uparrow$\cr
			\hline
			
        EnvDrop~\cite{tan2019learning}&42.8&38.4&34.3&28.3\\
        VLN$\circlearrowright$BERT~\cite{hong2021vln}&50.2&45.8&41.6&37.3\\
        \hline
        HAMT~\cite{Chen2021HistoryAM} (baseline)&53.3&50.3&45.2&41.2\\
        AACL&&&&\\
 \specialrule{.1em}{.05em}{.05em}

		
			\end{tabular}}}

	\vspace{-0.4cm}
\end{table*}
\end{comment}

\begin{comment}
\begin{figure*}[t]
\begin{centering}
\includegraphics[width=0.98\linewidth]{figures/visualization.pdf}
\par\end{centering}
\vspace{-0.2cm}
\caption{\label{fig:visualization}Visualization of the action selections by the baseline method~\cite{Chen2021HistoryAM} and our AACL. The green boxes denote the correct actions and the red boxes denote the wrong ones. 
}
\vspace{-0.4cm}
\end{figure*}
\end{comment}
\begin{comment}
\begin{figure*}[!htb]
\begin{centering}
\includegraphics[width=0.98\linewidth]{figures/visualization_2.pdf}
\par\end{centering}
\vspace{-0.2cm}
\caption{\label{fig:visualization_2}Visualization of the object predictions by our AACL and CLIP. 
}
\vspace{-0.6cm}
\end{figure*}
\end{comment}



\section{Experiments}
\subsection{Experimental Setup}
\label{Experimental Setup}
\textbf{Datasets.} We evaluate the proposed AACL on several popular VLN benchmarks with both fine-grained instructions (R2R~\cite{anderson2018vision}) and high-level instructions (REVERIE~\cite{qi2020reverie} and R2R-Last~\cite{Chen2021HistoryAM}). R2R includes 90 photo-realistic indoor scenes with 7189 trajectories. The dataset is split into train, val seen, val unseen, and test unseen sets with 61, 56, 11, and 18 scenes, respectively. REVERIE replaces the fine-grained instructions in R2R with high-level instructions which mainly target at object localization. R2R-Last only uses the last sentence of the original R2R instruction as the instruction. 


\begin{table}

    \caption{Comparison on R2R-Last.}
    \vspace{-0.2cm}
    \small
    \centering
    \resizebox{\linewidth}{!}{
    \begin{tabular}{l|cccc}
    \specialrule{.1em}{.05em}{.05em}
		
			\multirow{2}{*}{Method}&\multicolumn{2}{c|}{Val Seen}&\multicolumn{2}{c}{Val Unseen}\cr\cline{2-5}
			&SR$\uparrow$&SPL$\uparrow$&SR$\uparrow$&SPL$\uparrow$\cr
			\hline
			
        EnvDrop~\cite{tan2019learning}&42.8&38.4&34.3&28.3\\
        VLN$\circlearrowright$BERT~\cite{hong2021vln}&50.2&45.8&41.6&37.3\\
        \hline
        HAMT~\cite{Chen2021HistoryAM} (baseline)&53.3&50.3&45.2&41.2\\
        HAMT+AACL (ours)&\textbf{54.2}&\textbf{51.1}&\textbf{47.2}&\textbf{42.1}\\
 \specialrule{.1em}{.05em}{.05em}
    \end{tabular}
}
\label{tab:com on r2rlast}
\vspace{-0.2cm}
\end{table}

\begin{table}
\fontsize{4}{4}\selectfont
    \caption{Ablation Study on R2R. The baseline agent we choose is HAMT~\cite{Chen2021HistoryAM}.}
\vspace{-0.2cm}
    \small
    \centering
    \resizebox{0.7\linewidth}{!}{
    {\renewcommand{\arraystretch}{1.0}
    \begin{tabular}{l|ccc}
    \specialrule{.1em}{.05em}{.05em}
		
			\multirow{2}{*}{Method}&\multicolumn{3}{c}{Val Unseen}\cr\cline{2-4}
   &NE$\downarrow$&SR$\uparrow$&SPL$\uparrow$\cr

			\hline
			
       
        separate embedding&3.66&66.67&61.19\\
       w/o contrast&3.42&67.94&61.48\\ 
       w/o refine&3.45&67.82&62.33\\
            full model&\textbf{3.41}&\textbf{68.54}&\textbf{62.96}\\
        
 \specialrule{.1em}{.05em}{.05em}
    \end{tabular}
}}
\label{tab:ablation}
\vspace{-0.4cm}
\end{table}



\textbf{Evaluation Metrics.} We adopt the common metrics used in previous works~\cite{Chen2021HistoryAM,anderson2018vision,qi2020reverie} to evaluate the model performance: 1) Navigation Error (NE) calculates the average distance between the agent stop position and the target viewpoint, 2) Trajectory Length (TL) is the average path length in meters, 3) Success Rate (SR) is the ratio of stopping within 3 meters to the goal, 4) Success rate weighted by Path Length (SPL) makes the trade-off between SR and TL, 5) Oracle Success Rate (OSR) calculates the ratio of containing a viewpoint along the path where the target object is visible, 6) Remote Grounding Success Rate (RGS) is the ratio of performing correct object grounding when stopping, and 7) Remote Grounding Success weighted by Path Length (RGSPL) weights RGS by TL. 1)--4), 3)--4), and 2)--7) are used for evaluation on R2R, R2R-Last, and REVERIE, respectively. 

\textbf{Baselines.} 
In this work, we choose two strong baseline agents, HAMT~\cite{Chen2021HistoryAM} and DUET~\cite{Chen2022ThinkGA} to verify AACL's effectiveness. In HAMT, a hierarchical transformer is adopted for storing historical observations and actions. In contrast, DUET keeps track of all visited and navigable locations through a topological map. 

\textbf{Implementation Details.} 
We implement our model using the MindSpore Lite tool~\cite{mindspore}.
The batch size is set to 8, 8, 4 on R2R, R2R-Last, and REVERIE, respectively. The temperature parameter $\tau$ is set to 0.5. 
The loss weight $\lambda_{1}$ is set to 0.2 on all datasets, and the loss weight $\lambda_{2}$ is set to 1, 1, and 0.01 on R2R, REVERIE, and R2R-Last, respectively.
The residual ratio in Eq.~\ref{adapter} is set to 0.8 empirically. During object concept mapping, we remain top 5 object predictions for each observation.
The learning rate of the concept refining adapter is set to 0.1.  

\begin{figure*}[!htb]
\begin{centering}
\includegraphics[width=0.98\linewidth]{visualization_3.pdf}
\par\end{centering}
\caption{\label{fig:visualization_3}Visualization examples of action selection ((a) and (b)) and object concept mapping ((c)). In (a) and (b), the baseline is HAMT~\cite{Chen2021HistoryAM}. The green boxes denote the correct actions and the red boxes denote the wrong ones.
}
\vspace{-0.6cm}
\end{figure*}


\subsection{Quantitative Results}



\textbf{Comparison with the State-of-the-Arts (SOTAs).} Table~\ref{tab:com with sota}\footnote{The original value 2.29 of NE under Val Unseen in HAMT is a typo, which is actually 3.62 confirmed by the author.}, Table~\ref{tab:com with sota on reverie}, and Table~\ref{tab:com on r2rlast} present the performance comparison between the SOTA methods and AACL, where we can find that AACL establishes new state-of-the-art results in most metrics on R2R, REVERIE and R2R-Last. These results show that AACL is useful not only when the instructions are fine-grained but also when the instruction information is limited, demonstrating that the proposed actional atomic concepts can effectively enhance the observation features, simplify their alignment to the linguistic instruction features, and therefore improve the agent performance. Moreover, we can find that AACL consistently outperforms the two strong baselines on these three benchmarks especially under Unseen scenarios, showing that AACL can be used as a general tool for the multi-modal alignment. 






\begin{comment}
\begin{table}[!t]
\begin{minipage}{.5\linewidth}
\centering
\caption{Comparison on R2R-Last dataset.}
\resizebox{0.98\linewidth}{!}{{
\setlength{\tabcolsep}{0.6em}
    {\renewcommand{\arraystretch}{1.1}
    \begin{tabular}{l|ccc}
    \specialrule{.1em}{.05em}{.05em}
		
			Method&NE$\downarrow$&SR$\uparrow$&SPL$\uparrow$\cr

			\hline
			
        baseline&3.64&65.94&60.44\\
        separate embedding&3.66&66.67&61.19\\
       AACL-w/o contrastive&3.42&67.94&61.48\\ 
       AACL-w/o refine&3.45&67.82&62.33\\
            AACL-Full&\textbf{3.41}&\textbf{68.54}&\textbf{62.96}\\
        
 \specialrule{.1em}{.05em}{.05em}
    
    

    \end{tabular}}}
}
\vspace{0.2em}

\label{tab:com on r2rlast}
\end{minipage}
\begin{minipage}{.5\linewidth}
\centering
\caption{Ablation Study on R2R dataset.}
\resizebox{0.88\linewidth}{!}{{
\setlength{\tabcolsep}{0.6em}
    {\renewcommand{\arraystretch}{1.1}
    \begin{tabular}{l|ccc}
    \specialrule{.1em}{.05em}{.05em}
		
			Method&NE$\downarrow$&SR$\uparrow$&SPL$\uparrow$\cr

			\hline
			
        baseline&3.64&65.94&60.44\\
        separate embedding&3.66&66.67&61.19\\
       AACL-w/o contrastive&3.42&67.94&61.48\\ 
       AACL-w/o refine&3.45&67.82&62.33\\
            AACL-Full&\textbf{3.41}&\textbf{68.54}&\textbf{62.96}\\
        
 \specialrule{.1em}{.05em}{.05em}
    

    \end{tabular}}}
}
\vspace{0.2em}

\label{tab:ablation}
\end{minipage}

\end{table}
\end{comment}








\textbf{Ablation Study.} Table~\ref{tab:ablation} gives the ablation study of AACL. ``separate embedding'' means using the separate embedding scheme (Eq.~\ref{separate}) only for the visual feature and the directional feature. By comparing the results between ``separate embedding'' and ``w/o contrast'', we can find that the direct introduction of actional atomic concepts under the separate embedding strategy can already improve the navigation performance (1.27\% increase on SR), showing their effectiveness for enhancing the observation features. By comparing the results between ``w/o contrast'' and ``w/o refine'', we can observe that the proposed observation contrast strategy can effectively regularize the observation representation and improve the performance (0.85\% increase on SPL). The comparison between ``w/o refine'' and ``full model'' further shows the effectiveness of the concept refining adapter, demonstrating that the instruction-oriented object concept extraction can facilitate better cross-modal alignment. 

\begin{comment}
\begin{table}[!htb]
\begin{minipage}{.5\linewidth}
\centering
\caption{Comparison of different object numbers.}
\resizebox{0.68\linewidth}{!}{{
\setlength{\tabcolsep}{0.5em}
    {\renewcommand{\arraystretch}{1.0}
    \begin{tabular}{l|ccc}
        \specialrule{.1em}{.05em}{.05em}
		
			Method&NE$\downarrow$&SR$\uparrow$&SPL$\uparrow$\cr
    	\hline
			
        $k$=1&3.52&67.52&\textbf{62.24}\\
        $k$=5&\textbf{3.42}&\textbf{67.94}&61.48\\
        $k$=10&3.47&66.88&60.99\\
        
 \specialrule{.1em}{.05em}{.05em}
    \end{tabular}}}
}
\vspace{0.2em}
\label{tab:comparison on object number}
\end{minipage}
\begin{minipage}{.5\linewidth}
\centering
\caption{Comparison with random  concepts.}
\resizebox{0.78\linewidth}{!}{{
\setlength{\tabcolsep}{0.5em}
    {\renewcommand{\arraystretch}{1.0}
    \begin{tabular}{l|ccc}
        \specialrule{.1em}{.05em}{.05em}
		
			Method&NE$\downarrow$&SR$\uparrow$&SPL$\uparrow$\cr

    	\hline
			random object&3.60&66.37&61.33\\
			selected object&3.50&66.50&61.39\\
			random action&3.50&66.71&60.75\\
			selected action&\textbf{3.42}&\textbf{67.94}&\textbf{61.48}\\
        
 \specialrule{.1em}{.05em}{.05em}
    \end{tabular}}}
}
\label{tab:comparison w random}
\end{minipage}
\vspace{-0.4cm}
\end{table}
\end{comment}


\subsection{Qualitative Results}
Figure~\ref{fig:visualization_3} visualizes some results of action decision and object concept mapping. We can find that by introducing the actional atomic concepts, the agent is able to perform better cross-modal alignment for improving action decisions. In Figure~\ref{fig:visualization_3}(a), although the candidate observations do not contain the visual appearance of ``kitchen'', with the help of the actional atomic concepts,  AACL successfully chooses the right action whose paired action concept matches the one mentioned in the instruction (``turn right''), while the baseline selects the wrong one. From the  instruction attention comparison in the lower part of Figure~\ref{fig:visualization_3}(a), we can also observe that AACL attends to the right part (``right'') of the instruction at step 2, while the baseline attends to the wrong part (``and'').  In Figure~\ref{fig:visualization_3}(b), within the actional atomic concept, AACL successfully chooses the correct action asked in the instruction. In Figure~\ref{fig:visualization_3}(c), we can observe that the probability of ``kitchen'' of AACL is higher than that of CLIP for the GT action (top-1 vs. top-4), showing that the concept refining adapter enables more instruction-oriented object concept extraction, which is useful for selecting correct actions. 




\section{Conclusion}
In this work, we propose Actional Atomic-Concept Learning, which helps VLN agents demystify the alignment in VLN tasks through actional atomic concepts formed by language. 
During navigation, each visual observation is mapped to the specific actional atomic concept through the VLN environment and CLIP. A concept refining adapter is constructed to enable the instruction-oriented concept extraction. An observation co-embedding module is introduced to use concept features to regularize observation features.  
Experiments on public
VLN benchmarks show that AACL achieves new SOTA results. Benefiting from these human-understandable actional atomic concepts, AACL shows excellent interpretability in making action decision. 




\clearpage

\section{Acknowledgements}
This work was supported in part by National Key R\&D Program of China under Grant No. 2020AAA0109700, National Natural Science Foundation of China (NSFC) under Grant No.61976233, Guangdong Outstanding Youth Fund (Grant No. 2021B1515020061), Guangdong Natural Science Foundation under Grant 2017A030312006, Shenzhen Fundamental Research Program (Project No. RCYX20200714114642083, No. JCYJ20190807154211365), and the Fundamental Research Funds for the Central Universities, Sun Yat-sen University under Grant No. 22lgqb38. 
We gratefully acknowledge the support of MindSpore, CANN (Compute Architecture for Neural Networks), and Ascend AI Processor used for this research.

