\section{Introduction}
\label{sec:intro}

Monocular 3D human pose estimation aims to accurately regress the 3D locations of human joints in the camera coordinate system from a single image. It plays an important role in many applications, such as action recognition and human-computer interaction. Compared to the monocular systems, multi-view capture systems are expensive and inconvenient to set up and operate, which prevents them from being widely used in practice.
To tackle the monocular 3D HPE task, some approaches~\cite{cai2019exploiting, lee2018propagating, sun2017compositional, sun2018integral, yang20183d} regress 3D joint coordinates or heat maps directly from an image via a convolutional neural network (CNN)~\cite{krizhevsky2017imagenet, lecun1998gradient}. Directly regressing from the image space suffers from the large parameter searching space, which always leads to a sub-optimal solution. 
Recently, Martinez et al.~\cite{martinez2017simple} construct a simple fully connected network using only 2D keypoints as input and achieve promising 3D HPE performance, showing that by using 2D joint positions, the 3D human pose can also be efficiently and accurately estimated. Inspired by them and considering the thoroughly investigated 2D HPE, many works decompose the problem into two subtasks, i.e., the 2D HPE and 2D-to-3D pose lifting~\cite{liu2020comprehensive, zeng2021learning, zou2021modulated, xu2021graph}. Therewith, 2D-to-3D pose lifting springs up as a fundamental problem in this area that our work devotes to. 

However, human skeleton topology is inherently sparse and graph-structured. Fully connected neural networks are less effective in modeling graph-structured data due to their simple connections among all nodes and the probability of over-fitting. To leverage the information of the human skeletal topology, some works~\cite{ci2019LCN, xu2021graph, zeng2021learning, zou2021modulated} propose to model the human body with GCN and have achieved SOTA results. 
For example, Ci et al.~\cite{ci2019LCN} introduced a locally connected network to enhance the representation capability of GCN. Liu et al.~\cite{liu2020comprehensive} explored the weight sharing and feature transformation that occurs before or after feature aggregation in GCN. Nonetheless, existing GCN-based 3D HPE methods update the node features by aggregating their neighbors' information without considering the different contributions of those nodes in different motion patterns.

Instead of considering all the joints of a skeleton as a whole, Xue et al.~\cite{xue2022partaware} demonstrated that the human skeleton exhibits obvious part-wise inconsistency of motion patterns, which is also reported in SRNet~\cite{zeng2020srnet}. However, these works are limited to priory structural information of limb groupings and ignore investigating the latent groups underlying human joint synergies. Besides bio-connection between joints, other complex synergies exist between joints, such as maintaining balance. Moreover, joints in a limb are utilized as a whole to calculate the relationship with other body parts in previous works, which results in a lower accuracy of those peripheral joints, such as wrists and feet. 

To address the problems mentioned above in monocular 3D HPE, we propose a novel architecture: Hop-wise GraphFormer with Intragroup Joint Refinement (HopFIR).
The first key component of HopFIR is a novel Hop-wise GraphFormer (HGF) module to employ k-hop neighbors. In HGF, the information of every hop of every joint is aggregated into the hidden space, i.e. we get $N \times k$ groups of features for a skeleton model with $N$ joints. Meanwhile, a hop-wise transformer-like attention mechanism is designed to extract the correlation among feature groups, which computes the similarity by the dot product of the node feature and the group feature. The proposed HGF enables the network to discover latent joint interactions in human joint synergy. Because HGF leverage less prior information about the human body and ignores the interaction among joints in a limb, especially the interaction of peripheral joints associated with a limb. In this regard, we introduced an Intragroup Joint Refinement (IJR) module to strengthen the correlation of intra-group joints grouped by limb prior information. Moreover, a residual block, the HopFIR block is built by two HGF modules, followed by one IJR module. The proposed architecture achieves optimal regression accuracy with a stack of three HopFIR blocks.

In conclusion, our work makes the following contributions:

$\bullet$ To the best of our knowledge, we design the first Hop-wise GraphFormer module to explore potential joint correlations underlying human joint synergy. We also proved that other GCN-based methods can benefit from the proposed HGF module efficiently as shown in Fig.~\ref{fig:overview}.

$\bullet$ We design an Intragroup Joint Refinement module, which attends intragroup joints to refine joint features through the associated limb, especially the wrists and feet. The IJR modules promote HGF modules to discover the latent synergies among joints.

$\bullet$ We proposed a novel Hop-wise GraphFormer with Intragroup Joint Refinement (HopFIR) for 3D HPE, which is built entirely with HGF modules and IJR modules. Specifically, two HGF modules and one IJR module are coupled into a HopFIR block.

$\bullet$ Extensive experiments demonstrate the effectiveness and generalization ability of the proposed modules and HopFIR architecture by providing new state-of-the-art results on two challenging datasets, i.e., Human3.6M~\cite{ionescu2013h36m} and MPI-INF-3DHP~\cite{mehta20173dhp}.


\section{Related Work}
\label{sec:related work}

\textbf{3D Human Pose Estimation.} 
Early works~\cite{ramakrishna2012reconstructing, sminchisescu20083d} used handcrafted features, perspective relationships, and geometric constraints to estimate the 3D human pose. 
Recent pose estimation approaches can be generally divided into two categories. 
The first category of networks regresses 3D human joints from the image directly~\cite{pavlakos2017coarse, zhou2017towards}. Pavlakos et al.~\cite{pavlakos2017coarse} exploited CNN to predict voxel-wise likelihoods for each joint. Zhou et al.~\cite{zhou2017towards} directly embedded a kinematic object model into the networks to learn the general multi-articulate object pose.
Approaches in the second category
decouple the 3D HPE task into 2D pose estimation from an image and 3D pose estimation from the detected 2D joints (2D-to-3D). For example, Martinez et al.~\cite{martinez2017simple} proposed a simple yet effective baseline with fully-connect networks and prove that 3D human poses can be regressed simply and effectively from 2D keypoints. Our paper follows this pipeline and focuses on the 2D-to-3D pose lifting. 
Instead of treating all joints of a skeleton as a whole, grouping the joints with consideration of their interactions is crucial in promoting the 3D human pose regression accuracy.
Xue et al.~\cite{xue2022partaware} divided the human skeleton graph into five groups according to the limbs to explore part-wise motion inconsistency.
Zeng et al.~\cite{zeng2020srnet} split the human joints into local regions and recombine the global information from the rest of the joints.
Different from the approaches mentioned above, HopFIR groups joints by k-hop of each joint and prior limb information, which enables the network to discover latent connections between groups in different human joint synergy.

\textbf{Graph Neural Network.}
GCNs~\cite{defferrard2016convolutional, gori2005new, kipf2016semi, scarselli2008graph} generalize the capability of CNNs by performing convolution operations on graph-structured data. Existing GCNs can be divided into two categories: the spectral-based approaches~\cite{defferrard2016convolutional} and the spatial-based approaches~\cite{kipf2016semi}.
Our approach falls into the second category, which applies message-passing operations on the graph nodes and their neighbors.

Due to the graph-structure topology of the human skeleton, many works~\cite{ci2019LCN, liu2020comprehensive} introduce GCN to tackle the 3D HPE task. 
Zhao et al.~\cite{zhao2019semantic} proposed a SemGCN to learn the semantic relationships between human joints.
Zou et al.~\cite{zou2021modulated} proposed a weight modulation and an affinity modulation based on SemGCN.
These above methods aggregate the first-order neighborhood messages to update the feature matrix by assigning different weights for different nodes. 
Some works~\cite{zhu2021posegtac, zou2020highorder} extend the first-order neighbors to high-order neighbors in the spatial domain directly.
Zeng et al.~\cite{zeng2021learning} designed a hierarchical fusion block by dividing the fusion procedure into two stages, where all the high-order neighbors of a node are aggregated into a feature in the first stage and fuse it with the node feature and the first-order neighbor in the second stage.
Zhao et al.~\cite{zhao2022graformer} introduce Chebyshev graph convolution to fuse information among the k-hop neighbors of a joint directly.
All of them update features by aggregating each node's own k-hop neighborhood information in a GCN layer. However, HopFIR considers the k-hop groups of all nodes to reconstruct the k-hop feature of a node through the proposed attention mechanism, which can enhance the representation capability of GCNs. 

\textbf{Graph Attention.}
Graph attention networks~\cite{velivckovic2017gat} is a pioneer work that pays attention to the data in graph structure by assigning attention weight for each node. Recently, transformer~\cite{vaswani2017attention} is proposed for machine translation tasks, proving the capacity of attention on sequence input. ViT~\cite{dosovitskiy2020vit} introduces the Transformer into computer vision and achieves excellent performance. Inspired by them, some researchers further introduce the Transformer for 3D HPE.
Zheng et al.~\cite{zheng2021poseformer} designed a spatial-temporal model to model the human joint relations, which treats each 2D pose as an input token in Temporal Transformer Module and considers each joint as a patch in Spatial Module.
Xue et al.~\cite{xue2022partaware} presented the temporal transformer with a time window and the multi-head attention between the limb group and its part-wise dictionary feature.
Zhao et al.~\cite{zhao2022graformer} applied self-attention to capture global information by calculating the similarity of all nodes. 

Different from the existing graph attention mechanism for 3D HPE, we propose the intergroup multi-head attention mechanism among the k-hop groups of all nodes, which assign the attention weights by computing the similarity between the node feature and k-hop group feature. Moreover, we introduce the intragroup multi-head self-attention in limb groups to refine the joint features and promote the HGF to discover the latent synergies among joints.

\section{The proposed HopFIR}
\label{sec:formatting}
This paper proposes a novel architecture to regress 3D human pose from $N$ given 2D keypoints $X\in\mathbb{R}^{N\times2}$. The proposed framework mainly consists of Hop-wise GraphFormer (HGF) modules and Intragroup Joint Refinement (IJR) modules. In the following, we first review the vanilla Graph Convolutional Network and Transformer in Sec.~\ref{subsec:vanilla}. Then, the HGF module and IJR module are introduced in Sec.~\ref{subsec:HGF} and Sec.~\ref{subsec:IJR}, respectively. Finally, we present the network architecture in Sec.~\ref{subsec:architecture}.
\begin{figure}[t]
  \centering
 
   \includegraphics[width=1\linewidth]{group3.pdf}
   \caption{The illustration of the human skeletal graph and the groups in HGF and IJR modules. The k-hop neighbors of a joint are set as a group in HGF modules where (a) indicates the 1-hop groups of different joints and (b) indicates k-hop groups of joint 9. (c) indicates the joints grouped by prior limb information.
   }
   \label{fig:group}
\end{figure}

\subsection{Vanilla GCN and Transformer}
\label{subsec:vanilla}
\textbf{GCN.} A graph is defined as $\mathcal{G} =(\mathcal{V}, \mathcal{E})$ where $\mathcal{V}$ is a set of N nodes and $\mathcal{E}$ is the adjacency matrix representing edges between the nodes. Given a collection of input features $H^l \in \mathbb{R}^{N \times D}$,
a generic GCN layer that aggregates neighborhood information can be formulated as follows:
\begin{equation}
\label{e1}
    H^{(l+1)}=\sigma(\tilde{A} H^l W)
\end{equation}
where $W\in\mathbb{R}^{D \times D'}$is the learnable weight matrix which transforms the feature dimension from $D$ to $D'$, 
and $\sigma(.)$ is the activation function, such as ReLU~\cite{nair2010relu}. $H^{(l+1)}$ is the updated feature matrix, 
$\tilde{A}\in \mathbb{R}^{N\times N}$ is the symmetrically normalized affinity matrix~\cite{kipf2016semi} with added self-connections,
$A \in \{0,1\}^{N \times N}$ is the adjacency matrix. The $(i, j)th$ entry $a_{ij}=1$ represent node $j$ is the neighbor of node $i$. Otherwise, they are not connected and $a_{ij}=0$. Therefore, the none-neighbor nodes have a weak influence on each other in vanilla GCN, which hinders the modeling of underlying motion connections in 3D HPE. 

\textbf{Transformer.} Transformer architecture relies entirely on self-attention to compute representations of its input and output. The self-attention function maps the inputs to the queries $Q$, keys $K$, and values $V$ by weight matrices $W_Q$, $W_K$, and $W_V$, respectively, and the matrix of outputs is calculated as:
\begin{equation}
    Attention(Q, K, V)=Softmax(QK^T/\sqrt{d})V
\end{equation}
where $d$ is the feature dimension of $Q$, and $\frac{1}{\sqrt{d}}$ is a scaling factor to prevent extremely small gradients.
The multi-head attention which performs self-attention in parallel projects the queries, keys, and values $P$ times with different linear projections to each subspace, respectively.
\begin{equation}
    MultiHead(Q, K, V) = Concat(D_1,...,D_P)W_{o} 
\end{equation}
where $W_O$ is the projection matrix of outputs, $D_p = Attention(QW_Q^p, KW_K^p, VW_V^p)$, and $p\in [1,..,P].$ 

\subsection{Hop-wise GraphFormer}
\label{subsec:HGF}
Previous GCNs studies just aggregate multi-hop neighborhood information~\cite{zeng2021learning, zou2020highorder} or assign an attention weight to each one-order neighbor~\cite{zhao2019semantic,zou2021modulated} for 3DHPE. To effectively capture the node's neighborhood message and increase the representational capacity of GCNs, we introduce the hop-wise GraphFormer (HGF) module which treats each hop as a group (Fig.\ref{fig:group} a,b) and computes the attention weights for each hop. By considering the relationship within $k$ hops, we can obtain $N \times k$ groups for a skeleton graph with $N$ joints, which provides enough combinations of joints for discovering the latent correlations among joints in different human joint synergy.

We first define the k-hop matrix $A^k$ as: 
\begin{equation}
    a^k_{ij} = \begin{cases}
    1, &d(v_i, v_j)=k\\
    0, &otherwise
    \end{cases}
\end{equation}
where $d(v_i, v_j)$ denotes the distance of the shortest path between $v_i$ and $v_j$ on the skeleton graph.
The k-hop neighborhood information is aggregated with a weighted sum of the target node's k-hop neighbors which is named HopGCN:
\begin{equation}
\label{e5}
    s^k_i = \sum_{j}a^k_{ij}h_jW^k
\end{equation}
where $s^k_i \in \mathbb{R}^{D}$ is a hidden representation of the k-hop neighborhood information.
Eq.~\ref{e5} is similar to Eq.~\ref{e1} but with an extended definition of $A$ to $k$ hops. The weight matrix $W$ is assigned to each hop respectively. 

Before aggregating hidden representation $s^k_i$ to the target node, we propose a transformer-like attention mechanism computing the similarity between the node feature $h_i$ and k-hop hidden representation, as shown in Eq.~\ref{e7}:
\begin{equation}
\label{e7}
    z^k_i=\sum_jsoftmax_j(\frac{h_i^Ts^k_j}{\sqrt{d}})s^k_j.
\end{equation}
Due to the structure of the human skeleton and the joint synergy, the k-hop neighborhood of joint $i$ is related to that of other joints. For example, the synergy of hop $S_5^1$ is related to hop $S_6^1$ and $S_3^1$, as shown in Fig.~\ref{fig:group}. By packing together $z^k_i$, $h_i$ and $s^k_i$ into matrices $Z^k$, $H$ and $S^k$, $Z^k$ can be calculated in parallel:
\begin{equation}
    Z^k=softmax(\frac{H{S^k}^T}{\sqrt{d}})S^k
\end{equation}
Note that one can also reconstruct $Z^k$ using the self-attention mechanism purely on node features or hop features. The performance comparison of these cases is discussed in the experimental results.

We further reduce the dimension of hidden representation by a fully connected layer based on the order of the neighborhood, considering different amounts of information related to the target node, as formulated in Eq.~\ref{eq:fc}.
\begin{equation}\label{eq:fc}
    r^k_i=F^k(z^k_i)
\end{equation}
where $r^k_i$ is the refined representation of the k-hop neighborhood information with respect to node $i$, and $F^k$ is the mapping function. Then, the refined representation is concatenated to the updated node feature. 

All the refined representations of hop-wise neighborhoods and the target node feature $h_i$ are encoded to a D-dimensional vector:
\begin{equation}
\label{e10}
    h'_{i}=F(h_i, r_i^1, r_i^2, r_i^3, ..., r_i^k)  
\end{equation}
where $h'_i$ is the final updated feature of node $i$ in this layer, and $F$ is the aggregation function. 
Packing together the updated features of all nodes, Eq.~\ref{e10} can be rewritten as:
\begin{equation}
    H' = F(H, R^1, R^2, R^3, ..., R^k).
\end{equation}
Thus, we get the core layer of the HGF module, the Hop-wise attention(HA) layer, which extracts latent correlations between feature groups and aggregates k-hop neighborhood information.
With the increment of $k$,
we get more $N$ groups to explore the motion patterns underlying those groups.
One can adjust $k$ based on the tasks and pipelines. In HopFIR architecture, three hops are enough to achieve optimal performance according to the experimental results.

To fuse the current global information and original 2D information, we concatenate all the joint features in a batch and feed them into a fully connected layer to extract current global information. Then we concatenate the extracted global information, 2D information, and the output of HA on the feature dimension of the joint feature to fuse all the information by another fully connected layer.
\begin{figure*}[t]
  \centering
 
  \includegraphics[width=0.9\linewidth]{framework11.pdf}
   \caption{The HopFIR architecture and the details of HGF modules and IJR modules. The HopFIR block is a residual block~\cite{he2016resnet} built by two HGF modules, followed by one IJR module. The proposed architecture with three blocks achieves optimal performance.}
   \label{fig:framework}
\end{figure*}

\subsection{Intragroup Joint Refinement}
\label{subsec:IJR}
The proposed HGFs split the skeleton graph into different groups based on the k-hop neighborhood of each joint, which pays attention to the key groups in different joint synergies. However, 
HGF leverages less prior information about the human body and ignores the interaction among joints in a limb, 
especially the interaction of peripheral joints associated with limbs, such as wrists and feet. 
In this regard, we introduce an Intragroup Joint Refinement (IJR) module to strengthen the correlation of intra-group joints grouped by limb prior information as shown in Fig.~\ref{fig:group}(c). The HGF features of joints in each limb group are refined by using multi-head self-attention~\cite{vaswani2017attention} in the IJR modules, as denoted in Eq.~\ref{eq:mhsa}.
\begin{equation}
\label{eq:mhsa}
    H_g = MHSA(MHSA(H)_g)
\end{equation}
where $H$ is the feature matrix from HGF module, $MHSA(H)_g$ is the feature matrix of the group $g$ updated by global multi-head self-attention, and $H_g$ is the final feature matrix of the group $g$ updated by IJR module. More details of the IJR module are shown in Fig.~\ref{fig:framework}.
\subsection{HopFIR Architecture}
\label{subsec:architecture}
The HopFIR architecture consists of the proposed Hop-wise GraphFormer (HGF) modules and Intragroup Joint Refinement (IJR) modules, as illustrated in Fig.~\ref{fig:framework}. The residual HopFIR block which contains two HGFs and one IJR module is designed as the basic block in HopFIR.
In the HGF module, the input is first normalized by a layer normalization~\cite{ba2016ln}. Then, the operations described in Sec.~\ref{subsec:HGF} is performed through HopGCN, HA, FC and information fusion in the HGF module, respectively. 
After a batch normalization~\cite{ioffe2015bn} and a PReLU~\cite{he2015prelu} activation function, the output is residual connected with the normalized input from the layer normalization. 
Moreover, we define an embedding layer to map the input to the latent space and a HG module to transform the output into 3D space. 
The HG module is a variant of HGF designed for the output layer, more details are shown in the supplementary material.
The whole HopFIR accepts 2D keypoints as the input, which can be obtained via an off-the-shelf 2D detector.

The graph is obtained by adding a normalized globally learnable k-hop graph to the skeleton graph and then symmetrically normalizing it like~\cite{zou2021modulated}.
We use the L1-norm loss and L2-norm loss to compute the error between ground truth and prediction with the weighed summation as follows:
\begin{equation}
    L = \alpha \sum_{n=1}^N \Vert Y_n - \hat{Y_n} \Vert_2 + \beta \sum_{n=1}^N \Vert Y_n - \hat{Y_n} \Vert_1
\end{equation}
where N is the joint number, $\hat{Y_n}$ is the predicted 3D position of joint $n$, $Y_n$ is the ground truth, $\alpha$ = 1 and $\beta$ = 0.1.

\begin{table*}[t]
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{\linewidth}{!}{
    \begin{tabular}{l c c c c c c c c c c c c c c c c}
        \toprule
        \textbf{Protocol \#1} & Dir. & Disc. & Eat & Greet & Phone & Photo & Pose & Pur. & Sit & SitD. & Smoke & Wait & WalkD. & Walk & WalkT. & Avg.\\
        \midrule\midrule
        Martinez et al.~\cite{martinez2017simple} ICCV'2017& 51.8&  56.2&  58.1&  59.0& 69.5 & 78.4 & 55.2 & 58.1 & 74.0 & 94.6 & 62.3 & 59.1 & 65.1 & 49.5 & 52.4 & 62.9\\ 
       
       
       
       
        Zhao et al.~\cite{zhao2019semantic} CVPR'2019& 47.3 & 60.7 & 51.4 & 60.5 & 61.1 &\textbf{49.9} &\underline{47.3} & 68.1 & 86.2 &\textbf{55.0} & 67.8 & 61.0 &\textbf{42.1} & 60.6 & 45.3 & 57.6\\
        Ci et al.~\cite{ci2019LCN} ICCV'2019& 46.8 & 52.3 &\textbf{44.7} & 50.4 & 52.9 & 68.9 & 49.6 & 46.4 & 60.2 & 78.9 & 51.2 & 50.0 & 54.8 & 40.4 & 43.3 & 52.7\\
        Liu et al.~\cite{liu2020comprehensive} ECCV'2020& 46.3 & 52.2 & 47.3 & 50.7 & 55.5 & 67.1 & 49.2 & 46.0 & 60.4 & 71.1 & 51.5 & 50.1 & 54.5 & 40.3 & 43.7 & 52.4\\
        Xu et al.~\cite{xu2021graph} CVPR'2021& 45.2 & 49.9 & 47.5 & 50.9 & 54.9 & 66.1 & 48.5 & 46.3 & 59.7 & 71.5 & 51.4 & 48.6 & 53.9 & 39.9 & 44.1 & 51.9\\
        Zhao et al.~\cite{zhao2022graformer} CVPR'2022& 45.2 & 50.8 & 48.0 & 50.0 & 54.9 & 65.0 & 48.2 & 47.1 & 60.2 & 70.0 & 51.6 & 48.7 & 54.1 & 39.7 & 43.1 & 51.8\\
        Pavllo et al.~\cite{pavllo20193d} CVPR'2019&47.1 &50.6 &49.0 &51.8 &53.6 &61.4 &49.4& 47.4 &59.3& 67.4 &52.4 &49.5& 55.3 &39.5& 42.7 &51.8\\
        Cai et al.~\cite{cai2019exploiting} ICCV'2019&46.5 &48.8 &47.6 &50.9& 52.9& 61.3& 48.3& 45.8& 59.2& 64.4 &51.2 &48.4 &53.5 &39.2 &41.2 &50.6\\
        zeng et al.~\cite{zeng2020srnet} ECCV'2020&\underline{44.5} &\underline{48.2} & 47.1 & \textbf{47.8} & 51.2 &\underline{56.8} & 50.1 &\underline{45.6} & 59.9 & 66.4 & 52.1 &\textbf{45.3} & 54.2 & 39.1 &\underline{40.3} & 49.9\\ 
        Zou et al.~\cite{zou2021modulated} ICCV'2021& 45.4 & 49.2 & 45.7 & 49.4 &\underline{50.4} & 58.2 & 47.9 & 46.0 &\underline{57.5} & 63.0 &\underline{49.7} & 46.6 & 52.2 &\underline{38.9} & 40.8 &\underline{49.4}\\ 
       
       
        \midrule
       
       
        Ours &\textbf{43.9} &\textbf{47.6} &\underline{45.5} &\underline{48.9} &\textbf{50.1} &58.0 &\textbf{46.2} &\textbf{44.5} &\textbf{55.7} &\underline{62.9} &\textbf{49.0} &\underline{45.8} &\underline{51.8} &\textbf{38.0} &\textbf{39.9} &\textbf{48.5}\\
        \bottomrule
        \textbf{Protocol \#2} & Dir. & Disc. & Eat & Greet & Phone & Photo & Pose & Pur. & Sit & SitD. & Smoke & Wait & WalkD. & Walk & WalkT. & Avg.\\
        \midrule\midrule
        Martinez et al.~\cite{martinez2017simple} ICCV'2017& 39.5 & 43.2 & 46.4 & 47.0 & 51.0 & 56.0 & 41.4 & 40.6 & 56.5 & 69.4 & 49.2 & 45.0 & 49.5 & 38.0 & 43.1 & 47.7\\ 
       
        Ci et al.~\cite{ci2019LCN} ICCV'2019& 36.9 & 41.6 & 38.0 & 41.0 & 41.9 & 51.1 & 38.2 & 37.6 & 49.1 & 62.1 & 43.1 & 39.9 & 43.5 & 32.2 & 37.0 & 42.2\\
        Liu et al.~\cite{liu2020comprehensive} ECCV'2020& 35.9 & 40.0 & 38.0 & 41.5 & 42.5 & 51.4 & 37.8 & 36.0 & 48.6 & 56.6 & 41.8 & 38.3 & 42.7 & 31.7 & 36.2 & 41.2\\
        Cai et al.~\cite{cai2019exploiting} ICCV'2019& 36.8 & 38.7 & 38.2 & 41.7 & 40.7 & 46.8 & 37.9 & 35.6 & 47.6 &\underline{51.7} & 41.3 & 36.8 & 42.7 & 31.0 & 34.7 & 40.2\\
        Pavllo et al.~\cite{pavllo20193d} CVPR'2019& 36.0 & 38.7 & 38.0 & 41.7 & 40.1 & 45.9 & 37.1 & \underline{35.4} & 46.8 & 53.4 & 41.4 & 36.9 & 43.1 &\textbf{30.3} & 34.8 & 40.0\\
        zeng et al.~\cite{zeng2020srnet} ECCV'2020&\underline{35.8} & 39.2 &\underline{36.6} &\textbf{36.9} & 39.8 & 45.1 & 38.4 & 36.9 & 47.7 & 54.4 &\textbf{38.6} & 36.3 &\textbf{39.4} & \textbf{30.3} & 35.4 & 39.4\\ 
        Zou et al.~\cite{zou2021modulated} ICCV'2021&\textbf{35.7} &\underline{38.6} &\textbf{36.3} & \underline{40.5} &\textbf{39.2} &\textbf{44.5} &\underline{37.0} &\underline{35.4} &\underline{46.4} &\textbf{51.2} & 40.5 &\underline{35.6} &\underline{41.7} & 30.7 &\underline{33.9} &\underline{39.1}\\
        \midrule
       
       
        Ours &\underline{35.8} &\textbf{38.1} &\underline{36.6} & 40.9 &\underline{39.4} &\textbf{44.5} &\textbf{36.7} &\textbf{34.8} &\textbf{45.5} &\underline{51.7} &\underline{40.0} &\textbf{35.5} & 41.8&30.5&\textbf{33.8} &\textbf{39.0}\\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Quantitative comparison on Human3.6M with detected 2D poses as input under Protocol \#1 and Protocol \#2 in millimeters. The best results are highlighted in bold and the second-best results are presented with underlines.}
    \label{table:2D}
\end{table*}

\begin{table*}[ht]
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{\linewidth}{!}{
    \begin{tabular}{l c c c c c c c c c c c c c c c c}
        \toprule
        \textbf{Protocol \#1} & Dir. & Disc. & Eat & Greet & Phone & Photo & Pose & Pur. & Sit & SitD. & Smoke & Wait & WalkD. & Walk & WalkT. & Avg.\\
        \midrule\midrule
        Zhou et al.~\cite{zhou2019hemlets}(+) ICCV'2019& 34.4 & 42.4 & 36.6 & 42.1 & 38.2 & \textbf{39.8} & 34.7 & 40.2 & 45.6 & 60.8 & 39.0 & 42.6 & 42.0 & 29.8 & 31.7 & 39.9\\
        Ci et al.~\cite{ci2019LCN}(+)($\ast$) ICCV'2019& 36.3 & 38.8 & 29.7 & 37.8 & 34.6 & 42.5 & 39.8 & 32.5 & 36.2 & \textbf{39.5} & 34.4 & 38.4 & 38.2 & 31.3 & 34.2 & 36.3\\
        \midrule
        Martinez et al.~\cite{martinez2017simple} ICCV'2017& 37.7 & 44.4 & 40.3 & 42.1 & 48.2 & 54.9 & 44.4 & 42.1 & 54.6 & 58.0 & 45.1 & 46.4 & 47.6 & 36.4 & 40.4 & 45.5\\ 
        Zhao et al.~\cite{zhao2019semantic} CVPR'2019& 37.8 & 49.4 & 37.6 & 40.9 & 45.1 & 41.4 & 40.1 & 48.3 & 50.1 & 42.2 & 53.5 & 44.3 & 40.5 & 47.3 & 39.0 & 43.8\\
       
        Cai et al.~\cite{cai2019exploiting} ICCV'2019& 33.4 & 39.0 & 33.8 & 37.0 & 38.1 & 47.3 & 39.5 & 37.3 & 43.2 & 46.2 & 37.7 & 38.0 & 38.6 & 30.4 & 32.1 & 38.1\\
        Liu et al.~\cite{liu2020comprehensive} ECCV'2020& 36.8 & 40.3 & 33.0 & 36.3 & 37.5 & 45.0 & 39.7 & 34.9 & 40.3 & 47.7 & 37.4 & 38.5 & 38.6 & 29.6 & 32.0 & 37.8\\
       
        Zou et al.~\cite{zou2021modulated} ICCV'2021& - & - & - & - & - & - & - & - & - & - & - & - & - & - & - & 37.4\\ 
        zeng et al.~\cite{zeng2020srnet} ECCV'2020& 35.9 & 36.7 & 29.3 & 34.5 & 36.0 & 42.8 & 37.7 & 31.7 & 40.1 & 44.3 & 35.8 & 37.2 & 36.2 & 33.7 & 34.0 & 36.4\\
        Xu et al.~\cite{xu2021graph} CVPR'2021& 35.8 & 38.1 & 31.0 & 35.3 & 35.8 & 43.2 & 37.3 & 31.7 & 38.4 & 45.5 & 35.4 & 36.7 & 36.8 & 27.9 & 30.7 & 35.8\\
        Zhao et al.~\cite{zhao2022graformer} CVPR'2022& 32.0 & 38.0 & 30.4 & 34.4 & 34.7 & 43.3 & 35.2 & 31.4 & 38.0 & 46.2 & 34.2 & 35.7 & 36.1 & 27.4 & 30.6 & 35.2\\
        \midrule
        Ours& \textbf{31.3} & \textbf{34.0} & \textbf{28.0} & \textbf{32.0} & \textbf{33.1} & 42.1 & \textbf{34.1} & \textbf{28.1} & \textbf{33.6} & 39.8 & \textbf{31.7} & \textbf{32.9} & \textbf{33.8} & \textbf{26.7} & \textbf{28.9} & \textbf{32.7}\\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Quantitative comparison on Human3.6M with under Protocol \#1 in millimeters. Ground truth 2D keypoints are used as input. (+) uses extra data from MPII~\cite{andriluka2014mpii}. ($\ast$) uses pose scales in both training and testing.
    The best results are highlighted in bold.}
    \label{table:GT1}
\end{table*}

\section{Experiments}
In this section, we first introduce the experimental setup and implementation details of the HopFIR networks. Next, we conduct several ablation studies regarding the proposed architecture. Our experimental results and comparisons with state-of-the-art methods are presented in the last.

\subsection{Datasets and Evaluation Protocols}
\textbf{Datesets.}  Human3.6M~\cite{ionescu2013h36m} is currently the largest publicly available dataset for 3D human pose estimation with 3.6 million video frames. It captures the accurate 3D human joint positions from four camera viewpoints and records 11 subjects performing 15 assigned actions. Following previous works~\cite{zou2021modulated, xu2021graph, zhao2019semantic}, we train our model on five subjects (S1, S5, S6, S7, S8) and test on two subjects (S9, S11). In contrast to Human3.6M, MPI-INF-3DHP~\cite{mehta20173dhp} includes complex outdoor scenes, which are usually used to evaluate the generalization ability of the proposed methods. We use the test set of this dataset to verify the generalization of our model.

\textbf{Evaluation Protocols.}  For Human3.6M~\cite{ionescu2013h36m}, we evaluate our model on two standard evaluation protocols, the mean per-joint position error (MPJPE) and the mean per-joint position error after Procrustes alignment (P-MPJPE), which are referred as protocol \#1 and protocol \#2 respectively.
For MPI-INF-3DHP~\cite{mehta20173dhp}, we follow previous works~\cite{zou2021modulated, ci2019LCN, zhao2019semantic} and report the Percentage of Correct Keypoints (PCK) with a threshold of 150mm and the Area Under Curve (AUC) for a range of PCK thresholds.

\subsection{Implementation Details}

We make use of the ground truth 2D joint locations provided in the dataset to align the 3D and 2D poses following the setting of~\cite{zhao2019semantic}. 
Moreover, we do not use data augmentation during training with 2D ground truth input, which further verifies the efficacy of our model. Following previous work~\cite{zou2021modulated}, we obtain 2D pose detections using the cascaded pyramid network(CPN)~\cite{chen2018cpn}.
The models are implemented using the PyTorch framework and optimized via Adam~\cite{kingma2014adam}. We initialize the weights in HopFIR using~\cite{glorot2010xavierinit}. All experiments are conducted on a single NVIDIA RTX 3090 GPU.
3D pose regression from 2D detections is more challenging than that from 2D ground truth as the former needs to deal with some extra uncertainty in the 2D space. 
Therefore, we set different configurations for them to tackle the uncertainty. In experiments with 2D ground truth as the input, we train the HopFIR networks with an initial learning rate of 0.001, a decay factor of 0.90 per 4 epochs, a batch size of 64, channels of 128, and PReLU activation~\cite{he2015prelu}. When using detected 2D poses, we train the HopFIR networks with the initial learning rate of 0.006, a decay factor of 0.95 per 4 epochs but 0.2 for the first 4 epochs, the batch size of 256, the channel size of 256,  and LeaklyReLU activation~\cite{maas2013leakyrelu}.
Meanwhile, we apply Dropout\cite{JMLR:v15:dropout} with a dropout rate of 0.5 to avoid overfitting.

\subsection{Ablation Study}
\begin{table}[t]
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{75mm}{!}{
   
   
       
       
       
       
       
       
       
    \begin{tabular}{l|c|c|c|c}
        \toprule
        Method & Channels & Params & MPJPE & P-MPJPE \\
        \midrule\midrule
        GCN &128 &0.36M &40.63 &31.65\\ 
        HopGCN &128 &0.59M &39.15 &31.40\\ 
        HopGCN + IJR &128 &2.15M &36.62 &29.23\\ 
        HopGCN + HGF & 128 & 1.05M &35.19 &28.81 \\ 
        HopGCN + HopFIR & 128 & 2.15M &\textbf{32.67} &\textbf{26.20} \\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Ablation experiments on the proposed different module.
   
    }
    \label{table:layer}
\end{table}
\begin{table}[t]
    \small
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{75mm}{!}{
   
   
   
   
   
   
   
   
   
   
   
   
   
       
       
    \begin{tabular}{l|ccc||ccc}
        \toprule
        \multirow{2}{*}{Attention} &\multicolumn{3}{c||}{HA} &\multicolumn{3}{c}{HA+W}\\
        \cline{2-2}\cline{3-3}\cline{4-4}\cline{5-5}\cline{6-6}\cline{7-7}
        &HSS &SSS &SHH &HSS &SSS &SHH\\
        \hhline{-|---||---}
        Params  &\multicolumn{3}{c||}{2.15M} &\multicolumn{3}{c}{2.50M}\\ 
        \hhline{-|---||---}
        MPJPE   &\textbf{32.67} &34.18 & 34.28 & \textbf{33.29} & 33.53 & 33.90\\ 
        P-MPJPE &\textbf{26.20} &27.70 & 27.71 & \textbf{27.16} & 27.40 & 27.41\\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Quantitative comparison of HA layers with different similarity computing approaches.
   
    }
    \label{table:HA_attn}
\end{table}
\begin{table}[t]
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{83mm}{!}{
   
   
   
   
   
   
   
   
   
   
   
   
   
    \begin{tabular}{l|c|c|c|c|c}
        \toprule
        \multicolumn{2}{l|}{Method} & Channels & Params & MPJPE & P-MPJPE \\
        \midrule\midrule
        \multicolumn{2}{l|}{SemGCN~\cite{zhao2019semantic}} & 128 & 0.27M & 42.14 & 33.53 \\ 
        \multicolumn{2}{l|}{SemGCN + HA(HSS)} & 128 & 0.49M &\textbf{38.41} & \textbf{30.56} \\ 
        \multicolumn{2}{l|}{SemGCN + HA(SSS)} & 128 & 0.49M & 41.30 & 33.07 \\ 
        \multicolumn{2}{l|}{SemGCN + HA(SHH)} & 128 & 0.49M & 38.81 & 31.05 \\ 
        \midrule
        \multicolumn{2}{l|}{SemGCN~\cite{zhao2019semantic} w/ Non-local~\cite{wang2018nonlocal}} & 128 & 0.43M & 40.78 & 31.46 \\ 
        \multicolumn{2}{l|}{SemGCN w/ Non-local +HA(HSS)} & 128 & 0.66M & 38.03 & 30.50 \\ 
        \multicolumn{2}{l|}{SemGCN w/ Non-local +HA(SSS)} &128 &0.66M &\textbf{37.75} &30.17 \\ 
        \multicolumn{2}{l|}{SemGCN w/ Non-local +HA(SHH)} &128 &0.66M &37.94 &\textbf{29.71} \\ 
        \midrule
        \multicolumn{2}{l|}{Modulated GCN~\cite{zou2021modulated}} & 128 & 0.29M & 38.25 & 30.06 \\ 
        \multicolumn{2}{l|}{Modulated GCN +HA(HSS)+W} & 128 &0.96M &36.54 & 29.09\\ 
        \multicolumn{2}{l|}{Modulated GCN +HA(SSS)+W} & 128 &0.96M &\textbf{36.14} &\textbf{29.02}\\ 
        \multicolumn{2}{l|}{Modulated GCN +HA(SHH)+W} & 128 &0.96M &37.38 &30.02\\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Comparison of the improved performance of proposed HA layer added on different approaches. We test on SemGCN~\cite{zhao2019semantic} and Modulated GCN~\cite{zou2021modulated} that GCN-based method.
   
    }
    \label{table:HAmodule}
\end{table}

\begin{figure}[t]
  \centering
 
   \includegraphics[width=0.9\linewidth]{joint1.pdf}
   \caption{Comparison of
   MPJPE of peripheral joints on the test set of the Human3.6M. R and L denote right and left, respectively.}
   \label{fig:joint_error}
\end{figure}

\begin{table}[t]
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{60mm}{!}{
    \begin{tabular}{c|c|c|c|c}
        \toprule
        Num-K & Channels & Params & MPJPE & P-MPJPE \\
        \midrule\midrule
        1 & 128 & 1.88M & 35.88 & 28.76 \\ 
       
        2 & 128 & 2.03M & 34.96 & 27.74 \\ 
       
        3 & 128 & 2.15M &\textbf{32.67} &\textbf{26.20} \\ 
       
        4 & 128 & 2.27M & 35.58 & 28.35  \\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Ablation study on different number of the k-hop. The units of MPJPE and P-MPJPE are millimeters (mm).}
    \label{table:hopnum}
\end{table}

\begin{table}[t]
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{60mm}{!}{
    \begin{tabular}{c|c|c|c|c}
        \toprule
        Num-Block & Channels & Params & MPJPE & P-MPJPE \\
        \midrule\midrule
        1 & 128 &0.80M &37.33 &30.71\\ 
       
        2 & 128 &1.47M &34.21 &27.63 \\ 
       
        3 & 128 &2.15M &\textbf{32.67} &\textbf{26.20}\\ 
       
        4 & 128 &2.82M &33.84 &27.63\\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Ablation study on the number of the HopFIR block. The units of MPJPE and P-MPJPE are millimeters (mm).}
    \label{table:block_num}
\end{table}
\begin{table}[t]
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{60mm}{!}{
    \begin{tabular}{c|c|c|c|c}
        \toprule
        Block & Channels & Params & MPJPE & P-MPJPE \\
        \midrule\midrule
        (H)(I) & 128 &1.78M &34.43 &27.79 \\ 
       
        (I)(H) & 128 &1.78M &34.13 &27.56 \\ 
       
        (H)(I)(H) & 128 & 2.15M &35.08 &27.98 \\ 
       
        (I)(H)(H) & 128 &2.15M &34.20 &27.60  \\ 
       
        (H)(H)(I) & 128 &2.15M &\textbf{32.67} &\textbf{26.20}\\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Ablation study on the design of the HopFIR block. (I) is IJR module and (H) is HGF module. The units of MPJPE and P-MPJPE are millimeters (mm).}
    \label{table:block}
\end{table}

The proposed HopFIR contains two main components: Hop-wise GraphFormer (HGF) module and Intragroup Joint Refinement (IJR) module. We conduct a comprehensive ablation study on Human3.6M to validate the effectiveness of each component under controlled settings.
Note that all the input is the 2D ground truth to eliminate the extra uncertainty from the 2D pose detector.

\textbf{Effectiveness of Different Modules.}  We remove IJR and HGF separately to verify the effectiveness of the HGF module and IJR module and conduct experiments on first-order neighbors which remove all modules and the multi-hop mechanism.
In Table~\ref{table:layer},
one can see that each module improves the performance compared to the GCN-only approach. Coupling these two layers to form a HopFIR block achieves further performance improvement. 
The HopFIR networks reduce the MPJPE to 32.67mm, which achieves 7.2\% improvement compared to the GraFormer~\cite{zhao2022graformer}.

\textbf{Error on Peripheral Joints.}  We report the regression accuracy of wrists and feet in Fig.~\ref{fig:joint_error}. Compared to some previous methods~\cite{zou2021modulated, zhao2019semantic,zeng2020srnet}, the HopFIR network with IJR modules outperforms previous SOTA methods on all wrists and feet, such as Rfoot 6.7mm, Lfoot 7.7mm, LWrist 9.6mm and RWrist 7.7mm.
The experimental results verify that the intragroup joint attention within in limb group strengthens the capability of the HopFIR.

\textbf{Effectiveness of HA Layer.}  Hop-wise attention is designed to capture interactions among the k-hop groups in different human motion patterns. We calculate the similarity between the group and the target joint to reconstruct the k-hop group feature. Self-attention among groups is an obvious choice. One can also compute the similarity between the node features. 
In Table~\ref{table:HA_attn}, we show the experimental results of three different ways to design the HA layer. HSS is the method selected in this paper, where H, S, and S are $Q$, $K$, and $V$, and H and S represent the node feature and the k-hop group feature, respectively. Note that we do not apply a linear transformation of $Q$, $K$, and $V$ like attention~\cite{vaswani2017attention}, so we also show the result of the linear transformation, which is denoted as HA + W. The experiment results show that HSS similarity achieves better performance in the HopFIR architecture, but one can choose the type of similarity according to the network property.  

We further insert the HA layer into previous SOTA pose estimation methods to investigate its generalization, such as SemGCN~\cite{zhao2019semantic} and MGCN~\cite{zou2021modulated}. Note that no changes have been made to their source code but insert the HA layer before their information aggregate stage.
The experimental results in Table~\ref{table:HAmodule} show that the HA layer can improve these previous SOTA networks by a large margin, especially SemGCN~\cite{zhao2019semantic}, from 42.14mm to 38.41mm, resulting in a relative 8.9\% improvement. 
Experiments on the above methods of aggregating first-order neighbor information demonstrate the effectiveness of HA, which also indicates that latent joint grouping can recognize the latent human joint synergies.

\textbf{Different Number of Hop.}  HopFIR is designed to extract the correlation between feature groups, thus we set different hop $k$ to discover various latent connections underlying human joint synergies, the results are shown in Table~\ref{table:hopnum}.
One can observe that the MPJPE gradually decreases from 35.88 $mm$ to 32.67 $mm$ as the number of hops increases, and reaches the best performance at 3 hops. Therefore, the optimal number of hops is three for 3D HPE, which indicates that we get $16\times3$ groups from the skeleton graph. Each of the 16 groups corresponds to a potential correlation among coupled nodes at different distances, while three is enough to recognize the potential joint synergies.

\textbf{Structure design of the HopFIR block.}  To investigate the structure of the HopFIR block, experiments of HopFIR block number and various combinations of HGF modules and IJR modules were performed. 
As the number of blocks increases, the error gradually decreases and reaches the best performance at three.
As shown in Table~\ref{table:block}, (H)(H)(I) achieve the optimal results, which reduces the error to 32.67mm.
In addition, all results except 1 block are better than GraFormer~\cite{zhao2022graformer}, which indicates that the HopFIR has significant human pose representation capability.


\subsection{Comparison with State-of-the-art}
We compare the HopFIR with some state-of-the-art methods on Human3.6M under Protocol\#1 and Protocol\#2 which is shown in Table~\ref{table:2D}.
Our method reaches an MPJPE of 48.50mm. Compared with the previous best results~\cite{zou2021modulated}, we outperform it on all 16 joints.
Due to the uncertainty of 2D detections, we also investigate the capability of HopFIR networks using ground truth 2D key points as input to bypass the influence of detections. As shown in Table~\ref{table:GT1}, HopFIR obtains surprisingly better performance when given precise 2D joint information and attains state-of-the-art performance, which verifies its effectiveness.

\textbf{Cross-Dataset Results on MPI-INF-3DHP.}  Table~\ref{table:mpi3d} further compares HopFIR and previous methods on cross-dataset scenarios to validate its generalization ability. We only train our model on the Human3.6M dataset and test it on the test set of the MPI-INF-3DHP dataset. 
The results show that our approach obtains better results compared with other methods, which verifies the generalization ability of our approach to unseen scenarios.

\textbf{Qualitative Results.}  In Fig.~\ref{fig:qualitative}, we show the visual results of HopFIR on Human3.6M in the world space. As we can see, HopFIR networks are able to predict 3D joint positions more accurately, even for difficult poses that MGCN~\cite{zou2021modulated} failed. 
\begin{table}[t]
    \centering
    \setlength{\extrarowheight}{0pt}
   
   
    \setlength{\aboverulesep}{0pt}
    \setlength{\belowrulesep}{0pt}
    \resizebox{80mm}{!}{
    \begin{tabular}{l|c|c|c|c|c}
       
       
        \toprule
        \multirow{2}{*}{Methods}&\multicolumn{4}{c|}{PCK} & \multirow{2}{*}{AUC} \\
        \cline{2-2}\cline{3-3}\cline{4-4}\cline{5-5}
        & GS & no GS & Outdoor & All\\
        \midrule\midrule
        Martinez et al.~\cite{martinez2017simple} & 49.8 & 42.5 & 31.2 & 42.5 & 17.0\\ 
        Ci et al.~\cite{ci2019LCN} & 74.8 & 70.8 & 77.3 & 74.0 & 36.7\\ 
        zeng et al.~\cite{zeng2020srnet} & - & - & 80.3 & 77.6 & 43.8\\ 
        Li et al.~\cite{li2019generating} & 70.1 & 68.2 & 66.6 & 66.9 & -\\ 
        Zhao et al.~\cite{zhao2022graformer}&80.1 &77.9 &74.1 &79.0 & 43.8\\ 
        Liu et al.~\cite{liu2020comprehensive} (weight unsharing)& 77.6 & 80.5 & 80.1 & 79.3 & 47.6\\ 
        Xu et al.~\cite{xu2021graph} & 81.5 & 81.7 & 75.2 & 80.1 & 45.8\\
        Nie et al.~\cite{nie2021lifting} & - & - & - & 83.5 & 45.9\\
        Zou et al.~\cite{zou2021modulated} & 86.4 & \textbf{86.0} &85.7 & 86.1 & 53.7\\ 
        \midrule
       
       
        Ours & \textbf{89.1} & 85.9 & \textbf{85.9} & \textbf{87.2} & \textbf{57.0}\\ 
        \bottomrule
    \end{tabular}
    }
    \caption{Quantitative comparisons on MPI-INF-3DHP test set. GS denotes green screen.
   
    }
    \label{table:mpi3d}
\end{table}

\begin{figure}[t]
  \centering
  \includegraphics[width=1\linewidth]{Qualitative4.pdf}

  \caption{Qualitative visual results of our method and MGCN~\cite{zou2021modulated} on Human3.6M dataset~\cite{ionescu2013h36m}. The black lines are Ground Truth (GT) and the red lines are predictions by HopFIR and MGCN.
  }
  \label{fig:qualitative}
\end{figure}
\section{Conclusions}
We present a novel architecture for the 3D human pose estimation, that is, the Hop-wise GraphFormer with Intragroup Joint Refinement. The proposed HGF improves the GCN-based pose estimation networks by grouping the joints by k-hop neighborhood and capturing the potential joint correlations in different joint synergies. Because the peripheral joint strongly interacts with other intra-limb joints, the proposed IJR applies intragroup attention to refining the peripheral joint features through the associated limb. The HopFIR, mainly composed of the HGF modules and the IJR module, achieves new state-of-the-art results while keeping a modest model size. 
{\small
