\section{Introduction}\label{sec:intro}

Short length block codes are an attractive choice for use cases with stringent latency requirements \cite{ccsds}. However, they suffer from poor error correction performance. One key reason for this is the presence of short cycles in the Tanner graph of the code, making their decoding sub-optimal. Unfortunately at such short lengths, codes designed without cycles have worse error correction performance\cite{ryan2009channel}. Additionally, when fading channels are considered, the Log Likelihood Ratio (LLR) computation suffers from errors because of imperfect Channel State Information (CSI) and equalization, which makes the Belief Propagation (BP) decoder sub-optimal \cite{yazdani2011}. Hence modifications to the traditional BP decoder are necessary to improve the performance. 




In \cite{nachmani2016}, the authors demonstrate the advantage of using neural networks for improving the BP decoder by placing multiplicative weights along the edges of the Tanner graph structure and unrolling the iterations, resulting in a neural BP decoder.  
Since then, there have been many works that explored the possibility of reducing the complexity of the neural BP decoders. 
In \cite{noms2017}, the authors show that instead of using multiplicative weights, comparable gains can be achieved by using offset factors combined with min-sum approximation, which reduces the implementation cost. Later in \cite{nachmani2018}, the authors explore the possibility of reducing the number of weights needed by entangling them across iterations in a recurrent fashion and show that the performance is still comparable to the fully parameterized model. More recently, the authors in \cite{lian2019} propose entangling the weights not only across iterations but also across the edges. To compensate for the performance loss, a shallow neural network referred to as Parameter Adapter Network (PAN), is used to select different weights for different SNRs regions resulting in a performance very close to \cite{nachmani2018}.

Despite the success of these works, there are two key open problems that we answer in this work. The first is the trade-offs associated with entanglement, which is limited to only AWGN channels in  \cite{nachmani2016,noms2017,nachmani2018,lian2019}. We consider non-AWGN channels such as ETU and show that the trade-offs are quite different from AWGN. The second is the interpretation of the learned weights. While it has been conjectured that the use of normalization or offsets to modify the beliefs helps mitigate the detrimental effects of short cycles \cite{nachmani2016,noms2017}, there is no supporting evidence. We present empirical evidence that the weights learned are directly related to the short cycles present in the code and investigate how they help in improving the reliability of the posterior LLRs. 











Our main contributions can be summarized are as follows:
\begin{itemize}
    \item \emph{Complexity Trade-off:} We explore the complexity vs performance trade-off for neural augmented min-sum decoders across  various channels. We show that we benefit significantly by having more number of weights for a complex channel while fewer weights are sufficient for a simple channel such as AWGN and thus the choice of parameters can be made based on the channel conditions.
    
    
   
    
   
    

    \item \emph{Robustness and adaptivity:} We show that the learned weights are fairly robust to channel variations \textit{i.e,} the decoder trained on one channel still outperforms the classical decoders on other channels. We further show that these neural decoders can be quickly adapted to new channels with minimal additional training, to achieve additional gains.
    
   
    \item \emph{Over-the-air channels:} We evaluate neural min-sum decoders for over-the-air channels using USRPs. We show that that the fully parameterized min-sum decoder outperforms the entangled variant. 
    
   
    
   

    \item \emph{Interpretation:} We provide insights into the learned decoder by establishing a connection between the number of cycles in the Tanner graph and the weights learned. We also show that the learned weights reduce the correlation between the beliefs propagated, which is a desirable property since more independent beliefs result in more reliable estimation of the posterior LLRs \cite{yazdani2004}.
\end{itemize}
   


\section{System Model}\label{sec:sysmodel}
In this work, we consider linear block codes. A $(N,K)$ block code maps a message of length $K$  to a codeword of length $N$ and is uniquely described by its parity check matrix $\mathbb{H}$ of dimensions $(N-K) \times N$, where the rate of the code is $R=\sfrac{K}{N}$. The linear block code can be also represented using a bipartite graph, known as Tanner graph, which can be constructed using its parity check matrix $\mathbb{H}$. The Tanner graph consists of two types of nodes. We refer to them as Check Nodes (CN) that represent the parity check equations and Variable Nodes (VN) that represent the symbols in codeword. There is an edge present between a check node $c$ and variable node $v$ if $\mathbb{H}(c,v) = 1$.

We consider a system with Binary Phase Shift Keying (BPSK) modulation that transmits a random vector $X \in \{-1,1\}^N$, where $N$ is the code length. The generated vector is then passed through a channel to receive $Y$ as follows
\begin{equation*}
    Y = \text{Channel}(X) + W,
\end{equation*}
where Channel($X$) applies the effect of channel on the transmit vector $X$ and $W$ is the noise at the receiver. As a special case, when the channel is AWGN the LLR at the receiver for $v^{\text{th}}$ symbol can be calculated as
\begin{equation*}
    l_v = \frac{p(Y_v = y_v | X_v = -1)}{p(Y_v = y_v | X_v = 1)} = \frac{2y_v}{\sigma^2},
\end{equation*}
where $\mbox{Var}(W) = \sigma^2$ is the noise variance.

Apart from the AWGN channel, we also consider multi-path fading propagation channels from the 3GPP specifications \cite{3gppts36101,3gppts36104}. These channels result in interference and distortion at the receiver because of the multi-path components and also the  mobility of the users. Specifically, we consider multi-path fading ETU channel, which has high delay spread and hence hard to achieve perfect equalization. We use the MATLAB LTE Toolbox to simulate the transmission and receiving of data. Finally, we also test the decoder on the real-world data by transmitting and receiving over the air using a USRP N200 SDR setup in non line of sight (NLOS) multi-path environment. More comprehensive results for other channels and the code for the algorithm can be found at: \href{https://github.com/sravan-ankireddy/nams}{https://github.com/sravan-ankireddy/nams}


\section{Background}\label{sec:background}
In this section, we briefly review the BP decoding and existing classical and data driven methods for its improvement. We then look at the min-sum variant of the BP decoder and the corresponding methods for its improvement. We also introduce and review the baseline algorithms that we will be comparing against.
\subsection{Neural Belief Propagation decoder}
The BP decoder is an iterative soft-in soft-out decoder that operates on the Tanner graph structure of the linear block codes to compute the posterior LLRs of the received vector. We also refer to these messages as beliefs. In each iteration, the check nodes and the variable nodes process the information to update the beliefs passed along the edge. Operating in such an iterative fashion allows for incremental improvement in the estimated LLRs, converging towards the right codeword that is obtained by binary quantization after the final iteration.

During the first half of the iteration, at the VN $v$, the received channel LLR $l_v$ is combined with the  rest of the incoming beliefs $\mu^{t-1}_{c',v}$ from check node to calculate a new updated belief, to be passed to the check nodes in next iteration. Hence, the message from VN $v$ to CN $c$ at iteration $t$ can be computed as
\begin{equation}
    \mu^t_{v,c} = l_v + \sum_{c' \in N(v) \setminus c} \mu^{t-1}_{c',v} \mbox{          }, 
\end{equation}
where $N(v)\setminus c$ is the set of all check nodes connected to variable node $v$ except $c$.

\iffalse

\begin{center}
\begin{tikzpicture}[
  vnode/.style = {shape=circle,draw,minimum size=1em,fill=blue!100},
  cnode/.style = {shape=square,draw,minimum size=1em,fill=red!100},
  edge/.style = {-Latex},
  ]
 
  \node[] (ch) at (0,-1.5);
  \node[vnode,label=right:$v$] (c) at (0,0);
  \node[cnode,label=right:$c_i$] (ci) at (-3,2) ;
  \node[cnode,label=right:$c_j$] (cj) at (-1,2);
  \node[cnode,label=right:$c_k$] (ck) at (1,2);
  \node[cnode,label=right:$c_l$] (cl) at (3,2);

 
  \draw[edge, -<-=0.5] (c) to node[right, xshift=1mm] {$l_v$} (ch);
  \draw[edge, red, thick, ->-=0.5] (c) to node[left, xshift=-1mm, yshift=-1mm] {$\mu^t_{c_i,v}$} (ci);
  \draw[edge, -<-=0.5] (c) to node[right, xshift=-1mm, yshift=3mm] {$\mu^{t-1}_{c_j,v}$} (cj);
  \draw[edge, -<-=0.5] (c) to  node[right, xshift=1mm, yshift=3mm] {$\mu^{t-1}_{c_k,v}$} (ck);
  \draw[edge, -<-=0.5] (c) to node[right, xshift=5mm, yshift=3mm] {$\mu^{t-1}_{c_l,v}$} (cl);
\end{tikzpicture}
\end{center}
\fi 


During the latter half of the iteration, at the CN $c$, the message from the CN to any VN is calculated based on the criterion that the incoming beliefs  $\mu^t_{v',c}$ at any check node should always satisfy the parity constraint. Specifically, the message from CN $c$ to VN $v$ at iteration $t$ is given by
\begin{equation}\label{eq:cn_spa}
    \mu^t_{c,v} = 2 \tanh^{-1} \left( \prod_{v' \in M(c) \setminus v} \text{tanh} \left( \frac{\mu^t_{v',c}}{2}  \right)  \right),
\end{equation}

where $M(c) \setminus v$ is the set of all variable nodes connected to check node $c$ except $v$.

\iffalse
\begin{center}
\begin{tikzpicture}[
  vnode/.style = {shape=circle,draw,minimum size=1em,fill=blue!100},
  cnode/.style = {shape=square,draw,minimum size=1em,fill=red!100},
  edge/.style = {-Latex},
  ]
 
  \node[cnode,label=right:$c$] (c) at (0,0);
  \node[vnode,label=right:$v_i$] (vi) at (-3,-2) ;
  \node[vnode,label=right:$v_j$] (vj) at (-1,-2);
  \node[vnode,label=right:$v_k$] (vk) at (1,-2);
  \node[vnode,label=right:$v_l$] (vl) at (3,-2);

 
  \draw[edge, blue, thick, ->-=0.5] (c) to node[left, xshift=-1mm, yshift=1mm] {$\mu^t_{v,c_i}$} (vi);
  \draw[edge, -<-=0.5] (c) to node[right, xshift=-1mm, yshift=-2mm] {$\mu^{t}_{v,c_j}$} (vj);
  \draw[edge, -<-=0.5] (c) to  node[right, xshift=1mm, yshift=-2mm] {$\mu^{t}_{v,c_k}$} (vk);
  \draw[edge, -<-=0.5] (c) to node[right, xshift=5mm, yshift=-2mm] {$\mu^{t}_{v,c_l}$} (vl);
\end{tikzpicture}
\end{center}
\fi 

Finally, at the end of iteration $t$, we combine all the incoming beliefs to estimate the posterior belief as
\begin{equation}
    o^{t}_{v} = l_v + \sum_{c' \in N(v)} \mu^{t}_{c',v} \mbox{        }.
\end{equation}

In \cite{nachmani2016}, it is shown that the decoding performance of BP can be improved by introducing trainable weights along the edges of the Tanner graph, resulting in a Weighted Belief Propagation (WBP), at the VN as
\begin{equation}\label{eq:wbp}
    \mu^t_{v,c} =  w_v^t l_v + \sum_{c' \in N(v) \setminus c} w_{c',v}^t \mu^{t}_{c',v} \mbox{          }. 
\end{equation}









\subsection{Neural Min-Sum decoding}\label{sec:nms}

To reduce the computational complexity of BP decoding, a hardware friendly variant known as \textit{min-sum} approximation is used in practice. This simplifies \eqref{eq:cn_spa} to 
\begin{equation}\label{eq:cn_ms}
    \mu^t_{c,v} =  \min_{v' \in M(c) \setminus v}  ( |\mu^t_{v',c}|) \prod_{v' \in M(c)  \setminus v} \text{sign} \left(\mu^t_{v',c} \right).
\end{equation}
While the min-sum approximation simplifies the computation, it also comes with a loss in performance. It is readily shown in \cite{chen2002_nms} that min-sum approximation is always greater than the true LLR from BP. To correct for this, two popular techniques have been introduced \cite{chen2002_nms, chen2005_oms, chen2005}. 
The first is to use a normalization factor $\alpha$ to scale the beliefs appropriately as
\begin{equation}\label{eq:nms}
    \mu^t_{c,v} =  \alpha \left( \min_{v' \in M(c) \setminus v} ( |\mu^t_{v',c}|) \right) \prod_{v' \in M(c)  \setminus v} \text{sign} \left(\mu^t_{v',c} \right).
\end{equation}

Another approach is to use an offset factor which reduces the beliefs at the CN by $\beta$, while maintaining the sign, as
\begin{equation}\label{eq:oms}
    \mu^t_{c,v} =   \mbox{max} \left( \min_{v' \in M(c) \setminus v} ( |\mu^t_{v',c}|) - \beta, 0 \right) \prod_{v' \in M(c)  \setminus v} \text{sign} \left(\mu^t_{v',c} \right).
\end{equation}

While these methods have been shown to improve the error correction performance of min-sum algorithm empirically, there has been no analytical solution to calculate the optimal normalization or offset parameters. One of the key reasons for this is the difficulty in modeling the posterior probabilities. The authors in \cite{chen2002_nms} use Monte Carlo simulations to select the best normalization factors and also remark the difficulty in the theoretical analysis beyond first iteration. 

Similar to the neural BP decoder, the neural min-sum decoder can be formulated by using different normalization or offset factors along different edges \cite{noms2017,nachmani2018}. The equations \eqref{eq:nms} and \eqref{eq:oms} can be modified respectively to produce Neural Normalized Min-Sum (NNMS) and Neural Offset Min-Sum (NOMS) algorithms, given by
\begin{equation}\label{eq:nnms}
    \mu^t_{c,v} =  \alpha_{v',c}^t \left( \min_{v' \in M(c) \setminus v} ( |\mu^t_{v',c}|) \right) \prod_{v' \in M(c)  \setminus v} \text{sign} \left(\mu^t_{v',c} \right),
\end{equation}
\begin{equation}\label{eq:noms}
\begin{split}
    \mu^t_{c,v} =   \mbox{max} \left( \min_{v' \in M(c) \setminus v} ( |\mu^t_{v',c}|) - \beta_{v',c}^t, 0 \right) \\ \prod_{v' \in M(c)  \setminus v} \text{sign} \left(\mu^t_{v',c} \right),
\end{split}
\end{equation}
respectively, where the coefficients $\alpha_{v',c}^t$ and $\beta_{v',c}^t$ denote trainable weights. 














The drawback of using neural network based decoders is the increase in storage and computational complexity. Hence, it is important to study the gains achieved and explore
possible simplifications to the neural decoder with little to no trade-off in performance.

\subsection{Reducing the complexity via entangling the weights}\label{sec:ent}
To reduce the storage complexity of WBP, the authors in \cite{nachmani2018} propose entangling the weights across iterations.
Further, in \cite{lian2019}, the authors propose entangling weights not only across iterations but across edges as well. However, this results in a loss of performance, which is compensated using a parameter adapter network that selects different weights for different SNR regions, known as BP-PAN. Similary, for the neural min-sum decoders, the authors in \cite{nachmani2018,lian2019} conclude that entangling the weights across iterations or/and edges only comes with a negligible loss for AWGN channels.





 
 
 
 
 
 

\subsection{Non-AWGN channels}
In channels with multi-path and Doppler components, the received LLRs are more unreliable compared to AWGN channel, which makes the decoding harder. Hence, whether the observation in \cite{nachmani2018,lian2019} that entangling the weights lead to negligible reliability remains the same for non-AWGN channels remains an open problem. In the following, we study the effect of weight entanglements for various channels, such as ETU and the over-the-air channels, beyond AWGN channels. 






 










\section{Neural Augmented Min-Sum Decoding}\label{sec:nams}
In this section, we propose combining the augmentation of normalization and offsets to form a more generalized neural min-sum decoder. We refer to this as Neural Augmented Min-Sum (NAMS) decoder. The modified CN update rule at iteration $t$ is given by
\begin{equation}\label{eq:nams}
\begin{split}
    \mu^t_{c,v} =  \alpha^t_{v',c} \left( \mbox{max}\left( \min_{v' \in M(c) \setminus v} ( |\mu^t_{v',c}|) - \beta^t_{v',c},0 \right) \right) \\ \prod_{v' \in M(c)  \setminus v} \text{sign} \left(\mu^t_{v',c} \right).
\end{split}
\end{equation}


We consider the min-sum approximation and limit the modifications to only CN updates to keep the complexity low.













\textbf{Entanglement.} We consider three variants of NAMS with different levels of entanglement.

\begin{enumerate}
    \item \textbf{NAMS-full}: Different weights for all edges across all iterations as seen in \eqref{eq:nams}.
    
    
    \item \textbf{NAMS-relaxed}: Entangled weights across the iterations.
    \begin{equation*}
    \begin{split}
    \mu^t_{c,v} =  \alpha_{v',c} \left( \mbox{max} \left( \min_{v' \in M(c) \setminus v} ( |\mu^t_{v',c}|) - \beta_{v',c}, 0 \right) \right) \\ \prod_{v' \in M(c)  \setminus v} \text{sign} \left(\mu^t_{v',c} \right).
    \end{split}
    \end{equation*}
    \item \textbf{NAMS-simple}: Entangled weights across the iterations and edges.
\begin{equation*}
    \begin{split}
    \mu^t_{c,v} =  \alpha \left( \mbox{max} \left( \min_{v' \in M(c) \setminus v} ( |\mu^t_{v',c}|) - \beta, 0 \right) \right) \\ \prod_{v' \in M(c)  \setminus v} \text{sign} \left(\mu^t_{v',c} \right).
    \end{split}
\end{equation*}

\end{enumerate}




\textbf{Loss function and Training.} We use the \textit{binary cross entropy with logits} loss function to measure the difference between the soft output of the each layer and the transmitted codeword and take the sum of losses across each layer to calculate the multiloss term\cite{nachmani2018} as
\begin{equation*}
    L(o,x) = -\frac{1}{N}\sum_{t=1}^{T}\sum_{v=1}^{N} x_v \log (o_{v}^{t}) + (1-x_v) \log (1 - o_{v}^{t}),
\end{equation*}

where $o_{v}^{t}$ is the output for symbol $v$ at iteration $t$ of NAMS and $x$ is the transmitted codeword.

To keep the comparison fair, we use the same amount of training data as \cite{noms2017}. Each mini-batch consists of 10 codewords per SNR point in the training region and Stochastic Gradient Descent (SGD) method is used to learn the weights. We use the Adam optimizer with a initial learning rate of 0.005 and use cosine annealing learning rate scheduler \cite{cosann2016}










\begin{figure*}[h]
  \centerline{\subfigure[AWGN]{  \includegraphics[width=0.5\textwidth]{figs/BCH_63_36_AWGN_ent.eps}
    \label{fig:bch_63_36_AWGN}}
    \hfil
    \subfigure[ETU]{\includegraphics[width=0.5\textwidth]{figs/BCH_63_36_ETU_ent.eps}
    \label{fig:bch_63_36_ETU}}
    }
    \captionsetup{font=small}
  \caption{BER for BCH(63,36) : On the left we see that entanglement of weights has almost no effect on performance for AWGN channel. However, for ETU channel on the right, NAMS-full outperforms NAMS-simple by more than 2 dB at a BER of $10^{-3}$.}
  \label{fig:ber_ent}
\end{figure*}

\section{Results}\label{sec:results}
We evaluate the error correction performance of different neural decoders and study the trade-off associated with entanglement for various channels. We also look at the robustness of the learned decoders by testing across channels. Additionally, we also study the adaptability of the learned decoders via additional training on the new channel. Finally, we demonstrate the performance of our NAMS decoder on practical data using USRP N210 SDR transmitter and receiver setup.








\subsection{Entanglement vs channels}
In \figref{fig:ber_ent} we plot the the Bit Error Rate (BER) performance of three variants of the NAMS decoder with varying degrees of entanglement. We choose BCH(63,36) specifically to enable direct comparison against the existing neural decoders \cite{nachmani2016,noms2017,nachmani2018,lian2019}. We can see from \figref{fig:bch_63_36_AWGN} that the performance is almost indistinguishable across the variants for AWGN channel. In \figref{fig:bch_63_36_ETU}, we perform the same experiment for ETU channel. For this simulation, we estimate the channel at the receiver using the pilots of OFDM symbols and use the MMSE equalizer from LTE MATLAB toolbox to perform equalization. We can see from \figref{fig:bch_63_36_ETU} that even after equalization, the choice of entanglement has significant impact on the error correction performance. At a moderate BER of $10^{-3}$, the NAMS-full outperforms NAMS-simple by more than 2 dB. 

\textbf{AWGN vs. ETU.} 
One of the key differences between the AWGN and ETU channels is that the multi-path components in the ETU channel create a correlation between adjacent received symbols. 
To understand the impact of this better, we compute the expected value of correlation coefficient between the consecutive received symbols post equalization \textit{i.e,} $\mathbb{E} [\rho_{l_v,l_{v+1}}]$ where $l_v$ is LLR of the $v^{\text{th}}$ received symbol. From this, we see that the correlation between adjacent symbols is much higher for ETU channel. For the setting in \figref{fig:ber_ent}, we observed an expected correlation coefficient of 0.02 and 0.15 between adjacent symbols for AWGN and ETU channels respectively. We conjecture that through training, the decoder learns to compensate for this imperfect equalization and improve the error correction performance.










\begin{figure}[h]
    \centering
 	\includegraphics[width=1.0\linewidth]{figs/BCH_63_36_ETU_bench.eps}
 	\captionsetup{font=small}
 	\caption{BER for BCH(63,36) ETU channel: NAMS outperforms NOMS and NNMS by 0.1-0.3 dB. Additionally, NAMS-full is close to WBP but with lower complexity.}
 	\label{fig:nams_comp}
\end{figure}

\begin{figure}[!h]
    \centering
 	\includegraphics[width=1.0\linewidth]{figs/BCH_63_36_ETU_robust.eps}
 	\captionsetup{font=small}
 	\caption{BER for BCH(63,36) ETU channel: The neural min-sum family is more robust than BP-PAN. Among the neural min-sum family, NAMS-full is up to 0.5 dB more robust compared to NOMS-full and NNMS-full.}
  	\label{fig:nams_robust}
\end{figure}

\subsection{Comparison with existing neural decoders}

For a complete comparison, we first compare NAMS-full against the neural BP decoders WBP \cite{nachmani2016} and BP-PAN \cite{lian2019}. \figref{fig:nams_comp} shows that with similar number of weights, the performance of NAMS-full is comparable to that of WBP decoder but with much lower complexity because of the min-sum approximation. Additionally, we also see that NAMS-full outperforms BP-PAN by more than 1dB, by taking advantage of the large number of weights. \footnote{We note that the results in \figref{fig:nams_comp} are based on our PyTorch implementation. When the simulations are ran using the publicly available code for NOMS and NNMS, the performance was poor and hence we use our own implementation of these algorithms with the hyper-parameters optimized for the ETU channels.}

In \figref{fig:nams_comp}, we compare the performance of fully parameterized NAMS against existing neural min-sum decoders for the ETU channels. \figref{fig:nams_comp} shows that NAMS-full outperforms NOMS \cite{noms2017} and NNMS \cite{nachmani2018} by 0.2-0.3 dB.


















 








\textbf{Robustness.\ } We test the robustness by taking a decoder trained on AWGN channel and testing on ETU channel.
\figref{fig:nams_robust} shows that when the channel changes from AWGN to ETU, the NAMS-full decoder still outperforms the 
original BP decoder. Additionally, the degradation in performance because of the change of channel is lower for NAMS-full and is up to 0.4 dB more robust compared to other min-sum decoders. It is also interesting to note that the performance of BP-PAN\cite{lian2019} degrades significantly when the decoder is directly reused on ETU channel. This is mainly because the weight selection by PAN is heavily dependent on the SNR which does not translate directly when the channel is changed. This shows that while BP-PAN\cite{lian2019} learns very well on a simple channel with very few parameters, NAMS takes advantage of the larger number of weights and generalizes well, making it more robust. 


\textbf{Adaptivity.\ } Given the dynamic nature of channel conditions experienced in practical wireless channels, we would like our decoder to be able to adapt to new channels quickly. 
To this end, we train our existing decoder, that was trained on AWGN channel, on minimal ($10-15 \%$) amount of data from ETU channel, and observe that we
recover the reliability of matched training (i.e., training on the ETU channel), 
showing that the NAMS decoder has good adaptive capabilities.\footnote{Complete results available at github repository.}


\subsection{Robustness and adaptivity to new channels}
While using the same channel conditions for training and testing gives the best performance \cite{heli2020}, we also desire the learned decoder to be ($a$) reasonably robust to any changes in channel conditions and ($b$) quickly adaptable to new channels. 

\subsection{Over The Air channels}
Apart from learning on the simulated data, we train and test the NAMS decoder on a real multi-path fading environment by using two USRP N200 series RF-transceivers to communicate over the air. We generate random data, encode it using BCH(63,36) code and modulate using BPSK. We add a synchronization preamble and transmit over the channel. At the receiver, we capture the frames, correct for frequency offset and perform channel equalization. We then demodulate the data to estimate the LLR and feed it to the NAMS decoder for inference.
\figref{fig:ota_tx_pow} shows that for a given power level, for example 3 dBm, NAMS-full corrects up to $2.88 \times$ more errors comapred to NAMS-simple, which has only two weights.

\begin{figure}[h]
    \centering!
 	\includegraphics[width=1.0\linewidth]{figs/BCH_63_36_OTA_ent.eps}
 	\captionsetup{font=small}
 	\caption{BER for BCH(63,36) OTA channel: at 3 dBm, NAMS-full corrects 2.88 $\times$ more errors compared to NAMS-simple.}
 	\label{fig:ota_tx_pow}
\end{figure}









\section{Interpretation}\label{sec:interpretation}
While neural decoders achieve some impressive gains, it is natural to wonder about the reason for these gains. In \cite{nachmani2016,nachmani2018,lian2019}, it has been conjectured that the weights mitigate the effect of cycles and thus improve the performance. While this is an intuitively sound explanation, the evidence for the same is lacking. We provide a first empirical evidence that the weights are closely associated with the cycles in the graph.







For ease of analysis, we consider the NNMS-full version of the neural min-sum decoder, which only has the normalization factors. We first study the relation between the weights learned and the cycles in the graph. 

\textbf{Connection to cycles.} We look at three different rates of length 63 BCH codes, which have different number of length-4 cycles. For each code, we measure the average weight across all edges and iterations. Given the degradation associated with the cycles, we would expect the learned weights to attenuate more in presence of higher number of cycles. From \tabref{cyclesAndWeights}, we can see that this holds true, with BCH(63,30) having the highest number of cycles and hence the least average weight, while the default weight in traditional min-sum decoder would be 1. This shows that the weights being learned are indeed commensurate with the cycles present. 






Since these cycles lead to correlation between the incoming beliefs at the variable nodes, we proceed to empirically estimate and compare these correlations to better understand the effect of the weights.

\begin{table}[!htp]\centering
\captionsetup{font=small}
\scriptsize
\begin{tabular}{lrrrr}\toprule
\multirow{2}{*}{Code} &\multirow{2}{*}{Length-4 cycles} &\multicolumn{2}{c}{\text{Average weight}} \\\cmidrule{3-4}
& &AWGN\hspace{2mm} &ETU \\\midrule
BCH (63,30) &10122\hspace{4mm} &0.2632\hspace{2mm} &0.4079 \\
BCH (63,36) &5909\hspace{5mm} &0.2987\hspace{2mm} &0.4318 \\
BCH (63,57) &1800\hspace{5mm} &0.3990\hspace{2mm} &0.6583 \\
\bottomrule
\end{tabular}
\vspace{2mm}
\caption{Number of short cycles present vs the weights learned. Weights are inversely proportional to number of cycles indicating that the weights correct more when number of cycles is high.}
\label{cyclesAndWeights}
\end{table}

\textbf{Correlation.} We consider the AWGN channel for this analysis, where the correlation between incoming beliefs at nodes is because of the cycles in Tanner graph. To find if the learned weights mitigate this effect, we measure and compare the expected pairwise  correlation coefficient between the incoming messages at the variable nodes \textit{i.e,} $\mathbb{E}[\rho_{\mu_{v,c_i}, \mu_{v,c_j}}]$, where $c_i, c_j \in M(v)$ and $i \neq j$ and the expectation is over the message and the channel. We observe from \figref{fig:corr_pos} that the correlation is reduced considerably across all the bit positions, which supports our conjecture that the weights are improving the reliability of the posterior probabilities by reducing the correlation. 

\begin{figure}[ht]
    \centering
 	\includegraphics[width=1\linewidth]{figs/BCH_63_36_corr.eps}
 	\captionsetup{font=small}
 	\caption{BCH(63,36) AWGN channel at SNR 4dB. The expected pairwise correlation between incoming messages at variable nodes is reduced across all positions.}
 	\label{fig:corr_pos}
\end{figure}
























  









\section{Conclusion and Remarks}\label{sec:conclusion}
In this work we provided a generalized framework for augmenting the min-sum variant of BP decoder for linear block codes. Through our simulations, we show that having more weights in the decoder is more advantageous in presence of complex channels. We also provide empirical evidence showing relation between the underlying code structure and the weights learned; we show that the weights learned are attenuating the effect of these cycles and improve the reliability of the posterior LLRs. As a future direction, we would like to understand more about the connections between the choice of hyper parameters and the structure of the code, that can help in faster learning of the weights. We also believe that a faster adaptation to newer channels is possible by using meta learning techniques. On the theoretical front, it would be interesting to analyze the performance of  neural decoders using Gaussian approximation \cite{gamal2001}.

















    



















    



    
    


\vspace{-.8em} 
 \medskip
 \small
 \bibliographystyle{IEEEtran}
 