\section{Introduction}

Dependence measures lie at the core of statistics and machine learning, with their widespread applications in diverse fields such as feature screening \citep{JMLR:v21:19-537}, template matching \citep{goshtasbyTwoStageCrossCorrelation1984}, document clustering \citep{zhangDocumentClusteringCorrelation2012}, image retrieval \citep{liStatisticalCorrelationAnalysis2002} and graph structure learning \citep{JMLR:v22:20-1137}.

Proposed in the late 19th century, Pearson's $r$ (\citeyear{pearsonMathematicalContributionsTheory1896}) has been one of the main tools for scientists and engineers to study bivariate dependence during the 20th century. It is remarkably unaffected by the passage of time \citep{leerodgersThirteenWaysLook1988} and still goes strong in the 21th century, providing a benchmark for other new proposed dependence measures \citep{szekelyMeasuringTestingDependence2007, reshefMeasuringDependencePowerfully2016, panBallCovarianceGeneric2020}.

It is usually taken for granted that Pearson's $r$ can only capture linear relationship, as depicted in classical textbooks and contemporary literatures. For example:

\begin{quotation}
    ``When we are interested in the joint distribution of two random variables, it is useful to have a summary of how much the two random variables depend on each other. The covariance and correlation are attempts to measure that dependence, but they only capture a particular type of dependence, \textbf{namely linear dependence}"\citep{degrootProbabilityStatistics2012}
\end{quotation}

See also \citet{moodIntroductionTheoryStatistics1974}, \citet{ casellaStatisticalInference2002} and \citet{wassermanAllStatisticsConcise2004} for similar claims. 

To overcome the linear limitation of Pearson's $r$, a number of dependence measures are proposed. To name a few, Hoeffding's independence test \citep{hoeffdingNonParametricTestIndependence1948}, Hilbert-Schmidt Independence Criterion \citep{JMLR:v6:gretton05a}, distance correlation \citep{szekelyMeasuringTestingDependence2007}, randomized dependence coefficient \citep{lopez-pazRandomizedDependenceCoefficient2013}, Heller-Heller-Gorfine distance \citep{hellerConsistentMultivariateTest2013}, maximal information coefficient \citep{reshefMeasuringDependencePowerfully2016}, ${G^2}$ statistic \citep{wangGeneralizedRsquaredDetecting2017}, Ball Covariance \citep{panBallCovarianceGeneric2020} and cross rank coefficient $\xi $ \citep{chatterjeeNewCoefficientCorrelation2021}. Among these contributions, distance correlation (dCor) and maximal information coefficient(MIC) have attracted special attention. The former is said to represent one of the most important breakthroughs in measuring the dependence between two random variables\citep{panBallCovarianceGeneric2020}, while the latter is regarded as a measure for the 21 century \citep{speedCorrelation21stCentury2011}. Without exception, all these measures break the linear limitation by taking approaches completely different from that of Pearson's $r$, and none of them can be upward compatible with Pearson's $r$.  

Our research shows that the long-standing consensus that \textit{Pearson's $r$ is only a linear statistic} might be wrong. We find that Pearson's $r$ has potential to capture both linear and nonlinear association as long as the relationship is monotonic. In fact, Pearson's $r$ reflects the strength of arbitrary monotonic relationships, it just undervalues those nonlinear ones. All we need is to adjust the underestimation. We derive a new covariance inequality which is tighter than the famous Cauchy-Schwarz Inequality \citep{ casellaStatisticalInference2002}, and leverage it to adjust Pearson's $r$. For nonlinear monotonic scenarios, the adjusted $r$ outperforms not only its couterpart, but also several recently proposed measures.

Our main contributions are as follows:

\begin{itemize}
    \item We proposed an ajusted $r$ which can measure both linear and nonlinear monotonic dependence. The adjusted $r$ is more powerful for testing nonlinear monotonic independence, and more accurate for measuring nonlinear monotonic independence.
    \item  The adjusted $r$ is as simple and even more intuitive than Pearson's $r$ itself. It is upward compatible with Pearson's $r$. In fact, it reverts to Pearson's $r$ and Spearman's $\rho$ under certain conditions.
    \item Several propositions in textbooks and literatures about covariance and correlation are revised. 
\end{itemize}

\section{Why to Adjust Pearson's $r$} 
\label{why-to-adjust} 
We will start with our motivation to adjust Pearson's $r$.

\subsection{The Anti-intuitive Side of Pearson's $r$}

As stated in textbooks \citep{wassermanAllStatisticsConcise2004}, Pearson's $r$ between observed random sample $X$ and $Y$ is simply the ratio between the covariance and the product of standard deviation, i.e., $r = {{{\mathop{\rm cov}} \left( {X,Y} \right)} \mathord{\left/
    {\vphantom {{{\mathop{\rm cov}} \left( {X,Y} \right)} {sd\left( X \right)sd\left( Y \right)}}} \right.
    \kern-\nulldelimiterspace} {sd\left( X \right)sd\left( Y \right)}}$. We prefer the formula $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} $ to $sd\left( X \right)sd\left( Y \right)$ in the denominator, although they are mathematically equivalent. $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} $ tells that the denominator in Pearson's $r$ is the \textit{geometric mean} of ${\mathop{\rm var}} \left( X \right)$  and  ${\mathop{\rm var}} \left( Y \right)$. Just like more famous \textit{arithmetic mean} ${\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right)$, the \textit{geometric mean} $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} $ is just another type of average value of  ${\mathop{\rm var}} \left( X \right)$  and  ${\mathop{\rm var}} \left( Y \right)$. Thus, the essence of the denominator is (average) \textit{variance} rather than \textit{covariance}. And the paradigm behind Pearson's $r$ is to scale covariance with variance, i.e.,$r = {{covariance} \mathord{\left/
    {\vphantom {{covariance} {variance}}} \right.
    \kern-\nulldelimiterspace} {variance}}$. Although the two concepts \textit{variance} and \textit{covariance} sound similar, they are quite different things in nature: the former is a measure for dispersion characterization, while the latter a measure of relationship between two variables, just as its affix $co - $ implies. Regarding their disparate nature, it is counter-intuitive to scale covariance with variance. \textit{Covariance} should be scaled by \textit{covariance}, rather than \textit{variance}, although this average \textit{variance} happens to be bound for the \textit{covariance}.

\subsection{Boundedness Doesn't Ensure Optimum Unless Being Supremum}

According to Cauchy-Schwarz Inequality \citep{ casellaStatisticalInference2002} 
\begin{equation}
    \left| {{\mathop{\rm cov}} \left( {X,Y} \right)} \right| \le \sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} ,    
\end{equation}
the geometric mean variance $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} $ mathematically provides bound for covariance ${\mathop{\rm cov}} \left( {X,Y} \right)$, making the latter dimensionless and comparable. However, a crucial issue that has been neglected so far is that boundedness doesn't ensure optimum. With the mean inequality series \citep{hardyInequalities1952}, we can derive that $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)}  \le {\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right)$, since geometric mean is always less than or equal to arithmetic mean for non-negative values. We get immediately an even looser bound for covariance, i.e., ${\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right)$, with which we can define another correlation statistic, ${r^ - }\left( {X,Y} \right) = {{{\mathop{\rm cov}} \left( {X,Y} \right)} \mathord{\left/
    {\vphantom {{{\mathop{\rm cov}} \left( {X,Y} \right)} {{\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right)}}} \right.
    \kern-\nulldelimiterspace} {{\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right)}}$. We proved that the capture range of ${r^ - }$ is limited to identical relationships, i.e., $Y =  \pm X + b$, which are special cases of linear relationships $Y = aX + b$ with $a$ fixed to  $ \pm 1$(described later in Theorem \ref{thr4}; Corollary \ref{clr3}). In fact, any bound can be employed to scale covariance to a dimensionless and comparable statistic, and the geometric mean variance is just an option, not the only choice, let alone the optimal one. Scaling the ${\mathop{\rm cov}} \left( {X,Y} \right)$ to the range $\left[ { - 1, + 1} \right]$ is not the only thing that matters. Theoretically, the inequality behind Pearson's $r$,  $\left| {{\mathop{\rm cov}} \left( {X,Y} \right)} \right| \le \sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} $, is always strict in nonlinear scenarios, which implies that it can be improved. Now that the capture range of ${r^ - }$ can be extended from identical to linear relationship with a tighter bound, the capture range of $r$ can likewise be extended with an even tighter bound. The key is to find the supremum for covariance.

\section{How to Adjust Pearson's $r$}
\subsection{Tighter Bound Leads to Wider Capture Range}
    
Choosing or establishing an appropriate inequality is usually a key breakthrough in the solution of a problem. With Rearrangement Inequality \citep{hardyInequalities1952}, we derive the covariance inequality series, which provides tighter bound for covariance.
    
\begin{theorem}\label{thr1}
    For an observed random sample $\left( {X,Y} \right) = \left\{ {\left( {{x_i},{y_i}} \right):i = 1,2, \cdots ,n} \right\}$, the covariance inequality series holds:
    \begin{equation}
        {\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right) \ge \sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)}  \ge \left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right| \ge \left| {{\mathop{\rm cov}} \left( {X,Y} \right)} \right|
    \end{equation}

\end{theorem}

Here, ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)$ is the rearrangement covariance, i.e., the covariance between the rearranged versions of $X$ and $Y$ in perfect covarying status, which is defined as follows.


\begin{definition}[Rearrangement covariance]\label{def1}
  Let ${X^ \uparrow },{X^ \downarrow }$ and ${Y^ \uparrow },{Y^ \downarrow }$ be increasing and decreasing rearrangement of $X$ and $Y$ respectively,
  \begin{equation}
      \begin{array}{l}
          {X^ \uparrow } \buildrel \Delta \over = \left\{ {{x_{(1)}},{x_{(2)}}, \cdots ,{x_{(n)}}} \right\},{x_{(1)}} \le {x_{(2)}} \le  \cdots  \le {x_{(n)}},{x_{(i)}} \in X   \\
          {X^ \downarrow } \buildrel \Delta \over = \left\{ {{x_{[1]}},{x_{[2]}}, \cdots ,{x_{[n]}}} \right\},{x_{[1]}} \ge {x_{[2]}} \ge  \cdots  \ge {x_{[n]}},{x_{[i]}} \in X \\
          {Y^ \uparrow } \buildrel \Delta \over = \left\{ {{y_{(1)}},{y_{(2)}}, \cdots ,{y_{(n)}}} \right\},{y_{(1)}} \le {y_{(2)}} \le  \cdots  \le {y_{(n)}},{y_{(i)}} \in Y   \\
          {Y^ \downarrow } \buildrel \Delta \over = \left\{ {{y_{[1]}},{y_{[2]}}, \cdots ,{y_{[n]}}} \right\},{y_{[1]}} \ge {y_{[2]}} \ge  \cdots  \ge {y_{[n]}},{y_{[i]}} \in Y
      \end{array}
  \end{equation}
  neither $X$ nor $Y$ is constant for all values. Rearrangement covariance is defined as:
  \begin{equation}
      {\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right) \buildrel \Delta \over = \left\{ {\begin{array}{*{20}{c}}
                  {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \uparrow }} \right) = \sum\limits_{i = 1}^n {\frac{{\left( {{x_{(i)}} - \bar x} \right)\left( {{y_{(i)}} - \bar y} \right)}}{{n - 1}}} ,\;\;if{\mathop{\rm cov}} \left( {X,Y} \right) \ge 0} \\
                  {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \downarrow }} \right) = \sum\limits_{i = 1}^n {\frac{{\left( {{x_{(i)}} - \bar x} \right)\left( {{y_{[i]}} - \bar y} \right)}}{{n - 1}}} ,\;\;if{\mathop{\rm cov}} \left( {X,Y} \right) < 0}
              \end{array}} \right.
  \end{equation}
\end{definition}

Rearrangement covariance locates the boundary of covariance between $X$ and $Y$ on the same side. If $X$ and $Y$ tend to covary in the same direction, rearrangement covariance is defined as the covariance of $X$ and $Y$ in similar order; if $X$ and $Y$ tend to covary in the opposite direction, it is defined as the covariance of $X$ and $Y$ in opposite order. Note that when ${\mathop{\rm cov}} \left( {X,Y} \right) = 0$, the value of ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)$ can be assigned either ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \uparrow }} \right)$ or ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \downarrow }} \right)$. For simplicity, we just set it as ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \uparrow }} \right)$ here.

Theorem \ref{thr1} shows that the absolute rearrangement covariance is tighter than the geometric mean of covariance (in Cauchy-Schwarz Inequality), which is tighter in turn than the arithmetic mean. And the equality conditions are given in the following theorems.

\begin{theorem}\label{thr2}
  The equality in $\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right| \ge \left| {{\mathop{\rm cov}} \left( {X,Y} \right)} \right|$ holds if and only if each of $X$, $Y$ is a.s. a monotonic function of the other.
\end{theorem}

Theorem \ref{thr2} shows that $\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|$ is supremum of covariance between $X$ and $Y$ in arbitrary monotonic scenarios, including both linear and nonlinear ones.

\begin{theorem}\label{thr3}
  The equality in $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)}  \ge \left| {{\mathop{\rm cov}} \left( {X,Y} \right)} \right|$ holds if and only if each of $X$, $Y$ is a.s. a linear function of the other, i.e., $Y = aX + b$ for some constants $a$ and $b$, where $a \ne 0$ \citep{degrootProbabilityStatistics2012}.
\end{theorem}

Theorem \ref{thr3} shows that $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} $ is sharp bound of covariance between $X$ and $Y$ only in linear scenarios; otherwise, the bound is never attainable and the inequality is always strict.

\begin{theorem}\label{thr4}
  The equality in ${\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right) \ge \left| {{\mathop{\rm cov}} \left( {X,Y} \right)} \right|$ holds if and only if each of $X$, $Y$ is a.s. an identical function (with shift) of the other, i.e., $Y =  \pm X + b$ for some constant $b$.
\end{theorem}

Theorem \ref{thr4} shows that ${\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right)$ is sharp bound for the covariance between $X$ and $Y$ only in identical scenarios; otherwise, the bound is never attainable. For non-identical linear scenarios, the inequality is always strict, let alone those nonlinear ones.


\begin{theorem}\label{thr5}
  The equality in $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)}  \ge \left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|$ holds if and only if the augmented linear condition is met. Here, augmented linear condition means $Y$ is arbitrary permutation of ${aX + b,a \ne 0}$  and $sgn\left( {{\mathop{\rm cov}} \left( {X,Y} \right)} \right) = sgn\left( a \right)$.
\end{theorem}

\begin{theorem}\label{thr6}
  The equality in ${\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right) \ge \sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} $  holds if and only if the variance of $X$ and $Y$ is identical.
\end{theorem}

Complete proofs of all the theorems aforementioned and undermentioned are available in Appendix \ref{appendix_proof}, and also available on https://arxiv.org/abs/2205.04571.

Now, we leverage the covariance inequality series to define dependence statistics. Taking covariance as the same numerator and these three bounds as the denominators respectively, we get three corresponding dependence statistics ${r^ - }$, $r$, and ${r^\# }$. Here, ${r^\# }$ is the adjusted $r$ we proposed.

\begin{definition}[The degenerated $r$]\label{def2}
  ${r^ - }$ is defined as:
  \begin{equation}
      {r^ - }\left( {X,Y} \right) = \frac{{{\mathop{\rm cov}} \left( {X,Y} \right)}}{{{\textstyle{1 \over 2}}\left( {{\mathop{\rm var}} \left( X \right) + {\mathop{\rm var}} \left( Y \right)} \right)}}\;
  \end{equation}
\end{definition}

${r^ - }$  is a statistic which is only able to capture identical relationship, i.e., $Y =  \pm X + b$. The mathematical symbol $minus -$ is utilized here to imply its narrower capture range.

\begin{definition}[Pearson's $r$]\label{def3}
  Pearson's $r$ is defined as \citep{degrootProbabilityStatistics2012}:
  \begin{equation}r\left( {X,Y} \right) = \frac{{{\mathop{\rm cov}} \left( {X,Y} \right)}}{{\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} }}\;\end{equation}
\end{definition}

\begin{definition}[The adjusted $r$]\label{def4}
  The adjusted $r$ is defined as:
  \begin{equation}{r^\# }\left( {X,Y} \right) = \frac{{{\mathop{\rm cov}} \left( {X,Y} \right)}}{{\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|}}\end{equation}
\end{definition}

We borrow the musical sharp symbol $\# $ to denote the adjusted $r$, signifying that the statistic has sharp value because of its sharp bound. Just like $C\# $ is pronounced as \textit{C-sharp}, ${r^\# }$ is pronounced as \textit{r-sharp}.

\begin{theorem}\label{thr6.5}
    The adjusted $r$ posesses the fundamental properties as a dependence statistic in the following way:
    \begin{itemize}
        \item Standardization: The value of ${r^\# }\left( {X,Y} \right)$ must be between $-1$ and $+1$ inclusive, i.e., $ - 1 \le {r^\# }\left( {X,Y} \right) \le 1$;
        \item Attainability: ${r^\# }\left( {X,Y} \right) =  \pm 1$ holds if and only each of $X$, $Y$ is a.s. an increasing/decreasing monotonic function of the other;
        \item Symmetry: ${r^\# }\left( {X,Y} \right) = {r^\# }\left( {Y,X} \right)$;
        \item Shift invariance:${r^\# }\left( {X + b,Y} \right) = {r^\# }\left( {X,Y} \right)$
        \item Scale invariance:${r^\# }\left( {aX,Y} \right) = sgn\left( a \right){r^\# }\left( {X,Y} \right)$
        \item Zero correlation: The population value of  ${r^\# }\left( {X,Y} \right)$ is zero if $X$ and $Y$ are mutually independent.
    \end{itemize}
\end{theorem}


The relationship among these three statistics are depicted in Figure \ref{fig_01}. With covariance as the same numerator and the denominators arranged in ascending order, it is immediate that $\left| {{r^\# }} \right| \ge \left| {{r^ - }} \right| \ge \left| r \right|$.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.75\textwidth]{fig/figure_01.pdf}
  \caption{Covariance inequality series and dependence statistics. The inequality series provides three bounds for the covariance. These three bounds lead to three different dependence statistics which capture different types of relationships. The tighter the bound, the wider the capture range.}\label{fig_01}
\end{figure}

The difference lies not only on their magnitudes, but also their capture ranges. The capture range of adjusted $r$ is wider than that of Pearson's $r$, which is wider than that of degenerated $r$. Along the inequality series from right to left, the capture range is expanding from identical, through linear and finally to (linear and nonlinear) monotonic relationships, with the bounds getting tighter and tighter.


\begin{corollary}\label{clr1}
  $\left| {{r^\# }\left( {X,Y} \right)} \right| \le 1$ and the equality holds if and only each of $X$, $Y$ is a.s. a monotonic function of the other.
\end{corollary}

\begin{corollary}\label{clr2}
  $\left| {r\left( {X,Y} \right)} \right| \le 1$ and the equality holds if and only if each of $X$, $Y$ is a.s. a linear function of the other, i.e., $Y = aX + b$ for some constants $a$ and $b$, where $a \ne 0$ \citep{degrootProbabilityStatistics2012}.
\end{corollary}


\begin{corollary}\label{clr3}
  $\left| {{r^ - }\left( {X,Y} \right)} \right| \le 1$ and the equality holds if and only each of $X$, $Y$ is a.s. an identical function (with shift) of the other, i.e., $Y =  \pm X + b$ for some constant $b$.
\end{corollary}

To sum up, the capture ranges of these three dependence statistics are expanding in the following way:
\begin{equation}
  \begin{array}{*{20}{c}}
      {\left\{ {capture\;range\;of\;{r^ - }} \right\}}& \subset &{\left\{ {capture\;range\;of\;r} \right\}}& \subset &{\left\{ {capture\;range\;of\;{r^\# }} \right\}}\\
       \uparrow &{}& \uparrow &{}& \uparrow \\
      {identical\;relationship}&{}&{linear\;relationship}&{}&{monotonic\;relationship}
      \end{array}
\end{equation}

If a statistic is applied to a scenario out of its capture range, underestimation will arise. The following corollaries depict the underestimation theoretically.

\begin{corollary}\label{clr4}
  $r\left( {X,Y} \right) \le {r^\# }\left( {X,Y} \right)$ and the equality holds if and only if the augmented linear condition is met. Here, augmented linear condition means $Y$ is arbitrary permutation of $aX + b,a \ne 0$  and $sgn\left( {{\mathop{\rm cov}} \left( {X,Y} \right)} \right) = sgn\left( a \right)$
\end{corollary}

Corollary \ref{clr4} shows that Pearson's $r$ will undervalue non-(augmented)linear monotonic relationships.

\begin{corollary}\label{clr5}
  $\left| {{r^ - }\left( {X,Y} \right)} \right| \le \left| {r\left( {X,Y} \right)} \right|$, and the equality holds if and only if the variance of $X$ and $Y$ is identical.
\end{corollary}


\begin{corollary}\label{clr6}
  For linear relationship $Y = aX + b,a \ne 0$, we have:
  \begin{itemize}
      \item ${r^ - }\left( {X,Y} \right) = {{2a} \mathord{\left/
          {\vphantom {{2a} {\left( {1 + {a^2}} \right)}}} \right.
          \kern-\nulldelimiterspace} {\left( {1 + {a^2}} \right)}}$;
      \item  the degenerated $r$ will not underestimate the strength of linear relationships if and only if the relationship between $Y$ and $X$ is identical, i.e.,  $Y =  \pm X + b$.
  \end{itemize}
\end{corollary}

Corollary \ref{clr5} and \ref{clr6} show that the degenerated $r$ will undervalue non-identical linear relationships.

It is to noted that a statistic with wider capture range is not meant to replace a narrower one. To capture identical relationship, ${r^ - }$ is good enough and even the best choice, in sense that it measures identity accurately and will not be confused with other non-identical linear relationships; in contrast, if $r$ is employed, a score of 0.7 can either be strength of identity, or linearity, which will suffer from confusion. In short, to capture identical dependence, ${r^ - }$ is the best choice; to capture linear dependence, $r$ might be the best possible; for general monotonic dependence,  ${r^\# }$ might be the right one.


\subsection{Permutahedron Provides a Clear Picture for Covariance and Correlation}

For an intuitive explanation of the covariance and the adjusted $r$, we will resort to permutahedron. Permutahedron is a very interesting polyhedron because its vertices are in a one-to-one correspondence with permutations \citep{vinceRearrangementInequalityPermutahedron1990}, and therefore it can visualize the covarying spectrum intuitively.

To take a prototype situation, let $X = \left( {1,2,3,4} \right)$ and $Y$ be arbitrary permutation of $\left\{ {1,2,4,5} \right\}$. With reference $X = \left( {1,2,3,4} \right)$ fixed in ascending order, there are two perfect covarying $Y$ sequences, i.e., $\left( {1,2,4,5} \right)$ and $\left( {5,4,2,1} \right)$, in similar and opposite order respectively. We can place all the permutations and the corresponding ${r^\# }\left( {X,Y} \right)$ on a permutahedron P4 as shown in Figure \ref{fig_02}. Assume $\pi $ and $\pi '$ are two permutations of $\left\{ {1,2,4,5} \right\}$, there will be an edge directing from node $\pi $ to node $\pi '$  where node $\pi '$  is obtained by interchange neighbouring pairs ${\pi _j},{\pi _{j + 1}}$ of $\pi $ if ${\pi _j} < {\pi _{j + 1}}$.

\begin{figure}[!h]
    \centering
    \includegraphics[width=0.75\textwidth]{fig/figure_02.pdf}
    \caption{Permutahedron P4 as a schematic to illustrate the covarying status}
    \label{fig_02}
\end{figure}

Permutahedron provides a systematic picture for the covarying status between $X$ and $Y$. There are totally $n! = 4! = 24$ covarying states, which is the same as the number of permutations of $Y$ against the fixed $X$. Through any path from similar-order node $\left( {1,2,4,5} \right)$ to opposite-order node $\left( {5,4,2,1} \right)$, the actual covariance of the precursor  $\pi $ is always greater than that of the successor  $\pi '$ . Whenever there is a swap between the neighbouring elements in $Y$, the covariance of $X$ and $Y$ monotonically decreases.

\begin{proposition}\label{pro1}
  ${r^\# }\left( {{X^ \uparrow },\pi } \right) < {r^\# }\left( {{X^ \uparrow },\pi '} \right)$ if  $\pi '$ is obtained by interchange neighbouring pairs ${\pi _j},{\pi _{j + 1}}$ of $\pi $, where ${\pi _j} < {\pi _{j + 1}}$.
\end{proposition}

With the permutahedron, it is obvious that covariance quantifies the extent of all the covarying states exactly. Through any path forward, the covariance will decrease to negative ones, and finally reach the lower limit, ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \downarrow }} \right)$; through any path backward, the covariance will increase to positive ones, and finally reach the upper limit ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \uparrow }} \right)$. Thus, ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \downarrow }} \right)$ and ${\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^ \uparrow }} \right)$ prove to be the infimum and supremum and for ${\mathop{\rm cov}} \left( {X,Y} \right)$.


As standardized version of covariance, a correlation statistic should increase to $ + 1$ if the covarying status approaches to the supremum node, and decrease to $ - 1$ with the covarying status being closer to the infimum node. If we anchor these two endpoints to be $ \pm 1$, the magnitude of covarying status can naturally be expressed as the percentage of the maximum possible covariance that is actually obtained, or as the ratio of the actual covariance to the maximum possible covariance on the same side. Since the absolute values of supremum and infimum are not necessarily equal, we will scale the actual covariance with supremum or infimum respectively, according to the sign of the covariance.

As shown in Figure \ref{fig_02}, from the bottom up, the ${r^\# }\left( {X,Y} \right)$ monotonically decreases just as the covariance does. The farther away from the node $\left( {1,2,4,5} \right)$, the smaller the ${r^\# }$ score; the closer to the node $\left( {5,4,2,1} \right)$, the greater the absolute ${r^\# }$ score is. Following the path indicated by the arrows, the covarying status is from similar order, through complete disorder, to opposite order, and the ${r^\# }\left( {X,Y} \right)$ values will correspondingly vary from positive one ($+1$), through near zero, to negative one ($-1$), expressing directly and thoroughly the covarying status between $X$ and $Y$ in a standardized way.

Generally, the score of a correlation statistic is fairly easy to interpret when at an extreme (e.g., $-1$, 0, or $+1$).However, for values somewhere in the middle the correlation is more difficult to interpret. With the Permutahedron shown in Figure \ref{fig_02}, a clear picture covering all possible values is systematiclly presented. 

The essence of correlation is about order, the extent of how things varies in the same or opposite direction. Putting things in order, we get the ideal covariance, i.e., the maximum or minimum possible covariance. Compared with Pearson's $r$, we scale the actual covariance with the `perfect' \textit{covariance} instead of their average \textit{variance}. The numerator and the denominator share the same mathematical form and are homogeneous in nature. With the sharp bound, we are able to make most of the covariance, resulting a new correlation statistic which is able to measure the strength of monotonic relationship accurately.

It is also to be noted that the above mentioned intuitive interpretation about covariance and the adjusted $r$ is distribution free, since no assumptions about the population are required for the covarying process. 


\section{New Insights into Traditional Correlation Statistics}
Let's revisit two traditonal correlation statistics, Pearson's $r$ and Spearman's $\rho$, after the intuitively exploring the covariance and correlation. 
\subsection{ New Insights into Pearson's $r$}
A surprising insight into $r$ is that it can reflect not only the strength of linear relationships, but also the strength of nonlinear monotonic relationships. The only difference is that it measures (augmented) linear relationships to full extent, but nonlinear ones to some extent.

\begin{corollary}\label{clr7}
  Pearson's $r$ can measure the strength of arbitrary monotonic relationship in the following way:
  \begin{equation}
      Score\;of\;Pearson's\;r = {\textstyle{{\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|} \over {\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} }}} \times Stength\;of\;monotonic\;relationship\end{equation}
\end{corollary}

Here, the strength of arbitrary monotonic relationship is accurately expressed by the adjusted $r$, lying in the interval $\left[ { - 1, + 1} \right]$; and ${{\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|} \mathord{\left/
  {\vphantom {{\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|} {\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} }}} \right.
  \kern-\nulldelimiterspace} {\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} }}$ is the proportionality coefficient. It is immediate that ${{\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|} \mathord{\left/
  {\vphantom {{\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|} {\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} }}} \right.
  \kern-\nulldelimiterspace} {\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} }} \le 1$ and the equality holds if and only if the augmented linear condition is met, with Theorem \ref{thr5}.

The proportionality coefficient quantifies the extent of underestimation, which is determined by the type of relationship and its nonlinearity. The greater the nonlinearity, the severer the underestimation.

\begin{example}
  Figure \ref{fig_03} shows the proportionality coefficient of Pearson's $r$ with respect to nonlinear order $k$ for power functions $Y = {X^k}$. For the linear case, $k = 1$, the proportionality coefficient is $1.00$; with nonlinear order $k = 23$, the proportionality coefficient takes the value $0.50$; with even higher nonlinear order $k = 100$, the proportionality coefficient drops to $0.26$. 
\end{example}
  

\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.75\textwidth]{fig/figure_03.png}
    \caption{The proportionality coefficient of Pearson's $r$ with respect to nonlinear order $k$ for power function $Y = {X^k}$, $X \sim U\left( {0,1} \right)$.}\label{fig_03}
\end{figure}


An interesting question might arise in one's mind: how can the simple replacement of $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)} $ with $\left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right|$ leads to the capture range expanding from linearity to (nonlinear) monotonicity? The capture range is inherited from covariance itself, which can potentially detect and measure arbitrary monotonic dependence, if scaled properly. Pearson's $r$ is also measuring nonlinear monotonicity to some extent. The replacement is nothing more than adjusting the underestimation.

All we know that uncorrelatedness doesn't imply independence, and a further proposition is often depicted in textbooks: zero-correlation only means that there is no linear relationship. However, this proposition is inaccurate. In fact, zero-correlation is equivalent to zero-covariance. In sense that covariance is actually measuring arbitrary monotonic relationship, zero-correlation implies that there is no monotonic relationship between $X$ and $Y$, including those nonlinear ones. Geometrically, zero-correlation means the two sequence are completely out of order, a covarying status farthest away from similar order and opposite order, as already depicted in Figure \ref{fig_02}. Since the path length from the supremum node to the infimum node is ${{n(n - 1)} \mathord{\left/
    {\vphantom {{n(n - 1)} 2}} \right.
    \kern-\nulldelimiterspace} 2}$, thus the node with minimum score lies farthest away from both nodes approximately at the distance of ${{n(n - 1)} \mathord{\left/
    {\vphantom {{n(n - 1)} 4}} \right.
    \kern-\nulldelimiterspace} 4}$.

Another insight into Pearson's $r$ might be more surprising: the score of Pearson's  $r$  doesn't necessarily represents the extent of linearity. It is no doubt that the extent of linearity can be quantified by $r$ score, while the converse is not necessarily true. Pearson's $r$ is unable to distinguish noised linearity from perfect nonlinear monotonicity. For a value such as $r\left( {X,Y} \right) = 0.70$, the relationship between $Y$ and $X$ can be either noised linear function, i.e., $Y = X + \varepsilon $, with $X \sim U\left( {0,1} \right),\varepsilon  \sim N\left( {0,0.3} \right)$; or nonlinear monotonic function, i.e., $Y = {X^{10}}$, without any noise added. Thus, a moderate   score can imply either moderate linearity, or perfect nonlinear monotonicity. Obviously, moderate linearity can never be equivalent to perfect monotonicity. The confusion resides in the fact that the Pearson's $r$ underestimates the nonlinear monotonicity. As for ${r^\# }$, such confusion is avoidable.

\begin{example}
Figure \ref{fig_04} shows the scatter plot of perfect nonlinear relationship (left) and noised linear relationship (right). Same score (both as 0.70) implies that Pearson's $r$ is unable to distinguish these two patterns. With adjusted $r$, these two patterns will not be confused, since the ${r^\# }$ score on the left is 1.00 while that on the right is 0.72. 
\end{example}


\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.75\textwidth]{fig/figure_04.png}
    \caption{Scatter plot of perfect nonlinear relationship (left) and noised linear relationship (right). The red lines depict the monotonic function $Y = {X^{10}}$ (left) and linear function $Y = X$(right). There are 100 data points lying exactly on the curve (left) and loosely spreading around a straight line (right) on each panel.}\label{fig_04}
\end{figure}


\subsection{ New Insights into Spearman's $\rho$}
Traditionally, for measuring general monotonic relationships rather than only linear ones, we resort to rank-based correlation statistics, such as Spearman's $\rho$. Spearman's $\rho$ is simply the application of Pearson's $r$ in the data converted to ranks before calculating the statistic. Now that $\rho$ and $r$ share the same mathematical formula, why can the former measure nonlinear monotonicity while the latter only linearity? The underline theoretical foundation remains unrevealed.

In fact, the key is not the ranking, but still the sharp bound. Let $X$ and $Y$ be ranks themselves, Spearman's $\rho$ is equivalent to the adjusted $r$.

\begin{theorem}\label{thr7}
    If $X$ and $Y$ are both permutations of $\left\{ {1,2, \cdots ,n} \right\}$, then we have:
    \begin{itemize}
        \item $\sqrt {{\mathop{\rm var}} \left( X \right){\mathop{\rm var}} \left( Y \right)}  = \left| {{\mathop{\rm cov}} \left( {{X^ \uparrow },{Y^{ \uparrow  \downarrow }}} \right)} \right| = {\textstyle{1 \over {12}}}n\left( {n + 1} \right)$;
        \item  Spearman's $\rho$ is equivalent to the adjusted $r$, i.e., $\rho \left( {X,Y} \right) = {r^\# }\left( {X,Y} \right)$.
    \end{itemize}

\end{theorem}

In other words, if calculated on the ranks, ${r^\# }\left( {X,Y} \right)$ reverts to $\rho \left( {X,Y} \right)$. Thus, the reason why Spearman's $\rho$ can measure nonlinear monotonicity is that the denominator happens to be sharp bound. In fact, it is another special case of the augmented linear condition stated in Theorem \ref{thr5} and Corollary \ref{clr4}: $Y$ is permutation of $aX + b$, with $a = 1,b = 0$ and $X = \{ 1,2, \cdots ,n\} $. To measure arbitrary monotonic dependence, all we need is a sharp bound, rather than degenerating raw values into ranks.

Without exception, all measures designed for monotonic dependence are utilizing the order information. However, what we utilize here is the original information about order, rather than the ranks. With the sample size $n$,  there are totally ${{\left( {{n^3} - n} \right)} \mathord{\left/
    {\vphantom {{\left( {{n^3} - n} \right)} 6}} \right.
    \kern-\nulldelimiterspace} 6}$ possible $\rho$ values between $ - 1$ and $ + 1$ \citep{kendallNewMeasureRank1938}, whatever raw values are and however correlated patterns differ. The sensitivity of $\rho$ is inadequate. Mapping numerical values to their ranks does of course produce a certain loss of information.A very small difference between two values may no longer be distinguished from a very large difference, since both could be mapped to adjacent ranks and, therefore, have an equal (transformed) distance of 1 \citep{ruizFormalEmpiricalAnalysis2012}. On the contrary, the sensitivity of  ${r^\# }$ is the same as that of the original covariance and Pearson's $r$, without any information loss.

Another advantage of ${r^\# }$ over Spearman's $\rho$ is that the former is comparable with $r$, while the latter is not. For nonlinear correlation, the score of Spearman's $\rho$ might be remarkably greater than the score of Pearson's $r$. One may attempt to search for nonlinear relationships in data by checking whether the score of Spearman's $\rho$ far exceeds that of the Pearson's $r$. However, it might be meaningless and even impossible to compare these two scores directly, since they are measuring correlation on quite different bases. $\left| \rho  \right|$ can be either greater or less than $\left| r \right|$, and the difference  $\left| \rho  \right| - \left| r \right|$ is confusing. On the contrary, $\left| {{r^\# }} \right| - \left| r \right|$ is always depicting the underestimation magnitude with a non-negative value. As aforementioned, nonlinearity gives rise to underestimation, and the extent of underestimation increases with the degree of nonlinearity. Thus, it is reasonable to define monotonic nonlinearity by the extent of underestimation, i.e., $\left| {{r^\# }} \right| - \left| r \right|$, which lies in the range $\left[ {0,1} \right]$. Zero value implies no monotonic nonlinearity, and the value will approach to positive one with the increase of nonlinearity.

Pearson's $r$ and Spearman's $\rho$ are two (of the) most popular classical association measures, with the former benchmarking linear dependence and the latter benchmarking monotonic dependence. The adjusted $r$ provides a unified formalization for both linear and nonlinear monotonicity, with $r$ and $\rho$ as its special cases.

\section{Performance Evaluation of the Adjusted $r$}
After exploring the intuitive and theoretical properties of the adjusted $r$, we will evaluate its performance. Generally, characterising the dependence between two variables includes two main tasks  \citep{geenensHellingerCorrelation2020}:

\begin{itemize}
    \item detecting the existense or absense of the relaitonship, or testing the null hypothesis that the two variables are independent; 
    \item and measuring how strong the relationship is ,or quantifying the strength of any dependence that may exist between the two variables.
\end{itemize}

Correspondingly, performance evaluation should be carried out separately for these two types of tasks, with the performance metrics also being two-fold: its power for detecting dependence and its accuracy for measuring strength. We will first evaluate the accuracy performance since our main purpose is to propose a more accurate measure for nonlinear monotonic relationship.

\subsection{Accuracy Analysis for Measuring Monotonic Strength}
The basic question to be asked when measuring any attribute is how accurate is this measurement, and there should be no exception for dependence measurement. ISO 5725 uses two terms ``trueness" and ``precision" to describe the accuracy of a measurement method. ``Trueness" refers to the closeness of agreement between the arithmetic mean of measured results and the true or accepted reference value. ``Precision" refers to the closeness of agreement between measured values \citep{internationalstandardizationorganizationAccuracyTruenessPrecision1994}.

Dependence statistics are most time used as descriptive measures, and their accuracy performance is somewhat neglected in literature. We don't think correlation statistics should be evaluated in just unsupervised way. Without a benchmark value to compare with, different statistics can only speak for themselves. In order to evaluate the accuracy performance, it is fundamental to obtain the true value, or at least an accepted reference value for the relationship strength. In fact, if the relationship is known in advance, the coefficient of determination, $R-squared$, which is defined as the proportion of variance for one variable explained by the other variable, can serve as an intuitive and natural measure of relationship strength \citep{doksumCorrelationCurvesLocal1994} . We will take its square root, $R$, as the conventional true value of the relationship strength, or the effect size. A simple evidence is that Pearson's $r$, as the golden standard for measuring linearity, is equivalent to $R$, as long as the relationship is linear. Thus, it is acceptable to use $R$ as the reference value.

\begin{definition}[Conventional true value]\label{def6}
    For a noised functional relationship $Y = f\left( X \right) + \varepsilon $, the conventional true value of the relationship strength is defined as the square root of coefficient of determination:
    \begin{equation}
        R = \sqrt {1 - {{{\mathop{\rm var}} \left( \varepsilon  \right)} \mathord{\left/
        {\vphantom {{{\mathop{\rm var}} \left( \varepsilon  \right)} {{\mathop{\rm var}} \left( Y \right)}}} \right.
        \kern-\nulldelimiterspace} {{\mathop{\rm var}} \left( Y \right)}}}
    \end{equation}
\end{definition}

\begin{proposition}\label{pro2}
    For a noised linear function $Y = f\left( X \right) + \varepsilon  = aX + b + \varepsilon $, we have $\left| {r\left( {X,Y} \right)} \right| = R$. Here, $\varepsilon $ is independent noise added to $f\left( X \right)$.
\end{proposition}

Following ISO 5475, we evaluate the accuracy for a certain correlation statistic in the following way:
\begin{itemize}
    \item
          the trueness is expressed in terms of bias and computed as the difference between the reference and the median score of that statistic;
    \item
          precision is expressed in terms of imprecision and computed as the interquartile range of the scores; and
    \item
          the comprehensive performance of trueness and precision, or the total displacement from a reference value, can be represented as the mean absolute error, $MAE$ for short, and calculated as the mean of the absolute values of the difference between the measured value and the actual value. A measure with lower $MAE$ value is better.
\end{itemize}

We investigate the accuracy performance of adjusted $r$, along with the degenerated $r$, Pearson's $r$, Spearman's $\rho$, and two other leading dependence statistics, distance correlation (dCor) and maximal information coefficient (MIC). The simulation is conducted on up to 50 types of different monotonic scenarios, covering all basic elementary functions, several composite functions and typical special functions (see Table \ref{tbl_04} in Appendix \ref{appendix_scen}). Figure \ref{fig_05} shows the scatter plots of conventional true value versus measured values by these six statistics in different scenarios. The slashed diagonal line denotes an ideal statistic, implying that the measured value is always the same as the conventional true value. Apparently, the closeness to this benchmark line determines the performance of a statistic for a certain scenario. The median value of each statistic among scenarios at each $R$ level are also calculated and denoted by the non-transparent red line.

We first look at the extreme values on both sides. It is expected that a statistic will score zero when $X$ and $Y$ are independent, i.e., $R = 0$, and score unity when there is perfect monotonic dependence, i.e., $R = 1$. Only Spearman's $\rho$ and the adjusted $r$ meet these two basic requirements. MIC also scores $ + 1$ when  $R = 1$, however, it tends to overestimate the strength when $R$ is near zero, as also reported in other literature before \citep{chatterjeeNewCoefficientCorrelation2021}. The remaining statistics, ${r^ - }$, $r$, and even the dCor, will underestimate the strength if the relationship is nonlinear, and never converge to $ + 1$ even when $R$ approaches to 1.

Then we take a close look at the intermediate values. It can be seen in Figure \ref{fig_05} that the non-transparent red line of the adjusted $r$ is the closest one to the slashed line, which means the measured values by ${r^\# }$ possess the minimum error. To further quantify the accuracy, we add four boxplots at four representative $R$ levels (approximately, 0.25, 0.50, 0.75, and 1.00) for each statistic. We calculate the difference between the median value and true value to stand for trueness, and the interquartile range for precision. The results are listed in Table \ref{tbl_01}. The adjusted $r$ has the highest trueness in all these representative levels. As for precision, the adjusted $r$ also outperforms all other statistics except MIC. Although MIC possesses the best precision, it suffers from lower trueness. In other words, MIC is a biased statistic, tending to overestimate the strength when the strength is weak, or very noised; and underestimate it when the signal is strong, as shown in Figure \ref{fig_05}.

\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.75\textwidth]{fig/figure_05.png}
    \caption{Scatter plots of conventional true value versus measured value by six correlation statistics in 50 monotonic scenarios. The six investigated statistics, i.e., ${r^ - },\;r,\;{r^\# },\;\rho $, dCor and MIC, are located in six panels respectively. In each panel, there are totally 50 transparent green lines, representing the measured value (Y-axis) with respect to conventional true value (X-axis) for 50 scenarios. The scenarios are arranged in ascending order according to their transparency in the legend. Lines with higher transparency level are farther away from the benchmark line. The scenario prefixes I- and H- refer to Inverse and Hyperbolic respectively. For example, I-H-Tangent stands for Inverse hyperbolic tangent function. Details of the functions are depicted in  Table \ref{tbl_04}.}
    \label{fig_05}
\end{figure}


\begin{table}[htbp]
  \centering
    \begin{tabular}{m{0.2\textwidth}<{\centering}m{0.1\textwidth}<{\centering}m{0.1\textwidth}<{\centering}m{0.1\textwidth}<{\centering}m{0.1\textwidth}<{\centering}m{0.1\textwidth}<{\centering}m{0.1\textwidth}<{\centering}}
    \toprule
    \multirow{2}{*}{Reference Value} & \multicolumn{6}{c}{Measured Value} \\
\cmidrule{2-7}  &  ${r^ - }$ &  $r$  &  ${r^\# }$  &  $\rho$  & dCor  & MIC \\
    \midrule
    \multirow{2}{*}{0.2436 } & 0.0581  & 0.2084  & \textbf{0.2134 } & 0.2022  & 0.2012  & 0.1867  \\
          & (0.1073)  & (0.1411)  & (0.1411)  & (0.1469)  & (0.1267)  & \textbf{(0.0264) } \\
    \midrule
    \multirow{2}{*}{0.4902 } & 0.2101  & 0.4438  & \textbf{0.4540 } & 0.4370  & 0.4203  & 0.2726  \\
          & (0.3542)  & (0.3002)  & (0.2935)  & (0.3297)  & (0.2956)  & \textbf{(0.1096)} \\
    \midrule
    \multirow{2}{*}{0.7465 } & 0.4369  & 0.6907  & \textbf{0.7116 } & 0.6875  & 0.6766  & 0.4681  \\
          & (0.6607)  & (0.4709)  & (0.4073)  & (0.5132)  & (0.4718)  & \textbf{(0.3163)} \\
    \midrule
    \multirow{2}{*}{1.0000 } & 0.6883  & 0.9185  & \textbf{1.0000 } & 0.9999  & 0.9729  & 1.0000  \\
          & (0.9371)  & (0.6382)  & (0.0005)  & (0.0101)  & (0.3290)  & \textbf{(0.0000)} \\
    \bottomrule
    \end{tabular}%
    \caption{Median measured values by six correlation statistics in 50 monotonic scenarios at four differenct reference levels i.e., 0.2436, 0.4902, 0.7465, 1.0000, approximately equal to 0.25, 0.50, 0.75 and 1.00. Values in parentheses represent the corresponding interquartile range.}
  \label{tbl_01}%
\end{table}%


We can also explain precision with the equitability perspective: a measure of dependence is said to be equitable if it assigns similar scores to equally noisy relationships of different types \citep{reshefDetectingNovelAssociations2011}. In sense that the relationship strength $R$ is the converse of noise level, the interquartile range of the measured values at certain $R$ level can also be utilized to quantify the equitability for different statistics. As shown in Table \ref*{tbl_01}, the interquartile value of MIC is minimal among all the statistics, which leads to the same conclusion \textit{MIC is the most equitable statistic} as that in other literatures such as \citep{reshefMeasuringDependencePowerfully2016}. The adjusted is the second most equitable statistic, only inferior to MIC. But we still insist that only equitability is not enough to evaluate the performance of a statistic, since it reflects only the precision, but omits the trueness.


The overall performance in terms of MAE is ordered as ${r^\# }\left( {0.062} \right) \succ \rho \left( {0.105} \right)$ $ \succ dCor\left( {0.130} \right) \succ r\left( {0.154} \right) \succ MIC\left( {0.158} \right) \succ {r^ - }\left( {0.271} \right)$. The adjusted $r$ possesses significant accuracy advantage over other statistics. If we are looking for a statistic that can measure the strength of both linear and nonlinear monotonic relationship accurately, the adjusted $r$ might be the right choice.

We also investigate two typical sets of functions $Y = {X^k} + \varepsilon $  and $Y = {e^{kX}} + \varepsilon $, with different parameters $k$, and the results are shown in Figure \ref{fig_06}.  Almost all the statistics suffer from the increase of the nonlinear order $k$ and the statistics drop dramatically if the nonlinear order rises sharply. However, ${r^\# }$ is the least affected one, and still get the minimum mean absolute error. And the performance ranking for all six statistics remains unchanged.

In addition to simulated scenarios, we also investigate the performance of these statistics on real life scenarios provided as statistical reference datasets by NIST \citep{nationalinstituteofstandardsandtechnologyNISTStandardReference2003}. Among all the six monotonic scenarios, i.e., \textit{Filip, Chwirut1, Hahn1, Rat43, Roszman1}, and \textit{Thurber},  the adjusted $r$ provides the most accurate values (shown in Figure \ref{fig_07}), as compared to the verified values by NIST. Detailed measurement results are listed in Table \ref{tbl_02}.

It is to be noted that above mentioned scenarios are limited to monotonic ones. To test the performance on typical nonmonotonic relationships, we also explored 16 nonmonotonic scenarios depicted in previous literatures \citep{reshefDetectingNovelAssociations2011} and \citep{simonCommentDetectingNovel2014}. As expected, the MAE value of ${r^\# }$ is as great as 0.419, seriously inferior to those of MIC (0.097) and dCor (0.364). However, the performance is also significantly superior to those of $\rho$ (0.430) and $r$ (0.437). As for accuracy performance, ${r^\# }$ outperforms $r$  and $\rho$ in not only monotonic, but also nonmonotonic scenarios.



\begin{landscape}
   
\begin{table}[htbp]
    \centering
    \caption{Comparison of true value and measured value by six correlation statistics in real life scenarios verified by NIST. Measurement errors are listed below their measured values. Raw data and more detailed scenario descriptions are available at https://www.itl.nist.gov/div898/strd/.}
    \renewcommand{\arraystretch}{1.85}
      \begin{tabular}{m{0.08\textwidth}<{\centering}m{0.2\textwidth}<{\centering}m{0.3\textwidth}<{\centering}m{0.1\textwidth}<{\centering}m{0.08\textwidth}<{\centering}m{0.08\textwidth}<{\centering}m{0.08\textwidth}<{\centering}m{0.08\textwidth}<{\centering}m{0.08\textwidth}<{\centering}m{0.08\textwidth}<{\centering}}
      \toprule
      \multirow{2}{*}{Scenario} & \multirow{2}{*}{Pattern} & \multirow{2}{*}{Description} & \multirow{2}{*}{\parbox{0.08\textwidth}{\centering Reference value}} & \multicolumn{6}{c}{Measured Value} \\
      \cmidrule{5-10} &  &  &  & ${r^ - }$ & $r$ & ${r^\# }$ & $\rho$ & dCor & MIC \\    
      \midrule
      \multirow{2}{*}{Filip} & \multirow{2}{*}{\includegraphics[height=.123\textwidth,width=0.2\textwidth]{fig/figure_Filip.png}} & \multirow{2}{*}{\parbox{0.3\textwidth}{\centering Not available}} & \multirow{2}{*}{0.9984 } & 0.0681  & 0.9356  & 0.9979  & 0.9943  & 0.9375  & 1.0000  \\
            &       &       &       & -0.9303  & -0.0627  & -0.0004  & -0.0041 & -0.0608  & -0.0016  \\
      \midrule
      \multirow{2}{*}{Chwirut1} & \multirow{2}{*}{\includegraphics[height=.123\textwidth,width=0.2\textwidth]{fig/figure_Chwirut1.png}} & \multirow{2}{*}{\parbox{0.3\textwidth}{\centering Ultrasonic response, versus metal distance.}} & \multirow{2}{*}{0.9900 } & 0.1165  & 0.8423  & 0.9915  & 0.9845  & 0.9003  & 0.9861  \\
            &       &       &       & -0.8735  & -0.1477  & -0.0015  & -0.0055 & -0.0896  & -0.0038  \\
      \midrule
      \multirow{2}{*}{Hahn1} & \multirow{2}{*}{\includegraphics[height=.123\textwidth,width=0.2\textwidth]{fig/figure_Hahn1.png}} & \multirow{2}{*}{\parbox{0.3\textwidth}{\centering Coefficient of thermal expansion, versus temperature}} & \multirow{2}{*}{0.9999 } & 0.0421 & 0.8303  & 0.9999 & 0.9993 & 0.8859 & 1.0000  \\
            &       &       &       & -0.9578  & -0.1696  & -0.0000 & -0.0006 & -0.1140  & -0.0001  \\
      \midrule
      \multirow{2}{*}{Rat43} & \multirow{2}{*}{\includegraphics[height=.123\textwidth,width=0.2\textwidth]{fig/figure_Rat43.png}} & \multirow{2}{*}{\parbox{0.3\textwidth}{\centering Dry weight of onion bulbs and tops, versus growing time}} & \multirow{2}{*}{0.9872 } & 0.0307  & 0.9514  & 0.9825 & 0.9357 & 0.9654  & 0.9968  \\
            &       &       &       & -0.9565  & -0.0358  & -0.0047 & -0.0515 & -0.0218  & -0.0096  \\
      \midrule
      \multirow{2}{*}{Roszman1} & \multirow{2}{*}{\includegraphics[height=.123\textwidth,width=0.2\textwidth]{fig/figure_Roszman1.png}} & \multirow{2}{*}{\parbox{0.3\textwidth}{\centering Number of quantum defects, versus the excited energy state}} & \multirow{2}{*}{0.9992 } & 0.0002 & 0.9341 & 1.0000 & 0.9800  & 0.9616  & 0.8887  \\
            &       &       &       & -0.9990  & -0.0651  & -0.0008 & -0.0192 & -0.0376  & -0.1106  \\
      \midrule
      \multirow{2}{*}{Thurber} & \multirow{2}{*}{\includegraphics[height=.123\textwidth,width=0.2\textwidth]{fig/figure_Thurber.png}} & \multirow{2}{*}{\parbox{0.3\textwidth}{\centering Semiconductor electron mobility, versus density}} & \multirow{2}{*}{0.9998 } & 0.0055  & 0.9597  & 0.9996 & 0.9972 & 0.9751  & 0.9995  \\
            &       &       &       & -0.9943  & -0.0401  & -0.0002 & -0.0026 & -0.0247  & -0.0003  \\
      \bottomrule
      \end{tabular}%
    \label{tbl_02}%
\end{table}%
    
    
\end{landscape}



\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.75\textwidth]{fig/figure_06.png}
  \caption{Scatter plots of conventional true value versus measured value by six correlation statistics in two types of noisy monotonic relationships, $Y = {X^k} + \varepsilon $ and $Y = {e^{kX}} + \varepsilon $, with $X \sim U\left( {0,1} \right)$ and the nonlinear order k varying from 1 to 10.}
  \label{fig_06}
\end{figure}

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.75\textwidth]{fig/figure_07.png}
  \caption{Measurement error of the six correlation statistics in several real-life scenarios. The conventional true values are derived from the verified results by NIST. For the sake of clarity, square-root transformation is carried out on Y-axis.}
  \label{fig_07}
\end{figure}


\subsection{Power Analysis of Testing Monotonic Dependence}
We carried out a numerical simulation to compare the empirical power performance of the adjusted $r$ versus other dependence statistics. The empirical power of a dependence statistic is defined as its ability to discern between dependent and independent samples that share equal marginal forms. We follow the procedure depicted in \citep{simonCommentDetectingNovel2014}, which has been widely adopted in recent works such as \citep{jiangRobustDependenceMeasure2018} and \citep{panBallCovarianceGeneric2020}.

We compared the power distribution in nearly 70 monotonic scenarios (see Table \ref{tbl_05} in Appendix \ref{appendix_scen}). The simulation details are described in Appendix \ref{appendix_sim_power}, and the simulation results are shown in Figure \ref{fig_08}. The power values of ${r^\# }$  concentrate mainly on high values. The median power value of ${r^\# }$ is 0.765, followed by $r$ (0.763), dCor (0.574), ${r^ - }$ (0.431), $\rho$ (0.239) and MIC (0.181). The power performance ranking is distinct from the accuracy ranking in the last section. It is notable that:
\begin{itemize}
  \item
        Pearson's $r$ follows very closely behind ${r^\# }$, and there is no significant difference between their power distributions;
  \item
        although $r$ is a statistic for linearity and ${r^ - }$ a statistic for identity, both of them rank higher for testing arbitrary monotonic dependence and even outperform Spearman's $\rho$ which is designed for monotonic relationships.
\end{itemize}

One should not be surprised by the above results. In fact, both $r$ and ${r^ - }$ are scaled versions of covariance, which has the potential to express the covarying status or the monotonicity thoroughly. Their only defect is underestimation. However, underestimation will not affect the statistical power, unless the underestimation is too extreme and the resulting score is so trivial to be distinguished from the near-zero scores between two independent variables. It provides a theoretical explanation why lower Pearson's $r$ score and higher statistical power are simultaneously observed so often for nonlinear monotonic dependence.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.75\textwidth]{fig/figure_08.png}
  \caption{Power comparison among six correlation statistics. For a thorough illustration of the power distribution, the plot for each statistic in different panel is comprised of three subplots, a density plot depicting the estimated probability density (upper), a half boxplot to locate the quantiles (middle), and a scatter plot to reveal the positions of the raw values (lower). }
  \label{fig_08}
\end{figure}

In fact, Pearson's $r$ is powerful enough to detect almost arbitrary monotonic relationships, not limited to the linear ones. Only when the monotonic nonlinearity is extremely strong, the difference will be obvious. For a further illustration, we compared six types of scenarios, i.e., $Y = {X^k}$, $Y = {e^{kX}}$, $Y = Einstei{n_1}\left( {kX} \right)$,  $Y = E\left( {kX} \right)$,  $Y = \eta \left( {kX} \right)$ and  $Y = erf\left( {kX} \right)$, in order to compare the power performance in extremely nonlinear monotonicity. For all these scenarios, the nonlinearity becomes stronger with the parameter $k$ being greater. As shown in Figure \ref{fig_09}, with small $k$, the three power lines of  ${r^ - }$, $r$, and ${r^\# }$ almost coincide with each other. However, when parameter $k$ is set as extreme value, these three lines separate from each other, and ${r^\# }$ supersedes Pearson's $r$ and  ${r^ - }$ clearly.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.75\textwidth]{fig/figure_09.png}
  \caption{Power Comparison among ${r^ - }$, $r$ and ${r^\# }$ in slightly and extremely nonlinear monotonic scenarios. X-axis is the Signal-to-Noise Ratio in decibels, representing different noise levels. See Appendix \ref{appendix_sim_power} for more details about Signal-to-Noise Ratio.  }
  \label{fig_09}
\end{figure}

It is well known that there is no single statistics dominates in power in all cases. ${r^\# }$ is designed to detect and measure monotonic dependence, not for general dependence. It might also fail for nonmonotonic scenarios. We investigated its power performance in the aforementioned 16 nonmonotonic scenarios, in which the mean power values of ${r^\# }$ is only 0.558, which is significantly lower than those of MIC (0.954) and dCor (0.844). However, it is also competitive as compared to Pearson's $r$ (0.558) and Spearman's $\rho$ (0.513). In a word, whether the relationship is monotonic or not, ${r^\# }$ achieve similar or higher power than Pearson's $r$ and Spearman's $\rho$.

\section{Discussion}
The adjusted $r$ has intuitively simple interpretation and solid mathematical foundation.The adjusted $r$ is simple and straightforward. It is more intuitive even than Pearson's $r$ itself. Meanwhile, the adjusted $r$ has solid theoretical foundation. The mathematical foundation of the adjusted $r$ is the proposed covariance inequality, which is tighter than the famous Cauchy-Schwarz Inequality. The new inequality leads to the breakthrough. 

Tighter bound implies wider capture range. The new inequality reveals supremum of covariance, and we are able to make most of the covariance. Both advantages and limitations are inherited from covariance itself. On the one hand, the adjusted $r$ is more accurate in measuring nolinear monotonic dependence and more powerful in testing nonlinear monotonic dependence. On the other hand, the limitation of covariance is monotonic dependence. As a resut, the ajusted $r$ might also fail to capture nonmonotonic dependence. 

During the exploration of the adjusted $r$, several misunderstandings about covariance and Pearson's $r$ are corrected. The relevant content about covariance, correlation, and Pearson's $r$ in textbooks might need to be rewritten. We gain several new insights, including but not limited to:
\begin{itemize}
  \item
        Covariance is NOT just a measure of linear dependence. It can test and measure nonlinear monotonic relationship if scaled properly;
  \item
       Zero-correlation means that there is neither linear dependence, NOR nonlinear monotonic dependence;
  \item
        Pearson's $r$ can measure linear dependence to full extent, and NONLINEAR monotonic dependence to some extent;
  \item
        The strength of linear relationship can be quantified exactly by Pearson's $r$, but the  magnitude of Pearson's $r$ does NOT necessarily express the extent of linearity;
  \item
        Pearson's $r$ is statistically powerful enough to test not only linear but also NONLINEAR monotonic relationship, except those extremely nonlinear ones;
  \item
        Pearson's $r$ can be adjusted to measure NONLINEAR monotonic relationships accurately, in term of ${r^\# }$ proposed here. In other words, Pearson's $r$ is a good measure not only for linear relationship, but also for NONLINEAR monotonic relationship, if adjusted.
\end{itemize}

In comparison with traditional Pearson's $r$, the adjustment seems trivial, nothing more than replacing the conservative bound with a sharp one. In spite of its simple form, the adjustment leads to a genuinely new paradigm. We no longer scale \textit{covariance} with \textit{variance}; rather, we scale \textit{actual covariance} with \textit{perfect covariance}. We find that the new paradigm is also applicable to several more recently proposed dependence statistics, such as distance correlation \citep{szekelyEnergyData2017} , and Ball correlation \citep{panBallCovarianceGeneric2020} , which also follow the paradigm ${{covariance} \mathord{\left/
  {\vphantom {{covariance} {variacne}}} \right.
  \kern-\nulldelimiterspace} {variacne}}$. Preliminary empirical results show that if adjusted as ${{actual\;covariance} \mathord{\left/
  {\vphantom {{actual\;covariance} {perfect\;covariacne}}} \right.
  \kern-\nulldelimiterspace} {perfect\;covariacne}}$, they can also gain appealing properties such as being more accurate (see Figure \ref{fig_11} in Appendix \ref{appendix_sharp_dcor_MIC}). The new paradigm is expected to improve these more recently proposed statistics as well.




\newpage

