\section{Introduction} 
The transmission of cellular information plays an important role in various biological functions. In this process, environmental information is encoded to extracellular molecules, which generate a cellular response (Fig. \ref{fig-Schematic diagram of cellular information transmission}(a)). However, in many cases, extracellular molecules exhibit considerable fluctuations (extrinsic noise). For example, in developmental processes, cells detect their positional information through morphogen molecules. However, the concentration of morphogen molecules fluctuates because of the stochasticity of the diffusions \cite{Houchmandzadeh2002}. Another example is chemotaxis, in which the reaction between extracellular molecules and receptors largely fluctuates because of the stochasticity of the reaction \cite{Ueda2007,Miyanaga2007}. Regardless of such noisy conditions, cells often respond to their environments robustly, and this response mechanism has attracted considerable attention \cite{Libby2007,Andrews2007,Tkacik2008a,Tkacik2008b,Morishita2008,Tostevin2009,Kobayashi2010,Mugler2013,Hasegawa2018,Monti2018}. 


Intracellular reactions also exhibit significant fluctuations because of the stochasticity of the reactions \cite{Elowitz2002,Bar-Even2006,Newman2006,Taniguchi2010}. The fluctuations of intracellular reactions (intrinsic noise) generally reduce the information transmitted from the environment to the cellular response \cite{Gregor2007,Cheong2011,Uda2013,Selimkhanov2014}, and so intrinsic noise is thought to be disadvantageous for cellular information transmission. However, are there no advantageous effects of intrinsic noise?


Recent studies simulating biochemical reactions have demonstrated that the transmitted information barely decreases with respect to extrinsic noise when the intrinsic noise is large \cite{Koumura2014,Fujii2017,Tottori2019}. In other words, robustness against the extrinsic noise increases with the level of the intrinsic noise. It has therefore been suggested that robustness against extrinsic noise is one of the advantageous characteristics of intrinsic noise \cite{Koumura2014,Fujii2017,Tottori2019}. In this paper, we call this phenomenon intrinsic noise-induced robustness (INIR).

Previous studies on this mechanism have focused on specific biochemical reactions \cite{Koumura2014,Fujii2017,Tottori2019}; hence, their models were too complicated to clarify the relation between INIR and the input--output of a system. Moreover, the mechanism of INIR remains elusive. In this paper, we address these issues by analyzing simple models. 

This paper is organized as follows. In Sec. \ref{section-Linear model}, we analyze a model in which the input--output relation is linear. This model is an extension of the Gaussian channel, which is a standard model in information theory \cite{Cover}. We show that the decrease in transmitted information with respect to extrinsic noise is smaller when the intrinsic noise is larger. Therefore, even this simple model reproduces the INIR phenomenon. Moreover, we demonstrate that the intrinsic noise dependency of the robustness against the extrinsic noise becomes stronger when the variance of the intrinsic noise is larger than that of the input distribution. 

In Sec. \ref{section-Threshold model}, we analyze a threshold model in which the output depends on whether the input exceeds the threshold. When the threshold is equal to the mean of the input, INIR is realized in the threshold model (Sec. \ref{subsection-theta0}). In contrast, when the threshold is much larger than the mean of the input, the threshold model exhibits stochastic resonance (SR) (Sec. \ref{subsection-SR}). Under the SR phenomenon, noise acts to increase the level of transmitted information \cite{Gammaitoni1998}, and it is experimentally verified that SR appears in biological systems such as neurons \cite{Douglass1993,Levin1996,Russell1999,Gluckman1996,Stacey2000,Stacey2001,Srebro1999,Mori2002,Kitajo2007,Stufflebeam2000,Tanaka2008,Ward2010}. Interestingly, when the threshold model exhibits SR, INIR is not always realized. However, the threshold model with SR can increase both robustness against extrinsic noise and the transmitted information simultaneously, which is impossible for the linear model and the threshold model without SR. In other words, SR solves the trade-off between robustness against extrinsic noise and the transmitted information.

In Sec. \ref{section-Mechanism}, we explain why INIR is realized in the linear model and in the threshold model without SR, but is not always realized in the threshold model with SR. In Sec. \ref{section-Discussion}, we discuss the biological significance of our results and the relevance of other studies.


\section{Linear model\label{section-Linear model}}

\begin{figure}[btp]
\begin{center}
	\includegraphics[width=75mm]{fig1.pdf}\\[-1pc]
	 \caption{
	Schematic diagram of cellular information transmission.
	(a) 
	Environmental information is encoded to extracellular molecules. Cells detect extracellular molecules and determine proper responses via intracellular reactions.
	(b)
	The environment and the cellular responses correspond to input $X$ and output $Y$, respectively. Furthermore, fluctuations of extracellular molecules and intracellular reactions correspond to extrinsic noise $Z_{ex}$ and intrinsic noise $Z_{in}$, respectively.
	}
	\label{fig-Schematic diagram of cellular information transmission}
\end{center}
\end{figure}

We consider a channel composed of the input $X$, extrinsic noise $Z_{ex}$, intrinsic noise $Z_{in}$, and output $Y$ (Fig. \ref{fig-Schematic diagram of cellular information transmission}). The input $X$ corresponds to environmental information detected by the cells, and the output $Y$ corresponds to the cellular response against the environment. Furthermore, the extrinsic noise $Z_{ex}$ corresponds to the fluctuation of extracellular molecules, and the intrinsic noise $Z_{in}$ corresponds to the fluctuation of intracellular reactions. For example, in developmental processes, $X$ corresponds to the concentration of morphogen molecules, which encodes the positional information, and $Y$ corresponds to the concentration of gene expressions \cite{Tkacik2008a,Tkacik2008b}.  $Z_{ex}$ corresponds to the fluctuation of morphogen molecules because of the stochasticity of diffusions \cite{Houchmandzadeh2002}, and $Z_{in}$ corresponds to the fluctuation of gene expressions caused by the stochasticity of intracellular reactions \cite{Gregor2007,Tkacik2008a,Tkacik2008b}. 

Here, the input $X$, extrinsic noise $Z_{ex}$, and intrinsic noise $Z_{in}$ are assumed to obey Gaussian distributions $N(0,1)$, $N(0,\sigma_{ex}^2)$, and $N(0,\sigma_{in}^2)$, respectively, where $\sigma_{ex}^2$ is the variance of the extrinsic noise, and $\sigma_{in}^2$ is the variance of the intrinsic noise. Furthermore, the output $Y$ is given by the following:

\begin{align}
	Y&=X+Z_{ex}+Z_{in}
	\label{Linear response}
\end{align}
This model is an extension of the Gaussian channel, which is a standard model in information theory \cite{Cover}. In this paper, we call this the ``linear model.''

To quantify the information transmitted from the input $X$ to the output $Y$, we use the mutual information between $X$ and $Y$, $I(X;Y)$ \cite{Cover}. Higher values of $I(X;Y)$ denote that a large amount of information is transmitted from $X$ to $Y$. When $X$ and $Y$ are continuous random variables, $I(X;Y)$ is given by \cite{Cover}:

\begin{align}
	I(X;Y)&=\int_{Y}\int_{X}p(x,y)\log_{2}\frac{p(x,y)}{p(x)p(y)}dxdy
\end{align}
where $p(x,y)$ is the joint probability density function of $X$ and $Y$, and $p(x)$ and $p(y)$ are the marginal probability density functions of $X$ and $Y$, respectively. In the linear model, $I(X;Y)$ can be derived analytically as \cite{Cover}:

\begin{align}
	I(X;Y)&=\frac{1}{2}\log_{2}\left(1+\frac{1}{\sigma_{ex}^2+\sigma_{in}^2}\right)
	\label{LRM-MI}
\end{align}

The mutual information $I(X;Y)$ decreases with the intrinsic noise $\sigma_{in}^2$ when $\sigma_{ex}^2=0$ (Fig. \ref{linear model}(a)). However, the decrease in $I(X;Y)$ with respect to extrinsic noise $\sigma_{ex}^2$ is reduced at higher levels of intrinsic noise $\sigma_{in}^2$ (Fig. \ref{linear model}(b)). In other words, the degree of robustness against extrinsic noise increases with the level of intrinsic noise. Therefore, INIR is realized in the linear model.

\begin{figure}[btp]
\begin{center}
	(a)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig2a.pdf}
	\end{minipage}
	(b)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig2b.pdf}
	\end{minipage}\\
	(c)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig2c.pdf}
	\end{minipage}
	(d)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig2d.pdf}
	\end{minipage}
	 \caption{
	 Linear model. 
	 (a) 
	 Mutual information $I(X;Y)$ against intrinsic noise $\sigma_{in}^2$ at $\sigma_{ex}^2=0$. 
	 (b) 
	 Mutual information $I(X;Y)$ against extrinsic noise $\sigma_{ex}^2$. The mutual information $I(X;Y)$ is normalized at $\sigma_{ex}^2=0$.
	 (c)
	 Definition of the robustness function $\delta_{q}(\sigma_{in}^2)$.
	 (d)
	 Intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$. While $\delta_{q}(\sigma_{in}^2)\propto\sigma_{in}^2$ when $\sigma_{in}^2>1$ (green), $\delta_{q}(\sigma_{in}^2)\propto\sigma_{in}^{2q}$ when $\sigma_{in}^2<1$ (blue). 
	 }
	\label{linear model}
\end{center}
\end{figure}

To investigate the robustness against extrinsic noise in more detail, we define the robustness function $\delta_{q}(\sigma_{in}^2)$ as follows: 


\begin{align}
	\delta_{q}(\sigma_{in}^2):=\left\{\sigma_{ex}^{*2}\left|\frac{I(X;Y)|_{\sigma_{ex}^2=\sigma_{ex}^{*2}}}{I(X;Y)|_{\sigma_{ex}^2=0}}=q,\ \ \ q\in(0,1)\right.\right\}
	\label{the definition of the robustness function}
\end{align}

Therefore, the robustness function $\delta_{q}(\sigma_{in}^2)$ indicates the extrinsic noise $\sigma_{ex}^2$ when the normalized mutual information $I(X;Y)/I(X;Y)|_{\sigma_{ex}^2=0}$ decreases to $q$ for $q\in(0,1)$ (Fig. \ref{linear model}(c)). For example, when $q=0.5$, $\delta_{q}(\sigma_{in}^2)$ indicates the extrinsic noise $\sigma_{ex}^2$ when the mutual information $I(X;Y)$ has decreased by 50\%; when $q=0.99$, the robustness function $\delta_{q}(\sigma_{in}^2)$ indicates the extrinsic noise $\sigma_{ex}^2$ when the mutual information $I(X;Y)$ has decreased by 1\%. Higher values of $\delta_{q}(\sigma_{in}^2)$ indicate higher robustness against extrinsic noise $\sigma_{ex}^2$.

In the linear model, the robustness function $\delta_{q}(\sigma_{in}^2)$ can be derived analytically as:

\begin{align}
	\delta_{q}(\sigma_{in}^2)&=\frac{1}{\left(1+\sigma_{in}^{-2}\right)^{q}-1}-\sigma_{in}^2
	\label{LRM-the robustness function}
\end{align}
(see Appendix \ref{appendix-LRM-robustness-1}). The robustness function $\delta_{q}(\sigma_{in}^2)$ increases with intrinsic noise $\sigma_{in}^2$ for any $q$ (Fig. \ref{linear model}(d)). Therefore, INIR is clearly demonstrated. 

Moreover, the intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$ changes at $\sigma_{in}^2=1$ (Fig. \ref{linear model}(d)). The intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$ in $\sigma_{in}^2\gg1$ is stronger than that in $\sigma_{in}^2\ll1$. Indeed, Eq. (\ref{LRM-the robustness function}) can be approximated as follows:

\begin{align}
	\delta_{q}(\sigma_{in}^2)
	\propto\begin{cases}
		\sigma_{in}^{2q} & (\sigma_{in}^2\ll1)\\
		\sigma_{in}^2 & (\sigma_{in}^2\gg1)
	\end{cases}
	\label{LRM-the robustness function-approx}
\end{align}
(see Appendix \ref{appendix-LRM-robustness-2}). Considering $q\in(0,1)$, Eq. (\ref{LRM-the robustness function-approx}) indicates that the intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$ in $\sigma_{in}^2\gg1$ is stronger than that in $\sigma_{in}^2\ll1$.

As the input $X$ has variance 1, the intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$ becomes stronger when the intrinsic noise $\sigma_{in}^2$ is much larger than the variance of the input $X$. Therefore, the input distribution plays an important role in INIR. 






\section{Threshold model\label{section-Threshold model}}
\subsection{The case $\boldsymbol{\theta=0}$\label{subsection-theta0}}
In Sec. \ref{section-Linear model}, we showed that the linear model realizes INIR. Moreover, we revealed that INIR is stronger when the intrinsic noise is much larger than the variance of the input. 

However, the input--output relation in cellular information transmission is often nonlinear. In particular, some form of threshold response is often observed \cite{Ozbudak2004, Melen2005, Narula2012}. Therefore, in this subsection, we introduce a threshold model and examine whether it reproduces the same results as the linear model. 

The input $X$, extrinsic noise $Z_{ex}$, and intrinsic noise $Z_{in}$ are assumed to obey Gaussian distributions $N(0,1)$, $N(0,\sigma_{ex}^2)$, and $N(0,\sigma_{in}^2)$, respectively, as for the linear model. However, the output $Y$ is given by:

\begin{align}
	Y&=\begin{cases}
		1 & (X+Z_{ex}+Z_{in}\geq\theta)\\
		0 & (X+Z_{ex}+Z_{in}<\theta)
	\end{cases}
	\label{Threshold response}
\end{align}
Therefore, the output $Y$ is 1 when $X+Z_{ex}+Z_{in}$ is greater than the threshold $\theta$, whereas the output $Y$ is 0 when $X+Z_{ex}+Z_{in}$ is less than $\theta$. In this paper, we call this the ``threshold model.'' 

In the threshold model, the input $X$ is a continuous random variable, but the output $Y$ is a discrete random variable. Therefore, the mutual information $I(X;Y)$ in the threshold model is given by \cite{Cover}:

\begin{align}
	I(X;Y)&=\sum_{Y}\int_{X}p(x,y)\log_{2}\frac{p(x,y)}{p(x)p(y)}dx
\end{align}

For simplicity, in this subsection, we discuss the case of $\theta=0$ (Fig. \ref{threshold model}(a)). As input $X$ has a mean of 0, $\theta=0$ corresponds to the case where the mean of the input $X$ is equal to the threshold $\theta$. In Sec. \ref{subsection-SR}, we will discuss the case of $\theta\neq0$.

\begin{figure}[btp]
\begin{center}
	(a)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig3a.pdf}
	\end{minipage}
	(b)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig3b.pdf}
	\end{minipage}\\
	(c)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig3c.pdf}
	\end{minipage}
	(d)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig3d.pdf}
	\end{minipage}
	 \caption{
	 Threshold model ($\theta=0$). 
	 (a) 
	 When $\theta=0$, the threshold $\theta$ is equal to the mean of the input $X$. 
	 (b)--(d)
	 Numerical solutions (dots) and approximate solutions (line).
	 (b) 
	 Mutual information $I(X;Y)$ against the intrinsic noise $\sigma_{in}^2$ when $\sigma_{ex}^2=0$. 
	 (c) 
	 Mutual information $I(X;Y)$ against extrinsic noise $\sigma_{ex}^2$. The mutual information $I(X;Y)$ is normalized at $\sigma_{ex}^2=0$.
	 (d)
	 The intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$. $\delta_{q}(\sigma_{in}^2)\propto\sigma_{in}^2$ when $\sigma_{in}^2>2/(\pi\ln2)$ (green), $\delta_{q}(\sigma_{in}^2)\propto\sigma_{in}$ when $((1-q)^2/(4q^2))(2/(\pi\ln2))<\sigma_{in}^2<2/(\pi\ln2)$ (yellow), and $\delta_{q}(\sigma_{in}^2)\propto{\rm const}$ when $\sigma_{in}^2<\min\{((1-q)^2/(4q^2))(2/(\pi\ln2)),2/(\pi\ln2)\}$ (blue). 
	 }
	\label{threshold model}
\end{center}
\end{figure}

Unlike the linear model, the mutual information $I(X;Y)$ in the threshold model ($\theta=0$) cannot be derived analytically. However, it can be approximated as:

\begin{align}
	I(X;Y)&\approx1-\frac{1}{\sqrt{\frac{2}{(\pi\ln2)}\left(\sigma_{ex}^2+\sigma_{in}^2\right)^{-1}+1}}
	\label{TRM-theta0-MI}
\end{align}
(see Appendix \ref{appendix-TRM-MI-1}). This approximate solution matches the numerical solution closely (Fig. \ref{threshold model}(b), (c)).

The mutual information $I(X;Y)$ in the threshold model ($\theta=0$) decreases with the intrinsic noise $\sigma_{in}^2$ at $\sigma_{ex}^2=0$ (Fig. \ref{threshold model}(b)). However, the decrease in mutual information $I(X;Y)$ with respect to extrinsic noise $\sigma_{ex}^2$ occurs at a slower rate as the intrinsic noise $\sigma_{in}^2$ increases (Fig. \ref{threshold model}(c)). In other words, robustness against the extrinsic noise increases as the intrinsic noise becomes higher. Therefore, INIR is also realized in the threshold model ($\theta=0$). 

To investigate the robustness against extrinsic noise in more detail, we examine the robustness function $\delta_{q}(\sigma_{in}^2)$. As explained in Sec. \ref{section-Linear model}, the robustness function $\delta_{q}(\sigma_{in}^2)$ represents the extrinsic noise $\sigma_{ex}^2$ when the normalized mutual information $I(X;Y)/I(X;Y)|_{\sigma_{ex}^2=0}$ decreases to $q$ for $q\in(0,1)$. Large values of $\delta_{q}(\sigma_{in}^2)$ indicate high robustness against extrinsic noise.

The robustness function $\delta_{q}(\sigma_{in}^2)$ in the threshold model ($\theta=0$) cannot be derived analytically because we do not have an exact expression for $I(X;Y)$. However, $\delta_{q}(\sigma_{in}^2)$ can be derived approximately using the approximate solution of $I(X;Y)$ (Eq. (\ref{TRM-theta0-MI})). Thus:

\begin{align}
	\delta_{q}(\sigma_{in}^2)
	&\approx\frac{\left\{1-qI_{0}\right\}^2}{\left\{2-qI_{0}\right\}I_{0}}\frac{2}{(\pi\ln2)}-\sigma_{in}^2
	\label{TRM-the robustness function}
\end{align}
where 

\begin{align}
	I_{0}=1-\frac{1}{\sqrt{\frac{2}{(\pi\ln2)}\sigma_{in}^{-2}+1}}
	\label{TRM-the robustness function-I0}
\end{align}
(see Appendix \ref{appendix-TRM-robustness-1}). The robustness function $\delta_{q}(\sigma_{in}^2)$ increases with the intrinsic noise $\sigma_{in}^2$ for any $q$ (Fig. \ref{threshold model}(d)). Therefore, INIR is clearly demonstrated. 

Moreover, the intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$ changes at $\sigma_{in}^2=2/(\pi\ln2)$ (Fig. \ref{threshold model}(d)). The intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$ in $\sigma_{in}^2\gg2/(\pi\ln2)$ is stronger than that in $\sigma_{in}^2\ll2/(\pi\ln2)$. Furthermore, when $\sigma_{in}^2$ or $q$ is sufficiently small, the robustness function $\delta_{q}(\sigma_{in}^2)$ does not change with respect to the intrinsic noise $\sigma_{in}^2$. Indeed, Eq. (\ref{TRM-the robustness function}) can be approximated as follows:

\begin{align}
	\delta_{q}(\sigma_{in}^2)
	&\propto\begin{cases}
		\sigma_{in}^{2} & \left(\sigma_{in}^2\gg\frac{2}{\pi\ln2}\right)\\
		\sigma_{in} & \left(\frac{(1-q)^2}{4q^2}\frac{2}{\pi\ln2}\ll\sigma_{in}^2\ll\frac{2}{\pi\ln2}\right)\\
		{\rm const} & \left(\sigma_{in}^2\ll\min\left(\frac{(1-q)^2}{4q^2}\frac{2}{\pi\ln2},\frac{2}{\pi\ln2}\right)\right)
	\end{cases}
	\label{TRM-the robustness function-approx}
\end{align}
(see Appendix \ref{appendix-TRM-robustness-2}). Therefore, Eq. (\ref{TRM-the robustness function-approx}) indicates that the intrinsic noise dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$ in $\sigma_{in}^2\gg2/(\pi\ln2)$ is stronger than that in $\sigma_{in}^2\ll2/(\pi\ln2)$. As the variance of the input $X$ is 1 and $2/(\pi\ln2)$ is close to 1, when the intrinsic noise is much larger than the variance of the input, the intrinsic noise dependency of the robustness against the extrinsic noise is particularly strong in the threshold model.

Furthermore, Eq. (\ref{TRM-the robustness function-approx}) suggests that the robustness function $\delta_{q}(\sigma_{in}^2)$ does not change when $\sigma_{in}^2\ll\min\{((1-q)^2/(4q^2))(2/(\pi\ln2)),2/(\pi\ln2)\}$. In particular, when $q\leq1/3$, the region in which $\delta_{q}(\sigma_{in}^2)\propto\sigma_{in}$ disappears, and $\delta_{q}(\sigma_{in}^2)$ remains constant when $\sigma_{in}^2\ll2/(\pi\ln2)$.



\subsection{The case $\boldsymbol{\theta\neq0}$\label{subsection-SR}}
In Sec. \ref{subsection-theta0}, we discussed the threshold model in the case $\theta=0$. In this subsection, we consider the case $\theta\neq0$ (Fig. \ref{threshold model SR}(a)). The mutual information $I(X;Y)$ is an even function of $\theta$. Therefore, we can consider the case $\theta\geq0$ without loss of generality. 


\begin{figure*}[btp]
\begin{center}
	(a)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig4a.pdf}
	\end{minipage}
	(b)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{50mm}
		\includegraphics[width=50mm,height=35mm]{./fig4b.pdf}
	\end{minipage}
	(c)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig4c.pdf}
	\end{minipage}\\
	(d)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig4d.pdf}
	\end{minipage}
	(e)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig4e.pdf}
	\end{minipage}
	(f)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig4f.pdf}
	\end{minipage}
	(g)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig4g.pdf}
	\end{minipage}
	\caption{
	 Threshold model ($\theta\neq0$). 
	 (a) When $\theta>0$, the threshold $\theta$ is larger than the mean of the input $X$.
	 (b), (c) Mutual information $I(X;Y)$ with respect to intrinsic noise $\sigma_{in}^2$ when $\sigma_{ex}^2=0$. (b) Numerical solutions. (c) Approximate solutions (lines). Numerical solutions (dots).
	 (d), (e) $\sigma_{in}^2$-dependency of the robustness function $\delta_{q}(\sigma_{in}^2)$.
	 (f), (g) $\sigma_{in}^2$, $\theta$-dependency of $\partial\delta_{q}/\partial\sigma_{in}^2$. 
 	 (d)--(g) $\partial I(X;Y)/\partial \sigma_{in}^2>0$ (yellow), $\partial I(X;Y)/\partial \sigma_{in}^2<0$ and $\partial^2 I(X;Y)/(\partial \sigma_{in}^2)^2<0$ (green), $\partial I(X;Y)/\partial \sigma_{in}^2<0$ and $\partial^2 I(X;Y)/(\partial \sigma_{in}^2)^2>0$ (blue).
	 }
	\label{threshold model SR}
\end{center}
\end{figure*}


The mutual information $I(X;Y)$ in the threshold model cannot be derived analytically, but the following approximate expression can be obtained:

\begin{align}
	I(X;Y)\approx&\left(1-\frac{1}{\sqrt{\frac{2}{(\pi\ln2)}\left(\sigma_{ex}^2+\sigma_{in}^2\right)^{-1}+1}}\right)\nonumber\\
	&\times\exp\left(-\frac{\theta^2}{(\pi\ln2)\left(\frac{2}{(\pi\ln2)}+\sigma_{ex}^2+\sigma_{in}^{2}\right)}\right)
	\label{TRM-SR-MI}
\end{align}
(see Appendix \ref{appendix-TRM-SR-MI-1}). This approximate solution matches the numerical solution well (Fig. \ref{threshold model SR}(c)).

When $\theta=0$, that is, the threshold $\theta$ is equal to the mean of the input $X$, the mutual information $I(X;Y)$ decreases monotonically with respect to the intrinsic noise $\sigma_{in}^2$ (Fig. \ref{threshold model SR}(c)(red)). When $\theta$ is much larger than 0, that is, the threshold $\theta$ is much larger than the mean of the input $X$, the mutual information $I(X;Y)$ is maximized with a moderate value of the intrinsic noise $\sigma_{in}^2$ (Fig. \ref{threshold model SR}(c)). This phenomenon is called ``stochastic resonance'' (SR) \cite{Stocks2000, Stocks2001}.

Why does SR appear in the threshold model for large $\theta$? This is the same as asking why the mutual information $I(X;Y)$ is maximized with a moderate intrinsic noise $\sigma_{in}^2$ for large $\theta$. This mechanism can be explained as follows. When $\theta$ is large, that is, the threshold $\theta$ is much larger than the mean of the input $X$, the input $X$ barely exceeds the threshold $\theta$. Therefore, little of the input information is transmitted to the output in the absence of intrinsic noise $\sigma_{in}^2$. However, when the intrinsic noise $\sigma_{in}^2$ is added to the input $X$, the width of the input distribution increases, and the number of times the input $X$ exceeds the threshold $\theta$ increases. Therefore, the transmitted input information increases. Nonetheless, when the intrinsic noise $\sigma_{in}^2$ is too large, the input information is hidden by the large noise, and in this case, the transmitted input information decreases. Through this mechanism, the mutual information $I(X;Y)$ can be maximized with a moderate intrinsic noise $\sigma_{in}^2$ at large $\theta$.

Stocks has already reported that SR appears in the same threshold model \cite{Stocks2000, Stocks2001}. However, the condition under which SR appears has only been illustrated through numerical calculations. We can derive the approximate condition under which SR appears: $\theta>2$ (see Appendix \ref{appendix-TRM-SR-MI-2}). As the length of $X$ is scaled by its variance, SR appears when the threshold $\theta$ is more than twice the variance of the input $X$, which is consistent with the numerical results from a previous study \cite{Munakata2005}. 

To investigate the robustness of the threshold model against extrinsic noise at $\theta>0$, we examine the robustness function $\delta_{q}(\sigma_{in}^2)$ in the threshold model at $\theta=1$ (Fig. \ref{threshold model SR}(d)) and at $\theta=3$ (Fig. \ref{threshold model SR}(e)). As mentioned in the previous sections, when the robustness function $\delta_{q}(\sigma_{in}^2)$ is large, the robustness against extrinsic noise $\sigma_{ex}^2$ is high. When $\theta>0$, we cannot derive the robustness function $\delta_{q}(\sigma_{in}^2)$ analytically. Therefore, we derive $\delta_{q}(\sigma_{in}^2)$ numerically (Fig. \ref{threshold model SR}(d), (e)). 

When SR does not appear ($\theta=1$), the robustness function $\delta_{q}(\sigma_{in}^2)$ increases monotonically with the intrinsic noise $\sigma_{in}^2$ for any $q$ (Fig. \ref{threshold model SR}(d)). However, when SR appears ($\theta=3$), the robustness function $\delta_{q}(\sigma_{in}^2)$ does not always increase with the intrinsic noise $\sigma_{in}^2$ (Fig. \ref{threshold model SR}(e)). Therefore, INIR is not always realized in the presence of SR. 

We further examine the region where INIR is not realized. Defining $\sigma_{in,\max}^2$ as the intrinsic noise $\sigma_{in}^2$ that maximizes the mutual information $I(X;Y)$, the robustness function $\delta_{q}(\sigma_{in}^2)$ decreases with the intrinsic noise when $\sigma_{in}^2<\sigma_{in,\max}^2$ (Fig. \ref{threshold model SR}(e)(yellow)). In other words, the robustness against extrinsic noise $\sigma_{ex}^2$ decreases with respect to the intrinsic noise $\sigma_{in}^2$ when the mutual information $I(X;Y)$ increases with respect to $\sigma_{in}^2$. This is trivial, because the mutual information $I(X;Y)$ first increases and then decreases with extrinsic noise $\sigma_{ex}^2$ when $\sigma_{in}^2<\sigma_{in,\max}^2$. Therefore, when the intrinsic noise $\sigma_{in}^2$ decreases in $\sigma_{in}^2<\sigma_{in,\max}^2$, the range of the extrinsic noise $\sigma_{ex}^2$ for which the mutual information $I(X;Y)$ increases will expand. Thus, the robustness against extrinsic noise $\sigma_{ex}^2$ increases as the intrinsic noise $\sigma_{in}^2$ decreases when $\sigma_{in}^2<\sigma_{in,\max}^2$.

Interestingly, even when $\sigma_{in}^2>\sigma_{in,\max}^2$, there is a region where the robustness function $\delta_{q}(\sigma_{in}^2)$ decreases with the intrinsic noise $\sigma_{in}^2$ (Fig. \ref{threshold model SR}(e)). In other words, the robustness against the extrinsic noise $\sigma_{ex}^2$ decreases with respect to the intrinsic noise $\sigma_{in}^2$ even when the mutual information $I(X;Y)$ decreases with respect to $\sigma_{in}^2$. Defining $\sigma_{in,c}^2$ as the intrinsic noise $\sigma_{in}^2$ at which the mutual information $I(X;Y)$ changes from convex upward to convex downward, the robustness function $\delta_{q}(\sigma_{in}^2)$ decreases with respect to the intrinsic noise $\sigma_{in}^2$ when $\sigma_{in,\max}^2<\sigma_{in}^2<\sigma_{in,c}^2$ and $q$ is close to 1 (Fig. \ref{threshold model SR}(e)(green)). In other words, the robustness against the extrinsic noise $\sigma_{ex}^2$ decreases as the intrinsic noise $\sigma_{in}^2$ increases when the mutual information $I(X;Y)$ is decreasing and convex upward with respect to $\sigma_{in}^2$.

We have approximately demonstrated that the robustness function $\delta_{q}(\sigma_{in}^2)$ decreases with respect to the intrinsic noise $\sigma_{in}^2$ when the mutual information $I(X;Y)$ is decreasing and convex upward against the intrinsic noise $\sigma_{in}^2$, and $q$ is close to 1.

In the threshold model, the $\sigma_{in}^2$-dependency of $I(X;Y)$ is the same as the $\sigma_{ex}^2$-dependency of $I(X;Y)$, and $I(X;Y)$ is decreasing against $\sigma_{in}^2$. Therefore, 

\begin{align}
	\frac{\partial I(X;Y)}{\partial \sigma_{ex}^2}=\frac{\partial I(X;Y)}{\partial \sigma_{in}^2}<0
\end{align}
is satisfied. When $\partial I(X;Y)/\partial \sigma_{ex}^2<0$, the robustness function $\delta_{q}(\sigma_{in}^2)$ becomes close to 0 as $q\to1$. Therefore, when $\partial I(X;Y)/\partial \sigma_{ex}^2<0$ and $q\sim1$,

\begin{align}
	&\left.I(X;Y)\right|_{\sigma_{ex}^2=\delta_{q}}\nonumber\\
	&=\left.I(X;Y)\right|_{\sigma_{ex}^2=0}+\left.\frac{\partial I(X;Y)}{\partial\sigma_{ex}^2}\right|_{\sigma_{ex}^2=0}\delta_{q}+O\left(\delta_{q}^2\right)
	\label{Taylor expansion of MI}
\end{align}
is satisfied. Substituting Eq. (\ref{Taylor expansion of MI}) into Eq. (\ref{the definition of the robustness function}), we obtain

\begin{align}
	\delta_{q}(\sigma_{in}^2)&=-\frac{(1-q)}{\left.\frac{\partial\ln I(X;Y)}{\partial\sigma_{ex}^2}\right|_{\sigma_{ex}^2=0}}
	\label{the robustness function at q1}
\end{align}
$\partial\ln I(X;Y)/\partial\sigma_{ex}^2|_{\sigma_{ex}^2=0}$ corresponds to the slope of the normalized mutual information $I(X;Y)/I(X;Y)|_{\sigma_{ex}^2=0}$ against the extrinsic noise $\sigma_{ex}^2$ at $\sigma_{ex}^2=0$. When $\partial\ln I(X;Y)/\partial\sigma_{ex}^2|_{\sigma_{ex}^2=0}$ is large, the robustness function $\delta_{q}(\sigma_{in}^2)$ at $q\sim1$ is also large. 

Differentiating Eq. (\ref{the robustness function at q1}) with respect to the intrinsic noise $\sigma_{in}^2$, we have

\begin{align}
	\frac{\partial \delta_{q}(\sigma_{in}^2)}{\partial \sigma_{in}^2}
	&=\frac{(1-q)\frac{\partial}{\partial\sigma_{in}^2}\left(\left.\frac{\partial \ln I(X;Y)}{\partial\sigma_{ex}^2}\right|_{\sigma_{ex}^2=0}\right)}{\left(\left.\frac{\partial \ln I(X;Y)}{\partial\sigma_{ex}^2} \right|_{\sigma_{ex}^2=0}\right)^2}
	\label{the robustness function differentiated by the intrinsic noise}
\end{align}
In the threshold model, $I(X;Y)$ depends on the summation of $\sigma_{in}^2$ and $\sigma_{ex}^2$. Therefore, 

\begin{align}
	\frac{\partial}{\partial\sigma_{in}^2}\left(\left.\frac{\partial \ln I(X;Y)}{\partial\sigma_{ex}^2}\right|_{\sigma_{ex}^2=0}\right)
	=\left.\frac{\partial^2 \ln I(X;Y)}{\left(\partial\sigma_{in}^2\right)^2}\right|_{\sigma_{ex}^2=0}
	\label{the mutual information differentiated by the intrinsic noise twice}
\end{align}
is satisfied. From Eqs. (\ref{the robustness function differentiated by the intrinsic noise}) and (\ref{the mutual information differentiated by the intrinsic noise twice}), and for $q<1$, 

\begin{align}
	{\rm sgn}\left(\frac{\partial \delta_{q}(\sigma_{in}^2)}{\partial \sigma_{in}^2}\right)
	&={\rm sgn}\left(\left.\frac{\partial^2 \ln I(X;Y)}{\left(\partial\sigma_{in}^2\right)^2}\right|_{\sigma_{ex}^2=0}\right)
	\label{relation between delta robustness function and delta mutual information}
\end{align}
is satisfied, where ${\rm sgn}(\cdot)$ is the sign function. 

Here, from simple calculations, 

\begin{align}
	\frac{\partial^2 I(X;Y)}{\left(\partial\sigma_{in}^2\right)^2}<0\Rightarrow\frac{\partial^2 \ln I(X;Y)}{\left(\partial\sigma_{in}^2\right)^2}<0
\end{align}
is satisfied. Therefore, from Eq. (\ref{relation between delta robustness function and delta mutual information}), 

\begin{align}
	\frac{\partial^2 I(X;Y)}{\left(\partial\sigma_{in}^2\right)^2}<0\Rightarrow\frac{\partial \delta_{q}(\sigma_{in}^2)}{\partial \sigma_{in}^2}<0
	\label{equation-green-region}
\end{align}
is satisfied. Therefore, when $I(X;Y)$ is decreasing and convex upward with respect to $\sigma_{in}^2$, the robustness function $\delta_{q}(\sigma_{in}^2)$ at $q\sim1$ decreases with $\sigma_{in}^2$. Moreover, 

\begin{align}
	\frac{\partial^2 \ln I(X;Y)}{\left(\partial\sigma_{in}^2\right)^2}>0\Rightarrow\frac{\partial^2 I(X;Y)}{\left(\partial\sigma_{in}^2\right)^2}>0
\end{align}
is satisfied. Therefore, from Eq. (\ref{relation between delta robustness function and delta mutual information}), 

\begin{align}
	\frac{\partial \delta_{q}(\sigma_{in}^2)}{\partial \sigma_{in}^2}>0\Rightarrow\frac{\partial^2 I(X;Y)}{\left(\partial\sigma_{in}^2\right)^2}>0
	\label{equation-blue-region}
\end{align}
is also satisfied. Hence, when $I(X;Y)$ is decreasing against $\sigma_{in}^2$, and $\delta_{q}(\sigma_{in}^2)$ at $q\sim1$ is increasing against $\sigma_{in}^2$, $I(X;Y)$ is always convex downward with respect to $\sigma_{in}^2$. 

We now verify Eqs. (\ref{equation-green-region}) and (\ref{equation-blue-region}) numerically (Fig. \ref{threshold model SR}(f),(g)). When $q$ is close to 1, Eqs. (\ref{equation-green-region}) and (\ref{equation-blue-region}) are satisfied (Fig. \ref{threshold model SR}(f)). However,  when $q$ is far from 1, Eqs. (\ref{equation-green-region}) and (\ref{equation-blue-region}) are not satisfied (Fig. \ref{threshold model SR}(g)). These results are as expected. 

Therefore, it has been shown that the robustness function $\delta_{q}(\sigma_{in}^2)$ at $q\sim1$ decreases with the intrinsic noise $\sigma_{in}^2$ when $I(X;Y)$ is decreasing and convex upward against $\sigma_{in}^2$ (Fig. \ref{threshold model SR}(f)(green)). Thus, INIR is not always realized in the threshold model with SR, even when $I(X;Y)$ is decreasing against $\sigma_{in}^2$. However, this is not necessarily bad, and may indeed be an advantageous property. In the linear model and the threshold model without SR, reducing the intrinsic noise $\sigma_{in}^2$ increases the mutual information $I(X;Y)$, but decreases the robustness function $\delta_{q}(\sigma_{in}^2)$. Hence, there is a trade-off between the transmitted information and the robustness against extrinsic noise in the linear model and the threshold model without SR. In contrast, in the threshold model with SR, the mutual information $I(X;Y)$ and the robustness function $\delta_{q}(\sigma_{in}^2)$ at $q\sim1$ can be simultaneously increased by reducing the intrinsic noise $\sigma_{in}^2$ when $I(X;Y)$ is decreasing and convex upward with respect to $\sigma_{in}^2$ (Fig. \ref{threshold model SR}(f)(green)). Therefore, SR solves the trade-off problem between the transmitted information and the robustness against extrinsic noise.



\section{Mechanism\label{section-Mechanism}}
\begin{figure}[btp]
\begin{center}
	(a)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig5a.pdf}
	\end{minipage}
	(b)\hspace{-3.0mm}
	\begin{minipage}[t][][b]{40mm}
		\includegraphics[width=40mm,height=35mm]{./fig5b.pdf}
	\end{minipage}
	 \caption{
	 Threshold model.  $\sigma_{in}^2$-dependencies of (a) $H(Y)$ and (b)  $H(Y|X)$. $\partial I(X;Y)/\partial \sigma_{in}^2<0$ (solid line), $\partial I(X;Y)/\partial \sigma_{in}^2>0$ (dashed line).
	 }
	\label{entropy}
\end{center}
\end{figure}


In Sec. \ref{subsection-SR}, we showed that when $\partial I(X;Y)/\partial\sigma_{in}^2<0$ and $q\sim1$, the sign of $\partial\delta_{q}(\sigma_{in}^2)/\partial\sigma_{in}^2$ strongly depends on the sign of $\partial^2 I(X;Y)/(\partial\sigma_{in}^2)^2$. The robustness against extrinsic noise increases with respect to the intrinsic noise when $\partial^2 I(X;Y)/(\partial\sigma_{in}^2)^2>0$ (Fig. \ref{threshold model SR}(f)(blue)), whereas it decreases when $\partial^2 I(X;Y)/(\partial\sigma_{in}^2)^2<0$ (Fig. \ref{threshold model SR}(f)(green)). In this subsection, we explain the relation between $\partial\delta_{q}(\sigma_{in}^2)/\partial\sigma_{in}^2$ and $\partial^2 I(X;Y)/(\partial\sigma_{in}^2)^2$ more intuitively by decomposing $I(X;Y)$.

The mutual information $I(X;Y)$ is decomposed as follows:

\begin{align}
	I(X;Y)&=H(Y)-H(Y|X)
	\label{the decomposition of the mutual information into entropies}
\end{align}
where $H(Y)$ is the entropy of $Y$ and $H(Y|X)$ is the entropy of $Y$ given by $X$, which is expressed as:

\begin{align}
	H(Y)&=-\sum_{Y}p(y)\log_{2}p(y)\\
	H(Y|X)&=-\int_{X}p(x)\sum_{Y}p(y|x)\log_{2}p(y|x)dx
\end{align}
where $X$ is a continuous random variable and $Y$ is a discrete random variable.

In the threshold model, when $\partial I(X;Y)/\partial\sigma_{in}^2<0$, both $H(Y)$ and $H(Y|X)$ are increasing and convex upward with respect to the intrinsic noise $\sigma_{in}^2$ (Fig. \ref{entropy}(a), (b)(solid line)), and thus satisfy

\begin{align}
	&\frac{\partial H(Y)}{\partial \sigma_{in}^2}\geq0,&\frac{\partial H(Y|X)}{\partial \sigma_{in}^2}\geq0
	\label{first derivative of H(Y) and H(Y|X)}\\
	&\frac{\partial^2 H(Y)}{\left(\partial \sigma_{in}^2\right)^2}\leq0,&\frac{\partial^2 H(Y|X)}{\left(\partial \sigma_{in}^2\right)^2}\leq0
	\label{second derivative of H(Y) and H(Y|X)}
\end{align}
These inequalities are also satisfied for the extrinsic noise $\sigma_{ex}^2$.

From Eq. (\ref{the decomposition of the mutual information into entropies}), when $\partial H(Y)/\partial \sigma_{in}^2>\partial H(Y|X)/\partial \sigma_{in}^2$, $I(X;Y)$ increases with $\sigma_{in}^2$. However, when $\partial H(Y)/\partial \sigma_{in}^2<\partial H(Y|X)/\partial \sigma_{in}^2$, $I(X;Y)$ decreases with $\sigma_{in}^2$. Therefore, we can interpret $\partial H(Y)/\partial \sigma_{in}^2$ as the beneficial effect of noise, acting to increase the transmitted information, and $\partial H(Y|X)/\partial \sigma_{in}^2$ as the harmful effect of noise, acting to decrease the transmitted information. Moreover, from Eq. (\ref{second derivative of H(Y) and H(Y|X)}), both effects of noise decrease as the intrinsic noise $\sigma_{in}^2$ increases.  

From Eqs. (\ref{the decomposition of the mutual information into entropies}) and (\ref{second derivative of H(Y) and H(Y|X)}), when $\partial^2 I(X;Y)/(\partial\sigma_{in}^2)^2>0$ (Fig. \ref{threshold model SR}(f)(blue)), the following is satisfied:

\begin{align}
	&\left|\frac{\partial^2 H(Y)}{\left(\partial \sigma_{in}^2\right)^2}\right|<\left|\frac{\partial^2 H(Y|X)}{\left(\partial \sigma_{in}^2\right)^2}\right|
\end{align}
Therefore, when $\partial^2 I(X;Y)/(\partial\sigma_{in}^2)^2>0$ (Fig. \ref{threshold model SR}(f)(blue)), the change in the harmful effect of noise $\partial H(Y|X)/\partial \sigma_{in}^2$ is greater than that in the beneficial effect of noise $\partial H(Y)/\partial \sigma_{in}^2$. Thus, we focus on the change in the harmful effect of noise $\partial H(Y|X)/\partial \sigma_{in}^2$. As mentioned above, from Eq. (\ref{second derivative of H(Y) and H(Y|X)}), the harmful effect of noise $\partial H(Y|X)/\partial \sigma_{in}^2$ decreases as the intrinsic noise $\sigma_{in}^2$ increases.  Therefore, when $\sigma_{in}^2$ is larger, the decrease in transmitted information as a result of noise is smaller. Thus, the robustness against extrinsic noise increases at higher levels of intrinsic noise.

In contrast, when $\partial^2 I(X;Y)/(\partial\sigma_{in}^2)^2<0$ (Fig. \ref{threshold model SR}(f)(green)), the following is satisfied:

\begin{align}
	&\left|\frac{\partial^2 H(Y)}{\left(\partial \sigma_{in}^2\right)^2}\right|>\left|\frac{\partial^2 H(Y|X)}{\left(\partial \sigma_{in}^2\right)^2}\right|
\end{align}
Therefore, when $\partial^2 I(X;Y)/(\partial\sigma_{in}^2)^2<0$ (Fig. \ref{threshold model SR}(f)(green)), the change in the beneficial effect of noise $\partial H(Y)/\partial \sigma_{in}^2$ is greater than that in the harmful effect of noise $\partial H(Y|X)/\partial \sigma_{in}^2$. Thus, we focus on the change in the beneficial effect of noise $\partial H(Y)/\partial \sigma_{in}^2$. From Eq. (\ref{second derivative of H(Y) and H(Y|X)}), the beneficial effect of noise $\partial H(Y)/\partial \sigma_{in}^2$ decreases as the intrinsic noise $\sigma_{in}^2$ increases. Therefore, when $\sigma_{in}^2$ is larger, the increase in transmitted information as a result of noise is smaller. Thus, the robustness against extrinsic noise decreases with the intrinsic noise.


\section{Discussion\label{section-Discussion}}
In this paper, we first analyzed the robustness against extrinsic noise in a linear model. We showed that the robustness against extrinsic noise increases with intrinsic noise. A threshold model in which the threshold is equal to the mean of the input was then analyzed, and we showed that the robustness against extrinsic noise again increases with the level of intrinsic noise. Therefore, INIR is realized in these models.  Moreover, we discovered that the intrinsic noise dependency of the robustness against extrinsic noise is stronger when the intrinsic noise is larger than the variance of the input in these models. 

This raises the question of whether the intrinsic noise is larger than the variance of the input in actual biological systems. In other words, is the robustness against the extrinsic noise strongly dependent on the intrinsic noise in actual biological systems? To address this question, we focus on the mutual information $I(X;Y)$ when the intrinsic noise is larger than the variance of the input, i.e., $\sigma_{in}^2>1$ in the linear model or $\sigma_{in}^2>2/(\pi\ln2)$ in the threshold model. In these cases, $I(X;Y)<0.5$ and $I(X;Y)<1-1/\sqrt{2}\approx0.29\cdots$ are satisfied in the linear model and in the threshold model, respectively. Therefore, we can estimate whether the robustness against the extrinsic noise is strongly dependent on the intrinsic noise from whether the mutual information is less than $0.3\sim0.5$ bits. Note that the input and noise distributions and input--output relations in actual biological systems are not exactly the same as those in our models. Therefore, this estimation is only approximate. With reference to several experiments that measure the mutual information between input molecules and cellular responses such as gene expressions, the range of the mutual information is $0.6\sim1.2$ bits \cite{Cheong2011,Uda2013,Selimkhanov2014,Voliotis2014,Suderman2017,Potter2017,Granados2017,Granados2018,Ruiz2018}. Therefore, the robustness against the extrinsic noise may not be  strongly dependent on the intrinsic noise at the cellular level. However, in computational studies analyzing a dendritic spine, which is a small part of a neuron, the range of the mutual information is $0.1\sim0.3$ bits \cite{Koumura2014,Fujii2017,Tottori2019}. Therefore, the robustness against the extrinsic noise may be  strongly dependent on the intrinsic noise in dendritic spines.


We showed that larger levels of intrinsic noise produce a higher degree of robustness against extrinsic noise in both the linear and threshold model when the threshold is equal to the mean of the input. However, larger values of intrinsic noise are not necessarily advantageous. This is because more intrinsic noise decreases the transmitted information, regardless of extrinsic noise. Therefore, in basic terms, cells may evolve to decrease the level of intrinsic noise as a means of increasing their transmitted information. However, intrinsic noise is often a significant component of biological systems \cite{Elowitz2002,Bar-Even2006,Newman2006,Taniguchi2010,Koumura2014,Fujii2017,Tottori2019}, and we only argue that the robustness against extrinsic noise is high in such cases. 

We further showed that SR appears in the threshold model when the threshold is much larger than the mean of the input. In this model, there is a region where the robustness against extrinsic noise decreases with the intrinsic noise, meaning that INIR is not always realized. However, in the threshold model with SR, the robustness against extrinsic noise and the transmitted information can be simultaneously increased by reducing the intrinsic noise. Therefore, SR solves the trade-off problem between the robustness against extrinsic noise and the transmitted information.

In artificial experimental settings,  SR has been observed in various biological systems, especially neuronal systems \cite{Douglass1993,Levin1996,Russell1999,Gluckman1996,Stacey2000,Stacey2001,Srebro1999,Mori2002,Kitajo2007,Stufflebeam2000,Tanaka2008,Ward2010}. Douglass et al. were the first to report the appearance of SR in biological systems, namely a sensory neuron in a crayfish tail fan \cite{Douglass1993}. Following this study, it has been reported that SR appears in the cercal sensory neurons of crickets \cite{Levin1996}, hippocampal slices of rats \cite{Gluckman1996,Stacey2000,Stacey2001}, electrical sensory organs of paddlefish \cite{Russell1999}, human vision \cite{Srebro1999,Mori2002,Kitajo2007}, and human audition \cite{Stufflebeam2000,Tanaka2008,Ward2010}. Moreover, the threshold response is observed in many biochemical reactions \cite{Ozbudak2004, Melen2005, Narula2012}, which may indicate the existence of SR. These reactions not only detect weak signals by utilizing noise, but also ensure robustness against additional noise. 

However, in natural environments, the threshold systems may not exhibit SR. This is because the mutual information is higher when the threshold is closer to the mean of the input. Indeed, in developmental processes, where the input--output relation between morphogen molecules and gene expressions is the threshold response, the number of cells with high gene expressions is similar to that with low gene expressions \cite{Gregor2007,Tkacik2008b}, which indicates that the threshold is close to the mean of the input. It has been reported that, even when the threshold is close to the mean of the input, SR appears if there are several threshold systems and the entire output is determined by the summation of individual outputs \cite{Stocks2000,Stocks2001}. Therefore, when the threshold  is equal to the mean of the input, SR does not appear in the information transmission by a single cell, but does appear in the information transmission by multiple cells \cite{Suderman2017}. For example, tissues and organs composed of multiple cells respond to the same input with multiple cells. Therefore, if individual cells exhibit a nonlinear response such as a threshold response, SR appears regardless of the relation between the input and the threshold. Indeed, several theoretical studies suggest that if multiple neurons transmit information, SR will even appear when the mean of the input is equal to the threshold \cite{Collins1995,Pei1996a,Chialvo1997,Stocks2001a}.

In this paper, we discussed SR in a static system, but SR can also appear in dynamical systems \cite{Gammaitoni1998}. Moreover, the transmitted information is conventionally quantified by the signal-to-noise ratio (SNR) in dynamical systems, rather than by the mutual information \cite{Gammaitoni1998}. However, even if it was quantified by other measures, the transmitted information and the robustness against the extrinsic noise could be simultaneously increased when SR appears, because there is always a region where the transmitted information is decreasing and convex upward with respect to the intrinsic noise. An example is given in Appendix \ref{appendix-sensitivity}.

In this paper, we assumed that the intrinsic noise and extrinsic noise obey Gaussian distributions. However, other types of distribution are often observed in biological systems \cite{Thattai2001,Shahrezaei2008,Taniguchi2010,Charlebois2011}. Moreover, we examined systems showing a linear response and a threshold response. However, smooth nonlinear response systems are often observed, as represented by Hill functions in biological systems.  To construct more a general theoretical framework and uncover the characteristics of INIR, analyses of these nonlinear systems should be considered.

Experimental verification of our theory is also necessary. Several experiments have measured the mutual information between extracellular molecules and cellular responses such as gene expressions \cite{Cheong2011,Uda2013,Selimkhanov2014,Voliotis2014,Suderman2017,Potter2017,Granados2017,Granados2018,Ruiz2018}. Furthermore, it is possible to generate extrinsic noise by changing the concentration of extracellular molecules from trial to trial. Therefore, we can measure the extrinsic noise dependency of the mutual information, i.e., the robustness against the extrinsic noise. Recent experimental studies have reported that the gene expression noise level can be changed without changing the gene expression mean by constructing synthetic gene circuits \cite{Farquhar2019}. This method enables the intrinsic noise to be modified without changing the input--output relation. Therefore, we could measure the intrinsic noise dependency of the robustness against extrinsic noise. These procedures would allow us to experimentally verify whether the robustness against extrinsic noise increases with the intrinsic noise in  a linear system, a threshold system without SR, and a threshold system with SR. 




\begin{acknowledgments}
This work was supported by the Creation of Fundamental Technologies for Understanding and Control of Biosystem Dynamics, CREST (JPMJCR12W3), of the Japan Science and Technology Agency, and by the Japan Society for the Promotion of Science (JSPS), KAKENHI Grant Numbers 17H06300, 17H6299, 18H03979 and 19K22860. M.F. was funded by the JSPS, KAKENHI Grant Numbers 16K12508 and 19K20382.
\end{acknowledgments}



