


\subsection{Stereophonic Reproduction}

Consider a stereophonic reproduction setup as shown in \figurename~\ref{fig:refsystemrelative} with base angle $\phi_0$ and loudspeaker distance $r_l$. 
The two loudspeakers are reproducing delayed and attenuated versions of the same signal. 
The gains applied to the left and right loudspeaker are denoted as $g_L$ and $g_R$, respectively, while the delays are $\tau_L$ and $\tau_R$.
The \gls{icld} is defined as $\text{ICLD}=20\log_{10}\frac{g_L}{g_R}=G_L-G_R$, where $G_L=20\log_{10}g_L$ and $G_R=20\log_{10}g_R$. 
The \gls{ictd}, on the other hand, is defined as $\tau_R-\tau_L$.
Notice how these definitions are given such that whenever \gls{ictd} and \gls{icld} have the same sign, their effect is consistent to one another. 
For instance, when both are positive, the left loudspeaker is louder and its signal also arrives earlier. 

If the \gls{ictd} is below the \emph{echo threshold},
the listener will perceive a single fused auditory event~\cite{BlauertBook}.
The echo threshold is strongly stimulus-dependent, 
and varies between $2$~ms (for clicks) and $40$~ms (for speech)~\cite{BlauertBook}.
For \glspl{ictd} between $1$~ms and the echo threshold, 
the auditory event is localised at the 
loudspeaker whose signal arrives first~\cite{BlauertBook,Litovsky:1999hc}.
This effect is called ``law of the first wavefront"~\cite{gardner1968historical}.
For \glspl{ictd} smaller than $1$~ms, 
a single fused ``phantom'' sound source is localised in a position that depends on both \gls{ictd} and \gls{icld}.
This psychoacoustic effect is called ``summing localization"~\cite{BlauertBook}. 
In the literature, summing localization and the law of the first wavefront 
are collectively referred to as ``precedence effect"~\cite{BlauertBook}.

\begin{figure}[tb]
\centering
\includegraphics[width=\columnwidth]{plot_11_williams_only-eps-converted-to.pdf}
\caption{Williams psychoacoustic curves, together with 
the panning curve associated to the \gls{psr} method with $18.7$~cm inter-microphone distance and to the ORTF microphone pair.
Amplitude methods are associated to points on the y-axis ($\text{ICTD}=0$~ms).}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:williamsonly}
\end{figure}

\figurename~\ref{fig:williamsonly} shows Williams time-amplitude psychoacoustic curves, 
which represent all the \gls{icld}-\gls{ictd} pairs that render the phantom source in the direction of the left and right loudspeaker~\cite{williams1999microphone}.
Other time-amplitude psychoacoustic curves are also available in the literature~\cite{Franssen:1964lp,rumsey2001spatial,simon2010time}. 

The psychoacoustic curves show that if one wishes to render a phantom source in the direction of the left (respectively, right) loudspeaker it is possible to use an \gls{icld} of about $15$ dB (respectively, $-15$~dB) without any \gls{ictd} (amplitude panning).
It is then possible to continuously pan between the two loudspeakers using \glspl{icld} that vary between these two extremes (notice that the psychoacoustic curves do not give information on how to do this exactly, and only provide information about the extreme directions).
\figurename~\ref{fig:williamsonly} also shows that it is possible to render a sound source in the direction of the left loudspeaker by reducing the \gls{icld} but increasing the \gls{ictd} (time-amplitude panning). 
If the \gls{icld} is reduced to zero (time panning), it is still possible to displace the phantom source all the way to the left (respectively, right) loudspeaker with an \gls{ictd} of about $1$~ms (respectively, $-1$~ms), which corresponds to the onset of the law of the first wavefront. 

Note that most of the available stereophonic panning curves assume the standard stereophonic setup involving a phantom source panned between two loudspeakers positioned symmetrically to the left and right of the listener at a specified distance and can only provide suboptimal performance for panning lateral sources.



\subsection{Stereophonic recording}

\begin{figure}[t]
    \centering
   
    \input{figures/mic_ref_system.tex}
   
    \caption{Reference system for the stereophonic recording system with incoming plane wave.}
    \pgfkeysvalueof{/tikz/dsp/label}}{fig:refsystemmics}
\end{figure}

Consider the reference system in \figurename~\ref{fig:refsystemmics}. 
Here, the microphones are positioned on a circle with radius $r_m$ facing outwards.
The inter-microphone distance is denoted by  $d$ and is related to the array radius by 
\begin{equation}
d=2r_m\sin\left(\frac{\phi_m}{2}\right)~,    
\pgfkeysvalueof{/tikz/dsp/label}}{eq:drm}
\end{equation}
where $\phi_m$ is the angle separating the two microphones, typically referred to as microphone base-angle. 
The value $\theta_s$ denotes the angle of an incoming plane wave.
The arrangement of the two microphones on a circle (as opposed to positioning them on the x-axis) is preferred in this paper because it facilitates the extension to the multichannel case, and aids the comparison with \gls{psr}, as discussed later. 

The first systematic approach to the problem of two-channel stereophonic recording
is attributed to Blumlein~\cite{eargle2004microphone,rumsey2001spatial}. 
The Blumlein pair consists of two figure-8 microphones positioned 
orthogonally to each other ($\phi_m=90^\circ$) in the same position ($d=0$~cm).
Here, each microphone is connected to the corresponding loudspeaker without mixing, which means that the inter-microphone time and level differences are identical to \glspl{ictd} and \glspl{icld}, respectively. 
Systems without mixing are also the focus of the rest of this paper.
In the specific case of the Blumlein pair, the \glspl{ictd} are zero, since the microphone pair is coincident.


A variety of other stereophonic arrangements have been proposed in the past few decades (see e.g.~\cite{eargle2004microphone, rumsey2001spatial}).
These include the ORTF, DIN, NOS and cardioid XY pair, the characteristics of which are summarised in  
Table~\ref{table:mics}. 
The operating curve associated to the ORTF is shown in \figurename~\ref{fig:williamsonly}.




\renewcommand{\arraystretch}{1.2}
\begin{table}[tb]
\begin{center}
\begin{tabular}{l|c|c|c|c}
\multicolumn{1}{c|}{\begin{tabular}[c]{@{}c@{}}Microphone\\ arrangement\end{tabular}} 
& \multicolumn{1}{c|}{\begin{tabular}[c]{@{}c@{}}Inter-mic.\\ distance $d$ \end{tabular}} & \multicolumn{1}{c|}{\begin{tabular}[c]{@{}c@{}}Microphone \\ base-angle $\phi_m$\end{tabular}} & \multicolumn{1}{c|}{\begin{tabular}[c]{@{}c@{}}Directivity\\ pattern\end{tabular}}  & \multicolumn{1}{c}{\begin{tabular}[c]{@{}c@{}}Coverage \\ angle\end{tabular}} \\ \hline \hline

PSR~\cite{desena2013}                                         & Adjustable                                                                                                    & $\phi_0$                   & Custom                                   & $\phi_0$  \\ \hline
Blumlein~\cite{eargle2004microphone}                               & $0$~cm                                                                                                     & $90^\circ$                                                                                               & Figure-8                                 & $68^\circ$                                                                              \\\hline
90-deg XY~\cite{eargle2004microphone}                                     & $0$~cm                                                                                                     & $90^\circ$                                                                                               & Cardioid                                 & $176^\circ$                                                                             \\\hline
ORTF~\cite{eargle2004microphone}                                        & $17$~cm                                                                                                  & $110^\circ$                                                                                              & Cardiod                                  & $94^\circ$                                                                              \\\hline
DIN~\cite{eargle2004microphone}                                         & $20$~cm                                                                                                  & $90^\circ$                                                                                               & Cardioid                                 & $100^\circ$                                                                             \\\hline
NOS~\cite{eargle2004microphone}                                         & $30$~cm                                                                                                  & $90^\circ$                                                                                               & Cardioid                                 & $80^\circ$                                                                              \\
\end{tabular}
\end{center}
\caption{Characteristics of popular microphone arrangements. Here, the coverage angle denotes the range of angles that results in $\text{ICTD}$-$\text{ICLD}$ pairs within the Williams curves.}
\pgfkeysvalueof{/tikz/dsp/label}}{table:mics}
\end{table}





\glsreset{psr}
\subsection{Perceptual Soundfield Reconstruction}\pgfkeysvalueof{/tikz/dsp/label}}{sec:psr}
The methods discussed in the previous subsection were mostly designed on a trial-and-error basis. 
A systematic framework for the design of circular microphone arrays was proposed in~\cite{desena2013}, based on earlier work by Johnston and Lam ~\cite{Johnston:2000oe} and termed \gls{psr}.
This section summarizes the design procedure, but for the specific case of a stereophonic setup.

In \gls{psr}, each microphone is connected to a corresponding loudspeaker. 
The microphone base-angle, $\phi_m$, is identical to the loudspeaker base-angle, $\phi_0$, which allows a straightforward extension to full $360^\circ$ multichannel rendering. 
The inter-microphone delay (and thus \gls{ictd}) is: 
\begin{equation}
    \text{ICTD}(\theta_s)=\frac{d}{c}\sin\theta_s=\frac{1}{c}2r_m\sin\left(\frac{\phi_0}{2}\right)\sin\theta_s~,
    \pgfkeysvalueof{/tikz/dsp/label}}{eq:ictdmic}
\end{equation} 
where $c$ is the speed of sound ($c=343$ m/s in dry air at $20^\circ$). 
Notice that in the \gls{psr} literature, the $\text{ICTD}$ is expressed as a function of the array radius, $r_m$.
The remainder of this paper, on the other hand, uses the inter-microphone distance, $d$, which facilitates the comparison with other popular stereophonic microphone arrangements.


Consider the \gls{ictd} obtained for a plane wave with $\theta_s=\frac{\phi_0}{2}$, i.e. in the direction of the left microphone:
\begin{equation}
    \text{ICTD}\left(\phi_0/2\right) = \frac{d}{c}\sin\left(\frac{\phi_0}{2}\right)~.
    \pgfkeysvalueof{/tikz/dsp/label}}{ref:ictdmax}
\end{equation}

Given the value of $\text{ICTD}\left(\phi_0/2\right)$, psychoacoustic time-amplitude curves can be used to find the minimum \gls{icld} necessary to render a phantom source in the direction of the left loudspeaker. 
Rather than using Franssen curves as in~\cite{desena2013}, which are considered to be not sufficiently precise for quantitative design~\cite{BlauertBook}, this paper uses Williams curves.
Setting the $\text{ICLD}$ to be equal to the Williams value yields the constraint  $\text{ICLD}\left(\phi_0/2\right)=\text{ICLD}_{W,L}(\text{ICTD}\left(\phi_0/2\right))$, where $\text{ICLD}_{W,L} (\text{ICTD})$ is the Williams curve associated to the left loudspeaker (the top curve in \figurename~\ref{fig:williamsonly}).
The value $\text{ICLD}_{W,L}(\text{ICTD}\left(\phi_0/2\right))$ will denoted by $\text{ICLD}_{W}$ in the following.
For $\theta_s=-\frac{\phi_0}{2}$ (i.e. the direction of the right loudspeaker) one has $\text{ICLD}\left(-\phi_0/2\right)=\text{ICLD}_{W,R}(\text{ICTD}\left(-\phi_0/2\right))$,
which, due to the symmetry of the problem, is equivalent to $\text{ICLD}\left(-\phi_0/2\right)=-\text{ICLD}_{W}$. 
A third trivial point can be added, i.e. $\text{ICLD}(0)=0$~dB.

It remains to choose how to connect these three points.
Two choices were explored in \cite{desena2013}: a simple straight line~\cite{Hacihabiboglu:2010fq}, or a modified version of the tangent panning law~ \cite{desena2013}.
While both approaches were shown to lead to a good localization accuracy (i.e. the phantom source was shown to be perceived close to the intended direction $\theta_s$), the latter approach allows to link the design with other methods (e.g. \gls{vbap}~\cite{Pulkki:1997jt}) and consists of the following parametric function: 
\begin{equation}
	\text{ICLD}(\theta_s)=20\log_{10}\frac{\sin\left(\frac{\phi_0}{2}+\beta+\theta_s\right)}{\sin\left(\frac{\phi_0}{2}+\beta-\theta_s\right)}~,
	\pgfkeysvalueof{/tikz/dsp/label}}{eq:tanpangenphi}
\end{equation}
where $\beta$ is a free parameter that is used to satisfy the constraint $\text{ICLD}(\phi_0/2)=\text{ICLD}_{W}$ (the other two points are then also satisfied due to symmetry), which results in 
\begin{equation}
	\beta = \arctan\left(\frac{10^{\frac{\text{ICLD}_{W}}{20}} \sin(\phi_0)}{1-10^{\frac{\text{ICLD}_{W}}{20}}\cos(\phi_0)}\right)~.
	\pgfkeysvalueof{/tikz/dsp/label}}{eq:beta}
\end{equation}

To summarize, in order to obtain the \gls{psr} panning curve one should (a) set the free parameter $d$ (or, equivalently,  $r_m$), (b) obtain $\text{ICTD}\left(\phi_0/2\right)$ from (\ref{ref:ictdmax}), (c) obtain $\text{ICLD}_{W}$ from Williams psychoacoustic curve and $\beta$ from (\ref{eq:beta}) and (d) obtain $\text{ICTD}(\theta_s)$ and $\text{ICLD}(\theta_s)$ from (\ref{eq:ictdmic}) and (\ref{eq:tanpangenphi}), respectively.
The result of this procedure is a family of panning curves parametrized by the value of the inter-microphone distance $d$.
In the extreme case $d=0$, one obtains an amplitude-only method ($ \text{ICTD}(\theta_s)=0~\forall~\theta_s$).
As the value of $d$ increases, the \glspl{ictd} increase while the \glspl{icld} dictated by~(\ref{eq:tanpangenphi}) decrease, thus achieving stereophonic rendering with a different time/amplitude trading ratio.
\figurename~\ref{fig:williamsonly} shows the panning curve obtained for $d=18.7$~cm.

The so-obtained $\text{ICLD}(\theta_s)$ and $\text{ICTD}(\theta_s)$ can be viewed either in the context of stereophonic panning, whereby they can be used to directly control appropriate loudspeaker gains and delays, or in the context of stereophonic recording.
In the latter case, the microphone directivity patterns can be designed so as to emulate $\text{ICLD}(\theta_s)$.
Thus, one wishes to set $\text{ICLD}(\theta_s)=20\log_{10}\frac{\Gamma_L(\theta_s)}{\Gamma_R(\theta_s)}$, where $\Gamma_L(\theta_s)$ and $\Gamma_R(\theta_s)$ denote the directivity patterns of the left and right microphones.
In general, first-order microphones (i.e. microphones with $\Gamma(\theta_s)$ of the type $\Gamma(\theta_s)=a_0+a_1\cos(\theta_s)$) are not sufficiently directive to achieve the necessary \glspl{icld}.
Second-order microphones are already sufficient for this purpose~\cite{desena2013}.
The remainder of this paper will refer to PSR \emph{panning/recording} to emphasize the fact that application is possible in both contexts.

Notice that PSR aims at panoramic reproduction where a listener is allowed not only to move their head (e.g.~to reduce localization ambiguities) but also to freely rotate their orientation with respect to an arbitrary, reference reproduction axis. 
In order to provide an orientation-independent homogeneous localization for a symmetric loudspeaker setup, stereophonic pairwise panning laws can be used as a starting point. 
Despite the fact that these laws provide suboptimal localization performance for lateral sources, sources that are positioned within the range of near-peripheral azimuths (i.e.~$\pm{}\phi_0/2$ with respect to the listener's frontal direction) can be rendered accurately. 
From a purely physical point of view, PSR was also shown, via an energetic analysis of the reproduced sound field, to provide a good reproduction of directional properties of the sound field in a wide listening area~\cite{desena2013}.

\subsection{Auditory system modelling}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:audmod}

The auditory system estimates the directions of sound sources based on a combination 
of monaural and binaural cues~\cite{BlauertBook}.
Localization in the horizontal plane is mostly reliant on binaural cues, particularly on differences in the time of arrival and on difference in level of a sound wave at the two ears.
\Glspl{itd} are caused by the different time of arrival of sound waves radiated by sources outside the median plane.
At low frequencies the auditory system
analyses the interaural time difference between the signals' fine structure~\cite{BlauertBook}.
At higher frequencies this mechanism becomes ambiguous, 
and the time differences between the signals' envelopes are used instead~\cite{BlauertBook}.
The maximum naturally occurring \gls{itd} is approximately $0.65$~ms~\cite{rumsey2001spatial,BlauertBook}.
\Glspl{ild} are caused by the acoustical shadowing of the head
and are strongly frequency-dependent.
At low frequencies the head is approximately transparent 
to the sound wave and the level differences are small.
As the wavelength approaches the size of the human head, the level differences become sensible.
The highest natural \gls{ild} is in the region of $20$~dB~\cite{BlauertBook}.

A common confusion in this context is to assume that \glspl{ictd} are identical to \glspl{itd}, and \glspl{icld} are identical to \glspl{ild}, which is incorrect. 
The wavefronts of each loudspeaker reach both ears and form interference patterns at the position of the ears,
which, in turn, lead to a complex relationship between \gls{ictd}-\gls{icld} and \gls{itd}-\gls{ild}.
Also notice that, as opposed to \glspl{ictd} and \glspl{icld}, \glspl{itd} and \glspl{ild} are frequency-dependent. 





The mechanisms by which the auditory system interprets the \gls{itd} and \gls{ild} cues
are complex and not yet fully understood~\cite{BlauertBook}.
Experimental evidence
suggests that humans use two main mechanisms for source localization, 
and that these mechanisms are to a certain degree independent from one another~\cite[p.173]{BlauertBook}.
The first interprets the interaural time shifts between the signals' fine structure and uses signal components below $1.6$~kHz.
The second interprets the interaural level differences and time shifts of the envelopes \emph{jointly}.
The latter mechanism seems to be dominant for signals with significant
frequency content above $1.6$~kHz~\cite[p.173]{BlauertBook}.

A first, most notable attempt to model binaural processing
was made by Jeffress in 1948~\cite{jeffress1948place}, who hypothesised that sound localization is governed by a mechanism of
running cross-correlation between the two channels.
While today this is still considered to be an adequate mean of measuring 
\glspl{itd}, it does not account for the presence of \glspl{ild}~\cite{BlauertBook}.
Lindemann~\cite{lindemann1986extension} 
proposed a model that incorporates this information in the cross-correlation 
mechanism by way of inhibitory elements that are physiologically plausible.
Gaik~\cite{gaik1993combined} extended this model further, 
based on the observation that \glspl{itd} and \glspl{ild} due to point-like sources in free field come in specific pairs.
For instance the \gls{itd} and \gls{ild} values for a source in the median plane are both small.
On the other hand, for a source to the right/left, both \gls{itd} and \gls{ild} are high.
In fact, in these cases the acoustic wave arriving 
at the far ear is both attenuated (because of head shadowing) 
and delayed (because of propagation time).

Gaik observed that when inconsistent \gls{itd}-\gls{ild} pairs (e.g. a left-leading \gls{ild} and a right-leading \gls{itd}) are presented over headphones, the auditory event width increases, and sometimes two separate events appear~\cite{gaik1993combined,BlauertBook}.
In other words, inconsistent \gls{itd}-\gls{ild} pairs cause increased localization uncertainty.
These unnatural conditions can arise also with multiple sources radiating coherent signals, as in stereophonic reproduction.
Indeed, although each loudspeaker acts as a free-field source, the signals due to the different loudspeakers add up at the ears, creating interference phenomena that may result in inconsistent \gls{itd}-\gls{ild} cues.
Quantifying the deviation between the reproduced \gls{itd}-\gls{ild} pairs and
the ones associated to natural sources is therefore useful
to study the localization uncertainty due to different multichannel methods.
A study presented by Pulkki and Hirvonen in~\cite{pulkki2005localization} 
goes in this direction.
For a given multichannel method they 
find the angle of the closest free-field source in terms of \gls{ild} 
and \gls{itd}, separately.
This model gives useful predictions when the angles corresponding to the \gls{ild} and to the \gls{itd} coincide.
However, in most cases the \gls{itd} and \gls{ild} cues provide contradicting information, 
and therefore the model output is hard to interpret~\cite{pulkki2005localization}.


\subsection{Localization uncertainty in the center of the sweet-spot}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:center}

\begin{figure}[t]
\centering
\includegraphics[width=0.95\columnwidth]{plot_11_oncentre-eps-converted-to.pdf}
\caption{Contour plot of localization uncertainty as a function of \gls{icld} and \gls{ictd}
in the center of the sweet-spot.
The dashed white lines denote delays that result in the loudspeaker signals arriving at the same time at one of the two ears.
Overlaid are the Williams' curves and the time-amplitude panning curve associated to the \gls{psr} method with $d=18.7$~cm.}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:hictdicldcentre}
\end{figure}

\figurename~\ref{fig:hictdicldcentre} shows the localization uncertainty 
produced by the proposed model 
for a listener in the center of the sweet-spot and for a stereophonic reproduction system with base angle $\phi_0=60^\circ$ and $r_l=2$~m (see~\figurename~\ref{fig:refsystemrelative}).
Two areas in the second and fourth quadrants have a significant localization uncertainty.
These areas correspond to cases where \gls{icld} and \gls{ictd} provide inconsistent information: one 
loudspeaker is leading in terms of \gls{icld} (i.e. it is louder)
while the other is leading in terms of \gls{ictd} (i.e. it arrives earlier).
This indicates that inconsistent \gls{ictd}-\gls{icld} pairs somehow 
translate to unnatural \gls{itd}-\gls{ild} cues,
which is in agreement with the experimental findings of Leakey in~\cite{leakey1959some}.
It may also be observed in \figurename~\ref{fig:hictdicldcentre} 
that large \gls{icld} values (outside $\approx \pm 13$ dB) result in a
low localization uncertainty for all \glspl{ictd}.
Here, one loudspeaker signal is masking the other.

Finally, it can be  observed that for  \glspl{ictd} around $\pm 0.3$~ms
the localization uncertainty increases even in the first and third quadrants, where  \glspl{ictd}-\glspl{icld} pairs are consistent.
At  \glspl{ictd} of approximately $\pm 0.3$~ms, the two loudspeaker signals arrive at the same time at one of the two ears (more specifically, at the left ear for $\text{ICTD}\approx+0.3$~ms and at the right ear for  $\text{ICTD}\approx-0.3$~ms). 
Appendix~\ref{sec:appoverlap} proves that these \glspl{ictd} can be approximated as
\begin{equation}
    \tau_o \approx\pm \frac{r_h}{c}\left[\cos\left(\theta_e-\frac{\phi_0}{2}\right)+\frac{\phi_0}{2}+\theta_e-\frac{\pi}{2}\right]~,
    \pgfkeysvalueof{/tikz/dsp/label}}{eq:tauinter}
\end{equation}
where $r_h$ denotes the head radius and $\theta_e$ denotes the angle between the forward-looking direction and the ear.
For $r_h=9$~cm, $\theta_e=100^\circ$ and $\phi_0=60^\circ$, then $\tau_o=\pm0.27$~ms.
If signals of both loudspeakers arrive at the same time at one of the ears,  that ear effectively receives  one instance of the rendered acoustic event, whereas the other ear receives two instances. 
A number of psychoacoustic studies investigated effects of presenting three coherent stimuli, two to one ear and the third to the other ear. 
It was found that the perceived event had ``complex spatial structure" including cases where, depending on relative delays between the three stimuli, two distinct acoustic sources could be perceived~\cite{BlauertBook}. 
The results of the proposed model are in agreement with the findings of these experiments.
In fact, simulations not shown here for space reasons, confirm that as the head radius $r_h$ changes, the areas with higher \glspl{ictd} in the first and third quadrants move in accordance to~(\ref{eq:tauinter}).
If one wishes to avoid these areas, the \gls{ictd} values should be restricted to the open interval $\text{ICTD}\in]-\tau_o, \tau_o[$. 
In \figurename~\ref{fig:hictdicldcentre}, notice how the \gls{psr} operating curve associated to $d=18.7$~cm avoids the areas with higher localization uncertainty around $\text{ICTD} \approx \pm 0.3$~ms. 
This is so by construction, as will be discussed later in Section~\ref{sec:timeinteffect}.












\subsection{Localization uncertainty in off-center positions}

\begin{figure}[tb]
\centering
     \begin{subfigure}{0.95\columnwidth}
         \centering
         \includegraphics[width=\textwidth]{figures/plot_11_offcentre_m10cm-eps-converted-to.pdf}
         \caption{Left off-center position $\mathbf{x}=\{-10,0\}$~cm}
         \pgfkeysvalueof{/tikz/dsp/label}}{fig:hictdicldoffcentrel10}
     \end{subfigure}
     \vspace{0.5cm}
     
     \begin{subfigure}{0.95\columnwidth}
         \includegraphics[width=\textwidth]{figures/plot_11_offcentre_10cm-eps-converted-to.pdf}
         \caption{Right off-center position $\mathbf{x}=\{10,0\}$~cm}
         \pgfkeysvalueof{/tikz/dsp/label}}{fig:hictdicldoffcentre10}
     \end{subfigure}
     \vspace{0.5cm}
     
     \begin{subfigure}{0.95\columnwidth}
         \includegraphics[width=\columnwidth]{figures/plot_11_offcentre_20cm-eps-converted-to.pdf}
         \caption{Right off-center position $\mathbf{x}=\{20,0\}$~cm}
         \pgfkeysvalueof{/tikz/dsp/label}}{fig:hictdicldoffcentre20}
     \end{subfigure}
     
\caption{Contour plot of localization uncertainty as a function of \gls{icld} and \gls{ictd} for a listener in the off-center positions $\mathbf{x}=\{-10,0\}$~cm,  $\mathbf{x}=\{10,0\}$~cm and $\mathbf{x}=\{10,0\}$~cm.
Overlaid are the time-amplitude panning curve associated to the \gls{psr} method with $d=18.7$~cm and to the ORTF microphone pair.
}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:hictdicldoffcentre}
\end{figure}



Figure~\ref{fig:hictdicldoffcentre} shows the localization uncertainty 
for a listener in a position $10$~cm to the left, and in a position $10$~cm and $20$~cm to the right of the sweet-spot.
The listener is still looking ahead, in a direction parallel to the y-axis.
It may be observed that these plots are almost identical to the on-center plot of~\figurename~\ref{fig:hictdicldcentre} but shifted horizontally.
This implies that the dominant effect is the change in the \glspl{ictd} observed by the listener as a consequence of having moved closer to the right loudspeaker.
The change in observed \glspl{icld}, on the other hand, appears to have a minor effect. 
Likewise, the relative change of direction of the loudspeakers also appears to have a minor effect (in the $0.2$ m position, the two loudspeakers appear at $25^\circ$ and $-35^\circ$ with respect to the listener, compared to $\pm 30^\circ$ for the central position). 


Notice how amplitude panning methods, which are associated to the line $\text{ICTD}=0$ ms (corresponding to the y-axis), lie in an area with increased localization uncertainty for these off-center positions. 
For the rightward positions in \figurename~\ref{fig:hictdicldoffcentre10} and~\ref{fig:hictdicldoffcentre20}, the localization uncertainty is particularly high for positive \gls{icld} values, which are meant to render phantom sources between the midline and the left loudspeaker.
Viceversa, for the leftward position \figurename~\ref{fig:hictdicldoffcentrel10}, the localization uncertainty is high for negative \gls{icld} values, 
which are meant to render phantom sources between the midline and the right loudspeaker. 
Notice also how the \gls{psr} panning curve associated to $d=18.7$~cm largely avoids areas with higher localization uncertainty in both leftward and rightward positions $10$~cm off-center. 




\subsection{Definition of relative inter-channel time and level differences}

In order to aid the interpretation of Figures~\ref{fig:hictdicldoffcentre}, it is useful to define two new quantities: the \emph{relative} \gls{ictd} and \gls{icld}, i.e. equivalent inter-channel time and level difference as they are observed at a point away from the center of the loudspeaker array.
These quantities will be referred to as \gls{rictd} and \gls{ricld}, respectively.

Let the observation point be positioned at $(x,y)$ (see \figurename~\ref{fig:refsystemrelative}).
Appendix~\ref{sec:rictld} provides a closed-form approximation for the \gls{ricld} and \gls{rictd} under the assumption of small displacement compared to the loudspeaker distance:
\begin{align}
\begin{split}
\text{RICLD} & \approx \text{ICLD} - \frac{x}{r_l} \frac{20\sin\left(\frac{\phi_0}{2}\right)}{\log_e(10)}~, \\
\text{RICTD} & \approx \text{ICTD}-x\frac{2}{c}\sin\left(\frac{\phi_0}{2}\right)~.
\end{split}
\pgfkeysvalueof{/tikz/dsp/label}}{eq:ricltd}
\end{align}

\subsection{Considerations on the effect of \protect\gls{rictd} and \protect\gls{ricld} in amplitude panning}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:amplitude}

Consider now again the $\mathbf{x}=\{10,0\}$~cm off-center plot in \figurename~\ref{fig:hictdicldoffcentre10}.
The point $\text{ICTD}=0$~ms and $\text{ICLD}=5$ dB (i.e. the left loudspeaker leads in amplitude) is associated to $\text{RICTD}=-0.29$~ms and $\text{RICLD}=4.7829$ dB, which have opposite sign and thus provide inconsistent information. 
The \gls{ricld} is now smaller than $\text{ICLD}=5$~dB because the right loudspeaker is closer to the listener.
The change, however, is small ($0.22$~dB), and the left loudspeaker still leads in amplitude. 
The \gls{rictd}, on the other hand, has changed significantly with respect to \gls{ictd}, and the right loudspeaker is now leading in time. 
To summarise, the left loudspeaker is louder and thus leads in terms of amplitude, but the right loudspeaker is now closer and thus leads in terms of time.
Even though the \gls{ictd} and \gls{icld} were consistent, the horizontal shift of the listener caused the \gls{rictd} and \gls{ricld} to become inconsistent, which, in turn, led to a high localization uncertainty. 

Negative values of \gls{icld}, on the other hand, are less problematic in terms of localization uncertainty for a listener in this position. 
For instance, $\text{ICTD}=0$~ms and $\text{ICLD}=-5$ dB is associated to relative values $\text{RICTD}=-0.29$~ms and $\text{RICLD}=-5.2171$ dB, which have the same sign and provide consistent information. 

Similar arguments can be made for a listener moving to the left (i.e. $x<0$), as shown in \figurename~\ref{fig:hictdicldoffcentrel10}. 
In this case, however, the plots shift horizontally to the \emph{left} and the critical area of the plot becomes the bottom one. 


In summary, amplitude methods lead to higher localization uncertainty whenever the listener moves to one side but aims to render sound sources in directions closer to the opposite side.
Section~\ref{sec:ti} will show that time-amplitude methods provide a lower localization uncertainty in positions away from the center of the sweet-spot. 

\subsection{Further considerations on \protect\gls{rictd} and \protect\gls{ricld}}
Notice how the \gls{ricld} is a function of $\frac{x}{r_l}$, while the \gls{rictd} is independent from $r_l$. 
This implies that for large loudspeaker arrays, movement of the observation point away from the center of the sweet-spot causes a larger change of \gls{rictd} compared to \gls{ricld}.
This holds also even for living-room-sized arrays.
Consider for instance the \gls{rictd} that will result in full perceived shift of the phantom source to one loudspeaker, i.e.  $\text{RICTD}=1$ ms.
Assuming $\text{ICTD}=0$~ms, $\text{ICLD}=0$~dB and $\phi_0=\pi/3$ ($60$ degrees), this is already achieved at $x=\frac{0.001 c}{2\sin\left(\phi_0/2\right)}=0.34$ m. 
In that position, the \gls{ricld}, on the other hand, is only $1.5$ dB.
In other words, a phantom source that appears halfway between loudspeakers for a listener in the center of the sweet-spot, collapses onto one of the loudspeakers when the listener moves to a position $34$~cm on either side. 
This compares to approximately $3$~cm on either side for binaural cross-talk cancellation using two  loudspeakers~\cite{rose2002sweet}. 
It should be noted that the degradation is gradual in both cases, which makes defining quantitatively the sweet-spot size difficult.

Vertical displacement is less significant than horizontal displacement.
This can be inferred by the fact that the first-order approximation~(\ref{eq:ricltd}) does not  depend on $y$ (it can be shown that $y$ appears starting from the second-order term). 
This is confirmed by \figurename~\ref{fig:spatial0radius}, which shows the localization uncertainty as a function of listener position for PSR panning/recording associated to $d=0$~cm. 
The plot shows that the localization uncertainty varies almost exclusively as a function of the lateral displacement, $x$. 
For this reason, the simulations in the remainder of the paper will focus on listener displacement along the $x$ axis ($y=0$).


\begin{figure}[tb]
     \centering
     \includegraphics[width=0.9\columnwidth]{figures/coincident-eps-converted-to.pdf}
     \caption{Localization accuracy of PSR for $d=0$~cm (i.e. coincident PSR) as a function of the listener position around the center of the sweet spot.}
     \pgfkeysvalueof{/tikz/dsp/label}}{fig:spatial0radius}
\end{figure}










\section{Introduction}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:compmodel}
\input{introduction.tex}

\section{Background}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:background}
\input{background.tex}




\section{Localization Uncertainty Model}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:proposedmodel}
\input{model.tex}



\section{Localization Uncertainty in Stereophonic Reproduction}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:ti}
\input{effect.tex}


\section{Time-Amplitude Panning/Recording Methods}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:titrading}
\input{time_intensity.tex}

\section{Summary and Conclusions}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:conclusions}
\input{conclusions.tex}

\begin{appendices}


\section{}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:appoverlap}
\input{appendix_icd.tex}

\section{}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:rictld}
\input{appendix_ricld_rictd.tex}

\end{appendices}


\section*{Acknowledgments}
The authors would like to thank Niccol\`{o} Antonello for pointing out that the proposed computational model has a statistical interpretation. 

\bibliographystyle{IEEEtran}

\subsection{Calculation of \glspl{ild} and \glspl{itd}}
\pgfkeysvalueof{/tikz/dsp/label}}{ref:calculate}

The \gls{itd} and \gls{ild} values are calculated as follows.
Acoustic sources are modelled as point sources in the free field. 
The \glspl{hrtf} are taken as the Kemar mannequin measurements from the CIPIC database (subject 25)~\cite{algazi2001cipic}. 
The sampling frequency is $44.1$~kHz.
The response of the cochlea is modelled using a gammatone filter-bank~\cite{slaney1993efficient} with 24 center frequencies equally spaced on the \gls{erb} scale between $60$~kHz and $15$~kHz~\cite{glasberg1990derivation}.
As a rough model of the neuron firing probability, the bandpass signals are half-wave rectified below $1.5$~kHz, while above $1.5$~kHz the envelope of each bandpass signal is taken using the discrete Hilbert transform~\cite{gaik1993combined}. 
The resulting signals are fed to 24 binaural processors that calculate the \gls{itd} and \gls{ild} values independently.
The \gls{itd} is calculated as the location of the maximum of the cross-correlation function evaluated over time lags between $[-0.7, 0.7]$~ms~\cite{gaik1993combined, faller2004source}.
The \gls{ild} is calculated as the energy ratio of the left and right channel~\cite{faller2004source}.
Altogether, the model produces a set of 24 \gls{itd}-\gls{ild} pairs (48 values in total).



\figurename~\ref{fig:natural_pairs} show the \gls{itd}-\gls{ild} pairs associated to free-field sources in the frontal horizontal plane.
The free-field sources are reproducing $50$~ms long white noise sample, multiplied by a Tukey window with taper parameter $5\%$.
Each simulation is repeated ten times to average out the effect of different noise realizations. 
Each point on the plot corresponds to a free field source positioned at angles between $\theta=-90^\circ$ and $\theta=90^\circ$ with an angular resolution of $5^\circ$.
In the remainder of this paper, these values will be denoted as $\text{FITD}_i(\theta)$ and $\text{FILD}_i(\theta)$, respectively, where $i$ is the critical band index and $\theta$ is the free-field source angle with respect to the listener's look direction.

It can be observed in \figurename~\ref{fig:natural_pairs} that the interaural cues are highly correlated, i.e. larger \gls{itd} values are typically associated to larger \gls{ild} values, which is due to the concurrent effect of sound propagation and diffraction around the head~\cite{gaik1993combined}.
Also, the maximum \gls{ild} values increase with frequency, which is due to the increasing head shadowing  associated to decreasing wave lengths~\cite{gaik1993combined}.
Some small asymmetries are observed, which are possibly due to noise or asymmetries in the measurement setup of the \gls{hrtf} dataset. 



\begin{figure}[tb]
\centering
    \includegraphics[width=0.95\columnwidth]{PlotNaturalness-eps-converted-to.pdf}
    \caption{The figure shows the \gls{itd}-\gls{ild} pairs associated to point-like free-field sources in each critical band.}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:natural_pairs}
\end{figure}

\subsection{Distance between \gls{itd}-\gls{ild} pairs}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:distance}
Let $\text{ILD}_i$ and $\text{ITD}_i$ denote the \gls{itd} and \gls{ild} values in the $i$-th critical band as observed by a listener under stereophonic reproduction (or other acoustical conditions). 

In order to combine the information of \gls{itd} and \gls{ild} cues across critical bands, it is useful to normalize all quantities to the maximum values of the free-field cues:
\begin{align}
\overline{\text{ITD}}_i &= \frac{\text{ITD}_i}{\max_{\substack{\theta}}|\text{FITD}_i(\theta)|}~, \\
\overline{\text{ILD}}_i&= \frac{\text{ILD}_i}{\max_{\substack{\theta}}|\text{FILD}_i(\theta)|}~.
\end{align}
The free-field pairs, $\text{FITD}_i(\theta)$ and $\text{FILD}_i(\theta)$, 
are normalized in the same way and are
denoted as $\overline{\text{FITD}}_i(\theta)$ and $\overline{\text{FILD}}_i(\theta)$, respectively.

Notice that while $\overline{\text{FITD}}_i(\theta)\in[-1,1]$ and $\overline{\text{FILD}}_i(\theta)\in[-1,1]$,
$\overline{\text{ITD}}_i$ and $\overline{\text{ILD}}_i$ can in theory be outside that range.
Indeed, there is no guarantee that the acoustical condition being analyzed has \gls{itd}-\gls{ild} values outside the range of values occurring for free-field sources.

\begin{figure}
\centering 
    \newlength\fheight
    \newlength\fwidth
    \setlength{\fheight}{0.45\columnwidth}
    \setlength{\fwidth}{0.45\columnwidth}
\input{figures/plot_distances_example.tex}\quad\quad\quad\quad
\caption{The figure shows the normalized \gls{itd} and \gls{ild} pairs associated to free-field sources for the $18$th critical band (black dots), and an example of a  $\{\overline{\text{ITD}},\overline{\text{ILD}}\}$ point in $\{0.5,-0.5\}$.}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:points}
\end{figure}

\begin{figure}
\centering 
    \setlength{\fheight}{4cm}
    \setlength{\fwidth}{0.8\columnwidth}
\input{figures/plot_distances_dist.tex}
\caption{Distance between $\{\overline{\text{ITD}}_{18}, \overline{\text{ILD}}_{18}\}=\{0.5,-0.5\}$ and the free-field point $\{\overline{\text{FITD}}_{18}(\theta), \overline{\text{FILD}}_{18}(\theta)\}$ as a function of direction $\theta$ and for different distance functionals.}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:distance_05}
\end{figure}



The aim is now to select a distance functional between 
$\{\overline{\text{ITD}}_i, \overline{\text{ILD}}_i\}$ and $\{\overline{\text{FITD}}_i(\theta), \overline{\text{FILD}}_i(\theta)\}$
according to some meaningful psychoacoustic criterion.
Consider the distance defined by the classical $p$-norm:
\begin{equation} \left(\left|\overline{\text{ITD}}_i-\overline{\text{FITD}}_i(\theta)\right|^{p}+\left|\overline{\text{ILD}}_i-\overline{\text{FILD}}_i(\theta)\right|^{p}\right)^{\frac{1}{p}}~.
\pgfkeysvalueof{/tikz/dsp/label}}{eq:distance}
\end{equation}

\figurename~\ref{fig:points} shows an example of an observed $\{\overline{\text{ITD}}_{15}, \overline{\text{ILD}}_{15}\}$ point positioned at $\{0.5,-0.5\}$ in the $15$th critical band (the one centered at $6.07$~kHz).
The distance between each free-field source $\{\overline{\text{FITD}}_{15}(\theta), \overline{\text{FILD}}_{15}(\theta)\}$ and the $\{\overline{\text{ITD}}_{15}, \overline{\text{ILD}}_{15}\}=\{0.5,-0.5\}$ point is plotted in \figurename~\ref{fig:distance_05} as a function of the free-field source angle, $\theta$, for different distance functionals.

The experimental evidence shows that subjects presented with contradicting \gls{itd}-\gls{ild} pairs are likely to report split auditory events~\cite{gaik1993combined}.
The Euclidean distance ($p=2$) does not emulate this behaviour, as it leads to a single minimum in $\theta=0$, as shown in \figurename~\ref{fig:distance_05}. 
The Manhattan distance ($p=1$) is nearly constant in the $[-30^\circ,30^\circ]$ angular sector.
The $0.5$-distance, on the other hand,
causes two sharp minima, 
one of which is centred in the direction corresponding to the \gls{itd} cue,
which is compatible with the psychoacoustic evidence~\cite[p.170]{BlauertBook}. 
Other values of $p$ close to $0.5$ would also retain this behaviour.
In the next section it is shown that
the model has very good predictive power even without careful tuning of $p$.

The distance defined in~(\ref{eq:distance}) does not satisfy the triangle inequality for $p<1$.
However, the same distance raised to power $p$, does satisfy all the properties of a distance~\cite{kolmogorov1999elements}.
Hence, the $p$-norm distance used here is:
\begin{equation}
\xi_i\left(\theta|\overline{\text{ITD}}_i,\overline{\text{ILD}}_i\right) = \left|\overline{\text{ITD}}_i-\overline{\text{FITD}}_i(\theta)\right|^{p}+\left|\overline{\text{ILD}}_i-\overline{\text{FILD}}_i(\theta)\right|^{p}~.
\pgfkeysvalueof{/tikz/dsp/label}}{eq:dpi}
\end{equation}

The objective is to obtain a function of $\theta$ that quantifies the likelihood that a sound is perceived in that direction.
Intuitively, this function should be inversely proportional to $\xi_i(\theta|\overline{\text{ITD}}_i,\overline{\text{ILD}}_i)$, such that whenever the distance is small, the likelihood is high (and viceversa).
Let this function be
\begin{equation}
f_i\left(\theta|\overline{\text{ITD}}_i,\overline{\text{ILD}}_i\right) =
Ke^{-\xi_i\left(\theta|\overline{\text{ITD}}_i,\overline{\text{ILD}}_i\right)}~,
\pgfkeysvalueof{/tikz/dsp/label}}{eq:inversefunc}
\end{equation}
where $K$ is a positive constant. 
Although other choices are available\cite{De-Sena:2013}, the advantage of~(\ref{eq:inversefunc}) is that it gives the model an explicit statistical interpretation in the \gls{ml} framework, as will be discussed in the next subsection.

The next step is to integrate the information from the different critical bands.
The mechanisms governing this stage of perception are generally regarded 
as complex and not well understood~\cite{pulkki2005localization, BlauertBook}.
Here, the information across critical bands is combined as a loudness-weighted average:
\begin{equation}
f\left(\theta|\overline{\text{ITD}},\overline{\text{ILD}}\right) = \frac{K}{N}\sum_{i=1}^N w_i e^{-\xi_i\left(\theta|\overline{\text{ITD}}_i,\overline{\text{ILD}}_i\right)}~,
\pgfkeysvalueof{/tikz/dsp/label}}{eq:like}
\end{equation}
where $w_i$ are the loudness weights, $\overline{\text{ILD}}$ denotes the vector $\overline{\text{ILD}}=\left[\overline{\text{ILD}}_1,...,\overline{\text{ILD}}_N\right]$ and $\overline{\text{ITD}}=\left[\overline{\text{ITD}}_1,...,\overline{\text{ITD}}_N\right]$~, and $N$ is the number of critical bands ($N=24$).
The loudness weights are set according to the procedure proposed in~\cite{supper2005onset}, which results in critical bands with a higher signal level weighting more than critical bands with little active content~\cite{supper2005onset}.
The procedure involves (a) calculating the SPL levels in each critical band, (b) converting them to phon levels through the BS ISO 226:2003 equal-loudness contours, and (c) converting them to $w_i$ weights using a function where a $10$ phon reduction leads to a halving of the weight, in line with Stevens's model~\cite{stevens1957psychophysical}.  



\subsection{Statistical interpretation}
Suppose that $\{\overline{\text{ITD}}_i, \overline{\text{ILD}}_i\}$ are noisy observations of a point-like free-field source at angle $\theta_0$ associated to a set of true $\{\overline{\text{FILD}}_i(\theta_0), \overline{\text{FITD}}_i(\theta_0)\}$ values:
\begin{align}
    \begin{split}
        \overline{\text{ITD}}_i&=\overline{\text{FITD}}_i(\theta_0)+u_i~,\\
        \overline{\text{ILD}}_i&=\overline{\text{FILD}}_i(\theta_0)+v_i~.
    \end{split}
\end{align}
where $u_i$ and $v_i$ are the noise components, which may arise for instance as a consequence of room reverberation or reproduction with multiple loudspeakers.
Let the joint distribution of the noise components be the following mixture of zero-mean bivariate theta-generalized normal distributions~\cite{goodman1973multivariate}:
\begin{equation}
f\left(u_1,...,u_N;v_1,...,v_N\right) = \frac{1}{N}\sum_{i=1}^N Ce^{-\frac{\left|u_i\right|^{p}+
\left|v_i\right|^p}{2\sigma^2}}~,
\pgfkeysvalueof{/tikz/dsp/label}}{eq:likestat}
\end{equation}
where $\sigma$ represents a standard deviation and $C$ is a normalization constant.

Notice that if $\frac{1}{2\sigma^2}=1$, equation (\ref{eq:likestat}) becomes identical to (\ref{eq:like}).
Suppose now that the auditory system estimates the true value $\theta_0$ using an \gls{ml} approach. 
Within this context, $f\left(\theta|\overline{\text{ITD}},\overline{\text{ILD}}\right)$, seen here as a function of $\theta$,  takes the meaning of a likelihood function. 

Although the objective of the proposed model is not to estimate $\theta$ itself (which is left for future work), the shape of the likelihood function gives information about how difficult it is to estimate it, as will be explained in the next subsection.

\subsection{Calculation of the localization uncertainty}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:quantunc}

The likelihood function quantifies the probability that a subject would perceive a source in a given direction $\theta$. 
From this perspective, a uniform (constant) likelihood function would result in a maximally uncertain (diffuse) event.
At the other extreme, an impulsive likelihood function would result in a minimally uncertain event.
Various measures can be used in this context.
This paper uses a modified version of the circular variance~\cite{sena2017localization,mardia2009directional}.
This measure is drawn from the field of directional statistics~\cite{mardia2009directional}, and has values between $0$, which is associated to an impulsive function, and $1$, which is associated to a constant function.
Calculating the modified circular variance involves normalising $f\left(\theta|\overline{\text{ITD}},\overline{\text{ILD}}\right)$ so that it sums to one (e.g. by adjusting the $K$ constant), and calculating the (modified) first cosine and sine moments:
\begin{align}
    \alpha&=\int_{0}^{2\pi}f\left(2\theta|\overline{\text{ITD}},\overline{\text{ILD}}\right)\cos(\theta)d\theta,\\
    \beta&=\int_{0}^{2\pi}f\left(2\theta|\overline{\text{ITD}},\overline{\text{ILD}}\right)\sin(\theta)d\theta~, 
\end{align}
where the modification consists of using $f\left(2\theta|\overline{\text{ITD}},\overline{\text{ILD}}\right)$ instead of $f\left(\theta|\overline{\text{ITD}},\overline{\text{ILD}}\right)$. 
This modification is motivated by the fact that the likelihood function only takes values in $\left[-\frac{\pi}{2},\frac{\pi}{2}\right]$, which would not lead to a circular variance of $1$ in case of a constant likelihood function. 
Then, the circular variance is given by~\cite{mardia2009directional}
\begin{equation}
   \text{H}\left(\overline{\text{ITD}},\overline{\text{ILD}}\right)=1-\sqrt{\alpha^2+\beta^2}~.
\end{equation}
 


The final step is to normalize the value of $\text{H}$.
Indeed, since $f\left(\theta|\overline{\text{ITD}},\overline{\text{ILD}}\right)$ is never impulsive (even for free-field sources), the value of $\text{H}$ is always larger than zero.
For instance, for $p=0.7$, the minimum $\text{H}$ is $0.49$. 
The following normalization yields values close to zero for estimates associated to free-field sources:
\begin{equation}
    \overline{\text{H}}\left(\overline{\text{ITD}},\overline{\text{ILD}}\right)=\frac{\text{H}\left(\overline{\text{ITD}},\overline{\text{ILD}}\right)-\text{H}_{\text{min}}}{1-\text{H}_{\text{min}}}
    \pgfkeysvalueof{/tikz/dsp/label}}{eq:end}
\end{equation}
where $\overline{\text{H}}\left(\overline{\text{ITD}},\overline{\text{ILD}}\right)$ denotes the final localization uncertainty estimates of the proposed model and $\text{H}_{\text{min}}=\min_{\theta}\text{H}\left(\overline{\text{FILD}}(\theta),\overline{\text{FITD}}(\theta)\right)$.

It should be noted that, in practice, all calculations above are made using a discretization of the angles $\theta$. 
The simulations in the remainder of this paper use a resolution of $5^\circ$.



\subsection{Model validation}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:experiment}
A formal listening test with 19 subjects was carried out in~\cite{desena2013} using a modified MUSHRA test~\cite{mushra}.
The subjects answered the question 
``How certain are you of the direction of the source?"
by giving a score on a continuous scale from 0 to 100.
The test was carried out in an audio booth using four synthesized $5$-channel surround sound methods: (a) pair-wise tangent panning law~\cite{Bernfeld:1973wd} (equivalent to horizontal \gls{vbap}); (b) near-field corrected second-order Ambisonics 
with mode-matching decoding at low frequency and
maximum-energy decoding at high frequency
~\cite{daniel2003spatial, Poletti:2000dk, daniel1998ambisonics};
(c) second-order Ambisonics with in-phase decoding~\cite{daniel2000reprsentation};
and (d) the quasi-coincident microphone array proposed in~\cite{desena2013}. 
The test was run for three sound source directions 
and two seating positions, and included both a reference (a loudspeaker in the intended direction) and an anchor (an approximately diffuse soundfield). 
The rendered virtual source directions were in front of the listener and $\pm{}18^{\circ}$ to the right and left of the front direction. 
Three excerpts (female speech, african bongo and cello) were used as representatives of common program material, and the resulting subjective scores were averaged across the three excerpts.
Details of the experiment are available in~\cite{desena2013}. 







In order to validate the proposed model, the experiment was replicated here through simulations.
The stimuli used in the simulations were (a) long white noise burst ($500$~ms), (b) short white noise burst ($50$~ms), (c) short pink noise burst ($50$~ms), and (d) impulsive sound. 
All stimuli (except for the impulsive sound) were multiplied by a Tukey window with a $5\%$ taper parameter.

The loudspeaker signals for each of the $5$-channel surround sound methods mentioned above were then calculated. 
Notice that the experiment and the simulations employed multichannel sound reproduction where more than two loudspeakers can be substantially active in some of the methods used (i.e.~Ambisonics). 
While the model will be used in a stereophonic context in this paper, it is independent from the number of loudspeakers that the input ear signals originate from. 

The acoustic path from each loudspeaker to the head was simulated in free-field conditions using appropriate time delays and distance attenuation (inverse square law) that depend on the position of the head.
The ear pressure signals were obtained using the Kemar mannequin \glspl{hrtf}.
The so-obtained ear pressure signals associated with multiple loudspeakers were added together. 
Finally, the resulting ear pressure signals were fed to the proposed model, and localization uncertainty estimates were obtained using (\ref{eq:end}).

\figurename~\ref{fig:pearson} shows the absolute Pearson correlation between the subjective scores and 
the localization uncertainty estimates, as a function of the parameter $p$ for different stimuli. 
All the types of stimuli result in an absolute Pearson correlation stronger than $-0.98$ for $p\in[0.7,0.9]$, 
confirming that the model predictions are strongly correlated with the experimental data.
The reason why the correlation is negative is that subjects' answers to the question ``How certain are you of the direction of the source?" are merely inverted with respect to localization uncertainty.

All types of random noise perform particularly well. 
This indicates that the choice of stimulus does not appear to be critical when aiming to predict the localization uncertainty of an experiment involving a varied programme material like the one in~\cite{desena2013} (which included speech, bongos and cellos).
Furthermore, since the free-field pairs $\{\overline{\text{FITD}}_i(\theta), \overline{\text{FILD}}_i(\theta)\}$ were obtained using short white noise bursts, the model appears to be robust to a mismatch between the stimulus used for the free-field pairs and the one used in the acoustic scene under analysis (in a machine learning context, these would be akin to training and testing, respectively).




Fine tuning of the $p$-norm distance is also not critical as long as contradicting \gls{itd}-\gls{ild} cues lead to a an approximately bimodal likelihood function.
For the short white noise burst, values of $p$ between $0.5$ and $1.0$ all give correlation coefficients of approximately $-0.99$. 
The Euclidean distance ($p=2$), on the other hand, yields a much weaker correlation of $-0.84$.
At the other extreme, a small $p$ also leads to weak correlations, indicating that norms closer to the $l_0$-norm do not provide an effective measure in this context.

The simulations in the remainder of this paper use the short white noise burst, and a distance function with $p=0.7$ (an approximate midpoint of the $p\in[0.5,1.0]$ interval with~$-0.99$ correlation). 
\figurename~\ref{fig:corrmodel} shows the scatter plot comparing the model estimates with the subjective scores.

Note that the selection of an earlier set of results from an experiment using a five channel setup for the validation of the proposed model is deliberate. 
While the derivation of the model is based on two-channel stereophony as the simplest possible spatial audio reproduction system, its predictive power extends beyond that as is shown.

Note also that the experiment did not use any head or position tracking and the listeners were instructed to keep as still as possible. 
Such a listening scenario is not entirely ecologically valid. 
However, since the proposed model is neither dynamic nor does it incorporate any mechanism to account for listener motion, the results from the experiment are indicative of the predictive power of the model. 

\begin{figure}
\centering 
\begin{tikzpicture}
\begin{axis}[
    width=\columnwidth,
    height=4cm,
	grid=major,
	xmin=0,
	xmax=2,
	ymax=1,
	ymin=0.8,
	cycle list name=exotic,
    xlabel={$p$},
    ylabel={Absolute correlation}]
    \draw[dashed](0.7,0) -- (0.7,1);

    
    






\addplot coordinates {
(0.05,0.269255)
(0.075,0.701248)
(0.1,0.868561)
(0.125,0.927652)
(0.15,0.952472)
(0.175,0.965472)
(0.2,0.973072)
(0.25,0.979701)
(0.3,0.982028)
(0.4,0.984116)
(0.5,0.985789)
(0.6,0.987392)
(0.7,0.98861)
(0.8,0.989102)
(0.9,0.988583)
(1,0.986821)
(1.25,0.975872)
(1.5,0.954383)
(1.75,0.922287)
(2,0.881172)};
\pgfkeysvalueof{/tikz/dsp/label}}{white}

\addplot coordinates {
(0.05,0.314888)
(0.075,0.716641)
(0.1,0.873759)
(0.125,0.933155)
(0.15,0.958751)
(0.175,0.970817)
(0.2,0.976714)
(0.25,0.981117)
(0.3,0.982165)
(0.4,0.982866)
(0.5,0.984258)
(0.6,0.98627)
(0.7,0.988352)
(0.8,0.990033)
(0.9,0.990955)
(1,0.99085)
(1.25,0.984978)
(1.5,0.970146)
(1.75,0.946618)
(2,0.915957)};
\pgfkeysvalueof{/tikz/dsp/label}}{pink}


\addplot coordinates {
(0.05,0.253757)
(0.075,0.701906)
(0.1,0.871371)
(0.125,0.930226)
(0.15,0.955124)
(0.175,0.967908)
(0.2,0.975109)
(0.25,0.98143)
(0.3,0.983489)
(0.4,0.985435)
(0.5,0.987155)
(0.6,0.988805)
(0.7,0.990021)
(0.8,0.990457)
(0.9,0.989829)
(1,0.987914)
(1.25,0.976462)
(1.5,0.954452)
(1.75,0.922002)
(2,0.880821)};
\pgfkeysvalueof{/tikz/dsp/label}}{longwhite}


\addplot coordinates {
(0.05,0.045344)
(0.075,0.53543)
(0.1,0.793597)
(0.125,0.888738)
(0.15,0.9256)
(0.175,0.942518)
(0.2,0.951843)
(0.25,0.961767)
(0.3,0.966707)
(0.4,0.971257)
(0.5,0.973521)
(0.6,0.975035)
(0.7,0.976023)
(0.8,0.976386)
(0.9,0.975925)
(1,0.974408)
(1.25,0.9643)
(1.5,0.941856)
(1.75,0.90346)
(2,0.847472)};
\pgfkeysvalueof{/tikz/dsp/label}}{impulse}



\end{axis}
\end{tikzpicture}
\caption{Absolute Pearson correlation between the model predictions and the testing set as a function of the distance functional parameter $p$ for different stimuli: short white noise burst~(\ref{white}), long white noise burst~(\ref{longwhite}), impulse~(\ref{impulse}) and pink noise~(\ref{pink}). The short white noise burst curve~(\ref{white}) is not visible because it coincides almost exactly with the long white noise burst curve~(\ref{longwhite}). The dashed vertical line denotes the value chosen in the remainder of the simulations in this paper ($p=0.7$).}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:pearson}
\end{figure}

\begin{figure}
\centering 
    \newlength\figW
    \newlength\figH
    \setlength{\figW}{0.45\columnwidth}
    \setlength{\figH}{0.45\columnwidth}
\input{figures/plot_scatter.tex}
\caption{Scatter plot comparing the subjective scores of the experiment presented in~\cite{desena2013} and the estimates of the proposed model for $p=0.7$ and for short white noise burst. The blue line indicates the best-fitting line. The Pearson correlation is $-0.99$.}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:corrmodel}
\end{figure}







\subsection{Effect of time-amplitude trading on localization uncertainty using PSR panning/recording}
\pgfkeysvalueof{/tikz/dsp/label}}{sec:timeinteffect}


\begin{figure}[tb]
     \centering
     \begin{subfigure}{\columnwidth}
         \centering
         \includegraphics[width=\textwidth]{0-eps-converted-to.pdf}
         \caption{On-center}
         \pgfkeysvalueof{/tikz/dsp/label}}{fig:meshoncenter}
     \end{subfigure}
     
     \begin{subfigure}{\columnwidth}
         \centering
         \includegraphics[width=\textwidth]{10-eps-converted-to.pdf}
         \caption{Off-center by $10$~cm}
         \pgfkeysvalueof{/tikz/dsp/label}}{fig:mesh10cm}
     \end{subfigure}
     
     \begin{subfigure}{\columnwidth}
         \centering
         \includegraphics[width=\textwidth]{20-eps-converted-to.pdf}
         \caption{Off-center by $20$~cm}
         \pgfkeysvalueof{/tikz/dsp/label}}{fig:mesh20cm}
     \end{subfigure}
        \caption{Localization uncertainty as a function of inter-microphone distance, $d$, and plane wave angle, $\theta_s$, in (a) the center position, (b) in a position $10$~cm off-center to the right, i.e. $(x,y)=(10, 0)$~cm, and (c) in a position $20$~cm off-center to the right, i.e. $(x,y)=(20, 0)$~cm. The colorbar is the same for all three figures to facilitate a comparison between the three figures.}
        \pgfkeysvalueof{/tikz/dsp/label}}{fig:radiusangle}
\end{figure}

\figurename~\ref{fig:radiusangle} shows the localization uncertainty for PSR panning/recording as a function of inter-microphone distance $d$ and plane wave angle $\theta_s$ for three different observation positions. 
A number of observations can be made:
\begin{itemize}[leftmargin=*]
    \item As the distance from the center of the sweet spot increases, so does the maximum localization uncertainty. 
    \item In the center position (\figurename~\ref{fig:meshoncenter}), the localization uncertainty is lowest around $d=0$ (i.e. amplitude methods). Two ``sidebands'' appear in the plot beyond $d=20$~cm. These sidebands correspond to the vertical stripes observed in  \figurename~\ref{fig:hictdicldcentre} at $\pm\tau_o$.
    \item At $x=10$~cm (\figurename~\ref{fig:mesh10cm}), the localization uncertainty is higher for inter-microphone distance $d=0$~cm, i.e. amplitude methods.
    It remains high in plane wave directions around the midline, while it reduces for other directions, especially around the $\theta_s\in[10^\circ, 20^\circ]$ range.
    This can be observed also in \figurename~\ref{fig:hictdicldoffcentre10}, which shows the panning curve associated to $d=18.7$~cm. 
    Here, the panning curves avoids the areas with higher localization uncertainty coming in from the left.
    \item At $x=20$~cm (\figurename~\ref{fig:mesh20cm}), the localization uncertainty tends to reduce for higher $d$, where it also has a lower variability as a function of source angles $\theta_s$. 
\end{itemize}


\figurename~\ref{fig:cumulative} shows the localization uncertainty as a function of inter-microphone distance $d$.
Here the results are averaged across angles between the midline and the left loudspeaker, which are more difficult to render when the listener moves to the right of the center.
The curve associated to the center position shows that, in that position, the inter-microphone distance with the lowest uncertainty is close to $d=0$~cm, as expected.
In typical use cases, one would like to minimize the uncertainty as it is observed in a whole region around the center of the sweet spot, instead of specific positions. 
\figurename~\ref{fig:cumulative} shows the localization uncertainty averaged for listener displacements between $x=0$~cm and $x=5$~cm (with a $1$~cm resolution). 
Here, the absolute minimum is achieved at $d=20$~cm. 
\figurename~\ref{fig:cumulative} also shows the localization uncertainty averaged for displacements between $x=0$~cm and $x=15$~cm. 
In this case, the average uncertainty increases significantly at small $d$, and the absolute minimum moved to $d\approx 30$~cm.

\begin{figure}[tb]
\small
    \centering
   
   
    \setlength{\figH}{5cm}
    \setlength{\figW}{0.8\columnwidth}
    \input{figures/cumulative.tex}
    \caption{Localization uncertainty as a function of inter-microphone distance~$d$. Results are averaged across angles between the midline and the left loudspeaker.}
    \pgfkeysvalueof{/tikz/dsp/label}}{fig:cumulative}
\end{figure}

In summary, the results suggest that smaller inter-microphone distances (amplitude methods) are preferable for positions close to the center, while larger inter-microphone distances result in lower localization uncertainty at larger distances. 
There is thus a trade off between lower localization uncertainty in the center of the sweet spot and away from the center.


Notice that if one can assume that only one listener is present, and that some form of user tracking is available, then the results here suggest that it would be beneficial to correct the $\text{ICTD}$s such that the $\text{RICTD}=0$. 
In other words, if the location of the (single) listener is known, one can correct the time delays such that the listener experiences amplitude panning. 

If, on the other hand, the listener's position is not being tracked, it is typically difficult to predict how far the listener/s will move away from the center of the sweet spot. 
A possible compromise is to use the inter-microphone distance that results in the largest \gls{ictd} to be no larger than $\tau_o$, and thus avoid the areas with slightly higher localization uncertainty in the first and third quadrant shown in \figurename~\ref{fig:hictdicldcentre}.
One can find this inter-microphone distance by equating $\text{ICTD}_\text{max}$ with $\tau_o$, and isolating $d$:
\begin{equation}
    d= r_h \frac{\cos\left(\theta_e-\frac{\phi_0}{2}\right)+\frac{\phi_0}{2}+\theta_e-\frac{\pi}{2}}{\sin\left(\frac{\phi_0}{2}\right)}~.
    \pgfkeysvalueof{/tikz/dsp/label}}{eq:optrm}
\end{equation}
For instance, the inter-microphone distance for $\theta_e=100^\circ$ and $r_h=9$~cm, is $d=18.7$~cm, which is the one used for the panning curve shown in \figurename~\ref{fig:hictdicldcentre} and \ref{fig:hictdicldoffcentre}.


\subsection{Comparison with popular stereophonic microphone arrangements}

This subsection presents the results of a comparison between three versions of PSR for $d=0, 18.7, 37.2$~cm, and the popular stereophonic microphone arrangements summarized in Table~\ref{table:mics}. 
The comparison involves choosing which source angles $\theta_s$ to simulate. 
While it is clear that for PSR the source angles should be within the microphone base-angle ($60^\circ$), the choice for other microphone arrangements is ambiguous. 
Indeed, sound engineers typically make a choice of the relative position of the sources of interest (e.g. an orchestra) often based on artistic considerations.
In order to make a more unambiguous choice here, the range of angles is chosen according to which source angles results in $\text{ICTD}$-$\text{ICLD}$ pairs within the Williams curves.
The resulting ranges are termed here \emph{coverage angles}, and are reported in Table~\ref{table:mics} for the popular microphone arrangements.
In the simulations that follow, a set of thirty $\theta_s$ are uniformly taken within the coverage angle for each microphone arrangement.

\figurename~\ref{fig:comparison} shows the localization uncertainty as a function of lateral displacement $x$. 
\figurename~\ref{fig:latdispl} shows the average across source angles, while \figurename~\ref{fig:latdisplvar} shows the excursion across source angles, defined as the difference between the largest and the smallest localization uncertainty across source angles.
A number of observations can be made:
\begin{itemize}[leftmargin=*]
\item The amplitude methods, i.e. Blumlein pair, PSR with $d=0$~cm, and cardioid XY pair (all of which are indicated by a $\times$ marker) have very similar performance; the difference between them stems from the slightly different distribution of $\text{ICLD}$s across source angles $\theta_s$.
\item In comparison to time-amplitude methods, the average uncertainty of the amplitude methods is lower for $|x|<2$~cm but higher beyond $|x|>10$~cm.
\item The uncertainty excursion of the amplitude methods is significantly higher than  time-amplitude methods beyond $|x|>10$~cm, reaching almost $0.5$ at $|x|=20$~cm.
This is associated to the difficulty in rendering sound sources associated to direction opposite to the lateral movement of the listener, as discussed in Section~\ref{sec:amplitude}, and it means that that sound sources at different angles will be perceived differently.
\item In the center ($x=0$~cm), the average uncertainty is lowest for the amplitude methods, and then, in order, for PSR ($d=18.6$~cm), ORTF ($d=17$~cm), DIN ($d=20$~cm), PSR ($d=37.2$~cm) and NOS ($d=30$~cm). PSR ($d=18.6$~cm) is lowest among time-amplitude methods because it is the only one that does not cross the $\pm\tau_o$ threshold (see \figurename~ \ref{fig:hictdicldcentre}).
\item The PSR with $d=18.6$~cm has a reasonably low average uncertainty across the lateral displacement, a low excursion within $|x|<15$~cm, but a higher excursion compared to other time-amplitude methods beyond $|x|>15$~cm.
\item The NOS and PSR version with $d=37.2$~cm have a higher average uncertainty in the center, but a smaller variation of the average uncertainty across $x$, and a low overall excursion. 
\item The PSR ($d=18.6$~cm), ORTF ($d=17$~cm), DIN ($d=20$~cm), and NOS ($d=30$~cm) all have low excursion, with little discernible difference between them across $x$. 
\end{itemize}

Overall, these results further confirm that amplitude methods have a better performance at the center of the sweet-spot, while time-amplitude methods have a lower overall uncertainty and a lower excursion away from the sweet-spot. 
A lower excursion means that whenever multiple sound sources are present at different angles, they are rendered with a similar localization uncertainty, which is an appealing property. 

A further advantage of \gls{psr} over the other time-amplitude methods, is that it has been shown to have a high localization \emph{accuracy}, meaning that subjects would perceive sound sources in directions close to the intended $\theta_s$ ~\cite{desena2013}.
This is also related to the fact that in PSR the coverage angle is identical to the loudspeaker base-angle, while the other methods will have (at least) a compression/decompression of angles from the coverage angle to the loudspeaker base-angle. 
An additional advantage of having the coverage angle identical to the loudspeaker base-angle (and microphone base-angle) is that the extension to the multichannel case is straightforward. 


\begin{figure}
\small
     \centering
     \begin{subfigure}{\columnwidth}
         \centering
    \setlength{\figH}{6.5cm}
    \setlength{\figW}{0.9\columnwidth}
    \input{figures/lateral_displacement_avg.tex}
    \caption{Localization uncertainty averaged across $\theta_s$ angles.}
    \pgfkeysvalueof{/tikz/dsp/label}}{fig:latdispl}
    \end{subfigure}
    \vspace{0.3cm}
    
     \begin{subfigure}{\columnwidth}
         \centering
    \setlength{\figH}{6.5cm}
    \setlength{\figW}{0.9\columnwidth}
    \input{figures/lateral_displacement_var.tex}
    \caption{Localization uncertainty excursion across $\theta_s$ angles calculated as the Difference between maximum and minimum localization uncertainty.}
    \pgfkeysvalueof{/tikz/dsp/label}}{fig:latdisplvar}
    \end{subfigure}
\caption{Comparison of different PSR versions with popular stereophonic microphone arrangements.}
\pgfkeysvalueof{/tikz/dsp/label}}{fig:comparison}
\end{figure}


\subsection{Comments on the relation to multichannel PSR}
The PSR stereophonic setup considered thus far can be seen as a subset of a multichannel circular array. 
In~\cite{desena2013}, it was shown that having only the two loudspeakers closest to $\theta_s$ as active yields a lower spatial fluctuation of the active intensity vector field and thus a larger sweet spot. 
This turns the overall multichannel design problem into a number of stereophonic problems. 

Consider the case of $M$ microphones distributed uniformly around the circle with $\phi_m=\phi_0=2\pi/M$.
Each of these microphones is connected to a loudspeaker in the same angular configuration, without mixing.
Johnston and Lam proposed in \cite{Johnston:2000oe} a microphone array with $M=5$, resulting in a base angle of $\phi_0=2\pi/5=72^\circ$, and radius $r_m=15.5$~cm. 
This array radius is very close to the value needed to avoid \gls{ictd} larger than $\tau_o=\pm0.3$~ms. 
Indeed, using (\ref{eq:drm}) and (\ref{eq:optrm}) for $\phi_0=2\pi/5=72^\circ$ yields $r_m=16.2$~cm.
Johnston and Lam stated that $r_m=15.5$~cm would conserve \gls{itd} cues that the listener would have experienced in the recording space, but without providing a rigorous explanation (it could indeed be argued, as Bernfeld did in~\cite{Bernfeld:1973wd}, that correct \gls{itd} cues can also be achieved by using \glspl{icld} alone).
This paper shows that Johnston and Lam's choice of array radius is the one that leads to a good compromise between low localization uncertainty in central and non-central listener positions.
The same choice was made in~\cite{desena2013}, where the directional accuracy was also improved significantly. 



