\section{Introduction}

Statistical inference procedures based on the minimization of a statistical distance, alternatively a divergence, have a natural appeal. 
The method based on likelihood, the canonical approach to inference in statistical data analysis, is itself a minimum divergence method; the
maximum likelihood estimator minimizes the likelihood disparity (Lindsay, 1994), a version of the Kullback-Leibler diverence. Among the 
different formats of minimum divergence inference, the approach based on the minimization of density-based divergences is of particular 
importance, as in this case the resulting procedures combine a high degree of model efficiency with strong robustness properties. 

The central elements in the present research are the collections of the density-based minimum divergence procedures based on the density power divergence
of Basu et al.~(1998). The density power divergence (DPD) is a popular class of divergence measures which has significant application not
just in the area of statistics, but also in machine learning, mathematical information theory, and many other branches of science. The popularity
and utility of this procedure make it important to study other similar divergences in search of competitive or better statistical (and other)
properties. Indeed, one such divergence that is known to us and one that has also left its unmistakable mark on the area of robust statistical inference
is the logarithmic density power divergence (LDPD); see, e.g., Jones et al.~(2001), Fujisawa and Eguchi (2008), Basu et al.~(2011) and Fujisawa (2013). 

Both the ordinary DPD and the LDPD belong to the functional density power divergence class that we will define in the next section. 
These two families of divergences have also been referred to as the BHHJ and the JHHB classes, or the type 1 and type 0 classes, or 
the $\beta$-divergence and the $\gamma$-divergence classes; more details about their applications may be found in  
Jones et al.~(2001), Fujisawa and Eguchi (2008), Cichocki and Amari (2010) and Basu et al.~(2011), among others. However, while the DPD belongs to the class of Bregman divergences, the LDPD does not. The DPD is also a single-integral, non-kernel divergence (Jana and Basu, 2019); the LDPD is not a singe integral divergence, although it is a non-kernel one.  
 The non-kernel divergences 
have also been called \textit{decomposable} (Broniatowski et al., 2012). The divergences within the DPD family have been shown to possess 
strong robustness properties in statistical applications. The LDPD family is also a very useful one in statistical inference. The 
applicability of the latter class of divergences in mathematical information theory has been explored by Kumar and Sundaresan~(2015a, b).


Our basic aim
in this work is to characterize the class of functional density power divergences. Essentially, each functional density power divergence corresponds
to a function with the non-negative real line as its domain. The DPD corresponds to the identity function, while the LDPD corresponds to the log function. 
Within the class of functional density power divergences, we will characterize the class of functions which generate legitimate divergences. In turn, this will provide a characterization of the functional density power divergence class. 




\section{The DPD and the LDPD}
Suppose $(\mathbb{R},\mathcal{B}_{\mathbb{R}}, \mu)$ is a measure space on the real line. Define the notation
\begin{equation}
\label{defn}
 \mathcal{L}_{\mu} := \left\{ f : \mathbb{R} \longrightarrow \mathbb{R} | \; f \geq 0, \;f \;\text{measurable}, \; \int_{\mathbb{R}} f \;d\mu = 1 \right\}.
\end{equation}
A divergence on $\mathcal{L}_{\mu}$ is a non-negative function $D : \mathcal{L}_{\mu} \times \mathcal{L}_{\mu} \longrightarrow [0,\infty] $ with the property 
\begin{equation}
\label{prop}
D(g,f) =0 \iff g=f \; a.e.[\mu].
\end{equation}
For the sake of brevity we will drop the dominating measure $\mu$ from the notation; it will be understood that all the integrations and almost sure statements are with respect to this dominating measure. We also suppress the dummy variable of integration from the expression of the integerals in the rest of the paper. +



One of the most popular examples of such families of divergences is the {\it density power divergence} (DPD) of Basu et al. (1998) defined as 
\begin{equation}
\label{DPD}
\text{DPD}_{\alpha}(g,f) :=  \int_{\mathbb{R}} f^{\alpha+1} - \Big( 1+\dfrac{1}{\alpha} \Big) \int_{\mathbb{R}} f^{\alpha}g + \dfrac{1}{\alpha}  \int_{\mathbb{R}} g^{\alpha+1}, \;\; {\rm for~all} \; \; g, f \in \mathcal{L}_{\mu},
\end{equation}  
where $\alpha$ is a non-negative real number. For $\alpha=0$, the definition is to be understood in a limiting sense as $\alpha \downarrow 0$, and the form of the divergence then turns out to be,
\begin{equation}
\label{DPD2}
\text{DPD}_{\alpha=0}(g,f) :=  \int_{\mathbb{R}} g \log \dfrac{g}{f}, \;\; {\rm for~all} \; \; g,f \in \mathcal{L}_{\mu},
\end{equation}  
which is actually the likelihood disparity (Lindsay, 1994); it is also a version of the Kullback-Leiber divergence. For $\alpha = 1$, the divergence in~(\ref{DPD}) reduces to the squared $L_2$ distance. It is easy and straightforward to check that the definition in~(\ref{DPD}) satisfies the condition in~(\ref{prop}).




Another common example of a related divergence class is the {\it logarithmic density power divergence} (LDPD) family of Jones et al. (2001) defined as  
\begin{equation}
 \label{defnldpd}
 \text{LDPD}_{\alpha} (g,f):= \log \Big(\int_{\mathbb{R}} f^{\alpha+1} \Big) - \Big( 1+\dfrac{1}{\alpha} \Big) \log \Big(\int_{\mathbb{R}} f^{\alpha}g \Big)+ \dfrac{1}{\alpha} \log \Big( \int_{\mathbb{R}} g^{\alpha+1} \Big), \;\; {\rm for~all} \; \; g, f \in \mathcal{L}_{\mu}, \alpha \geq 0.
 \end{equation} 
Its structural similarity with the DPD family is immediately apparent. It is obtained by replacing the identity function on each component of the integral by the log function. This family is also known to produce highly robust estimators with good efficiency. Fujisawa and Eguchi (2008) and Fujisawa (2013), in fact, argue that the minimum divergence estimators based on the LDPD are more successful in minimizing the bias of the estimator under heavy contamination in comparison to the minimum DPD estimators. However, also see Kuchibhotla et al. (2018) for some counter views. 

 
\section{The Functional Density Power Divergence}

Further exploration of the divergences within the DPD family leads to the observation that this class of divergences may be extended to a more general family of divergences, called the \textit{functional density power divergence family} having the form 
\begin{equation}
\label{defn2}
\text{FDPD}_{\phi,\alpha}(g,f) := \phi \Big(\int_{\mathbb{R}} f^{\alpha+1} \Big) - \Big( 1+\dfrac{1}{\alpha} \Big) \phi \Big(\int_{\mathbb{R}} f^{\alpha}g \Big)+ \dfrac{1}{\alpha} \phi \Big( \int_{\mathbb{R}} g^{\alpha+1} \Big), \;\; {\rm for~all} \; \; g, f \in \mathcal{L}_{\mu},
\end{equation} 
where $\phi : [0, \infty) \longrightarrow [-\infty, \infty]$ 
is a pre-assigned function, $\alpha$ is a non-negative real number and $\text{FDPD}_{\phi, \alpha}$ is a divergence in the sense of (\ref{prop}). Note that the expression given in~(\ref{defn2}) need not necessarily define a divergence for all $\phi$ as it does not always satisfy the condition stated 
in~(\ref{prop}). In the following we will identify the class of functions $\phi$ for which the quantity defined in (\ref{defn2}) is actually a divergence, thus providing a characterization of the FDPD class.  

Within the FDPD class, the case $\alpha=0$ has to be again understood in a limiting sense and this limiting divergence exists under some constraints on the function $\phi$. For example, if we assume $\phi$ is continuously differentiable on $[0,\infty)$, then the divergence for $\alpha=0$ can be defined as
\begin{equation}
\label{zero case}
\text{FDPD}_{\phi,\alpha=0} (g,f) := \phi^{\prime}(1) \int_{\mathbb{R}} g \log \dfrac{g}{f}, \;\; {\rm for~all} \; \; g,f \in \mathcal{L}_{\mu},
\end{equation}
where $\phi^{\prime}$ is the derivative of $\phi$. Obviously we require $\phi^{\prime}(1)$ to be positive for the above to be a divergence. Note that, the divergence in~(\ref{zero case}) is actually the likelihood disparity with a different scaling constant and therefore the divergences $\text{FDPD}_{\phi, \alpha=0}$ are effectively equivalent to the likelihood disparity for inferential purposes. For the DPD and the LDPD, in fact, the scaling constant $\phi^{\prime}(1)$ equals unity. The characterization of the FDPD, therefore, is not an interesting problem for $\alpha = 0$; 
hence, we will not concern ourselves with the $\alpha=0$ case in the following.

\begin{remark}
Suppose, $\phi$ is a strictly increasing and convex function on the non-negative real line. Then it is straightforward to check that the expression defined in~(\ref{defn2}) does indeed satisfy the condition in~(\ref{prop}) and therefore defines a legitimate divergence which belongs to the FDPD class. Note that $\phi^{\prime}(1)$ is necessarily positive in this case. The identity function which relates to the DPD family belongs to this class of $\phi$ functions.

\end{remark}


\begin{remark}
That the class of  functions described in the previous remark does not completely characterize the FDPDs can be seen by choosing  $\phi(x) = \log x,$ for all $x \geq 0$, with the convention that $\log 0 := -\infty$. In this case $\phi$ is a concave function but the corresponding FDPD  also satisfies~(\ref{prop}) and gives rise to the logarithmic density power divergence  (LDPD) family already introduced in Section 2. In this case $\phi^{\prime}(1) = 1$.
\end{remark}


We expect that the members of the FDPD family will possess useful robustness properties which could make it interesting to examine these divergences and therefore it is natural to ask whether we can characterize all the functions $\phi$ which give rise to a divergence and thus can obtain a complete description of the FDPD family. As already indicated, the main objective of this article is to discuss this characterization under certain standard assumptions.


\section{Characterization of the FDPD family}
In this section we will assume that the dominating measure $\mu$ is actually the Lebesgue measure on the real line and therefore the FDPD is a family of divergences on the space of probability density functions.

Our first result states a general sufficient condition on the function $\phi$ which will guarantee that $\text{FDPD}_{\phi,\alpha}$ is a valid divergence for all $\alpha > 0.$

\begin{proposition}
\label{propn1}
Suppose $\phi :[0,\infty) \longrightarrow [-\infty,\infty]$ is a function such that the function $\psi : [-\infty, \infty) \longrightarrow [-\infty,\infty]$ defined as $\psi (x) := \phi(e^x), \;{\rm for~all}~ x \in [-\infty, \infty)$ is convex and strictly increasing on its domain. Then $\text{FDPD}_{\phi,\alpha}$ is a valid divergence for each fixed $\alpha > 0$, according to the definition in~(\ref{defn2}).
\end{proposition}

\begin{proof}
In order to show that $\text{FDPD}_{\phi,\alpha}$ is a valid divergence, we have to show that $\text{FDPD}_{\phi,\alpha} (g,f) \geq 0, \;\; {\rm for~all} \; g,f \in \mathcal{L}_{\mu},$ and equality holds if and only if $g=f, \; a.e.[\mu].$ For $\alpha > 0$, using the convexity of the function $\psi$ we can conclude that 
\begin{eqnarray}
\alpha \phi \Big(\int_{\mathbb{R}} f^{\alpha+1} \Big) +  \phi \Big( \int_{\mathbb{R}} g^{\alpha+1} \Big) &=& \alpha \psi \Big(\log \int_{\mathbb{R}} f^{\alpha+1} \Big)+  \psi \Big(\log \int_{\mathbb{R}} g^{\alpha+1} \Big) \nonumber\\
\label{convexstep} & \geq & (1+\alpha) \psi \Big( \dfrac{\alpha}{1+\alpha} \log \int_{\mathbb{R}} f^{\alpha+1} + \dfrac{1}{1+\alpha}  \log \int_{\mathbb{R}} g^{\alpha+1} \Big).
\end{eqnarray}
On the other hand, for $\alpha >0$, using \textit{H\"{o}lder's inequality} on the functions $f^{\alpha}$ and $g$ with dual indices $\Big( \dfrac{1+\alpha}{\alpha}, 1+\alpha \Big)$ we obtain,
\begin{equation}
\label{holder}
  {\big( \int_{\mathbb{R}} f^{\alpha+1}\big)}^{\frac{\alpha}{1+\alpha}} {\big( \int_{\mathbb{R}} g^{\alpha+1}\big)}^{\frac{1}{1+\alpha}} \geq \int_{\mathbb{R}} f^{\alpha}g,
\end{equation}
which is equivalent to
\begin{equation}
\label{holder2}
\dfrac{\alpha}{1+\alpha} \log \int_{\mathbb{R}} f^{\alpha+1} + \dfrac{1}{1+\alpha}  \log \int_{\mathbb{R}} g^{\alpha+1} \geq \log \int_{\mathbb{R}} f^{\alpha}g.
\end{equation}
Expression~(\ref{convexstep}) and~(\ref{holder2}) along with the strict monotonicity of $\psi$ imply that,
\begin{eqnarray}
\label{inc}
\alpha \phi \Big(\int_{\mathbb{R}} f^{\alpha+1} \Big) +  \phi \Big( \int_{\mathbb{R}} g^{\alpha+1} \Big) & \geq & (1+\alpha) \psi \Big( \dfrac{\alpha}{1+\alpha} \log \int_{\mathbb{R}} f^{\alpha+1} + \dfrac{1}{1+\alpha}  \log \int_{\mathbb{R}} g^{\alpha+1} \Big) \nonumber \\
\label{exp}& \geq & (1+\alpha) \psi \Big( \log \int_{\mathbb{R}} f^{\alpha}g \Big)
  =  (1+\alpha) \phi \Big( \int_{\mathbb{R}} f^{\alpha}g \Big),
\end{eqnarray}
which is equivalent to the statement that $\text{FDPD}_{\phi,\alpha}(g,f) \geq 0$. For the equality $\text{FDPD}_{\phi,\alpha}(g,f)=0$ to hold, we must have equality in~(\ref{convexstep}) and~(\ref{inc}). By strict monotonicity of $\psi$, the equality in~(\ref{inc}) implies equality in~(\ref{holder}) which will happen only if $f^{1+\alpha}=g^{1+\alpha}, \; a.e.[\mu]$, which is equivalent to $f=g, a.e.[\mu]$. On the other hand, if $f=g,\; a.e.[\mu]$, then clearly $\text{FDPD}_{\phi,\alpha}(g,f)=0$ by~(\ref{defn2}). This completes our proof.
\end{proof} 
 Now we shall show that the condition on $\phi$ stated in Proposition~\ref{propn1} is indeed a necessary condition for generating a divergence family under some mild smoothness assumptions on $\phi$.


\begin{proposition}
\label{propn2}
Suppose $\phi : [0,\infty) \longrightarrow [-\infty, \infty]$ is a function which is continuous on $[0,\infty)$ and twice continuously differentiable on $(0, \infty)$ with finite second order derivative on $(0, \infty)$. Moreover, suppose that $\text{FDPD}_{\phi,\alpha}$ is a valid divergence for all $\alpha > 0$. Then the function $\psi : [-\infty, \infty) \longrightarrow [-\infty,\infty]$ defined as $\psi (x) := \phi(e^x), \;{\rm for~all}~ x \in [-\infty, \infty)$ is convex and strictly increasing on its domain.  
\end{proposition}

\begin{proof}
We shall use the idea of computing the divergence between two uniform probability density functions and extracting the property of the function $\phi$ from it. First take $\theta, \tau >0$ and 
$$ f(x) = \dfrac{1}{\theta}I_A(x), \;\; g(x) = \dfrac{1}{\tau}I_B(x),$$
where $I_A(x)$ denotes the indicator function of the set $A$, and the indicated sets $A$ and $B$ are defined as $A = [0, \theta]$ and $B = [0, \tau]$.  Easy computations show that
\begin{equation}
\label{values}
\int_{\mathbb{R}} f^{1+\alpha} = \theta^{-\alpha}, \;\; \int_{\mathbb{R}} g^{1+\alpha} = \tau^{-\alpha}, \;\; \int_{\mathbb{R}} f^{\alpha}g = \begin{cases}
 \theta^{-\alpha}, \;\; \text{if } \theta > \tau, \\
 \theta^{1-\alpha}\tau^{-1}, \;\; \text{if } \theta \leq \tau.
\end{cases}
\end{equation}
Therefore, the property that $\text{FDPD}_{\phi,\alpha}(g,f) \geq 0$ along with equality if and only if $g=f, a.e.[\mu]$ yields that for all $\alpha>0$,
\begin{equation}
\label{exp2}
\phi(\theta^{-\alpha}) - \Big(1+\dfrac{1}{\alpha}\Big) \phi( \theta^{-\alpha}) + \dfrac{1}{\alpha} \phi(\tau^{-\alpha}) > 0, \;\; {\rm for~all} \; \theta >\tau >0,
\end{equation}
and
\begin{equation}
\label{exp3}
\phi(\theta^{-\alpha}) - \Big(1+\dfrac{1}{\alpha}\Big) \phi(\theta^{1-\alpha}\tau^{-1} ) + \dfrac{1}{\alpha} \phi(\tau^{-\alpha}) > 0, \;\; {\rm for~all} \; \tau >\theta >0.
\end{equation}
Equation~(\ref{exp2}) simplifies to give $\phi(\theta^{-\alpha}) < \phi (\tau^{-\alpha})$ for all $\theta > \tau >0$, which helps us to conclude that $\phi$ is strictly increasing on $(0,\infty)$. Continuity of $\phi$ at $0$ guarantees that $\phi$ is basically strictly increasing on $[0,\infty)$ which proves the fact that $\psi$ is strictly increasing on $[-\infty,\infty)$.

Now fix any $x,y \in \mathbb{R}$ such that $x>y$. Put $\theta = \exp(-\frac{x}{\alpha})$ and $\tau = \exp(-\frac{y}{\alpha})$ in Equation~(\ref{exp3}). Notice that $x>y$ will guarantee that $\theta< \tau$. Therefore we get
\begin{equation}
\label{exp4}
\phi(e^x) + \dfrac{1}{\alpha}\phi(e^y) > \Big(1+\dfrac{1}{\alpha}\Big) \phi \Big(\exp \Big(\dfrac{(\alpha-1)x+y}{\alpha}\Big)\Big), \;\; {\rm for~all} \; x>y \in \mathbb{R}, \;\; {\rm for~all} \; \alpha >0.
\end{equation}
Expression~(\ref{exp4}) on simplification yields
\begin{equation}
\label{exp5}
\dfrac{\alpha}{1+\alpha}\psi(x) + \dfrac{1}{1+\alpha}\psi(y) >  \psi  \Big(\dfrac{(\alpha-1)x+y}{\alpha}\Big), 
\end{equation} 
for all $x > y \in {\mathbb R}$, and for all $\alpha > 0$. Rewriting $\lambda = \frac{\alpha}{1 + \alpha}$, we conclude
\begin{equation}
\label{exp6}
\lambda\psi(x) + (1-\lambda)\psi(y) >  \psi  \Big( \dfrac{2\lambda-1}{\lambda}x + \dfrac{1-\lambda}{\lambda}y \Big),
\end{equation}
for all $x > y \in {\mathbb R}$ and for all $\lambda \in (0, 1)$. Now take real numbers $x_1>x_2>x_3$ and put $x=x_1,y=x_3$ and $\lambda = \frac{x_1-x_3}{2x_1-x_2-x_3} \in (0,1)$ in~(\ref{exp6}). Then an easy computation shows that
\begin{equation}
\label{exp7}
\dfrac{x_1-x_3}{2x_1-x_2-x_3}\psi(x_1) + \dfrac{x_1-x_2}{2x_1-x_2-x_3}\psi(x_3) > \psi (x_2),
\end{equation}
for all $x_1,x_2,x_3 \in \mathbb{R}$, $x_1>x_2>x_3$. Simplification of the expression in  (\ref{exp7}) leads to the relation
\begin{equation}
\label{exp8}
\dfrac{\psi(x_1)-\psi(x_2)}{x_1-x_2} > \dfrac{\psi(x_2)-\psi(x_3)}{x_1-x_3}, 
\end{equation}
for all $x_1,x_2,x_3 \in \mathbb{R}$, $x_1>x_2>x_3$. Now we shall use the differentiability assumption on $\phi$. Since $\phi$ is twice continuously differentiable with finite second order derivative on $(0,\infty)$, it implies that $\psi$ is also twice continuously differentiable on $\mathbb{R}$ with finite second order derivative. Taking $x_1 \downarrow x_2$ in~(\ref{exp8}), we conclude that
\begin{equation}
\label{exp9}
\psi^{\prime}(x_2) \geq \dfrac{\psi(x_2)-\psi(x_3)}{x_2-x_3} \;\; {\rm for~all} \; x_2,x_3 \in \mathbb{R}, \; x_2>x_3.
\end{equation}
To show $\psi$ is convex on $\mathbb{R}$ it is enough to show that $\psi^{\prime \prime}(x) \geq 0$ for all $x \in \mathbb{R}$. Suppose, if possible, there exists a real number 
$x_0$ such that $\psi^{\prime \prime}(x_0) < 0$. By continuity of $\psi^{\prime \prime}$ we can get $a<x_0<b$ such that $\psi^{\prime \prime} (x) <0$ for all $x \in [a,b]$. Then by~(\ref{exp9}) we have
\begin{equation}
\label{exp10}
\psi^{\prime}(b) \geq \dfrac{\psi(b)-\psi(a)}{b-a}=\psi^{\prime}(c), \;\; \text{for some } c \in (a,b).
\end{equation}
But $\psi^{\prime \prime}<0$ on $[a,b]$ guarantees that $\psi^{\prime}(c) > \psi^{\prime}(b)$ as $c<b$. This gives us a contradiction. Hence, $\psi$ is convex on $\mathbb{R}$. Monotonicity of $\psi$ now guarantees that $\psi$ is convex on $[-\infty,\infty)$. This completes the proof.
\end{proof}

\begin{remark}
Note that if $\phi$ is convex and strictly increasing then $\psi$ defined as in Proposition~(\ref{propn1}) is strictly convex and strictly increasing. On the other hand, if $\phi=\log$, then $\psi$ is basically the identity function and therefore convex and strictly increasing.
\end{remark}


Propositions 1 and 2 provide the complete characterization of the FDPD family and class of $\phi$ functions generating them. We trust that this characterization describes the class within which one can search for suitable minimum divergence procedures exhibiting good balance between model efficiency and robustness. 

