-1
$\begingroup$

Let $X$ and $Y$ be two continuous random variables with common support and with PDF $f(x)$ and $g(y)$. For any $0 \leq \alpha \leq 1$, and any constant $\beta$ within the support of $X$ and $Y$ such that $\Pr [Y>\beta ] >0$ and that $\Pr[Y\leq \beta ] > 0$, we define \begin{equation} D_{\alpha,\beta}(f \| g) = 1 - \alpha \Pr[X \leq \beta \mid Y>\beta ] - ( 1- \alpha) \Pr[X > \beta \mid Y\leq\beta ] \:, \tag{1}\end{equation} as a measure of distance of $f$ from $g$ at point $\beta$. Two questions:

1) Is $D_{\alpha,\beta}(f \| g)$ a divergence of distribution $f$ from $g$ at point $\beta$?

2) Is there any relationship between (1) and the well-known distance (or divergence) measures?

Special case of $\alpha = \Pr[Y>\beta]$ is important. Moreover, for notational simplicity, I did not mention that $D_{\alpha,\beta}(f \| g)$ also depends on the joint distribution of $X$ and $Y$.

*Definition of statistical divergence: https://en.wikipedia.org/wiki/Divergence_(statistics)

*Definition of statistical distance: https://en.wikipedia.org/wiki/Statistical_distance

$\endgroup$
2
  • $\begingroup$ define "a divergence of distribution", is it a seminorm? something weaker/stronger? $\endgroup$
    – kodlu
    Commented Nov 30, 2015 at 22:47
  • $\begingroup$ @kodlu, added into question. $\endgroup$
    – Jeff
    Commented Dec 1, 2015 at 12:27

1 Answer 1

2
$\begingroup$

First of all here, your $D_{\alpha,\beta}(f||g)$ is not properly defined. It is not a function of $f$ and $g$ (for given $\alpha$ and $\beta$), as it also depends on the joint distribution of $X$ and $Y$. Also, you need to ensure that $P(Y>\beta)>0$ and $P(Y\le\beta)>0$ ($\beta$ being in the support of $Y$ would not be enough).

Further, at least one of the probabilities $P(\beta<X\le b)$ or $P(b<X\le\beta)$ equals $0$, depending on whether $\beta>b$ or $\beta\le b$. So, at least one of the conditional probabilities $P(X\le b|X>\beta)$ or $P(X>b|X\le\beta)$ equals $0$, provided that both are defined. So, if $0<\alpha<1$ and $Y=X$, then the right-hand side of (1) never equals $0$, which violates the second one of the two conditions of being a divergence that you referred to.

If now $\alpha=1$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X\le b)=1$ and $P(X>\beta)>0$. Similarly, if $\alpha=0$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X>b)=1$ and $P(X\le\beta)>0$.

To conclude, in no way is $D_{\alpha,\beta}(f||g)$ a divergence.

Addendum: It appears that you have made two changes: (i) added the conditions $P(Y>\beta)>0$ and $P(Y\le\beta)>0$ to ensure that the conditional probabilities $P(X\le b|X>\beta)$ and $P(X>b|X\le\beta)$ are defined and (ii) specialized $b$ in these conditional probabilities to $\beta$. Of course, this specialization does not help: since your (non-)function $D_{\alpha,\beta}(\cdot|\cdot)$ was not a divergence in the more general setting, it is "even less so" after the specialization. Indeed, if $Y=X$, then the right-hand side of (1) is now always $1\ne0$, which violates, for all $\alpha\in[0,1]$, the second one of the two conditions of being a divergence that you referred to. In fact, your candidate for a divergence will take the maximum possible value ($1$) precisely when it was supposed to take the smallest possible value ($0$).

Also, the dependence of your $D_{\alpha,\beta}(f||g)$ on the joint distribution of $X$ and $Y$ is not just a matter of "notational simplicity" or lack thereof. Indeed, according to the definition you referred to, a divergence must be a function of the individual distributions of $X$ and $Y$, and your $D_{\alpha,\beta}(f||g)$ is not such a function.

What I also showed is that, even if we overlook this "function of the individual distributions" requirement and allow a divergence in a broader sense to be a function of the random variables $X$ and $Y$ themselves (or of their joint distribution), then your $D_{\alpha,\beta}(f||g)$ still will not be a divergence, even in that broader sense -- because it will be nonzero when $Y=X$.

A question to ask here is this: even if this were a divergence, why would it be of interest? Cf. e.g. the Kullback--Leibler divergence, which arises naturally in statistics and information theory. Does your $D_{\alpha,\beta}(f||g)$ arise in a natural context?

$\endgroup$
3
  • $\begingroup$ Thanks @IosifPinelis for your notes. I have revised the question, and you may revise your answer accordingly. $\endgroup$
    – Jeff
    Commented Dec 2, 2015 at 3:50
  • $\begingroup$ I have added an addendum regarding the edited version of the question. $\endgroup$ Commented Dec 2, 2015 at 16:39
  • $\begingroup$ You are right that this function works in opposite of "divergence". Actually, it shows similarity of distributions of $X$ and $Y$. So, we may redefine the divergence as $1-D_{\alpha,\beta}(f\|g)$. The main problem of the KL divergence is that it does not monotonically map the similarity of $f$ and $g$ to some real value. That is, $f_1$ might be more similar to $g$ than $f_2$ (with point-wise comparison), but has higher KL divergence than $f_2$. However, (1) offers a monotonic mapping, at least in my simulations. Possible application would be in the detection theory, among others. $\endgroup$
    – Jeff
    Commented Dec 2, 2015 at 21:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .