Skip to main content
added 1858 characters in body
Source Link
Iosif Pinelis
  • 127.8k
  • 8
  • 107
  • 229

First of all here, your $D_{\alpha,\beta}(f||g)$ is not properly defined. It is not a function of $f$ and $g$ (for given $\alpha$ and $\beta$), as it also depends on the joint distribution of $X$ and $Y$. Also, you need to ensure that $P(Y>\beta)>0$ and $P(Y\le\beta)>0$ ($\beta$ being in the support of $Y$ would not be enough).

Further, at least one of the probabilities $P(\beta<X\le b)$ or $P(b<X\le\beta)$ equals $0$, depending on whether $\beta>b$ or $\beta\le b$. So, at least one of the conditional probabilities $P(X\le b|X>\beta)$ or $P(X>b|X\le\beta)$ equals $0$, provided that both are defined. So, if $0<\alpha<1$ and $Y=X$, then the right-hand side of (1) never equals $0$, which violates the second one of the two conditions of being a divergence that you referred to.

If now $\alpha=1$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X\le b)=1$ and $P(X>\beta)>0$. Similarly, if $\alpha=0$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X>b)=1$ and $P(X\le\beta)>0$.

To conclude, in no way is $D_{\alpha,\beta}(f||g)$ a divergence.

Addendum: It appears that you have made two changes: (i) added the conditions $P(Y>\beta)>0$ and $P(Y\le\beta)>0$ to ensure that the conditional probabilities $P(X\le b|X>\beta)$ and $P(X>b|X\le\beta)$ are defined and (ii) specialized $b$ in these conditional probabilities to $\beta$. Of course, this specialization does not help: since your (non-)function $D_{\alpha,\beta}(\cdot|\cdot)$ was not a divergence in the more general setting, it is "even less so" after the specialization. Indeed, if $Y=X$, then the right-hand side of (1) is now always $1\ne0$, which violates, for all $\alpha\in[0,1]$, the second one of the two conditions of being a divergence that you referred to. In fact, your candidate for a divergence will take the maximum possible value ($1$) precisely when it was supposed to take the smallest possible value ($0$).

Also, the dependence of your $D_{\alpha,\beta}(f||g)$ on the joint distribution of $X$ and $Y$ is not just a matter of "notational simplicity" or lack thereof. Indeed, according to the definition you referred to, a divergence must be a function of the individual distributions of $X$ and $Y$, and your $D_{\alpha,\beta}(f||g)$ is not such a function.

What I also showed is that, even if we overlook this "function of the individual distributions" requirement and allow a divergence in a broader sense to be a function of the random variables $X$ and $Y$ themselves (or of their joint distribution), then your $D_{\alpha,\beta}(f||g)$ still will not be a divergence, even in that broader sense -- because it will be nonzero when $Y=X$.

A question to ask here is this: even if this were a divergence, why would it be of interest? Cf. e.g. the Kullback--Leibler divergence, which arises naturally in statistics and information theory. Does your $D_{\alpha,\beta}(f||g)$ arise in a natural context?

First of all here, your $D_{\alpha,\beta}(f||g)$ is not properly defined. It is not a function of $f$ and $g$ (for given $\alpha$ and $\beta$), as it also depends on the joint distribution of $X$ and $Y$. Also, you need to ensure that $P(Y>\beta)>0$ and $P(Y\le\beta)>0$ ($\beta$ being in the support of $Y$ would not be enough).

Further, at least one of the probabilities $P(\beta<X\le b)$ or $P(b<X\le\beta)$ equals $0$, depending on whether $\beta>b$ or $\beta\le b$. So, at least one of the conditional probabilities $P(X\le b|X>\beta)$ or $P(X>b|X\le\beta)$ equals $0$, provided that both are defined. So, if $0<\alpha<1$ and $Y=X$, then the right-hand side of (1) never equals $0$, which violates the second one of the two conditions of being a divergence that you referred to.

If now $\alpha=1$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X\le b)=1$ and $P(X>\beta)>0$. Similarly, if $\alpha=0$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X>b)=1$ and $P(X\le\beta)>0$.

To conclude, in no way is $D_{\alpha,\beta}(f||g)$ a divergence.

First of all here, your $D_{\alpha,\beta}(f||g)$ is not properly defined. It is not a function of $f$ and $g$ (for given $\alpha$ and $\beta$), as it also depends on the joint distribution of $X$ and $Y$. Also, you need to ensure that $P(Y>\beta)>0$ and $P(Y\le\beta)>0$ ($\beta$ being in the support of $Y$ would not be enough).

Further, at least one of the probabilities $P(\beta<X\le b)$ or $P(b<X\le\beta)$ equals $0$, depending on whether $\beta>b$ or $\beta\le b$. So, at least one of the conditional probabilities $P(X\le b|X>\beta)$ or $P(X>b|X\le\beta)$ equals $0$, provided that both are defined. So, if $0<\alpha<1$ and $Y=X$, then the right-hand side of (1) never equals $0$, which violates the second one of the two conditions of being a divergence that you referred to.

If now $\alpha=1$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X\le b)=1$ and $P(X>\beta)>0$. Similarly, if $\alpha=0$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X>b)=1$ and $P(X\le\beta)>0$.

To conclude, in no way is $D_{\alpha,\beta}(f||g)$ a divergence.

Addendum: It appears that you have made two changes: (i) added the conditions $P(Y>\beta)>0$ and $P(Y\le\beta)>0$ to ensure that the conditional probabilities $P(X\le b|X>\beta)$ and $P(X>b|X\le\beta)$ are defined and (ii) specialized $b$ in these conditional probabilities to $\beta$. Of course, this specialization does not help: since your (non-)function $D_{\alpha,\beta}(\cdot|\cdot)$ was not a divergence in the more general setting, it is "even less so" after the specialization. Indeed, if $Y=X$, then the right-hand side of (1) is now always $1\ne0$, which violates, for all $\alpha\in[0,1]$, the second one of the two conditions of being a divergence that you referred to. In fact, your candidate for a divergence will take the maximum possible value ($1$) precisely when it was supposed to take the smallest possible value ($0$).

Also, the dependence of your $D_{\alpha,\beta}(f||g)$ on the joint distribution of $X$ and $Y$ is not just a matter of "notational simplicity" or lack thereof. Indeed, according to the definition you referred to, a divergence must be a function of the individual distributions of $X$ and $Y$, and your $D_{\alpha,\beta}(f||g)$ is not such a function.

What I also showed is that, even if we overlook this "function of the individual distributions" requirement and allow a divergence in a broader sense to be a function of the random variables $X$ and $Y$ themselves (or of their joint distribution), then your $D_{\alpha,\beta}(f||g)$ still will not be a divergence, even in that broader sense -- because it will be nonzero when $Y=X$.

A question to ask here is this: even if this were a divergence, why would it be of interest? Cf. e.g. the Kullback--Leibler divergence, which arises naturally in statistics and information theory. Does your $D_{\alpha,\beta}(f||g)$ arise in a natural context?

Source Link
Iosif Pinelis
  • 127.8k
  • 8
  • 107
  • 229

First of all here, your $D_{\alpha,\beta}(f||g)$ is not properly defined. It is not a function of $f$ and $g$ (for given $\alpha$ and $\beta$), as it also depends on the joint distribution of $X$ and $Y$. Also, you need to ensure that $P(Y>\beta)>0$ and $P(Y\le\beta)>0$ ($\beta$ being in the support of $Y$ would not be enough).

Further, at least one of the probabilities $P(\beta<X\le b)$ or $P(b<X\le\beta)$ equals $0$, depending on whether $\beta>b$ or $\beta\le b$. So, at least one of the conditional probabilities $P(X\le b|X>\beta)$ or $P(X>b|X\le\beta)$ equals $0$, provided that both are defined. So, if $0<\alpha<1$ and $Y=X$, then the right-hand side of (1) never equals $0$, which violates the second one of the two conditions of being a divergence that you referred to.

If now $\alpha=1$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X\le b)=1$ and $P(X>\beta)>0$. Similarly, if $\alpha=0$ and $Y=X$, then the right-hand side of (1) equals $0$ only if $P(X>b)=1$ and $P(X\le\beta)>0$.

To conclude, in no way is $D_{\alpha,\beta}(f||g)$ a divergence.