Let $\mathscr P _0$ and $\mathscr P _1$ be two non-overlapping sets of probability distributions defined on $(\Omega,\mathcal{A})$. Consider the distance defined as $$D_u(P_0,P_1)=\int_\Omega \left(\frac{p_1}{p_0}\right)^u p_0 \mathrm{d}\mu<\infty.$$ Two distributions are chosen from each set $Q_0\in\mathscr P _0$ and $Q_1\in\mathscr P _1$ such that $$D_u(Q_0,Q_1)\geq D_u(P_0,P_1)\quad \forall (P_0,P_1)\in \mathscr P _0\times \mathscr P _1,\forall u\in[0,1]$$

Is it true that (A:)$$\int \min(q_0,t q_1)\mathrm{d}\mu\geq \int \min(p_0,t p_1)\mathrm{d}\mu\quad \forall (P_0,P_1)\in \mathscr P _0\times \mathscr P _1,\forall t$$or equaivalently (B:) $$Q_0\left[\frac{q_1}{q_0}>t\right]\geq P_0\left[\frac{q_1}{q_0}>t\right]\quad\ \forall P_0\in \mathscr P _0,\forall t$$ $$Q_1\left[\frac{q_1}{q_0}\leq t\right]\geq P_1\left[\frac{q_1}{q_0}\leq t\right]\quad\ \forall P_1\in \mathscr P _1,\forall t$$


$\bullet$ One can consider A or B since both conditions are equivalent.

$\bullet$ $p_0$ and $p_1$ are densities of $P_0$ and $P_1$ and the same goes to $q_0$ and $q_1$ with $Q_0$ and $Q_1$.

What I know:

From Huber's paper (pages 260-261) Theorem 6.1 I know that if the distance is the $f$-divergence, i.e. $D_f$, then A and B are correct. Additionally, if A and B are correct, then $Q_0$ and $Q_1$ minimize $D_f$ (iff condition).

Huber considers $$Q_{jt}=(1-t)Q_{0t}+t Q_{1t}\\q_{jt}=(1-t)q_{0t}+t q_{1t}$$ and finds the first and second derivatives of $D_f(Q_{0t},Q_{1t})$. He then shows that the second derivative is $\geq 0$ (convex) and hence $(Q_{00},Q_{10})$ minimizes $D_f$ if and only if the first derivative evaluated at $t=0$ is $\geq 0$ for all $(Q_{01},Q_{11})\in(\mathscr P _0\times\mathscr P _1)$. He shows that this is really the case, hence the claim is true.

I think that this result can be strenghtened, i.e. if $(Q_0,Q_1)$ maximizes $D_u$ for all $u\in[0,1]$, then it should satisfy A or equivalently B. I dont know how to proceed.

Addendum: It seems that the question eventually boils down to finding $(Q_0,Q_1)$ which maximizes $D_u$ for all $u\in[0,1]$ and fails to minimize $D_f$ for at least one $f$. This will be a counterexample to the claim (of course if there exists such a pair).

  • $\begingroup$ Are you trying to replace f-divergence with any divergence $D_u$ or there is sth more? $\endgroup$ – Henry.L Apr 21 '17 at 16:31
  • $\begingroup$ @Henry.L I am trying to replace $D_f$ with $D_u$. For $\Longrightarrow$ there is no problem. Because if $Q_0$ and $Q_1$ satisfy stochastic ordering, then they also minimize all f-divergences. $D_u$ is just a sub-set of f-divergences with $f=-x^u$ with an exception that $f(1)\neq 0$. I am interested in $\Longleftarrow$, For this case if $Q_0$ and $Q_1$ minimize all f-divergences, I know that they also satisfy stochastic ordering condition. $\endgroup$ – Seyhmus Güngören Apr 21 '17 at 18:43
  • $\begingroup$ @Henry.L Now just replace $D_f$ by $D_u$. Will we have the same conclusion? namely if $Q_0$ and $Q_1$ maximize $D_u$ for all $u\in[0,1]$, will $Q_0$ and $Q_1$ satisfy A or B as given in the question? $\endgroup$ – Seyhmus Güngören Apr 21 '17 at 18:43
  • $\begingroup$ I am kind of skeptical but cannot come up with a counter-example at the moment, will try to think of it... $\endgroup$ – Henry.L Apr 21 '17 at 22:01
  • $\begingroup$ @Henry.L thank you very much for your interest. I will be happy to see a counterexample, if there exists one;) $\endgroup$ – Seyhmus Güngören Apr 21 '17 at 22:05

Answer: this does NOT extends to other cases than f-divergence.

For the case $u=0$, there are many counter examples since $D_u$ is not separating any densities. So I think it suffices to consider $u\in (0,1]$.

Let $U^n=Unif(0,\frac{1}{n})$ the uniform measure on unit interval. And $\mathscr{P}_0$ corresponds to those with odd $n$, $\mathscr{P}_1$ contains exactly those with even $n$. Let $Q_0=U^n,Q_1=U^m$, thus $D_u(Q_0,Q_1)< \infty$ if $n<m$; and $=\infty$ otherwise due to the fact that $$\frac{q_{1}}{q_{0}}=\frac{m\boldsymbol{1}_{[0,\frac{1}{m}]}(\omega)}{n\boldsymbol{1}_{[0,\frac{1}{n}]}(\omega)}=\begin{cases} \frac{m}{n}>1 & \omega\in[0,\frac{1}{m}]\\ 0 & \omega\in[\frac{1}{m},\frac{1}{n}]\\ \infty & \omega\in[\frac{1}{n},1] \end{cases},n<m$$.(It suffices to consider the integration over the last segment w.r.t. $Q_0$'s density $q_0$.)

Therefore $D_u(Q_0,Q_1)\geq D_u(P_0,P_1)\quad \forall (P_0,P_1)\in \mathscr P _0\times \mathscr P _1,\forall u\in[0,1]$ as long as $Q_0=U^n,Q_1=U^m$ satisfying $n\geq m$. Now for any pair $Q_0=U^n,Q_1=U^m\text{ such that },n\geq m$, $$\frac{q_{1}}{q_{0}}=\frac{m\boldsymbol{1}_{[0,\frac{1}{m}]}(\omega)}{n\boldsymbol{1}_{[0,\frac{1}{n}]}(\omega)}=\begin{cases} \frac{m}{n}\leq1 & \omega\in[0,\frac{1}{n}]\\ \infty & \omega\in[\frac{1}{n},1] \end{cases},n\geq m$$ Let $t=1$ now and$ \left\{ \omega\in[0,1]\mid\frac{q_{1}}{q_{0}}>t\right\} =[\frac{1}{n},1]$ $$Q_{1}\left[\frac{q_{1}}{q_{0}}>t\right]=m\cdot\mu\left[[\frac{1}{n},1]\cap[0,\frac{1}{m}]\right]=m\cdot\mu\left[\frac{1}{n},\frac{1}{m}\right]=m[{\frac{1}{m}-\frac{1}{n}}]>0$$

Choose $P_{1}=U^{k}$ for any $k\geq n$ $$P_{1}\left[\frac{q_{1}}{q_{0}}>t\right]=k\cdot\mu\left[[\frac{1}{n},1]\cap[0,\frac{1}{k}]\right]=0$$

  • $\begingroup$ I havent read completely yet but $\mathscr P_0$ and $\mathscr P_1$ are non-overlapping sets so $\mathscr P_0\neq \mathscr P_1$. $\endgroup$ – Seyhmus Güngören Apr 21 '17 at 23:03
  • $\begingroup$ @SeyhmusGüngören I correct it now, mindless typo. $\endgroup$ – Henry.L Apr 21 '17 at 23:06
  • $\begingroup$ Thank you very much once again. This is a nice counterexample. I have a few comments. Choosing $P_1=U^k$ for any $k>n$ as $k$ is even and $n$ is odd. "the uniform measures on unit interval" probably better and if $\mu$ is the lebesgue measure wouldnt we get $[1/n,1/m]=1/m-1/n$? why is it at the denominator? and one last question: for which f-divergence wouldnt this example work? I mean it would fail to satisfy the distance condition. $\endgroup$ – Seyhmus Güngören Apr 22 '17 at 1:58
  • $\begingroup$ @SeyhmusGüngören (1)Yes (2)When I edited the post I did not realize I put it in denominator and the error diffuse.., but that does not change the nature of counter example (3)That is worth another question... $\endgroup$ – Henry.L Apr 22 '17 at 2:06
  • $\begingroup$ "Sure" from (1) to (3) $\endgroup$ – Seyhmus Güngören Apr 22 '17 at 2:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.