2
$\begingroup$

Let $\mathscr P _0$ and $\mathscr P _1$ be two non-overlapping sets of probability distributions defined on $(\Omega,\mathcal{A})$. Consider the distance defined as $$D_u(P_0,P_1)=\int_\Omega \left(\frac{p_1}{p_0}\right)^u p_0 \mathrm{d}\mu<\infty.$$ Two distributions are chosen from each set $Q_0\in\mathscr P _0$ and $Q_1\in\mathscr P _1$ such that $$D_u(Q_0,Q_1)\geq D_u(P_0,P_1)\quad \forall (P_0,P_1)\in \mathscr P _0\times \mathscr P _1,\forall u\in[0,1]$$

Is it true that (A:)$$\int \min(q_0,t q_1)\mathrm{d}\mu\geq \int \min(p_0,t p_1)\mathrm{d}\mu\quad \forall (P_0,P_1)\in \mathscr P _0\times \mathscr P _1,\forall t$$or equaivalently (B:) $$Q_0\left[\frac{q_1}{q_0}>t\right]\geq P_0\left[\frac{q_1}{q_0}>t\right]\quad\ \forall P_0\in \mathscr P _0,\forall t$$ $$Q_1\left[\frac{q_1}{q_0}\leq t\right]\geq P_1\left[\frac{q_1}{q_0}\leq t\right]\quad\ \forall P_1\in \mathscr P _1,\forall t$$

Notes:

$\bullet$ One can consider A or B since both conditions are equivalent.

$\bullet$ $p_0$ and $p_1$ are densities of $P_0$ and $P_1$ and the same goes to $q_0$ and $q_1$ with $Q_0$ and $Q_1$.

What I know:

From Huber's paper (pages 260-261) Theorem 6.1 I know that if the distance is the $f$-divergence, i.e. $D_f$, then A and B are correct. Additionally, if A and B are correct, then $Q_0$ and $Q_1$ minimize $D_f$ (iff condition).

Huber considers $$Q_{jt}=(1-t)Q_{0t}+t Q_{1t}\\q_{jt}=(1-t)q_{0t}+t q_{1t}$$ and finds the first and second derivatives of $D_f(Q_{0t},Q_{1t})$. He then shows that the second derivative is $\geq 0$ (convex) and hence $(Q_{00},Q_{10})$ minimizes $D_f$ if and only if the first derivative evaluated at $t=0$ is $\geq 0$ for all $(Q_{01},Q_{11})\in(\mathscr P _0\times\mathscr P _1)$. He shows that this is really the case, hence the claim is true.

I think that this result can be strenghtened, i.e. if $(Q_0,Q_1)$ maximizes $D_u$ for all $u\in[0,1]$, then it should satisfy A or equivalently B. I dont know how to proceed.

Addendum: It seems that the question eventually boils down to finding $(Q_0,Q_1)$ which maximizes $D_u$ for all $u\in[0,1]$ and fails to minimize $D_f$ for at least one $f$. This will be a counterexample to the claim (of course if there exists such a pair).

$\endgroup$
6
  • $\begingroup$ Are you trying to replace f-divergence with any divergence $D_u$ or there is sth more? $\endgroup$
    – Henry.L
    Commented Apr 21, 2017 at 16:31
  • $\begingroup$ @Henry.L I am trying to replace $D_f$ with $D_u$. For $\Longrightarrow$ there is no problem. Because if $Q_0$ and $Q_1$ satisfy stochastic ordering, then they also minimize all f-divergences. $D_u$ is just a sub-set of f-divergences with $f=-x^u$ with an exception that $f(1)\neq 0$. I am interested in $\Longleftarrow$, For this case if $Q_0$ and $Q_1$ minimize all f-divergences, I know that they also satisfy stochastic ordering condition. $\endgroup$ Commented Apr 21, 2017 at 18:43
  • $\begingroup$ @Henry.L Now just replace $D_f$ by $D_u$. Will we have the same conclusion? namely if $Q_0$ and $Q_1$ maximize $D_u$ for all $u\in[0,1]$, will $Q_0$ and $Q_1$ satisfy A or B as given in the question? $\endgroup$ Commented Apr 21, 2017 at 18:43
  • $\begingroup$ I am kind of skeptical but cannot come up with a counter-example at the moment, will try to think of it... $\endgroup$
    – Henry.L
    Commented Apr 21, 2017 at 22:01
  • $\begingroup$ @Henry.L thank you very much for your interest. I will be happy to see a counterexample, if there exists one;) $\endgroup$ Commented Apr 21, 2017 at 22:05

1 Answer 1

2
$\begingroup$

Answer: this does NOT extends to other cases than f-divergence.

For the case $u=0$, there are many counter examples since $D_u$ is not separating any densities. So I think it suffices to consider $u\in (0,1]$.

Let $U^n=Unif(0,\frac{1}{n})$ the uniform measure on unit interval. And $\mathscr{P}_0$ corresponds to those with odd $n$, $\mathscr{P}_1$ contains exactly those with even $n$. Let $Q_0=U^n,Q_1=U^m$, thus $D_u(Q_0,Q_1)< \infty$ if $n<m$; and $=\infty$ otherwise due to the fact that $$\frac{q_{1}}{q_{0}}=\frac{m\boldsymbol{1}_{[0,\frac{1}{m}]}(\omega)}{n\boldsymbol{1}_{[0,\frac{1}{n}]}(\omega)}=\begin{cases} \frac{m}{n}>1 & \omega\in[0,\frac{1}{m}]\\ 0 & \omega\in[\frac{1}{m},\frac{1}{n}]\\ \infty & \omega\in[\frac{1}{n},1] \end{cases},n<m$$.(It suffices to consider the integration over the last segment w.r.t. $Q_0$'s density $q_0$.)

Therefore $D_u(Q_0,Q_1)\geq D_u(P_0,P_1)\quad \forall (P_0,P_1)\in \mathscr P _0\times \mathscr P _1,\forall u\in[0,1]$ as long as $Q_0=U^n,Q_1=U^m$ satisfying $n\geq m$. Now for any pair $Q_0=U^n,Q_1=U^m\text{ such that },n\geq m$, $$\frac{q_{1}}{q_{0}}=\frac{m\boldsymbol{1}_{[0,\frac{1}{m}]}(\omega)}{n\boldsymbol{1}_{[0,\frac{1}{n}]}(\omega)}=\begin{cases} \frac{m}{n}\leq1 & \omega\in[0,\frac{1}{n}]\\ \infty & \omega\in[\frac{1}{n},1] \end{cases},n\geq m$$ Let $t=1$ now and$ \left\{ \omega\in[0,1]\mid\frac{q_{1}}{q_{0}}>t\right\} =[\frac{1}{n},1]$ $$Q_{1}\left[\frac{q_{1}}{q_{0}}>t\right]=m\cdot\mu\left[[\frac{1}{n},1]\cap[0,\frac{1}{m}]\right]=m\cdot\mu\left[\frac{1}{n},\frac{1}{m}\right]=m[{\frac{1}{m}-\frac{1}{n}}]>0$$

Choose $P_{1}=U^{k}$ for any $k\geq n$ $$P_{1}\left[\frac{q_{1}}{q_{0}}>t\right]=k\cdot\mu\left[[\frac{1}{n},1]\cap[0,\frac{1}{k}]\right]=0$$

$\endgroup$
8
  • $\begingroup$ I havent read completely yet but $\mathscr P_0$ and $\mathscr P_1$ are non-overlapping sets so $\mathscr P_0\neq \mathscr P_1$. $\endgroup$ Commented Apr 21, 2017 at 23:03
  • $\begingroup$ @SeyhmusGüngören I correct it now, mindless typo. $\endgroup$
    – Henry.L
    Commented Apr 21, 2017 at 23:06
  • $\begingroup$ Thank you very much once again. This is a nice counterexample. I have a few comments. Choosing $P_1=U^k$ for any $k>n$ as $k$ is even and $n$ is odd. "the uniform measures on unit interval" probably better and if $\mu$ is the lebesgue measure wouldnt we get $[1/n,1/m]=1/m-1/n$? why is it at the denominator? and one last question: for which f-divergence wouldnt this example work? I mean it would fail to satisfy the distance condition. $\endgroup$ Commented Apr 22, 2017 at 1:58
  • $\begingroup$ @SeyhmusGüngören (1)Yes (2)When I edited the post I did not realize I put it in denominator and the error diffuse.., but that does not change the nature of counter example (3)That is worth another question... $\endgroup$
    – Henry.L
    Commented Apr 22, 2017 at 2:06
  • $\begingroup$ "Sure" from (1) to (3) $\endgroup$ Commented Apr 22, 2017 at 2:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .