Let $P$ be a prior probability distribution over some events $A$ and $B$, and suppose that we want to minimise an $f$-divergence between $P$ and the set of all distributions $Q$ that satisfy that constraint that $Q(B|A) = q$ for some fixed $q \geq P(B|A)$. Let $P_{f}$ denote the result of minimising a given $f$-divergence with this constraint. Is it always true, for any $f$, that $P_{f}(A) \leq P(A)$? I know that this holds for several examples (Kullback Leibler divergence, Hellinger distance, Inverse Kullback Leibler divergence), but is it true in general?
Minimising the f-divergence to a conditional probability constraint.
King Kong
- 631
- 3
- 16