Information monotonicity of divergence => function of $f$-divergence

Question

It is well-known that $f$-divergences defined on $\mathcal P(\mathcal X)$ where $\mathcal X$ is a measure space with $\sigma$-algebra $\mathcal B$ satisfy the property of information monotonicity:

For every $f$-divergence and every stochastic kernel \begin{equation*} \begin{split} K: \mathcal{X} \times \mathcal{B} \mapsto[0,1] \\ D_{f}(K P, K Q) \leq D_{f}(P, Q) \end{split} \end{equation*} Ref: Thm 14. Liese, Vadja (2006). On Divergences and Informations in Statistics and Information Theory.

Conversely, if a divergence can be written as an integral of some function, and if it satisfies information monotonicity, then Amari showed that it must be an $f$-divergence. Ref: Appendix A. Amari (2009). Alpha-Divergence Is Unique, Belonging to Both -Divergence and Bregman Divergence Classes

This led Amari to conjecture, in the same paper that: When a divergence $D$ satisfies information monotonicity, it must be a function of an $f$ -divergence.

I haven't found any follow-ups to this conjecture in the literature. Does anyone know a reference for a proof or a counterexample?

Answer 1

It is known that, in general, a monotonic divergence measure does not have to a monotonically-increasing function of an $f$-divergence. See discussion after Definition 1 (and footnote 3) in

• Polyanskiy and Verdú, Arimoto Channel Coding Converse and Rényi Divergence, 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2010.

The situation is different for divergence measures that are decomposable, meaning they can be written as \begin{align} D(p||q) = \sum_{i=1}^n f(p_i, q_i) \end{align} where $n$ is the size of the alphabet. It is known that for binary alphabet, $n=2$, there are divergence measures that are monotonic and decomposable but are not $f$-divergences (nor functions of $f$-divergences); see Lemma 1 in:

• J. Jiao, T. A. Courtade, A. No, K. Venkat, and T. Weissman, "Information measures: the curious case of the binary alphabet," IEEE Trans. Inform. Theory, 2014.

For $n\ge 3$, any monotonic and decomposable divergence must be an $f$-divergence; see the above paper by Jiao et al. and Theorem 1 in

• Pardo and Vajda, About Distances of Discrete Distributions Satisfying the Data Processing Theorem of Information Theory, IEEE Trans on Information Theory, 1997.

(Importantly, Pardo and Vajda do not mention that the proof of Theorem 1 only works for $n\ge 3$; this was first pointed out in Jiao et al.)