I would like to understand the relationship between minimising the KL divergence $P \mapsto D_{KL}[P,Q]$ and the reverse KL divergence $P\mapsto D_{KL}^*[P,Q]=D_{KL}[Q,P]$ for probability measures $P$ and (fixed) $Q$.
Both are $f$-divergences $$D_f[P,Q]= \int f\left(\frac {dP}{dQ}\right)dQ,$$
with respective function $f(t)= t\log t$ and $f^*(t)=-\log t$.
Denote by $\mathcal C$ the cone of convex functions $[0, \infty) \mapsto \mathbb{R}$ that vanish at $1$. A nice property of $f$-divergences, which relates the KL divergences to its reversal, is given in Proposition 2 of On f-Divergences: Integral Representations, Local Behavior, and Inequalities. I. Sason (2018).
Let $f \in \mathcal{C},$ and let $f^{*}:(0, \infty) \mapsto \mathbb{R}$ be the conjugate function, given by $$ f^{*}(t)=t f\left(\frac{1}{t}\right) $$ for $t>0 .$ Then, $f^{*} \in \mathcal{C} ; f^{* *}=f,$ and for every pair of probability measures $(P, Q)$ $$ D_{f}(P \| Q)=D_{f^{*}}(Q \| P) $$
I am wondering whether anyone has come across this notion of duality between convex functions
$$\mathcal{C}\to \mathcal{C}: f \mapsto f^*.$$
It does not seem related to the usual notion of convex duality (Fenchel-Legendre transform).
References would be much appreciated.
