Is there a notion of a higher Kullback-Leibler divergence?

Question

For discrete probability distributions $P$ and $Q$ defined on the same sample space, $\mathcal{X}$, the Kullback-Leibler divergence is defined as $$ D_{\mathrm{KL}}(P \parallel Q)=\sum_{x \in \mathcal{X}} P(x) \log \frac{P(x)}{Q(x)} $$

Is there a notion of a higher Kullback-Leibler divergence? In other words, has the following divergence been studied:

$$ D^k_{\mathrm{KL}}(P \parallel Q)=\sum_{x \in \mathcal{X}} P(x) \left(\log \frac{P(x)}{Q(x)}\right)^k. $$

This is with a view to generalizing Stam's inequality.

Answer 1

You might be interested in the notion of renyi divergence. There have been generalizations of Stam's inequality to this setting, see Variants of the Entropy Power Inequality by Bobkov and Marsiglietti.

Note that this is not for your proposed variant of the KL divergence. A common way to generalizing ways of measuring distances between two distributions is via $f$-divergences. These satisfy many useful properties that things like the KL divergence (for example, the data-processing inequality).

Your formula may be written as an $f$-divergence for $f_k(x) = x(\ln x)^k$. All the literature on $f$-divergences I know of assumes that $f(x)$ is convex (it is quite key to the theory), but while $f_1$ is convex (e.g. viewing the KL divergence as an $f$-divergence fits into the existing theory well, which is of course by design), $f_k$ need not be convex for $k>1$ (and one can check that $f_2$ is already non-convex), so your proposed generalization is in a technical sense quite different than the KL divergence.