0
$\begingroup$

Does anyone know about a statistical divergence of this type? \begin{equation} \text{D}(P||Q) = \frac{1}{2} \left[\text{KL}(M||P) + \text{KL}(M||Q)\right] \end{equation} where $M = \frac{1}{2} [P+Q]$.

This divergence is very similar to the Jensen-Shannon Divergence $\text{D}(P||Q) = \text{KL}(P||M) + \text{KL}(Q||M)$ but where the distributions in the argument of the statistical divergence appear in the second argument of the KL divergences.

I am interested in knowing if such divergence exists in the literature and to properties of such divergence. Thanks!

$\endgroup$

2 Answers 2

1
$\begingroup$

That is J-divergence, commonly just called 'symmetrised KL divergence.'

$\endgroup$
1
  • $\begingroup$ I remember hearing this is the metric K & L originally called 'divergence', its components being called 'discriminations.' First citation seems to confirm this. $\endgroup$ Commented Apr 21, 2020 at 18:20
1
$\begingroup$

This might help as a place to start. Sgarro has generalized the Kullback-Leibler divergence $D_{\rm KL}(Q||P)$ to multiple distributions by introducing the average divergence

$$D’(P_1,\ldots P_k)=\frac{1}{k(k-1)}\sum_{i,j=1}^k D_{\rm KL}(P_i||P_j),$$

so what you have is related to his measure of divergence by $$ D(P,Q)=3D’(P,Q,M)-\frac12D_{\rm KL}(P||Q) -\frac12D_{\rm KL}(Q||P)-\frac12D_{\rm KL}(P||M) -\frac12D_{\rm KL}(Q||M). $$ where $M=\frac12(P+Q).$

A. Sgarro, “Informational divergence and the dissimilarity of probability distributions”, Calcolo(18): 293–302 (1981).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.