0
$\begingroup$

Does anyone know about a statistical divergence of this type? \begin{equation} \text{D}(P||Q) = \frac{1}{2} \left[\text{KL}(M||P) + \text{KL}(M||Q)\right] \end{equation} where $M = \frac{1}{2} [P+Q]$.

This divergence is very similar to the Jensen-Shannon Divergence $\text{D}(P||Q) = \text{KL}(P||M) + \text{KL}(Q||M)$ but where the distributions in the argument of the statistical divergence appear in the second argument of the KL divergences.

I am interested in knowing if such divergence exists in the literature and to properties of such divergence. Thanks!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

That is J-divergence, commonly just called 'symmetrised KL divergence.'

Improve this answer
$\endgroup$
1
  • $\begingroup$ I remember hearing this is the metric K & L originally called 'divergence', its components being called 'discriminations.' First citation seems to confirm this. $\endgroup$
    – Christian Chapman
    Commented Apr 21, 2020 at 18:20
1
$\begingroup$

This might help as a place to start. Sgarro has generalized the Kullback-Leibler divergence $D_{\rm KL}(Q||P)$ to multiple distributions by introducing the average divergence

$$D’(P_1,\ldots P_k)=\frac{1}{k(k-1)}\sum_{i,j=1}^k D_{\rm KL}(P_i||P_j),$$

so what you have is related to his measure of divergence by $$ D(P,Q)=3D’(P,Q,M)-\frac12D_{\rm KL}(P||Q) -\frac12D_{\rm KL}(Q||P)-\frac12D_{\rm KL}(P||M) -\frac12D_{\rm KL}(Q||M). $$ where $M=\frac12(P+Q).$

A. Sgarro, “Informational divergence and the dissimilarity of probability distributions”, Calcolo(18): 293–302 (1981).

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .