Does anyone know about a statistical divergence of this type? \begin{equation} \text{D}(P||Q) = \frac{1}{2} \left[\text{KL}(M||P) + \text{KL}(M||Q)\right] \end{equation} where $M = \frac{1}{2} [P+Q]$.

This divergence is very similar to the Jensen-Shannon Divergence $\text{D}(P||Q) = \text{KL}(P||M) + \text{KL}(Q||M)$ but where the distributions in the argument of the statistical divergence appear in the second argument of the KL divergences.

I am interested in knowing if such divergence exists in the literature and to properties of such divergence. Thanks!


2 Answers 2


This might help as a place to start. Sgarro has generalized the Kullback-Leibler divergence $D_{\rm KL}(Q||P)$ to multiple distributions by introducing the average divergence

$$D’(P_1,\ldots P_k)=\frac{1}{k(k-1)}\sum_{i,j=1}^k D_{\rm KL}(P_i||P_j),$$

so what you have is related to his measure of divergence by $$ D(P,Q)=3D’(P,Q,M)-\frac12D_{\rm KL}(P||Q) -\frac12D_{\rm KL}(Q||P)-\frac12D_{\rm KL}(P||M) -\frac12D_{\rm KL}(Q||M). $$ where $M=\frac12(P+Q).$

A. Sgarro, “Informational divergence and the dissimilarity of probability distributions”, Calcolo(18): 293–302 (1981).


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .