# Jensen-Shannon Divergence of Sample Distributions

Given normal distributions with a single positional and variation parameter each, $$p_1=\big[\mu_1, \sigma_1\big]$$, $$p_2=\big[\mu_2, \sigma_2\big]$$, we define their Jensen-Shannon divergence as: $$\text{JS}(p_1, p_2)=\big|\int_{\mathbb{R}}\frac{p_1+p_2}{2}\ln\big(\frac{p_1+p_2}{2}\big)-\frac{1}{2}\big(p_1\ln p_1+p_2\ln p_2\big)dx\big|$$ We write $$p_1(s)=\big[\mu_1,\sigma_1/\sqrt{s}\big]$$, $$p_2(s)=\big[\mu_2,\sigma_2/\sqrt{s}\big]$$ to signify their corresponding sampling distributions, $$s>0$$.

Question: Are there any resources that establish a relationship, ideally some identity, between the $$\text{JS}(p_1, p_2)$$ and $$\text{JS}(p_1(s), p_2(s))$$?

EDIT For simplicity, $$1/\sqrt{s}=\lambda$$, and we will refer to the distributions above in terms of their coordinates (as brackets). We will superscript the Jensen-Shannon by $$\lambda$$ to say that each distribution is denormalized by a factor $$\lambda$$. Since the Jensen-Shannon is $$1$$-homogenous ($$f(\lambda x)=\lambda f(x)$$), we can extract $$\lambda$$ as below, then translating by $$x\mapsto x+\mu_1$$, $$\frac{1}{\lambda}\text{JS}^{\lambda}\big([0, \sigma_1\lambda],[\mu_2-\mu_1, \sigma_2\lambda]\big)$$ We scale by $$x\mapsto x\lambda$$ to eliminate the $$\lambda$$ factors from the $$\sigma$$ terms. $$=\text{JS}\big([0, 1], [(\mu_2-\mu_1)/\lambda, \sigma_2/\sigma_1])$$ Of course for the original divergence, similar operations give us $$\text{JS}(p_1, p_2)=\text{JS}\big([0, 1], [(\mu_2-\mu_1), \sigma_2/\sigma_1]\big)$$ So really the question becomes comparing the Jensen-Shannon divergences of two distributions, the first $$[0, 1]$$ and the second $$[\mu_2-\mu_1, \sigma_2/\sigma_1]$$, where the second Jensen-Shannon has scaled the positional argument by $$1/\lambda$$.

Therefore I propose we consider this question to be a multiplicative property (if any) of the position whereas the translation-invariance property is an additive property.

• Do you mean the corresponding sampling distribution of the means as a function of sample size $s$? Apr 18, 2019 at 14:22
• Yes, I am wondering if there is some inequality or relationship between the Jensen-Shannon of distributions and their respective $s$ sampling distributions. I understand the Jensen-Shannon is invariant under translation of position ($\mu_1+\lambda, \mu_2+\lambda$), so I'm curious to see if there similar properties w.r.t. the variances. Apr 18, 2019 at 16:04