1
$\begingroup$

I'm interested in the Kullback-Leibler divergence on multimodal gaussian mixtures.

For positive, real weights $\sum_{1\leq k\leq m}w_k=\sum_{m+1\leq k\leq n}w_k=1$, univariate Gaussians $g_k\equiv g(\mu_k,\sigma_k)$, and a convention $g_{1,\ldots,n}^{w_1,\ldots,w_n}=\sum_{1\leq k\leq n}w_kg_k$

Finally, Kullback-Leibler divergence $\text{D}_\text{KL}(g_{1,\ldots,n}^{w_1,\ldots,w_n}|\>g_{m+1,\ldots,2n}^{w_{m+1},\ldots,w_{2n}})=\int_{\mathbb{R}}g_{1,\ldots,n}^{w_1,\ldots,w_n}\ln(\frac{g_{1,\ldots,n}^{w_1,\ldots,w_n}}{g_{m+1,\ldots,2n}^{w_{m+1},\ldots,w_{2n}}})dx$

Question: Is there any research on the behavior of the Kullback-Leibler divergence in the cases when its comparing a mixture whose weights $(w_1,\ldots,w_n)$ shifted away from a uniform weighting $(1/n,\ldots,1/n)$?

i.e. $\text{D}_\text{KL}(g_{1,\ldots,n}^{w_1,\ldots,w_n}|g_{1,\ldots,n}^{1/n,\ldots,1/n})$

$\endgroup$
2

0

You must log in to answer this question.

Browse other questions tagged .