7
$\begingroup$

Do we have an exact formula to compute the KL divergence between 2 mixtures of Gaussians (i.e convex combinations of a finite number of Gaussian distributions)?

If not exactly known, are there good upperbounds that are known for this quantity?

$\endgroup$
5
$\begingroup$

There is no closed form expression, for approximations see:

A lower and an upper bound for the Kullback-Leibler divergence between two Gaussian mixtures are proposed. The mean of these bounds provides an approximation to the KL divergence which is shown to be equivalent to a previously proposed approximation in:

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I looked into this paper quite a lot. How good do you think the result is? It seems to be quite loose an upperbound. As in when there is no mixture but just 2 Gaussians then this upperbound seems to be a additive error of dimension away from the true KL which is known in this case. $\endgroup$ – gradstudent Sep 14 '18 at 20:59
  • $\begingroup$ What is your conclusion regarding the tightness of this approximation? Were you able to find tighter bounds? This is the first paper that shows up in Google search. $\endgroup$ – Ankitp Oct 16 '18 at 4:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.