so Silverman in his 1986 book mentioned about approximating distributions with Gaussian mixture models but he didn't go much further into the topic...I'm just wondering, say I'm given a N-dimensional uniform box (and as a further extension to this, any arbitrary distribution) $u(\bar{x})$ , is there a neat way to approximately it with a (truncated) k-kernel Gaussian mixture model $g(\bar{x}) = \sum_{i} \omega_i N(\mu_i,\sigma^2_i)$?

I've attempted this by trying to minimise some sort of measurable divergence between the two, say the Hellinger distance or the Kullback–Leibler divergence, but analytical solutions don't seem to be plausible and in terms of numerical approaches, the the computational costs climb up way too rapidly as the number of kernels/dimension goes up.

I wonder if any previous studies have been done on this topic before but haven't been quite successful in finding anything useful... if someone can kindly give me some leads or perhaps point me to the relevant literature on this it'd be greatly appreciated!

Cheers!

  • Let’s start simple: If $u$ is the uniform distribution on [0,1], what Gaussian mixture $g$ best approximates it in terms of the Hellinger or KL metric? Is a mixture of two Gaussians enough? – Matt F. Dec 7 at 3:35
  • It's enough to be a significant improvement over a single kernel approximation but obviously the more kernels you add the better the approximation would get. – crystfallZ Dec 7 at 3:49
  • The general question seems premature in the absence of some numerics on this case first. – Matt F. Dec 7 at 6:15

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.