Is there any closed form expression for Rényi entropy of a set variables with multivariate Gaussian distribution?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Yes. First, do a change of variable in the integral to convert it to the Renyi entropy of a set of uncorrelated Gaussians with standard deviation $1$. The integral now splits into a product of 1-dimensional integrals, where each one is the Renyi entropy of a 1-dimensional Gaussian.
An alternative approach is to write the integral in polar co-ordinates and split the integral into the product of a spherical integral (which is equal to something like the determinant of the covariance matrix multiplied by the volume of a sphere) and a radial integral. The radial integral can be written in terms of gamma functions (or, beta functions) using the right change of variable.
answered Sep 15, 2011 at 18:29
-
$\begingroup$ If $X$ is $\mathcal{N}(\mu, K)$ random vector then $K$ can be written as $K=U\Lambda U^T$ where $\Lambda$ is a diagonal matrix with the eigen values of $K$ and $U$ is orthonormal. If we define $Y=U^T(X-\mu)$, then $Y$ is a Gaussian vector of independent random variables with mean $0$ and covariance matrix $\Lambda$. Do you mean to say that Renyi entropy of $X$ is equal to The Renyi entropy of $Y$? $\endgroup$– AshokCommented Sep 29, 2011 at 5:12
-
$\begingroup$ Yes. Just use the definition of Renyi entropy and do the change of variables. $\endgroup$ Commented Sep 29, 2011 at 14:33