Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

I'm wondering if there's any general technique that gives the total variation distance between a distribution on $\mathbb{R}^n$ and $N(0, I_n)$.

My understanding is that Stein's method gives only Wasserstein distance in higher dimension because the characterization of multivariate Gaussian is a second-order differential equation (while it is a first-order differential equation in one-dimensional case) so more regularity is required on test functions and thus it yields a weaker distance. And I understand that it is possible to improve Wasserstein distance to total variation distance if the distribution is log-concave.

What is the usual way to handle the total variation distance to multivariate Gaussian? I'm primarily interested in approximating $N(0,I_n)$ but the approximating distribution is not necessarily log-concave. Perhaps there's some easy way for this special case? Or is there any impossibility result?

share|improve this question
    
Unfortunately I cannot help you, since I just found out about the Stein's method. However, I was wondering if you could point me in the direction of the material behind the second paragraph of your question. I have a similar problem: I am trying to upper-bound the total variation distance between $N(0,I_n)$ and another distribution. My other distribution happens to be a mixture of multivariate Gaussians with unit variance, but non-zero vectors of means. Thus, I am wondering about the applicability of the last sentence of the second paragraph to mixtures of log-concave distributions. –  Bullmoose May 31 '13 at 8:30

2 Answers 2

Stein's method doesn't give total variation approximation in one dimension, either, without some kind of additional assumptions. This has nothing to do with Stein's method; for an impossibility result, any discrete distribution has maximal (1 or 2 depending on your normalization convention) total variation distance to any continuous (e.g. Gaussian) distribution. But of course you can approximate any distribution by a discrete distribution, in Wasserstein distance for example.

share|improve this answer

I am not sure the following helps.

If $f$ is a prob.distribution on $\mathbb{R}^n$ having the same covariance matrix as $N(0, I_n)$, then \begin{eqnarray*}D(f\|N(0, I_n))&=& H(N(0, I_n))-H(f)\\\ &=& \frac{1}{2}\log((2\pi e)^n)-H(f)\end{eqnarray*} where $D(\cdot \| \cdot)$ denotes the KL divergence $H$ denotes the Shannon entropy.

Now, by Pinsker's inequality, \begin{eqnarray*}\|f-N(0, I_n)\|_1 &\le& \sqrt{2 D(f\|N(0, I_n))}\\\ &=& \sqrt{\log((2\pi e)^n)-2 H(f)}\end{eqnarray*}

So the conclusion is that if the entropy $f$ is closer to that of Gaussian, $f$ will be closer to $N(0, I_n)$ in total variation. But it works only for those $f$ which also has covariance matrix $I_n$.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.