1
$\begingroup$

I'm wondering if there's any general technique that gives the total variation distance between a distribution on $\mathbb{R}^n$ and $N(0, I_n)$.

My understanding is that Stein's method gives only Wasserstein distance in higher dimension because the characterization of multivariate Gaussian is a second-order differential equation (while it is a first-order differential equation in one-dimensional case) so more regularity is required on test functions and thus it yields a weaker distance. And I understand that it is possible to improve Wasserstein distance to total variation distance if the distribution is log-concave.

What is the usual way to handle the total variation distance to multivariate Gaussian? I'm primarily interested in approximating $N(0,I_n)$ but the approximating distribution is not necessarily log-concave. Perhaps there's some easy way for this special case? Or is there any impossibility result?

Improve this question
$\endgroup$
1
  • $\begingroup$ Unfortunately I cannot help you, since I just found out about the Stein's method. However, I was wondering if you could point me in the direction of the material behind the second paragraph of your question. I have a similar problem: I am trying to upper-bound the total variation distance between $N(0,I_n)$ and another distribution. My other distribution happens to be a mixture of multivariate Gaussians with unit variance, but non-zero vectors of means. Thus, I am wondering about the applicability of the last sentence of the second paragraph to mixtures of log-concave distributions. $\endgroup$
    – Bullmoose
    May 31, 2013 at 8:30

2 Answers 2

Reset to default
1
$\begingroup$

Stein's method doesn't give total variation approximation in one dimension, either, without some kind of additional assumptions. This has nothing to do with Stein's method; for an impossibility result, any discrete distribution has maximal (1 or 2 depending on your normalization convention) total variation distance to any continuous (e.g. Gaussian) distribution. But of course you can approximate any distribution by a discrete distribution, in Wasserstein distance for example.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Stein's method can be used to derive total-variation bounds. This is because $d_{TV}(p_1,p_2) = max_{||h|| \leq 1} (E_{p_1}(h) - E_{p_2}(h) )$. The properties of the solution of the "Stein differential equation" for gaussian approximation in 1D are given in Lemma 2.4 of Chen, Goldstein, Shao (2011). $\endgroup$
    – Guillaume Dehaene
    Feb 4, 2016 at 10:25
  • $\begingroup$ @Guillaume: You seem to have misread what I wrote. What I said is that total variation approximation is simply false without some kind of additional assumptions, so it is equally out of reach for any method. This is the kind of "impossibility result" the OP asked about. On the other had, if you do have appropriate additional assumptions, then Stein's method is often the best way to reach total variation bounds. $\endgroup$
    – Mark Meckes
    Feb 4, 2016 at 14:11
  • 1
    $\begingroup$ Incidentally, the OP's question suggests that they might be familiar with arxiv.org/abs/math/0606073, from which you can see that I know perfectly well that Stein's method can be used to derive total variation bounds. $\endgroup$
    – Mark Meckes
    Feb 4, 2016 at 14:16
0
$\begingroup$

I am not sure the following helps.

If $f$ is a prob.distribution on $\mathbb{R}^n$ having the same covariance matrix as $N(0, I_n)$, then \begin{eqnarray*}D(f\|N(0, I_n))&=& H(N(0, I_n))-H(f)\\\ &=& \frac{1}{2}\log((2\pi e)^n)-H(f)\end{eqnarray*} where $D(\cdot \| \cdot)$ denotes the KL divergence $H$ denotes the Shannon entropy.

Now, by Pinsker's inequality, \begin{eqnarray*}\|f-N(0, I_n)\|_1 &\le& \sqrt{2 D(f\|N(0, I_n))}\\\ &=& \sqrt{\log((2\pi e)^n)-2 H(f)}\end{eqnarray*}

So the conclusion is that if the entropy $f$ is closer to that of Gaussian, $f$ will be closer to $N(0, I_n)$ in total variation. But it works only for those $f$ which also has covariance matrix $I_n$.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.