Are known expressions for total variation distance between $N(0,\sigma^2)$ and $N(0,\sigma^2+\epsilon)$ for small $\epsilon$? The only thing I seem to find is things are expression about the mean but not if we change variance slightly.
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
5
There's also a softer argument based on properties of the heat kernel, which applies in higher dimensions as well, in Lemma 4.9 of this paper of Klartag. It shows that in $n$ dimensions, the total variation distance between centered Gaussian distributions with covariances $\alpha I_n$ and $\beta I_n$ is at most $$ C \sqrt{n} \left|\frac{\beta}{\alpha} - 1\right|, $$ where $C > 0$ is an absolute constant (which can be made explicit if you want).
answered Feb 12, 2014 at 9:10
-
$\begingroup$ AH! interesting I am a probabilist but I am trying to show an integral satisfy a Holder continuous condition. Naturally I converted the language into total variation distance! You have just reverted it back to the original problem! $\endgroup$– lost1Commented Feb 12, 2014 at 14:00
-
1$\begingroup$ This seems to easily follow from Pinsker's inequality. At least the \sqrt{n} part. The proof in the paper is very nice. $\endgroup$ Commented May 11, 2016 at 6:59
-
$\begingroup$ @mr.gondolier If I did the calculation right, then Pinsker gives $\sqrt{\frac{n}{2}[ (\beta/\alpha)^2 - 1 - \log(\beta / \alpha)]}$, which reproduces this result when $\alpha$ and $\beta$ are very different, but which is worse when $\beta / \alpha \approx 1$. $\endgroup$ Commented May 11, 2016 at 15:01
-
$\begingroup$ @MarkMeckes I have $\sqrt{n/2 (\beta/\alpha-1-\log \beta/\alpha)}$. So asymptotically when $\beta/\alpha \approx 1$, KL divergence is quadratic and it gives the same result. $\endgroup$ Commented May 14, 2016 at 5:42
-
$\begingroup$ @mr.gondolier You're right, I was alternately using $\alpha$ and $\beta$ as variance in one place and standard deviation in another. So Pinsker does give another nice approach, more elementar than the heat kernel approach, and simpler to carry out than the direct computation. $\endgroup$ Commented May 14, 2016 at 11:03
$\begingroup$
$\endgroup$
5
As marcoromito wrote, this is an elementary calculation. However, I thought I would record a nice approximation that I stumbled across. Whether it is new, I have no idea.
ADDED: The following sentence has changed.
By scaling, we can restrict ourselves to $\mathrm{tvd}(\epsilon)$ which is the total variation distance between $N(0,1)$ and $N(0,(1+\epsilon)^2)$ for $\epsilon\ge 0$. The points where the two densities are equal are $\pm x_0$ where $$x_0 = \frac{(1+\epsilon)\sqrt{2\ln(1+\epsilon)}}{\sqrt{\epsilon(2+\epsilon)}}.$$ Integrating the difference between the densities over $[-x_0,x_0]$ we find $$\mathrm{tvd}(\epsilon) = \mathrm{erf}\left( \frac{(1+\epsilon)\sqrt{\ln(1+\epsilon)}}{\sqrt{\epsilon(2+\epsilon)}}\right) - \mathrm{erf}\left( \frac{\sqrt{\ln(1+\epsilon)}}{\sqrt{\epsilon(2+\epsilon)}}\right). $$ The interesting thing is that for small $\epsilon\ge 0$ $$ \mathrm{tvd}(\epsilon) = \mathrm{tvd'}(\epsilon) + O(\epsilon^5)$$ where $$ \mathrm{tvd'}(\epsilon) = \frac{2^{3/2}\,\epsilon }{\pi^{1/2}e^{1/2}(2+\epsilon)}.$$ This rational approximation is remarkably precise for small $\epsilon$.
Experimentally (based on plotting the ratio in Maple and seeing a smooth curve), $$1 \le \frac{\mathrm{tvd}(\epsilon)}{\mathrm{tvd'}(\epsilon)} \le \sqrt{e\pi/8} \approx 1.03318$$ for all $\epsilon\gt 0$.
answered Feb 8, 2014 at 1:51
-
$\begingroup$ yeah, the last bit is exactly what i wanted. I got something similar, but not the same, so i need to chck. I was doing some pde stuff and estimating an integral against greens function of the heat equation, and want to show this integral is Holder and the expression basically the total variation distance of two gaussians. $\endgroup$– lost1Commented Feb 8, 2014 at 3:46
-
$\begingroup$ I am interested in how you got the 1.0332 and what is the bound for non unit variance? $\endgroup$– lost1Commented Feb 8, 2014 at 3:48
-
$\begingroup$ You have an extra factor of $2+\epsilon$ on the bottom, which I disagree with. $\endgroup$– lost1Commented Feb 11, 2014 at 15:11
-
$\begingroup$ @lost1: Sorry, I had a mistake. My formula is for standard deviation $1+\epsilon$, not variance $1+\epsilon$. I changed it. Do you agree now? $\endgroup$ Commented Feb 12, 2014 at 5:35
-
$\begingroup$ i verified this. working with variance $v$ and $v+\epsilon$ is (imo) nicer, because I needed to perturb it for a general $v$ but thank you. $\endgroup$– lost1Commented Feb 12, 2014 at 14:48
$\begingroup$
$\endgroup$
For two measures with densities f,g, the total variation distance is $$ \int_{f>g}(f(x)-g(x))\,dx $$ For two Gaussian measures with the same mean and different variances it is easy to identify the set $\{f>g\}$ and to obtain the formula for the distance in terms of the partition function of the standard Gaussian measure
