MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

L1 distance between gaussian measures: Definition

Let $P_1$ and $P_0$ be two gaussian measures on $\mathbb{R}^p$ with respective "mean,Variance" $m_1,C_1$ and $m_0,C_0$ (I assume matrices have full rank). I know that calculating the L1 distance between $P_1$ and $P_0$: $$d_1=\int|dP_1-dP_0|$$

The easy case

is an easy exercice when $C_0=C_1=C$: $$d_1=2(2\Phi(\sigma/2)-1)$$ where $$\sigma=\|C^{-1/2}(m_1-m_0)\|.$$ (norm of $\mathbb{R}^p$) and $\Phi$ is the cdf of a gaussian mean zero variance 1 reall variable.

I don't remember the name of $\sigma$ (RKHS norm? Cameron martin ?) but it can also be written: $\|\mathcal{L}\|_{L_2(P_{1/2})}$ where $\mathcal{L}$ is the log likelihood ratio function and $P_{1/2}$ is the normal distribution with mean $(m_1+m_0)/2$ and variance $C$.

My question is about how to extend that type of result for the case when $C_0\neq C_1$ (explicit calculation of the L1 distance).

I see two possible reductions of the problem if calculous are too complicated:

  1. search for an inequality relating the L1 distance and some norm of the likelihood ratio
  2. search for some exact expression in a particular case, for example $C_1$ and $C_0$ diagonal.

Reduction 1 gets a partial answer with the general inequality

$$d_1\leq 2\sqrt{K(P_1,P_0)}$$ (due to pinsker or Lecam I don't remember) where $$K(P_1,P_0)=\int \log \left(\frac{dP_1}{dP_0} \right ) dP_1$$ is the kullback divergence.

I am not really satisfyed with this answer since in the case $C_1=C_0$ it is suboptimal, it does not include an "half measure" $P_{1/2}$ (could include $(P_0+P_1)/2$ by using twice the inequality but I don't really like this interpolation),...

share|cite|improve this question
Shouldn't your formula read $d_1=4\Phi(\sigma/2)-2$, which takes the expected values $d_1=0$ when $\sigma=0$ and $d_1\to2$ as $\sigma\to\infty$? – George Lowther Aug 4 '10 at 21:42
oops, thanks, I have updated the question... – robin girard Aug 5 '10 at 7:04
I have tried to answer a somehow related question on stats.stackexchange to start a discussion on the interpolation question… – robin girard Mar 7 '11 at 9:21
You can get a little bit better with your strategy. Take as intermediate measure $P_{1/2}$ a Gaussian with mean $m_0$ and variance $C_1$ or vice versa. Then you just estimate by using the triangle inequality and the resulting both distances as follows: Firstly, use the exact formula for the difference of the two Gaussians with the same variance. Secondly, use Kullback-Leibler divergence to estimate the both Gaussians with the same mean. – André Schlichting May 13 '11 at 14:51
Do you have a citation for the exact formula in the case where variances are equal? – user80963 Oct 2 '15 at 9:53

Another bound can be obtained via the Hellinger distance $d_1 \leq \frac{\sqrt{2}}{2} H(P,Q)$. (not sure how tight/loose the obtained bound will be though)

share|cite|improve this answer

What you wrote is also a total variation distance between two Gaussian measures and $\sigma$ is calculated indeed via norm of Cameron-Martin space.

I'm not sure what to do for example in the diagonal case, one should look for the proof when correlation matrices are the same. In this case ellipsoids related to covariance operators have the same orientation, and one can try to find the Borel set that gives the supremum of total variation norm and hence to $L^1$ distance. Just because the ellipsoids are the same, maybe finding the optimal Borel set is a solvable excercise (again look for the proof when $C_0=C_1$). (I think it could be some set like a half plane -- at least in one dimension it is rather easy excercise -- find $L^1$ norm between Gaussians with different variances -- this set is just a set where one density is bigger than the other one, in multidimensional case when $C_0=C_1$ this set is a half plane due to the good orientation of the ellipsoids).

For diagonal case if the previous statement is true, one can try to find again a good Borel set related to the spectral values.

Otherwise I'm not sure that for general $C_0,C_1$ could be calculated since there is no symmetry in the problem (look the example about half planes, that I wrote above).

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.