Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

L1 distance between gaussian measures: Definition

Let $P_1$ and $P_0$ be two gaussian measures on $\mathbb{R}^p$ with respective "mean,Variance" $m_1,C_1$ and $m_0,C_0$ (I assume matrices have full rank). I know that calculating the L1 distance between $P_1$ and $P_0$: $$d_1=\int|dP_1-dP_0|$$

The easy case

is an easy exercice when $C_0=C_1=C$: $$d_1=2(2\Phi(\sigma/2)-1)$$ where $$\sigma=\|C^{-1/2}(m_1-m_0)\|.$$ (norm of $\mathbb{R}^p$) and $\Phi$ is the cdf of a gaussian mean zero variance 1 reall variable.

I don't remember the name of $\sigma$ (RKHS norm? Cameron martin ?) but it can also be written: $\|\mathcal{L}\|_{L_2(P_{1/2})}$ where $\mathcal{L}$ is the log likelihood ratio function and $P_{1/2}$ is the normal distribution with mean $(m_1+m_0)/2$ and variance $C$.

My question is about how to extend that type of result for the case when $C_0\neq C_1$ (explicit calculation of the L1 distance).

I see two possible reductions of the problem if calculous are too complicated:

  1. search for an inequality relating the L1 distance and some norm of the likelihood ratio
  2. search for some exact expression in a particular case, for example $C_1$ and $C_0$ diagonal.

Reduction 1 gets a partial answer with the general inequality

$$d_1\leq 2\sqrt{K(P_1,P_0)}$$ (due to pinsker or Lecam I don't remember) where $$K(P_1,P_0)=\int \log \left(\frac{dP_1}{dP_0} \right ) dP_1$$ is the kullback divergence.

I am not really satisfyed with this answer since in the case $C_1=C_0$ it is suboptimal, it does not include an "half measure" $P_{1/2}$ (could include $(P_0+P_1)/2$ by using twice the inequality but I don't really like this interpolation),...

share|improve this question
    
Shouldn't your formula read $d_1=4\Phi(\sigma/2)-2$, which takes the expected values $d_1=0$ when $\sigma=0$ and $d_1\to2$ as $\sigma\to\infty$? –  George Lowther Aug 4 '10 at 21:42
    
oops, thanks, I have updated the question... –  robin girard Aug 5 '10 at 7:04
    
I have tried to answer a somehow related question on stats.stackexchange to start a discussion on the interpolation question stats.stackexchange.com/questions/7912/… –  robin girard Mar 7 '11 at 9:21
    
You can get a little bit better with your strategy. Take as intermediate measure $P_{1/2}$ a Gaussian with mean $m_0$ and variance $C_1$ or vice versa. Then you just estimate by using the triangle inequality and the resulting both distances as follows: Firstly, use the exact formula for the difference of the two Gaussians with the same variance. Secondly, use Kullback-Leibler divergence to estimate the both Gaussians with the same mean. –  André Schlichting May 13 '11 at 14:51
add comment

1 Answer

Another bound can be obtained via the Hellinger distance $d_1 \leq \frac{\sqrt{2}}{2} H(P,Q)$. (not sure how tight/loose the obtained bound will be though)

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.