2
$\begingroup$

I am interested in a lower bound on the Bhattacharya distance between two independent multivariate Gaussian distributions. To be precise, consider zero-mean independent Gaussian distributions $p_1\sim\mathcal{N}(\mathbb{0},\Theta_1^{-1})$ and $p_2\sim\mathcal{N}(\mathbb{0},\Theta_2^{-1})$ (note that $\Theta_1$, $\Theta_2$ are $n\times n$ positive definite matrices). Then the Bhattacharya distance between $p_1$ and $p_2$ can be expressed as:

\begin{align*} d_B(p_1,p_2)=\frac{1}{2} \ln\left(\frac{|\Theta_1+\Theta_2)/2|}{|\Theta_2|^{1/2}|\Theta_2|^{1/2}}\right). \end{align*}

I would prefer the lower bound is in terms of some matrix norm of $(\Theta_1-\Theta_2)$, for e.g., $||\Theta_1-\Theta_2||_1$ (L1 norm) or $||\Theta_1-\Theta_2||_2$ (L2 norm) or $||\Theta_1-\Theta_2||_F$ (Frobenius norm).

One may note that $2d_B(p_1,p_2)=D(p_1||p)+D(p_2||p)$, where $p=(p_1+p_2)/2$ and $D(\cdot||\cdot)$ is the KL-divergence function, but I could not make use of this property to get any answer.

$\endgroup$
3
  • $\begingroup$ What is $|\Theta|$? And by the $L^1$ and $L^2$ norms, do you mean the induced operator norms? $\endgroup$ Sep 21, 2012 at 18:42
  • $\begingroup$ $|\Theta|$ is the determinant of $\Theta$. By L1 and L2 norms, I meant matrix norms (mathworld.wolfram.com/MatrixNorm.html). $\endgroup$
    – adas
    Sep 21, 2012 at 20:11
  • 2
    $\begingroup$ Roughly speaking, if $\Theta_1^{-1}\Theta_2 \approx I$, then $d_B$ can be lower bounded by $\frac{1}{8}\|\Theta_1-\Theta_2\|^2$, but in general, there seems to be no uniform lower bound ... $\endgroup$
    – Suvrit
    Sep 23, 2012 at 11:45

1 Answer 1

2
$\begingroup$

The KL divergence is given by $$\hbox{Tr}(\Sigma_0\Sigma_1^{-1})- \ln(|\Sigma_0\Sigma_1^{-1}|)$$ Let $\mu_{-}$ be the smallest eigenvalue of $\Sigma_0\Sigma_1^{-1}$ and $\mu_{+}$ the largest, then

$$\mu_{-} = ||\Sigma_1 \Sigma_0^{-1} ||_2$$

$$\mu_{+} = ||\Sigma_0 \Sigma_1^{-1} ||_2$$

$$\hbox{Tr}(\Sigma_0\Sigma_1^{-1}) > n\mu_{-}$$ $$-\ln(|\Sigma_0\Sigma_1^{-1}|) > -n\ln(\mu_{+})$$ $$KL > n ||\Sigma_1\Sigma_0^{-1}||_2 -n\ln(||\Sigma_0\Sigma_1^{-1}||_2)$$

With the lower bound on the KL you can get a lower bound on the distance you're looking for. It doesn't seem you can get away with looking at the norm of the difference between the matrices.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.