2
$\begingroup$

I am interested in a lower bound on the Bhattacharya distance between two independent multivariate Gaussian distributions. To be precise, consider zero-mean independent Gaussian distributions $p_1\sim\mathcal{N}(\mathbb{0},\Theta_1^{-1})$ and $p_2\sim\mathcal{N}(\mathbb{0},\Theta_2^{-1})$ (note that $\Theta_1$, $\Theta_2$ are $n\times n$ positive definite matrices). Then the Bhattacharya distance between $p_1$ and $p_2$ can be expressed as:

\begin{align*} d_B(p_1,p_2)=\frac{1}{2} \ln\left(\frac{|\Theta_1+\Theta_2)/2|}{|\Theta_2|^{1/2}|\Theta_2|^{1/2}}\right). \end{align*}

I would prefer the lower bound is in terms of some matrix norm of $(\Theta_1-\Theta_2)$, for e.g., $||\Theta_1-\Theta_2||_1$ (L1 norm) or $||\Theta_1-\Theta_2||_2$ (L2 norm) or $||\Theta_1-\Theta_2||_F$ (Frobenius norm).

One may note that $2d_B(p_1,p_2)=D(p_1||p)+D(p_2||p)$, where $p=(p_1+p_2)/2$ and $D(\cdot||\cdot)$ is the KL-divergence function, but I could not make use of this property to get any answer.

Improve this question
$\endgroup$
3
  • $\begingroup$ What is $|\Theta|$? And by the $L^1$ and $L^2$ norms, do you mean the induced operator norms? $\endgroup$
    – Mark Meckes
    Commented Sep 21, 2012 at 18:42
  • $\begingroup$ $|\Theta|$ is the determinant of $\Theta$. By L1 and L2 norms, I meant matrix norms (mathworld.wolfram.com/MatrixNorm.html). $\endgroup$
    – adas
    Commented Sep 21, 2012 at 20:11
  • 2
    $\begingroup$ Roughly speaking, if $\Theta_1^{-1}\Theta_2 \approx I$, then $d_B$ can be lower bounded by $\frac{1}{8}\|\Theta_1-\Theta_2\|^2$, but in general, there seems to be no uniform lower bound ... $\endgroup$
    – Suvrit
    Commented Sep 23, 2012 at 11:45

1 Answer 1

Reset to default
2
$\begingroup$

The KL divergence is given by $$\hbox{Tr}(\Sigma_0\Sigma_1^{-1})- \ln(|\Sigma_0\Sigma_1^{-1}|)$$ Let $\mu_{-}$ be the smallest eigenvalue of $\Sigma_0\Sigma_1^{-1}$ and $\mu_{+}$ the largest, then

$$\mu_{-} = ||\Sigma_1 \Sigma_0^{-1} ||_2$$

$$\mu_{+} = ||\Sigma_0 \Sigma_1^{-1} ||_2$$

$$\hbox{Tr}(\Sigma_0\Sigma_1^{-1}) > n\mu_{-}$$ $$-\ln(|\Sigma_0\Sigma_1^{-1}|) > -n\ln(\mu_{+})$$ $$KL > n ||\Sigma_1\Sigma_0^{-1}||_2 -n\ln(||\Sigma_0\Sigma_1^{-1}||_2)$$

With the lower bound on the KL you can get a lower bound on the distance you're looking for. It doesn't seem you can get away with looking at the norm of the difference between the matrices.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .