Question

I am interested in a lower bound on the Bhattacharya distance between two independent multivariate Gaussian distributions. To be precise, consider zero-mean independent Gaussian distributions $p_1\sim\mathcal{N}(\mathbb{0},\Theta_1^{-1})$ and $p_2\sim\mathcal{N}(\mathbb{0},\Theta_2^{-1})$ (note that $\Theta_1$, $\Theta_2$ are $n\times n$ positive definite matrices). Then the Bhattacharya distance between $p_1$ and $p_2$ can be expressed as:

\begin{align*} d_B(p_1,p_2)=\frac{1}{2} \ln\left(\frac{|\Theta_1+\Theta_2)/2|}{|\Theta_2|^{1/2}|\Theta_2|^{1/2}}\right). \end{align*}

I would prefer the lower bound is in terms of some matrix norm of $(\Theta_1-\Theta_2)$, for e.g., $||\Theta_1-\Theta_2||_1$ (L1 norm) or $||\Theta_1-\Theta_2||_2$ (L2 norm) or $||\Theta_1-\Theta_2||_F$ (Frobenius norm).

One may note that $2d_B(p_1,p_2)=D(p_1||p)+D(p_2||p)$, where $p=(p_1+p_2)/2$ and $D(\cdot||\cdot)$ is the KL-divergence function, but I could not make use of this property to get any answer.

Answer 1

The KL divergence is given by $$\hbox{Tr}(\Sigma_0\Sigma_1^{-1})- \ln(|\Sigma_0\Sigma_1^{-1}|)$$ Let $\mu_{-}$ be the smallest eigenvalue of $\Sigma_0\Sigma_1^{-1}$ and $\mu_{+}$ the largest, then

$$\mu_{-} = ||\Sigma_1 \Sigma_0^{-1} ||_2$$

$$\mu_{+} = ||\Sigma_0 \Sigma_1^{-1} ||_2$$

$$\hbox{Tr}(\Sigma_0\Sigma_1^{-1}) > n\mu_{-}$$ $$-\ln(|\Sigma_0\Sigma_1^{-1}|) > -n\ln(\mu_{+})$$ $$KL > n ||\Sigma_1\Sigma_0^{-1}||_2 -n\ln(||\Sigma_0\Sigma_1^{-1}||_2)$$

With the lower bound on the KL you can get a lower bound on the distance you're looking for. It doesn't seem you can get away with looking at the norm of the difference between the matrices.