It seems that there are two separate definitions for the Fisher information, and I'm wondering if there is some relationship between the two.

The first is the so-called Fisher information which appears in some versions of the log-Sobolev inequality. It has the form $I(f) = \int_X \frac{|\nabla f|^2}{f} dx$. Notice that the derivatives are in the observation space $X$. This quantity seems to be important in functional analysis.

The second is the so called Fisher information metric or Fisher-Rao metric. For a parameterized family of probability densities $f(x,\theta)$, we can express it in the following way: $$g^{FR}_f(\theta_i, \theta_j) = \int_X \frac{\nabla_{\theta_i} f \,\nabla_{\theta_j} f}{f} dx.$$

Note that the derivatives here are in the statistical manifold, not the observation space. It's straightforward to generalize this to a non-parametrized model with Frechet derivatives. This metric is important in statistics and probability because it is in some sense a canonical metric.

What I'm trying to understand is whether there some relationship between these two. I can see that if $X$ is the real line, and we perturb $f$ by translating it, then the $I(f)$ and the norm of the translation in $g_f^{FR}$ are the same. However, I don't see much else relating them.

Does anyone have any pointers?

3more comments