# What is the relationship between the Fisher Information and the Fisher Information metric?

It seems that there are two separate definitions for the Fisher information, and I'm wondering if there is some relationship between the two.

The first is the so-called Fisher information which appears in some versions of the log-Sobolev inequality. It has the form $I(f) = \int_X \frac{|\nabla f|^2}{f} dx$. Notice that the derivatives are in the observation space $X$. This quantity seems to be important in functional analysis.

The second is the so called Fisher information metric or Fisher-Rao metric. For a parameterized family of probability densities $f(x,\theta)$, we can express it in the following way: $$g^{FR}_f(\theta_i, \theta_j) = \int_X \frac{\nabla_{\theta_i} f \,\nabla_{\theta_j} f}{f} dx.$$

Note that the derivatives here are in the statistical manifold, not the observation space. It's straightforward to generalize this to a non-parametrized model with Frechet derivatives. This metric is important in statistics and probability because it is in some sense a canonical metric.

What I'm trying to understand is whether there some relationship between these two. I can see that if $X$ is the real line, and we perturb $f$ by translating it, then the $I(f)$ and the norm of the translation in $g_f^{FR}$ are the same. However, I don't see much else relating them.

Does anyone have any pointers?

• Possibly relevant: stats.stackexchange.com/questions/154724/… – SMD Jun 5 '18 at 18:39
• Thanks for the reference. However, that question is entirely about the second type of Fisher information, where the derivatives are in the statistical manifold and doesn't use the first type, where the derivatives are in the state space. – Gabe K Jun 6 '18 at 13:01
• Have you heard about the fact that the Fisher information functional is the derivative of entropy along the heat semigroup? – S.Surace Jul 10 '18 at 20:51
• Thanks for bringing that up. That's actually exactly what I've been looking into recently. It appears this yields some interesting relationships, but let me work out the details before I say anything too publicly. – Gabe K Jul 11 '18 at 2:24
• After spending some more times with these concepts, I think the deepest that I can say so far is: the Fisher information functional is the derivative of the entropy along the heat semigroup, while the Fisher-Rao metric is the Hessian of the relative entropy $H(\nu||\mu)$ wrt. the first argument. I don't know whether this helps your quest. If you want, I can add another answer to expand on this. – S.Surace Oct 27 '18 at 13:56

Let $$X$$ be a smooth manifold. The Fisher-Rao metric is invariant under the action of diffeomorphisms by pushforward, while the Fisher information functional $$I$$ is not; even under scalings on $$X=\mathbb{R}^n$$, the squared norm of the gradient will pick up a non-trivial scaling factor.