Fisher Information of variance of difference between random variable and gaussian

I'm reading through the following paper: https://arxiv.org/pdf/0704.1751.pdf

I'm stuck in the middle of page 8, at the statement:

$$E[||S(X)-S^*(X)||^2] = J(X) - J(X^*)$$

Where $$S(X)$$ is the score of the random vector $$X$$, $$S^*(X)$$ is the score of the gaussian r.v. with the same covariance as $$X$$, and $$J(X)$$ is the Fisher information of $$X$$ defined to be $$\text{tr}(\text{Cov}(S(X), S(X)))$$.

What I have so far is:

\begin{align} E[||S(X)-S^*(X)||^2] &= E[||S(X)||^2 - 2(S(X)\cdot S^*(X)) + ||S^*(X)||^2]\\ &= E[||S(X)||^2] - 2E[S(X)\cdot S^*(X)] + E[||S^*(X)||^2]\\ &= J(X) - 2E[S(X)\cdot S^*(X)] + J(X^*) \end{align}

From this point it suffices to show that: $$E[S(X)\cdot S^*(X)] = J(X^*)$$

It can be shown that the score of the associated Gaussian is $$-K^{-1}x$$, where $$K=\text{Cov}(X),$$ so in other words, we look to show: $$E[S(X)\cdot (-K^{-1}X)] = E[(K^{-1}X)\cdot (K^{-1}X)]$$

How can this be shown? Or is there an alternative way to show this?

Thanks.

• Do you have a response to the answer below? Mar 6, 2022 at 3:26

$$\newcommand{\R}{\mathbb R}\newcommand{\tr}{\operatorname{tr}}$$Let $$p$$ be the pdf of $$X$$, so that $$\begin{equation*} S(x)=\frac{\nabla p(x)}{p(x)} \tag{1}\label{1} \end{equation*}$$ for all $$x\in\R^n$$ and $$\begin{equation*} \int_{\R^n} p(x-t)x\,dx=[E(X+t)=]t \tag{2}\label{2} \end{equation*}$$ for all $$t\in\R^n$$.
If (say) $$\int|\nabla p(x)|\,(|x|+1)dx<\infty$$, then, differentiating \eqref{2} in $$t$$ at $$t=0$$ and using the Fubini theorem, we get $$\begin{equation*} -\int_{\R^n} \nabla p(x)x^\top\,dx=I, \tag{3}\label{3} \end{equation*}$$ where $$I$$ is the identity matrix; that is, $$\begin{equation*} ES(X)X^\top=-I=EX S(X)^\top. \end{equation*}$$
So, $$\begin{equation*} ES(X)\cdot(K^{-1}X)=ES(X)^\top K^{-1}X=E\tr S(X)^\top K^{-1}X =E\tr K^{-1}XS(X)^\top=\tr K^{-1}EXS(X)^\top =-\tr K^{-1}. \end{equation*}$$ Also, $$\begin{equation*} E(K^{-1}X)\cdot(K^{-1}X)=EX^\top K^{-2}X=E\tr K^{-2}XX^\top =\tr K^{-2}EXX^\top =\tr K^{-1}. \end{equation*}$$
Thus, indeed $$\begin{equation*} ES(X)\cdot(-K^{-1}X)=E(K^{-1}X)\cdot(K^{-1}X). \end{equation*}$$