I have two Gaussian random variables $$X \sim \mathcal N(0, I), \ \ \ W \sim \mathcal N(0, \sigma\cdot I)$$ and I known a parametric change-of-variable $Y(\theta) = T(X; \theta)$.
I would like to find the differential entropy $h(X) = - \int_{\Omega} p(x) \log p(x) dx$ of the transformed $X$ with that additive noise $W$:
$$Z(\theta) = T(X; \theta) + W \\ h(Z(\theta)) = \ ?$$
or at least how it depends on $\theta$, i.e. the solution in the form $h(Z(\theta)) = C + g(\theta)$ would suffice (up to an unknown constant).
We know how the differential entropy behaves under the change-of-variable: $$ h(T(X))= h(X) - \mathbb E_{x\sim X} \log \det |\nabla T(x)|$$
So I tried to apply the Taylor expansion to pull the noise term into the first argument of $T$, i.e. something like
$$ T(X; \theta) + W = T\big(X + \nabla T^{-1}(X;\theta) \cdot W; \theta \big) + o(||\nabla T^{-1}||_{\infty})$$
and then $$h(Z(\theta)) = h(T(X; \theta) + W)\approx h(T\big(X + \nabla T^{-1}(X) \cdot W; \theta \big)) \\ = h(X + \nabla T^{-1}(X;\theta) \cdot W) - \mathbb E_{x\sim X} \log\det|\nabla T(x + \nabla T^{-1}(x;\theta) \cdot W;\theta)| \\ = h(X + \nabla T^{-1}(X;\theta) \cdot W) - \mathbb E_{x\sim X} \log\det|\nabla T(x;\theta)| \ + \ o(||\nabla^2 T^{-1}||_{\infty})$$
where the first term could be estimated from the entropy of a Gaussian. But even in 1D numerical experiments showed that this seems to be a very crude approximation unless $T$ is very smooth.
I was wondering if there were any cool identities like the de Bruijn’s identity or the one described in this paper that could help me estimate exact entropy of $h(Z(\theta))$?
