Consider a real-valued random variable $X$ with mean $0$ and variance $1$ and let $Z$ be an independent standard Gaussian random variable. Now consider the stochastic process $Y(t) = \sin(t) X + \cos(t) Z $ for $t \in [0, \frac{\pi}{2}]$. We will be interested in $S(Y(t))$, the relative entropy of $Y(t)$ with respect to the Gaussian, more specifically $ S(Y(t)) = \int \frac{d\mu}{d\nu} \log\left( \frac{d\mu}{d\nu}\right)d\nu$ where $ \nu $ is the standard Gaussian probability measure and $\mu$ is the probability measure induced by $Y(t)$.

From the entropy power inequalities for relative entropy, we can get upper bounds on $S(Y(t))$. Is there any general result or technique to lower bound the relative entropy of $Y(t)$ under reasonable conditions on $X$? Specifically, we are interested in the following case: $S(X) \geq \epsilon$ for some positive constant $\epsilon$, and so $S(Y(\pi/2)) \geq \epsilon$. Moreover, $S(Y(0))=0$. We would like to obtain a lower bound on $S(Y(t))$ as a function of $t$ as $t \rightarrow 0$. If needed, we can assume that $X$ is nice in various ways, e.g., it has bounded density and is sub-Gaussian.

  • $\begingroup$ If you are willing to scale $Y(t)$ so that it's power is 1, Lemma 1 (then Lemma 2 and the rest of the paper) might help, as it turns bounding $S(Y(t)/\sqrt{\langle Y(t)^2\rangle})$ into bounding two mutual informations. sciencedirect.com/science/article/pii/S0019995878904138 $\endgroup$ Oct 25 '17 at 23:07
  • $\begingroup$ Thank You. I failed to mention in the original post that, $X$ can be taken to be mean 0 and variance 1, thus we are taking a power preserving combination. I have edited the post to reflect this. $\endgroup$ Oct 26 '17 at 6:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.