Skip to main content
1 of 2

relation between multivariate estimation error and differential entropy

On page 255 of the book "Elements of information theory" by Thomas M. Cover and Joy A. Thomas, there is a theorem: For any random variable $X$ and estimator $\hat{X}$, $E(X-\hat{X})^2 \geq min_{\hat{X}} E(X-\hat{X})^2 = E(X-E(X))^2 = var(X) \geq \frac{1}{2\pi e}e^{2h(X)}$. Right after theorem there is a corollary relating estimation error and differential entropy: Given random variable $X$, side information $Y$ and estimator $\hat{X}(Y)$, $E(X-\hat{X}(Y))^2 \geq \frac{1}{2\pi e}e^{2h(X|Y)}$.

However, I'm not sure whether the corollary is true for multi-variables: Given multi random variable $X \in \mathbb{R}^m, m > 1 $, side information $Y \in \mathbb{R}^n, n \geq 1 $ and estimator $\hat{X}(Y)$, $E(|X-\hat{X}(Y)|^2) \geq \frac{1}{2\pi e}e^{2h(X|Y)}$, where $|\cdot|$ denotes the 2-norm.