Skip to main content
MathOverflow

Return to Question

Bumped by Community user
formatting, added tags
Source Link
YCor
YCor
  • 63.9k
  • 5
  • 187
  • 286

relation Relation between multivariate estimation error and differential entropy

On page 255 of the book "Elements of information theory" by Thomas M. Cover and Joy A. Thomas, there is a theorem: For any random variable $X$ and estimator $\hat{X}$, $E(X-\hat{X})^2 \geq min_{\hat{X}} E(X-\hat{X})^2 = E(X-E(X))^2 = var(X) \geq \frac{1}{2\pi e}e^{2h(X)}$.$$E(X-\hat{X})^2 \geq \min_{\hat{X}} E(X-\hat{X})^2 = E(X-E(X))^2 = \mathrm{var}(X) \geq \frac{1}{2\pi e}e^{2h(X)}.$$ Right after theorem there is a corollary relating estimation error and differential entropy: Given random variable $X$, side information $Y$ and estimator $\hat{X}(Y)$, $E(X-\hat{X}(Y))^2 \geq \frac{1}{2\pi e}e^{2h(X|Y)}$.

However, I'm not sure whether the corollary is true for multi-variables: Given multi random variable $X \in \mathbb{R}^m, m > 1 $, side information $Y \in \mathbb{R}^n, n \geq 1 $ and estimator $\hat{X}(Y)$, $E(|X-\hat{X}(Y)|^2) \geq \frac{1}{2\pi e}e^{2h(X|Y)}$, where $|\cdot|$ denotes the 2-norm.

relation between multivariate estimation error and differential entropy

On page 255 of the book "Elements of information theory" by Thomas M. Cover and Joy A. Thomas, there is a theorem: For any random variable $X$ and estimator $\hat{X}$, $E(X-\hat{X})^2 \geq min_{\hat{X}} E(X-\hat{X})^2 = E(X-E(X))^2 = var(X) \geq \frac{1}{2\pi e}e^{2h(X)}$. Right after theorem there is a corollary relating estimation error and differential entropy: Given random variable $X$, side information $Y$ and estimator $\hat{X}(Y)$, $E(X-\hat{X}(Y))^2 \geq \frac{1}{2\pi e}e^{2h(X|Y)}$.

However, I'm not sure whether the corollary is true for multi-variables: Given multi random variable $X \in \mathbb{R}^m, m > 1 $, side information $Y \in \mathbb{R}^n, n \geq 1 $ and estimator $\hat{X}(Y)$, $E(|X-\hat{X}(Y)|^2) \geq \frac{1}{2\pi e}e^{2h(X|Y)}$, where $|\cdot|$ denotes the 2-norm.

Relation between multivariate estimation error and differential entropy

On page 255 of the book "Elements of information theory" by Thomas M. Cover and Joy A. Thomas, there is a theorem: For any random variable $X$ and estimator $\hat{X}$, $$E(X-\hat{X})^2 \geq \min_{\hat{X}} E(X-\hat{X})^2 = E(X-E(X))^2 = \mathrm{var}(X) \geq \frac{1}{2\pi e}e^{2h(X)}.$$ Right after theorem there is a corollary relating estimation error and differential entropy: Given random variable $X$, side information $Y$ and estimator $\hat{X}(Y)$, $E(X-\hat{X}(Y))^2 \geq \frac{1}{2\pi e}e^{2h(X|Y)}$.

However, I'm not sure whether the corollary is true for multi-variables: Given multi random variable $X \in \mathbb{R}^m, m > 1 $, side information $Y \in \mathbb{R}^n, n \geq 1 $ and estimator $\hat{X}(Y)$, $E(|X-\hat{X}(Y)|^2) \geq \frac{1}{2\pi e}e^{2h(X|Y)}$, where $|\cdot|$ denotes the 2-norm.

Source Link
CASIA-LH
CASIA-LH
  • 21
  • 1

relation between multivariate estimation error and differential entropy

On page 255 of the book "Elements of information theory" by Thomas M. Cover and Joy A. Thomas, there is a theorem: For any random variable $X$ and estimator $\hat{X}$, $E(X-\hat{X})^2 \geq min_{\hat{X}} E(X-\hat{X})^2 = E(X-E(X))^2 = var(X) \geq \frac{1}{2\pi e}e^{2h(X)}$. Right after theorem there is a corollary relating estimation error and differential entropy: Given random variable $X$, side information $Y$ and estimator $\hat{X}(Y)$, $E(X-\hat{X}(Y))^2 \geq \frac{1}{2\pi e}e^{2h(X|Y)}$.

However, I'm not sure whether the corollary is true for multi-variables: Given multi random variable $X \in \mathbb{R}^m, m > 1 $, side information $Y \in \mathbb{R}^n, n \geq 1 $ and estimator $\hat{X}(Y)$, $E(|X-\hat{X}(Y)|^2) \geq \frac{1}{2\pi e}e^{2h(X|Y)}$, where $|\cdot|$ denotes the 2-norm.