2
$\begingroup$

Consider this setting:

$Y=X+N$

where $N$ is a Gaussian standard random variable and $X$ is another arbitrarily distributed r.v. You can think of this $X$ as a message being transmitted over an AWGN channel the output of which is the r.v. $Y$. I am wondering if anybody can introduce me some good resources on the connection between $MMSE = E[(X- E[X|Y])^2]$ and mutualinput-output information, namely $I(X:Y)= E[log \frac {p_{X,Y}}{p_X p_Y}]$

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

This must be extremely relevant

"Mutual Information and Minimum Mean-square Error in Gaussian Channels" by Dongning Guo, Shlomo Shamai (Shitz), and Sergio Verdu

http://arxiv.org/pdf/cs/0412108

Improve this answer
$\endgroup$
1
  • $\begingroup$ This is relevant Anadim. I had seen this paper before, but I am looking for a more detailed presentation of the stuff. By the way, very good reference! $\endgroup$
    – Farshid
    Commented May 29, 2011 at 16:23
1
$\begingroup$

You might want to read Section 6 of: Clustering with Bregman Divergences. Banerjee et al. In particular, in that section the authors develop a rate distortion theory using Bregman divergences.

The reason I mention this paper is because both squared Euclidean distance and KL-Divergence are Bregman divergences; Mutual information is the Bregman Information corresponding to KL-Divergence.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .