Minimum Mean Square Error (MMSE) and Mutual Information (I)

Question

Consider this setting:

$Y=X+N$

where $N$ is a Gaussian standard random variable and $X$ is another arbitrarily distributed r.v. You can think of this $X$ as a message being transmitted over an AWGN channel the output of which is the r.v. $Y$. I am wondering if anybody can introduce me some good resources on the connection between $MMSE = E[(X- E[X|Y])^2]$ and mutualinput-output information, namely $I(X:Y)= E[log \frac {p_{X,Y}}{p_X p_Y}]$

Answer 1

This must be extremely relevant

"Mutual Information and Minimum Mean-square Error in Gaussian Channels" by Dongning Guo, Shlomo Shamai (Shitz), and Sergio Verdu

http://arxiv.org/pdf/cs/0412108

Answer 2

You might want to read Section 6 of: Clustering with Bregman Divergences. Banerjee et al. In particular, in that section the authors develop a rate distortion theory using Bregman divergences.

The reason I mention this paper is because both squared Euclidean distance and KL-Divergence are Bregman divergences; Mutual information is the Bregman Information corresponding to KL-Divergence.