# Rate-Distortion theory: What is the distribution of distortion on an optimal Gaussian encoder?

If we wish to encode a gaussian source, $X\sim\mathcal{N}(0,\sigma^2)$ at rate $R$, then decode it to create an estimate $\hat{X}$, rate-distortion theory tells us that the lowest mean-squared-error we can achieve through the encoder-decoder is $\sigma^2 2^{-2R}.$ That is, using an optimal encoder and decoder, we can achieve $E(X-\hat{X})^2=\sigma^2 2^{-2R}.$ (possibly asymptotically as we let code block lengths increase)

What might the distribution of $\hat{X}|_{X=x}$ be, for such encoders and decoders? That is, how is our estimate distributed around the right answer?

For illustration, we know obviously that no choice of enc/dec could yield a deterministic $\hat{X}|_{X=x}=x+\sqrt{\sigma^22^{-2R}},$ even though this distribution would give us the correct mean-squared-error. (Otherwise we could design a lossless decoder!)

I believe that it should be possible to design some random coding scheme where, as the code block lengths get large, eventually each $i$th message in our block, $\hat{X_i}|_{X_i=x_i}$, becomes close in distribution to $\mathcal{N}(x_i,\sigma^22^{-2R}).$

Equivalently, I am trying to find a sequence of encoders and decoders where:

$$D(p^n||q^n)\rightarrow 0, \text{ as }n \uparrow \infty$$

where $x^n$ denotes a block of $n$ messages, $\hat{X}^n$ is the decoded estimate of all these $n$ messages, $p^n$ is the distribution of ${\hat{X}^n|_ {X^n=x^n}}$ and $q^n$ is the distribution of $\mathcal{N}(x^n,\sigma^22^{-2R}I_{n\times n})$.

Finding such an encoder/decoder pair would essentially allow us to properly say that quantization of a gaussian source can be modeled as gaussian noise.

• This is expected, because the test channel that minimizes rate in the construction of $R(D)$ for a Gaussian source $X$ is: $$X \longrightarrow \otimes \longrightarrow \!\!\!\! \underset{\underset{N\sim \mathcal{N}(0,1)}{\big\uparrow}}{\oplus} \!\!\!\! \longrightarrow \otimes \longrightarrow \alpha \cdot (\alpha X+\beta N),$$ with $N\perp X, \ \alpha = \sqrt{1-D/\sigma_X^2}$ and $\beta=\sqrt{D/\sigma_X^2}.$ Feb 16, 2016 at 19:27