Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
Put another way: after conditioning on the encoding, the source letters are no longer necessarily independent. This question's objective function keeps track of this non-independence, the source coding problem does not.
Tom, thanks for the proof. I think this is actually subtly different than source coding under logarithmic loss. Choosing the best encoder here is minimizing a different objective: in this case (1) the "loss function" simultaneously depends on all $n$ source letters and the encoder word in some complex way. in the source coding problem (2) the loss function is an average of $n$ identical single-letter losses. Your proof shows that the minima of the two problems are the same.
One interpretation of MGL is that outputs of the form (codeword + noise) expose more of the system's entropy when the input codebook is structured compared to when the codebook is all the words with iid letters. (In the first case you can (partially) decode and get a noise vector estimate, in the second case the codeword and noise are undifferentiated.) $${}$$ In light of this, to approach your question I would first try and understand more precisely how your $X$ falls between a "optimal-for-decoding" dictionary and the "noise" dictionary.
