# Higher moments of information and Renyi entropy

For a given discrete probability distribution, Shannon entropy can be though as an expectation value $\langle - \log p \rangle$ (see also: What is entropy, really?, What is the role of the logarithm in Shannon's entropy? - Stats.SE).

Are higher moments of information $\langle (- \log p)^k \rangle$ ever used?

(I can imagine some scenarios where we are not interested in the average information, but its fluctuations or 'pessimistic'/'optimistic' scenarios. However, I don't remember seeing it in applied anywhere.)

Side note: they do carry the same, ehkm, information as Rényi entropy, as they are related by $$2^{z H_{1-z}(X)} = \sum_i p_i^{1-z} = \sum_i p_i 2^{-z \log p_i} = \sum_k \tfrac{z^k}{k!} \sum_i p_i (- \log p_i)^k.$$

EDIT: By "higher" I mean $k>1$. So, for example, is the variance of information used for something?

• what's 'ehkm'?? Commented Dec 18, 2014 at 20:59
• @Memming An utterance to express that here it is awkward to use information in another, informal meaning. Commented Dec 18, 2014 at 22:21