Return to Question

For a given discrete probability distribution, Shannon entropy can be though as an expectation value $\langle - \log p \rangle$ (see also: What is entropy, really?, What is the role of the logarithm in Shannon's entropy? - Stats.SE).

Are higher moments of information $\langle (- \log p)^k \rangle$ ever used?

(I can imagine some scenarios where we are not interested in the average information, but its fluctuations or 'pessimistic'/'optimistic' scenarios. However, I don't remember seeing it in applied anywhere.)

Side note: they do carry the same, ehkm, information as Rényi entropy, as they are related by $$ 2^{z H_{1-z}(X)} = \sum_i p_i^{1-z} = \sum_i p_i 2^{-z \log p_i} = \sum_k \tfrac{z^k}{k!} \sum_i p_i (- \log p_i)^k. $$

EDIT: By "higher" I mean $k>1$. So, for example, is the variance of information used for something?

For a given discrete probability distribution, Shannon entropy can be though as an expectation value $\langle - \log p \rangle$ (see also: What is entropy, really?, What is the role of the logarithm in Shannon's entropy? - Stats.SE).

Are higher moments of information $\langle (- \log p)^k \rangle$ ever used?

(I can imagine some scenarios where we are not interested in the average information, but its fluctuations or 'pessimistic'/'optimistic' scenarios. However, I don't remember seeing it in applied anywhere.)

Side note: they do carry the same, ehkm, information as Rényi entropy, as they are related by $$ 2^{z H_{1-z}(X)} = \sum_i p_i^{1-z} = \sum_i p_i 2^{-z \log p_i} = \sum_k \tfrac{z^k}{k!} \sum_i p_i (- \log p_i)^k. $$

EDIT: By "higher" I mean $k>1$. So, for example, is the variance of information used for something?

For a given discrete probability distribution, Shannon entropy can be though as an expectation value $\langle - \log p \rangle$ (see also: What is entropy, really?, What is the role of the logarithm in Shannon's entropy? - Stats.SE).

Are higher moments of information $\langle (- \log p)^k \rangle$ ever used?

(I can imagine some scenarios where we are not interested in the average information, but its fluctuations or 'pessimistic'/'optimistic' scenarios. However, I don't remember seeing it in applied anywhere.)

Side note: they do carry the same, ehkm, information as Rényi entropy, as they are related by $$ 2^{z H_{1-z}(X)} = \sum_i p_i^{1-z} = \sum_i p_i 2^{-z \log p_i} = \sum_k \tfrac{z^k}{k!} \sum_i p_i (- \log p_i)^k. $$

EDIT: By "higher" I mean $k>1$. So, for example, is the variance of information used for something?

For a given discrete probability distribution, Shannon entropy can be though as an expectation value $\langle - \log p \rangle$ (see also: What is entropy, really?, What is the role of the logarithm in Shannon's entropy? - Stats.SE).

Are higher moments of information $\langle (- \log p)^k \rangle$ ever used?

(I can imagine some scenarios where we are not interested in the average information, but its fluctuations or 'pessimistic'/'optimistic' scenarios. However, I don't remember seeing it in applied anywhere.)

Side note: they do carry the same, ehkm, information as Rényi entropy, as they are related by $$ 2^{z H_{1-z}(X)} = \sum_i p_i^{1-z} = \sum_i p_i 2^{-z \log p_i} = \sum_k \tfrac{z^k}{k!} \sum_i p_i (- \log p_i)^k. $$

EDIT: By "higher" I mean $k>1$. So, for example, is the variance of information used for something?

For a given discrete probability distribution, Shannon entropy can be though as an expectation value $\langle - \log p \rangle$ (see also: What is entropy, really?, What is the role of the logarithm in Shannon's entropy? - Stats.SE).

Are higher moments of information $\langle (- \log p)^k \rangle$ ever used?

(I can imagine some scenarios where we are not interested in the average information, but its fluctuations or 'pessimistic'/'optimistic' scenarios. However, I don't remember seeing it in applied anywhere.)

Side note: they do carry the same, ehkm, information as Rényi entropy, as they are related by $$ 2^{z H_{1-z}(X)} = \sum_i p_i^{1-z} = \sum_i p_i 2^{-z \log p_i} = \sum_k \tfrac{z^k}{k!} \sum_i p_i (- \log p_i)^k. $$

For a given discrete probability distribution, Shannon entropy can be though as an expectation value $\langle - \log p \rangle$ (see also: What is entropy, really?, What is the role of the logarithm in Shannon's entropy? - Stats.SE).

Are higher moments of information $\langle (- \log p)^k \rangle$ ever used?

(I can imagine some scenarios where we are not interested in the average information, but its fluctuations or 'pessimistic'/'optimistic' scenarios. However, I don't remember seeing it in applied anywhere.)

Side note: they do carry the same, ehkm, information as Rényi entropy, as they are related by $$ 2^{z H_{1-z}(X)} = \sum_i p_i^{1-z} = \sum_i p_i 2^{-z \log p_i} = \sum_k \tfrac{z^k}{k!} \sum_i p_i (- \log p_i)^k. $$

EDIT: By "higher" I mean $k>1$. So, for example, is the variance of information used for something?