Question

Observation: Entropy is a metric over some non commutative ring. Indeed, if we exponentiate the standard entropy definition

$\displaystyle H(X) = -\sum_{x \in \mathcal{X}} p(x) \ln p(x).$

we'll get

$\displaystyle exp(H(X)) = -\prod_{x \in \mathcal{X}} p(x)^{p(x)}.$

The later is just a sum of squares if we interpret multiplication as summation and power as multiplication. This triggers all kind of questions, the most important one: can summation of probabilities be demoted and, possibly, entirely excluded from the theory?

Answer 1

Section 2.5 of David Ellerman's "Counting distinctions: on the conceptual foundations of Shannon’s information theory" overviews the history of the sum of probability squares formula known as Gini coefficient later consolidated with Shannon's definition via Tsallis entropy.