MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Observation: Entropy is a metric over some non commutative ring. Indeed, if we exponentiate the standard entropy definition

$\displaystyle H(X) = -\sum_{x \in \mathcal{X}} p(x) \ln p(x).$

we'll get

$\displaystyle exp(H(X)) = -\prod_{x \in \mathcal{X}} p(x)^{p(x)}.$

The later is just a sum of squares if we interpret multiplication as summation and power as multiplication. This triggers all kind of questions, the most important one: can summation of probabilities be demoted and, possibly, entirely excluded from the theory?

share|cite|improve this question
Unfortunately, the $x^y$ operation is not just non-commutative. It is also non-associative (and non-Lie-bracket), so you won't get anything out of noncommutative algebra. Entropy is strange. Live with it. – darij grinberg Jan 20 '11 at 18:49
Hi Tegiri, what is the motivation for this question? – Kirill Shmakov Oct 14 '11 at 19:05
@darij grinberg: can you share a perspective on entropy from the point of view of non-associative algebra? I'd love to hear more about its strangeness. – Tom LaGatta Feb 26 '13 at 6:11

Section 2.5 of David Ellerman's "Counting distinctions: on the conceptual foundations of Shannon’s information theory" overviews the history of the sum of probability squares formula known as Gini coefficient later consolidated with Shannon's definition via Tsallis entropy.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.