Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Observation: Entropy is a metric over some non commutative ring. Indeed, if we exponentiate the standard entropy definition

$\displaystyle H(X) = -\sum_{x \in \mathcal{X}} p(x) \ln p(x).$

we'll get

$\displaystyle exp(H(X)) = -\prod_{x \in \mathcal{X}} p(x)^{p(x)}.$

The later is just a sum of squares if we interpret multiplication as summation and power as multiplication. This triggers all kind of questions, the most important one: can summation of probabilities be demoted and, possibly, entirely excluded from the theory?

share|improve this question
6  
Unfortunately, the $x^y$ operation is not just non-commutative. It is also non-associative (and non-Lie-bracket), so you won't get anything out of noncommutative algebra. Entropy is strange. Live with it. –  darij grinberg Jan 20 '11 at 18:49
2  
Hi Tegiri, what is the motivation for this question? –  Kirill Shmakov Oct 14 '11 at 19:05
    
@darij grinberg: can you share a perspective on entropy from the point of view of non-associative algebra? I'd love to hear more about its strangeness. –  Tom LaGatta Feb 26 '13 at 6:11
add comment

1 Answer

Section 2.5 of David Ellerman's "Counting distinctions: on the conceptual foundations of Shannon’s information theory" overviews the history of the sum of probability squares formula known as Gini coefficient later consolidated with Shannon's definition via Tsallis entropy.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.