MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Observation: Entropy is a metric over some non commutative ring. Indeed, if we exponentiate the standard entropy definition

$\displaystyle H(X) = -\sum_{x \in \mathcal{X}} p(x) \ln p(x).$

we'll get

$\displaystyle exp(H(X)) = -\prod_{x \in \mathcal{X}} p(x)^{p(x)}.$

The later is just a sum of squares if we interpret multiplication as summation and power as multiplication. This triggers all kind of questions, the most important one: can summation of probabilities be demoted and, possibly, entirely excluded from the theory?

share|cite|improve this question

closed as unclear what you're asking by Qiaochu Yuan, Alex Degtyarev, abx, Anton, Daniel Moskovich Apr 3 at 10:12

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question.If this question can be reworded to fit the rules in the help center, please edit the question.

7  
Unfortunately, the $x^y$ operation is not just non-commutative. It is also non-associative (and non-Lie-bracket), so you won't get anything out of noncommutative algebra. Entropy is strange. Live with it. – darij grinberg Jan 20 '11 at 18:49
3  
Hi Tegiri, what is the motivation for this question? – Kirill Shmakov Oct 14 '11 at 19:05
    
@darij grinberg: can you share a perspective on entropy from the point of view of non-associative algebra? I'd love to hear more about its strangeness. – Tom LaGatta Feb 26 '13 at 6:11

Section 2.5 of David Ellerman's "Counting distinctions: on the conceptual foundations of Shannon’s information theory" overviews the history of the sum of probability squares formula known as Gini coefficient later consolidated with Shannon's definition via Tsallis entropy.

share|cite|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.