3
$\begingroup$

Observation: Entropy is a metric over some non commutative ring. Indeed, if we exponentiate the standard entropy definition

$\displaystyle H(X) = -\sum_{x \in \mathcal{X}} p(x) \ln p(x).$

we'll get

$\displaystyle exp(H(X)) = -\prod_{x \in \mathcal{X}} p(x)^{p(x)}.$

The later is just a sum of squares if we interpret multiplication as summation and power as multiplication. This triggers all kind of questions, the most important one: can summation of probabilities be demoted and, possibly, entirely excluded from the theory?

Improve this question
$\endgroup$
3
  • 7
    $\begingroup$ Unfortunately, the $x^y$ operation is not just non-commutative. It is also non-associative (and non-Lie-bracket), so you won't get anything out of noncommutative algebra. Entropy is strange. Live with it. $\endgroup$
    – darij grinberg
    Commented Jan 20, 2011 at 18:49
  • 3
    $\begingroup$ Hi Tegiri, what is the motivation for this question? $\endgroup$
    – Kirill Shmakov
    Commented Oct 14, 2011 at 19:05
  • $\begingroup$ @darij grinberg: can you share a perspective on entropy from the point of view of non-associative algebra? I'd love to hear more about its strangeness. $\endgroup$
    – Tom LaGatta
    Commented Feb 26, 2013 at 6:11

1 Answer 1

Reset to default
2
$\begingroup$

Section 2.5 of David Ellerman's "Counting distinctions: on the conceptual foundations of Shannon’s information theory" overviews the history of the sum of probability squares formula known as Gini coefficient later consolidated with Shannon's definition via Tsallis entropy.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged .