8
$\begingroup$

(Migrating from math.stackexchange.com per commenter suggestion)

I was reading Baez, Fritz, and Leinster's "A Characterization of Entropy in Terms of Information Loss", and wondered if, instead of working in $\mathbf{FinMeas}$, we could work in any (e.g.) regular category, by categorifying conditional entropy. Actually, it seems easier to categorify perplexity (exponential entropy), which eliminates the logarithm and its arbitrary base.

The perplexity of a probability distribution $p$ is $$PP(p)=\frac{1}{\prod_x{p_x}^{p_x}}$$

In $\textbf{Grp}$, the above formula seems to suggest defining the [conditional] perplexity of a morphism $f$ as something like $\operatorname{End}(\operatorname{ker}(f))$. My vague intuition is that the kernel is what is lost by $f$, and its endomorphism monoid determines its information content since any morphism in $\textbf{Grp}$ factors into an endomorphism followed by a monomorphism.

However, even if $\operatorname{End}(\operatorname{ker}(f))$ happens to be the right way to define perplexity in $\textbf{Grp}$, maybe it only works because the cosets of a kernel are all isomorphic, and if we want it to work more generally we'd need to do something like $\frac{\operatorname{End}(\operatorname{dom} f)}{\operatorname{End}(\operatorname{im} f)}$, if that even makes sense.

Does anyone have any insights as to the "right" way to categorify entropy/perplexity, or any references that might be helpful?

A bonus question is: why is entropy so much more popular than perplexity? Do people just like addition more than multiplication? Historical accident? It's a bit...confusing.

$\endgroup$
2
  • 2
    $\begingroup$ concerning the bonus question: see mathoverflow.net/q/398217/11260 $\endgroup$ Oct 10, 2022 at 21:18
  • 8
    $\begingroup$ Sorry for the self-promotion, but since you ask for references, I wrote a book on just this sort of thing: "Entropy and diversity: the axiomatic approach" (free online version available: maths.ed.ac.uk/~tl/ed). I hadn't heard the term "perplexity" before today; in my book, it's referred to as diversity of order 1. $\endgroup$ Oct 10, 2022 at 21:50

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.