## Question:

Might there be a natural geometric interpretation of the exponential of entropy in Classical and Quantum Information theory? This question occurred to me recently via a geometric inequality concerning the exponential of the Shannon entropy.

## Original motivation:

The weighted AM-GM inequality states that if $\{a_i\}_{i=1}^n,\{\lambda_i\}_{i=1}^n \in \mathbb{R}_+^n$ and $\sum_{i=1}^n \lambda_i = 1$, then:

\begin{equation} \prod_{i=1}^n a_i^{\lambda_i} \leq \sum_{i=1}^n \lambda_i \cdot a_i \tag{1} \end{equation}

As an application, we find that if $H(\vec{p})$ denotes the Shannon entropy of a discrete probability distribution $\vec{p} = \{p_i\}_{i=1}^n$ and $r_p^2 = \lVert \vec{p} \rVert^2 $ is the $l_2$ norm of $\vec{p}$ then:

\begin{equation} e^{H(\vec{p})} \geq \frac{1}{r_p^2} \tag{2} \end{equation}

This result follows from the observation that if $a_i = p_i$ and $\lambda_i = p_i$,

\begin{equation} e^{-H(\vec{p})} = e^{\sum_i p_i \ln p_i} = \prod_{i=1}^n p_i^{p_i} \tag{3} \end{equation}

\begin{equation} \sum_{i=1}^n p_i^2 = \lVert \vec{p} \rVert^2 \tag{4} \end{equation}

and using (1), we may deduce (2) where equality is obtained when the Shannon entropy is maximised by the uniform distribution i.e. $\forall i, p_i = \frac{1}{n}$.

## A remark on appropriate geometric embeddings:

If we consider that the Shannon entropy measures the quantity of hidden information in a stochastic system at the state $\vec{p} \in [0,1]^n$, we may define the level sets $\mathcal{L}_q$ in terms of the typical probability $q \in (0,1)$:

\begin{equation} \mathcal{L}_q = \{\vec{p} \in [0,1]^n: e^{H(\vec{p})} = e^{- \ln q} \} \tag{5} \end{equation}

which allows us to define an equivalence relation over states $\vec{p} \in [0,1]^n$. Such a model is appropriate for events which may have $n$ distinct outcomes.

Now, we'll note that $e^{H(\vec{p})}$ has a natural interpretation as a measure of hidden information while $e^{-H(\vec{p})}$ may be interpreted as the typical probability of the state $\vec{p}$. Given (5), a natural relation between these measures may be found using the Hyperbolic identities:

\begin{equation} \cosh^2(-\ln q) - \sinh^2(-\ln q) = 1 \tag{6} \end{equation}

\begin{equation} \cosh(-\ln q) - \sinh(-\ln q) = q \tag{7} \end{equation}

where $2 \cdot \cosh(-\ln q)$ is the sum of these two measures and $2 \cdot \sinh(-\ln q)$ may be understood as their difference. This suggests that the level sets $\mathcal{L}_q$ have a natural Hyperbolic embedding in terms of Hyperbolic functions.

## References:

Olivier Rioul. This is IT: A Primer on Shannon’s Entropy and Information. Séminaire Poincaré. 2018.

David J.C. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press 2003.

John C. Baez, Tobias Fritz, Tom Leinster. A Characterization of Entropy in Terms of Information Loss. Arxiv. 2011.