Let $G$ be a locally compact group. A measure $\mu$ is the right-Haar measure on $G$ if for every $g\in G$ and $E\subseteq G$ Borel set $\mu(Eg)=\mu(E)$. It is known that every locally compact group has a unique right-Haar measure (up to a multiplicative constant). Similarly one can define the left-Haar measure as being invariant for left shifts. However, in the case $G$ is compact this distinction is irrelevant, since both left and right Haar measures are equivalent up to a constant (note that $G$ is not nececssarily commutative, so being left and right Haar measures are in fact two different properties).

I am interested on a natural notion of entropy for measures on compact groups. In this case I expect the Haar measure to maximize the entropy. Is there an analog of information theory developed in this framework? I am particularly interested in the case where

I am particularly interested on the case where $G=O(n)$, the group of orthogonal transformations of $\mathbb{R}^n$.

Any references or insights are greatly welcome!

---------------------------- Update ------------------------------

Due to a request, I will further explain. Before I start, for notation and definitions I refer to Cover & Thomas book (Elements of Information Theory).

My question is very much related to Intrinsic significance of differential entropy: in the case of real random variables, differential entropy is troublesome due to the fact that there is no canonical reference space (in the discrete case, we are always working with r.v.'s over $\{1,\ldots,n\}$), so for example differential entropy is not invariant under re-scalings, since $$ h(a X) = h(X) + \log(a), $$ as opposed to the discrete case where $h(aX)=h(X)$.

However, in the case your r.v.'s are defined over a compact group $G$ (we may restrict here to the ones that are absolutely continuous w.r.t. Haar), the re-scaling problem disappears, as we have a reference space $G$. I wonder if having this reference space helps at all in terms of having information-theoretic identities/inequalities similar to the discrete case (e.g., Data Processing Inequality).

Finally, from what I have been reading lately, it seems that this is hopeless: just the fact of having an infinite measure space seems to break down all nice properties of the discrete entropy. However, I want to keep this question open to further comments from people with more acquaintance with information theory.

meaninglessbutirrelevantor something —compactness does not make the distinction lose its meaning! – Mariano Suárez-Alvarez♦ Apr 18 '14 at 19:26