I wonder if the following idea has any real world, or perhaps mathematical, applications.
For simplicity, suppose you are interested in two different real variables, which we will denote by $X$ and $Y$. You have made $N$ observations of $(x, y)$, thus resulting in $N$ data points in the $xy$-plane. One can represent the data by a real $2 \times N$ matrix.
My first remark is related to the following quantity: $X - E[X]$ (of course, a similar discussion would hold for $Y - E[Y]$). This corresponds to $$ \vec{X} - \frac{\sum_{i=1}^N x_i}{N}\left(1, \ldots, 1\right)^T = \vec{X} - (\vec{X}, \vec{v}) \vec{v},$$ where $$ \vec{v} = \frac{1}{\sqrt{N}}(1, \ldots, 1)^T$$ and $T$ denotes the transpose. Here $\vec{X}$ denotes the $N \times 1$ vector $(x_1, \ldots, x_N)^T$.
This reminds me of the standard $N$-dimensional representation $V$ of $O(N)$, which is not irreducible. Indeed, it is the following direct sum $$V = (\vec{v}) \oplus (\vec{v})^\perp$$ where $(\vec{v})$ is just the real span of $v$ and $\perp$ denotes the orthogonal complement. The map $X \mapsto X - E[X]$ corresponds to the orthogonal projection $V \to (\vec{v})^\perp$. This is the first remark.
The second remark is that the covariance matrix of $X$ and $Y$ is, by definition, just the Gram matrix of the projections of $\vec{X}$ and $\vec{Y}$ onto $(\vec{v})^\perp$. Thus, if the projections of $\vec{X}$ and $\vec{Y}$ are linearly independent, this is just the $2 \times 2$ matrix representing the Euclidean inner product on $\mathbb{R}^N$ with respect to the basis consisting of the projections of $\vec{X}$ and $\vec{Y}$ onto $(\vec{v})^\perp$. This is my second remark.
We thus see that standard probability theory is linked (and in some sense dual) to the representation theory of $O(N)$. It thus seems to possible to define a probability theory that is linked to some other Lie group, say $Sp(N)$ for example (I am sure it was already done for $U(N)$), or even an exceptional Lie group ($F_4$, $G_2$, $E_6$, $E_7$ or $E_8$).
I wonder if this is interesting, if it has been done and if it has any interesting applications, whether in mathematics or in the "real" world (Mathematics is real too, but I guess it is a different kind of reality than the reality of the physical world... It is getting philosophical now!).
