I am chemist and ask for apologies for all my mathematical inabilities when asking this question in advance, but after quite a bit of searching I found that this problem could be "open" or at least hard enough to find addressed in the literature and also advanced enough that it's possibly suitable to be asked here.

I work over a subset of the finite groups called point groups these are all the ((essentially) finite) subgroups of $O(3)$. The "degeneracies" arising in those groups are of importance in Chemistry so I started to investigate them. With "degeneracy" the dimensions of the irreps over $\mathbb R$, (this is of essential importance) occurring for the group under consideration is meant. Whenever a group is represented by an irrep of dimension $n>1$ we speak about $n$-fold degeneracy.

The term degeneracy in this context relates to the fact that the quantum mechanical eigenstates of (symmetric) systems/molecules form such $n$-dimensional sub spaces of the Hilbert space. Since the Hamilton operator is self-adjunct it makes sense to regard representations over $\mathbb R$ instead of over the usually more elegant $\mathbb C$.

My first question is, under which conditions in terms of group elements degeneracy can occur in a group? And the second question is how is the maximal dimension $n_{\max}$ of real irreps, over all irreps of the group, determined by the structure of the group?

My primary observation is that groups that contain exactly one generator $y$ of order $m=3$, like a group $\langle x,y,...| x^2=y^m = 1 = ...\rangle $ have $n_{\max}=2$

Then there are just a very few point groups with $n_{\max} > 2$. We call them "high symmetry groups". Basically its the symmetry groups of the tetrahedron, the octahedron and the icosahedron (with 2 or 3 certain subgroups of theirs), where the former two have $n_{\max}=3$, the latter $n_{\max}=5$.

These high symmetry groups all have two generators, the tetrahedral groups one of order $m=3$, the octahedral groups one of order $m=4$ and the icosahedral groups one of order $5$.

So I would assume that there is a connection between the order of the generators and the degeneracies that can occur in a group. What it is exactly remains very obscure to me. So I would be very grateful about any hints also to the literature.

*Edit*

Since the audience is so fantastically knowledgeable, I can't resist to make a small comment on the motivation of my research in the hope it might ring some bell and give rise to more inspiring comments:

The motivation of my question is, that you can see certain interesting physical properties in the states (of physical systems) if they are degenerate. "States" are some manifestations of irreps where we have direct numerical access to and a good intuition about their visual representations. It happens that there are certain, it seems, deep connections between the angular momentum operator, which is essentially an infinitesimal rotation in the (physical space) $\mathbb R^3$, and the occurrence of degeneracy (at least if it's 2-fold). At the same time we see that these states which are instances of degenerate representations are transformed into each others by (finite) rotations. Such that the question was arising if all such degeneracies are related to rotations, or if there is at least something in the structure of rotations that is general in $d>1$ dimensional representations.

There was recently a result that suggested that there is a "hidden" anti-unitary symmetry (state transforming operator) at the basis of **any** $2$-fold degeneracy of the form

$$ \mathcal{O} = i \begin{pmatrix} 0 & 1 \\ -1 & 0 \end{pmatrix} \mathcal{K}$$ (with $\mathcal{K}$ as complex conjugation operator) that transforms between the two degenerate irreps. Such an operator can be easily constructed from the angular momentum operator, for example, but not only, constructions of operators like {\mathcal{O}} based on spin or time reversal and others are possible if one considers more general "parametrizations" of physical systems then only $\mathbb R^3$. Anyway, as far as symmetries of the real space $\mathbb R^3$ are concerned it seems that rotation (and infintesimal rotation) are crucial and I aim to understand what "crucial" here exactly means.

What I find in particular intriguing in the light of the answer from @QiaochuYuan is the connection with "non-Abelianess", because the defining relation of $J$, the angular momentum or also spin operator is the so called "angular momentum algebra", that is a commutator relation between its components

$$ [J_i,J_j] = i \varepsilon_{ijk} J_k$$

(with the Levi-Civita symbol $\varepsilon$). So this seems to suggest that angular momentum is an essential source of non-commutativity somehow. I like then to understand what types of sources else there might be, for this non-commutativity in quantum mechanical systems, especially if only representations of states in $\mathbf{R}^3$ are considered. One thing which complicates the question is that in most systems rotational symmetry "is broken" but degeneracy can nevertheless occur. Then my question would be, what is the explicit form of the $\mathcal{O}$ operator and if it can be continuously related to angular momentum if one views the "symmetry breaking" as a continuous process.

(Sorry for the post-answer edit, I hope it complies with the MO rules!)

3more comments