Using irreducible characters of the orthogonal group as basis for homogeneous symmetric polynomials

Question

The irreducible characters of the orthogonal group $O(2N)$ are given by $$ o_\lambda(x_1,x_1^{-1},...x_N,x_N^{-1})=\frac{\det(x_j^{N+\lambda_i-i}+x_j^{-(N+\lambda_i-i)})}{\det(x_j^{N-i}+x_j^{-(N-i)})}$$

I was playing with them as basis for the space of homogeneous symmetric polynomials. I wanted to write the function $$p_3=\sum_{i=1}^N (x_i^3+x_i^{-3})$$ as a linear combination of $o_\lambda$'s.

I started with $N=2$ and in that case I found that $$ p_3=o_{(3)}-2o_{(2,1)}+o_{(1)}.$$

However, I then tried with $N=4$ and in that case I found that $$ p_3=o_{(3)}-o_{(2,1)}+o_{(1,1,1)}.$$

I was expecting the coefficients to be the same, i.e. to be independent of $N$. (Independence of $N$ indeed holds when using characters of the unitary group, in which case the coefficients are the characters of the permutation group.)

Have I made some mistake or are the coefficients indeed dependent on $N$?

Answer 1

The coefficients do depend on $N$. A way to get around this and deal with "universal characters" was found by Koike and Terada (Young-diagrammatic methods for the representation theory of the classical groups of type $B_n$, $C_n$, $D_n$).

Answer 2

(I have looked more carefully at this theory of "universal characters" mentioned by Stanley and am updating this answer according to what I learned. All of this was contained in the answer by Stanley, I am just unpacking it for the sake of those like me and the OP who may be confused.)

The following material is in the paper Young-diagrammatic methods for the representation theory of the classical groups of type $B_n$, $C_n$, $D_n$ by Koike and Terada, also in works by King (such as Modification Rules and Products of Irreducible Representations of the Unitary, Orthogonal, and Symplectic Groups, Journal of Mathematical Physics 12, 1588, 1971) and a very readable 1938 book by Murnaghan (The Theory of Group Representations).

$\DeclareMathOperator\Or{O}\DeclareMathOperator\U{U}$So functions $o_\lambda$ correspond to irreducible characters of $\Or(2N)$ only when $\ell(\lambda)\le N$. The set of such actual characters forms a basis for the space of symmetric functions on variables $\{x_1,x_1^{-1},\dotsc,x_N,x_N^{-1}\}$. However, the coefficients in the expansion of power sums in general depend on $N$.

On the other hand, partitions with $\ell(\lambda)>N$ do not define irreducible representations, so $o_\lambda$ is not an actual character. In that case they are replaced by another symmetric function, $o_\lambda\to o_{\widetilde{\lambda}}$ with a modified partition $\widetilde{\lambda}$.

When these functions are used, the large-$N$ expansions continue to hold for small $N$. In that sense they are 'universal'.

When a Schur function $s_\mu$ is decomposed in terms of $o_\lambda$, which corresponds to the branching rule of $\U(2N)\supset \Or(2N)$, one has $\ell(\lambda)\le \ell(\mu)$, so if $\ell(\mu)\le N$ only actual characters appear. This is not true if $N<\ell(\mu)\le 2N$, and then universal characters must also be used. This is not discussed in the classic book "Representation theory" by Fulton and Harris, for example, where only the case $\ell(\mu)\le N$ appears.

The modified partition $\widetilde{\lambda}$ is defined as follows. For $O(2N)$, let $m=2\ell(\lambda)-2N$. Then remove from the Young diagram of $\lambda$ a total of $m$ adjacent boxes, starting from the bottom of the first column and keeping always at the boundary of the diagram. In this way the changes to be implemented are, in sequence, $\lambda'_1\to\lambda'_2-1$, then $\lambda'_2\to\lambda'_3-1$, etc. until the procedures stops at some column $c$. If $m$ is too large and there are not enough boxes to accommodate this procedure, or if the remaining diagram is not a partition, then $o_\lambda=0$; otherwise $o_\lambda=(-1)^{c-1}o_{\widetilde{\lambda}}$.

For example, the universal decomposition for $p_4$ is $$p_4=o_4-o_{31}+o_{211}-o_{1111}+o_0.$$ If $\lambda=(1,1,1,1)$ and $N=6$, we must remove $8-6=2$ boxes, in which case we get $c=1$ and $\widetilde{\lambda}=(1,1)$. Hence, for $O(6)$ we have $o_{1,1,1,1}=o_{1,1}$ and the universal relation reduces to $p_4=o_4-o_{31}+o_{211}-o_{11}+o_0$. When $N=4$, we must remove $8-4=4$ boxes from $\lambda=(1,1,1,1)$. We end up with no boxes at all, so $o_{1,1,1,1}=o_\emptyset$ for $O(4)$. We must remove $6-4=2$ boxes from $\lambda=(2,1,1)$, arriving at $o_{2,1,1}=o_2$. The relation reduces to $p_4=o_4-o_{31}+o_{2}$. Finally, take $N=2$. We cannot remove $8-2=6$ boxes from $(1,1,1,1)$, so $o_{1,1,1,1}=0$ for this group; when we remove $4-2=2$ boxes from $(3,1)$, the result is not the diagram of a partition, so $o_{3,1}=0$ for this group; removing $6-2=4$ boxes from $(2,1,1)$ leads to the empty partition, and the removing procedure ends in the second column so $c=2$, hence $o_{2,1,1}=-o_\emptyset$ for this group. Thereby the relation reduces to $p_4=o_4$.

Unfortunately, the modification rule is incorrectly stated in the recent "The Random Matrix Theory of the Classical Compact Groups" (Cambridge University Press, 2019), by Elizabeth Meckes.