# Counting the orbits of a set of tabloids under the action of a Young subgroup

Let $\lambda = (\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_k)$ and $\mu = (\mu_1 \geq \mu_2 \geq \cdots \geq \mu_\ell)$ be partitions of a positive integer $n$. As in Fulton's book on Young tableaux, recall that a tabloid is an equivalence class of numberings (with the distinct numbers $1,2,\ldots,n$) of a Young diagram, two being equivalent if corresponding rows contain the same entries. The symmetric group $S_n$ then acts in a natural way on tabloids. Let $M^\lambda$ denote the complex vector space with basis the tabloids of shape $\lambda$. Since $S_n$ acts on tabloids, $M^\lambda$ is naturally an $S_n$-representation.

Now let $S_\mu$ denote the Young subgroup of $S_n$ corresponding to the partition $\mu$. For example if $n=7$ and $\mu=(3,2,2)$ then $S_\mu = S_{\{1,2,3\}} \times S_{\{4,5\}} \times S_{\{6,7\}} \cong S_3 \times S_2 \times S_2$.

I am interested in computing the dimension of the $S_\mu$-invariant subspace of $M^\lambda$, i.e. $dim (M^\lambda)^{S_\mu}$. This amounts to counting the number of orbits under $S_\mu$ in the set of tabloids of shape $\lambda$.

A simple reformulation in terms of balls and boxes'' would be: with $\lambda$ and $\mu$ the fixed partitions as above, I have $n$ balls in $\ell$ different colors, where there are $\mu_i$ many balls of the $i$-th color. I want to count the number of ways to distribute these balls into $k$ many boxes, where the $j$-th box can hold at most $\lambda_j$ many balls.

Does anyone know a formula for this number? Alternatively, any concrete information about these numbers would be appreciated.

The number of ways to place the balls so that the $j$th box holds exactly $\lambda_j$ balls is the number $N_{\lambda\mu}$ of matrices of nonnegative integers with row sum vector $\lambda$ and column sum vector $\mu$. There is no simple formula for these numbers. For connections with symmetric functions, see for instance Prop. 7.5.1, equation (7.31), Cor. 7.12.3, and Exercise 7.77(c) of Enumerative Combinatorics, vol. 2. Note that if the $j$th box holds at most $\lambda_j$ balls then it holds exactly $\lambda_j$ balls since $\sum \mu_i = \sum \lambda_i$.