# Plane partitions not containing (1,1,1)

A plane partition is a subset of $\mathbb Z_{\geqslant0}^3$ s.t. if it contains $(i+1,j,k)$ or $(i,j+1,k)$ or $(i,j,k+1)$ it also contains $(i,j,k)$.

What is the generating function $R(q)$ of (volumes of) plane partitions not containing the cell $(1,1,1)$?

Since a plane partition containing $(1,1,1)$ contains a $2\times2\times2$ cube, mod $q^8$ this g.f. coincides with MacMahon’s $$M(q)=\frac1{\prod(1-q^i)^i}=1+q+3q^2+6q^3+13q^4+24q^5+48q^6+86q^7+160q^8+282q^9+500q^{10}+\ldots$$ and computing a couple more terms $$R(q)=1+q+3q^2+6q^3+13q^4+24q^5+48q^6+86q^7+159q^8+279q^9+488q^{10}+\ldots$$ OEIS doesn’t know this sequence, but in the form $$R(q)=1+\frac{q-3q^3+6q^6-10q^{10}+\ldots}{[(1-q)(1-q^2)(1-q^3)\ldots]^3}.$$ a pattern is evident.

So actually the question is

How to prove that $R(q)=1+\frac1{(q)_\infty^3}\sum\limits_i(-1)^i\frac{i(i-1)}2q^{\tfrac{i(i-1)}2}$?

AFAIK the statement can be found somewhere in the literature on representation theory of quantum toroidal something. But surely there is a well-known combinatorial (in the broad sense, not necessarily bijective) proof?

Context 1. G.f. for plane partitions not containing $(1,1,0)$ is $1/(q)_\infty^2\cdot\sum(-1)^iq^{\frac{i(i+1)}2}$. See the subsection about «V-shaped partitions» in Stanley's Enumerative Combinatorics for a (simple) combinatorial proof. Perhaps there is something like that proof in «(1,1,1)-case».

Context 2. G.f. for plane partitions not containing $(1,1,1)$ but intersecting axes at $(i,0,0)$, $(0,j,0)$ and $(0,0,k)$ resp. is $\genfrac[]{0pt}{}{i+j}i\genfrac[]{0pt}{}{j+k}j\genfrac[]{0pt}{}{k+i}kq^{i+j+k+1}$. So maybe it’s possible to compute $R(q)$ as $1+\sum_n q^{n+1}\sum_{i+j+k=n}\genfrac[]{0pt}{}{i+j}i\genfrac[]{0pt}{}{j+k}j\genfrac[]{0pt}{}{k+j}k$. And for the internal sum I know the answer at least at $q=1$: https://math.stackexchange.com/q/177209/

• Am I correct that you mean $R(q)=1+\sum_{n=0}^\infty q^{n+1}\sum_{i+j+k=n}\genfrac[]{0pt}{}{i+j}i\genfrac[]{0pt}{}{j+k}k\genfrac[]{0pt}{}{k+i}k$ Commented Jan 9, 2015 at 8:11
• @Hjalmar Yes, of course, thank you (I'll fix the typo) Commented Jan 9, 2015 at 10:39
• Regard a plane partition as a two-dimensional array $(\pi_{ij})_{i,j\geq 1}$ in the usual way. The present question asks to count plane partition $n$ with $\pi_{22}\leq 2$. If $v(n)$ denotes the number of plane partitions of $n$ with $\pi_{22}=0$, then by EC1, Prop. 2.5.1, $\sum_{n\geq 0} v(n)q^n = \left( \sum_{m\geq 0} (-1)^mq^{{m+1\choose 2}}\right)\prod_{i\geq 1}(1-q^i)^{-2}$. This suggests counting plane partitions of $n$ with $\pi_{22}\leq k$ for $k\geq 2$ (and similar problems). Commented Sep 14, 2020 at 18:13
• @RichardStanley We also have some formulas for $\pi_{nm}=0$ in arXiv:1512.08779. In that case an answer is given by Brion's theorem. What really puzzles me is that the formula for $R(q)-1$ also looks like some version of Brion's theorem (each triple (x,y,z) in the answer below should correspond to a vertex etc)… Commented Sep 14, 2020 at 21:47

Correcting typos, we have $$R(q)=1+\sum_{i,j,k=0}^\infty\frac{(q)_{i+j}(q)_{i+k}(q)_{j+k}}{(q)_i^2(q)_j^2(q)_k^2}\,q^{i+j+k+1}$$ (where $(q)_k=\prod_{j=1}^k(1-q^j)$). By the $q$-binomial theorem, $$\frac{(q)_{i+j}}{(q)_i(q)_j}=\sum_{x=0}^i\frac{1}{(q)_x(q)_{i-x}}(-1)^xq^{\binom x2+(j+1)x}.$$ Using this, and the expansions obtained by cyclically permuting $(i,j,k)$, we get \begin{align*}R(q)-1&=\sum_{i,j,k=0}^\infty\sum_{x,y,z=0}^{i,j,k}\frac{(-1)^{x+y+z}q^{\binom x2+\binom y2+\binom z3+(j+1)x+(k+1)y+(i+1)z+i+j+k+1}}{(q)_x(q)_{i-x}(q)_y(q)_{j-y}(q)_z(q)_{k-z}}\\ &=\sum_{x,y,z=0}^\infty \frac{(-1)^{x+y+z}q^{\binom x2+\binom y2+\binom z2+(y+2)x+(z+2)y+(x+2)z+1}}{(q)_x(q)_y(q)_z}\\ &\quad\times\sum_{i=0}^\infty\frac{q^{(1+z)i}}{(q)_i}\sum_{j=0}^\infty\frac{q^{(1+x)j}}{(q)_j}\sum_{k=0}^\infty\frac{q^{(1+y)k}}{(q)_k}\\ &=\frac 1{(q)_\infty^3}\sum_{x,y,z=0}^\infty {(-1)^{x+y+z}q^{\binom x2+\binom y2+\binom z2+(y+2)x+(z+2)y+(x+2)z+1}} ,\end{align*} where we first replaced $(i,j,k)$ by $(i+x,j+y,k+z)$ and then computed the inner sums using again a special case of the $q$-binomial theorem. We now observe that $$\binom x2+\binom y2+\binom z2+(y+2)x+(z+2)y+(x+2)z+1=\binom{x+y+z+2}2.$$ Since there are $\binom n2$ solutions $(x,y,z)$ to $x+y+z+2=n$, we get indeed $$R(q)=1+\frac 1{(q)_\infty^3}\sum_{n=0}^\infty(-1)^n\binom n2q^{\binom n2}.$$