The coefficient of a specific monomial in the expansion of the following polynomial

Question

Let $a_{n,k}$ be the coefficient of $$X_1^{\frac{k(n-1)}{2}}X_2^{\frac{k(n-1)}{2}}\cdots X_n^{\frac{k(n-1)}{2}}$$ in the expansion of the real polynomial $$\left(\prod\limits_{1\leq i<j\leq n}(X_j-X_i)\right)^k,$$where $n,k$ are positive integers such that $n>1$ and $k(n-1)\equiv0 \pmod 2$.

Since $$\prod\limits_{1\leq i<j\leq n}(X_j-X_i)=\sum\limits_{\sigma\in S_n}sgn(\sigma)\prod\limits_{i=1}^nX_{\sigma_i}^{i-1},$$ here the sum is computed over all permutations $\sigma$ of the set $\{1,2,\cdots,n\}$, we can get the following simple results:

$(1)a_{n,1}=0$ for every odd positive integer $n>1$;

$(2)a_{n,2}\neq 0$ for every positive integer $n>1$.

I want to ask whether $a_{n,k}$ is equal to 0 or not when $k>2$.

Answer 1

Let's call your polynomial $\Delta$. Then $\Delta$ is anti-symmetric in the $x_i$ (if you switch $x_i$ to $x_j$ then $\Delta$ turns into $-\Delta$). If $k$ is odd, then $\Delta^k$ also has the property, so $a_{n,k}=0$ whenever $k$ is odd.

I will now show that $(-1)^{\binom{n}{2} m} a_{n,2m}>0$. Write $k=2m$. Let $T$ be the torus $\theta_1 + \theta_2 + \cdots + \theta_n=0$ inside $(\mathbb{R}/2 \pi \mathbb{Z})^n$, equipped with the Haar measure $d V$ with volume one.

Since $\Delta^{k}$ is homogenous of degree $k \binom{n}{2}$, it cannot contain monomials of the form $(x_1 x_2 \cdots x_n)^e$ for any $e$ other than $k \tfrac{n-1}{2}$. Therefore, we can extract $a_{n,k}$ by a contour integral over $T$:

$$a_{n,k} = \int_T \Delta(e^{i \theta_1}, \ldots, e^{i \theta_n})^{2m} dV$$ $$=\int_T \prod_{a<b} \left( e^{i \theta_a} - e^{i \theta_b} \right)^{2m} dV$$ $$=\int_T \prod_{a<b} \left( e^{i2 \theta_a} - 2 e^{i (\theta_a+\theta_b)} + e^{i 2 \theta_b} \right)^{m} dV$$ $$=\int_T e^{i (n-1) \sum \theta_a} \prod_{a<b} \left( e^{i (\theta_a- \theta_b)} - 2 + e^{i (\theta_b-\theta_a)} \right)^{m} dV$$ $$=(-2)^{\binom{n}{2} m} \int_T \prod_{a<b} \left( 1 - \cos (\theta_a - \theta_b) \right)^{m} dV.$$ Going from the second-to-last line to the last line, I used that $\sum \theta_a=0$ on $T$.

The integrand is nonnegative (and positive except on a set of measure zero) so the final integral is positive.

Answer 2

This is not an answer but a conjecture based on some trials. $$a_{n,k} = \begin{cases} 1, & n=1 \\ 0, & n\gt 1, k\text{ odd} \\ (-1)^{t\binom n2}\displaystyle\frac{(tn)!}{(t!)^n}, & k=2t. \end{cases} $$ Note that $\frac{(tn)!}{(t!)^n}$ is the number of ways of partitioning a set of size $tn$ into $n$ sets of size $t$.

Oh, now I see that this is a special case of Macdonald's constant-term conjecture, see this article, which has references.