Have you ever seen this matrix? Each row is obtained from the previous one by multiplying each element by the corresponding element of the next cyclic permutation of $(a_1,\dots, a_n)$: $$\left( \begin{array}{llllllll} 1 & 1 & 1 & \dots & 1 & 1 \\ a_1 & a_2 & a_3 & \dots & a_{n1} & a_{n} \\ a_1 a_2 & a_2a_3 & a_3 a_4 & \dots & a_{n1} a_n & a_{n} a_1 \\ a_1 a_2a_3 & a_2a_3 a_4 & a_3 a_4a_5 & \dots & a_{n1} a_n a_1& a_{n} a_1 a_2 \\ a_1 a_2a_3a_4 & a_2a_3 a_4 a_5& a_3 a_4a_5 a_6& \dots & a_{n1} a_n a_1 a_2& a_{n} a_1 a_2a_3 \\ \dots & \dots & \dots & \dots & \dots & \dots \\ a_1 a_2a_3a_4 \dots a_{n2}& a_2a_3 a_4 a_5\dots a_{n1} & a_3 a_4a_5 a_6\dots a_n&\dots & a_{n1} a_n a_1 a_2\dots a_{n4}& a_{n} a_1 a_2a_3\dots a_{n3} \\ a_1 a_2a_3a_4 \dots a_{n1}& a_2a_3 a_4 a_5\dots a_{n} & a_3 a_4a_5 a_6\dots a_1& \dots & a_{n1} a_n a_1 a_2\dots a_{n3}& a_{n} a_1 a_2a_3\dots a_{n2} \\ \end{array} \right)$$ I would like to know if there is a closed formula for the determinant; of course it is invariant (up to sign) under cyclic permutations of $(a_1,a_2,\dots,a_n)$.

2$\begingroup$ Interesting matrix. The determinant is an irreducible polynomial for $n = 3$ and for $n = 4$, but I wouldn't be too surprised if it has some nontrivial combinatorial formulas. $\endgroup$ – darij grinberg Nov 25 '20 at 18:11

$\begingroup$ @SandraS Welcome to MathOverflow! $\endgroup$ – Per Alexandersson Nov 29 '20 at 21:00
Denote the matrix $A$, and index all $a_i$, and all rows and columns starting from $0$ for convenience. If, say, $a_0 = 0$, then $\det A = (1)^{\lfloor (n  1) / 2 \rfloor} \prod_{i = 1}^{n  1} a_i^i$ by substituting and computing the remaining upper triangular determinant. Let's further assume that all $a_i$ are nonzero.
Let $p_k = \prod_{i = 0}^{k  1} a_i$ ($p_0 = 1$ by convention). Put $a'_i = a_i p_n^{1/n}$, and construct $A'$ similarly. Alternatively, $A'$ is $A$ with $i$th row multiplied by $p_n^{i/n}$. Defining $p'_k$ similarly, we now have $p'_k = p_k p_n^{k / n}$, in particular $p'_n = 1$.
After multiplying $i$th column of $A'$ by $p'_i$, we arrive at a Hankel matrix $$P' = \begin{pmatrix} 1 & p'_1 & \ldots & p'_{n  1} \\ p'_1 & p'_2 & \ldots & 1 \\ \ldots & \ldots & \ldots & \ldots \\ p'_{n  1} & 1 & \ldots & p'_{n  2}\end{pmatrix},$$ which is a row permutation of a circulant $(1, \ldots, p'_{n  1})$. With all substitutions in mind, we have $$\det A = (1)^{\lfloor(n  1) / 2\rfloor} \prod_{k = 0}^{n  1} p_k \cdot \prod_{k = 0}^{n  1}\left(\sum_{j = 0}^{n  1} p'_j e^{i\frac{2\pi jk}{n}} \right).$$
One can see that the problem is as general as arbitrary circulant determinant.

$\begingroup$ Great, I have checked this out and I think one should multiply $i$th column of $A'$ by $p'_i p_n^{1/n}.$ $\endgroup$ – SandraS Nov 26 '20 at 8:59

$\begingroup$ My reasoning is, $A'$ looks like $A$ with $a'_i$ replacing $a_i$. If we multiply $i$th column of $A$ by original $p_i$, then each element becomes a prefix product $a_1 a_2 \ldots$ (possibly looping over $a_n$ to $a_1$, and so on). There seems to be no reason the same shouldn't apply to $A'$ as well. $\endgroup$ – Mikhail Tikhomirov Nov 26 '20 at 9:20