Your determinant is closely related to the Van der Monde determinant ${\rm det}[\eta^{(j-1)(k-1)}]$ ($1 \leq j,k \leq n$).
Let $A = [\eta^{(j-1)(k-1)}]$ and note that $A$ is the character table of a cyclic group of order $n.$ I will sketch a method to evaluate the determinant of the complex character table of any finite group $G.$
Let $G$ be a finite group an let $B = [\chi_{k}(g_{j})]$, where $\{g_{j} : 1 \leq j \leq t\}$ is a full set of representatives for the conjugacy classes of $G$ and $\{\chi_{k} : 1 \leq k \leq t \}$is the set of complex irreducible characters of $G$.
The orthogonality relations for group characters tell us that $B\overline{B}^{T}$ is diagonal with $r$-th diagonal entry $|C_{G}(g_{r})|.$
Furthermore, we note that $BB^{T} = PB\overline{B}^{T}$, where $P$ is the permutation of $\{1,2,\ldots ,t \}$ which interchanges the class of $g_{j}$ with the class of $g_{j}^{-1}$ (again from the orthogonality relations).
Note that $P$ is the permutation matrix associated to a product of $\frac{t-s}{2}$ disjoint $2$-cycles (and $s$ $1$-cycles) where there are exactly $s$ conjugacy classes such that $g_{j}$ is conjugate to $g_{j}^{-1}.$
It follows that ${\rm det}(B)^{2} = (-1)^{\frac{t-s}{2}} \prod_{j=1}^{t} |C_{G}(g_{j})|$ and that ${\rm det}(B) = (i)^{\frac{t-s}{2}} \sqrt{\prod_{j=1}^{t} |C_{G}(g_{j})|}$ where the square root is real and positive, and $i$ is one choice of $\sqrt{-1}.$
When $G$ is cyclic of order $n$ (which is your case), we note that $t = n$ and that $s= 1$ if $n$ is odd and $s= 2$ if $n$ is even, while also $|C_{G}(g_{j})| = n$ for each $j$, so the above formula simplifies considerably.