I might as well write an answer with the proof I referenced to above (found as theorem 110 <a href="http://qspace.library.queensu.ca/bitstream/1974/5235/1/Novak_Jonathan_I_200909_PhD.pdf">here</a>). Hopefully Darij will write a more detailed answer tomorrow.

The first thing to observe is that your matrix $A$ is the image of the element
$$\omega=\sum_{\sigma\in S_n} x^{|\pi|}\pi$$
in the regular representation of the group algebra $\mathbb C[S(n)]$. Next notice that this element factors as
$$\omega=(x+J_1)(x+J_2)\cdots(x+J_n)$$
where $J_k$ are the Jucys-Murphy elements defined as $J_k=\sum_{ s < k} (s,k)$.

Let's denote $\Xi_n=\lbrace J_1,J_2,\dots,J_n,0,0,\dots\rbrace$. It is a theorem that for any symmetric function $f\in \Lambda$, the mapping $f\to f(\Xi_n)$ sends symmetric polynomials onto elements of the class algebra $\mathcal Z(n)$. 

Now since $f(\Xi_n)\in \mathcal Z(n)$, by Schur's lemma it acts as a scalar on any irreducible representation $V^{\lambda}$ of $\mathbb C[S(n)]$. Jucys theorem says that the central character of $f(\Xi_n)$ acting on $V^{\lambda}$ can be obtained by simply substituting the alphabet $\Xi_n$ with the content alphabet
$$A_{\lambda}=\lbrace c(\square): \square\in \lambda\rbrace.$$
These two facts are proved in Jucys' article, <a href="http://www.sciencedirect.com/science/article/pii/0034487774900196">"Symmetric polynomials and the center of the symmetric group ring"</a>.

So, in particular, the central character of $\omega$ is
$$\prod_{\square\in \lambda}(x+c(\square)),$$
and, putting things together, from the decomposition $\mathbb C[S(n)]=\bigoplus_{\lambda \vdash n} (\dim \lambda)V^{\lambda}$, we obtain
$$\det(A)=\prod_{k=1}^{n-1}(x^2-k^2)^{r_k},$$
where
$$r_k=\sum_{\lambda\vdash n, k\in A_{\lambda}} \dim \lambda.$$