This isn't the answer you're looking for, but I discovered it while attempting to find a geometric approach.

Consider the polynomials $$p_j(x) = (x-x_1)\cdots (x-x_j) = \sum_{i=0}^{n-1} a_{i,j} x^i,$$ where $p_0(x)=1$ by convention, $0\leq j\leq n-1$. Of course, $a_{j,j}=1$ and $a_{i,j}=0$ if $i>j$, and $a_{i,j}$ is the $i$th symmetric polynomial in $x_1,\ldots, x_j$ up to sign. If we multiply the Vandermonde matrix $[x_i^{j-1}]$ by the upper unipotent matrix $[a_{i-1,j-1}]$, we get the matrix $[p_{j-1}(x_i)]$. This is a lower triangular matrix, since $p_{j-1}(x_i) =0$ if $i \leq j-1$, and the diagonal entries are $p_{i-1}(x_i)$. Clearly, $$\prod_{i=1}^n p_{i-1}(x_i) = \prod_{i=1}^n (x_i-x_1) \cdots (x_i-x_{i-1}) = \prod_{1\leq i < j\leq n} (x_j-x_i).$$

Maybe there's a way of seeing this product geometrically as a natural affine transformation?