Ok, the posibillity of no formula existing fascinated me so I followed Wolfgang's advise (even if it was not meant for me) and found a formula on the internet. I quote:

Theorem 3. Let $p(z)$ be any monic polynomial of degree $n$ and let $M_n$ be the family of
all $n × n$ matrices over $\{−1,+1\}$. Then
$$L_n = 2^{−n^2} \sum_{X \in M_n} p(\textrm{Per} X) \pi(X),$$
where Per$(X)$ is the permanent of $X$ and $\pi(X)$ is the product of the entries of $X$.

From: On the number of Latin squares by Brendan D. McKay and Ian M. Wanless.

However the following quote from the same article is maybe more relevant in some respects:

The literature contains quite a few exact formulas for $R_n$, but none of them appear very efficient for explicit computation (though Saxena [16] managed to compute $R_7$ using such a formula).

($L_n$ is the number of nice matrices, $R_n$ is the number of nice matrices whose first row and column read $1, \ldots, n$. $L_n$ can easily be computed from $R_n$ but is much bigger. The problem here more than anything else is that the numbers $R_n$ and $L_n$ grow perversely fast with $n$.)