I will sharpen my earlier comment a little. The number of such matrices is divisible by $(p-1)^{2n-2}$ (though it may be zero). Let $T$ be the group of invertible diagonal matrices, and let $\mathcal{M}$ denote the set of invertible matrices of full rank ( though not necessarily determinant 1). Then $M \to TMU^{-1}$ defines an action of $T \times T$ on $\mathcal{M},$ and the only pairs $(T,U)$ which fix any element are those with $T = U$ both scalar. Hence $(T \times T)/{\rm diag} (S)$ acts semi-regularly on $\mathcal{M},$ where $S$ is the subgroup of $T$ consisting of scalars. Thus $|\mathcal{M}|$ is divisible by $(p-1)^{2n-1}$, and hence, as user74230 notes in comments, the number of matrices in $\mathcal{M}$ which have determinant $1$ is a multiple of $(p-1)^{2n-2}.$ Also, the number of such matrices is divisible by $\frac{n!}{2},$ since $A_{n}$ acts by right multiplication (in its representation by permutation matrices) on such matrices, and no non-identity element has any fixed point.
Actually, I realise that this implies that when $n \geq p,$ the number of such matrices is divisible by $p.$ For when $p =2$ the number is zero, and when $p$ is odd and $n \geq p,$ then $|A_{n}|$ is divisible by $p^{\frac{n - \sigma_{p}(n)}{p-1}}$, where $\sigma_{p}(n)$ is the sum of the digitdigits in the base $p$ expansion of $n.$