Is it known whether there is a prime $p=4k+1$ such that $k!+1$ is divisible by $p$?
(I conjectured that such primes don't exist, but couldn't prove it.)
There are no such primes $p$. Write $p=4k+1$ as $a^2+b^2$ with $a$ odd and $b$ even, and by changing the sign of $a$ if necessary suppose that $a\equiv 1 \pmod 4$. Note that $a$ is uniquely defined. Gauss showed that (see, for example, Binomial coefficients and Jacobi sums for references, and proofs of this and other similar congruences)
$$
\binom{2k}{k} \equiv 2a \pmod p.
$$
Also, by Wilson's theorem we know that $(2k!)^2\equiv (4k)! \equiv -1 \pmod p$. Using this, and squaring Gauss's congruence, we get
$$
4a^2 (k!)^4 \equiv -1 \pmod{p}.
$$
If now $k! \equiv -1 \pmod p$ then we conclude that $4a^2 + 1\equiv 0 \pmod p$.
Since $4a^2+1$ is $1\pmod 4$ and at most $4p$, we must have $4a^2+1 =p = (2a)^2 +1^2$. But by the (essential) uniqueness of writing $p$ as a sum of two squares this forces $a=1$ (and so $p=5$). For $p=5$ we verify directly the claim.