Although the question was completely answered by others, I want to provide some more input.
- An illuminating $p$-adic point of view is given in Section 7.1.6, "The Kazandzidis Congruences", of the (excellent) book "A Course in $p$-adic Analysis" by Alain M. Robert. The main result of the section is the following result, mentioned by Alexei Ustinov, and attributed there to Kazandzidis: $$\forall p > 2: \binom{pn}{pk} \equiv \binom{n}{k} \pmod {p^{2+\varepsilon}nk(n-k)\binom{n}{k}\mathbb{Z}_p}, \quad \varepsilon = 1_{p>3}.$$
The section is part of the chapter on the Morita $p$-adic Gamma function $\Gamma_p: \mathbb{Z}_p \to \mathbb{Z}_p$, which is a continuous function given on integers $n>2$ by $$\Gamma_p(n)=(-1)^n \prod_{1\le j <n, p\nmid j} j,$$ at least when $p$ is an odd prime. Legendre's formula shows that the $p$-adic valuation of both terms in $F_p$ is exactly $1$, and so it remains to understand the $p$-adic valuation of $$\binom{p^{n+1}}{p^n} / \binom{p^n}{p^{n-1}},$$ which is a difference of two elements of $\mathbb{Z}_p^{*}$. As mentioned in the section, L. van Hamme had observed that $$\binom{pa}{pb} / \binom{a}{b} =\frac{\Gamma_p(pa)}{\Gamma_p(pb)\Gamma_p(p(a-b))},$$ and so it remains to compute $$| \frac{\Gamma_p(pa)}{\Gamma_p(pb)\Gamma_p(p(a-b))} - 1|_p,$$ which explains the link with $p$-adic analysis. Properties of the logarithm in $\mathbb{Z}$ show that in fact the above valuation is exactly $$|\log \Gamma_p(pa) - \log \Gamma_p(pb) - \log \Gamma_p(p(a-b))|_p.$$ Now it just a matter of finding out what is the Taylor expansion of $f(x):=\log \Gamma(px)$. It can be shown that $f$ is an odd function, as so $$f(x) = \sum_{i \ge 1} a_i x^{2i-1},$$ and then $$f(a)-f(b)-f(a-b)= ab(a-b)\sum_{i \ge 2} a_i \cdot \frac{a^{2i-1}-b^{2i-1}-(a-b)^{2i-1}}{ab(a-b)}.$$ (Note the vanishing of the linear term $i=1$!) Since $\frac{a^{2i-1}-b^{2i-1}-(a-b)^{2i-1}}{ab(a-b)}\in \mathbb{Z}_p$ (this is an integer polynomial in $a$ and $b$), we obtain $$|f(a)-f(b)-f(a-b)|_p \le |ab(a-b)|_p \cdot \max_{i \ge 2} |a_i|_p \cdot |\frac{a^{2i-1}-b^{2i-1}-(a-b)^{2i-1}}{ab(a-b)}|_i.$$ Bounding the term $\max_{i \ge 2} |a_i|_p \cdot |\frac{a^{2i-1}-b^{2i-1}-(a-b)^{2i-1}}{ab(a-b)}|_i$ is somewhat technical, and it is related to the Bernoulli numbers, as mentioned in Alexei's answer. It can be shown that the term $i=2$ satisfies $|a_2\cdot 3|_p \le p^{-3+1_{p=3}}$ and that the rest has valuation larger or equal to that. This yields the Kazandzisids Congruences by the previous arguments. by plugging $a=p^n,b=p^{n-1}$, we obtain a bound on the $p$-adic valuation of $F_p$.