Binomial supercongruences: is there any reason for them? One of the recent questions, in fact
the answer
to it, reminded me about the binomial sequence
$$
a_n=\sum_{k=0}^n{\binom{n}{k}}^2{\binom{n+k}{k}}^2,
\qquad n=0,1,2,\dots,
$$
of the Apéry numbers. The numbers $a_n$ come as
denominators of rational approximations to $\zeta(3)$ in Apéry's
famous proof of the irrationality of the number.
There are many nice properties of the sequence, one of these
is the observation that for primes $p\ge5$,
$$
a_{pn}\equiv a_n\pmod{p^3},
\qquad n=0,1,2,\dots.
$$
The congruence was conjectured for $n=1$ by S. Chowla, J. Cowles and M. Cowles
and proved in the full generality by I. Gessel (1982).
There is probably nothing strange in the congruence
(which belongs, by the way, to the class of supercongruences as it happens
to hold for a power of prime higher than one). But already the classical
binomials behave very similarly: for primes $p\ge5$,
$$
\binom{pm}{pn}\equiv\binom mn\pmod{p^3},
$$
the result due to G.S. Kazandzidis (1968). There are many other examples
of modulo $p^3$ congruences, most of them explicitly or implicitly related
to some modular objects, but that is a different story. My question is:
what are the grounds for the above (very simple) congruence for binomial
coefficients to hold modulo $p^3$? Not modulo $p$ or $p^2$ but $p^3$.
I do not ask you to prove the supercongruence but to indicate a general
mechanism which provides some kind of evidence for it and can be used
in other similar problems.
My motivation rests upon my own research on supercongruences; most of them
are just miracles coming from nowhere...
 A: Andrew Granville, in his beautiful article Binomial coefficients modulo prime powers (1997), attributes to Jacobsthal the congruence
$$
\binom{np}{mp} \Big/ \binom{n}{m} \equiv 1 \mod{p^{3+\mathrm{ord}_p\{nm(n-m)\}}} \quad \quad \quad (p \geq 5).
$$
In Chapter 7 of Alain's Course in $p$-Adic Analysis (GTM 198), the same congruence is attributed to Kazandzidis. 
This is a (mild?) generalization of Wolstenholme's theorem, $\binom{np}{mp} \equiv \binom{n}{m} \mod{p^3}$.
As both references explain, the general congruence is best understood via the $p$-adic analog of Stirling's asymptotic formula for the $\Gamma$-function. The $p$-adic logarithm of the left-hand side admits a convergent power series expansion involving the $p$-adic Bernoulli numbers - just as the higher order terms in the usual Stirling formula involve the usual Bernoulli numbers. This is covered by formulas (35) and (38) in Granville's article. When $p \geq 5$, all terms in the expansion are seen to have at least the stated valuation $\delta := 3+\mathrm{ord}_p\{nm(n-m)\}$, and the only term with valuation possibly $\delta$ has the form $B_{p-3} \times p^3 \times nm(n-m) \times u_p$, where $u_p \in \mathbb{Z}_p^{\times}$ is a $p$-adic unit. 
Therefore the congruence holds modulo $p^{4 + \mathrm{ord}_p(nm(n-m))}$ if and only if $p \mid B_{p-3}$. Those are the Wolstenholme primes referred to by Greg Kuperberg in his answer.
[I came to this old question via the "Related" column for the new post A congruence involving binomial coefficients by Richard Stanley, which proposes a different generalization of Wolstenholme's $\binom{2p}{p} \equiv 2 \mod{p^3}$. ]
A: I learned the second congruence as a version of Wolsteholme's theorem, and I would be a bit surprised if Kazandzidis was the first person to observe the equivalence between this form and any other form of Wolsteholme's result.  As for the "reason" that this result is true, I wrote a proof for the Wikipedia page which is mostly, but not entirely, a direct counting argument and which you could call "the grounds" for the congruence.   The conceptual content of the argument is as follows:
The result holds modulo $p^2$ because you can divide the $pm$-set into $m$ cycles of length $p$ and rotate them separately.  You obtain $\binom{m}{n}$ equivalence classes of subsets of size 1, and the other equivalence classes have order $p^2$ or higher.
Then you can examine $\binom{-p}{p}$, interpreted as $\binom{p^4-p}{p}$.  This binomial coefficient is algebraically equivalent to $\binom{2p}{p}/2$ up to sign.  The orbit decomposition in the previous paragraph establishes a second relation with $\binom{2p}{p}$.  The conclusion is that the mod $p^2$ contribution vanishes for one non-trivial pair $(m,n)$, and if it vanishes once, it vanishes always.  This vanishing  principle extends mod $p^4$ — your second binomial congruence holds mod $p^4$ — provided that it vanishes once "by accident".  A prime for which this happens is called a Wolstenholme prime.  Two such primes are known, 16843 and 2124679.
Another remark:  There are two pieces of evidence that the $p^2$ congruence (Babbage's theorem) is entirely combinatorial, but the extension to $p^3$ (Wolstenholme) is essentially algebraic.  First, that Wolstenholme's theorem doesn't hold for the primes $p=2,3$.  Second, that Babbage's theorem has a $q$-analogue for Gaussian binomial coefficients, with the same orbit proof.  But Wolsteholme's extension does not have a $q$-analogue as far as I know.  In fact, you can tell that the known $q$-Babbage's theorem doesn't extend, because the difference polynomial doesn't have any more cyclotomic factors.

I stand corrected on a couple of points.  First, on the math, there is a paper by George Andrews, "q-Analogs of the binomial coefficient congruences of Babbage, Wolstenholme, and Glaisher", that does give some version of a q-analogue of Wolstenholme's theorem.  The proof given there is algebraic, so it does not shed light on possible combinatorial "explanations".  Second, according to Andrews, the full binomial interpretation of Wolstenholme's result is due to Glaisher.
