There is an excellent issue of Statistical Science that address the James-Stein phenomena from various aspects.
https://www.jstor.org/stable/i23208816
Question What does it mean that a James-Stein estimator beats least squares estimator?
It means that the JS estimator has a smaller risk than LSE w.r.t. a
prescribed risk function $R(\delta)=E_{\theta}L(\delta,\theta)$; which
is equivalent to say that if we choose $L^2$ loss function, then when
the dimension is higher than 3, the LSE for the mean is no longer
admissible. The only admissible estimator should be JS estimator.
The original JS estimator assumes multi-dimensional Gaussian distribution. But later with consideration of admissibility, there are various cases where shirkage estimator(JS estimator) actually beats the LSE(also MLE) and other frequentist estimators. And "beating" also depends on the $L^2$ loss function you choose, and hence the risk function $R$ you choose for this decision problem. To illustrate this point, we consider Gaussian case with $L^2$ loss function in the explanation below.
There are two approaches that seem very intuitive to me.
(1)Linkage between JS estimator and the Diffusion process.
Brown [1] pointed out that under the framework of decision theory, it could be shown that the admissibility and the recurrence of Brownian motion is equivalent. Moreover, Brown shown in his major theorem that via a variational minimization problem, the admissibility of an estimator can be discussed using recurrence of corresponding Brownian motion. Such a variational approach can be extended to various other situations.
Theorem 5.1 in [1], modified. A necessary condition for $\delta$ to be
admissible with given risk $R$ is that there exists a non-negative measure $F$ s.t. its
density $f^{*}<\infty$ and $\delta(x)=\delta_{F}(x)$(generalized Bayes
rule w.r.t. F) for almost all $x\in E^{m}$ w.r.t. Lebesgue measure.
Furthermore
(A) If $\{Z_{t}\}$ is transient then $\delta$ is inadmissible.
(B) If $\{Z_{t}\}$ is recurrent and the risk set w.r.t. $R$ is bounded uniformly
then $\delta$ is inadmissible.
where $\{Z_{t}\}$ is the diffusion process in $E^{m}$ along with its
infinitesimal generator with local mean
$\nabla(logf^{*})=\delta_{F}(x)-x$ and a local variance covariance
matrix $2I$.
Therefore since 1d and 2d Brownian motions are recurrent, so is the mean estimator; but when it goes to 3d, Brownian motions are transient and hence mean estimators are no longer admissible and beaten by JS estimator.
This is probably one of the most celebrated results derived from Bayesian statistics and it integrated the stochastic process so seamlessly that I believe it even lead to later MCMC simulation developments.
(2)Geometric interpretation of JS estimator.(taught by Prof.M.P. :)
Zhao and Brown [3] pointed out that if we restricted ourselves to spherically symmetric estimators, then the naive geometric optimal estimator derived from simple Euclidean geometry $$\delta_{NGO}(Z)=\left[1-\frac{p-1}{\left\Vert Z\right\Vert ^{2}}\right]Z$$ is providing exact amount of shrinkage as JS estimator did. This approach is intuitive in sense that spherical symmetric estimators of the form $\delta(X)=\tau(\|X\|)X$ with a scalar function $\tau$, the key of using this class of estimators is that (i) By admissibility consideration, estimators that are not spherically symmetric are inadmissible (ii) By structure of the spherical estimator, it provides a natural geometry that is isometric to Euclidean geometry. By (i)(ii), the problem of choosing an admissible estimator becomes a "compass-and-ruler" problem of geometry. Amazing!
Reference
[1]Brown, Lawrence D. "Admissible estimators, recurrent diffusions, and insoluble boundary value problems." The Annals of Mathematical Statistics 42.3 (1971): 855-903. http://projecteuclid.org/euclid.aoms/1177693318
[2]https://stats.stackexchange.com/questions/13494/intuition-behind-why-steins-paradox-only-applies-in-dimensions-ge-3
[3]Brown, Lawrence D., and Linda H. Zhao. "A Geometrical Explanation of Stein Shrinkage." Statistical Science (2012): 24-30.
