show/hide this revision's text 2 added 337 characters in body

Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing.

It is claimed that if one wants to estimate the mean $\Theta$ of
Gaussian distributed vector $ y$~$ N(\Theta, \sigma^2 Id)$, then the naive estimation - (i.e. just take $y$ as an estimation) is not good for size of vector greater or equal 3.

"Not good" means that (quote Wikipedia) "James–Stein estimator always achieves lower (Mean squared error (MSE) than the least squares estimator".

Question please clarify the sentence above. I wonder the following - usually when we calculate MSE we need some distribution on the estimated parameter $\Theta$ and averaging in MSE is taken over this distribution also.

So what distribution is assumed ? Or may be for ANY distribution it holds true ?

PS

Stein's example is more general (quote Wikipedia):

Stein's example (or phenomenon or paradox), in decision theory and estimation theory, is the phenomenon that when three or more parameters are estimated simultaneously, there exist combined estimators more accurate on average (that is, having lower expected mean-squared error) than any method that handles the parameters separately. This is surprising since the parameters and the measurements might be totally unrelated.

A Paradox?

Popular articles have appeared hailing the James-Stein estimator a paradox; one should use the price of tea in China to obtain a better estimate of the chance of rain in Melbourne!

(Quote from Deane Yang's suggested page http://jmanton.wordpress.com/2010/06/05/comments-on-james-stein-estimation-theory/ )
show/hide this revision's text 1

The James–Stein estimator - counterintuitive estimation of the mean. What means it is better than least squares ? (Understanding Wikipedia)

Background James-Stein estimator and Stein's phenomenon, as described in Wikipedia are rather counterintuitive and amazing.

It is claimed that if one wants to estimate the mean $\Theta$ of
Gaussian distributed vector $ y$~$ N(\Theta, \sigma^2 Id)$, then the naive estimation - (i.e. just take $y$ as an estimation) is not good for size of vector greater or equal 3.

"Not good" means that (quote Wikipedia) "James–Stein estimator always achieves lower (Mean squared error (MSE) than the least squares estimator".

Question please clarify the sentence above. I wonder the following - usually when we calculate MSE we need some distribution on the estimated parameter $\Theta$ and averaging in MSE is taken over this distribution also.

So what distribution is assumed ? Or may be for ANY distribution it holds true ?

PS

Stein's example is more general (quote Wikipedia):

Stein's example (or phenomenon or paradox), in decision theory and estimation theory, is the phenomenon that when three or more parameters are estimated simultaneously, there exist combined estimators more accurate on average (that is, having lower expected mean-squared error) than any method that handles the parameters separately. This is surprising since the parameters and the measurements might be totally unrelated.