I'm trying to familiarize myself with the latest results in finite sample statistics. It seems to me that these results can be classified into two categories:
- Unsurprising results confirm that the asymptotic behavior of a statistic behaves similarly to the finite sample behavior of a statistic. For example:
It is well known that the maximum likelihood estimator is asymptotically normally distributed with rate $O(1/\sqrt n)$. Many recent papers confirm similar behavior for the MLE in the finite sample regime (e.g. Spokoiny's Parametric estimation. Finite sample theory). These results currently require stronger assumptions than the classical results, but it seems possible that these results will eventually be strengthened to match their classical counterparts.
Many results on random matrices have very similar results in the finite sample and asymptotic regimes. For example, bounds on the minimum and maximum eigenvalues of random matrices are comparable in both regimes.
- Surprising Results show that a statistic has different behavior in the finite sample regime than in the asymptotic regime. For example:
- The empirical mean is an asymptotically normal estimator of the true mean, but the empirical mean is not subgaussian for finite samples. There's been considerable work on finding new mean estimators that are subgaussian for finite samples (e.g. Devroye et. al.'s Sub-Gaussian mean estimators).
What are some other surprising results of finite sample statistics?
