5
$\begingroup$

I am reading V. Vapnik's "Statistical Learning Theory". The author layouts following two statistical laws related to empirical CDF. I am looking for reference about proofs on these two laws.

Let $F_l(x)$ be the empirical CDF from $l$ i.i.d samples drawing from same distribution with CDF $F(x)$.

Iterated Logarithm Law: $$ \mathbb{P}\left( \limsup_{l \to \infty} \sup_x \sqrt{\frac{l}{\ln\ln l}}|F_l(x) - F(x)| = 1 \right) = 1 $$

Smirnov Law: $$ \lim_{l \to \infty} \mathbb{P}\left( l \int (F_l(x) - F(x))^2 dF(x) < \epsilon \right) = 1 - \frac{2}{\pi} \sum_{k=1}^\infty \int_{(2k-1)\pi}^{2k\pi} \frac{\exp(-\lambda^2\epsilon/2)}{\sqrt{-\lambda \sin \lambda}} d\lambda $$

some comments

  1. Iterated Logarithm Law: I think it can't trivially be implied by classical iterated logarithm law for i.i.d. sequence, because the law holds true uniformly for all $x$.

  2. Smirnov Law: I know the law was proved first by Smirnov in Russian article. I am looking for English reference with formal proof of this law.

$\endgroup$
  • $\begingroup$ Note the statistic appearing in the Smirnov Law is better known as the Cramer-von Mises statistic (or Cramer-von Mises-Smirnov). $\endgroup$ – P.Windridge Aug 27 '15 at 19:24
  • $\begingroup$ This looks might it may contain hints-> epubs.siam.org/doi/abs/10.1137/1119082 but I don't have access. It makes the useful point that you only need to consider the uniform[0,1] distribution (assuming $F$ is continuous). I guess the easiest proof would proceed by writing the statistic in terms of the empirical process, using the fact that that converges to a Gaussian process, and then using something about the distribution of the latter. But this is just idle speculation :) $\endgroup$ – P.Windridge Aug 27 '15 at 19:52
  • $\begingroup$ Here is an article that summarises the proof I suggested above = projecteuclid.org/euclid.aop/1020107767 (the pdf is open access, projecteuclid.org/download/pdf_1/euclid.aop/1020107767) $\endgroup$ – P.Windridge Aug 27 '15 at 20:06
  • 1
    $\begingroup$ A pedagogical comment: The Smirnov (von Mises - Cramer) statistic uses the integral of the squared empirical process, whereas the Kolmogorov-Smirnov statistic uses the maximum absolute value of the EP. $\endgroup$ – P.Windridge Aug 27 '15 at 20:29
1
$\begingroup$

For Smirnov's result I think the easiest approach (at least on a "hand waving" level) is via empirical processes:

As long as $F$ is continuous then it suffices to consider uniform distributions.

Let $\Delta_n(t) = n^{1/2}(F_n(t) - t)$.

Then the Smirnov Cramer von Mises statistic you wrote is $$ \int_0^1 \Delta_n(t)^2 dt. $$

But $\Delta_n(t)$ converges weakly to a Brownian bridge [Kolmogorov, Doob, Donsker, Andersson, .... proofs dripping from many books].

So, sweeping away some boring technicalities, $$ \int_0^1 \Delta_n(t)^2 dt \to \int_0^1 BB(t)^2 dt $$
where $BB$ is a standard Brownian bridge.

The distribution of the RHS is that which Smirnov derived.

[I think the link https://projecteuclid.org/download/pdf_1/euclid.aop/1020107767 I posted contains some refs on how to evaluate the distribution of the integral of a squared BB].

$\endgroup$
  • $\begingroup$ Thanks for answer my question. Any idea about the iterated logarithm law?? $\endgroup$ – Yan Zhu Aug 27 '15 at 22:47
  • $\begingroup$ I think Kallenberg's foundations of modern probability has it but I don't have my copy to hand $\endgroup$ – P.Windridge Aug 28 '15 at 6:31
  • 1
    $\begingroup$ I couldn't find that form of LIL in Kallenberg,and realised actually I haven't seen it before. BTW convergence of the empirical process to a Brownian bridge is Theorem 14.15 p278 in the second edition of Kallenberg. (You can find a more precise version by googling "Hungarian Embedding" or KMT embedding). $\endgroup$ – P.Windridge Aug 29 '15 at 20:55
3
$\begingroup$

I think the proofs are given in the book http://www.amazon.com/Probability-The-Classical-Limit-Theorems/dp/110762827X (Probability: The Classical Limit Theorems, by Henry McKean). Concerning the Smirnov Law, see also http://projecteuclid.org/euclid.aoms/1177730243 (On the Kolmogorov-Smirnov Limit Theorems for Empirical Distributions, by W. Feller).

$\endgroup$
  • $\begingroup$ I checked the references. What they refer to Kolmogorov-Smirnov Law is different from what I mentioned here. Could you please doubly check them? $\endgroup$ – Yan Zhu Aug 18 '15 at 1:40
  • $\begingroup$ Indeed they are different. I found the initial Smirnov's paper but it is in Russian: mathnet.ru/php/… Maybe this paper will be of some help: google.ru/… $\endgroup$ – Zurab Silagadze Aug 18 '15 at 16:04

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.