I am reading V. Vapnik's "Statistical Learning Theory". The author layouts following two statistical laws related to empirical CDF. I am looking for reference about proofs on these two laws.

Let $F_l(x)$ be the empirical CDF from $l$ i.i.d samples drawing from same distribution with CDF $F(x)$.

Iterated Logarithm Law: $$ \mathbb{P}\left( \limsup_{l \to \infty} \sup_x \sqrt{\frac{l}{\ln\ln l}}|F_l(x) - F(x)| = 1 \right) = 1 $$

Smirnov Law: $$ \lim_{l \to \infty} \mathbb{P}\left( l \int (F_l(x) - F(x))^2 dF(x) < \epsilon \right) = 1 - \frac{2}{\pi} \sum_{k=1}^\infty \int_{(2k-1)\pi}^{2k\pi} \frac{\exp(-\lambda^2\epsilon/2)}{\sqrt{-\lambda \sin \lambda}} d\lambda $$

some comments

  1. Iterated Logarithm Law: I think it can't trivially be implied by classical iterated logarithm law for i.i.d. sequence, because the law holds true uniformly for all $x$.

  2. Smirnov Law: I know the law was proved first by Smirnov in Russian article. I am looking for English reference with formal proof of this law.

  • $\begingroup$ Note the statistic appearing in the Smirnov Law is better known as the Cramer-von Mises statistic (or Cramer-von Mises-Smirnov). $\endgroup$ Aug 27, 2015 at 19:24
  • $\begingroup$ This looks might it may contain hints-> epubs.siam.org/doi/abs/10.1137/1119082 but I don't have access. It makes the useful point that you only need to consider the uniform[0,1] distribution (assuming $F$ is continuous). I guess the easiest proof would proceed by writing the statistic in terms of the empirical process, using the fact that that converges to a Gaussian process, and then using something about the distribution of the latter. But this is just idle speculation :) $\endgroup$ Aug 27, 2015 at 19:52
  • $\begingroup$ Here is an article that summarises the proof I suggested above = projecteuclid.org/euclid.aop/1020107767 (the pdf is open access, projecteuclid.org/download/pdf_1/euclid.aop/1020107767) $\endgroup$ Aug 27, 2015 at 20:06
  • 1
    $\begingroup$ A pedagogical comment: The Smirnov (von Mises - Cramer) statistic uses the integral of the squared empirical process, whereas the Kolmogorov-Smirnov statistic uses the maximum absolute value of the EP. $\endgroup$ Aug 27, 2015 at 20:29

2 Answers 2


For Smirnov's result I think the easiest approach (at least on a "hand waving" level) is via empirical processes:

As long as $F$ is continuous then it suffices to consider uniform distributions.

Let $\Delta_n(t) = n^{1/2}(F_n(t) - t)$.

Then the Smirnov Cramer von Mises statistic you wrote is $$ \int_0^1 \Delta_n(t)^2 dt. $$

But $\Delta_n(t)$ converges weakly to a Brownian bridge [Kolmogorov, Doob, Donsker, Andersson, .... proofs dripping from many books].

So, sweeping away some boring technicalities, $$ \int_0^1 \Delta_n(t)^2 dt \to \int_0^1 BB(t)^2 dt $$
where $BB$ is a standard Brownian bridge.

The distribution of the RHS is that which Smirnov derived.

[I think the link https://projecteuclid.org/download/pdf_1/euclid.aop/1020107767 I posted contains some refs on how to evaluate the distribution of the integral of a squared BB].

  • $\begingroup$ Thanks for answer my question. Any idea about the iterated logarithm law?? $\endgroup$
    – Yan Zhu
    Aug 27, 2015 at 22:47
  • $\begingroup$ I think Kallenberg's foundations of modern probability has it but I don't have my copy to hand $\endgroup$ Aug 28, 2015 at 6:31
  • 1
    $\begingroup$ I couldn't find that form of LIL in Kallenberg,and realised actually I haven't seen it before. BTW convergence of the empirical process to a Brownian bridge is Theorem 14.15 p278 in the second edition of Kallenberg. (You can find a more precise version by googling "Hungarian Embedding" or KMT embedding). $\endgroup$ Aug 29, 2015 at 20:55

I think the proofs are given in the book http://www.amazon.com/Probability-The-Classical-Limit-Theorems/dp/110762827X (Probability: The Classical Limit Theorems, by Henry McKean). Concerning the Smirnov Law, see also http://projecteuclid.org/euclid.aoms/1177730243 (On the Kolmogorov-Smirnov Limit Theorems for Empirical Distributions, by W. Feller).

  • $\begingroup$ I checked the references. What they refer to Kolmogorov-Smirnov Law is different from what I mentioned here. Could you please doubly check them? $\endgroup$
    – Yan Zhu
    Aug 18, 2015 at 1:40
  • $\begingroup$ Indeed they are different. I found the initial Smirnov's paper but it is in Russian: mathnet.ru/php/… Maybe this paper will be of some help: google.ru/… $\endgroup$ Aug 18, 2015 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.