# Reference to iterated logarithm law and Smirnov law of empirical CDF

I am reading V. Vapnik's "Statistical Learning Theory". The author layouts following two statistical laws related to empirical CDF. I am looking for reference about proofs on these two laws.

Let $F_l(x)$ be the empirical CDF from $l$ i.i.d samples drawing from same distribution with CDF $F(x)$.

Iterated Logarithm Law: $$\mathbb{P}\left( \limsup_{l \to \infty} \sup_x \sqrt{\frac{l}{\ln\ln l}}|F_l(x) - F(x)| = 1 \right) = 1$$

Smirnov Law: $$\lim_{l \to \infty} \mathbb{P}\left( l \int (F_l(x) - F(x))^2 dF(x) < \epsilon \right) = 1 - \frac{2}{\pi} \sum_{k=1}^\infty \int_{(2k-1)\pi}^{2k\pi} \frac{\exp(-\lambda^2\epsilon/2)}{\sqrt{-\lambda \sin \lambda}} d\lambda$$

1. Iterated Logarithm Law: I think it can't trivially be implied by classical iterated logarithm law for i.i.d. sequence, because the law holds true uniformly for all $x$.

2. Smirnov Law: I know the law was proved first by Smirnov in Russian article. I am looking for English reference with formal proof of this law.

• Note the statistic appearing in the Smirnov Law is better known as the Cramer-von Mises statistic (or Cramer-von Mises-Smirnov). – P.Windridge Aug 27 '15 at 19:24
• This looks might it may contain hints-> epubs.siam.org/doi/abs/10.1137/1119082 but I don't have access. It makes the useful point that you only need to consider the uniform[0,1] distribution (assuming $F$ is continuous). I guess the easiest proof would proceed by writing the statistic in terms of the empirical process, using the fact that that converges to a Gaussian process, and then using something about the distribution of the latter. But this is just idle speculation :) – P.Windridge Aug 27 '15 at 19:52
• Here is an article that summarises the proof I suggested above = projecteuclid.org/euclid.aop/1020107767 (the pdf is open access, projecteuclid.org/download/pdf_1/euclid.aop/1020107767) – P.Windridge Aug 27 '15 at 20:06
• A pedagogical comment: The Smirnov (von Mises - Cramer) statistic uses the integral of the squared empirical process, whereas the Kolmogorov-Smirnov statistic uses the maximum absolute value of the EP. – P.Windridge Aug 27 '15 at 20:29

For Smirnov's result I think the easiest approach (at least on a "hand waving" level) is via empirical processes:

As long as $F$ is continuous then it suffices to consider uniform distributions.

Let $\Delta_n(t) = n^{1/2}(F_n(t) - t)$.

Then the Smirnov Cramer von Mises statistic you wrote is $$\int_0^1 \Delta_n(t)^2 dt.$$

But $\Delta_n(t)$ converges weakly to a Brownian bridge [Kolmogorov, Doob, Donsker, Andersson, .... proofs dripping from many books].

So, sweeping away some boring technicalities, $$\int_0^1 \Delta_n(t)^2 dt \to \int_0^1 BB(t)^2 dt$$
where $BB$ is a standard Brownian bridge.

The distribution of the RHS is that which Smirnov derived.

[I think the link https://projecteuclid.org/download/pdf_1/euclid.aop/1020107767 I posted contains some refs on how to evaluate the distribution of the integral of a squared BB].

• Thanks for answer my question. Any idea about the iterated logarithm law?? – Yan Zhu Aug 27 '15 at 22:47
• I think Kallenberg's foundations of modern probability has it but I don't have my copy to hand – P.Windridge Aug 28 '15 at 6:31
• I couldn't find that form of LIL in Kallenberg,and realised actually I haven't seen it before. BTW convergence of the empirical process to a Brownian bridge is Theorem 14.15 p278 in the second edition of Kallenberg. (You can find a more precise version by googling "Hungarian Embedding" or KMT embedding). – P.Windridge Aug 29 '15 at 20:55

I think the proofs are given in the book http://www.amazon.com/Probability-The-Classical-Limit-Theorems/dp/110762827X (Probability: The Classical Limit Theorems, by Henry McKean). Concerning the Smirnov Law, see also http://projecteuclid.org/euclid.aoms/1177730243 (On the Kolmogorov-Smirnov Limit Theorems for Empirical Distributions, by W. Feller).

• I checked the references. What they refer to Kolmogorov-Smirnov Law is different from what I mentioned here. Could you please doubly check them? – Yan Zhu Aug 18 '15 at 1:40
• Indeed they are different. I found the initial Smirnov's paper but it is in Russian: mathnet.ru/php/… Maybe this paper will be of some help: google.ru/… – Zurab Silagadze Aug 18 '15 at 16:04