5
$\begingroup$

I am reading V. Vapnik's "Statistical Learning Theory". The author layouts following two statistical laws related to empirical CDF. I am looking for reference about proofs on these two laws.

Let $F_l(x)$ be the empirical CDF from $l$ i.i.d samples drawing from same distribution with CDF $F(x)$.

Iterated Logarithm Law: $$ \mathbb{P}\left( \limsup_{l \to \infty} \sup_x \sqrt{\frac{l}{\ln\ln l}}|F_l(x) - F(x)| = 1 \right) = 1 $$

Smirnov Law: $$ \lim_{l \to \infty} \mathbb{P}\left( l \int (F_l(x) - F(x))^2 dF(x) < \epsilon \right) = 1 - \frac{2}{\pi} \sum_{k=1}^\infty \int_{(2k-1)\pi}^{2k\pi} \frac{\exp(-\lambda^2\epsilon/2)}{\sqrt{-\lambda \sin \lambda}} d\lambda $$

some comments

  1. Iterated Logarithm Law: I think it can't trivially be implied by classical iterated logarithm law for i.i.d. sequence, because the law holds true uniformly for all $x$.

  2. Smirnov Law: I know the law was proved first by Smirnov in Russian article. I am looking for English reference with formal proof of this law.

Improve this question
$\endgroup$
4
  • $\begingroup$ Note the statistic appearing in the Smirnov Law is better known as the Cramer-von Mises statistic (or Cramer-von Mises-Smirnov). $\endgroup$
    – P.Windridge
    Commented Aug 27, 2015 at 19:24
  • $\begingroup$ This looks might it may contain hints-> epubs.siam.org/doi/abs/10.1137/1119082 but I don't have access. It makes the useful point that you only need to consider the uniform[0,1] distribution (assuming $F$ is continuous). I guess the easiest proof would proceed by writing the statistic in terms of the empirical process, using the fact that that converges to a Gaussian process, and then using something about the distribution of the latter. But this is just idle speculation :) $\endgroup$
    – P.Windridge
    Commented Aug 27, 2015 at 19:52
  • $\begingroup$ Here is an article that summarises the proof I suggested above = projecteuclid.org/euclid.aop/1020107767 (the pdf is open access, projecteuclid.org/download/pdf_1/euclid.aop/1020107767) $\endgroup$
    – P.Windridge
    Commented Aug 27, 2015 at 20:06
  • 1
    $\begingroup$ A pedagogical comment: The Smirnov (von Mises - Cramer) statistic uses the integral of the squared empirical process, whereas the Kolmogorov-Smirnov statistic uses the maximum absolute value of the EP. $\endgroup$
    – P.Windridge
    Commented Aug 27, 2015 at 20:29

2 Answers 2

Reset to default
1
$\begingroup$

For Smirnov's result I think the easiest approach (at least on a "hand waving" level) is via empirical processes:

As long as $F$ is continuous then it suffices to consider uniform distributions.

Let $\Delta_n(t) = n^{1/2}(F_n(t) - t)$.

Then the Smirnov Cramer von Mises statistic you wrote is $$ \int_0^1 \Delta_n(t)^2 dt. $$

But $\Delta_n(t)$ converges weakly to a Brownian bridge [Kolmogorov, Doob, Donsker, Andersson, .... proofs dripping from many books].

So, sweeping away some boring technicalities, $$ \int_0^1 \Delta_n(t)^2 dt \to \int_0^1 BB(t)^2 dt $$
where $BB$ is a standard Brownian bridge.

The distribution of the RHS is that which Smirnov derived.

[I think the link https://projecteuclid.org/download/pdf_1/euclid.aop/1020107767 I posted contains some refs on how to evaluate the distribution of the integral of a squared BB].

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for answer my question. Any idea about the iterated logarithm law?? $\endgroup$
    – Yan Zhu
    Commented Aug 27, 2015 at 22:47
  • $\begingroup$ I think Kallenberg's foundations of modern probability has it but I don't have my copy to hand $\endgroup$
    – P.Windridge
    Commented Aug 28, 2015 at 6:31
  • 1
    $\begingroup$ I couldn't find that form of LIL in Kallenberg,and realised actually I haven't seen it before. BTW convergence of the empirical process to a Brownian bridge is Theorem 14.15 p278 in the second edition of Kallenberg. (You can find a more precise version by googling "Hungarian Embedding" or KMT embedding). $\endgroup$
    – P.Windridge
    Commented Aug 29, 2015 at 20:55
3
$\begingroup$

I think the proofs are given in the book http://www.amazon.com/Probability-The-Classical-Limit-Theorems/dp/110762827X (Probability: The Classical Limit Theorems, by Henry McKean). Concerning the Smirnov Law, see also http://projecteuclid.org/euclid.aoms/1177730243 (On the Kolmogorov-Smirnov Limit Theorems for Empirical Distributions, by W. Feller).

Improve this answer
$\endgroup$
2
  • $\begingroup$ I checked the references. What they refer to Kolmogorov-Smirnov Law is different from what I mentioned here. Could you please doubly check them? $\endgroup$
    – Yan Zhu
    Commented Aug 18, 2015 at 1:40
  • $\begingroup$ Indeed they are different. I found the initial Smirnov's paper but it is in Russian: mathnet.ru/php/… Maybe this paper will be of some help: google.ru/… $\endgroup$
    – Zurab Silagadze
    Commented Aug 18, 2015 at 16:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .