Let $X_1,X_2,\ldots,X_n$ be i.i.d. random variables in $\mathbb{R}$ with common cumulative distribution function (CDF) $F(x)$. The empirical approximation to $F(x)$ is defined as follows:

$$\hat{F}_n(x)=\frac{1}{n}\sum_{i=1}^n\mathbf{1}_{\{X_i\leq x\}}(x)\tag{1}$$

where $\mathbf{1}_A(x)$ is an indicator function: unity when $x\in A$ and zero otherwise.

Per an answer to my previous question, by Donsker's Theorem the scaled error process $G_n(x)\equiv\sqrt{n}\left(\hat{F}_n(x)-F(x)\right)$ converges to the Brownian bridge process $B(F(x))$.

I am interested in the error of a point estimate $\hat{F}_n(x_0)$. The first sentence of the "History" section of the Wikipedia article on the Donsker's Theorem states the following:

By the classical central limit theorem, for fixed $x$, the random variable $G_n(x)$ converges in distribution to a Gaussian (normal) random variable $G(x)$ with zero mean and variance $F(x)(1 − F(x))$ as the sample size $n$ grows.

I have two questions about this statement (in order of importance):

  1. What is the standard approach when $F(x)$ is unknown? Can one use the estimate $\hat{F}_n(x)$? (similarly to how we use the sample variance estimator to approximate the confidence interval for unknown mean given unknown variance)

  2. While I am familiar with the standard proofs and the use of the classical CLT, I don't see how one proves the statement above using standard CLT. Can someone provide a hint?


If you're only interested in estimating $F$ at a single point $x_0$, then you are really just estimating the probability $p_0 = \mathbb{P}\left(X\in (-\infty,x_0]\right)$. The indicator variables $B_i = [X_i\leq x_0]$ are i.i.d. Bernoulli with (potentially unknown) probability $p_0$, so you can apply the usual CLT machinery to get the stated result and from there a confidence interval for $p_0$.

Similar methods work if you care about $F$ at finitely many points. You only really need Donsker's Theorem to understand the whole process $G$ at once.

  • $\begingroup$ Aha -- the key word is "Bernoulli"! I think I understand how this works now. All this is new to me, and I appreciate your help. $\endgroup$ – Bullmoose Mar 24 '14 at 21:19

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.