Let $X_0, X_1, X_2, \ldots$ be a sequence of i.i.d. real-valued random variables on some probability space $(\Omega, \mathcal{F}, \mathbb{P})$ with continuous CDF $F(x)$ and define a sequence of empirical CDFs $F_n(x) = \frac{1}{n} \sum_{i = 1}^n \mathbf{1}_{\{X_i \leq x\}}(x)$ ($\mathbf{1}_A$ is an indicator function). It is well known that

$$\sup_{x \in \mathbb{R}} \left|F_n(x) - F(x)\right| \to 0 \text{ a.s.}$$

(This is the fundamental theorem of mathematical statistics, or FTMS, supposedly.) Are we then safe to say that

$$F_n(X_0) \to U \text{ in law}$$

where $U$ follows a standard uniform distribution? Or could we make the even stronger claim that we can redefine the random variables into a probability space such that the convergence happens almost surely?

It seems like this would be the case. FTMS suggests we can write $F_n(X_0) = F(X_0) + o(1)$ (holding almost surely) and the distribution of $F(X_0)$ is a standard uniform distribution. While $F_n$ is affected by the asymptotics, $X_0$ is not. The probability $X_0$ takes a value that is not covered by the convergence is zero thanks to FTMS being a uniform convergence result. Is it that easy? Am I overthinking this?

  • $\begingroup$ I don't see why $F_{X_0}(X_0)$ would be uniformly distributed for an arbitrary rv $X_0$. One obvious problem is that $F_{X_0}$ need not take all values in $[0,1]$. $\endgroup$ Apr 23 at 17:18
  • $\begingroup$ @ChristianRemling You are right. CDF should be continuous at least. I fixed the post. $\endgroup$
    – cgmil
    Apr 23 at 19:15

1 Answer 1


$\newcommand\Om\Omega\newcommand\om\omega\newcommand\R{\mathbb R}$This is indeed straightforward. Spelling out $$\sup_{x\in\R}|F_n(x)-F(x)|\to0 \text{ a.s.}, $$ we see that for some subset $N$ of $\Om$ of outer probability $0$ and all $\om\in\Om_0:=\Om\setminus N$ we have $$\sup_{x\in\R}\Big|\frac1n\sum_{j=1}^n 1(X_j(\om)\le x)-F(x)\Big|\to0$$ and hence $$\frac1n\sum_{j=1}^n 1(X_j(\om)\le X_0(\om))\to F(X_0(\om)).$$

So, indeed we have $$F_n(X_0)\to F(X_0)$$ a.s. and hence in law. Also, if $F$ is continuous, then the random variable $F(X_0)$ has the standard uniform distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.