Precise asymptotics for moments of order statistics of normal distribution

Question

Let $X_1, \cdots, X_n \sim N(0,1)$ be i.i.d. normal random variates. I am interested in understanding the first two moments of the quasi-range $X_{(n)}-X_{(n-1)}$ (i.e., the maximum value minus the second-largest value) as $n \to \infty$.

I am trying to use the quantile-function method of section 4.6 of David and Nagaraja's Order Statistics, but am having trouble with convergence of some approximations. The method presented there (originally due to Pearson, later expanded by David and Johnson) is to Taylor-expand the relation $X_{(r)} = Q(U_{(r)})$ about $\mathbb{E}[U_{(r)}]$, where $Q(u) = F^{-1}(u)$ is the quantile function (the inverse of the cdf) and $U_{(r)}$ are the rather-more-well-understood order statistics of the uniform $U[0,1]$ distribution.

The Taylor expansion gives us (Equation 4.6.2), writing $p_r = r/(n+1)$ and $q_r = 1-p_r$, the series $$X_{(r)} = Q(p_r) + \left(U_{(r)}-p_r\right)Q'(p_r) + \frac{1}{2}\left(U_{(r)}-p_r\right)^2Q''(p_r) + \frac{1}{6}\left(U_{(r)}-p_r\right)^2Q'''(p_r) + \cdots.$$

From here, we can use formulas for uniform order statistics to get the first two moments of $X_{(r)}$, as well as covariances $\mathrm{Cov}[X_{(r)},X_{(s)}]$; these are Equations 4.6.3-4.6.5 of David and Nagaraja. For reference, I give the equation for the expected value: $$\mathbb{E}[X_{(r)}] = Q(p_r) + \frac{p_rq_r}{2(n+2)}Q''(p_r) + \frac{p_rq_r}{(n+2)^2}\left[\frac{1}{3}(q_r-p_r)Q'''(p_r)+\frac{1}{8}p_rq_rQ''''(p_r)\right]+O\left(\frac{1}{(n+2)^3}\right).$$

We can use a neat trick (cf. Example 4.6 of ibid.) to calculate the derivatives of the quantile function for a normal distribution; this boils down to the fact that the pdf $f(x) = (2\pi)^{-1/2}\exp(-x^2/2)$ of a standard normal distribution satisfies $f'(x) = -x\cdot f(x)$. This gives us the general relation $$Q^{(k)}(u) = \frac{P_k(Q(u))}{f(Q(u))^k},$$ where $P(x)$ is a family of polynomials satisfying $P_1(x) = 1$ and $P_{k+1}(x) = kxP_k(x) + P'_k(x)$. Importantly, to leading order $P_{k+1}(x) = k!x^k + O(x^{k-1})$.

We can then use the standard approximations $$Q(1-u) \approx \sqrt{\log\left(\frac{1}{2\pi u^2}\right)}, \quad f(Q(1-u)) \approx u\sqrt{\log\left(\frac{1}{2\pi u^2}\right)}, $$ which are valid as $u \to 0$.

So if we want to compute the moments of $X_{(n)}$, say, we would take $p_r = n/(n+1)$ and $q_r = 1/(n+1)$. The above terms of the expected-value equation then become, writing for convenience $s = \sqrt{\log\left(\frac{(n+1)^2}{2\pi}\right)}$, $$\mathbb{E}[X_{(n)}] \approx s + \frac{1}{s}\left(\frac{n}{2(n+2)} + \frac{2n(n-1)}{3(n+2)^2} + \frac{3n^2}{4(n+2)^2} + \cdots\right) + O\left(\frac{1}{s^2}\right).$$

This is fine for estimating $\mathbb{E}[X_{(n)}] \approx s$, but if we want to look at the expected difference $\mathbb{E}[X_{(n)}-X_{(n-1)}]$, we need to understand the $1/s$ term, because the difference between the $s$-terms in the $X_{(n)}$ and $X_{(n-1)}$ term is on the order of $1/s$.

But it's not clear that the coefficient on the $1/s$ term in the series for $\mathbb{E}[X_{(n)}]$ even converges! Indeed, David and Nagaraja warn,

From a practical point of view, the most important feature of the expansion is that convergence may be slow or even nonexistent if $r/n$ is too close to 0 or 1.

For the moments of $X_{(n-1)}$, I think that we get coefficients that are geometrically decreasing, so things are fine. But the same convergence problem plagues our series for $\mathrm{Var}[X_{(n)}]$ and possibly $\mathrm{Cov}[X_{(n)},X_{(n-1)}]$, though I haven't checked the latter series thoroughly.

Question: is there another way to get asymptotic expressions for the first two moments of the normal order statistics $X_{(n)}$ and $X_{(n-1)}$ (as well as their covariance) that has better convergence properties?

Alternate suggestions for understanding the first two moments (or the entire distribution) of the quasi-range $X_{(n)}-X_{(n-1)}$ would also be helpful. It might be possible to use ad-hoc methods (e.g., Fisher–Tippett–Gnedenko) for $X_{(n)}$ and then the series above for $X_{(n-1)}$, but that would rely on the covariance series converging quickly, which I am not yet confident in.

Answer 1

I call $f$ the density and $F$ the cumulative distribution function of $\mathcal{N}(0,1)$. Since $X_{(n)} \ge X_{(n-1)}$, $$X_{(n)}-X_{(n-1)} = \int_\mathbb{R} 1_{[X_{(n-1)} \le x < X_{(n)}]}dx.$$ Taking expectations and applying Fubini theorem for positive functions yields $$E[X_{(n)}-X_{(n-1)}] = \int_\mathbb{R} P[X_{(n-1)} \le x < X_{(n)}]dx.$$ $$E[X_{(n)}-X_{(n-1)}] = \int_\mathbb{R} nF(x)^{n-1}(1-F(x))dx.$$ More generally, for every integer $d \ge 1$, $$(X_{(n)}-X_{(n-1)})^d = \int_{\mathbb{R}^d} 1_{[X_{(n-1)} \le x_1 < X_{(n)}]} \times \cdots \times 1_{[X_{(n-1)} \le x_d < X_{(n)}]}dx_1 \cdots dx_d.$$ $$(X_{(n)}-X_{(n-1)})^d = \int_{\mathbb{R}^d} 1_{[X_{(n-1)} \le \min(x_1,\ldots,x_d) \le \max(x_1,\ldots,x_d) < X_{(n)}]} dx_1 \cdots dx_d.$$ Taking expectations and applying Fubini theorem for positive functions yields $$E[(X_{(n)}-X_{(n-1)})^d] = \int_{\mathbb{R}^d} P[X_{(n-1)} \le \min(x_1,\ldots,x_d) \le \max(x_1,\ldots,x_d) < X_{(n)}] dx_1 \cdots dx_d.$$ $$E[(X_{(n)}-X_{(n-1)})^d] = \int_{\mathbb{R}^d} nF(\min(x_1,\ldots,x_d))^{n-1} \big( 1 - F (\max(x_1,\ldots,x_d)) \big) dx_1 \cdots dx_d.$$ If we want the asymptotic behaviour, we may observe that the function $b$ defined by $$b(x) := F(x)(1-F(x))/f(x)$$ is in $\mathcal{C}_0(\mathbb{R})$ and equivalent to $1/|x|$ as $|x| \to +\infty$. One can write $$E[X_{(n)}-X_{(n-1)}] = \int_\mathbb{R} nF(x)^{n-2}b(x)f(x)dx = \int_0^1 ny^{n-2}b(F^{-1}(y))dy.$$ We get $n/(n-1)$ times the mean value of $b \circ F^{-1}$ for the distribution Bêta$(1,n-1)$, which converges to $\delta_1$ as $n \to +\infty$. Using that $b(F^{-1}(y)) \to 0$ slowly as $y \to 1$, we will get that $E[X_{(n)}-X_{(n-1)}] \to 0$ slowly.

Answer 2

The desired asymptotics are as follows:
\begin{equation*} E(X_{(n)}-X_{(n-1)})\sim\frac1{\sqrt{2\ln n}}, \tag{1}\label{1} \end{equation*} \begin{equation*} E(X_{(n)}-X_{(n-1)})^2\sim\frac1{\ln n}, \tag{1a}\label{1a} \end{equation*} and, generally, for any natural $k$, \begin{equation*} E(X_{(n)}-X_{(n-1)})^k\sim \frac{k!}{(2\ln n)^{k/2}} \tag{1b}\label{1b} \end{equation*} as $n\to\infty$.

Indeed, $X_{(k)}=F^{-1}(U_{(k)})$, where $k\in[n]:=\{1,\dots,n\}$, $F$ is the standard normal c.d.f., and $U_{(k)}$ is the $k$th smallest order statistic for an i.i.d. sample of size $n$ from the uniform distribution over the interval $[0,1]$. So, \begin{equation*} EX_{(k)}=\int_0^1 du\,F^{-1}(u)g_k(u), \end{equation*} and hence \begin{equation*} d_n:=EX_{(n)}-EX_{(n-1)}=I_n:=\int_0^1 du\,F^{-1}(u)(g_n(u)-g_{n-1}(u)), \tag{2}\label{2} \end{equation*} where $g_k$ and $G_k$ are, respectively the p.d.f. and the c.d.f. of $U_{(k)}$.

Note that $1-F(x)=e^{-x^2/\sim2}$ as $x\to\infty$ and $F(x)=e^{-x^2/\sim2}$ as $x\to-\infty$. We write $a\sim b$ and $a\lesssim b$ to mean $a/b=1+o(1)$ and $a/b\le1+o(1)$, respectively. Also we denote by $\sim a$ any expression $b$ such that $b\sim a$. So, \begin{equation*} F^{-1}(u)\underset{u\uparrow1}\sim\sqrt{2\ln\frac1{1-u}} \quad\text{and}\quad F^{-1}(u)\underset{u\downarrow0}\sim\sqrt{2\ln\frac1u}. \tag{3}\label{3} \end{equation*} Therefore and because $F'(x)\sim x(1-F(x))$ as $x\to\infty$ and $F'(x)\sim-xF(x)$ as $x\to-\infty$, for \begin{equation*} z:=(F^{-1})'=\frac1{F'\circ F^{-1}} \tag{3a}\label{3a} \end{equation*} we have \begin{equation*} z(u)\underset{u\uparrow1}\sim\frac1{(1-u)\sqrt{2\ln\frac1{1-u}}} \quad\text{and}\quad z(u)\underset{u\downarrow0}\sim\frac1{u\sqrt{2\ln\frac1u}}. \tag{4}\label{4} \end{equation*} So, recalling \eqref{2}, integrating by parts, and using the known (and easily obtained) expression for $G_k$, we get \begin{equation*} d_n=I_n=\int_0^1 du\,z(u)(G_{n-1}(u)-G_n(u)) \\ =\int_0^1 du\,z(u) nu^{n-1}(1-u)=I_{n1}+I_{n2}, \tag{5}\label{5} \end{equation*} where \begin{equation*} I_{n1}:=\int_0^{1-1/\sqrt m} du\,z(u) nu^m(1-u),\quad I_{n2}:=\int_{1-1/\sqrt m}^1 du\,z(u) nu^m(1-u), \end{equation*} and \begin{equation*} m:=n-1. \end{equation*}

Next, \begin{equation*} |I_{n1}|\le C n(1-1/\sqrt m)^{m-1}\lesssim C me^{-\sqrt m}=o(1/\sqrt{\ln n}) \tag{6}\label{6} \end{equation*} (as $n\to\infty$), where $C:=\int_0^1 du\,|z(u)|u(1-u)<\infty$, in view of \eqref{4}.

Using \eqref{4} again and also the substitution $u=e^{-w}$, we get \begin{equation*} I_{n2}\sim J_{n2}:=\int_{1-1/\sqrt m}^1 \frac{du}{\sqrt{2\ln\frac1{1-u}}} nu^m \sim\int_0^{\sim1/\sqrt m} \frac{dw}{\sqrt{2\ln\frac1w}} me^{-mw} \tag{6a}\label{6a} \end{equation*} and hence \begin{equation*} \sqrt{2\ln m}\, I_{n2} \sim\int_0^{\sim\sqrt m}dt\, r_m(t) e^{-t}, \end{equation*} where \begin{equation*} r_m(t):=\frac{\sqrt{\ln m}}{\sqrt{\ln\frac mt}}. \end{equation*} Note that $r_m(t)\to1$ for each real $t>0$ (as $n\to\infty$), and $\sup_{t\in[0,\sim\sqrt m]}r_m(t)\lesssim\sqrt2$. So, by dominated convergence, \begin{equation*} \sqrt{2\ln n}\, I_{n2} \sim\int_0^\infty dt\,e^{-t}=1. \tag{7}\label{7} \end{equation*}

Collecting \eqref{2}, \eqref{5}, \eqref{6}, and \eqref{7}, we obtain \eqref{1}.

---

To get \eqref{1a}, observe first that, in view of \eqref{3a},
\begin{equation} X_{(n)}-X_{(n-1)}=F^{-1}(U_{(n)})-F^{-1}(U_{(n-1)})=\int_{U_{(n-1)}}^{U_{(n)}} du\,z(u), \end{equation} whence \begin{equation} \begin{aligned} D_n&:=E(X_{(n)}-X_{(n-1)})^2 \\ &=\int_0^1\int_0^1 du\,dv\,z(u)z(v)\,P(u,v\in[U_{(n-1)},U_{(n)}]) \\ &=2\int_0^1 du\,\int_u^1 dv\,z(u)z(v)\,P(U_{(n-1)}<u<v<U_{(n)}) \\ &=2\int_0^1 du\,\int_u^1 dv\,z(u)z(v)\,nu^{n-1}(1-v) \\ &=2\int_0^1 du\,z(u)nu^{n-1}\int_u^1 dv\,z(v)\,(1-v). \\ \end{aligned} \end{equation} The contribution of the values of the integrand of the latter double integral $u<1-1/\sqrt m$ is negligible (cf. \eqref{6}. So, in view of \eqref{4}, \begin{equation} D_n\sim2\int_{1-1/\sqrt m}^1\frac{du}{(1-u)\sqrt{2\ln\frac1{1-u}}}\,nu^{n-1}K(u), \end{equation} where \begin{equation} K(u):=\int_u^1 \frac{dv}{\sqrt{2\ln\frac1{1-v}}} =\int_{\sqrt{2\ln\frac1{1-u}}}^\infty dy\,e^{-y^2/2} \sim\frac{1-u}{\sqrt{2\ln\frac1{1-u}}} \end{equation} as $u\uparrow1$; here we used the substitution $y=\sqrt{2\ln\frac1{1-v}}$. So, \begin{equation} D_n\sim\int_{1-1/\sqrt m}^1\frac{du}{\ln\frac1{1-u}}\,nu^m. \end{equation}
The latter integral is dealt with quite similarly to the previously considered integral $J_{n2}$, defined in \eqref{6a}. This results in \eqref{1a}, and \eqref{1b} is proved similarly. $\quad\Box$