4
$\begingroup$

Suppose that $f \colon [0, 1] \to \mathbb{R}$ is a $1$-Lipschitz function. Define the uniform norm $\|f\|_\infty = \sup_{x} |f(x)|$.

Given $\{U_j\}_{j=1}^\infty$ independent and identically distributed uniform random variables on the unit interval $[0, 1]$, I am curious to know about the behavior of the random variable
$$ \Delta_n(f) := \Big|\max_{j \leq n} |f(U_j)| - \|f\|_\infty\Big| $$ I would roughly expect that $\Delta_n \sim \frac{1}{n}$, with high probability simply because on average one of the $n$ samples $U_1, \dots, U_n$ should lie within $1/n$ of the maximum, and therefore, the error should be of this order. I am also interested to what degree this can be made uniform over Lipschitz functions $f$, i.e., by studying the maximal quantity $\sup_{f \in \mathcal{F}} \Delta_n(f)$ where $\mathcal{F}$ is a suitable family of Lipschitz functions (say with $f(0) = 0$). Is this question studied? If so, are there any references available?

$\endgroup$
0

2 Answers 2

4
$\begingroup$

$\newcommand\De\Delta\newcommand{\Ga}{\Gamma}\newcommand\ga{\gamma}$Let $X_j:=g(U_j)$, where $g:=|f|$. Then the $X_i$'s are i.i.d. random variables and $g$ is a $1$-Lipschitz function. We have $$M:=\max_{[0,1]}g=g(u)$$ for some $u\in[0,1]$.

For any real $h>0$, \begin{align*} P(\De_n(f)>h)&=P(M-\max_1^n X_j>h) \\ &=P(\max_1^n X_j<M-h) \\ &=P(X_1<M-h)^n \\ &=P(g(U_1)<g(u)-h)^n \\ &=P(g(u)-g(U_1)>h)^n \\ &\le P(|U_1-u|>h)^n \tag{1}\label{1} \\ &\le(1-h)^n\le e^{-nh}; \end{align*} the $\le$ inequality in \eqref{1} holds because $g$ is $1$-Lipschitz.

So, $\De_n(f)$ is stochastically dominated by an exponentially distributed random variable with mean $1/n$, which confirms your expectation. In particular, $E\De_n(f)\le1/n$.


Consider now \begin{equation*} \De_n(F):=\sup_{f\in F}\De_n(f), \end{equation*} where $F$ is the set of all $1$-Lipschitz functions on $[0,1]$. Let us show that \begin{equation*} E\De_n(F)\sim\frac{\ln n}{2n} \tag{2}\label{2} \end{equation*} and \begin{equation*} \De_n(F)\Big/\frac{\ln n}{2n}\to1 \tag{3}\label{3} \end{equation*} in probability (as $n\to\infty$).

Indeed, it is easy to see that \begin{equation*} \De_n(F)=\frac12\,\max(2G_1,G_2,\dots,G_n,2G_{n+1}) =\frac{M_n}2+L_n, \tag{4}\label{4} \end{equation*} where \begin{equation*} M_n:=\max(G_1,G_2,\dots,G_n,G_{n+1}), \end{equation*} \begin{equation*} 0\le L_n\le\frac12\,\max(G_1,G_{n+1}), \tag{5}\label{5} \end{equation*} \begin{equation*} G_i:=U_{n:i}-U_{n:i-1} \end{equation*} for $i=1,\dots,n+1$; $U_{n:1}\le\dots\le U_{n:n}$ are the order statistics for the $U_j$'s; $U_{n:0}:=0$; and $U_{n:n+1}:=1$.

A result by Fisher 1929 (see e.g. formula (1.7)) is that for $x\in[0,1]$ we have \begin{equation*}\label{eq:fisher} P(M_n>x)=\sum_{j=1}^{n+1}(-1)^{j-1}\binom{n+1}j(1-jx)_+^n, \end{equation*} where $u_+:=\max(0,u)$. So, \begin{equation*} EM_n=\int_0^1dx\,P(M_n>x) =\sum_{j=1}^{n+1}(-1)^{j-1}\binom{n+1}j\int_0^1dx\,(1-jx)_+^n =\frac{\psi(n+2)+\ga}{n+1}\sim\frac{\ln n}n, \tag{6}\label{6} \end{equation*} where $\psi:=\Ga'/\Ga$ and $\ga=0.577\dots$ is Euler's gamma; these results for $EM_n$ were also given at the end of Section 1 of the linked paper.

Writing $EM_n^2=\int_0^1dx\,2x\,P(M_n>x)$, we similarly get \begin{equation*} Var\,M_n\sim\frac{\pi^2}{6n^2}. \end{equation*} So, by \eqref{6} and Chebyshev's inequality, \begin{equation*} M_n\Big/\frac{\ln n}n\to1 \tag{7}\label{7} \end{equation*} in probability. In view of \eqref{5}, it is easy to see that $EL_n=O(1/n)$ and hence $L_n/\frac{\ln n}n\to0$ in probability.

Now \eqref{2} and \eqref{3} follow from \eqref{4}, \eqref{6}, and \eqref{7}. $\quad\Box$


From the first equality in \eqref{4} and formula (2.2) of the linked paper, one can get an exact expression for the cdf of $\De_n(F)$: for all real $x\ge0$, \begin{equation*} P(2\De_n(F)\le x)\\ =\sum_{k=0}^{n-1}(-1)^k\binom{n-1}k [(1-kx)_+^n-2(1-(k+\tfrac12)x)_+^n+(1-(k+1)x)_+^n]. \end{equation*} This yields an exact expression for $E\De_n(F)$: \begin{equation*} E\De_n(F)=\frac1{2(n+1)}\Big(\psi(n)+\ga+\frac{2\sqrt{\pi}\,\Ga (n)}{\Ga(n+1/2)} -\frac1n\Big), \end{equation*} which, in turn, again yields \eqref{2}.

$\endgroup$
1
  • $\begingroup$ Exact expressions and asymptotics for the distribution of the supremum of the $\Delta_n(f)$ over all $1$-Lipschitz functions $f$ are now given. $\endgroup$ Commented Mar 8, 2023 at 16:51
3
$\begingroup$

As you correctly notice, for a given Lipschitz function $f$, the $\mathbb{E} \Delta_n(f) = \mathcal{O}(1/n)$, and it is relatively simple to convert your argument into a formal proof.

Moreover, this is well-concentrated: $$\mathbb{P}(\Delta_n(f) > \lambda / n) \leq \exp(-\Omega(\lambda))$$ follows by noticing that the expected number of $U_i$ in the interval $I$ of length $\lambda/n$ around the maximum of $f$ is at least $\lambda/2$. Therefore, if we take $X_i$ to be the indicator of the event $U_i \in I$ applying Hoeffding bound to the sum $\sum_i X_i$ yields the probability that $$\mathbb{P}(\forall i\leq n,\, U_i \not\in I) \leq \exp(-\Omega(\lambda)).$$

The latter quantity is actully much bigger $$\mathbb{E} \sup_{f \in \mathcal{F}} \Delta_n(f) \approx \frac{\log n}{n}.$$

More concretely, if we order $U_{1}, \ldots U_{n}$ in an increasing order $U_{(1)} \leq \ldots \leq U_{(n)}$, then $\sup_{f \in \mathcal{F}} \Delta_n(f)$ is essentially equal $\frac{1}{2}\max_{j} |U_{(j)} - U{(j-1)}|$ (up to small defects at the ends of interval that are easy to fix) - we can realize it by exhibiting a piecewise linear Lipschitz function $f$ which is $0$ on all $U_{(j)}$ and as large as $\frac{1}{2}|U{(j)} - U_{(j-1)}|$ the middle of the interval $[U_{(j-1)}, U_{(j)}]$.

To see that with constant probability, if we chose $n$ points uniformly at random on $[0,1]$, there will be a consecutive pair with distance $\Omega(\log n/n)$, we can just partition $[0,1]$ into intervals $I_0, I_1, \ldots I_m$ all of length $\frac{c \log n}{n}$ for some small constant $c$. The expected number of points $U_i$ falling into interval $I_k$ is equal to $c \log n$, and for each fixed interval $I_k$, we should be able to show that $\mathbb{P}(\forall_{i\leq n} U_i \not\in I_k) \geq n^{-c'}$ for some $c'$ depending on $c$. By choosing $c$ properly, we can make this probability at least $n^{-0.1}$. Therefore in expectation there is least $n^{0.8}$ empty intervals $I_k$, and a variance calculation is enough to deduce that with constant probability there are some empty intervals $I_k$. But clearly, if one of the intervals $I_k$ of length $\frac{c \log n}{n}$ is empty, there are two consecutive points among $U_{(i)}$ with distance between them at least $\frac{c \log n}{n}$. This completes the proof that $\mathbb{E}_U \sup_{f \in \mathcal{F}} \Delta_n(f) \geq \Omega(\frac{\log n}{n})$.

Upper bound, and moreover a concentration $$\Pr(\sup_{f \in \mathcal{F}} \Delta_n(f) \geq \lambda \frac{\log n}{n}) \leq \exp(-\Omega(\lambda))$$ can be shown in a very similar way.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .