UPDATE: The result below can be understood as an almost sure stochastic equicontinuity condition. I don't know of any result establishing primitives of almost sure stochastic equicontinuity. If you know, this would very likely help me answer the question.

I need the following almost sure convergence result:

Let $\|\cdot\|$ denote a norm on a functional space (could be $\|\cdot\|_{\infty}$ or $\|\cdot \|_2$ in $L^2$, for example). $\mathcal{G}=\{g:\mathcal{Z}\rightarrow \mathbb{R}\}$ of measurable functions. Let $Z_1,Z_2,\dots$ be $i.i.d$ random variables with $E[g(Z_i)]=0$ for any $g\in \mathcal{G}$. Let $\{\delta_n\}$ be a sequence of positive numbers such that $\delta_n\rightarrow 0$. Then, under ADDITIONAL ASSUMPTIONS, \begin{equation*} \sup_{g\in\mathcal{G} \text{ s.t. }\|g\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n g(Z_i)\right|\rightarrow_{a.s.}0. \end{equation*}

Note that all random variables are defined in the same probability space and are independent draws of the same distribution. The number $n$ here denotes the size of the sample, and the almost sure convergence is over the distribution of the $Z_i$ as $n\rightarrow \infty$. The space $\mathcal{G}$ does not depend on $n$, and $g$ is not random.

I need to find which reasonable assumptions could be made to establish this result. $\mathcal{G}$ is $P$-Donsker? Lipschitz functions? Bounds on moments? etc.

Attending requests for clarification of the background: this result is needed to establish a stochastic equicontinuity condition on a linear regression estimator with a generated regressor (it means that one of the regressors is estimated). This is a high level result, in the sense that it is a result for a generic estimator for the regressor, we want to give conditions that this estimator must satisfy, but not specify the estimator. I am using results in a paper by Chen, Linton and Keilegom (2003) which establish the consistency of the bootstrap for estimators which are based on the optimization of a function of the data, the parameter of interest and a nuisance infinite dimension parameter. So, suppose the model is $E[Y|X,W]=\beta X+\gamma h_0(W)$, but $h_0$ is not known. The term is thus estimated and the regression is done on $\hat{h}(W)$ instead. If it helps, you can think of $h_0(W)=E[V|V\leq 0, W]$, for another variable $V$ which is observed in the data, for example. (FIY: this is a huge simplification of the true situation, so don't give me suggestions about how to estimate the model I just described. In the actual model $V$ is not observed and must be predicted out of sample with machine learning and optimization methods.)

In order to apply one of the results in the paper I mentioned, I need to prove that some quantities such as the one above are $o_{a.s.}(1)$. For example, I give you a simplified version of one of them: \begin{equation*} \sup_{h,h_0\in\mathcal{H} \text{ s.t. }\|h-h_0\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n \left[X_i(h(W_i)-h_0(W_i))-E[X_i(h(W_i)-h_0(W_i))\right]\right| \end{equation*} I need to show that several objects that look more or less like the one above are $o_{a.s.}(1)$ as $n\rightarrow \infty$ for any sequence of positive $\delta_n\rightarrow 0$. Note that I can make assumptions about $\mathcal{H}$, $X$ and $W$. For example, I can say that $\mathcal{H}$ is the space of measurable negative Lipschitz functions, and I can say that $E[|X|^4]$ and $E[|W|^4]$ are finite. Basically I need conditions that would allow me to establish this result.

At the essence this is an empirical process. Define $\mathbb{G}_nf=\frac{1}{\sqrt{n}}\sum_{i=1}^n f(Z_i)-E[f(Z_i)]$ and let $\mathbb{G}$ be the Brownian Bridge, then if $\mathcal{G}$ is $P$-Donsker, $\mathbb{G}_ng\rightsquigarrow\mathbb{G}g$ for every fixed $g$. This is a convergence in distribution in a functional space. I need to show that $\sup_{\|g\|_{\mathcal{G}}\leq \delta_n} |\mathbb{G}_n g|\rightarrow_{a.s.} 0$.

I hope this is clearer and someone can help me. I ran out of ideas of things to try and need a fresh take.

  • $\begingroup$ I think you should wait a little bit more than 9 hours on Math Stack Overflow. $\endgroup$ Jul 4, 2020 at 23:01
  • 1
    $\begingroup$ To me this seems to some sort of "Rundumschlag" (german). Can you make your question more concrete, specifying the function space (what is the $\sup$-norm) and clarify your problem. For example is your $\sup$ measurable as a supremum over uncountable many functions? $\endgroup$ Jul 4, 2020 at 23:47
  • $\begingroup$ As stated the claim does not hold, as $Z_i$ could be uniform on $[-1,1]$ and for each $n$, the function $g$ could be an odd function from $[-1,1]$ to $[-\delta_n,\delta_n]$, chosen to satisfy $g(Z_i)=\delta_n$ for all $i \le n$. More care is needed in the supremum. $\endgroup$ Jul 5, 2020 at 2:49
  • $\begingroup$ @DieterKadelka Define any normed space, suppose that the norm exists. I even took out the $\sup$ from the description (I simply meant $||g||=\sup_{z\in\mathcal{Z}}|g(z)|$ and you can suppose that the functions $g$ are defined in $\mathcal{Z}$ with image in $\mathbb{R}$ and are bounded.) I think it is clear in the question that I have plenty of freedom to decide the functional space. If you know of a theorem that may achieve this, I can probably assume its requirements. I added explanations to the question. See new paragraph. $\endgroup$
    – Caetano
    Jul 5, 2020 at 5:27
  • $\begingroup$ @YuvalPeres Yes, it should be obvious that as stated it does not hold, as I mentioned that even to prove this convergence in probability one needs bounded second moments. Your example forgets the requirement in the statement that $E[g(Z_i)]=0$. Please see updated comments. $\endgroup$
    – Caetano
    Jul 5, 2020 at 5:50

1 Answer 1


A sufficient condition to establish the result above for the convergence in probability is if $\mathcal{G}$ has finite entropy with an envelope function $M$ which satisfies $E[M(Z)^2]<\infty$. This allows us to apply Theorem 1 in https://cowles.yale.edu/sites/default/files/files/pub/d10/d1059.pdf.

Unfortunately I could not find any result for almost sure stochastic equicontinuity, which is probably the answer to this problem. I will look it up further. If you know anything about this, please let me know.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.