# Show that $\sup_{\|g\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n g(Z_i)\right|\rightarrow_{a.s.}0.$ when $\delta_n\rightarrow 0$?

UPDATE: The result below can be understood as an almost sure stochastic equicontinuity condition. I don't know of any result establishing primitives of almost sure stochastic equicontinuity. If you know, this would very likely help me answer the question.

I need the following almost sure convergence result:

Let $$\|\cdot\|$$ denote a norm on a functional space (could be $$\|\cdot\|_{\infty}$$ or $$\|\cdot \|_2$$ in $$L^2$$, for example). $$\mathcal{G}=\{g:\mathcal{Z}\rightarrow \mathbb{R}\}$$ of measurable functions. Let $$Z_1,Z_2,\dots$$ be $$i.i.d$$ random variables with $$E[g(Z_i)]=0$$ for any $$g\in \mathcal{G}$$. Let $$\{\delta_n\}$$ be a sequence of positive numbers such that $$\delta_n\rightarrow 0$$. Then, under ADDITIONAL ASSUMPTIONS, $$\begin{equation*} \sup_{g\in\mathcal{G} \text{ s.t. }\|g\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n g(Z_i)\right|\rightarrow_{a.s.}0. \end{equation*}$$

Note that all random variables are defined in the same probability space and are independent draws of the same distribution. The number $$n$$ here denotes the size of the sample, and the almost sure convergence is over the distribution of the $$Z_i$$ as $$n\rightarrow \infty$$. The space $$\mathcal{G}$$ does not depend on $$n$$, and $$g$$ is not random.

I need to find which reasonable assumptions could be made to establish this result. $$\mathcal{G}$$ is $$P$$-Donsker? Lipschitz functions? Bounds on moments? etc.

Attending requests for clarification of the background: this result is needed to establish a stochastic equicontinuity condition on a linear regression estimator with a generated regressor (it means that one of the regressors is estimated). This is a high level result, in the sense that it is a result for a generic estimator for the regressor, we want to give conditions that this estimator must satisfy, but not specify the estimator. I am using results in a paper by Chen, Linton and Keilegom (2003) which establish the consistency of the bootstrap for estimators which are based on the optimization of a function of the data, the parameter of interest and a nuisance infinite dimension parameter. So, suppose the model is $$E[Y|X,W]=\beta X+\gamma h_0(W)$$, but $$h_0$$ is not known. The term is thus estimated and the regression is done on $$\hat{h}(W)$$ instead. If it helps, you can think of $$h_0(W)=E[V|V\leq 0, W]$$, for another variable $$V$$ which is observed in the data, for example. (FIY: this is a huge simplification of the true situation, so don't give me suggestions about how to estimate the model I just described. In the actual model $$V$$ is not observed and must be predicted out of sample with machine learning and optimization methods.)

In order to apply one of the results in the paper I mentioned, I need to prove that some quantities such as the one above are $$o_{a.s.}(1)$$. For example, I give you a simplified version of one of them: $$\begin{equation*} \sup_{h,h_0\in\mathcal{H} \text{ s.t. }\|h-h_0\|\leq \delta_n}\left| \frac{1}{\sqrt{n}}\sum_{i=1}^n \left[X_i(h(W_i)-h_0(W_i))-E[X_i(h(W_i)-h_0(W_i))\right]\right| \end{equation*}$$ I need to show that several objects that look more or less like the one above are $$o_{a.s.}(1)$$ as $$n\rightarrow \infty$$ for any sequence of positive $$\delta_n\rightarrow 0$$. Note that I can make assumptions about $$\mathcal{H}$$, $$X$$ and $$W$$. For example, I can say that $$\mathcal{H}$$ is the space of measurable negative Lipschitz functions, and I can say that $$E[|X|^4]$$ and $$E[|W|^4]$$ are finite. Basically I need conditions that would allow me to establish this result.

At the essence this is an empirical process. Define $$\mathbb{G}_nf=\frac{1}{\sqrt{n}}\sum_{i=1}^n f(Z_i)-E[f(Z_i)]$$ and let $$\mathbb{G}$$ be the Brownian Bridge, then if $$\mathcal{G}$$ is $$P$$-Donsker, $$\mathbb{G}_ng\rightsquigarrow\mathbb{G}g$$ for every fixed $$g$$. This is a convergence in distribution in a functional space. I need to show that $$\sup_{\|g\|_{\mathcal{G}}\leq \delta_n} |\mathbb{G}_n g|\rightarrow_{a.s.} 0$$.

I hope this is clearer and someone can help me. I ran out of ideas of things to try and need a fresh take.

• I think you should wait a little bit more than 9 hours on Math Stack Overflow. Jul 4, 2020 at 23:01
• To me this seems to some sort of "Rundumschlag" (german). Can you make your question more concrete, specifying the function space (what is the $\sup$-norm) and clarify your problem. For example is your $\sup$ measurable as a supremum over uncountable many functions? Jul 4, 2020 at 23:47
• As stated the claim does not hold, as $Z_i$ could be uniform on $[-1,1]$ and for each $n$, the function $g$ could be an odd function from $[-1,1]$ to $[-\delta_n,\delta_n]$, chosen to satisfy $g(Z_i)=\delta_n$ for all $i \le n$. More care is needed in the supremum. Jul 5, 2020 at 2:49
• @DieterKadelka Define any normed space, suppose that the norm exists. I even took out the $\sup$ from the description (I simply meant $||g||=\sup_{z\in\mathcal{Z}}|g(z)|$ and you can suppose that the functions $g$ are defined in $\mathcal{Z}$ with image in $\mathbb{R}$ and are bounded.) I think it is clear in the question that I have plenty of freedom to decide the functional space. If you know of a theorem that may achieve this, I can probably assume its requirements. I added explanations to the question. See new paragraph. Jul 5, 2020 at 5:27
• @YuvalPeres Yes, it should be obvious that as stated it does not hold, as I mentioned that even to prove this convergence in probability one needs bounded second moments. Your example forgets the requirement in the statement that $E[g(Z_i)]=0$. Please see updated comments. Jul 5, 2020 at 5:50

A sufficient condition to establish the result above for the convergence in probability is if $$\mathcal{G}$$ has finite entropy with an envelope function $$M$$ which satisfies $$E[M(Z)^2]<\infty$$. This allows us to apply Theorem 1 in https://cowles.yale.edu/sites/default/files/files/pub/d10/d1059.pdf.