Question

Let $\xi_{tn}(\theta),t=1,\dots,n$ be a real-valued martingale difference array indexed by a parameter $\theta \in \Theta \subset R$, where the set $\Theta$ is compact. Now, for all fixed $\theta \in \Theta$, the law of large numbers,

$\sum_{t=1}^n \xi_{tn} (\theta) \rightarrow 0$ in probability,

is assumed to hold. Then, what conditions make the following UNIFORM law of large numbers hold?

$\sup_{\theta \in \Theta} | \sum_{t=1}^n \xi_{tn}(\theta) | \rightarrow 0$ in probability.

Are there any articles or books about the result?

Answer 1

Even if $\xi$'s are independent (a specific case of your martingale-difference stitation), uniform LLN sometimes holds and sometimes it does not. This type of questions has been studied in Machine learning and, specifically, in Vapnik-Chervonenkis theory. The Glivenko-Cantelli theorem (see wikipedia) describes one situation where the convergence is uniform. More general results can be formulated in terms of VC (Vapnik-Chervonenkis) classes. Perhaps some literature is available for martingales, too.

Answer 2

An extension of the uniform Glivenko-Cantelli notion to martingale differences can be found here: www-stat.wharton.upenn.edu/~rakhlin/papers/emp_proc_dep.pdf