Strong Law of Large Numbers for weakly dependent random variables

Let Xi be a sequence of identically-distributed random variables with finite-range dependence (i.e. there exists I such that if |i-i'| ≥ I, then Xi and Xi' are independent), and a finite moment-generating function (i.e. EerXi < ∞ for all r ∈ R).

It's not too hard to show that Xi satisfies a strong law of large numbers, and I've got a proof written. However, I'm sure that this is a standard theorem in the probability literature, and I'd rather just cite it in the paper I'm writing. Do you have a good reference for this result?

Here are two follow-up generalizations: what if Xi instead has only a finite moment condition? Or what if Xi has exponential correlation decay (i.e. EXiXi' ≤ Ce-c|i-i'| for some positive c, C)?

This question sounds like an exercise: Split the sequence into I sequences of iid random variables. Apply the classical SLLN to each sequence. Recombine.

Tom: Of course it is true with exponential decay of the corellation function, but it is not easy.

The essential difficulty is that one wants to reduce the SLLN to an exponentially growing subsequence of N's. In the classical case, this is done by a martingale inequality. (Prob of a supremum of a martingale is dominated by the probability at end of the martingale.)

Once one moves away from an implicit martingale structure, then tricks have to be employed---of which the most obvious is that if the sequence of random variables is bounded, then obviously you can reduce to an exponentially growing subsequence. This point is much of the content of the paper of Lyons cited already.

Not sure that this would appear in a text book however. My sense is that these considerations are well-known.

• You're right. The question as I asked it is simple and can be solved just by splitting up the sequence. What about if there isn't just finite-range dependence, and there is instead exponential correlation decay? – Tom LaGatta Oct 27 '09 at 22:08

If they're 0,1 valued, then the following may be what you need. It's call Levy's Borel-Cantelli Lemmas:

Suppose that for natural numbers $n$, $E_n \in F_n$ (a sigma-algebra). Define $Z_n = \sum _{1\leq k \leq n} I_{E_k}$, the number of $E_1,\ldots,E_n$ which occur. Set $e_k = P(E_k | F_{k-1})$, and $Y_n = \sum_{1\leq k \leq n} e(k)$. Then, almost surely, (a) $Y_\infty < \infty$ implies $Z_\infty < \infty$ (b) $Y_\infty = \infty$ imples $Z_n / Y_n \rightarrow 1$.

This allows, in essence, for you to use the Borel-Cantelli lemmas even when the variables are dependent, as long as many variables are mostly independent from the earlier ones.

I know it isn't the strong law, but in many circumstances that you'd want a strong law (but don't have it) this suffices.

My reference is "Probability with Martingales", by D. Williams, and this is Theorem 12.15 (with proof) on page 124.

• My family X<sub>i</sub> is unbounded, so that doesn't work. In fact, my proof is simple and exploits the exponential Chebyshev inequality and classical Borel-Cantelli lemma, but surely it's written down or generalized somewhere. Another nice generalization of Borel-Cantelli is the <a href="books.google.com/… lemma</a>. – Tom LaGatta Oct 25 '09 at 17:27

I think Lyons answers this in his Strong laws of large numbers for weakly correlated random variables. This should be in a textbook somewhere, though, which I'd rather cite.