What conditions on a probability distribution defined by long-time averaging do I need to satisfy a central limit theorem?

Question

For integer $n$, $1 \le n \le N$, consider the random variables

$X_n = \cos[t \omega_n]$

For any fixed $N$, we can take the mean

$Y_N = \frac{1}{N} \sum_{n=1}^N X_n$

and define a (cumulative) distribution by averaging over long times:

$P(Y_N \le y) = \lim_{T \to \infty} \frac{1}{2 T} \lambda(\{t \in [-T,T] \mid Y_N \le y \} )$,

where $\lambda$ is the Lebesgue measure. If the $\omega_n$ are linearly independent over the rationals for all $n$, then the random variables $X_n$ are independent and identically distributed over long times. Since they are i.i.d., we can apply the central limit theorem to show that $p(Y_N = y)$, the probability density of $Y_N$, approaches a normal distribution with zero mean and variance $\sigma_0^2/N$, where $\sigma_0^2=1/2$ is the variance of each $X_n$.

Now, suppose that the $\omega_n$ are not all linearly independent, but are incommensurate (i.e. pairwise linearly independent). This would mean that the $X_n$ are not independent. In particular, suppose we consider only $N=2^M$ for integer $M$ and take

$w_n^{(M)} = \sum_{k=0}^{M-1} (-1)^{r_k} h_k$

where

$r_k = (n/2^k) \mod 2$

are the digits of $n$ in base 2. Though the $h_k$ may be linearly independent over the rationals, there are $2^M$ frequencies $w_n^{(M)}$ for each fixed $M$. Since $k$ only ranges from $0$ to $M-1$, these frequencies are not linearly independent.

So for fixed $M$, not only are the $2^M$ random variables $X_n^{(M)} = \cos[t \omega_n^{(M)}]$ not independent, their joint probability density is actually only has support on a $M$-dimensional subspace. This is not a stationary sequence (I think). I'm not sure if it is ergodic.

Now, we could express the $Y$ as a mean of just $M$ independent random variables by combining the $X_n^{(M)}$. This would seem to guarantee a variance only as small as $\sigma_0^2/M$ rather than $\sigma_0^2/2^M$.

However, numerical work confirmed my intuition that the the $Y$ actually approaches a normal distribution with the small variance $\sigma_0^2/2^M$. This suggests there is a CLT I could apply to get this result analytically. But when I read the standard texts, I can't find much in the way of CLT's for non-stationary processes. I'm trying to read about ergodicity, but I can't even tell if this sequence fits the descriptions.

Is this sequence ergodic? Does it satisfy a CLT?

Answer 1

In your example, algebraic manipulation gives $$ 2^{-M} \sum_{n=0}^{2^M-1} X_n^{(M)} = \prod_{j=0}^{M-1} \cos t h_j. $$ If the $h_j$'s are linearly independent over $\Bbb Q$, then, as you point out, the random variables $\cos t h_j$ approach independence as $t$ is chosen over larger and larger intervals. Therefore, in the limit, $$ Y = Z_1 ... Z_M, \qquad Z_i = \cos W_i, \qquad W_1,...,W_M {\rm\ \ i.i.d.\ uniform\ on\ } [0,2\pi). $$ By symmetry, ${\bf E}[Y]=0$, and since ${\bf E}[Z_i^2]=1/2$ for each $i$, we have ${\bf Var}[Y]=2^{-M}$. This may explain the result of your numerical experiments. However, $Y$ does not approach normal after rescaling: if $Y$ is rescaled to unit variance by setting $Y'_M:=2^{M/2} Y$, then $\log |Y'_M|=\log(\sqrt{2} |Z_1|)+...+\log(\sqrt{2} |Z_M|)$, so, since ${\bf E}[\log(\sqrt{2} |Z_i|)]<0$ and ${\bf Var}[\log(\sqrt{2} |Z_i|)]<\infty$, we can apply the CLT to $\log |Y'_M|$ to show that, as $M\to\infty$, $\log |Y'_M|$ will converge weakly to normal after rescaling, but $Y'_M$ converges weakly to 0.