Mean of i.i.d Random Variables With No Expected Value

Question

Let $X$ be an integer-valued random variable and let $X_n$ be the sum of $n$ independent realizations of $X$. I would like to understand the behavior of $X_n/n$ for large $n$ in some cases where $X$ has no expected value and therefore the central limit theorem (along with Chebyshev's Theorem, etc.) does not apply.

For a concrete example, let $X$ have the following distribution: $$Prob(X=0)=1/3\qquad Prob(X=4^k)=1/4^{k+1}\qquad Prob(X=-4^{k})=1/4^{k+1}$$ with all other probabilities zero.

Note that $X$ has no expected value so we can't use the central limit theorem. Still, I'd like to have a good way to estimate $Prob(X_n/n > M)$ for a given large $n$ and $M$.

Note also that in this example $Prob(X_n/n=M)$ can depend on number theoretic properties of $M$ (and in particular will not be monotonic in $M$), but I'm hoping that when $n$ is big and we consider $Prob(X_n/n > M)$, this sort of thing will wash out.

I've done some relevant calculations, but I wonder whether I'm missing either some knowledge or some insight that would make this easier to understand.

Answer 1

We can attack this problem by using Fourier transforms (i.e. characteristic functions). I'll consider the example in the problem where $X$ is a random variable taking the value $0$ with probability $1/3$, and $4^{k}$ and $-4^{k}$ with probability $1/4^{k+1}$ (for $k=0$, $1$, $\ldots$). I'll show that the probability that $|X_n|/n >M$ is about a constant times $1/M$ (more precise result below).

Fix a smooth function $\Phi$ compactly supported in $[-1,1]$ and approximating the characteristic function of that interval. Concretely, suppose $\epsilon$ is small and $\Phi(x)=1$ on $[-1+\epsilon, 1-\epsilon]$ and is between $0$ and $1$ on the rest of $[-1,1]$. Since $\Phi$ is smooth, its Fourier transform ${\hat \Phi}(\xi) = \int_{-\infty}^{\infty}\Phi(x) e^{-2\pi i x\xi} dx$ has rapid decay for $|\xi|$ large.

Now let $n$ and $M$ be large and consider $$ {\Bbb E}\Big(\Phi\Big(\frac{X_n}{nM}\Big)\Big). $$
Note that $$ \text{Prob} (|X_n| >nM) \le 1 -{\Bbb E}(\Phi(X_n/(nM))) \le \text{Prob}(|X_n| > (1-\epsilon)nM), $$ and so our problem is to understand the expectation above. By Fourier inversion, $$ {\Bbb E}(\Phi(X_n/(nM))) = \int_{-\infty}^{\infty} {\hat \Phi}(\xi) {\Bbb E}\Big( e^{2\pi i \xi X_n/(nM)}\Big) d\xi = \int_{-\infty}^{\infty} {\hat \Phi}(\xi) \Big( {\Bbb E}\Big( e^{2\pi i \xi X/(nM)}\Big)\Big)^{n} d\xi. $$

Now we compute that $$ {\Bbb E}\Big( e^{2\pi i \xi X/(nM)}\Big) = \frac{1}{3} + 2 \sum_{k=0}^{\infty} \frac{1}{4^{k+1}} \cos \Big( \frac{2\pi \xi 4^k}{nM}\Big) = 1 - 2\sum_{k=0}^{\infty} \frac{1}{4^{k+1}}\Big (1-\cos \Big( \frac{2\pi \xi 4^k}{nM}\Big) \Big). $$ Now using that $(1-\cos(x)) = O(\min(x^2, 1))$ we see that $$ \sum_{k=0}^{\infty} \frac{1}{4^{k+1}}\Big (1-\cos \Big( \frac{2\pi \xi 4^k}{nM}\Big) \Big) = O\Big( \frac{|\xi|}{nM}\Big). $$ Therefore, using $(1-x)^n = 1-nx +O(n^2 x^2)$ for $0\le x\le 1$,
$$ {\Bbb E}(\Phi(X_n/(nM))) = \int_{-\infty}^{\infty} {\hat \Phi}(\xi) \Big( 1- 2n \sum_{k=0}^{\infty} \frac{1}{4^{k+1}}\Big (1-\cos \Big( \frac{2\pi \xi 4^k}{nM}\Big) \Big) + O\Big(\frac{\xi^2}{M^2}\Big) \Big) d\xi. $$ Since ${\hat \Phi}$ has rapid decay, the error term above is $O(1/M^2)$ (with the implied constant depending only on the fixed function $\Phi$). Using Fourier inversion, we conclude that $$ {\Bbb E}({\Phi }(X_n/(nM))) = \Phi(0) - n \sum_{k=0}^{\infty} \frac{1}{4^{k+1}} \Big(2 \Phi(0) - \Phi\Big(\frac{4^k }{nM} \Big) -\Phi\Big(-\frac{4^k}{nM}\Big)\Big) + O\Big(\frac{1}{M^2}\Big). $$ Since $\Phi(0)=1$, we get $$ 1- {\Bbb E}({\Phi }(X_n/(nM))) = n \sum_{k=0}^{\infty} \frac{1}{4^{k+1}} \Big(2 - \Phi\Big(\frac{4^k }{nM} \Big) -\Phi\Big(-\frac{4^k}{nM}\Big)\Big) + O\Big(\frac{1}{M^2}\Big). $$ By our choice for $\Phi$, the main term above is $$ \ge 2n \sum_{k, 4^{k} \ge nM} \frac{1}{4^{k+1}}, $$ and is $$ \le 2n \sum_{k, 4^{k} \ge (1-\epsilon) nM} \frac{1}{4^{k+1}}. $$

Thus we have obtained a good understanding of the probability that $|X_n|/n$ is large. Note also that the precise answer will have discontinuities when $nM$ gets near a power of $4$, but in any case the probability is about a constant times $1/M$.