I have a sequence of centered independent random variables $X_i$ that are all bounded by one in absolute value. They are not identically distributed, though. I would like to know if the central limit theorem is still true for such a sequence. Putting $S_n= X_1+...+X_n$, do we have $$ c_n = P(\ {S_n\over\sigma(S_n)} \in [a,b] ) - {1\over \sqrt{2\pi}}\int_a^b exp(-t^2/2) dt \ \rightarrow \ 0\ ? $$ (let's assume $\sigma(S_n)$ goes to infinity with n). I guess it is true but I can't find a reference.

Also, what can be said from the rate of convergence of $c_n$ ? Since the $X_i$ are uniformly bounded, does $c_n$ goes to zero exponentially fast ?


Theorem (Billingsley, "probability and measure", example 27.4)

Let X_i a sequence of independent, uniformly bounded random variables with zero mean, such that $\sigma(S_n)$ goes to infinity with n. Then $S_n/\sigma(S_n)$ converges in law to the normalized Laplace-Gauss distribution.

This follows from the Lindeberg triangular array theorem. As pointed out in the other answers, the convergence can be slow. The Bernstein inequality may be used to bound the tail. Under the assumption of the previous theorem, for all n, we have

$$P(S_n>t) \leq exp(-t^2/(\sigma^2(S_n)+Ct/3))$$

where C is a bound for the |X_i|'s.

  • $\begingroup$ I didn't read carefully and had some details stupidly wrong in the "Added" part of my post. It's now correct, and consistent with what coudy writes. $\endgroup$ Jun 30 '10 at 14:33
  • $\begingroup$ I am not sure what you mean by the normalized Laplace-Gauss distribution, can you provide a reference to that? $\endgroup$
    – Three Diag
    Oct 31 '16 at 12:05
  • 1
    $\begingroup$ By Laplace-Gauss, I just mean the usual normal distribution with density f(x) = exp(-x²/2)/√(2π). $\endgroup$
    – coudy
    Oct 31 '16 at 13:28

For your first question, the answer is yes, and I don't understand why it isn't better known since all the classical proofs of the central limit theorem generalize easily to that setting. See this section of the Wikipedia page on the central limit theorem.

Added: I overstated slightly, since the classical proofs don't easily generalize to give the most general conditions. But in the OP's setting, as coudy points out, Lindeberg's condition implies that it's enough to have $\sigma(S_n) \to \infty$ (whereas if the $X_i$ were identically distributed we would of course have $\sigma(S_n) = \sqrt{n}\sigma(X)$).

For your second question, even under uniform boundedness, $c_n$ only goes to zero like $n^{-1/2}$ in general. See for example this paper.

  • $\begingroup$ I believe en.wikipedia.org/wiki/Azuma%27s_inequality implies that one can even drop the independance. $\endgroup$
    – Helge
    Jun 25 '10 at 14:44
  • 2
    $\begingroup$ You can't simply drop the independence; you need to replace it with something else, and here the proofs do get harder. But Azuma's inequality gives tail bounds, not a central limit theorem. For a sampling of central limit theorems for dependent random variables: en.wikipedia.org/wiki/… $\endgroup$ Jun 25 '10 at 14:56

There is an entire book on concentration inequalities that goes well beyond the iid normal case. Loosely speaking, any collection of variables satisfying some kind of Lipschitz property will have exponential tails that can often lead to a CLT-type form. Azuma's inequality is one example covered in the book.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.