Given a random variable $X$ with bounded moments such that $E[X] = 0, E[X^2] = 1$, let $F_n$ denote the distribution $\sum\limits_{i=1}^d\frac{X_i}{\sqrt{n}}$ where each $X_i$ is an independent copy of $X$. It is known that as $n \to \infty$, $F_n$ converges in distribution to a standard Gaussian.

The Berry-Esseen inequality then gives us a quantitative estimate for the rate of convergence. If $\Phi$ denotes the standard Gaussian distribution we have:: $$\sup|F_n - \Phi| \leq C \frac{1}{\sqrt{n}} $$

Where (throught the discussion) $C$ is a positive constant depending on the moments of $X$.

We may also look at a stronger type of convergence. Namely, convergence in total variation (in the $L_1$ norm). Clearly, without further demands from $X$, nothing can be expected. However, some sufficient conditions can be found. For example, when $X$ satisfies a Poincare type inequality or when $X$ has a finite Kullback-Leibler divergence from $\Phi$. In both those cases one may use the Entropic Central Limit Theorem (as stated here, http://arxiv.org/pdf/1105.4119v2.pdf, for example) to give a similar bound on the total variation distance. $$||F_n - \Phi||_{\mathrm{TV}} \leq C \frac{1}{\sqrt{n}}$$.

So, when all $X_i$ are identically distributed and satisfy some simple condition, convergence in Total Variation and convergence in distribution happen together at the same rate.

Now, let us consider the non-i.i.d case. Let $A_n = \{a_i\}_{i=1}^n$ and denote by $F_{A,n}$ the distribution of $\sum a_i X_i$, where again all $X_i$ are independent copies of $X$.

In this case, the Berry-Esseen inequality gives the following bound: $$\sup|F_{A,n} - \Phi| \leq C \frac{\left(\sum a_i^3\right)}{\left(\sum a_i^2\right)^{(3/2)}}$$

Looking at the Total Variation though, bounding the Kullback-Liebler divergence of $X$ from standard normal no longer makes sense, since the divergence of $a_iX_i$ can grow as a function of $a_i$.

The known bound of the Entropic CLT, given that $X$ satisfies a Poincare inequality (see http://www.tau.ac.il/~shiri/ptrf/clt.pdf) when joined with Pinsker's inequality then gives: $$||F_{A,n} - \Phi||_{\mathrm{TV}} \leq C\frac{\sqrt{\sum a_i^4}}{\sum a_i^2}$$ Unlike in the i.i.d case this gives a real difference between Total Variation distance and convergence in distribution.

I am not aware whether the Entropic CLT is tight. But, this suggests the existence of a nice enough random variable $X$ (satisfying Poincare inequality) and a set of coefficients $A$, such that $\sum a_iX_i$ convergence in distribution to a Gaussian in rate $O(\frac{1}{\sqrt{n}})$, while converging in Total Variation in a much slower rate (Consider for example the case where $a_i \sim \frac{1}{i^{(1/3)}}$) In particular, there is some event on the real line, which cannot be a finite union of intervals, on which the two distributions differ by a substantial amount.

My question is then:

  • Is there a natural example of such a random variable?

  • If so, is there a natural description to the event on which the distributions differ?

  • $\begingroup$ By $X$, you mean a sequence of distributions since you are talking about the non-identically distributed case, right? What does it mean for $X$ to satisfy a Poincare-type inequality? $\endgroup$ – Douglas Zare Jan 7 '16 at 5:17
  • $\begingroup$ Well, $X$ is fixed. The sequence is $a_iX$ in essence. Sp, they are not identical, but up to re-scaling all elements of the sequence are like $X$. A random variable satisfies Poincare inequality if there is a positive constant $c$ such that for every (differentiable) $f$, $c\mathrm{Var}(f(X)) \leq E[f'(x)]^2$. Basically, what is important here, that $X$ cannot be supported on 'atoms'. $\endgroup$ – Cain Jan 7 '16 at 7:15
  • $\begingroup$ Ok. I'm not sure what that implies for an absolutely continuous distribution. The Berry-Esseen bound is sharp (within a constant) when a random variable is well-approximated by lattice random variables. If you can smear a lattice random variable slightly (by a decreasing amount as you move out on the tails) so that it satisfies a Poincare inequality, then a candidate event would be that the sum is far from the lattice, say that the fractional part is between $1/4$ and $3/4$. $\endgroup$ – Douglas Zare Jan 7 '16 at 12:17
  • $\begingroup$ The thing is, it's not obvious how to 'smear' a discrete random variable into one which a Poincare inequality. Satisfying such inequalities, means the density cannot be too 'bumpy', in the sense that it cannot be 0 at any interval (otherwise just choose $f$ to be a bump function on the complement of the interval). $\endgroup$ – Cain Jan 8 '16 at 19:12

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.