Given a random variable $X$ with bounded moments such that $E[X] = 0, E[X^2] = 1$, let $F_n$ denote the distribution $\sum\limits_{i=1}^d\frac{X_i}{\sqrt{n}}$ where each $X_i$ is an independent copy of $X$. It is known that as $n \to \infty$, $F_n$ converges in distribution to a standard Gaussian.

The Berry-Esseen inequality then gives us a quantitative estimate for the rate of convergence. If $\Phi$ denotes the standard Gaussian distribution we have:: $$\sup|F_n - \Phi| \leq C \frac{1}{\sqrt{n}} $$

Where (throught the discussion) $C$ is a positive constant depending on the moments of $X$.

We may also look at a stronger type of convergence. Namely, convergence in total variation (in the $L_1$ norm). Clearly, without further demands from $X$, nothing can be expected. However, some sufficient conditions can be found. For example, when $X$ satisfies a Poincare type inequality or when $X$ has a finite Kullback-Leibler divergence from $\Phi$. In both those cases one may use the Entropic Central Limit Theorem (as stated here, http://arxiv.org/pdf/1105.4119v2.pdf, for example) to give a similar bound on the total variation distance. $$||F_n - \Phi||_{\mathrm{TV}} \leq C \frac{1}{\sqrt{n}}$$.

So, when all $X_i$ are identically distributed and satisfy some simple condition, convergence in Total Variation and convergence in distribution happen together at the same rate.

Now, let us consider the non-i.i.d case. Let $A_n = \{a_i\}_{i=1}^n$ and denote by $F_{A,n}$ the distribution of $\sum a_i X_i$, where again all $X_i$ are independent copies of $X$.

In this case, the Berry-Esseen inequality gives the following bound: $$\sup|F_{A,n} - \Phi| \leq C \frac{\left(\sum a_i^3\right)}{\left(\sum a_i^2\right)^{(3/2)}}$$

Looking at the Total Variation though, bounding the Kullback-Liebler divergence of $X$ from standard normal no longer makes sense, since the divergence of $a_iX_i$ can grow as a function of $a_i$.

The known bound of the Entropic CLT, given that $X$ satisfies a Poincare inequality (see http://www.tau.ac.il/~shiri/ptrf/clt.pdf) when joined with Pinsker's inequality then gives: $$||F_{A,n} - \Phi||_{\mathrm{TV}} \leq C\frac{\sqrt{\sum a_i^4}}{\sum a_i^2}$$ Unlike in the i.i.d case this gives a real difference between Total Variation distance and convergence in distribution.

I am not aware whether the Entropic CLT is tight. But, this suggests the existence of a nice enough random variable $X$ (satisfying Poincare inequality) and a set of coefficients $A$, such that $\sum a_iX_i$ convergence in distribution to a Gaussian in rate $O(\frac{1}{\sqrt{n}})$, while converging in Total Variation in a much slower rate (Consider for example the case where $a_i \sim \frac{1}{i^{(1/3)}}$) In particular, there is some event on the real line, which cannot be a finite union of intervals, on which the two distributions differ by a substantial amount.

My question is then:

  • Is there a natural example of such a random variable?

  • If so, is there a natural description to the event on which the distributions differ?

  • $\begingroup$ By $X$, you mean a sequence of distributions since you are talking about the non-identically distributed case, right? What does it mean for $X$ to satisfy a Poincare-type inequality? $\endgroup$ Jan 7, 2016 at 5:17
  • $\begingroup$ Well, $X$ is fixed. The sequence is $a_iX$ in essence. Sp, they are not identical, but up to re-scaling all elements of the sequence are like $X$. A random variable satisfies Poincare inequality if there is a positive constant $c$ such that for every (differentiable) $f$, $c\mathrm{Var}(f(X)) \leq E[f'(x)]^2$. Basically, what is important here, that $X$ cannot be supported on 'atoms'. $\endgroup$
    – Cain
    Jan 7, 2016 at 7:15
  • $\begingroup$ Ok. I'm not sure what that implies for an absolutely continuous distribution. The Berry-Esseen bound is sharp (within a constant) when a random variable is well-approximated by lattice random variables. If you can smear a lattice random variable slightly (by a decreasing amount as you move out on the tails) so that it satisfies a Poincare inequality, then a candidate event would be that the sum is far from the lattice, say that the fractional part is between $1/4$ and $3/4$. $\endgroup$ Jan 7, 2016 at 12:17
  • $\begingroup$ The thing is, it's not obvious how to 'smear' a discrete random variable into one which a Poincare inequality. Satisfying such inequalities, means the density cannot be too 'bumpy', in the sense that it cannot be 0 at any interval (otherwise just choose $f$ to be a bump function on the complement of the interval). $\endgroup$
    – Cain
    Jan 8, 2016 at 19:12
  • $\begingroup$ Ha you managed to find a solution here? In may case the X are Rademacher variable and I'm also trying to characterize A_n's properties $\endgroup$ Dec 7, 2022 at 5:59


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy