# Limits of binomial distribution

We know that as $n \to \infty$, the binomial distribution $B(n, p)$, with fixed $p$, after appropriate normalization, converges to a normal distribution. If $p = c/n$ for some constant $c$, then it converges to the Poisson distribution.

What happens for intermediate cases, say when $p = c n^{-\alpha}$ for fixed $c$ and $0 < \alpha < 1$. After appropriate normalization, what is the limiting distribution of $B(n,p)$?

-

Assume the distribution of $X_n$ is binomial $(n,p)$. Then, for every real number $t$, $$\mathrm E(\mathrm e^{\mathrm it(X_n-np)})=\mathrm e^{\mathrm itnp}(1-p+p\mathrm e^{\mathrm it})^n.$$ Assuming that $p\to0$ when $n\to\infty$, standard limited expansions yield $$\mathrm E(\mathrm e^{\mathrm it(X_n-np)})=\exp\left(-\tfrac12npt^2(1-p)+O(npt^3)\right).$$ Let us assume that $np\to\infty$ when $n\to\infty$ and choose $t=s/\sqrt{np}$ for a given $s$. Then $npt^2(1-p)\to s^2$ and $npt^3=s^3/\sqrt{np}\to0$, hence $$\mathrm E(\mathrm e^{\mathrm is(X_n-np)/\sqrt{np}})\to\mathrm e^{-s^2/2}.$$ Thus, $(X_n-np)/\sqrt{np}$ converges in distribution to the standard gaussian distribution when $n\to\infty$, as soon as $p\to0$ and $np\to\infty$.

-

I believe with a fairly standard scaling and shifting, we can recover a normal limit for any $0 < \alpha < 1$ by a standard application of the Lindeberg–Feller theorem.

Define the triangular array of random variables $X_{n,m}$, $1 \leq m \leq n$ such that each row contains iid elements distributed as $\mathrm{Bernoulli}(cn^{-\alpha})$ random variables. Then $S_n = \sum_{m=1}^n X_{n,m}$ is $B(n,p)$ with $p = c n^{-\alpha}$.

Now, shift and rescale by taking $Y_{n,m} = n^{-(1-\alpha)/2} (X_{n,m} - c n^{-\alpha})$. Note that $\mathbb E Y_{n,m} = 0$ and $\mathbb E Y_{n,m}^2 = n^{-1} c (1 - c n^{-\alpha})$.

We need only check the Lindeberg–Feller conditions:

1. $\sum_{m=1}^n \mathbb E Y_{n,m}^2 = c (1-cn^{-\alpha}) \to c > 0$, and
2. For all $\epsilon > 0$, $\lim_{n\to\infty} \sum_{m=1}^n \mathbb E |Y_{n,m}|^2 1_{(|Y_{n,m}|^2 > \epsilon)} = 0$.

The second one follows since $1_{(|Y_{n,m}|^2 > \epsilon)} = 0$ for all sufficiently large $n$ since $|Y_{n,m}|^2 \leq n^{-(1-\alpha)} (1+c)^2$ almost surely.

Hence $n^{-(1-\alpha)/2} (S_n - c n^{1-\alpha}) \;\xrightarrow{\;d\;}\; \mathcal N(0,c)$.

To see what fails in the case where $\alpha = 1$, note that the indicator function $1_{(|Y_{n,m}|^2 > \epsilon)}$ will no longer go to zero almost surely. Therefore the second condition will fail because there will be a contribution of $c n^{-1} (1-c/n)^2$ in expectation for each of the $n$ terms, which, when added up yields (obviously) a nonzero limit.

-

The total variation distance between $B(n,p)$ and Poisson($np$) goes to 0 as $n\to\infty$ whenever $p\to 0$. See http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=2086772 for example. This means that the Poisson and normal ranges overlap considerably and you don't need anything else.

-
I disagree with the last part of your last sentence, which puts undue significance in the result recalled in the first sentence. For example, if $np\to0$, the distance between Poisson $(np)$ and a Dirac mass at $0$ ALSO goes to $0$ hence in this regime, the result simply says that $B(n,p)$ converges to a Dirac mass at $0$. Similarly, if $np\to\infty$, Poisson $(np)$ is itself (when shifted by $np$ and rescaled by $\sqrt{np}$) approximately standard gaussian hence in this regime, the result is simply an unusual way of stating that a CLT holds. – Did Feb 14 '12 at 10:30
I don't see your point. If $np\to 0$, the explicit bounds on the tdv given in the linked paper show more than just accumulation at 0. However, there are stronger results in that case to be sure (very easily obtained). For $np\to\infty$, you are right that scaled Poisson is close to normal, but that's what "the Poisson and normal ranges overlap considerably" means. The OP asked what happens between the ranges where binomial is like Poisson and where binomial is like normal, and the correct answer is that there is nothing between them. – Brendan McKay Feb 14 '12 at 19:10
My point is that in these two regimes, the Poisson distribution is a red herring, for a Dirac distribution or for a gaussian one. (I read your comment by chance, please use @ when commenting to someone else than the author of the post.) – Did Feb 15 '12 at 0:48
@Didier Piau: Depending on the way $p$ varies as a function of $n$ (except for $p=1-o(1/n)$), $B(n,p)$ is well approximated by either the Poisson or Normal distribution. The details depend on the distance measure, but for most measures (total variation distance being one) the Poisson approximation remains better than the normal well into the range where the binomial is asymptotically normal. For tiny $p$, the Poisson approximation is better than the Dirac approximation. Do you disagree with any of these statements? – Brendan McKay Feb 15 '12 at 9:08
1. Let $X_1,X_2,...,X_n$ be independent and identically distributed random variables such that $E(X_i) = 0, Var(X_i) = 1$ and denote $W=n^{-1/2}\sum X_i$. By using Stein's method, we have that $$d_W(\mathcal{L}(W),N(0,1)) \leq \frac{5 E|X_1|^3}{n^{1/2}},$$ where $d_W$ is the Wasserstein metric.

2. Any binomial distribution $B(n, p)$ is the sum of $n$ independent Bernoulli trials $B(1, p)$, so you can consider $X_i$ as a normalized Bernoulli random variable and apply the above inequality.

-