Limits of binomial distribution

Question

We know that as $n \to \infty$, the binomial distribution $B(n, p)$, with fixed $p$, after appropriate normalization, converges to a normal distribution. If $p = c/n$ for some constant $c$, then it converges to the Poisson distribution.

What happens for intermediate cases, say when $p = c n^{-\alpha}$ for fixed $c$ and $0 < \alpha < 1$. After appropriate normalization, what is the limiting distribution of $B(n,p)$?

Answer 1

Assume the distribution of $X_n$ is binomial $(n,p)$. Then, for every real number $t$, $$ \mathrm E(\mathrm e^{\mathrm it(X_n-np)})=\mathrm e^{\mathrm itnp}(1-p+p\mathrm e^{\mathrm it})^n. $$ Assuming that $p\to0$ when $n\to\infty$, standard limited expansions yield $$ \mathrm E(\mathrm e^{\mathrm it(X_n-np)})=\exp\left(-\tfrac12npt^2(1-p)+O(npt^3)\right). $$ Let us assume that $np\to\infty$ when $n\to\infty$ and choose $t=s/\sqrt{np}$ for a given $s$. Then $npt^2(1-p)\to s^2$ and $npt^3=s^3/\sqrt{np}\to0$, hence $$ \mathrm E(\mathrm e^{\mathrm is(X_n-np)/\sqrt{np}})\to\mathrm e^{-s^2/2}. $$ Thus, $(X_n-np)/\sqrt{np}$ converges in distribution to the standard gaussian distribution when $n\to\infty$, as soon as $p\to0$ and $np\to\infty$.

Answer 2

I believe with a fairly standard scaling and shifting, we can recover a normal limit for any $0 < \alpha < 1$ by a standard application of the Lindeberg–Feller theorem.

Define the triangular array of random variables $X_{n,m}$, $1 \leq m \leq n$ such that each row contains iid elements distributed as $\mathrm{Bernoulli}(cn^{-\alpha})$ random variables. Then $S_n = \sum_{m=1}^n X_{n,m}$ is $B(n,p)$ with $p = c n^{-\alpha}$.

Now, shift and rescale by taking $Y_{n,m} = n^{-(1-\alpha)/2} (X_{n,m} - c n^{-\alpha})$. Note that $\mathbb E Y_{n,m} = 0$ and $\mathbb E Y_{n,m}^2 = n^{-1} c (1 - c n^{-\alpha})$.

We need only check the Lindeberg–Feller conditions:

1. $\sum_{m=1}^n \mathbb E Y_{n,m}^2 = c (1-cn^{-\alpha}) \to c > 0$, and
2. For all $\epsilon > 0$, $\lim_{n\to\infty} \sum_{m=1}^n \mathbb E |Y_{n,m}|^2 1_{(|Y_{n,m}|^2 > \epsilon)} = 0$.

The second one follows since $1_{(|Y_{n,m}|^2 > \epsilon)} = 0$ for all sufficiently large $n$ since $|Y_{n,m}|^2 \leq n^{-(1-\alpha)} (1+c)^2$ almost surely.

Hence $n^{-(1-\alpha)/2} (S_n - c n^{1-\alpha}) \;\xrightarrow{\;d\;}\; \mathcal N(0,c)$.

---

To see what fails in the case where $\alpha = 1$, note that the indicator function $1_{(|Y_{n,m}|^2 > \epsilon)}$ will no longer go to zero almost surely. Therefore the second condition will fail because there will be a contribution of $c n^{-1} (1-c/n)^2$ in expectation for each of the $n$ terms, which, when added up yields (obviously) a nonzero limit.

Answer 3

The total variation distance between $B(n,p)$ and Poisson($np$) goes to 0 as $n\to\infty$ whenever $p\to 0$. See http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=2086772 for example. This means that the Poisson and normal ranges overlap considerably and you don't need anything else.

Answer 4

1. Let $X_1,X_2,...,X_n$ be independent and identically distributed random variables such that $E(X_i) = 0, Var(X_i) = 1$ and denote $W=n^{-1/2}\sum X_i$. By using Stein's method, we have that $$d_W(\mathcal{L}(W),N(0,1)) \leq \frac{5 E|X_1|^3}{n^{1/2}},$$ where $d_W$ is the Wasserstein metric.

2. Any binomial distribution $B(n, p)$ is the sum of $n$ independent Bernoulli trials $B(1, p)$, so you can consider $X_i$ as a normalized Bernoulli random variable and apply the above inequality.