How to prove the equation holds in asymptotic sense

Question

The random variables $X_1,X_2,\cdots,X_{k}$ have a multinomial distribution if and only if their joint probability mass distribution is given by $$ \mathbf{P}\left\{X_{1}=n_{1}, X_{2}=n_{2}, \ldots, X_{k}=n_{k}\right\} = \frac{n !}{n_{1} ! n_{2} ! \ldots n_{k} !} p_{1}^{n_{1}} p_{2}^{n_{2}} \ldots p_{k}^{n_{k}} $$ where $p_{i}\in \left (0,1 \right ),n_{i}\in\left\{1,\ldots,n\right\},\sum_{i=1}^{k} n_{i}=n \text { and } \sum_{i=1}^{k} p_{i}=1 .$

We say that

1. A sequence of random variables $\xi_{n}=O_{\mathrm{P}}(1) $ if and only if for any $\epsilon>0$,there is $M>0$ such that $\mathbf{P}\left(\left|\xi_{n}\right| \ge M\right)<\epsilon, n\ge 1 $;
2. A sequence of random variables $\xi_{n}=o_{\mathbf{p}}\left( 1\right)$ if and only if $\xi_{n} \stackrel{\mathbf{p}}{\longrightarrow} 0$ as $n\rightarrow\infty.$

Show that \begin{equation} \boxed{2\sum_{i=1}^k n_i\log\frac{n_i}{np_i} = \sum_{i=1}^k\frac{(n_i-np_i)^2}{np_i} + o_{\mathbf{p}}(1)\quad (n\rightarrow \infty).} \end{equation}

---

The following is my thoughts,but I'm not sure if it's correct. Please provide corrections if necessary.

By Bernoulli’s law of large numbers, For every $\varepsilon>0$ we have

\begin{aligned} &\lim _{n \rightarrow \infty} \mathbf{P}\left(\left|\frac{n_{i}}{n}-p_{i}\right| \geq \varepsilon\right)=0,\quad\text{i.e.}\quad\frac{n_i-np_{i}}{np_{i}}=o_{\mathbf{p}}(1). \end{aligned}

By CLT,The asymptotic distribution of $\frac{n_i-n p_i}{\sqrt{n}}$ is $\mathcal{N}\left( 0,p_{i}(1-p_i) \right )$,i.e.$\frac{n_i-n p_i}{\sqrt{n}}\stackrel{\mathbf{d}}{\longrightarrow}\mathcal{N}\left( 0,p_{i}(1-p_i) \right ).$ Then $\frac{n_i-n p_i}{\sqrt{n}}=O_{\mathbf{p}}(1)\Leftrightarrow n_i-n p_i=O_{\mathbf{p}}(\sqrt{n}).$

\begin{aligned} 2\sum_{i=1}^k n_i\log\frac{n_i}{n p_i} &=2\sum_{i=1}^k n_i\log\left(1+\frac{n_i-np_i}{n p_i}\right)\\ &=2\sum_{i=1}^k n_i\left[\left(\frac{n_i-n p_i}{n p_i}\right)-\frac12\left(\frac{n_i-n p_i}{n p_i}\right)^2+O\left(\left(\frac{n_i-n p_i}{n p_i}\right)^3\right)\right]\\ &=2\sum_{i=1}^k n_i\left[\left(\frac{n_i-n p_i}{n p_i}\right)-\frac12\left(\frac{n_i-n p_i}{n p_i}\right)^2\right]+\sum_{i=1}^{k}n_{i}O_{\mathbf{p}}\left((\frac{\sqrt{n}}{n})^3\right)\\ &=2\sum_{i=1}^k n_i\left[\left(\frac{n_i-n p_i}{n p_i}\right)-\frac12\left(\frac{n_i-n p_i}{n p_i}\right)^2\right]+O_{\mathbf{p}}\left(\frac{1}{\sqrt{n}}\right)\\ &=\sum_{i=1}^{k}\frac{(n_i-n p_i)^2}{n p_i}-\sum_{i=1}^{k}\left(n_i-n p_i\right)\cdot O_{\mathbf{p}}\left(\left(\frac{n_i-n p_i}{n p_i}\right)^2\right)+O_{\mathbf{p}}\left(\frac{1}{\sqrt{n}}\right)\\ &=\sum_{i=1}^{k}\frac{(n_i-n p_i)^2}{n p_i}-\sum_{i=1}^{k}\left(n_i-n p_i\right)\cdot O_{\mathbf{p}}\left(\frac{1}{n}\right)+O_{\mathbf{p}}\left(\frac{1}{\sqrt{n}}\right)\\ &=\sum_{i=1}^{k}\frac{(n_i-n p_i)^2}{n p_i}+O_{\mathbf{p}}\left(\frac{1}{\sqrt{n}}\right)\\ &=\sum_{i=1}^{k}\frac{(n_i-n p_i)^2}{n p_i}+o_{\mathbf{p}}\left(1\right). \end{aligned}

Answer 1

I will first remark that your condition 1, that $\xi_n = O_P(1)$, is exactly the definition that the family $(\xi_n)_n$ is tight.

Here is a shorter proof than your computation. Let us use $\ln(1+x) = x + x^2 \phi(x)$ with $\phi$ continuous and $\phi(x) \to -1/2$ for $x\to 0$: \begin{align} 2\sum_i n_i \ln \frac{n_i}{n p_i} &= 2\sum_i n_i \frac{n_i - np_i}{n p_i} + 2\sum_i \frac{(n_i - np_i)^2}{n p_i} \frac{n_i}{n p_i} \phi\left(\frac{n_i-np_i}{n p_i}\right) \\ &= \sum_i \frac{(n_i - np_i)^2}{n p_i} \left( 2 + 2 \frac{n_i}{n p_i} \phi\left(\frac{n_i-np_i}{n p_i}\right) \right) . \end{align} by using that $\sum_i n_i = n = \sum_i np_i$. The law of large number ensures that for every $i$ $$ \frac{n_i}{np_i} \phi\left( \frac{n_i-np_i}{np_i}\right) \to \phi(0) = -1/2 $$ almost surely as $n\to\infty$. Since $(n_i - np_i)^2 / (np_i)$ is bounded for every $i$, $$2\sum_i n_i \ln \frac{n_i}{n p_i} - \sum_i \frac{(n_i - np_i)^2}{n p_i} \to 0 $$ in distribution, and thus (because 0 is constant) in probability.

Answer 2

NOT AND ANSWER, ONLY TOO WEAK INEQUALITIES AT THIS TIME

You should replace all $n_i$ by $X_i^{(n)}$ in your question, and your proof. I did not read the proof, but the answer is trivially yes.

Note that saying that a sequence $(\xi_n)$ is $O_P(1)$, means that the sequence of distributions of the $(\xi_n)$ is tight; and saying that a sequence $(\xi_n)$ is $o_P(1)$, means that the sequence $(\xi_n)$ goes to $0$ in probability.

The random variable $(X_1^{(n)},\ldots,X_k^{(n)})$ has the same distribution as the sum of $n$ i.i.d random variables with distribution $p_1\delta_{e_1}+\cdots+p_k\delta_{e_k}$, where $(e_1,\ldots,e_k)$ is the canonical basis of $\mathbb{R}^k$.

Theorem of Karl Pearson ensures that the random variables $\sum_{i=1}^k\frac{(X_i^{(n)}-np_i)^2}{np_i}$ converge in distribution to $\chi^2_{k-1}$ as $n \to +\infty$. Tightness of the the sequence of their distributions follows.

On the one hand, by convexity of the function $f : x \mapsto x\ln(x)$, $$\sum_{i=1}^k X_i^{(n)}\ln \frac{X_i^{(n)}}{np_i} = n \sum_{i=1}^k p_i f\Big(\frac{X_i^{(n)}}{np_i}\Big) \ge nf\Big(\sum_{i=1}^k p_i\frac{X_i^{(n)}}{np_i}\Big) = nf(1) = 0.$$ On the other hand, by concavity of the function $\ln$, \begin{eqnarray*} \sum_{i=1}^k X_i^{(n)} \ln \frac{X_i^{(n)}}{np_i} &\le& \sum_{i=1}^k X_i^{(n)}\Big(\frac{X_i^{(n)}}{np_i}-1\Big) \\ &=& \sum_{i=1}^k \Big(\frac{(X_i^{(n)})^2}{np_i}-X_i^{(n)}\Big) \\ &=& \sum_{i=1}^k \Big(\frac{(X_i^{(n)})^2}{np_i}-2X_i^{(n)}+np_i\Big) \\ &=& \sum_{i=1}^k\frac{(X_i^{(n)}-np_i)^2}{np_i}. \end{eqnarray*} Hence $\sum_{i=1}^k X_i^{(n)} \ln \frac{X_i^{(n)}}{np_i}$ is also $O_P(1)$.