Expected value of Taylor series with central moments of binomial variate

Question

I want to understand this entry, but do not understand how the $\mathcal{O}\left(\frac{1}{n^2}\right)$ in the accepted answer comes into play.

I reproduce the question here: We have $x \sim \mathrm{Bin}_{p,n}$ and want to approximate $E[\ln(x+1)]$ with a Taylor series. And I'm only interested in the case when the Taylor series is convergent for all values of $x$. (This condition implies $p> \frac{1}{2} $). Thus, \begin{equation} E[T_{x_0=np}\left(\ln(x+1)\right)]=\ln(np + 1) - \sum_{i=1}\frac{(-1)^{i}}{i(np-1)^{i}}E[(x-np)^{i}] \end{equation} In the accepted solution, the suggestion is to use the following approximation: \begin{equation} E[\ln(x+1)]=\ln(np + 1) - \frac{np(1-p)}{2(np+1)^{2}} + \mathcal{O}\left(\frac{1}{n^2}\right) \end{equation} Where the second summand comes from the variance term $(i = 2)$. I have not found an upper bound for central moments of Binomial variates that could lead to the big Oh notation. Plugging in a closed form expression for the respective central moments from this paper does not lead me to this approximation.

Answer 1

There are many ways to bound the Binomial central moments. A good (very general) estimate is provided by the Marcinkiweicz-Zygmund inequality, in the sharp form due to Burkholder [1], Writing $X-np$ as a sum of $n$ i.i.d. mean zero variables $Y_i$ taking values $1-p$ and $-p$, observe that the square function S(X) defined on page 87 in [1] is bounded by $n^{1/2}$. Theorem 3.1 page 87 in [1] then gives $$E[|X-np|^k] \le \bigl((k-1)n^{1/2}\bigr)^k \,.$$
The case $k=3$ is best dealt with directly, and here we cannot afford the absolute value inside the expectation: $$E[(X-np)^3] =\sum_{i,j,\ell \le n} E[Y_iY_jY_\ell]=\sum_{i \le n} E[Y_i^3] \le n \,.$$

[1] Burkholder, Donald L. "Sharp inequalities for martingales and stochastic integrals." Astérisque 157, no. 158 (1988): 75-94. http://www.numdam.org/article/AST_1988__157-158__75_0.pdf

Remark: In my answer I focus on the question regarding Binomial central moments. for estimating the logarithm, see Iosif Pinelis' answer.

Answer 2

"And I'm only interested in the case when the Taylor series is convergent for all values of $x$." -- There is no such Taylor series for $\ln$ (which is only defined on $(0,\infty)$).

However, to get the desired $O(1/n^2)$, you do not need a convergent series; you do not need any series at all. Instead, you need the Taylor expansion \begin{equation} \ln(1+u)=u-u^2/2+u^3/3+O(u^4) \tag{1} \end{equation} for $u\ge-1/2$ (say), with a universal constant in $O(u^4)$.

Indeed, letting \begin{equation} U:=\frac{X-np}{np+a}, \tag{2} \end{equation} for all real $a>0$ you can write \begin{align*} &E\ln(X+a)-\ln(np+a) \\ &=E\ln(1+U) \\ &=E\ln(1+U)1(U\ge-1/2) \\ &+E\ln(1+U)1(U<-1/2). \tag{3} \end{align*} By (1),
\begin{align*} &E\ln(1+U)\,1(U\ge-1/2) \\ &=E(U-U^2/2+U^3/3+O(U^4))\,1(U\ge-1/2) \\ &=E(U-U^2/2+U^3/3)+O(EU^4) \\ &-E(U-U^2/2+U^3/3)1(U<-1/2). \tag{4} \end{align*} Now we are going to use (say) Rosenthal's inequality (see e.g. formula (7) in this paper), which implies \begin{equation} EU^4=O(1/n^2) \tag{5} \end{equation} and \begin{equation} EU^6=O(1/n^3); \tag{6} \end{equation} here in what follows, the constants in $O(\cdot)$ may depend only on $p$ (and, in (11) and (12), also on $a$). By (6), for $m=0,1,2,3$ we have $|U|^m\,1(U<-1/2)|\le U^6(1/2)^{m-6}$ and hence \begin{equation} |EU^m\,1(U<-1/2)|\le EU^6(1/2)^{m-6}=O(1/n^3). \tag{7} \end{equation} Also, \begin{equation} EU=0,\quad EU^2=\frac{npq}{(np+a)^2} \tag{8} \end{equation} (where $q:=1-p$), $E(X-np)^3=n(pq^3-qp^3)=O(n)$ and hence \begin{equation} EU^3=O(1/n^2). \tag{9} \end{equation} By (4), (5), (7), (8), (9), \begin{align*} &E\ln(1+U)\,1(U\ge-1/2) \\ &=E(U-U^2/2+U^3/3)+O(1/n^2) \\ &=-\frac{npq}{2(np+a)^2}+O(1/n^2). \tag{10} \end{align*} By (2) and the obvious inequality $X\ge0$, we have $1+U\ge\frac a{np+a}$. Also, if $U<-1/2$, then $1+U<1/2$, whence $\ln(1+U)<0$ and hence $|\ln(1+U)|\le \ln\frac{np+a}a$. So, \begin{align*} &E|\ln(1+U)\,1(U<-1/2)| \\ &\le\ln\frac{np+a}a E1(U<-1/2)=O(1/n^2), \tag{11} \end{align*} by (7) with $m=0$. Now (3) and (10) yield \begin{align*} &E\ln(X+a)=\ln(np+a)-\frac{npq}{2(np+a)^2}+O(1/n^2), \tag{12} \end{align*} as desired.