# Using $\delta$-method to “estimate” undefined moments of a random variable?

I posted this over on MSE without much luck. Not sure if posting here is considered cross-posting but I can remove it if it is.

Let $$X\sim\mathcal N(\sqrt 2,1/x^2)$$. The expected value $$\mathsf EX^{-1}$$ is undefined; however, we can assign it a value via the Cauchy principal value interpretation $$\mathsf{E}X^{-1}\overset{\text{p.v.}}{=}\lim_{\epsilon\nearrow 0}\left(\int_{-\infty}^{-\epsilon}+\int_\epsilon^\infty\right)\frac{1}{t}f_X(t)\,\mathrm dt=\sqrt 2 x\mathcal D(x),$$ where $$\mathcal D(x):=e^{-x^2}\int_0^xe^{t^2}\,\mathrm dt$$ is Dawson's integral. Notice that as $$x\to\infty$$, $$\mathsf{Var}X\to 0$$ and so I wondered what would happen if I were to apply the $$\delta$$-method to estimating $$\mathsf EX^{-1}$$. Let $$g(X)=1/X$$. Expanding $$g$$ in a Taylor series about $$\mathsf EX=\sqrt 2$$ gives $$g(X)=\frac{1}{\sqrt 2}\sum_{k=0}^\infty\left(\frac{\sqrt 2-X}{\sqrt 2}\right)^k$$ and so evaluating the expected value simply requires knowing central moments of the normal distribution. We find $$\mathsf Eg(X) %=\frac{1}{\sqrt 2}\sum_{k=0}^\infty\mathsf E\left(\frac{\sqrt 2-X}{\sqrt 2}\right)^k %=\frac{1}{\sqrt 2}\sum_{k=0}^\infty\mathsf E\left(\frac{X-\sqrt 2}{\sqrt 2}\right)^{2k} =\frac{1}{\sqrt 2}\sum_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}.$$ Looking back at our expression for $$\mathsf EX^{-1}$$ it stands to reason that if we then divide $$\mathsf Eg(X)$$ by $$\sqrt 2 x$$ that we obtain an estimate for $$\mathcal D(x)$$ at large $$x$$, namely, $$\mathcal D(x)\sim\frac{1}{2x}\sum_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}=\frac{1}{2x}+\frac{1}{4x^3}+\mathcal O(x^{-5}).$$ Comparing these first two terms with eqn. (9) here seems to indicate the above expression is indeed an asymptotic expansion of $$\mathcal D(x)$$ for large $$x$$ and thus $$\mathsf E g(X)$$ provides us with an asymptotic expansion for the Cauchy principal value of $$\mathsf EX^{-1}$$ as $$x\to\infty$$.

There is no reason to stop here as we could go further and apply the same approach to "estimate" $$\mathsf{Var}X^{-1}$$ yielding $$\mathsf{Var}X^{-1}\sim %\mathsf Eg^2(X)-(\mathsf Eg(X))^2 \frac{1}{2}\sum_{k=0}^\infty\frac{(2k+1)(2k-1)!!}{(2x^2)^k}-\frac{1}{2}\left(\sum_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}\right)^2 =\frac{1}{4x^2}+\frac{1}{x^4}+\mathcal O(x^{-6}).$$

My question fundamentally has to do with what exactly the $$\delta$$-method is estimating when we apply it to estimating moments of random variables for which the moments to not exist? In this specific case, we could claim (at least I think) that the $$\delta$$-method gave us an asymptotic expansion for the Cauchy principal value of $$\mathsf EX^{-1}$$. However, for the second moment, the Cauchy principal value interpretation of the integral would give us $$\mathsf EX^{-2}\overset{\text{p.v.}}{=}\infty$$ whereas the $$\delta$$-method gave us a finite expression. So what the heck did the $$\delta$$-method estimate?

$$\newcommand\vp\varepsilon$$

1. Of course, your divergent series for $$EX^{-1}$$ and $$EX^{-2}$$ should be understood as asymptotic expansions. The delta method practically never involves series; it involves asymptotic expansions instead, with an appropriately controlled remainder.

2. Take any natural $$k$$. With the modifications discussed above, your delta method estimates a lot of things. In particular, for $$X\sim N(a,1/x^2)$$ with a real $$a>0$$ and $$x\to\infty$$ and for any $$\vp\in(0,a/2)$$, it estimates $$EX^{-k}1(|X|>\vp)=I_{k,a}+J_{k,a}(\vp),$$ where $$I_{k,a}:=EX^{-k}1(|X-a| $$J_{k,a}(\vp):=EX^{-k}1(|X|>\vp,|X-a|>a/2).$$ We have $$|J_{k,a}(\vp)|\le\vp^{-k}P(|X-a|>a/2)=2\vp^{-k}P(Z>ax/2)=o(x^{-m})\tag{1}$$ for any natural $$m$$, where $$Z\sim N(0,1)$$.

Using the series $$x^{-k}=\sum_{n=0}^\infty \binom{-k}{n}a^{-k-n}(x-a)^n$$ for $$x$$ with $$|x-a|, we get the asymptotic expansion \begin{aligned}I_{k,a}&\sim\sum_{n=0}^\infty \binom{-k}{n}a^{-k-n} E(X-a)^n1(|X-a| for any natural $$m$$ (cf. (1)). So, $$EX^{-k}1(|X|>\vp)\sim\sum_{n=0}^\infty \binom{-k}{n}a^{-k-n} E(X-a)^n.$$ In particular, $$EX^{-k}1(|X|>\vp)=\frac1{a^k}\Big(1+\frac{(k+1) k}{2 a^2 x^2}+\frac{(k+3) (k+2) (k+1) k}{8 a^4 x^4}\Big)+O(x^{-6}).$$

These results will hold if, instead of fixing $$\vp$$, we allow $$\vp$$ to go to $$0$$, but not overly too fast: in particular, the following very mild requirement is enough: $$\vp>e^{-ba^2x^2/k}$$ for some real $$b\in(0,1/2)$$ and all large enough $$x>0$$.

(For $$k=1$$, $$EX^{-k}$$ exists in $$\mathbb R$$ in the principal value sense -- that is, as $$\lim_{\vp\downarrow0}EX^{-1}1(|X|>\vp)$$, but this does not hold for any other natural $$k$$. This distinction has little, if anything, to do with the delta method.)

• @AaronHendrickson : Thank you for spotting the mistake with $|Z|$. This is now fixed. – Iosif Pinelis Mar 29 at 16:32