1
$\begingroup$

I posted this over on MSE without much luck. Not sure if posting here is considered cross-posting but I can remove it if it is.

Let $X\sim\mathcal N(\sqrt 2,1/x^2)$. The expected value $\mathsf EX^{-1}$ is undefined; however, we can assign it a value via the Cauchy principal value interpretation $$ \mathsf{E}X^{-1}\overset{\text{p.v.}}{=}\lim_{\epsilon\nearrow 0}\left(\int_{-\infty}^{-\epsilon}+\int_\epsilon^\infty\right)\frac{1}{t}f_X(t)\,\mathrm dt=\sqrt 2 x\mathcal D(x), $$ where $\mathcal D(x):=e^{-x^2}\int_0^xe^{t^2}\,\mathrm dt$ is Dawson's integral. Notice that as $x\to\infty$, $\mathsf{Var}X\to 0$ and so I wondered what would happen if I were to apply the $\delta$-method to estimating $\mathsf EX^{-1}$. Let $g(X)=1/X$. Expanding $g$ in a Taylor series about $\mathsf EX=\sqrt 2$ gives $$ g(X)=\frac{1}{\sqrt 2}\sum_{k=0}^\infty\left(\frac{\sqrt 2-X}{\sqrt 2}\right)^k $$ and so evaluating the expected value simply requires knowing central moments of the normal distribution. We find $$ \mathsf Eg(X) %=\frac{1}{\sqrt 2}\sum_{k=0}^\infty\mathsf E\left(\frac{\sqrt 2-X}{\sqrt 2}\right)^k %=\frac{1}{\sqrt 2}\sum_{k=0}^\infty\mathsf E\left(\frac{X-\sqrt 2}{\sqrt 2}\right)^{2k} =\frac{1}{\sqrt 2}\sum_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}. $$ Looking back at our expression for $\mathsf EX^{-1}$ it stands to reason that if we then divide $\mathsf Eg(X)$ by $\sqrt 2 x$ that we obtain an estimate for $\mathcal D(x)$ at large $x$, namely, $$ \mathcal D(x)\sim\frac{1}{2x}\sum_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}=\frac{1}{2x}+\frac{1}{4x^3}+\mathcal O(x^{-5}). $$ Comparing these first two terms with eqn. (9) here seems to indicate the above expression is indeed an asymptotic expansion of $\mathcal D(x)$ for large $x$ and thus $\mathsf E g(X)$ provides us with an asymptotic expansion for the Cauchy principal value of $\mathsf EX^{-1}$ as $x\to\infty$.

There is no reason to stop here as we could go further and apply the same approach to "estimate" $\mathsf{Var}X^{-1}$ yielding $$ \mathsf{Var}X^{-1}\sim %\mathsf Eg^2(X)-(\mathsf Eg(X))^2 \frac{1}{2}\sum_{k=0}^\infty\frac{(2k+1)(2k-1)!!}{(2x^2)^k}-\frac{1}{2}\left(\sum_{k=0}^\infty\frac{(2k-1)!!}{(2x^2)^k}\right)^2 =\frac{1}{4x^2}+\frac{1}{x^4}+\mathcal O(x^{-6}). $$

My question fundamentally has to do with what exactly the $\delta$-method is estimating when we apply it to estimating moments of random variables for which the moments to not exist? In this specific case, we could claim (at least I think) that the $\delta$-method gave us an asymptotic expansion for the Cauchy principal value of $\mathsf EX^{-1}$. However, for the second moment, the Cauchy principal value interpretation of the integral would give us $\mathsf EX^{-2}\overset{\text{p.v.}}{=}\infty$ whereas the $\delta$-method gave us a finite expression. So what the heck did the $\delta$-method estimate?

$\endgroup$
1
$\begingroup$

$\newcommand\vp\varepsilon$

  1. Of course, your divergent series for $EX^{-1}$ and $EX^{-2}$ should be understood as asymptotic expansions. The delta method practically never involves series; it involves asymptotic expansions instead, with an appropriately controlled remainder.

  2. Take any natural $k$. With the modifications discussed above, your delta method estimates a lot of things. In particular, for $X\sim N(a,1/x^2)$ with a real $a>0$ and $x\to\infty$ and for any $\vp\in(0,a/2)$, it estimates $$EX^{-k}1(|X|>\vp)=I_{k,a}+J_{k,a}(\vp),$$ where $$I_{k,a}:=EX^{-k}1(|X-a|<a/2),$$ $$J_{k,a}(\vp):=EX^{-k}1(|X|>\vp,|X-a|>a/2).$$ We have $$|J_{k,a}(\vp)|\le\vp^{-k}P(|X-a|>a/2)=2\vp^{-k}P(Z>ax/2)=o(x^{-m})\tag{1}$$ for any natural $m$, where $Z\sim N(0,1)$.

Using the series $$x^{-k}=\sum_{n=0}^\infty \binom{-k}{n}a^{-k-n}(x-a)^n$$ for $x$ with $|x-a|<a/2$, we get the asymptotic expansion $$ \begin{aligned}I_{k,a}&\sim\sum_{n=0}^\infty \binom{-k}{n}a^{-k-n} E(X-a)^n1(|X-a|<a/2) \\ &\sim\sum_{n=0}^\infty \binom{-k}{n}a^{-k-n} E(X-a)^n+o(x^{-m}) \end{aligned}$$ for any natural $m$ (cf. (1)). So, $$EX^{-k}1(|X|>\vp)\sim\sum_{n=0}^\infty \binom{-k}{n}a^{-k-n} E(X-a)^n.$$ In particular, $$EX^{-k}1(|X|>\vp)=\frac1{a^k}\Big(1+\frac{(k+1) k}{2 a^2 x^2}+\frac{(k+3) (k+2) (k+1) k}{8 a^4 x^4}\Big)+O(x^{-6}).$$

These results will hold if, instead of fixing $\vp$, we allow $\vp$ to go to $0$, but not overly too fast: in particular, the following very mild requirement is enough: $\vp>e^{-ba^2x^2/k}$ for some real $b\in(0,1/2)$ and all large enough $x>0$.


(For $k=1$, $EX^{-k}$ exists in $\mathbb R$ in the principal value sense -- that is, as $\lim_{\vp\downarrow0}EX^{-1}1(|X|>\vp)$, but this does not hold for any other natural $k$. This distinction has little, if anything, to do with the delta method.)

$\endgroup$
1
  • $\begingroup$ @AaronHendrickson : Thank you for spotting the mistake with $|Z|$. This is now fixed. $\endgroup$ – Iosif Pinelis Mar 29 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.