estimate the error term in CLT - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-19T11:59:57Z http://mathoverflow.net/feeds/question/39289 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/39289/estimate-the-error-term-in-clt estimate the error term in CLT mr.gondolier 2010-09-19T10:02:28Z 2010-09-20T20:51:48Z <p>Let $X_m = \frac{1}{\sqrt{m}}\sum_{k=1}^m Z_k$ where $Z_k$ are iid equally likely on $\{\pm 1\}$. Then $X_m$ convergens to $X \sim \mathcal{N}(0,1)$ in distribution by CLT.</p> <p>Let $f$ be a smooth bounded function on $\mathbb{R}$. Then $\mathbb{E}[f(X_m)] \to \mathbb{E}[f(X)]$. I wonder if there is any general method to give sharp asymptotic estimate of the error term $\mathbb{E}[f(X_m)] - \mathbb{E}[f(X)]$, which I expect to be $\Theta(1/m)$. The scaling constant should depend on $f$ (as well as the distribution of $Z_k$ if they are not binary). </p> <p>For law of large number, this type of estimate can be done via the <a href="http://en.wikipedia.org/wiki/Delta_method" rel="nofollow">Delta method</a> (e.g., to estimate $\mathbb{E}[f(\bar{Z})] - f(0)$). There must be a counterpart for CLT... I haven't found the <a href="http://en.wikipedia.org/wiki/Edgeworth_series" rel="nofollow">Edgeworth expansion</a> useful because it seems to work with distribution with densities.</p> <p><strong>Edited:</strong> To be clear, I am only interested in some specific nice function (e.g., $f(x) = x^2 e^{-x^2/4}$) and finding a sharp expansion for the error term of the form, say, $c/m + o(1/m)$, where $c$ will depend n $f$. As pointed by Mark, the worst-case rate of all bounded smooth function $f$ is $1/\sqrt{m}$, which agrees with the upper bound given by Stein's method.</p> http://mathoverflow.net/questions/39289/estimate-the-error-term-in-clt/39313#39313 Answer by Nate Eldredge for estimate the error term in CLT Nate Eldredge 2010-09-19T16:36:33Z 2010-09-19T16:36:33Z <p>The <a href="http://en.wikipedia.org/wiki/Berry%E2%80%93Esseen_theorem" rel="nofollow">Berry-Esseen theorem</a> is a classical result of this sort. It predicts errors on the order of $m^{-1/2}$, however.</p> http://mathoverflow.net/questions/39289/estimate-the-error-term-in-clt/39353#39353 Answer by Byron Schmuland for estimate the error term in CLT Byron Schmuland 2010-09-20T01:07:20Z 2010-09-20T01:07:20Z <p>If $X_m$ has cumulative distribution function $F_m$, and $X$ has cumulative distribution function $F$, then (at least formally) integration by parts gives you $$E(f(X_m))-E(f(X))=\int (F_m(x)-F(x)) df(x).$$ Now you can apply the Berry-Esseen bound. </p> http://mathoverflow.net/questions/39289/estimate-the-error-term-in-clt/39392#39392 Answer by Mark Meckes for estimate the error term in CLT Mark Meckes 2010-09-20T14:54:19Z 2010-09-20T14:54:19Z <p>Stein's method typically gives good Berry-Esseen type bounds for smooth test functions. See Chapter III of <a href="http://books.google.com/books?id=6C2g7_-KnhsC&amp;lpg=PP1&amp;dq=approximate%2520computation%2520of%2520expectations&amp;pg=PP1#v=onepage&amp;q&amp;f=false" rel="nofollow">Stein's book</a> (entirely viewable in Google Books). For example, specializing to your case of symmetric Bernoulli summands, equation (37) on p. 38 gives $$\vert \mathbb{E}f(X_m)-\mathbb{E}f(X)\vert \le \frac{2\Vert f' \Vert_\infty}{\sqrt{m}}.$$ For more general summands, there is some simple dependence on the third and fourth moments as well as $\Vert f \Vert_\infty$.</p> <p>Also, I'm pretty sure that $m^{-1/2}$ is the correct rate here even for Bernoullis, although I can't find a reference for a lower bound at the moment. Why do you expect better?</p> http://mathoverflow.net/questions/39289/estimate-the-error-term-in-clt/39398#39398 Answer by mr.gondolier for estimate the error term in CLT mr.gondolier 2010-09-20T15:42:53Z 2010-09-20T15:42:53Z <p>Sorry this is NOT an answer to my question... just some clarafications.</p> <p>The reason I think $m^{-1/2}$ is not tight is as follows. For example, take $f$ to be the characteristic function, we have</p> <p>$\mathbb{E}[e^{itX_m}] = (\mathbb{E}[e^{it Z/\sqrt{m}}])^m = (1 - t^2/(2m) + o(1/m))^m \to e^{-t^2/2} = \mathbb{E}[e^{itX}]$</p> <p>at rate $1/m$, because $m\log(1-1/m) \to -1$ at rate $1/m$.</p> <p>Also, it seems all moments of $X_m$ converge to the moments of $X$ at rate $1/m$. Doing a Taylor expansion for those nice $f$ should also yield a rate of $1/m$?</p>