most recent 30 from http://mathoverflow.net 2013-06-19T23:22:37Z http://mathoverflow.net/feeds/question/28463 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/28463/optimum-small-number-for-numerical-differentiation Joonas Pulakka 2010-06-17T06:30:34Z 2012-06-19T17:36:02Z

<p><a href="http://en.wikipedia.org/wiki/Numerical_differentiation#Practical_considerations" rel="nofollow">http://en.wikipedia.org/wiki/Numerical_differentiation#Practical_considerations</a> mentions the formula</p> <p>$h=\sqrt \epsilon * x$</p> <p>where $\epsilon$ is the machine epsilon (approx. 2.2e-16 for 64-bit IEEE 754 doubles), to calculate the optimum "small number" h to be used in differentiation, such as</p> <p>$\frac{f(x+h)-f(x)}{h}$</p> <p>But what if x is zero? Then h will be zero too, and division by zero is certainly not a way to do numerical differentiation. Is the article wrong? Is it otherwise correct, except that near zero (how near?) some small enough constant (how small?) should be used?</p>

http://mathoverflow.net/questions/28463/optimum-small-number-for-numerical-differentiation/28465#28465 Gerry Myerson 2010-06-17T07:00:15Z 2010-06-17T07:00:15Z

<p>If you click through to the reference given for the Wikipedia piece, you'll find an answer. The formula given there is to take $h$ to be roughly $\sqrt{\epsilon_f}x_c$, where $\epsilon_f$ isn't necessarily "machine epsilon," but more to the point, where $x_c$ isn't necessarily $x$. </p>

http://mathoverflow.net/questions/28463/optimum-small-number-for-numerical-differentiation/28701#28701 Dan Piponi 2010-06-19T00:44:45Z 2012-06-19T17:36:02Z

<p>What you want is a number $\epsilon$ that is small enough that it isn't zero, but its square is zero. Because then we have $f(x+\epsilon)=f(x)+\epsilon f'(x)+0$, for suitable $f$. No real number has this property, but if you're implementing a numerical method on a computer it's straightforward to implement a type containing numbers that <a href="http://en.wikipedia.org/wiki/Automatic_differentiation#Automatic_differentiation_using_dual_numbers" rel="nofollow">do have this property</a>. It'll probably give you better results than anything involving small real numbers.</p> <p>(BTW There have also been papers published that propose using a complex $\epsilon$ but I think these are misguided.)</p>

http://mathoverflow.net/questions/28463/optimum-small-number-for-numerical-differentiation/33472#33472 J. M. 2010-07-27T05:22:32Z 2010-07-27T05:22:32Z

<p>I use the technique given by J.C. Nash in the book "Compact Numerical Methods for Computers".</p> <p>On page 219, there is the formula</p> <p><code>$h=\sqrt{\epsilon}\left(|x|+\sqrt{\epsilon}\right)$</code></p> <p>where <code>$\epsilon$</code> is of course machine epsilon. I have also seen the following formula used (apologies, I no longer recall where this was from):</p> <p><code>$h=\sqrt{\epsilon}\max\left(|x|,\sqrt{\epsilon}\right)$</code></p> <p>The recipe for second derivatives is similar except that one now uses the cube root of machine epsilon instead of its square root.</p>