I wonder if the approximation $(1-hiA)\approx\exp(-hiA)$ is valid...

@DavidKetcheson thank you for the reference - I'll be reading that. Although the first thing that is done in this chapter is the trick I mentioned

@Piyush I am not sure myself - this is really an open ended question for discussion. In the motivating example one could ask for optimal convergence rates for certain classes of loss functions. In the general case maybe optimal in the sense that the difference between the real ode and the discretization stays smaller than epsilon for as large as possible step sizes. But you are welcome to point out other measures