Well, there seems to be a lot of literature. I have encountered similar questions once when discussing problems in deformation quantization. here the ordered field is simply the field of formal Laurent series $\mathbb{R}((\hbar))$ with the ordering that $\hbar$ is positive (this fixed it uniquely). The idea was to transfer as much of stuff from $C^*$-algebra theory to the formal star products which are algebras of the above non-Archimedian field. Ultimately, we wanted to develop a spectral theory for these algebras, which utterly failed :) Nevertheless, you might be interested in examples of such fields and the "completed Newton Puiseux field" might be a funny one. First you take the algebraic or real closure of the formal Laurent series yielding the Newton-Puiseux series. This is not yet complete (in the sense of convergent Cauchy sequences) so you still have to complete it. The result is again field (either real, i.e. ordered) or algebraically closed, the smallest one containing the formal Laurent series.
Let me probably clarify what I mean by completion here: since you have an ordered field, the naive $\epsilon$-definition of a Cauchy sequence makes sense with the main point that the $\epsilon$ is now a positive element of you field and not just a positive real number. So this defines a topology on your ordered field, but also a uniform structure. So one can speak about Cauchy sequences and completeness etc. It is then a standard argument to show that the completion (in the sense of uniform structures, i.e. taking Cauchy sequences module zero sequences) gives again an ordered field and the original one is included via constant sequences as usual. The slightly less trivial point is that if you start with an ordered field which is real closed then the completion is again real closed. This is needed to see that the completed Newton-Puiseux series are indeed both: real (or algebraically, in the case you started with $\mathbb{C}$ instead of $\mathbb{R}$) closed and (Cauchy) complete.
A first reading may be
Narici, N., Beckenstein, E., Bachman, G.: Functional Analysis and Valuation Theory. New York: Marcel Dekker, 1971.
However, the main point is that analysis does not work too well. The reasons is that whenever you need suprema, you're on your own. And this happens quite often in analysis :) The other problem arising is that $1/n$ is no longer a zero sequence, a fact which is also used very often in analysis: you named already the intermediate value theorem...
So my conclusion is that notions of positivity work well (needed a lot in formal DQ and representation theory of $^*$-algebras) but notions of calculus do not work well.