This is a question that I also struggle with sometimes. On the one hand, I understand the value of sweeping things under the carpet when students are not ready for them yet. When I learned Calculus in High School, we talked about -- but never properly defined -- limits (I'm can't recall if we did the limit derivatives). Yet, we managed to go pretty far into the material, e.g. establishing recurrence relations for integrals of the type $\int_a^b e^{-x}\sin(n x)$. This lack of definition was a very frustrating point for me, and when I finally learned about $(\epsilon,\delta)$ two years later, a wave of relief washed over me. Yet, I'm pretty sure that my cohorts did not feel the same way, hence my sympathy for teachers who want to keep things simple by hiding the definition.

At the same time, I don't want my Calc course to be a series of magic tricks, so I always insist on the logical construction of the course: we want to investigate slopes of tangents. We want to work exactly, not approximately. This is why we'll get into limits in the first place (not very historical, but a logical development). So what do I do?

• I briefly cover $(\epsilon,\delta)$ without really applying it. Just to show the difference between a "wordy" definition and a mathematical one.
• I insist on the fact that the limit laws are derived fro this rigorous definition. (You can sketch the proof for the sum of limits for instance).
• I am upfront about the fact that I don't expect my students to use the $(\epsilon,\delta)$ definition, though I like them to memorize it. The payoff will be later.
• I also stress that, historically, calculus was done without this definition for a long time: so it can be done, they will be able to do it, but it also has its limitations when dealing with more abstract material.

In a course that is set up in this way, it is quite natural to cover the limit definition of derivatives. There are a lot of good reasons why one should do that anyway, some of which have already been addressed. Functions which are defined piecewise do require this, and that includes important examples like $\exp(-1/x^2)$ and fun ones like $x^2\sin(1/x)$. The rigorous derivation of the derivative of $\sin x$ is another good example.

There are also wrong ways of doing this. In the comments, Holger pointed to the case study in the Notices article Teaching mathematics graduate students how to teach. Here, the problem asked to use the definition of derivative to compute the slope of a certain cubic at a point. By the time the exam rolls around, you have easier ways of doing this, so of course the students would feel that this is an arbitrary and confusing question.

One example that I have yet to see though is Taylor series: defining the derivative in this way makes it obvious that $$ f(a+h) \approx f(a)+f'(a)h+o(h)$$ and sets you up for the higher order ones. Yes, you can see that from the graph too, but at that level most of my students have a terrible time reasoning from an abstract graph.

Given how fundamental these ideas are, especially in Physics, I can never stress enough these kinds of relationship in my course.

This is a question that I also struggle with sometimes. On the one hand, I understand the value of sweeping things under the carpet when students are not ready for them yet. When I learned Calculus in High School, we talked about -- but never properly defined -- limits (I'm can't recall if we did the limit derivatives). Yet, we managed to go pretty far into the material, e.g. establishing recurrence relations for integrals of the type $\int_a^b e^{-x}\sin(n x)$. This lack of definition was a very frustrating point for me, and when I finally learned about $(\epsilon,\delta)$ two years later, a wave of relief washed over me. Yet, I'm pretty sure that my cohorts did not feel the same way, hence my sympathy for teachers who want to keep things simple by hiding the definition.

At the same time, I don't want my Calc course to be a series of magic tricks, so I always insist on the logical construction of the course: we want to investigate slopes of tangents. We want to work exactly, not approximately. This is why we'll get into limits in the first place (not very historical, but a logical development). So what do I do?

• I briefly cover $(\epsilon,\delta)$ without really applying it. Just to show the difference between a "wordy" definition and a mathematical one.
• I insist on the fact that the limit laws are derived fro this rigorous definition. (You can sketch the proof for the sum of limits for instance).
• This sets up for students how the mathematical edifice is built: abstract definitions to formalize intuition, big gun theorems proved rigorously from these definition (limit laws, derivative laws...). Add a few examples to the mix and then you're set up for practical, mechanical computations (the stuff that computers do).
• I am upfront about the fact that I don't expect my students to use the $(\epsilon,\delta)$ definition, though I like them to memorize it. The payoff will be later.
• I also stress that, historically, calculus was done without this definition for a long time: so it can be done, they will be able to do it, but it also has its limitations when dealing with more abstract material.

In a course that is set up in this way, it is quite natural to cover the limit definition of derivatives. There are a lot of good reasons why one should do that anyway, some of which have already been addressed. Functions which are defined piecewise do require this, and that includes important examples like $\exp(-1/x^2)$ and fun ones like $x^2\sin(1/x)$. The rigorous derivation of the derivative of $\sin x$ is another good example.

There are also wrong ways of doing this. In the comments, Holger pointed to the case study in the Notices article Teaching mathematics graduate students how to teach. Here, the problem asked to use the definition of derivative to compute the slope of a certain cubic at a point. By the time the exam rolls around, you have easier ways of doing this, so of course the students would feel that this is an arbitrary and confusing question.

[Actually, I took so long to write this that I've been ninja'd by Pietro on this example.] One example that I have yet to see though is Taylor series: defining the derivative in this way makes it obvious that $$ f(a+h) \approx f(a)+f'(a)h+o(h)$$ and sets you up for the higher order ones. Yes, you can see that from the graph too, but at that level most of my students have a terrible time reasoning from an abstract graph.

Given how fundamental these ideas are, especially in Physics, I can never stress enough these kinds of relationship in my course.