Maybe I have misunderstood, and please forgive me if so, but this question, as asked, seems like asking why we should teach understanding as opposed to rote memorization. There are options that include understanding other than limits, such as finding the linear approximation of a function. This is computable for polynomials by re - expansion, i.e. setting x = (x-a) +a, then picking off the coefficient of the linear term in (x-a). But just teaching rules denies the student the ability to use the concept in any situation other than the ones met before.
It is worth noting that there is a lot of historical precedent for teaching it as a limit, which occurs already in Euclid. I.e. Euclid characterizes the tangent to a circle as the unique line such that between it and any other line through the same point, one can interpose a secant (Prop. 16, Book III). (Strictly, he says equivalently that one cannot interpose another line between the tangent and the circle itself, i.e. every other line through the point is a secant.) Thus the tangent is the limit of those secants. Thus I believe one can easily say that the limiting point of view is the original one of Euclid. From this point of view, the idea of limit is the one used so fruitfully by the Greeks, and the contribution of the mathematicians of later times is to make that notion more precise.
On the other hand, if you want to avoid the conceptual difficulty students have with limits, you can follow Descartes instead, at least for derivatives of polynomials, and characterize the tangent line as the unique line such that subtracting its equation from the original function gives a polynomial with a double root at the given point. This leads to motivating the Zariski cotangent space, as M/M^2.
Both points of view also have a nice dynamic interpretation as realizing the tangent as the unique line intersecting the curve doubly at the point, understood as the limit of the two secant intersections,and measured by the presence of a double root.
But if you want a defense of limits, I suggest Euclid Prop. 16, Book III as ample precedent.
If you want a defense of making students practice using the limit definition, I propose that as noted above, this is the only way to get them to appreciate the fundamental theorem of calculus. That theorem cannot be appreciated by memorizing rules for derivatives, One must understand the definition and apply it to an abstractly defined area function. I suggest that one reason many students do not understand why the fundamental theorem of calculus is true, is that (again as noted above) they have not grasped either what an abstractly defined function is, nor what a derivative truly means.
So if you want them to understand the relation between the derivative and the integral, then I agree with others that they need to know what a function is and derivative is. The reasoning here is that once someone understands something, he can use it in more settings than could possibly be covered by any set of rules.
Another practical benefit of testing the use of the h-->0 definition to obtain derivatives of simple functions, is that it forces practice in algebra, trig identities, and exponentials, skills which most of my students are almost completely lacking.
However, I recommend you teach it any way that makes sense to you. after all you understand it, so whatever you say based on that understanding will be useful. Make up your mind what seems important to you, and go for it!