There is much discussion both in the education community and the mathematics community concerning the challenge of (epsilon, delta) type definitions in calculus and the student reception of them. The mathematical community often holds an upbeat opinion on the success of student reception of this (see examples below), whereas the education community often stresses difficulties and their "baffling" and "inhibitive" effect (see below). A typical educational perspective on this was recently expressed by Paul Dawkins in the following terms:
2.3. Student difficulties with real analysis definitions. The concepts of limit and continuity have posed well-documented difficulties for students both at the calculus and analysis level of instructions (e.g. Cornu, 1991; Cottrill et al., 1996; Ferrini-Mundy & Graham, 1994; Tall & Vinner, 1981; Williams, 1991). Researchers identified difficulties stemming from a number of issues: the language of limits (Cornu, 1991; Williams, 1991), multiple quantification in the formal definition (Dubinsky, Elderman, & Gong, 1988; Dubinsky & Yiparaki, 2000; Swinyard & Lockwood, 2007), implicit dependencies among quantities in the definition (Roh & Lee, 2011a, 2011b), and persistent notions pertaining to the existence of infinitesimal quantities (Ely, 2010). Limits and continuity are often couched as formalizations of approaching and connectedness respectively. However, the standard, formal definitions display much more subtlety and complexity. That complexity often baffles students who cannot perceive the necessity for so many moving parts. Thus learning the concepts and formal definitions in real analysis are fraught both with need to acquire proficiency with conceptual tools such as quantification and to help students perceive conceptual necessity for these tools. This means students often cannot coordinate their concept image with the concept definition, inhibiting their acculturation to advanced mathematical practice, which emphasizes concept definitions.
See the entire article (note that the online article provides links to the papers cited above).
To summarize, in the field of education, researches decidedly have not come to the conclusion that epsilon, delta definitions are either "simple", "clear", or "common sense". Meanwhile, mathematicians often express contrary sentiments. Two examples are given below.
...one cannot teach the concept of limit without using the epsilon-delta definition. Teaching such ideas intuitively does not make it easier for the student it makes it harder to understand. Bertrand Russell has called the rigorous definition of limit and convergence the greatest achievement of the human intellect in 2000 years! The Greeks were puzzled by paradoxes involving motion; now they all become clear, because we have complete understanding of limits and convergence. Without the proper definition, things are difficult. With the definition, they are simple and clear.
see Kleinfeld, Margaret; Calculus: Reformed or Deformed? Amer. Math. Monthly 103 (1996), no. 3, 230-232.
I always tell my calculus students that mathematics is not esoteric: It is common sense. (Even the notorious epsilon, delta definition of limit is common sense, and moreover is central to the important practical problems of approximation and estimation.)
see Bishop, Errett; Book Review: Elementary calculus. Bull. Amer. Math. Soc. 83 (1977), no. 2, 205--208.
Having presented some published references, I would like to complement the published references with anecdotal evidence drawn from this very page, namely the sentiment that
epsilon-delta definition is immediately appealing to students with a certain clarity of mind, etc.
Such sentiments I believe are common and reveal a belief that these convoluted definitions are "natural" and a failure to follow them possibly constitutes an absence of a "clarity of mind".
When one compares the upbeat assessment found in the mathematics community and the somber assessments common in the education community, sometimes one wonders whether they are talking about the same thing. How does one bridge the gap between the two assessments? Are they perhaps dealing with distinct student populations? Are there perhaps education studies providing more upbeat assessments than Dawkins' article would suggest?
Note 1. A number of editors have commented by now on the subject of alternatives. Thus, Neil Strickland notes that "People may think that rigorous analysis is valuable despite this [teaching difficulty], and that certain other proposed teaching methods are no better, but those are different questions." In response, I would like to mention that the comparison of distinct approaches to teaching analysis is a fascinating subject, on which I have first hand experience and much to say. However, this subject is not that of this question.
Note 2. Evidence from classroom experience comparing the two approaches has been presented in this recent publication based on the opinion poll of the students involved.