Question

Mathematicians used to hold plenty of false, but intuitively reasonable, ideas in analysis that were backed up with proofs of one kind or another (understood in the context of those times). Coming to terms with the counterexamples led to important new ideas in analysis.

1. A convergent infinite series of continuous functions is continuous. Cauchy gave a proof of this (1821). See Theorem 1 in Cours D'Analyse Chap. VI Section 1. Five years later Abel pointed out that certain Fourier series are counterexamples. A consequence is that the concept of uniform convergence was isolated and, going back to Cauchy's proof, it was seen that he had really proved a uniformly convergent series of continuous functions is continuous. For a nice discussion of this as an educational tool, see http://www.math.usma.edu/people/Rickey/hm/CalcNotes/CauchyWrgPr.pdf. [Edit: This may not be historically fair to Cauchy. See Graviton's answer for another assessment of Cauchy's work, which operated with continuity using infinitesimals in such a way that Abel's counterexample was not a counterexample to Cauchy's theorem.]

2. Lagrange, in the late 18th century, believed any function could be expanded into a power series except at some isolated points and wrote an entire book on analysis based on this assumption. (This was a time when there wasn't a modern definition of function; it was just a "formula".) His goal was to develop analysis without using infinitesmals or limits. This approach to analysis was influential for quite a few years. See Section 4.7 of Jahnke's "A History of Analysis". Work in the 19th century, e.g., Dirichlet's better definition of function, blew the whole work of Lagrange apart, although in a reverse historical sense Lagrange was saved since the title of his book is "Theory of Analytic Functions..."

3. Any continuous function (on a real interval, with real values) is differentiable except at some isolated points. Ampere gave a proof (1806) and the claim was repeated in lots of 19th century calculus books. See pp. 43--44, esp. footnote 11 on page 44, of Hawkins's book "Lebesgue's theory of integration: its origins and development". Here is a Google Books link: http://books.google.com/books?id=oV1aLqag6WwC&pg=PA43&lpg=PA43&dq=Ampere+1806&source=bl&ots=lV-HfqHza5&sig=Q_3U85rv2o8YEok--GjYWeJSZPs&hl=en&ei=-BZmTLm8CIL-8Aatpon8Dg&sa=X&oi=book_result&ct=result&resnum=1&ved=0CBYQ6AEwAA#v=onepage&q=Ampere%201806&f=false. In 1872 Weierstrass killed the whole idea with his continuous nowhere differentiable function, which was one of the first fractal curves in mathematics. For a survey of different constructions of such functions, see http://www.scribd.com/doc/29256978/Continuous-Nowhere-Differentiable-Functions-Johan-Thim

4. A solution to an elliptic PDE with a given boundary condition could be solved by minimizing an associated "energy" functional which is always nonnegative. It could be shown that if the associated functional achieved a minimum at some function, that function was a solution to a certain PDE, and the minimizer was believed to exist for the false reason that any set of nonnegative numbers has an infimum. Dirichlet gave an electrostatic argument to justify this method, and Riemann accepted it and made significant use of it in his development of complex analysis (e.g., proof of Riemann mapping theorem). Weierstrass presented a counterexample to the Dirichlet principle in 1870: a certain energy functional could have infimum 0 with there being no function in the function space under study at which the functional is 0. This led to decades of uncertainy about whether results in complex analysis or PDEs obtained from Dirichlet's principle were valid. In 1900 Hilbert finally justified Dirichlet's principle as a valid method in the calculus of variations, and the wider classes of function spaces in which Dirichlet's principle would be valid eventually led to Sobolev spaces. A book on this whole story is A. F. Monna, "Dirichlet's principle: A mathematical comedy of errors and its influence on the development of analysis" (1975), which is not reviewed on MathSciNet.