Examples of "unsuccessful" theories with afterlives I am looking for examples of mathematical theories which were introduced with a certain goal in mind, and which failed to achieved this goal, but which nevertheless developed on their own and continued to be studied for other reasons.
Here is a prominent example I know of:


*

*Lie theory: It is my understanding that Lie introduced Lie groups with the idea that they would help in solving differential equations (I guess, by consideration of the symmetries of these equations). While symmetry techniques for differential equations to some extent continue to be studied (see differential Galois theory), they remain far from the mainstream of DE research. But of course Lie theory is nonetheless now seen as a central topic in mathematics.


Are there some other examples along these lines?
 A: Ronald Fisher's theory of fiducial inference was introduced around 1930 or so (I think?), for the purpose of solving the Behrens–Fisher problem. It turned out that fiducial intervals for that problem did not have constant coverage rates, or in what then came to be standard terminology, they are not confidence intervals. That's not necessarily fatal in some contexts, since Bayesian credible intervals don't have constant coverage rates, but everyone understands that there are good reasons for that. Fisher wrote a paper saying that that criticism is unconvincing, and I wonder if anyone understands what Fisher was trying to say. Fisher was brilliant but irascible. (He was a very prolific author of research papers in statistical theory and in population genetics, a science of which he was one of the three major founders. I think he may have single-handedly founded the theory of design of experiments, but I'm not sure about that.) 
However, fiducial methods seem to be undergoing some sort of revival:
https://statistics.fas.harvard.edu/event/4th-bayesian-fiducial-and-frequentist-conference-bff4
A: I quote at length from the Wikipedia essay on the history of knot theory: 

In 1867 after observing Scottish physicist Peter Tait's experiments involving smoke rings, Thomson came to the idea that atoms were knots of swirling vortices in the æther. Chemical elements would thus correspond to knots and links. Tait's experiments were inspired by a paper of Helmholtz's on vortex-rings in incompressible fluids. Thomson and Tait believed that an understanding and classification of all possible knots would explain why atoms absorb and emit light at only the discrete wavelengths that they do. For example, Thomson thought that sodium could be the Hopf link due to its two lines of spectra.
Tait subsequently began listing unique knots in the belief that he was creating a table of elements. He formulated what are now known as the Tait conjectures on alternating knots. (The conjectures were proved in the 1990s.) Tait's knot tables were subsequently improved upon by C. N. Little and Thomas Kirkman.
James Clerk Maxwell, a colleague and friend of Thomson's and Tait's, also developed a strong interest in knots. Maxwell studied Listing's work on knots. He re-interpreted Gauss' linking integral in terms of electromagnetic theory. In his formulation, the integral represented the work done by a charged particle moving along one component of the link under the influence of the magnetic field generated by an electric current along the other component. Maxwell also continued the study of smoke rings by considering three interacting rings.
When the luminiferous æther was not detected in the Michelson–Morley experiment, vortex theory became completely obsolete, and knot theory ceased to be of great scientific interest. Modern physics demonstrates that the discrete wavelengths depend on quantum energy levels.

A: Continuing what was said by @GerryMyerson, the project of providing foundations for mathematics started by Frege was presented in a treatise called Grundgesetze der Arithmetik (Basic laws of arithmetic). The axioms of this treatise were proven inconsistent by Bertrand Russell in what we know today as Russell's paradox.
This paradox also affects naive set theory, understood as the theory comprising the following two axioms:


*

*Axiom of extensionality: $(x \in a \leftrightarrow x \in b) \rightarrow a = b$. That is, if two sets $a$ and $b$ have the same elements, then they're the same set.

*Axiom (scheme) of unrestricted comprehension: $x \in a \leftrightarrow$ P$x$, for each formula P$x$. That is, to each property P uniquely corresponds one set $a$.
Naive set theory, thus understood, follows from Frege's axioms and seems to capture very well the notion of set. But since $x \notin x$ is a formula, the axiom scheme of unrestricted comprehension guarantees that the following is an axiom:


*$x \in x \leftrightarrow x \notin x$
Now, when we ask whether $x \in x$ or $x \notin x$, we obtain contradictory situations in both cases.
This paradox was solved by discarding this axiomatisation of set theory and, hence, Frege's axiomatics. But some logicians, mathematicians and philosophers have considered that perhaps this wasn't the right way to solve this. Instead of rejecting this naive set theory or Frege's theory, they propose to reject the principle of explosion or ex contradictione sequitor quodlibet:


*$P \wedge \neg P \rightarrow Q$. That is, from a contradiction follows any formula or statement.


This research programme is often known as the paraconsistent programme, because they work with paraconsistent logics. A logic system is said to be paraconsistent iff the logical thesis (4) is not valid in general. Hence, if the theory is inconsistent, it doesn't mean that anything follows from it (which means that it still may be useful). You can find out more about his programme in:


*

*SEP: Paraconsistent Logic

*SEP: Inconsistent Mathematics

*SEP: Dialetheism
You will find there (specially in the second link) a whole programme for researching inconsistent mathematical theories, which are generally considered of no mathematical interest. (You will also find that mainly philosophers are working in this programme.)
Whether this programme is of any scientific value, that's for you to judge. But I accept this probably wasn't the kind of answer you were looking for. There is a chance, however, that you find it very interesting. I hope it helps in any case.
A: It's perhaps slightly (if any) exaggerated, but the development of algebraic number theory (particularly the study of cyclotomic fields) is strongly motivated by attempts to prove Fermat's last theorem.
And everyone knows the end of that story ...
To quote Wiki:
Fermat's last theorem:

The unsolved problem stimulated the development of algebraic number theory in the 19th century and the proof of the modularity theorem in the 20th century.

Cyclotimic field:

The cyclotomic fields played a crucial role in the development of modern algebra and number theory because of their relation with Fermat's last theorem. It was in the process of his deep investigations of the arithmetic of these fields (for prime n) – and more precisely, because of the failure of unique factorization in their rings of integers – that Ernst Kummer first introduced the concept of an ideal number and proved his celebrated congruences.

A: I think Dirac's equation is a good example. Dirac was looking for a special-relativistic version of Schrödinger's equation. For the probabilistic interpretation to work, it had to have only first-order time derivatives, unlike the field equations known at the time.
He found a Lorentz invariant field equation with first-order derivatives, and it turned out to have enormous theoretical importance since it kicked off the study of relativistic field theories and Lie group representations in physics.
But the Dirac equation isn't a relativistic version of Schrödinger's equation. It can't describe multiparticle entanglement, it doesn't violate Bell's inequality, you can't build a quantum computer in it, etc. From a modern perspective it's just the massive, spin-½ counterpart to Maxwell's equations.
A version of the Dirac equation appears in the Lagrangian of quantum electrodynamics and the Standard Model. But it's right alongside a version of Maxwell's equations, complete with second-order derivatives, which turned out not to be a problem after all.
It's often still taught in introductory courses that Dirac's equation explained the electron's spin and magnetic moment, but both of those retrodictions were essentially accidental. Dirac's argument for spin ½ would imply that all fundamental particles must have half-integer spin, which doesn't appear to be the case; and Weinberg says "there is really nothing in Dirac's line of argument that leads unequivocally to this particular value for the magnetic moment" (The Quantum Theory of Fields, Vol. 1, p. 14).
A: "The modern study of knots grew out an attempt by three 19th-century Scottish
physicists to apply knot theory to fundamental questions about the universe".
A: E.H. Moore's General Analysis was to be a unifying framework for (at least) analysis.  This goal was never achieved, in part due to the rather complicated formalism used by Moore.
Nonetheless, a part of that theory, called Moore-Smith sequences or nets, survived and even thrived as a way of describing topology where it is known that sequences do not suffice.
A: Motives and the standard conjectures were developed by Grothendieck to prove the last of the Weil conjectures. They failed at this as none of the standard conjectures were proven - despite some progress on this, I would say we are not closer to proving the Weil conjectures via the standard conjectures today than we were when they were first formulated - and Deligne showed that Grothendieck's earlier invention of etale cohomology was perfectly sufficient to prove the Weil conjectures.
However, since that time different notions of motive were constructed, with different useful properties, in addition to Grothendieck's, and many of them have found applications in areas of algebraic geometry and number theory, with the first really big one being Voevodsky's Fields medal-winning proof of the Milnor conjecture.
A: String Theory!
String Theory was born in the context of strong interactions inside atomic nuclei, since the 60s-70s. The theory turned out not suited to describe the strong force, and was supplanted around 1973 by the rising Quantum Chromodynamics (our current best model for the strong force interactions). Among the reasons for the failure, there was the mandatory presence of unwanted spin 2 particles...
Those particles are now interpreted as gravitons! And String Theory is now seen as a theory of quantum-gravity, describing all the known forces (electromagnetic, weak, strong) and gravity at the same time! That’s a pretty big afterlife!
A: (Converted from a comment to an answer as requested.)
Non-Euclidean geometry was initially developed in hopes of deriving the parallel postulate from the other axioms of Euclidean geometry, as can be seen in particular through the pioneering work of Saccheri in this area, who tried in vain to prove the parallel postulate by contradiction and ended up proving a large number of foundational results in what we would now call elliptic and hyperbolic geometry as a consequence. (See for instance this article of Fitzpatrick, or this McTutor article on Non-Euclidean geometry.)
Nowadays, the classical non-Euclidean geometries (the elliptic geometry of the sphere, and the hyperbolic geometry of hyperbolic space) play the important role of describing two of the basic model geometries in Riemannian geometry, namely the simply connected geometries of constant and isotropic positive or negative curvature respectively.  (In two dimensions, where Riemann curvature is effectively a scalar quantity, these two geometries, together with Euclidean geometry, are the only models needed; in higher dimensions there are however other model geometries of interest, such as the remaining five Thurston geometries of the geometrisation conjecture in three dimensions.)
A: This is a copy of a copy of some history of the origins of free probability by Dan Voiculescu extracted from a response by Roland Speicher, a developer of the field, to an MO-Q:
This is from his  article "Background and Outlook" in the Lectures Notes
"Free Probability and Operator Algebras", see
http://www.ems-ph.org/books/book.php?proj_nr=208

Just before starting in this new direction, I had worked with Mihai Pimsner,
  computing the K-theory of the reduced $C^*$-algebras of free groups. From the
  K-theory work I had acquired a taste for operator algebras associated with free
  groups and I became interested in a famous problem about the von Neumann
  algebras $L(\mathbb{F}_n)$ generated by the left regular representations of free groups,
  which appears in Kadison's Baton-Rouge problem list. The problem, which
  may have already been known to Murray and von Neumann, is:are $L(\mathbb{F}_m)$ and $L(\mathbb{F}_n)$ non-isomorphic if $m \not= n$?
This is still an open problem. Fortunately, after trying in vain to solve it,
  I realized it was time to be more humble and to ask: is there anything I can
  do, which may be useful in connection with this problem? Since I had come
  across computations of norms and spectra of certain convolution operators on
  free groups (i.e., elements of $L(\mathbb{F}_n)$), I thought of finding ways to streamline
  some of these computations and perhaps be able to compute more complicated
  examples. This, of course, meant computing expectations of powers of such
  operators with respect to the von Neumann trace-state $\tau(T) = \langle T e_e,e_e\rangle$, $e_g$
  being the canonical basis of the $l^2$-space.
The key remark I made was that if $T_1$, $T_2$ are convolution operators on $\mathbb{F}_m$
  and $\mathbb{F}_n$ then the operator on $\mathbb{F}_{m+n} = \mathbb{F}_m \ast \mathbb{F}_n$ which is $T_1 + T_2$, has moments $\tau((T_1 + T_2)^p)$ which depend only on the moments $\tau(T_j^k)$, $j = 1, 2$ , but not
  on the actual $T_1$ and $T_2$. This was like the addition of independent random
  variables, only classical independence had to be replaced by a notion of free
  independence, which led to a free central limit theorem, a free analogue of
  the Gaussian functor, free convolution, an abstract existence theorem for one
  variable free cumulants, etc.

A: The chromatic polynomial of a graph was originally introduced as part of an attempt to prove the four-color conjecture (now a theorem), but was unsuccessful in that goal.  However, the chromatic polynomial continues to be studied to this day as an interesting algebraic invariant of a graph.
A: Multiplication of quaternions was introduced for use in physics for purposes for which cross-products of vectors came to be used and have been used ever since.
But today quaternions are used in computer graphics. I suspect they also have other applications.
A: Logic and set theory were developed by Frege, Russell and Whitehead, Hilbert and others in the late 19th, early 20th centuries with the goal of providing a firm foundation for all of Mathematics. In this they failed miserably, but nevertheless they have continued to develop and to be studied for other reasons. 
A: Gauge-theory might be another example at the border to physics. The original idea of deriving physics from gauge-symmetries and indeed the use of the term/prefix "gauge-" (in German "Eich-") itself goes back a paper by Hermann Weyl in 1919 ("Eine neue Erweiterung der Relativitätstheorie"). In this paper he somehow tried to unify electrodynamics and general relativity using this approach, by postulating that the notion of scale (or "gauge") might be a local symmetry. This of course was a total failure as it contradicted several experiments.
It was only about a decade later that he and some others picked up the idea again, applied it to electromagnetism and quantum physics (this time with phase as a gauge) and made it work. And then of course in 1954 there came Yang and Mills and now Weyl's "failed idea" is at the core of the Standard model of physics. However the original goal of adding general relativity to the mix still hasn't been achieved.
A: The typical oracle methods of Computability theory AKA Recursion theory were shown to be insufficient to settle the P vs. NP problem by Baker, Gill and Solovay 1975.
Thus recursion theory became divorced from the problems of efficient computability and experienced a bit of a setback (not as many papers in Ann.Math. anymore etc.). 
Nevertheless it continued as the study of in principle computability. 
A: There is the ramified type theory of Bertrand and Russell, which at the time was seen as a great step forward and which is why they named it as the Principia Mathematica, hoping to set mathematics on the same solid foundations as Newton did with his Principia for natural philosophy, that is physics as it would be described now.
Its only with the flowering of a new field, computer science, did type theory come into its own ...
There is also Maxwells Theory, which as recognised was an enormous achievement, in particular by Heaviside, but was generally ignored, until a couple of decades later.
