Has the mathematics research community ever been led astray by a dumb mistake?

Question

This is a highly subjective question, but here goes.

Has anyone ever published a result that was "taken seriously" by the research community, but was then discovered to be incorrect because of a "dumb mistake"?

Allow me to (somewhat) clarify my terms:

1. By a "dumb mistake", I mean "any mistake that is immediately obvious to any professional on the community upon being pointed out". Another requirement for the mistake to count as "dumb" is that there be nothing "mathematically interesting" or educational about it. Examples would include arithmetic errors, errors of elementary algebra, accidentally using one formula in place of another intended one, transcribing an equation incorrectly, or "obvious" bugs in computer code. In order to count, the mistake should be able to be described very succinctly. I would say that "a failure to make an 'easy' conceptual leap" would generally not count, although I acknowledge that all of this is pretty subjective.

2. "Taken seriously" is definitely pretty ambiguous, but I would say that the result "made a splash" in the research community. Sufficient evidence to demonstrate this could be, say, the paper making the incorrect claim receiving a large number of citations, or a significant number of subsequent papers relying on the claim, or a statement by a "major figure" in the field listing a major open problem that follows from the claim, etc. It doesn't count if the mistake was almost immediately pointed out by someone else (although what counts as "almost immediately" is also ambiguous). Basically, the rough threshold that I'm wondering about is that the mistake "did some nontrivial damage to the research process by leading subsequent researchers astray for a time", even it was ultimately corrected.

I'm not really looking for a paper that made a big controversial claim, and most experts assumed that it was probably wrong but took a little while to find the mistake. I'm more looking for a paper that was broadly accepted to be correct when it was published, and it "flew under the radar" for a bit before anyone realized that the claim was wrong.

Answer 1

In 1970, Irwing Noel Baker "proved" that a transcendental entire function of one complex variable can have at most one completely invariant component of the set of normality. In fact he "proved" a more general statement that there cannot be two disjoint domains whose preimages are connected. His paper occupies three pages, and contains an elementary mistake (a geometric statement which looks completely evident at the first glance). During the period 1970-2018, Baker's argument was used in several papers.

In 2018 I tried to explain Baker's argument to Julien Duval, he could not understand it, and finally he found that it is incorrect.

In the same year a counterexample to Baker's theorem was constructed, and the original question about completely invariant domains remains unsolved.

Answer 2

People in the comments have already pointed out Daniel Biss's thesis. But he also made two other unrelated mistakes that to my mind are more interesting:

1. In his paper,

   D. Biss & B. Farb, $\mathcal{K}_g$ is not finitely generated, Invent. Math. 163(1), 2006, 213-226.

   Biss and Farb claim to prove that the so-called Johnson kernel subgroup of the mapping class group is not finitely generated. People definitely took this seriously when it came out, and it had consequences for other kinds of finiteness results (see, for instance, the discussion in Farb's paper "Some problems on mapping class groups and moduli space").

   The mistake here was that they gave an algorithm for constructing a bunch of curves on a surface and claimed that they were disjoint. Later on, Masatoshi Sato (then a PhD student at the University of Tokyo) carefully drew pictures of these curves and verified that they were not disjoint. The proof fell apart. Even better, it later turned out that the Johnson kernel subgroup is finitely generated (which might have been discovered much sooner if the community was not misled by Biss-Farb's theorem). See

   M. Ershov & S. He, On finiteness properties of the Johnson filtrations, Duke Math. J. 167, 2018, 1713-1759.

   and

   T. Church, M. Ershov, & A. Putman, On finite generation of the Johnson filtrations, J. Eur. Math. Soc. 24, 2022, no.8, 2875-2914.

2. In his papers

   D. Biss, A generalized approach to the fundamental group, Amer. Math. Monthly 107 (2000), no. 8, 711-720.

   and

   D. Biss, The topological fundamental group and generalized covering spaces, Topology Appl. 124 (2002), no. 3, 355-371.

   Biss studied the fundamental groups of spaces that do not have universal covers. His main theorem says that there is a natural topology on these fundamental groups making them into topological groups. This has been fairly influential, e.g. they were part of his work cited for the Morgan prize, and the second paper has 97 citations on google scholar. However, he makes some elementary point-set topology mistakes that invalidate his proofs (totally different from the mistakes in his other papers!). In fact, for many easy examples the purported "topological group structure" he defined is not continuous. This was pointed out in

   P. Fabel, The topological Hawaiian earring group does not embed in the inverse limit of free groups, Algebr. Geom. Topol. 5 (2005), 1585-1587.

   For a very nice discussion of the history, I recommend the blog post here.

Answer 3

The claim that the convolution of unimodal distributions is unimodal appeared as Theorem 3 of §32 in the very influential book by Gnedenko and Kolmogorov (the Russian original of 1949 of the English translation "Limit Distributions for Sums of Independent Random Variables" of 1954).

As discussed in Appendix II of the English translation, this claim is incorrect. Two simple errors in the "proof" of this claim are indicated there. Of those two, at least the second one, based on an incorrect identity, certainly seems to qualify as "dumb".

The claim and the "proof" appear to have originated in Lapin, A. I. "On some properties of stable laws." Dissertation, in Russian, 1947.

It also appears that the first counterexample to this claim was given by Chung in 1953 (see p. 11 in "Unimodality, Convexity, and Applications" by Dharmadhikari and Joag-dev, 1987).

Answer 4

Hilbert problem 16, second part asks for an upper estimate of the number of limit cycles of a polynomial system of differential equations in dimension 2. For long time it was believed that Henri Dulac in 1923 proved that the number of limit cycles is finite.

Dulac studied the Poincare map in a neighborhood of a limit cycle and obtained an infinite asymptotic expansion for it. This derivation occupies a whole book (143 pages). Then he concluded at the very end: since this map is analytic on $(0,\delta]$, and has an infinite asymptotic expansion as $x\to 0$, it can have only finitely many fixed points, and thus the number of limit cycles is finite (cycles correspond to fixed points of the Poincare map). This is an elementary mistake since the map can be of the form $\phi(x)=x+g(x)$ where $g$ is flat (has zero asymptotic expansion), and thus we can conclude nothing about the number of fixed points near $0$.

This mistake was spotted only in 1980s (probably nobody had patience to read Dulac's book to the very end before), and since then at least two proofs of Dulac's theorem were published, both enormously long and complicated, and I am not sure how well any of them is verified.

Answer 5

The following is a short summary of the attacks on OCB2 in Cryptography. This is more applied than other answers, but still fits the question fairly well, and has the fun detail that we were only saved from what could have possibly been the most devastating practical failure of cryptography by a patent.

In Cryptography, a common proof technique is that of proofs by reduction. These are essentially "constructive" proofs of implications, which take the general form

Scheme $\Pi$ has property $P\implies$ scheme $F(\Pi)$ has property $Q$.

What "constructive" here means is the combination of

• Preserving Running Time: Any algorithm $A$ for breaking property $Q$ of $F(\Pi)$ may be transformed into an algorithm $A'$ for breaking property $P$ of $\Pi$, such that the running times of $A$ and $A'$ are similar, and
• Preserving Advantage: For $A$ and $A'$ as above, if $A$ only breaks property $Q$ with some probability $p$, then $A'$ breaks property $P$ with some probability $p'$ not too much smaller than $p$.

Reductions are perhaps the core proof technique in cryptography. Relevant to the discussion below is their usage in symmetric encryption. Here, one starts with

• a block cipher $\Pi$, roughly encryption for "fixed-length" (e.g. 128 bits) inputs, which
• satisfies a weak security property $P$ (typically "Indistinguishabillity under Chosen Plaintext Attack", or IND-CPA. At a high level, this models passive adversaries, which may choose messages $m$ to be encrypted, and view the resulting ciphertexts that are transmitted, but may not modify these ciphertexts in any way),

and produces

• a general encryption scheme $F(\Pi)$, which may encrypt messages of arbitrary (not a priori bounded) length, which

• satisfies a strong security property $Q$ (for example, in IND-CCA2, one allows an adversary to arbitrarily modify the aformentioned ciphertexts).

For symmetric encryption, the above are typically called "block cipher modes of operation". One often needs not only a transformation $F$ that takes as input a block cipher $\Pi$, but additionally an auxiliary scheme $\Gamma$ called a "Message Authentication Code". I won't discuss this detail further though.

There are many popular modes of operation. The OCB modes have similar performance to "classical" modes, while achieving a stronger security property known as authenticated encryption with associated data (AEAD). An AEAD cipher takes as input two messages $(m, d)$, where

1. the message $m$ is kept private (along the lines of IND-CCA2 mentioned above), while
2. the data $d$ is publicly transmitted, but "authenticated", so an adversary who attempts to modify this (public) data will be caught.

This can be useful when messages $m$ require public components for processing. For example, the data $d$ may contain the ID of which key should be used to decrypt the message. Such data must be public, but should also not be able to be tampered with.

Over roughly a decade, Phil Rogaway developed the OCB Modes (OCB1, OCB2, OCB3) of operation. They have been very popular. OCB2 was an ISO standard (until the attack referred to below), and (a variant of) OCB3 continues to be one (only OCB2 is known to have a flawed security proof). OCB3 was selected as a "winner" of the CAESAR competition (a competition run by cryptographers to select a portfolio of cryptographic recommendations for future usage by standardization committees). These schemes all had reduction-based proofs of security. This doesn't mean that the schemes are secure, but it should mean that the only "real" way to attack them is to attack their base components $\Pi$, rather than their transformation $F(\Pi)$.

In 2018, Inoue and Minematsu found a simple attack on OCB2, that attacked the transformation $F$ rather than the components $\Pi$. This is to say that Rogaway's proof was wrong. The way it was wrong may be described simply in a high-level way. Roughly speaking (and perhaps in a post-hoc way, with knowledge of the attack), for OCB2 one may split $F = G\circ H$ into two parts.

1. First, one shows that $H(\Pi)$ produces what is known as a tweakable block cipher from a block cipher $\Pi$, and
2. Then, one shows that $G(\Pi')$ maps a tagged tweakable block cipher to an AEAD scheme.

This is to say that there was a small mismatch between the post-condition of $H$ and pre-condition of $G$. This small gap lead to a complete failure in the proof, in the sense that OCB2 admitted efficient, near-complete breaks in security, even for secure "base" schemes $\Pi$. This was missed by several standardization bodies, and many cryptographers (as of 2024, the OCB2 paper has nearly 600 citations).

---

As mentioned previously, this attack thankfully didn't have much practical impact. Despite OCB2 having many practical benefits compared to competing schemes, Rogaway patented the scheme, and had various licenses for it, namely

1. Free usage in open-source software,
2. Paid licenses available in commercial non-military software.

This was not enough for cryptographic library authors (see some discussion here) to widely adopt the OCB modes. So, the presence of the patents deterred adoptions of OCB modes, which saved us from OCB2 failing — despite its security proof.

Answer 6

I do not resist to mention an extra-mathematical case of the situation you are interested in. From Wikipedia article on Chromosome (the bold characters are mine):

The number of human chromosomes was published in 1923 by Theophilus Painter. By inspection through the microscope, he counted twenty-four pairs, which would mean forty-eight chromosomes. His error was copied by others and it was not until 1956 that the true number, forty-six, was determined by Indonesian-born cytogeneticist Joe Hin Tjio.

Answer 7

I'm surprised no one has yet mentioned a mathematical object sufficiently mainstream that it has its own Wikipedia page: Analytic sets

Wikipedia doesn't mention it, but the existence of these sets flew under the radar for a while; Lebesgue published a paper in which he claimed that the projection onto a line of the intersection of a family of plane sets is the intersection of their projections. (Hoare, G. T. Q., and N. J. Lord. "‘Intégrate, longueur, aire’the centenary of the Lebesgue integral." The Mathematical Gazette 86.505 (2002): 3-27.)

This is clearly false! Yet it took a few years until it was noticed; in June 1930, Lusin finally published his Leçons sur les ensembles analytiques et leurs applications in which he developed the theory of such sets. Lebesgue himself wrote the preface to the book in which he mentioned his "fruitful error":

A la réflexion, une Préface m'a semblé être le seul endroit où je pourrais avouer très haut ce que M. Lusin a soigneusement caché: l'origien de tous les problemès dont it va s'agir ici est une grossiere erreur de mon Memoire sur les fonctions représentables analytiquement. Fructueuse erreur, que je fus bien inspiré de la commettre!

[Translation: On reflection, a Preface seemed to me to be the only place where I could confess clearly what Mr. Lusin has carefully hidden: the origin of all the problems that will be discussed here is a gross error in my Memoir on analytically representable functions. A fruitful error, which I was well inspired to commit!]

(Unfortunately, I can't find a link to the original article in which Lebesgue claimed this since the above article refers only to his Oeuvres scientifiques, and a full bibliography seems to be on the internet, only fragments. I looked around, but I wasn't able to find it, neither on gallica.bnf.fr, nor on MathSciNet. Strangely, the article Who discovered analytic sets on Wikipedia mentions none of these facts, which seem important pieces in their discovery.)

Answer 8

I think that Jules Drach's nonlinear differential Galois theory is an example. It was published as his thesis: "Essai sur une théorie générale de l'intégration et sur la classification des transcendantes", Ann. Sci de l'École Norm. Sup., Serie 3, 15 (1898), pp. 243-384.

Drach continued to publish on this subject, and gave invited lectures to the International Congress of Mathematicians at Cambridge in 1912 and also at Toronto in 1924.

So I think it's safe to say that Drach's work was indeed taken seriously.

However, a major mistake was discovered by Vessiot not long after the publication of the thesis. In brief, the key construct - the "rationality group" - need not exist. In a letter of 3rd October 1898 to Drach, Vessiot went so far as to say that all the proofs must be given anew: "Et dans tous les cas, toutes les démonstrations me paraissent a créer a peu près de toutes pièces."

Was it a "dumb mistake"? Well, in a letter to Vessiot a few weeks later, Painlevé remarked that anybody who thought about it for five minutes would recognise the problem.

As far as I can discover, Drach's only published response to all this was a footnote in his 1912 ICM lecture acknowledging that in 1898 "Messrs. Painlevé and Vessiot ... had kindly drawn my attention to the ambiguity of certain statements."

• The letter from Vessiot to Drach, and from Painlevé to Vessiot are reproduced and translated into English in Pommaret's book Lie Groups and Mechanics,Gordon & Breach 1988.
• A. Cogliati in "Early history of infinite continuous groups, 1883–1898", Historia Mathematica 41 (2014) pp. 291-332 states that the topic of the Grand Prix des Sciences Mathématiques for 1902 was selected with Drach's thesis (and its issues) in mind.
  He quotes in the verdict of the jury (H. Poincaré, C. Jordan, P. Painlevé, É. Picard and P. Appel) that Vessiot's prize winning paper "fully relieves the lacunae which still afflicted the theory inaugurated by Drach on linear partial differential equations."

Answer 9

Alfred Kempe published a "proof" of the four colour conjecture in 1879. Another "proof" was published by Peter Guthrie Tait in 1880. Percy Heawood debunked Kempe's in 1890, and Julius Petersen debunked Tait's in 1891. But I'm not sure the mistakes count as "dumb".

Answer 10

"Growth in a Time of Debt." Reinhart, Carmen M., and Kenneth S. Rogoff. American economic review 100.2 (2010): 573-578.

A short article in economics, it now has nearly 5500 citations and is notable enough for its own wikipedia page. The article examines the association between national debt and growth of GDP across several countries, and its most important conclusion is that

When gross external debt reaches 60 percent of GDP, annual growth declines by about two percent; for levels of external debt in excess of 90 percent of GDP, growth rates are roughly cut in half.

When restricted to post-WWII economies, their figure 2 claims even further that GDP growth becomes negative on average if debt exceeds 90%. The result was taken seriously by many economists and politicians and was directly referenced by conservative economic policies in the USA, UK, and EU. They used the Reinhart and Rogoff claim to support spending cuts so that the "deadly" 90% debt ratio could be avoided.

In "Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff." Herndon, Thomas, Michael Ash, and Robert Pollin. Cambridge journal of economics 38.2 (2014): 257-279, it was shown that the Reinhart-Rogoff paper overestimated the growth dependence on debt ratio by a 2.5x factor and despite its short length contained numerous statistical errors. The "dumbest" of these was that when constructing the figures for their main result, the authors had misclicked and inadvertently failed to select a large percentage of the countries and years in their excel spreadsheet. The spreadsheet had not been included in the initial publication but was provided to the researchers by special request.

There's subjectivity regarding whether economics is considered a part of the "mathematics research community", but it's inarguable that a 6-page article which dramatically affected economic policy of multiple world powers over the course of three years due to a Microsoft Excel misclick is exceedingly dumb.