A post appeared yesterday (9/22/2020) entitled ``Fear of publishing a wrong result'' on the Mathematics Stack Exchange.


It made me think of the following--- What if an incorrect proof is published?''; or more poignantly, What if one discovers an incorrect proof to a longstanding result?''

Naturally, we might think to contact the editor of that journal and have them publish either a retraction of the paper or, if a corrected proof is supplied, publish that. But what if the journal is not amenable to that suggestion? Well, one may try to incorporate it in a natural fashion in one's own paper and submit it for publication? But what if no success is realized via that route?

For what it's worth, some years back I was studying the Ph.D. thesis of D. H. Lehmer in which, among other things, he presented what later would become known as the ``Lucas-Lehmer Test'' for determining the prime/composite character of a Mersenne number.

Alas, the proof Mr. Lehmer provided had a flaw in it. (Before I go any further, let me say that the flaw is correctable; and so, the Lucas-Lehmer Test is indeed a valid result!)

If I may, I ask that if anyone has discovered a flaw in a published proof, then please make us aware of the error, and what resolution (if any) came about. I think this is rather important because many results are quoted to prove other results, which in turn, are quoted for the same purpose. To cite a result that inherently has a flaw in its proof results in an incorrect proof. If the flaw is not correctable, then the result is incorrect. (A list a journal's errata updated and appearing at the end of each issue would be a start anyway in rectifying the oversights of the referees. But I am aware of no journal that does this.)

For now, if anybody is interested, the following identifies the error which appears in the proof of Theorem 5.1 (If N $\pm$ 1 is the rank of apparition of N, then N is a prime) found on page $442$ of ``An extended theory of Lucas' functions,'' Ann. Math., 31 (1930), 419--448.---

Consider first (if you are not familiar with the Lehmer sequences) some preliminaries.


Let $R$, and $Q$ be nonzero relatively prime integers. The Lehmer sequences are defined as

\begin{equation} U_{n+2}(\sqrt{R}, Q) = \sqrt{R}U_{n+1} - QU_{n}, \,\,\, U_{0} = 0, \,\,\, U_{1} = 1, \,\,\,\,\, n \in \{0, 1, \ldots\}. \,\,\,\,\,\,\,\,\,\, (1) \end{equation}

We define the rank of apparition of a prime $p$ in $\{U_{n}(\sqrt{R}, Q)\}$ to be the index of the first term that contains $p$ as a divisor and denote this by $\omega(p)$. We also let $\prod_{i=1}^{\nu} {p_{i}}^{\alpha_{i}}$ represent the prime factorization of an arbitrary odd integer $m$, $\Delta = R - 4Q$ be the discriminant of the characteristic equation of (1), and $\left(\frac{R\Delta}{p_{i}}\right)$ represent either the Legendre symbol or zero depending upon whether $p_{i} \mid R\Delta$ or not.

In his 1930 paper, Lehmer introduced the following generalization of Euler's totient function.

\begin{equation} T(m) = 2 \prod_{i=1}^{\nu} {p_{i}}^{\alpha_{i} - 1}\left\{p_{i} - \left(\frac{R\Delta}{p_{i}}\right)\right\} \,\,\,\,\,\,\,\,\,\, (2) \end{equation}

(Lehmer's Theorem 1.12) , Let $\{U_{n} (\sqrt{R}, Q)\}$ be an extended Lucas sequence and suppose that $N$ and $Q$ are relatively prime integers. Then, $U_{T(N)} \equiv 0 \pmod{N}$.

Remark: The following proposition is Lehmer's Theorem 5.1. Later in the paper, Lehmer uses it in his proof of what we know as the ``Lucas-Lehmer test.'' However, the proof of Theorem contains a flaw. It should also be noted that Lehmer's Theorem 5.1 is found in later literature, such as in H. C. Williams' 1998 book, Edouard Lucas and primality testing. However, in it, the generalized totient function described by (2) is introduced and the proof of Lehmer's Theorem 5.1 is indicated (alas) to follow from earlier results.

And so, after the identification of the error, a proof of Lehmer's Theorem 5.1 that does not depend on (2) is provided here.

(Lehmer's Theorem 5.1) Let $N$ and $2\Delta RQ$ be relatively prime and $N \pm 1$ be the rank of apparition of $N$. Then, $N$ is a prime.

An error in the proof of Theorem 5.1.

Lehmer's demonstration of Theorem 5.1 begins with the assumption that $N = \prod_{i=1}^{\nu} {p_{i}}^{\alpha_{i}}$ is composite. He observes from (2) that $T(N) \neq N \pm 1$, and then concludes by Lemma 1 that $U_\frac{T(N)}{2} \equiv 0 \pmod{N}$. In light of the hypothesis that the rank of apparition of $N$ is $N \pm 1$, Lehmer asserts that $U_{N \pm 1} \equiv 0 \pmod{N}$. This implies that $U_{\frac{T(N)}{2} - (N \pm 1)} \equiv 0 \pmod{N}$. At this point Lehmer remarks, ``$\left[p_{i} - \left(\frac{R\Delta}{p_{i}}\right)\right] \leq p_{i} + 1$, and so, $\frac{1}{2}T(N) < 2N$.'' The statement, however, is incorrect, as it says that for each $p_{i}$,

\begin{equation} \prod_{i=1}^{\nu} \left\{\frac{p_{i} - \left(\frac{R\Delta}{p_{i}}\right)}{p_{i}}\right\} \,\, < \,\, \prod_{i=1}^{\nu} \left\{\frac{p_{i} + 1}{p_{i}}\right\} \,\, < \,\, 2. \,\,\,\,\,\,\,\,\,\, (3) \end{equation}

To show that (3) is not always true, let's consider the example, $p_{1} = 3$, $p_{2} = 5$, $p_{3} = 7$, $p_{4} = 11$, and $p_{5} = 13$ under the assumption that all of the Legendre symbols are equal to -1. Hence, we have

\begin{equation} \prod_{i=1}^{5} \left\{\frac{p_{i} - \left(\frac{R\Delta}{p_{i}}\right)}{p_{i}}\right\} \, = \, \prod_{i=1}^{\nu} \left\{\frac{p_{i} + 1}{p_{i}}\right\} \, = \, \frac{4 \cdot 6 \cdot 8 \cdot 12 \cdot 14}{3 \cdot 5 \cdot 7 \cdot 11 \cdot 13} \, > \, 2. \end{equation}

The proof of Lehmer's Theorem 5.1 in his 1930 paper ends with the conclusion that $T(N) > N \pm 1$, which produces the contradiction $0 < \frac{1}{2}T(N) - (N \pm 1) < N \pm 1$. In other words, $U_{\frac{1}{2}T(N) - (N \pm 1)}$ is a term of $\{U_{n}(\sqrt{R}, Q)\}$ that contains $N$ as a divisor but occurs earlier in the sequence than $U_{N \pm 1}$ contrary to the assumption that the rank of apparition of $N$ is $N \pm 1$.

A corrected proof for Theorem 5.1

In Lehmer's Theorem 5.1, $N$ and $2\Delta RQ$ are relatively prime integers. So, it is without loss of generality that we let $N$ be an odd positive integer. The corrected demonstration of Theorem 5.1 follows will follow an immediate consequence of Theorem 2, the proof of which is an adaptation of an argument that appears in Riesel's 1994 book, Prime Numbers and Methods for Factorization for its Theorem 4.8, which asserts that $N$ is prime provided that $\prod_{i=1}^{k} {q_{i}}^{\alpha_{i}}$ is the prime factorization of $N + 1$, there exists a Lucas sequence $\{U_{n}(P, Q)\}$ such that $GCD\left(U_{\frac{N+1}{q_{i}}}, N\right) = 1$, and $U_{N+1} \equiv 0 \pmod{N}$.

Consider the following theorem.

Theorem 2 , Let $\nu \geq 1$ and $N = q_{1}^{\alpha_{1}}q_{2}^{\alpha_{2}} \cdots q_{\nu}^{\alpha_{\nu}}$ be composite, odd, and relatively prime to $RQ\Delta$, where for $i = 1, 2, \ldots$, the $q_{i}$ are distinct primes and $q_{i-1} < q_{i}$. Then, $\omega(N) < N - 1$.

Proof of Theorem 2. , First, let $\nu = 1$. Since $N$ is composite, then $\alpha_{1} > 1$. Furthermore, as $q_{1} \geq 3$, we have

\begin{equation} N - \omega(N) \,\, \geq \,\, N - (q_{1} + 1) \,\, \geq \,\, q_{1}^{2} - q_{1} + 1 \,\, > \,\, (q_{1} - 1)^{2} \,\, \geq \,\, 4 \,\, > \,\, 1. \end{equation}

Let $\nu > 1$ and $\hat{N} = q_{1}q_{2} \cdots q_{\nu}$. Since $\forall i: 1 \leq i \leq \nu$, $q_{i} \pm 1$ is even and $\omega(q_{i}) \mid (q_{i} \pm 1)$,

\begin{equation} \omega(N) \,\, \leq \,\, LCM\{\omega(q_{i}^{\alpha_{i}})\}^{\nu}_{i=1} \,\, \leq \,\, \frac{2\left(N / \hat{N}\right)(q_{1} + 1)(q_{2} + 1) \cdots (q_{\nu} + 1)}{2^{\nu}}. \end{equation}


\begin{equation} N - \omega(N) \,\, \geq \,\, \left(\frac{N}{\hat{N}}\right)\left[\hat{N} - \frac{(q_{1} + 1)(q_{2} + 2) \cdots (q_{\nu} + 1)}{2^{\nu-1}}\right] \end{equation}

\begin{equation} \hskip 71pt = \,\, N\left[1 - \left(\frac{1}{\hat{N}}\right)\frac{\left(q_{1} + 1\right)\left(q_{2} + 1\right) \cdots \left(q_{\nu} + 1\right)}{2^{\nu-1}}\right] \end{equation}

\begin{equation} \hskip 60pt = \,\, N\left[1 - \frac{\left(1 + \frac{1}{q_{1}}\right)\left(1 + \frac{1}{q_{2}}\right) \cdots \left(1 + \frac{1}{q_{\nu}}\right)}{2^{\nu-1}}\right] \end{equation}

\begin{equation} \hskip -20pt \geq \,\, N\left[1 - \left(\frac{4}{3}\right)\left(\frac{6}{5}\right) \left(\frac{1}{2}\right)\right] \end{equation}

\begin{equation} \hskip -45pt = \,\, N\left(\frac{1}{5}\right) \,\, \geq \,\, \frac{6}{5} \,\, > \,\, 1. \end{equation}

Therefore, $\omega(N) \, < \, N - 1$. $\blacksquare$

Has anyone encountered similar instances in journal readings?

  • 5
    $\begingroup$ If you are asking us to embarrass ourselves by listing our mistakes which "no one wants to rectify", that's unpleasant and not very nice. And if you're asking us to embarrass other people by publicizing their mistakes, that's quite a bit worse. It's bad enough to proclaim mistakes; but ones that "no one wants to rectify", that seems almost cruel. I don't think this question can lead to good things. $\endgroup$ – Zach Teitler Sep 24 at 2:37
  • 5
    $\begingroup$ I do not see a mathematical question here. If you have found and figured out how to rectify a significant mistake in a significant published paper, then you should write it up as a paper, submit it to the arXiv, and then submit it to a journal. Such things get published all the time. $\endgroup$ – Steven Landsburg Sep 24 at 2:48
  • 3
    $\begingroup$ This is not in my area, and I have made no attempt to check this, but it strikes me as overwhelmingly against the odds that an error in the proof of a major result would have gone unnoticed for 90 years! $\endgroup$ – Michael Renardy Sep 24 at 3:04
  • 2
    $\begingroup$ "Has anyone discovered a flaw in a published proof and no one wants to rectify the situation?" How would you know whether anyone wants to rectify the situation? You present a hypothetical situation in which a journal refuses to correct the flaw, but no evidence that this has happened – and even in this hypothetical case, it is evident that someone did want to rectify the situation, namely, the person who wrote to the journal to explain the flaw. You ask, "Has anyone encountered similar instances in journal readings?" but it's not clear what you mean (continued, next comment). $\endgroup$ – Gerry Myerson Sep 24 at 6:10
  • 2
    $\begingroup$ Do you mean, has anyone encountered an uncorrected error in a published paper? Or do you mean to imply that there has been a refusal to attend to an error in Lehmer's paper (though support for that position is not in evidence) and that is the kind of instance you are looking for? $\endgroup$ – Gerry Myerson Sep 24 at 6:14