Recently a ''bug'' was discovered in one of the most popular mathematics software, Wolfram Mathematica (see links here and here). It concerns the evaluation of the sum $$ \sum_{k=1}^{n-1} \frac{(-1)^{k-1}(k-1)!^2}{(n^2-1^2)\ldots(n^2-k^2)}, $$ a fairly straightforward computational exercise, as one would expect. Surprisingly, Mathematica incorrectly evaluates this sum to $$ \frac{1}{n^2}, $$ instead of the correct expression, which is $$\frac{1}{n^2}-\frac{2(-1)^{n-1}}{n^2 \binom{2n}{n}}.$$

Another user (in second link above) found that Maple 2020 *also* makes the same incorrect evaluation.

This raises the question whether we can trust widely used software like Mathematica and Maple with (much) more complex computational tasks, and in particular theorems and lemmas that appear in published literature that explicitly rely on large scale computations performed with such applications.

In some cases, the peer review process involves replicating computational results that appear in a manuscript, but this is (more often than not) not the case. Furthermore, it is not unlikely that reviewers will use the same software to double check these results as the author, thereby replicating the same mistake, or bug.

To what extent can we trust results that were obtained with the aid of extensive computations? At what point can we safely accept the ''truthfulness'' of a claim if its replication requires months (sometimes years) of number crunching performed by software that can make such elementary mistakes as above?

leastlikely to contain errors. The odds of hitting an error like this one and not noticing it numerically is much much lower than the odds of my screwing up a proof. $\endgroup$ – Noah Snyder Dec 12 '20 at 15:44