How to write computer-assisted mathematics well?

Question

Much has been said about writting good papers in mathematics. A short google query yields countless sources of advice. This skill also appears to be quite transferrable between basic branches of mathematics: a well-written paper in analysis follows the same basic principles as a well-written paper in algebra, etc.

Recently, I found my first (hopefully) publishable result that uses the help of a computer, and - to my embarassment - realised that I have very little idea what a well-written computer-assisted paper looks like. Surprisingly, I also had no success finding general writting advice on this subject.

It would probably be too broad to just ask "How to write computer-assisted mathematics well?", although to be honest that's the question to which I'm trying to find an answer. Let me try to be (marginally) more specific.

1. Are there any style manuals that specifically address the question of writting computer-assisted mathematics? Barring that, what are some well-known and well-written papers that one can try and emulate?

2. In my particular case, I have around 20 pages explaining how to reduce a certain problem in number theory / combinatorics to a finite computation and around 100 lines of Mathematica code that perform the computatio. Is it fair game to simply say, once the explanation is completed, "I took the argument above and I coded it up, and the computer produced 42 as output, so that's the solution"? If not, what are reasonable steps to take in order to ensure my findings are verifiable?

3. What steps, if any, should I take to make the code I used to perform computations accessible? In principle, the reader could recreate it themselves, the same way as they can retrace any technical computation that is routinely omitted, but that hardly seems polite.

Answer 1

Follow the golden rule: do unto readers as you would have them do unto you.

Include technical computations if doing so makes readers' life easier; omit them if doing so only obscures the exposition; move them to other sections or appendices if they are not useful on a first reading, but are still of potential value to (some) readers.

The code is no different. Include it in the text itself if doing so communicates something important about mathematics itself. Delegate it to appendix/external page otherwise. Include any comments on implementation, if these are not obvious.

Special considerations for code:

• When uploading a paper on arXiv, you can upload ancillary files. That is a good way to include code with your paper.
• Do not forget to comment your code. Code is meant to be read by people first, and computers second.
• In writing code, exercise the same care you would that you would in relying on someone else's result in your proof. Try to make sure that the other people's code that you use actually works correctly. This is especially difficult with closed proprietary systems such as Mathematica. You can still perform tests, and if possible write more than one implementation. I have ran into a non-insignificant number of bugs this way.

Answer 2

There are so many things to say here. "The concept of 'well' depends on the (position of the) observer." "Don't just involve the reader, recruit them!" "Reading code is more boring than running it." "Is it really proved if it takes a computer?" Tempting as it is to address these and other points, I will elaborate on just one: recruitment.

A good paper informs, but a well written paper inspires. Imagine that you want someone to not only verify your result but extend it. You should not only explain your work with utmost clarity, you should also indicate ways in which your work could be verified or extended. Ideally you have done some of this verification or extension yourself, and left some of the fun of (re-) discovery to the interested reader.

If you can encapsulate the ideas of the Mathematica code into a paragraph, that makes the paper more approachable than presenting a block of well commented code. My opinion (as a reader, not as a professional writer) is that code listings are best left to an appendix or the end of the paper. Only if you are writing an extremely literate program, where every subroutine teaches some mathematics to the reader, do you include it in the paper. Describing how the program performs in a run without explaining well why each computation branch was chosen has all the thrill of watching paint dry. Instead, try to challenge the reader to code with you, by describing the relevant portion of the computation, and then you presenting your solution and subtly asking the reader to come up with code that is good or better.

I have no examples that apply directly to your situation. For inspiration, I recommend the New Turing Omnibus by A.K. Dewdney, which is a collection of short articles in computer science. If you can write the core of your paper in the style of one of these articles, you can at least get people to read and understand the core, and leave the less exciting stuff for a series of appendices.

Gerhard "Doesn't Always Remove An Appendix" Paseman, 2020.06.25.

Answer 3

In theory, good writing is good writing. Writing (or for that matter, doing) computer-assisted mathematics well is, at bottom, not different from writing (or doing) any kind of mathematics well.

In practice, computer-assisted mathematics does pose special challenges. Mathematical writing serves (at least) two purposes: (1) you want to convey your ideas clearly so that people understand them; (2) you want to present your arguments in a way that allows others to confirm their correctness. It would be wonderful if you could accomplish both of these goals simultaneously, and sometimes you can, but computer-assisted mathematics tends to pose special challenges. In particular, one often has to address these two goals separately.

It sounds like you've already done a pretty good job of identifying these two distinct goals, and addressing them separately. Your 20 pages sounds like it's geared toward conveying your ideas clearly, and motivating the computational part. All the usual guidelines for writing mathematics apply here. If you've done your job well, the reader will understand what the computer-assisted part is supposed to do, and how it does it. All that's left is part (2), ensuring that the reader can confirm correctness without too much pain.

To do this, you should first convince yourself that the computation is correct. It is good scholarly practice to have some healthy skepticism of the correctness of computer code, whether it's someone else's code or your own. Victor Miller likes to tell the story of how, historically, several published papers on the computation of the number of primes less than $n$ suffered from the "curse" that all the entries in their table were correct, except for the last and largest value. For any kind of nontrivial computation where your stated theorems actually depend on the computation being correct, you should try to code up the computation in two completely different ways (or at least use two completely different computer algebra packages). If the computation is too large or complicated for this to be practical, then try to think of ways of generating corroborating evidence that your computation is being carried out correctly (e.g., checksums, as Terry Tao commented). For an example of "best practices," I recommend this StackOverflow answer to the question of how to verify the correctness of a computation of the digits of $\pi$. Of course, in your writeup, you should at least summarize the cross-checks that you performed.

Finally, there is the question of how to make your code available to others. This is an important question, but I think that this has already been addressed by the MO questions that commenters have linked to.

Answer 4

A few thoughts:

• Since you are using Mathematica, you have the option of writing the paper as a Mathematica notebook, including as much text as needed. (The same would apply for Python or R.) The Mathematica Journal is one place that publishes many papers in this format.

• It helps to include a simpler version of the code that calculates something, even if that something could be calculated better in other ways. This way readers can understand part of the procedure with a minimum of notation, just as they might want to read a proof of a simple case before reading a proof of the main theorem.

• It helps if the paper and the code are consistent :

  — All the numbers and graphs used in the paper should be generated by the code

  — Variable and function names should be the same on the code and the paper

  — Section titles should be consistent between the code and the paper

This may not be the easiest way to write but is the easiest way to read.