# Revisiting the unreasonable effectiveness of mathematics

## Question:

On balance, with theoretical advances in algorithmic information theory and Quantum Computation it appears that the remarkable effectiveness of mathematics in the natural sciences is quite reasonable. By effectiveness, I am generally referring to Wigner's observation that mathematical laws have remarkable generalisation power.

Might there be a modern review paper on the subject for mathematicians where the original question is re-evaluated in light of modern mathematical sciences?

## An information-theoretic perspective:

In order to motivate an information-theoretic analysis, it is worth observing that Occam's razor is an essential tool in the development of mathematical theories.

From an information-theoretic perspective, a Universe where Occam's razor is generally applicable is one where information is conserved. The conservation of information would imply that fundamental physical laws are generally time-reversible. Moreover, given that Occam's razor has an appropriate formulation within the context of algorithmic information theory as the Minimum Description Length principle this information-theoretic analysis generally presumes that the Universe itself may be simulated by a Universal Turing Machine.

David Deutsch and others have done significant work demonstrating the plausibility of the Physical Church-Turing thesis(which is consistent with the original Church-Turing thesis) and this would explain why mathematical methods are so effective in the natural sciences.

This brief analysis has emerged from informal discussions with a handful of algorithmic information theorists(Hector Zenil, Marcus Hutter, and others) and it makes me wonder whether complementary theories from mathematical physics might help mathematicians account for the remarkable effectiveness of mathematics in the natural sciences.

## Clarification of particular terms:

### Minimum Description Length principle:

Given data in the form of a binary string $$x \in \{0,1\}^*$$ the Minimum Description Length of $$x$$ is given by the Kolmogorov Complexity of $$x$$:

$$$$K_U(x) = \min_{p} \{|p|: U(p) = x\}$$$$

where $$U$$ is a reference Universal Turing Machine and $$p$$ is the shortest program that takes as input the empty string $$\epsilon$$ and outputs $$x$$.

### The Law of Conservation of Information:

The Law of Conservation of information which dates back to Von Neumann essentially states that the Von Neumann entropy is invariant to Unitary transformations. This is meaningful within the framework of Everettian quantum mechanics as a density matrix may be assigned to the state of the Universe. This way information is conserved as we run a simulation of the Universe forwards or backwards in time.

### The Physical Church-Turing thesis:

The Law of Conservation of information is consistent with the observation that all fundamental physical laws are time-reversible and computable. The research of David Deutsch(and others) on the Physical Church-Turing thesis explains how a Universal Quantum computer may simulate these laws. Michael Nielsen wrote a good introductory blog post on the subject [7].

The Physical Church-Turing thesis is a key point in this discussion as it provides us with a credible explanation for the remarkable effectiveness of mathematics in the natural sciences.

## A remark on effectiveness:

What I have retained from my discussions with physicists and other natural scientists is that the same mathematical laws with remarkable generalisation power are also constrained by Occam's razor. In fact, from an information-theoretic perspective the remarkable effectiveness of mathematics is a direct consequence of the effectiveness of Occam's razor. This may be partly understood from a historical perspective if one surveys the evolution of ideas in physics [10].

Given two compatible theories, Einstein generally argued that one should choose the simplest theory that yields negligible experimental error. To be precise, he stated:

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.-Einstein(1933)

As the application of Occam's razor generally requires a space of computable models, and algorithmic information theory carefully explains why simpler theories generalise better [8] it is fair to say that the notion of effectiveness may be made precise. However, the theory of algorithmic information was developed in the mid 1960s by Chaitin, Kolmogorov and Solomonoff which was after Wigner wrote his article in 1960.

## What is remarkable:

If we view the scientific method as an algorithmic search procedure then there is no reason, a priori, to suspect that a particular inductive bias should be particularly powerful. This much was established by David Wolpert via his No Free Lunch Theorems [11].

On the other hand, the history of natural science indicates that Occam's razor is remarkably effective. The effectiveness of this inductive bias has more recently been explored within the context of deep learning [12].

## References:

1. Eugene Wigner. The Unreasonable Effectiveness of Mathematics in the Natural Sciences. 1960.

2. David Deutsch. Quantum theory, the Church–Turing principle and the universal quantum computer. 1985.

3. Peter D. Grünwald. The Minimum Description Length Principle . MIT Press. 2007.

4. A. N. Kolmogorov Three approaches to the quantitative definition of information. Problems of Information and Transmission, 1(1):1--7, 1965

5. G. J. Chaitin On the length of programs for computing finite binary sequences: Statistical considerations. Journal of the ACM, 16(1):145--159, 1969.

6. R. J. Solomonoff A formal theory of inductive inference: Parts 1 and 2. Information and Control, 7:1--22 and 224--254, 1964.

7. Michael Nielsen. Interesting problems: The Church-Turing-Deutsch Principle. 2004. https://michaelnielsen.org/blog/interesting-problems-the-church-turing-deutsch-principle/

8. Marcus Hutter et al. (2007) Algorithmic probability. Scholarpedia, 2(8):2572.

9. Andrew Robinson. Did Einstein really say that? Nature. 2018.

10. The Evolution of Physics, Albert Einstein & Leopold Infeld, 1938, Edited by C.P. Snow, Cambridge University Press

11. Wolpert, D.H., Macready, W.G. (1997), "No Free Lunch Theorems for Optimization", IEEE Transactions on Evolutionary Computation 1, 67.

12. Guillermo Valle Pérez, Chico Camargo, Ard Louis. Deep Learning generalizes because the parameter-function map is biased towards simple functions. 2019.

• Is "effectiveness" a well defined expression or are you using it in a colloquial way? Jun 9 at 15:27
• What is the relevance of quantum computation here? This question might be clearer with fewer buzzwords. Jun 9 at 16:20
• I think I don't understand the premise. Is it: (universe can be simulated by a turing machine) implies (mathematics is useful for physics)?
– usul
Jun 9 at 17:14
• @MattF. I clarified particular terms. Physicists think that quantum mechanics may be used to simulate almost all physics, hence the most important contributions to the Physical Church-Turing thesis have been via theories of quantum computation. Jun 9 at 18:39
• @AidanRocke, the clarifications are helpful, but I still find the posture here off-putting. Phrases like "on balance it appears", "it is worth observing that", "it is fair to say that" make me wary of the claims that follow. Also I consider the "precise" quote from Einstein to be both unhelpful and fake: quoteinvestigator.com/2011/05/13/einstein-simple Jun 9 at 20:16

A 2013 issue of Interdisciplinary Science Reviews was entirely devoted to this topic. One viewpoint, by Jesper Lützen, struck me:

When Wigner claimed that the effectiveness of mathematics in the natural sciences was unreasonable it was due to a dogmatic formalist view of mathematics according to which higher mathematics is developed solely with a view to formal beauty. I shall argue that this philosophy is not in agreement with the actual practice of mathematics. Indeed, I shall briefly illustrate how physics has influenced the development of mathematics from antiquity up to the twentieth century. If this influence is taken into account, the effectiveness of mathematics is far more reasonable.

(the articles in this issue are behind a paywall, perhaps there is another way to access them...)

• Yes, the sometimes-stylish conceit that "pure" mathematics is guided by nothing but pure aesthetics... is not only recent, but inaccurate, in my observation. True, the current culture of write-as-many-papers-as-possible may have (among others) a perverse effect of people creating make-work for themselves. Jun 9 at 19:53
• ''Yes, the sometimes-stylish conceit that "pure" mathematics is guided by nothing but pure aesthetics... is not only recent, but inaccurate'' ~ This is hardly a recent conceit. Jun 11 at 15:50
• On the other hand, aesthetics is not really "pure" - there are way too many factors.
– Z. M
Jun 11 at 20:55
• Consistent also with the thrust of the book Where Mathematics Comes From by George Lakoff and Rafael E. Núñez. Jun 12 at 1:45
• Smacks also of the conceit Douglas Adams cautioned against: This is rather as if you imagine a puddle waking up one morning and thinking, 'This is an interesting world I find myself in — an interesting hole I find myself in — fits me rather neatly, doesn't it? In fact it fits me staggeringly well, must have been made to have me in it!.' // The perspective of the mystic versus that of the natural secular humanist. Jun 12 at 1:59

I have never seen any remotely plausible attempt at setting up a framework in which we are able to quantitatively calculate exactly how much effectiveness would be "reasonable," let alone calculate the probability that the effectiveness would be at least as unreasonable as the actually observed effectiveness.

The more I have pondered the previous paragraph, the more I have come to the conclusion that we should treat Wigner's essay as an exercise in cultivating our sense of wonder, and not as a sketch of a proposed scientific experiment or calculation.

• I agree that the effectiveness is somewhat vague which is why I specifically focus on Wigner's observation that mathematical laws have remarkable generalisation power. A key point in my analysis, and that of physicists with whom I have discussed the issue, is that these laws also satisfy Occam's razor. Jun 10 at 6:00
• 3. 3 effectiveness would be reasonable. 4 effectiveness would be ridiculous. Jun 10 at 6:27
• @AidanRocke I don't see anything in your analysis that lets us quantify the word "remarkable." Are mathematical laws remarkable or not? What quantitative calculation lets you answer that question? Jun 10 at 12:50
• @TimothyChow this is a good point to address. I have since added a section on what precisely is remarkable from an algorithmic perspective. Jun 10 at 14:16
• To your second paragraph, I think Wigner wrote his essay as an antithesis to a thesis that he never states - that "advanced mathematics should never be used in physics". I seem to remember that Wigner had substantial opposition to publishing his paper on representations of the Poincaré group, and for a long time a large community of physicists called the use of group theory in quantum mechanics the "Gruppenpest", i.e. the "pestilence of groups". Jun 10 at 17:57

I'm not sure why it has not yet been pointed out that all known applications of mathematics to explain or predict phenomena in the real world only rely on a very weak part of mathematics. For example, it is well known that ACA (see Reverse Mathematics) suffices for almost all real analysis, and one does not seem to need to go further than ATR for everything that is relevant to reality.

In that light, it should be obvious why applied mathematics is so effective: Clearly, ACA is the natural outcome of believing that there is some kind of real-world interpretation of PA that is standard (i.e. the domain only consists of the interpretations of "$$0$$" and terms of the form "$$1+\cdots+1$$"). Since we do believe that we are able to store and manipulate in physical media (e.g. computing systems) encoded representations of very large natural numbers up to $$2^{4096}$$ and beyond, and they seem to obey all the axioms of PA, we can justify that those encodings yield a model of ACA as well. A little bit more work can push this justification to ATR, if we assume that the concept of "well-ordering on $$ℕ$$" is an absolute notion. (Many logicians talk about ACA0 or ATR0, rather than ACA or ATR, but clearly the full induction schema is sound if the original system has a standard model.)

Note that ACA already can construct all finite Turing jumps, and ATR can go further. But frankly, what real-world phenomenon needs even the ω-th jump? The belief in the existence of the $$k$$-th finite jump corresponds to the belief that every $$k$$-quantifier arithmetical sentence has a well-defined truth-value, which we already believe once we believe PA is meaningful. Beyond that, well, conceptually we can iterate the jump along any explicitly constructed and proven well-ordering, so one might argue that it has meaning too, even if not in terms of concrete physical entities, at least in terms of well-defined conceptual notions.

Until someone demonstrates a mathematical theorem that has clear empirical real-world verification but cannot reasonably be expressed in any form that is provable in ATR, there is nothing surprising at all about effectiveness of applied mathematics, simply because applied mathematics can be carried out within a natural extension (e.g. ACA or ATR) of a system (PA) that had been specifically designed to reflect facts about real-world counting!

• This seems to beg the question. Why should a theory based on "counting" predict the behavior of the real world? Quantum field theory would suggest that "counting" is an emergent property, not something fundamental to "reality." Even in classical physics, there is nothing obviously countable about gravitational or electromagnetic fields. Jun 11 at 15:52
• @alephzero: In my answer I stated that, since real-world counting is correctly captured by ACA, it is unsurprising that all theorems of ACA are effective for the real world. Your question is why lots of other physical phenomena that do not appear to be directly related to counting also end up being also constrained by theorems of ACA. The answer is quite simple, and lies in understanding the results of reverse mathematics. Many other physical phenomena appear to be described by facts of real analysis, but the core structures underlying $ℝ$ is intrinsically related to $ℕ$... Jun 11 at 16:05
• @alephzero: In particular, a key well-known fact in reverse mathematics is that every continuous function $f$ on $ℝ$ can be encoded by its values on $ℚ$, and every member of $ℝ$ itself can be encoded by a subset of $ℕ$, so $f$ can be encoded by a subset of $ℕ$. This fact is how results about continuous functions can almost all be captured by facts about subsets of $ℕ$, and these facts turn out to be provable in ACA. Jun 11 at 16:08
• @user21820 Going from a vague concept of "real-world counting" all the way up to ACA seems like a big jump to me. Why can't I argue that bounded arithmetic already captures "real-word counting" and hence it is remarkable that stronger induction axioms, or the weak Koenig lemma, also seem to describe the real world? Jun 12 at 13:01
• @TimothyChow: That's why I explicitly said that it rests on "believing that there is some kind of real-world interpretation of PA that is standard". If you believe in PA− but only very limited induction, then you of course can't get to ACA. But that doesn't really make sense, because from any model of PA− the initial segment is a model of full PA. So simply belief in some kind of real-world interpretation of PA− that is standard suffices! (Of course you might argue that this claim itself cannot be proven without sufficiently strong meta-system, but the intuition is still explanatory.) Jun 12 at 17:37

The question (and some replies) seems to be arguing something of the following form:

There's no "unreasonable" effectiveness of mathematics. Of course maths is effective in physics! All we have to do is assume this rather basic mathematical principle (e.g. "algorithmic information theory" or "ACA") and there you go - maths being effective derives from just that assumption.

The problem though is that this is circular argument. You're assuming the Universe obeys some deeper/more fundamental mathematical principle in order to prove that maths will necessarily be useful to those scientists exploring its possibilities. That is, you're assuming the very thing you're trying to prove.

It's trivially easy to imagine universes that don't follow mathematical laws or even have some very deep mathematical reason at their root (there are entire fields of literature plausibly related to such thought experiments). If your answer to Wigner's problem is that our Universe isn't like that, and it has some deep mathematical root like the above ideas, the question then is why should our Universe be like that? Why that as a brute fact?

This, ultimately, is a philosophical question, not a purely mathematical one. And there are probably good sociological, historical and anthropic arguments for why certain kinds of mathematical explanations are usually better than other mathematical explanations. But I wouldn't expect humans to find why maths should be useful full stop - that's just as a deep question as "why is there something rather than nothing?". Any answer you come up with can still face the brute simplicity of another "why" question. We just have to humbly accept that as a species.

• This is plainly wrong. PA (and later ACA) was designed to model real-world counting. So of course mathematics that can be carried out in ACA are effective in real-world applications! This is not circular at all. If the world was different enough that counting had a different structure, we would be much less likely to invent PA! The problem with other responses is that they conflate "all mathematics" with "mathematics that is effective for the real-world", and I had already explained that in my answer. Jun 11 at 15:52
• @user21820 I think you have misunderstood my answer. The fact that ACA was designed to model real-world counting is a sociological explanation for why that particular form of mathematics is useful as opposed to some other form of mathematics (see my last paragraph). However, that sociological explanation does not explain why any maths at all should be useful i.e. why should there even be such a thing as "real-world number counting". That's what makes it circular. Jun 12 at 22:44
• From your comment, it's clear that you're answering a completely different question. The question asks why our mathematics, which is based on "a particular inductive bias", seems to be particularly effective in modelling some aspects of our world. You appear to be talking about whatever mathematics that suits any possible world. But it's incorrect for you to claim that I assumed some basic mathematical principle must hold in the real world (in any modal sense). All I said was that it does hold in this world, causing us to invent effective mathematics! Jun 13 at 3:50

The French logician Jean-Louis Krivine gives an interesting perspective in his essay Wigner, Curry et Howard.

My simplified summary of his explanation of the "unreasonable" effectiveness is that rather than simply "discovering" theorems, mathematicians "reverse-engineer" them from the human brain, where they were written by evolution.

The second-to-last paragraph summarizes a core part of Krivine's argument:

Nous venons de nous apercevoir que les lois mathématiques sont, en fait, des programmes écrits dans la mémoire morte de notre cerveau. Dire que le monde physique se conforme à de telles lois revient donc à dire qu’il se conforme à des programmes écrits dans notre cerveau. L’anthropocentrisme ridicule de cette affirmation saute aux yeux, nous voilà revenus au temps où les planètes, le soleil et les étoiles tournaient autour de la terre et où les oranges étaient divisées en quartiers pour que nous puissions les consommer plus facilement. Wigner a tout à fait raison de trouver cela « déraisonnable ».

In English:

We just observed that the mathematical laws are in fact programs that are hard-coded in our brains. To say that the physical world conforms to these laws thus amounts to saying that it conforms to programs written in our brain. The ridiculous anthropocentrism of this statement jumps to the eyes, we have returned to the time where the planets, the sun and the stars turned around the earth and where the oranges where divided in quarters so that we could consume them more easily. Wigner is indeed right to find this "unreasonable".

Thus Krivine dismisses the question, perhaps in a way similar to Lützen as quoted in Carlo Beenakker's answer.

As a rather recent revisitation of Wigner's article, one may also quote James Glimm, who wrote in the article "Mathematical perspectives" (Bull. Amer. Math. Soc. (N.S.) 47 (2009), no. 1, 127–136),

In simple terms, mathematics works. It is effective. It is essential. It is practical. Its force cannot be avoided, and the future belongs to societies that embrace its power. Its force is derived from its essential role within science, and from the role of science in technology. Wigner’s observations concerning The Unreasonable Effectiveness of Mathematics are truer today than when they were first written in 1960.