Revisiting the unreasonable effectiveness of mathematics

Question

Question:

On balance, with theoretical advances in algorithmic information theory and Quantum Computation it appears that the remarkable effectiveness of mathematics in the natural sciences is quite reasonable. By effectiveness, I am generally referring to Wigner's observation that mathematical laws have remarkable generalisation power.

Might there be a modern review paper on the subject for mathematicians where the original question is re-evaluated in light of modern mathematical sciences?

An information-theoretic perspective:

In order to motivate an information-theoretic analysis, it is worth observing that Occam's razor is an essential tool in the development of mathematical theories.

From an information-theoretic perspective, a Universe where Occam's razor is generally applicable is one where information is conserved. The conservation of information would imply that fundamental physical laws are generally time-reversible. Moreover, given that Occam's razor has an appropriate formulation within the context of algorithmic information theory as the Minimum Description Length principle this information-theoretic analysis generally presumes that the Universe itself may be simulated by a Universal Turing Machine.

David Deutsch and others have done significant work demonstrating the plausibility of the Physical Church-Turing thesis(which is consistent with the original Church-Turing thesis) and this would explain why mathematical methods are so effective in the natural sciences.

This brief analysis has emerged from informal discussions with a handful of algorithmic information theorists(Hector Zenil, Marcus Hutter, and others) and it makes me wonder whether complementary theories from mathematical physics might help mathematicians account for the remarkable effectiveness of mathematics in the natural sciences.

Clarification of particular terms:

Minimum Description Length principle:

Given data in the form of a binary string $x \in \{0,1\}^*$ the Minimum Description Length of $x$ is given by the Kolmogorov Complexity of $x$:

\begin{equation} K_U(x) = \min_{p} \{|p|: U(p) = x\} \end{equation}

where $U$ is a reference Universal Turing Machine and $p$ is the shortest program that takes as input the empty string $\epsilon$ and outputs $x$.

The Law of Conservation of Information:

The Law of Conservation of information which dates back to Von Neumann essentially states that the Von Neumann entropy is invariant to Unitary transformations. This is meaningful within the framework of Everettian quantum mechanics as a density matrix may be assigned to the state of the Universe. This way information is conserved as we run a simulation of the Universe forwards or backwards in time.

The Physical Church-Turing thesis:

The Law of Conservation of information is consistent with the observation that all fundamental physical laws are time-reversible and computable. The research of David Deutsch(and others) on the Physical Church-Turing thesis explains how a Universal Quantum computer may simulate these laws. Michael Nielsen wrote a good introductory blog post on the subject [7].

The Physical Church-Turing thesis is a key point in this discussion as it provides us with a credible explanation for the remarkable effectiveness of mathematics in the natural sciences.

A remark on effectiveness:

What I have retained from my discussions with physicists and other natural scientists is that the same mathematical laws with remarkable generalisation power are also constrained by Occam's razor. In fact, from an information-theoretic perspective the remarkable effectiveness of mathematics is a direct consequence of the effectiveness of Occam's razor. This may be partly understood from a historical perspective if one surveys the evolution of ideas in physics [10].

Given two compatible theories, Einstein generally argued that one should choose the simplest theory that yields negligible experimental error. To be precise, he stated:

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.-Einstein(1933)

As the application of Occam's razor generally requires a space of computable models, and algorithmic information theory carefully explains why simpler theories generalise better [8] it is fair to say that the notion of effectiveness may be made precise. However, the theory of algorithmic information was developed in the mid 1960s by Chaitin, Kolmogorov and Solomonoff which was after Wigner wrote his article in 1960.

What is remarkable:

If we view the scientific method as an algorithmic search procedure then there is no reason, a priori, to suspect that a particular inductive bias should be particularly powerful. This much was established by David Wolpert via his No Free Lunch Theorems [11].

On the other hand, the history of natural science indicates that Occam's razor is remarkably effective. The effectiveness of this inductive bias has more recently been explored within the context of deep learning [12].

References:

1. Eugene Wigner. The Unreasonable Effectiveness of Mathematics in the Natural Sciences. 1960.

2. David Deutsch. Quantum theory, the Church–Turing principle and the universal quantum computer. 1985.

3. Peter D. Grünwald. The Minimum Description Length Principle . MIT Press. 2007.

4. A. N. Kolmogorov Three approaches to the quantitative definition of information. Problems of Information and Transmission, 1(1):1--7, 1965

5. G. J. Chaitin On the length of programs for computing finite binary sequences: Statistical considerations. Journal of the ACM, 16(1):145--159, 1969.

6. R. J. Solomonoff A formal theory of inductive inference: Parts 1 and 2. Information and Control, 7:1--22 and 224--254, 1964.

7. Michael Nielsen. Interesting problems: The Church-Turing-Deutsch Principle. 2004. https://michaelnielsen.org/blog/interesting-problems-the-church-turing-deutsch-principle/

8. Marcus Hutter et al. (2007) Algorithmic probability. Scholarpedia, 2(8):2572.

9. Andrew Robinson. Did Einstein really say that? Nature. 2018.

10. The Evolution of Physics, Albert Einstein & Leopold Infeld, 1938, Edited by C.P. Snow, Cambridge University Press

11. Wolpert, D.H., Macready, W.G. (1997), "No Free Lunch Theorems for Optimization", IEEE Transactions on Evolutionary Computation 1, 67.

12. Guillermo Valle Pérez, Chico Camargo, Ard Louis. Deep Learning generalizes because the parameter-function map is biased towards simple functions. 2019.

Answer 1

A 2013 issue of Interdisciplinary Science Reviews was entirely devoted to this topic. One viewpoint, by Jesper Lützen, struck me:

When Wigner claimed that the effectiveness of mathematics in the natural sciences was unreasonable it was due to a dogmatic formalist view of mathematics according to which higher mathematics is developed solely with a view to formal beauty. I shall argue that this philosophy is not in agreement with the actual practice of mathematics. Indeed, I shall briefly illustrate how physics has influenced the development of mathematics from antiquity up to the twentieth century. If this influence is taken into account, the effectiveness of mathematics is far more reasonable.

(the articles in this issue are behind a paywall, perhaps there is another way to access them...)

Answer 2

I have never seen any remotely plausible attempt at setting up a framework in which we are able to quantitatively calculate exactly how much effectiveness would be "reasonable," let alone calculate the probability that the effectiveness would be at least as unreasonable as the actually observed effectiveness.

The more I have pondered the previous paragraph, the more I have come to the conclusion that we should treat Wigner's essay as an exercise in cultivating our sense of wonder, and not as a sketch of a proposed scientific experiment or calculation.

Answer 3

I'm not sure why it has not yet been pointed out that all known applications of mathematics to explain or predict phenomena in the real world only rely on a very weak part of mathematics. For example, it is well known that ACA (see Reverse Mathematics) suffices for almost all real analysis, and one does not seem to need to go further than ATR for everything that is relevant to reality.

In that light, it should be obvious why applied mathematics is so effective: Clearly, ACA is the natural outcome of believing that there is some kind of real-world interpretation of PA that is standard (i.e. the domain only consists of the interpretations of "$0$" and terms of the form "$1+\cdots+1$"). Since we do believe that we are able to store and manipulate in physical media (e.g. computing systems) encoded representations of very large natural numbers up to $2^{4096}$ and beyond, and they seem to obey all the axioms of PA⁻, we can justify that those encodings yield a model of ACA as well. A little bit more work can push this justification to ATR, if we assume that the concept of "well-ordering on $ℕ$" is an absolute notion. (Many logicians talk about ACA0 or ATR0, rather than ACA or ATR, but clearly the full induction schema is sound if the original system has a standard model.)

Note that ACA already can construct all finite Turing jumps, and ATR can go further. But frankly, what real-world phenomenon needs even the ω-th jump? The belief in the existence of the $k$-th finite jump corresponds to the belief that every $k$-quantifier arithmetical sentence has a well-defined truth-value, which we already believe once we believe PA is meaningful. Beyond that, well, conceptually we can iterate the jump along any explicitly constructed and proven well-ordering, so one might argue that it has meaning too, even if not in terms of concrete physical entities, at least in terms of well-defined conceptual notions.

Until someone demonstrates a mathematical theorem that has clear empirical real-world verification but cannot reasonably be expressed in any form that is provable in ATR, there is nothing surprising at all about effectiveness of applied mathematics, simply because applied mathematics can be carried out within a natural extension (e.g. ACA or ATR) of a system (PA) that had been specifically designed to reflect facts about real-world counting!

Answer 4

The question (and some replies) seems to be arguing something of the following form:

There's no "unreasonable" effectiveness of mathematics. Of course maths is effective in physics! All we have to do is assume this rather basic mathematical principle (e.g. "algorithmic information theory" or "ACA") and there you go - maths being effective derives from just that assumption.

The problem though is that this is circular argument. You're assuming the Universe obeys some deeper/more fundamental mathematical principle in order to prove that maths will necessarily be useful to those scientists exploring its possibilities. That is, you're assuming the very thing you're trying to prove.

It's trivially easy to imagine universes that don't follow mathematical laws or even have some very deep mathematical reason at their root (there are entire fields of literature plausibly related to such thought experiments). If your answer to Wigner's problem is that our Universe isn't like that, and it has some deep mathematical root like the above ideas, the question then is why should our Universe be like that? Why that as a brute fact?

This, ultimately, is a philosophical question, not a purely mathematical one. And there are probably good sociological, historical and anthropic arguments for why certain kinds of mathematical explanations are usually better than other mathematical explanations. But I wouldn't expect humans to find why maths should be useful full stop - that's just as a deep question as "why is there something rather than nothing?". Any answer you come up with can still face the brute simplicity of another "why" question. We just have to humbly accept that as a species.

Answer 5

The French logician Jean-Louis Krivine gives an interesting perspective in his essay Wigner, Curry et Howard.

My simplified summary of his explanation of the "unreasonable" effectiveness is that rather than simply "discovering" theorems, mathematicians "reverse-engineer" them from the human brain, where they were written by evolution.

The second-to-last paragraph summarizes a core part of Krivine's argument:

Nous venons de nous apercevoir que les lois mathématiques sont, en fait, des programmes écrits dans la mémoire morte de notre cerveau. Dire que le monde physique se conforme à de telles lois revient donc à dire qu’il se conforme à des programmes écrits dans notre cerveau. L’anthropocentrisme ridicule de cette affirmation saute aux yeux, nous voilà revenus au temps où les planètes, le soleil et les étoiles tournaient autour de la terre et où les oranges étaient divisées en quartiers pour que nous puissions les consommer plus facilement. Wigner a tout à fait raison de trouver cela « déraisonnable ».

In English:

We just observed that the mathematical laws are in fact programs that are hard-coded in our brains. To say that the physical world conforms to these laws thus amounts to saying that it conforms to programs written in our brain. The ridiculous anthropocentrism of this statement jumps to the eyes, we have returned to the time where the planets, the sun and the stars turned around the earth and where the oranges where divided in quarters so that we could consume them more easily. Wigner is indeed right to find this "unreasonable".

Thus Krivine dismisses the question, perhaps in a way similar to Lützen as quoted in Carlo Beenakker's answer.

Answer 6

As a rather recent revisitation of Wigner's article, one may also quote James Glimm, who wrote in the article "Mathematical perspectives" (Bull. Amer. Math. Soc. (N.S.) 47 (2009), no. 1, 127–136),

In simple terms, mathematics works. It is effective. It is essential. It is practical. Its force cannot be avoided, and the future belongs to societies that embrace its power. Its force is derived from its essential role within science, and from the role of science in technology. Wigner’s observations concerning The Unreasonable Effectiveness of Mathematics are truer today than when they were first written in 1960.