I believe I.M. Gelfand said that when beginning to learn a new subject, one should learn it like a physicist.
In this spirit, what are some helpful and surprising physical intuitions accompanying mathematical objects?
I especially am interested in intuitions that suggest nice proofs...and helpful new ways of thinking.
I don't know if you've heard of this before, but there is an extremely elegant physical proof of the existence and properties of the Fermat point. It even illustrates the degeneracy that starts to occur for largest angle at least $2\pi/3$.
Consider your desired triangle as a flat, sturdy, sheet. Make very small notches at the vertices. Now take three ideal ropes and at one end of each fix identical balls. Join all three free ends. Place the rope-ball configuration on the top of the triangle, and slide each of the ropes through the notches. Let the balls hang off the triangle.
Now, allow the system to assume its minimal potential energy. The intersection of the ropes will move to minimize the sum of the distances to the vertices, as potential energy is linear in height. So the intersection will move to the Fermat point.
But here is the tricky part. As the intersection stabilizes, we know that the forces on it must be zero. The forces all have the same magnitude, so if they are all zero then they must have equal angles between them. So the lines from the Fermat point X to the vertices of triangle ABC satisfy $\angle AXB = \angle BXC = \angle CXA = 2\pi/3$.
Of course, this suggests that in a suitably obtuse triangle the intersection will fall right through one of the notches, and indeed for angles greater than $2\pi/3$ the Fermat point is at the largest angle.
I view the Gauss-Bonnet theorem for a surface without boundary
via the physical analogy that if you push a dent into a maleable surface, or squeeze it,
the surface must compensate by curving elsewhere
to maintain $2 \pi \chi$.
Kirillov's orbit method in representation theory establishes a correspondence (which is not exact in general) between irreducible unitary representations of a Lie group $G$ and orbits of the action of $G$ on the dual of its Lie algebra $\mathfrak{g}^{\ast}$. The physical intuition behind the orbit method comes from the notion of quantizing a classical system: the rough idea, as I understand it, is that orbits of the action of $G$ on $\mathfrak{g}^{\ast}$ should be thought of as classical systems with $G$-symmetry, and the corresponding irreducible representations of $G$ should be thought of as the corresponding quantum systems.
Well, there are of course many examples but perhaps the Dirac $\delta$-function is the most striking one. With quite some violence to the math of usual calculus, Dirac's idea turned out to be exremely useful: the "function" everywhere zero and very high at $0$ so that the integral is $1$. Applications are ranging from Fourier analysis, (linear) PDE, quantum mechanics, and many more. Now of course we know how to make things rigorous in the framework of Schwartz's test function and distribution theory. But the intuition is clearly from physics.
Similar and also in this framework is the notion of oscillatory integrals...
Mr. McGuire: Benjamin.
Benjamin: Mr. McGuire.
Mr. McGuire: Benjamin.
Benjamin: Mr. McGuire.
Mr. McGuire: I want to say two words to you.
Benjamin: Yes, sir.
Mr. McGuire: Are you listening?
Benjamin: Yes, I am.
Mr. McGuire: Path integrals.
The Hilbert method for linear elliptic boundary-value problems start from the physical intuition that a steady system in stable configuration minimizes its energy.
There are several examples at the number theory and physics archive. To get you started let me mention the statistical-mechanical interpretation of the Riemann zeta function as the partition function of a system called the primon gas or Riemann gas.
I remember from Folland's PDE book an anecdote about Green convincing himself of the existence of a Green's Function:
Let $\Omega$ be a vacuum and $S$ a perfectly conducting shell grounded to zero potential. Place a negative charge at $x\in \Omega$. This induces a positive charge on the shell $S$. Indeed, the Green's Function $G(x,y)$ is the induced charge at a point $y$.
I would also add that the physical interpretations of gradients, divergence and curl are indespensible for REMEMBERING the various theorems of Gauss, Stokes and Green. For example, the curl of a velocity field is twice the angular acceleration, a fact that facilitates the order of differences of partials in the curl operator.
Archimedes figured out the area of a parabolic section by slicing it into pieces and moving the pieces to the other side of an imaginary lever and observing that they balance a certain triangle.
The Jacobi triple product formula is
$$\prod_{m=1}^\infty \left( 1 - x^{2m}\right) \left( 1 + x^{2m-1} y^2\right) \left( 1 +\frac{x^{2m-1}}{y^2}\right) = \sum_{n=-\infty}^\infty x^{n^2} y^{2n} $$
Borcherds found an elegant combinatorial argument for this formula based on counting the number of ways it is possible for a certain physical system to achieve particular energy levels. The physical model was based on the Dirac sea model of the vacuum.
Many combinatorial formulae find interpretations this way. For example the appearance of $n-\frac{1}{24}$ in Rademacher's formula for the partition function is less of a surprise for physicists familiar with vacuum energies.
My favourite example is that of braided fusion categories (a fusion category has finitely many equivalence classes of irreducible objects, but that is not essential here). This is basically the idea behind topological quantum computing. Without going into details, a nice interpretation of the equivalence classes of irreducible objects is as "particle types" or "charges". In a fusion category there is a monoidal operation $\otimes$, which can be interpreted as "composition of charges". Moreover, if $X$,$Y$ are two irreducible objects, the product $X \otimes Y$ can be decomposed into a direct sum $\bigoplus_i X_i$ of irreducible objects. This is called fusion, and can be interpreted as "fusing" particles. The irreducibles appearing in the decomposition are the particle types that can appear after fusing two particles.
The braiding of the category describes what happens if we interchange two particles. If the braiding is in fact a symmetry, that is, we have a symmetric monoidal category, the situation reduces to the well-known Bose/Fermi statistics in physics. The existence of dual objects corresponds to anti-particles. In particular, there is a morphism from the tensor unit (the "vacuum") to $X \otimes \overline{X}$ for every object $X$, interpreted as creating a particle together with it's anti-particle form the vacuum. Note that charge is conserved!
Pushing this analogy a bit further, one can interpret the "tangle diagrams" that are commonly used to describe morphisms in braided monoidal categories as the wordlines of particles. For example, a diagram with one input and two outputs can be interpreted as a particle splitting into two. Although this whole picture of worldlines should not be taken to literally, I find it a helpful when thinking about such diagrams.
The previous one was so classical!. This one is much less. James Glimm succeeded to prove in 1967 the existence of a global-in-time solution of the Cauchy problem for hyperbolic systems of conservation laws in one space dimension. His proof is probabilistic. The key of the proof is the construction and estimate of the potential of interaction. It involves pairs of waves that are approaching each other. Clearly a physical intuition.
I think, Elliott Sober gives a nice example (cf. Elliott Sober `Quine's Two Dogmas', Proceedings of the Aristotelian Society, Supplementary Volumes, Vol. 74 (2000), pp. 268-269):
For logic and mathematics to be tested empirically, one logical or mathematical statement would have to be pitted against an alternative and a framework of shared background assumptions would have to be supplied that permits the two statements to make different predictions about observations. It is instructive to examine a case in the history of mathematics in which this actually happened. Plateau was a 19th century French mathematician who wanted to figure out what the surface of least area is that fills various closed curves. A simple example of the type of problem that Plateau had in mind is a curve that has the shape of an ellipse. An elliptical curve can be filled by the ellipse it contains; any other surface that fills the curve will bulge into a third dimension and so must have more surface area. Although the answer to Plateau's question is obvious for this example, the answers aren't at all obvious for more complicated three-dimensional curves. What Plateau did was to dip wires bent in different shapes into soapy water and observe the resulting soap bubbles that adhered to the frames when they were removed from the water (Courant, R. and Robbins, H., 1941, What is Mathematics?, New York: Oxford University Press). Given the physical assumption that soap bubbles take on the surface of least area in this experiment, different mathematical hypotheses ('the surface of least area for curve C is s1' versus 'the surface of least area for curve C is s2', for example) make different predictions. This example is interesting in the history of mathematics precisely because it is so atypical. No such test of '2 + 3 = 5' against alternative arithmetic hypotheses has ever been carried out, nor is it remotely clear what this would be like.
For beginners, Polya's Induction and Plausible Reasoning has some nice examples of physical reasoning. For example he has a chapter on a shortest path problem that is solved mechanically and optically.