This interesting little fellow (a nice introduction is the video "Mock Modular Forms are Everywhere" by Cheng and Felder) popped up in some operator algebra (Witt / Virasoro Lie algebra) I was exploring, so I've been curious about where else the Dedekind $\eta$-function makes a cameo appearance and found that in physics it's related to the
- statistical parameters of solvable Ising models
(See "The Reasonable and Unreasonable Effectiveness of Number Theory in Statistical Mechanics" by G. Andrews and "Introduction to Exactly Solvable models in Statistical Mechanics" by C. Tracy.)
The difference between the average local occupation densities of two sub-lattices of a hard hexagon model of a lattice gas given on pp. 368-371 of Tracy is $R(\tau)=\frac{n(\tau)\eta(5\tau)}{\eta^2(3\tau)}$.
partition functions (statistical mechanics variety) for colored bosons moving on a line (1/24 is the associated Casimir energy) and one-color fermions
operator traces (characters) for the infinite dimensional Lie algebras $\widehat{su}_n$, equivalent to 2-Dim current algebras
partition function of a microscopic black-hole in a 5-Dim D-brane
string theory guage corrections
(For 2-5, see "Nucleon Structure, Duality and Elliptic Theta Functions" by W. Scott. For item 2, see also "Vertex Operators and Modular Forms" by G. Mason and M. Tuite.)
From pg. 39 of "Fivebrane instantons ..." and on pg. 11 of "D3 instantons ...," a correction to the field basis (of the RR axion dual to D3-branes) in type IIB string perturbation theory related to the action of S-duality in ten dimensions:
$\tilde{c_a} \mapsto \tilde{c_a}-\tilde{c}_{2,a}\:\epsilon(g)$ where, with $g=\binom{a\:\:b}{c\:\:d}$,
$$\exp(2\pi i \epsilon(g))=\frac{\eta\left [ \frac{a\tau+b}{c\tau+d} \right ]}{\left ( c\tau+d \right )^{\frac{1}{2}}\eta(\tau)}.$$
- partition function in 2+1 dimensions and vanishing chemical potential of non-relativistic fermions in a constant magnetic field
("Nonrelativistic Fermions in Magnetic Fields: a Quantum Field Theory Approach" by O. Espinosa, J. Gamboa, S. Lepe, and F. Mendez)
- physics of gauge theories and the Dirac operator
(See "The Logarithm of the Dedekind $\eta$ Function" by M. Atiyah.)
Michael Atiyah even goes so far as to say, "It seems therefore timely to attempt to survey the whole development of the theory of $\log(\eta)$, putting results in their natural order and in the appropriate general context. This is the aim of the present paper, in which the emphasis will be strongly geometrical. In a sense we shall show that the latest ideas from physics [circa 1987] provide the key to a proper understanding of Dedekind's original results."
[ 7.5 Edit 1/18/21: The lecture "A geometric approach to the modular flow on the space of lattices" by Bruce Bartlett interleaves topics presented by Atiyah above and Ghys below.]
- knots and dynamics
(See "Knots and Dynamics" by E. Ghys, and Chapter 2 A New Twist in Knot Theory in Dana MacKenzie's book What's Happening in the Mathematical Sciences Vol. 7.)
Ghys presents the equation $$24\log\eta\left(\frac{a\tau+b}{c\tau+d}\right)=24\: \log(\eta(\tau))+6\: \log(-(c\tau+d)^{2})+2\pi i\:\mathfrak{R}\left(\binom{a\: b}{c\: d}\right) $$
where $\mathfrak{R}$ is the Rademacher function, which he relates to the linking number
between two knots related to modular/Lorenz flow: “For every hyperbolic element $A=\binom{a\: b}{c\: d}$ in $PSL(2,Z)$, the linking number between the [modular/Lorenz] knot $k_A$ and the trefoil knot $l$ is equal to $\mathfrak{R}(A)$ ....”
- string/brane partition functions, propagators, and metrics
In "String Theory" by S. Nibbelink, $\eta$ occurs in the denominator of string partition functions for fermionic and bosonic zero modes (pp. 163-7).
A coefficient in the 10-dim metric for a 7-brane is given as $e^{\phi}=\tau_2 \eta^2\bar{\eta}^2|\prod_{i=1}^{k}(z-z_i)^{-\frac{1}{12}}|^2$ on pg. 493 of "Supergravity vacua and solitons" by G. Gibbons.
In what other contexts in physics does the Dedekind $\eta$ function take a bow?
(Edit) Moreover, since this is a community wiki and not a test question with one best answer but an attempt to come to a better understanding of the $\eta$-function and associated math and physics, I invite people to expand on any of the items with specifics (e.g., exact formulas), more references, and/or insightful commentaries (e.g., what you believe are important aspects of the references).
Other appearances: In Gliozzi's "The Infrared Limit of QCD Effective String" on pg. 14; Panero's "A numerical study of confinement in compact QED" on pg. 4; Zahed's "Holographic Pomeron and Primordial Viscosity" on pg. 1; Caselle and Pinn's "On the Universality of Certain Non-Renormalizable Contributions in Two-Dimensional Quantum Field Theory" on pg. 3; Billo, Casselle, and Pellegrini's "New numerical results and novel effective string predictions for Wilson loops" on pg. 6 and 15; and Basar, Kharzeev, Yee, and Zahed's "Holographic Pomeron and the Schwinger Mechanism" on pg. 7.