The origin of the natural base in statistical mechanics

Question

In modern treatments of statistical mechanics, the natural base is conventionally used for the Gibbs and Boltzmann entropy without careful justification. While I am aware that the properties of the Shannon entropy are invariant to the choice of base of the logarithm, I suspect that physicists might have careful theoretical justifications for this particular choice.

Question: Might there be a sound theory behind this convention?

So far a couple reasons occurred to me:

1. Stirling's log-factorial approximation:

\begin{equation} \ln N! \sim N \cdot \ln N - N \tag{1} \end{equation}

which is an essential tool in statistical mechanics.

2. The exponential function diagonalises the derivative operator:

\begin{equation} \forall \lambda \in (0,1), \frac{\partial}{\partial H} \exp(\lambda \cdot H) = \lambda \cdot \exp(\lambda \cdot H) \tag{2} \end{equation}

which may be useful whenever one wants to analyse variations in the exponential of entropy. The advantage of the exponential of entropy is that it is parameterisation invariant as pointed out by Tom Leinster on a related question.

As there might be subtle reasons I have ignored, any useful references on this question are more than welcome.

References:

1. von Neumann, John (1932). Mathematische Grundlagen der Quantenmechanik (Mathematical foundations of quantum mechanics) Princeton University Press., . ISBN 978-0-691-02893-4.

2. Landau and Lifshitz. Statistical Physics. Butterworth-Heinemann. 1980.

3. David J.C. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press 2003.

Answer 1

As Matt F. points out, we could just absorb a change of base of the logarithm into the coefficient. The reason that is not convenient in physics is that we would like the same coefficient $k$ to appear in the ideal gas law $pV=NkT$.

Note that in physics entropy changes can be defined in thermodynamic rather than statistical terms, as the absorbed heat in a reversible transition ($\Delta S=\int dQ/T$). So any change in the base of the logarithm has to be compensated by a change in the coefficient $k\mapsto k'$. And if we would define the entropy $S=k'\log_2 W$ using the base-2 logarithm, the ideal gas law would read $pV\ln 2=Nk'T$.

Answer 2

Something Gian-Carlo Rota made a big deal about emphasizing was that there is a real difference between the entropy of discrete and continuum probability distributions. For a discrete distribution, the Shannon entropy with $\sum p_{n}\log_{2}p_{n}$ is clearly dimensionless. However, the Boltzmann entropy, with $\int dx\, f(x)\ln f(x)$, is a horrible mess in terms of units. Fortunately, in quantum mechanics, Planck's constant $\hbar$ swoops in to fix the unit problem. (If you've never seen it before, it can seem like magic how such a fundamental-looking problem just evaporates.) So for real gasses (and other statistical physics systems), the problem goes away, but Rota emphasized that if we want to use entropy to describe other kinds of continuous distributions, the problem may still be there.

However, the quantum mechanical entropy only really cleans itself nicely if the natural logarithm is used. You can define an entropy using any basis $a$ for the logarithm of the available phase space $S=k\log_{a}\Omega$; that just means changing the definition of the Boltzmann constant $k$. For a lot of expressions, changing $k$ is all that's needed, but that logarithm defining $S$ is not actually the only place where the constant $e$ enters in the statistical mechanics of ideal gasses. It also shows up in the Sacker-Tetrode equation, $$S=Nk\ln\left[\frac{Ve^{5/2}}{N\hbar^{3}}\left(\frac{mkT}{2\pi}\right)^{3/2}\right],$$ and there is no way to make the $e$ in the argument go away entirely. Moreover, there is no way to get that weird factor of $e^{5/2}(2\pi)^{-3/2}\hbar^{-3}$ without the quantum mechanical treatment I alluded to above. (This is in contrast to the dependences on $N$, $k$, $V$, $T$, and $m$, which can be determined from purely classical statistical mechanics.)

Now, maybe one can say that $e$ is just showing up in that argument because, like the $\pi$ that is also present, $e$ is simply a ubiquitous constant that pops up all sorts of places that don't seem appropriate. However, it is clear that this is one instance where the natural logarithm gives an expression that is manifestly nicer than what you would get using a different base.