10
$\begingroup$

In modern treatments of statistical mechanics, the natural base is conventionally used for the Gibbs and Boltzmann entropy without careful justification. While I am aware that the properties of the Shannon entropy are invariant to the choice of base of the logarithm, I suspect that physicists might have careful theoretical justifications for this particular choice.

Question: Might there be a sound theory behind this convention?

So far a couple reasons occurred to me:

  1. Stirling's log-factorial approximation:

\begin{equation} \ln N! \sim N \cdot \ln N - N \tag{1} \end{equation}

which is an essential tool in statistical mechanics.

  1. The exponential function diagonalises the derivative operator:

\begin{equation} \forall \lambda \in (0,1), \frac{\partial}{\partial H} \exp(\lambda \cdot H) = \lambda \cdot \exp(\lambda \cdot H) \tag{2} \end{equation}

which may be useful whenever one wants to analyse variations in the exponential of entropy. The advantage of the exponential of entropy is that it is parameterisation invariant as pointed out by Tom Leinster on a related question.

As there might be subtle reasons I have ignored, any useful references on this question are more than welcome.

References:

  1. von Neumann, John (1932). Mathematische Grundlagen der Quantenmechanik (Mathematical foundations of quantum mechanics) Princeton University Press., . ISBN 978-0-691-02893-4.

  2. Landau and Lifshitz. Statistical Physics. Butterworth-Heinemann. 1980.

  3. David J.C. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press 2003.

$\endgroup$
2
  • 11
    $\begingroup$ Why does the choice of base matter? We can write Boltzmann’s equation as $S=k\ln W$ or $S=k’ \log_2 W$ with $k’=k\ln 2$. $\endgroup$
    – user44143
    Feb 23, 2022 at 1:34
  • 2
    $\begingroup$ @MattF. In brief, Occam's razor. If you do Statistical Mechanics calculations, within an hour you might use Stirlings log-factorial approximation 20 times. So from the vantage point of computational complexity, the natural base saves us a $\mathcal{O}(N)$ cost. But, the ultimate reason may be more subtle than this. Hence my question. $\endgroup$ Feb 23, 2022 at 10:29

2 Answers 2

13
$\begingroup$

As Matt F. points out, we could just absorb a change of base of the logarithm into the coefficient. The reason that is not convenient in physics is that we would like the same coefficient $k$ to appear in the ideal gas law $pV=NkT$.

Note that in physics entropy changes can be defined in thermodynamic rather than statistical terms, as the absorbed heat in a reversible transition ($\Delta S=\int dQ/T$). So any change in the base of the logarithm has to be compensated by a change in the coefficient $k\mapsto k'$. And if we would define the entropy $S=k'\log_2 W$ using the base-2 logarithm, the ideal gas law would read $pV\ln 2=Nk'T$.

$\endgroup$
2
  • 1
    $\begingroup$ This is a great point. A related insight that occurred to me is that the notion that the base is arbitrary presumes that Boltzmann's constant is arbitrary. However, the fundamental constants aren't defined independently of each other and they define the epistemic limits of modern theoretical physics. Perhaps a metrologist could present a theoretical justification along these lines. $\endgroup$ Feb 23, 2022 at 12:01
  • 1
    $\begingroup$ indeed, instead of redefining Boltzmann's constant we could redefine the unit of temperature; Kelvin and Celsius would then no longer be related by a simple shift of the zero-point; $\endgroup$ Feb 23, 2022 at 12:08
4
$\begingroup$

Something Gian-Carlo Rota made a big deal about emphasizing was that there is a real difference between the entropy of discrete and continuum probability distributions. For a discrete distribution, the Shannon entropy with $\sum p_{n}\log_{2}p_{n}$ is clearly dimensionless. However, the Boltzmann entropy, with $\int dx\, f(x)\ln f(x)$, is a horrible mess in terms of units. Fortunately, in quantum mechanics, Planck's constant $\hbar$ swoops in to fix the unit problem. (If you've never seen it before, it can seem like magic how such a fundamental-looking problem just evaporates.) So for real gasses (and other statistical physics systems), the problem goes away, but Rota emphasized that if we want to use entropy to describe other kinds of continuous distributions, the problem may still be there.

However, the quantum mechanical entropy only really cleans itself nicely if the natural logarithm is used. You can define an entropy using any basis $a$ for the logarithm of the available phase space $S=k\log_{a}\Omega$; that just means changing the definition of the Boltzmann constant $k$. For a lot of expressions, changing $k$ is all that's needed, but that logarithm defining $S$ is not actually the only place where the constant $e$ enters in the statistical mechanics of ideal gasses. It also shows up in the Sacker-Tetrode equation, $$S=Nk\ln\left[\frac{Ve^{5/2}}{N\hbar^{3}}\left(\frac{mkT}{2\pi}\right)^{3/2}\right],$$ and there is no way to make the $e$ in the argument go away entirely. Moreover, there is no way to get that weird factor of $e^{5/2}(2\pi)^{-3/2}\hbar^{-3}$ without the quantum mechanical treatment I alluded to above. (This is in contrast to the dependences on $N$, $k$, $V$, $T$, and $m$, which can be determined from purely classical statistical mechanics.)

Now, maybe one can say that $e$ is just showing up in that argument because, like the $\pi$ that is also present, $e$ is simply a ubiquitous constant that pops up all sorts of places that don't seem appropriate. However, it is clear that this is one instance where the natural logarithm gives an expression that is manifestly nicer than what you would get using a different base.

$\endgroup$
1
  • $\begingroup$ Respectfully, I disagree that you can't get rid of e^(5/2) for Sackur-Tetrode? In the derivation, Nk*5/2 emerges from Equipartition theorem which yields Nk*3/2 term (from translational internal energy), and Stirlings approximation on q_trans, which gets Nk. To make it into the more compact form, we put it to base e and fit it inside the natural logarithm. If one had chosen a different log base, e.g. log2, we would've put 2^(5/2) inside it instead. I do think e and ln have mathematical and natural significance, but that is my understanding of the emergence of e here. $\endgroup$
    – Eliot Behr
    Feb 2, 2023 at 20:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.