In modern treatments of statistical mechanics, the natural base is conventionally used for the Gibbs and Boltzmann entropy without careful justification. While I am aware that the properties of the Shannon entropy are invariant to the choice of base of the logarithm, I suspect that physicists might have careful theoretical justifications for this particular choice. 

>Question: Might there be a sound theory behind this convention? 

So far a couple reasons occurred to me: 

1. Stirling's log-factorial approximation: 

\begin{equation}
\ln N! \sim N \cdot \ln N - N \tag{1}
\end{equation}

which is an essential tool in statistical mechanics. 

2. The exponential function diagonalises the derivative operator: 

\begin{equation}
\forall \lambda \in (0,1), \frac{\partial}{\partial H} \exp(\lambda \cdot H) = \lambda \cdot \exp(\lambda \cdot H) \tag{2}
\end{equation}

which may be useful whenever one wants to analyse variations in the exponential of entropy. The advantage of the exponential of entropy is that it is parameterisation invariant [as pointed out by Tom Leinster on a related question](https://mathoverflow.net/questions/398217/geometric-interpretations-of-the-exponential-of-entropy).  

As there might be subtle reasons I have ignored, any useful references on this question are more than welcome. 

## References: 
1. von Neumann, John (1932). Mathematische Grundlagen der Quantenmechanik (Mathematical foundations of quantum mechanics) Princeton University Press., . ISBN 978-0-691-02893-4.

2. Landau and Lifshitz. Statistical Physics. Butterworth-Heinemann. 1980. 

3. David J.C. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press 2003.