Conceptual explanation for the appearance of entropy in $\frac{d}{dp}\|x\|_p$

Question

For $x\in \mathbb{R}^d$, an elementary computation yields that $$\frac{d}{dp}\log \|x\|_p =\frac{1}{p^2}\sum_{i=1}^d \frac{|x_i|^p}{\|x\|_p^p}\log \frac{|x_i|^p}{\|x\|_p^p}=-\frac{1}{p^2}\operatorname{Ent}(\mu_{x,p}),$$ where $\mu_{x,p}$ is the law of a random variable taking values in $\{1,\ldots,d\}$ that takes the value $i$ with probability proportional to $|x_i|^p$ and $\operatorname{Ent}(\mu_{x,p}) = -\sum \mu_{x,p}(i) \log \mu_{x,p}(i)$ is its entropy.

I've been curious about the following question: Is there a good/conceptual reason for an entropy to appear here?

Obviously the computation is very easy, but could I have predicted this equality without any computation?

Answer 1

As Von Neumann said "Nobody really know what entropy is." so it is quite difficult to give a conceptual reason. However I think your calculation appears and can be interpreted in the statistical-mechanics setting (so from a physicits' point of view) as the Free Energy https://en.wikipedia.org/wiki/Helmholtz_free_energy and the canonical ensemble https://en.wikipedia.org/wiki/Canonical_ensemble

Let $E$ such that $|X|=e^{- E}$. The partition function is $Z = \sum e^{-\beta E} = \|x\|^\beta_\beta$ with $p=\beta=\frac{1}{T}$ the inverse of temperature and $F=T\log(Z)$ is the free energy. Then the physics relation $$ F = \langle E \rangle- TS $$with $S$ the entropy and $\langle E\rangle$ the mean energy and this calculation $$\langle E\rangle =\frac{1}{Z}\sum e^{-\beta E}=\partial_{\beta}\log(Z) = \beta\partial_{\beta}[\frac{1}{\beta}\log(Z)]+\frac{1}{\beta}\log(Z)= \beta \partial_{\beta}[\log(\|x\|_\beta)] +F$$ gives $$\frac{1}{\beta}S = \beta \partial_{\beta}[\log(\|x\|_\beta)$$ which is what you wanted.