Is Li(x) the best possible approximation to the prime-counting function?

Question

The Prime Number Theorem says that $\lim_{n \to \infty} \frac{\pi(n)}{\mathrm{Li}(n)} = 1$, where $\mathrm{Li}(x)$ is the Logarithm integral function $\mathrm{Li}(x) = \int_2^x \frac{1}{\log(x)}dx$. It is also the case that $\lim_{n \to \infty} \frac{\pi(n)}{n/\log(n)} = 1$, as $\mathrm{Li}(x)$ and $\log(x)$ are asymptotically equivalent. However it seems that $\mathrm{Li}(n)$ is a better approximation to $\pi(n)$ (the Mathworld article states that this has been proven, I don't know in what precise sense).

There are also results for the absolute difference between $\pi(n)$ and $\mathrm{Li}(n)$; for instance $\pi(n) - \mathrm{Li}(n)$ is known to change sign infinitely often. We also know that the Riemann Hypothesis is equivalent to $$ \pi(n) - \mathrm{Li}(n) \in O \left (\sqrt{n} \log(n) \right ).$$

In addition, Riemann showed that we have $$\pi(n) = \mathrm{Li}(n) - \frac{1}{2} \mathrm{Li}\left ( \sqrt{n} \right ) - \sum_{\rho} \mathrm{Li}(x^\rho) + \text{lower order terms}$$ where $\rho$ runs over all the nontrivial zeroes of the Riemann zeta function.

Question: Is there a sense in which $\mathrm{Li}(n)$ is the best possible approximation to $\pi(n)$? Ideally, there would be some Bohr-Mollerup type theorem: $\mathrm{Li}(n)$ is uniquely characterized as being a good approximation to $\pi(n)$ which has some properties, such as analyticity and negative second derivative. Probably $\mathrm{Li}(n)$ isn't the best possible, for instance $\mathrm{Li}(n) - \frac{1}{2} \mathrm{Li}\left ( \sqrt{n} \right )$ might be better? Riemann also suggested $$\sum_{n \geqslant 1} \frac{\mu(n)}{n} \mathrm{Li} \left (x^\frac{1}{n}\right ),$$ where $\mu(n)$ is the Möbius function.

Answer 1

This is an extension of Emerton's comment. From my paperback 1990 reprint of "The Distribution of Prime Numbers" by A.E. Ingham (1932). Ingham is discussing your formula with the Mobius function, pages 105-106:

"Considerable importance was attached formerly to a function suggested by Riemann as an approximation to $\pi(x)$...This function represents $\pi(x)$ with astonishing accuracy for all values of $x$ for which $\pi(x)$ has been calculated, but we now see that its superiority over $\mathrm{Li}(x)$ is illusory...and for special values of $x$ (as large as we please) the one approximation will deviate as widely as the other from the true value."

In the next paragraph he shows that RH implies "And we can see in the same way that the function $\mathrm{Li}(x) - \frac{1}{2} \mathrm{Li}(x^{\frac{1}{2}})$ is `on the average' a better approximation than $\mathrm{Li}(x)$ to $\pi(x)$; but no importance can be attached to the latter terms in Riemann's formula even by repeated averaging."

Similar material is in Edwards pages 34-36 where he discusses relative size of terms, as Emerton comments. Edwards does not bother with comparing different approximations under the assumption of RH.

EDIT: I remember clearly that the error estimate of de la Vallee Poussin (the second comment of Thomas Bloom) shows that $\mathrm{Li}(x)$ is a better approximation for $\pi(x)$ than any rational function of $x$ and $\log x.$ I am not finding a reference that uses those exact words. However, Edwards comes pretty close in section 5.4, page 87. Here he displays the usual asymptotic expansion, formula (5), and points out that $\mathrm{Li}(x)$ is eventually a better approximation for $\pi(x)$ than the asymptotic expansion stopped at a finite number of terms. I think it likely that this also establishes the claim for any rational function of $x$ and $\log x.$

Answer 2

Whether for a finite set $\mathcal{R}$ of roots the approximation $$ \pi(x)\approx\mathrm{Li}(x)-\frac{1}{2}\mathrm{Li}(x^{1/2})-\sum_{\rho\in\mathcal{R}}\mathrm{Li}(x^\rho) $$ is "on average" better then $\pi(x)\approx\mathrm{Li}(x)-\frac{1}{2}\mathrm{Li}(x^{1/2})$ depends on the precise notion of average. Since $x^\rho=x^\sigma e^{i\gamma\log x}$, it might appear more natural to consider $\pi(e^t)$ instead of $\pi(x)$. Things are somewhat easier for $\Psi$, and there we get under RH $$ \lim_{T\rightarrow\infty}\frac{1}{T}\int_2^T\left(\Psi(e^t)-\Big(e^t-\sum_{\rho\in\mathcal{R}}\frac{e^{t\rho}}{\rho}\Big)\right)^2\frac{dt}{e^{t/2}} = \sum_{\rho\not\in\mathcal{R}}\frac{1}{|\rho|^2} $$ (insert explicit formula, throw away terms of small order, square out, interchange integral and summation). So after the proper rescaling including some roots actually does reduce the mean error.