After digging through a bunch of references (especially Davenport's book and https://arxiv.org/pdf/1205.6755.pdf), I sorted out the answer I was looking for. For all $T > 0$ except at the points of discontinuity of $N(2\pi T)$, and assuming the Riemann hypothesis, one has $$N(2 \pi T) =1+ \frac{1}{\pi} \theta(2\pi T) + \frac{1}{\pi}\operatorname{Arg}\zeta\left(\frac{1}{2}+2\pi i T\right),$$ where $\operatorname{Arg}$ is the principal branch of the argument, and where $\theta(T)$ is the Riemann-Siegel theta function. One has the asymptotic relation \begin{align*} N(2 \pi T) & =T \log T-T+\frac{7}{8}+\frac{1}{\pi}\operatorname{Arg}\zeta\left(\frac{1}{2}+2\pi i T\right)+O\left(\frac{1}{T}\right) \\ & = \int_1^T \log t \, dt -\frac{1}{8}+\frac{1}{\pi}\operatorname{Arg}\zeta\left(\frac{1}{2}+2\pi i T\right)+O\left(\frac{1}{T}\right) \end{align*} and the asymptotic expansion \begin{align*} N(2 \pi T) - \frac{1}{\pi}\operatorname{Arg}\zeta\left(\frac{1}{2}+2\pi i T\right) \sim T \log T - T+ \frac{7}{8}+\frac{1}{96 \pi^2 T}+ \frac{7}{11340\pi^4 T^3}+ \frac{31}{161280 \pi^6 T^5}+\cdots \end{align*} as $T \to \infty$, where the numerators and (1/2)denominators are as in OEIS Sequences A036282 and A114721, respectively. Here is a plot of the function $N(2\pi T)$ and its smooth approximation $1+\frac{1}{\pi}\theta(2\pi T)$.
NOTE: The Riemann--von Mangoldt formula for $N(T)$ is repaired to hold also at the points of discontinuity and without assuming the Riemann Hypothesis as follows. Let $$N_0(T) = \lim_{\epsilon \to 0} \frac{N(T+\epsilon)+N(T-\epsilon)}{2}.$$ Let $$\operatorname{Arg}_0 \zeta\left ( \frac{1}{2} +iT \right) = \lim_{s\to \frac{1}{2}^+} \operatorname{Arg} \zeta\left (s+iT \right).$$ Then one has $$N_0(T) =1+ \frac{1}{\pi} \theta(T) + \frac{1}{\pi}\operatorname{Arg}_0\zeta\left(\frac{1}{2}+iT\right)$$ for all $T \geq 0$.