Up to a factor of logarithm, $E(x)$ E(x)$'s oscillation has an amplitude which is of the same magnitude as that of $\frac{1}{x}\left(\psi(x)-x\right)$, that is the error in the prime number theorem. Specifically $$E(x)=O\left( e^{-c(\log x)^{3/5-\epsilon}}\right)$$ unconditionally, and $$E(x)=O\left( x^{-\frac{1}{2}+\epsilon}\right).$$ under RH.
Proof: Let $W(x)=\pi(x)-\text{li}(x)$ be the error term in the prime number theorem. Then
$$\sum_{p\leq x}\frac{1}{p}=\int_{2}^{x}\frac{1}{t}d\left(\pi\left(t\right)\right)=\int_{2}^{x}\frac{1}{t\log t}dt+\int_{2}^{x}\frac{1}{t}dW(t).$$
Integration by parts yields
$$\int_{2}^{x}\frac{1}{t}dW(t)=\frac{W(t)}{t}\biggr|_{2}^{x}+\int_{2}^{x}\frac{W(t)}{t^{2}}dt.$$
Since
$$\int_{2}^{x}\frac{W(t)}{t^{2}}dt=\int_{2}^{\infty}\frac{\pi(t)-\text{li}(t)}{t^{2}}dt+O_{\epsilon}\left(e^{-c\left(\log x\right)^{\frac{3}{5}-\epsilon}}\right),$$
it then follows that $$\sum_{p\leq x}\frac{1}{p}=\log\log x+b+O_{\epsilon}\left(e^{-c\left(\log x\right)^{\frac{3}{5}-\epsilon}}\right),$$
where $b=\int_{2}^{\infty}\frac{\pi(t)-\text{li}(t)}{t^{2}}dt+\frac{W(2)}{2}-\log\log2.$
