On the OP request, here is the plot of first 10000 partial sums.
Following Terry Tao's suggestion, here is the plot of ($n$th partial sum) $+2^{\frac32}/\sqrt{\pi n}$ for $n$ up to one million:
The thick line in the beginning actually consists of high frequency oscillations - in the range up to 2000 it looks like this:
(I hope there are no rounding artifacts, I calculated everything with 100 decimal digits precision)