Look at Section 18.4 in Ireland-Rosen "A classical introduction to modern number theory". Note $p\equiv 1 \pmod{4}$ and $p = \pi\cdot \bar{\pi}$ with $\pi = 1- iu \equiv 1 \pmod{2+2i}$. Let $\lambda: \mathbb{F}_p \to \langle i \rangle$ be the character of order $4$, which is equal to $(\tfrac{\cdot}{p})_4$. Theorem 5 there shows that $$a_p = \overline{\lambda(-a)} \, \pi + \lambda(-a)\,\bar{\pi}$$ where $a_p$ is the negative of your expression $\# E(\mathbb{F}_p)-p-1$. If $\lambda(-a)=1$ then $a_p= 2$. If $\lambda(-a)=i$ then $a_p=-2u$. If $\lambda(-a) = -1$ then $a_p=-2$. If $\lambda(-a)=-1$ then $a_p=-2u$.
So, yes $a_p\in\{-2u,-2,2,2u\}$.