Problem 1.2i in Atiyah-Macdonald states that for any ring $A$ and $f=a_0+\cdots+a_nx^n\in A[x]$, $f$ is a unit $\iff$ $a_0$ is a unit in $A$, and the $a_i$ are nilpotent.

Note that $A=\mathbb{Z}/n\mathbb{Z}$ has nilpotents $\iff$ $n$ is not squarefree.

In your specific case, $\mathbb{Z}/4\mathbb{Z}$ has 0 and 2 as nilpotents, and 1 and 3 as units, hence the units of $(\mathbb{Z}/4\mathbb{Z})[x]$ are, as you predicted, exactly the polynomials of the form $\pm1+2xp$ for $p\in(\mathbb{Z}/4\mathbb{Z})[x]$.

Problem 1.2i in Atiyah-Macdonald states that for any ring $A$ and $f=a_0+\cdots+a_nx^n\in A[x]$, $f$ is a unit $\iff$ $a_0$ is a unit in $A$, and the $a_i$ are nilpotent.

Note that $A=\mathbb{Z}/n\mathbb{Z}$ has nontrivial nilpotents $\iff$ $n$ is not squarefree.

In your specific case, $\mathbb{Z}/4\mathbb{Z}$ has 0 and 2 as nilpotents, and 1 and 3 as units, hence the units of $(\mathbb{Z}/4\mathbb{Z})[x]$ are, as you predicted, exactly the polynomials of the form $\pm1+2xp$ for $p\in(\mathbb{Z}/4\mathbb{Z})[x]$.