Expanding slightly on Greg Martin's answer, the symmetry applies across the imaginary axis as well as the real axis, so the only way a path can avoid closing up is if it crosses at most one axis at most once (and if it does cross an axis, the path -- if extended in the backward as well as forward direction -- will be mirror symmetric with respect to the axis it crosses). Note that within a quadrant the horizontal (respectively vertical) steps are alternately toward and away from the imaginary (respectively real) axis.

Intuitively the steps, on average, get larger the further you are from the axes. So if you're in the first quadrant and take a step to the right and then a step up, you expect your next step, to the left, to be somewhat larger than your previous step to the right (and similarly with the next step down, assuming you're still in the first quadrant). So in a sense the axes are exerting kind of a gravitational tug on sort of a random walk. Of course, nothing of the sort is literally going on. Still, it seems hard to imagine the stars (I mean Gaussian primes) aligning to keep a random walk going in perpetuity.

It seems more likely one might encounter closed paths that don't have any nice symmetry, such as
$$(a,b) \rightarrow (a+4,b) \rightarrow (a+4,b+8) \rightarrow (a-2,b+8) \rightarrow$$ $$(a-2,b+4) \rightarrow (a+2,b+4) \rightarrow (a+2,b+6) \rightarrow (a,b+6) \rightarrow (a,b)$$

(Sorry, if someone could replace that with a picture, that would be helpful.) As "unknown" pointed out, there are certainly closed square paths that stay in the first quadrant. There are also rectangles, such as the $2\times4$ rectangle with $8+13i$ for its lower left hand corner and the $2\times6$ one starting at $14+19i$. (I spotted these in a picture of Gaussian primes of norm less than 1000 in the paper "A Stroll Through the Gaussian Primes" by Gethner, Wagon, and Wick.) One might expect a souped-up (supped-up?) $k$-tuple conjecture to predict the existence of any closed-path pattern that isn't forbidden by the usual suspects. (Part of the souping up, though, is that not only are there primes at the specified corners, but everything else along the edges is composite.)

All in all, a nice problem -- and the spirals, reminiscient of Celtic knots, are really lovely!