In "Smooth Homology Spheres and their Fundamental Groups" Kervaire proves that i) every 4-dimensional homology sphere bounds a contractible smooth manifold, and ii) for $d \geq 5$, every $d$-dimensional homology sphere bounds a contractible smooth manifold, after perhaps changing it by connect-sum with an exotic sphere.
Hence, let $\Sigma^d$ be any homology sphere of dimension $d \geq 4$, and modify it if necessary by connected-sum with a homotopy sphere so that it bounds a contractible manifold $\Delta^{d+1}$. Then $$N := \Delta \cup_\Sigma \Delta$$ has an obvious (smooth) involution with fixed set $\Sigma$. Furthermore, this is easily seen to be a homotopy sphere (by Mayer--Vietoris and Seifert--van Kampen). It may not be diffeomorphic to $S^{d+1}$, but as it has an orientation-reversing involution it will have order $2$ in the group $\Theta_{d+1}$ of exotic spheres. When $d+1=5,6, 12, 13$ this group is trivial or $\mathbb{Z}/3$, and so $N$ must in fact be diffeomorphic to $S^{d+1}$ in these cases.
Thus $\Sigma$ need not be a homotopy sphere in general