Yes. See Theorem 7 of the paper A survey on Seifert fibre space conjecture by Jean-Philippe Préaux. One has to understand the "Seifert spaces modulo $\mathbb{P}$" introduced by Heil and Whitten. (This is the first time I've had to think about these - life is simpler when we assume orientability!)

The answer is "no" (see below for an example) but it is almost "yes". If $M$ does not have any real projective plane boundary components then this follows from Theorem 7 of the paper A survey on Seifert fibre space conjecture by Jean-Philippe Préaux.

In general, one has to understand the "Seifert spaces modulo $\mathbb{P}$" introduced by Heil and Whitten. (This is the first time I've had to think about these - life is simpler when we assume orientability!) See the above survey paper for references.

Here is the promised example. Consider the three torus $T = \mathbb{R}^3 / \mathbb{Z}^3$. There is a $\mathbb{Z}_2$ action via the "antipodal map" $\tau$ that acts as negation on all coordinates. Note that the fixed point set of $\tau$ is $P = \{(0,0,0), (1/2, 0, 0), \ldots, (1/2, 1/2, 1/2)\}$. Let $T' = T / \tau$. So $T'$ is not a three-manifold, due to the orbifold points at $P'$, the image of $P$. If we remove small neighbourhoods of all of the points of $P'$ we obtain a three manifold $T''$ which has fundamental group $\mathbb{Z}^3 \rtimes \mathbb{Z}_2$. So (sadly), the fundmental group is neither torsion free nor one of the desired groups.