There are examples analogous to Row's in dimensions $n>2$ which are orientable when $n$ is even. I'll give a bit of motivation for the example at the end.
Consider the action of the group $G= \mathbb{Z}^n\rtimes \{\pm I\}=\{ x \mapsto \pm x+ m, m\in \mathbb{Z}^n\}$ on $\mathbb{R}^n$. The subgroup $G_{m/2}=\{x,-x+m\}, m\in \mathbb{Z}^n $ is the stabilizer of $m/2\in \frac12\mathbb{Z}^n$. Remove open balls of radius $r<1/4$ about the lattice points $\frac12\mathbb{Z}^n$ to get the simply-connected manifold $V= \mathbb{R}^n -\mathcal{N}_r(\frac12\mathbb{Z}^n)$. When $n$ is even, $V$ admits an orientation which is $G$-invariant. This induces an orientation on $\partial V$. Since $G_{m/2}$ acts as the antipodal map on the sphere of radius $r$ about $m/2$, the quotient $W'=V/G$ will be a manifold with $2^n$ boundary components (corresponding to $\frac12\mathbb{Z}^n/\mathbb{Z}^n \cong (\mathbb{Z}/2\mathbb{Z})^n$) each of which is homeomorphic to $\mathbb{RP}^{n-1}$. The fundamental group of each boundary component will correspond to some $G_{m/2}$ up to conjugacy. $W'$ has a 2-fold cover $V/\mathbb{Z}^n$ which is homeomorphic to $T^n$ punctured at $2^n$ balls.
Take the $2^n$ boundary components of $W'$, and glue them together in pairs, so that when $n$ is even, the induced orientations get reversed, to get a manifold $W$. For concreteness, let's say that we identify the boundary components corresponding to $m/2+\mathbb{Z}^n$ and $m/2+\frac12^n +\mathbb{Z}^n$, inducing a homomorphism $\alpha_m:G_{m/2}\to G_{m/2+\frac12^n}$. In even dimensions, $W$ will be orientable. Since $\pi_1W'= G$, and the subgroup of a boundary component corresponding to the coset $m/2+\mathbb{Z}^n$ will be conjugate to $G_{m/2}$, we see that $\pi_1 W = G \ast_{m\in 0\times\{0,1\}^{n-1}} \alpha_m$ is a multiple HNN extension by the isomorphisms pairing the subgroups. Each HNN extension will introduce a new group element $t_m$ along with a relation of the form $t_mG_{m/2}t_m^{-1}=G_{m/2+\frac12^n}$. So we can give a relative presentation for the fundamental group as
$$\pi_1 W \cong \langle G, t_m | t_mG_{m/2}t_m^{-1}=G_{m/2+\frac12^n}, m\in 0\times\{0,1\}^{n-1}\rangle.$$ Note that there is some choice here of subgroup representative up to conjugacy which does not affect the overall group isomorphism type.
The claim is that $\pi_1 W$ does not split as a free product. This follows from the Kurosh subgroup theorem, and will be proved below.
Now suppose that $W$ is a non-trivial connect sum $W= W_1 \# W_2$. Then $\pi_1(W)=\pi_1(W_1)\ast \pi_1(W_2)$ by the Seifert-van Kampen theorem. Since $\pi_1(W)$ is not a non-trivial free product, that means that $\pi_1(W_1)=1$ (possibly after reindexing).
We need to show that $W_1'=W_1\backslash D^n$ is homeomorphic to the $n$-ball, and hence $W_1$ is the $n$-sphere. $W_1'$ lifts to the double cover of $W$ coming from the homomorphism $\pi_1(W)\to \mathbb{Z}/2\mathbb{Z}$, which is homeomorphic to $T^n \#( S^{n-1}\times S^1)^{\# 2^{n-1}}$ (a non-prime manifold). In turn, $W_1'$ lifts to the universal cover of this manifold which is a submanifold of $\mathbb{R}^n$ (because it is an infinite connect sum of $\mathbb{R}^n$s). Hence $W_1'$ is an $n$-ball by the Schoenflies Theorem, and we see that $W$ is irreducible and orientable when $n>2$ is even.
Now let's see why $\pi_1 W$ is freely indecomposable. Suppose that $\pi_1 W=A\ast B$. Since $ G < \pi_1 W$ is freely indecomposable, by the Kurosh subgroup theorem $G$ is conjugate to a subgroup of $A$ or $B$, let's say $A$. Moreover, the group $H=\pi_1 W/ \ll \mathbb{Z}^n \gg$ obtained by killing $\mathbb{Z}^n$ will be isomorphic to $\mathbb{Z}/2\mathbb{Z}\times \mathbb{Z}^{\ast 2^{n-1}}$ since $G/\mathbb{Z}^n\cong \mathbb{Z}/2\mathbb{Z}$. Thus we see that the image $\overline{A}$ of $A$ in $H$ will contain $\mathbb{Z}/2\mathbb{Z}$, and hence will be non-trivial. Moreover, the quotient $H$ will split as a free product $\overline{A}\ast B$. However, $\mathbb{Z}/2\mathbb{Z}\times \mathbb{Z}^{\ast 2^{n-1}}$ is not a free product, since it has a non-trivial center, a contradiction.
Motivation
If a finitely generated group $G$ splits as a free product, then any Cayley graph for $G$ (associated to a finite generating set) has more than one end. If $G< G'$ is finite index, then the Cayley graphs of $G$ and $G'$ are almost equivalent (quasi-isometric), in fact a Cayley graph for $G$ can be obtained from one for $G'$ by collapsing some finite trees equivariantly (this is basically the Reidemeister-Schreier method).
Hence if $G$ has more than one end, so does $G'$.
Now a theorem of Stallings implies that if a group $G'$ has more than one end, then $G'$ is a graph of groups with finite edge groups. Thus, in this example, we found a manifold whose fundamental group is an HNN extension over $\mathbb{Z}/2\mathbb{Z}$ subgroups, but itself is not a free product. But it has an index 2 subgroup that does split as a free product.
If an $n$-manifold $M$ is a connect sum, and $\pi_k(M)=0$ for $k < n-1$, then a similar argument shows that $\pi_1(M)=A\ast B$ is a non-trivial free product. So any manifold finitely covered by such an $M$ will have fundamental group splitting over a finite group. One can probably find many more examples with such properties. I don't know how to find an example which is a connect sum with simply-connected summands, but finitely covers a prime manifold.