You haven't quite given the full strength of Swarup's result, which is what makes it so useful: moreover, $H_2$ and $tHt^{-1}$ are conjugate into $J_2$. The geometric interpretation is that a natural square-complex realisation of $G$ has a free face.
There is no analogous result of the full statement of Swarup's theorem when the edge and vertex groups are of infinite rank. For example, take $G=\langle a,t\rangle$ and consider the retraction to $\mathbb{Z}=\langle t\rangle$ given by killing $a$. This gives us the semidirect product decomposition
$G=J\rtimes \mathbb{Z}$
where $J$ is free on $\{t^nat^{-n}\mid n\in\mathbb{Z}\}$. We can think of this as an HNN decomposition with vertex and both edge groups equal to $J$. Thus, we can't have one of the edge groups conjugate into a proper free factor of $J$, so the analogue of Swarup's theorem doesn't hold.
However, off the top of my head, I don't know if your weaker version of Swarup might work in the infinite rank case. (Note: In an earlier version I said it didn't, but I had misinterpreted what you wrote slightly. Apologies!)
By the way, there are many modern generalisations of Swarup's theorem. You might be interested in this paper of Diao and Feighn, which explains the square-complex point of view.