When are isomorphic copies of the base group in an $HNN$-extension subgroups of the base group (up to conjugacy)?

Question

Let $G=\langle H, t; A^t=B\rangle$ by an $HNN$-extension of $H$, $A$ and $B$ isomorpic subgroups of $H$ where conjugation by $t$ induces the isomorphism.

Assuming $H$ is a finite group it is a well-known consequence of Britton's Lemma that if $\hat{H}$ is a subgroup of $G$ with $\hat{H}\cong H$ then $\hat{H}$ is conjugate to $H$ in $G$.

I was wondering under what other conditions on $H$ this result would hold. Specifically,

What conditions can we place on $H$ such that if $\hat{H}$ is a subgroup of $G$ with $\hat{H}\cong H$ then $\hat{H}$ is conjugate to a subgroup of $H$.

Answer 1

If H is a finitely generated torsion group, or more generally has Serre's property FA (fixed point property on trees) then what you want is true. Indeed H will have a fixed point in the Bass-Serre tree of the HNN extension. Since all vertex stabilizers are conjugate on this case, you are done.

Answer 2

Property FA is equivalent to your property for groups $H$ that do not decompose as a free product. Indeed, suppose that your property holds but $H$ acts non-trivially on a simplicial tree. Then $H$ decomposes as a non-trivial amalgamated product $A*_CB$ (it is either that or an HNN extension which your property rules out). That is $H$ is not conjugate to a subgroup of either $A$ or $B$. Consider the free product $F=A*B$ and an HNN extension $E$ of $F$ conjugating two copies of $C$ there (one in $A$ and one in $B$) with free letter $t$. Then $A*_CB$ is isomorphic to the subgroup of $E$ generated by $tAt^{-1}$ and $B$ (it is proved in Lyndon and Schupp). Hence $H$ is a subgroup of $E$. By your assumption, $H$ is a subgroup of $F$. Hence by Kurosh's theorem $H$ is a non-trivial free product.

For groups that are free products, the properties are most probably also equivalent. Here is an idea. Consider $H=A*B$. Take a factor-group $H'$ of $H$ containing $A, B$, so that $H'$ does not contain a copy of $H$. One can assume also that all subgroups that are not conjugate to subgroups from, $A$ or $B$ are cyclic. That can be done by a result from Olshanskii's book "Geometry of defining relations in groups". Now consider the HNN extension $U$ of $H'$ where the free letter $t$ centralizes $B$: $tbt^{-1}=b, b\in B$. Then $H=A*B$ should be isomorphic to the subgroup of $U$ generated by $tAt^{-1}$ and $B$. This needs to be checked of course.

Update. If both $A,B$ are of order 2, Olshanskii's method does not work. But in that case, one can take the HNN extension of the symmetric group $S_3$ instead. In general, Olshanskii's method works if $A$ and $B$ do not contain involutions.