Using math software to show that the following groups are infinite? I would like to show that the following finitely presented group in 3 generators $P, Q, R$ is infinite in certain cases:
$$P^p, Q^q, R^r, (PQ)^2, (QR)^2, (PQR)^2, (QR^{r/2+1})^a (RQR^{r/2})^b$$
For example when $p=3, q=4, r=4$ and $a=5$, $b=1$ or when $p=5, q=3, r=6$ and $a=2, b=1$.
I gave it to gap via F:=FreeGroup("P","Q","R"); G:=F/[rels]; and tried IsInfinite but that never finished. Then I tried F:=FreeMonoid("P","Q","R"); G:=F/[[rel,Identity(F)],...]; and ReducedConfluentRewritingSystem with a couple of different reduction orderings but that never finished either.
Are there some other methods I can use?
 A: I think it might be more sensible to ask questions like this on the GAP forum, but I can help you with the first of your examples. I haven't tried the second. A standard method of trying to prove finitely presented groups infinite is to look for subgroups of low index and compute their abelian invariants, hoping that you will find a subgroup with an infinite abelian quotient.
To look for subgroups of low index, there is the LowIndexSubgroups command, which finds all subgroups up to a given index, and also GQuotients, which looks for homomorphisms onto specific groups. In your first example, the first approach found me subgroups up to index $12$, and the one  of index $12$ turned out to be the second derived group. I then tried systematically looking for homomorhisms of $G''$ onto simple groups. I found homomorphisms onto ${\rm PSL}(2,13)$ and ${\rm PSL}(2,25)$. The kernel of the first of these has a large (and probably infinite) $2$-quotient, but no infinite abelian quotient. I was luckier with the second. Here are the actual GAP commands that prove infiniteness.
gap> F := FreeGroup(3);;
gap> P:=F.1;; Q:=F.2;; R:=F.3;;
gap> p:=3;; q:=4;; r:=4;; a:=5;; b:=1;;
gap> rels:=[P^p,Q^q,R^r,(P*Q)^2,(Q*R)^2,(P*Q*R)^2,
 > (Q*R^(r/2 + 1))^a * (R*Q*R^(r/2))^b ];;
gap> G := F/rels;
   <fp group on the generators [ f1, f2, f3 ]>
gap> D := CommutatorSubgroup(G,G);;
gap> DD := CommutatorSubgroup(D,D);;
gap> Index(G,DD);
12
gap> homs := GQuotients(DD,PSL(2,25));;
gap> K := Kernel(homs[1]);;
gap> AbelianInvariants(K);
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
  0, 0, 0 ]

