**Given p and r:**  Pick your favorite group $G$ of order $r$.  It has a faithful transitive action on a set of size $m$ for some $m$, so you can take the semidirect product $\mathbb{F}_p^m \rtimes G$.  Alternately, if $G$ has an interesting automorphism group, pick one of the Sylow $p$-subgroups $H$ of $\text{Aut}(G)$ and you can take the semidirect product $G \rtimes H$.

**Given p and m:**  The group $GL_m(\mathbb{F}_p)$ contains a (say, Sylow) subgroup $G$ of order relatively prime to $p$, so again you can take the semidirect product $\mathbb{F}_p^m \rtimes G$.