This question grew out of this post.
Question: Is there a finitely generated, infinite, residually finite group such that every finite index subgroup has $p$-power index for a fixed prime $p$?
The $p$-adic integers $\mathbb Z_p$ give an example of a non-finitely generated such group. This is not entirely obvious, but follows from a result of Jean-Pierre Serre, which states that every finite index subgroup of $\mathbb Z_p$ is closed.
EDIT: André Henriques has pointed out that Rostislav Grigorchuk constucted a finitely generated, infinite, residually finite $2$-group. All finite quotients of this group have to be $2$-groups. Colin Reid asked in a comment whether there is a torsionfree example. So let me take the freedom to extend my question:
Question: Is there a torsionfree example?