Finitely generated, infinite, residually finite groups whose finite quotients are  $p$-groups. This question grew out of this post.

Question: Is there a finitely generated, infinite, residually finite group such that every finite index subgroup has $p$-power index for a fixed prime $p$?

The $p$-adic integers $\mathbb Z_p$ give an example of a non-finitely generated such group. This is not entirely obvious, but follows from a result of Jean-Pierre Serre, which states that every finite index subgroup of $\mathbb Z_p$ is closed.
EDIT: André Henriques has pointed out that Rostislav Grigorchuk constucted a finitely generated, infinite, residually finite $2$-group. All finite quotients of this group have to be $2$-groups. Colin Reid asked in a comment whether there is a torsionfree example. So let me take the freedom to extend my question:

Question: Is there a torsionfree example?

 A: Grigorchuk's group $G$ is an example of what you're looking for:
http://en.wikipedia.org/wiki/Grigorchuk_group.
Every element of $G$ has finite 2-power order, and so every finite quotient of $G$ is a 2-group.
A: Yes, there are torsion free examples. I do not know who constructed them first, but some examples can be found in papers by Grigorchuk and his co-authors. For example Bartoldi and Grigorchuk proved that a certain Fabrykowski-Gupta group $\Gamma $ has the following properties (see Propositions 6.4 and 6.5 in  arXiv:math/9911206):
(a) It is a subgroup of the automorphism group of a rooted tree.
(b) It is virtually torsion free.
(c) It satisfies the following 'congruence property': every finite index subgroup of $\Gamma $ contains a level stabilizer (i.e., the stabilizer of a level of the tree) and the index of every level stabilizer is a power of 3.
By (a) $\Gamma $ is residually finite.  By (b) there is a torsion free subgroup $K$ of finite index in $\Gamma $. Since every finite index normal subgroup of $K$ contains a finite index normal subgroup of $\Gamma $, (c) implies that every finite quotient of $K$ is a 3-group.
