Given a free group $F$ on $d$ generators and a normal subgroup $H$ of $F$ whose index is finite of prime power order, is there a systematic way to find the numbers of generators of $H/[H,F]$ and of $H/[H,F]H^p$?

This is really a cohomological question and has a simple cohomological answer. Recall that if a group $G$ acts on an abelian group $M$, then $M_G$ denotes the coinvariants of the action, that is, the quotient of $M$ by the subgroup generated by $\{\text{$mg(m)$ $$ $m \in $M, $g \in G$}\}$. The group $F$ acts on $H$ by conjugation, and thus there is an induced action of $F$ on $H_1(H)$. Key Observation : $H/[H,F] \cong (H_1(H;\mathbb{Z}))_F$ and $H/[H,F]H^p \cong (H_1(H;\mathbb{Z}/p))_F$. Indeed, we have $H/[H,H] \cong H_1(H;\mathbb{Z})$ and $H/[H,H]H^p \cong H_1(H;\mathbb{Z}/p)$ by definition, and quotienting by $[H,F]$ just kills off the $F$action. The other needed ingredient is the 5term exact sequence in group homology. Given a short exact sequence $$1 \longrightarrow A \longrightarrow B \longrightarrow C \longrightarrow 1$$ of groups and a ring of coefficients $R$, this 5term exact sequence takes the form $$H_2(B;R) \longrightarrow H_2(C;R) \longrightarrow (H_1(A;R))_B \longrightarrow H_1(B;R) \longrightarrow H_1(C;R) \longrightarrow 0.$$ Letting $Q = F/H$, we will apply this to the short exact sequence $$1 \longrightarrow H \longrightarrow F \longrightarrow Q \longrightarrow 1.$$ The key simplification that occurs is that $H_2(F;R) = 0$ since $F$ is free. We thus get exact sequences $$0 \longrightarrow H_2(Q;\mathbb{Z}) \longrightarrow H/[F,H] \longrightarrow H_1(F;\mathbb{Z}) \longrightarrow H_1(Q;\mathbb{Z}) \longrightarrow 0$$ and $$0 \longrightarrow H_2(Q;\mathbb{Z}/p) \longrightarrow H/[F,H]H^p \longrightarrow H_1(F;\mathbb{Z}/p) \longrightarrow H_1(Q;\mathbb{Z}/p) \longrightarrow 0.$$ If you understand $Q$ enough to calculate its first and second homologies, these short exact sequences allow you to determine $H/[F,H]$ and $H/[F,H]H^p$. 

