I'm a little confused by your augmentation map.  Do you mean the augmentation map on the group ring $\mathbb{Z}[F]$?  (And in the pro-$p$ case, the completed group ring $\mathbb{Z}_p[[F]]$?)

Assuming this is the case (and apologies for misinterpreting if not), there is a tremendous amount of machinery set up for dealing exactly with questions of this sort -- probably the best starting place is the phrase "pro-p Fox Differential Calculus."  (And so, indeed, your intuition that solving the discrete problem turns out to provide the correct pro-$p$ analog is correct.  It was Iwasawa who carefully established the fundamental analogy here.  In fact, thanks to the topology of $\mathbb{Z}_p$, in some ways the pro-p Fox calculus is nicer than the discrete version.)  In particular, if you filter the group ring $\mathbb{Z}_p[[F]]$ by powers of the augmentation ideal (the group-ring version of your $A$), you land upon the sequence of "dimension subgroups" of F.

These subgroups have shown up repeatedly in the analysis of pro-$p$-groups arising in the study of large Galois group arising from restricted ramification questions (as appears to be the case for you).  A couple of the highlights of the theory are the work of Vogel and Morishita interpreting number-theoretic analogs of the <i>a priori</i> knot-theoretic notion of Milnor invariants, refined versions of Golod-Shafarevich-type inequalities, and perhaps most relevant for your question, work of Arrigoni (e.g., "On Schur $\sigma$-groups") which I think explicitly answers questions of your type.  For a more fundamental reference, see Koch's "Galois theory of $p$-extensions.")

Sorry to be mostly hand-wavey -- I'm away from good references at the moment.