I'm a little confused by your augmentation map. Do you mean the augmentation map on the group ring $\mathbb{Z}[F]$? (And in the pro-$p$ case, the completed group ring $\mathbb{Z}_p[[F]]$?)

Assuming this is the case (and apologies for misinterpreting if not), there is a tremendous amount of machinery set up for dealing exactly with questions of this sort -- probably the best starting place is the phrase "pro-p Fox Differential Calculus." (And so, indeed, your intuition that solving the discrete problem turns out to provide the correct pro-$p$ analog is correct. It was Iwasawa who carefully established the fundamental analogy here. In fact, thanks to the topology of $\mathbb{Z}_p$, in some ways the pro-p Fox calculus is nicer than the discrete version.) In particular, if you filter the group ring $\mathbb{Z}_p[[F]]$ by powers of the augmentation ideal (the group-ring version of your $A$), you land upon the sequence of "dimension subgroups" of F.

These subgroups have shown up repeatedly in the analysis of pro-$p$-groups arising in the study of large Galois group arising from restricted ramification questions (as appears to be the case for you). A couple of the highlights of the theory are the work of Vogel and Morishita interpreting number-theoretic analogs of the a priori knot-theoretic notion of Milnor invariants, refined versions of Golod-Shafarevich-type inequalities, and perhaps most relevant for your question, work of Arrigoni (e.g., "On Schur $\sigma$-groups") which I think explicitly answers questions of your type. For a more fundamental reference, see Koch's "Galois theory of $p$-extensions.")

Sorry to be mostly hand-wavey -- I'm away from good references at the moment.

Do you mean the augmentation map on the group ring $\mathbb{Z}[F]$  (and in the pro-$p$ case, the completed group ring $\mathbb{Z}_p[[F]]$?) I ask only because this augmentation map comes up frequently and significantly in the study of large number-theoretic Galois groups.

Assuming this is the case (and apologies for misinterpreting if not -- hopefully the answer will still be of some use to you), there is a tremendous amount of machinery set up for dealing exactly with questions of this sort -- probably the best starting place is the phrase "pro-p Fox Differential Calculus." (And so, indeed, your intuition that solving the discrete problem turns out to provide the correct pro-$p$ analog is correct. It was Iwasawa who carefully established the fundamental analogy here. In fact, thanks to the topology of $\mathbb{Z}_p$, in some ways the pro-p Fox calculus is nicer than the discrete version.) In particular, if you filter the group ring $\mathbb{Z}_p[[F]]$ by powers of the augmentation ideal (the group-ring version of your $A$), you land upon the sequence of "dimension subgroups" of F.

These subgroups have shown up repeatedly in the analysis of pro-$p$-groups arising in the study of large Galois groups arising from restricted ramification questions (as appears to be the case for you). A couple of the highlights of the theory are the work of Vogel and Morishita interpreting number-theoretic analogs of the a priori knot-theoretic notion of Milnor invariants, refined versions of Golod-Shafarevich-type inequalities, and perhaps most relevant for your question, work of Arrigoni (e.g., "On Schur $\sigma$-groups") which I think explicitly answers questions of your type. For a more fundamental reference, see Koch's "Galois theory of $p$-extensions.")

Sorry to be mostly hand-wavey -- I'm away from good references at the moment.