Let $F_n$ be a free group of rank $n\ge 2$, and $F_n\rightarrow G$ a surjection with $G$ finite. Let $R$ be the kernel. From this, we get an action of $G$ on the abelianization $R/R'$ (a free abelian group of rank $|G|(n-1)+1$).
What is the structure of $R/R'$ as a $G$-representation? I seem to remember reading somewhere that this is just $\mathbb{Z}[G]^{n-1}\oplus\mathbb{Z}$. Is this right? I'm looking for a reference or a proof.
Your memory is correct, at least if you replace $\mathbb{Z}$ with a field $k$ of characteristic $0$. This is a theorem of Gaschütz. See
W. Gaschütz, Über modulare Darstellungen endlicher Gruppen, die von freien Gruppen induziert werden, Math. Z. 60 (1954), 274–286.
One way to prove it is to think of your free group as the fundamental groups of a graph $X$ with one vertex and $n$ loops. Then $R$ is the fundamental group of a regular cover $Y$ of $X$ with deck group $G$. Since the deck group acts freely on $Y$ its cellular chain complex is a chain complex of $G$-modules that can be identified with
$$0 \rightarrow k[G]^n \rightarrow k[G] \rightarrow 0.$$
When you take homology you know you get $k$ in degree $0$, and the theorem now follows from semisimplicity.
The question has been answered over fields of characteristic $0$ but not over $\mathbb Z$. as originally asked. It turns out that the statement is never true for $G$ noncyclic. It is proved in Lemma 3.4 of Free presentations and relation modules of finite groups by J. S. Williams that if $R/[R,R]\cong A\oplus P$ with $P$ projective and $A\neq 0$ and if $G$ is noncyclic, then $A$ is a faithful $G$-module. In particular, we cannot have $R/[R,R]\cong \mathbb Z\oplus \mathbb ZG^{n-1}$. Of course, if $n=1$ and $G$ is cyclic, then obviously $R/[R,R]\cong \mathbb Z$. The proof uses the decomposition over $\mathbb Q$ and some group cohomology to deduce that $\mathbb Q\otimes A$ must contain at least one of the $\mathbb QG$ copies if $G$ is not cyclic.
I think we may be able to extend the result to fields if the characteristic of the field $F$ does not divide the order of the group $G$.
Proposed Theorem: Let $F_n$ be a free group of rank $n \geq 2$ and let $G$ be a finite group. Suppose $\varphi: F_n \rightarrow G$ is surjective, and let $R = \ker(\varphi)$. If the characteristic of the field $F$ does not divide the order of $G$, then the abelianization of $R$, denoted $R/R'$ (where $R'$ is the commutator subgroup of $R$), is isomorphic as a $G$-module to $F[G]^{n-1} \oplus F$.
Proof Sketch: We first note that since $\varphi$ is surjective, $R$ is a normal subgroup of $F_n$. By the Nielsen-Schreier Theorem, $R$ is also free, and its rank is $1 + |G|(n - 1)$ due to the index of $R$ in $F_n$ being equal to $|G|$.
The group $G$ acts on $F_n$, and hence on $R$, by conjugation. This action is well-defined on the cosets of $R/R'$ and satisfies the module axioms, making $R/R'$ a $G$-module.
By Maschke's theorem, the assumption on the characteristic of $F$ ensures that the group algebra $F[G]$ is semisimple, which means that any $F[G]$-module, in particular $R/R'$, can be decomposed into a direct sum of simple modules.
When considering the cohomology of free groups, we take into account a resolution of $F_n$ that, when restricted to $R$, implies that $H^0(R, F)$ is isomorphic to $F$ and $H^q(R, F)$ is trivial for $q \geq 1$.
The Lyndon-Hochschild-Serre spectral sequence arises from the group extension $1 \rightarrow R \rightarrow F_n \rightarrow G \rightarrow 1$ and simplifies due to the trivial higher cohomology of $R$, leading to a short exact sequence involving $H^0(G, F)$, $H^0(G, F[G])$, and $H^0(G, I)$, where $I$ is the augmentation ideal in $F[G]$.
Finally, the elements of the augmentation ideal $I$ in $F[G]$ have coefficients that sum to zero. The abelianization $R/R'$ interacts with this ideal in a way that allows us to identify parts of $R/R'$ with non-trivial $G$ action. The short exact sequence mentioned earlier helps us analyze this interaction, revealing that the image of the first map is the $F$ summand arising from the trivial action, and the cokernel, $H^0(G, I)$, corresponds to elements of $R/R'$ whose images under the $G$ action lie in the augmentation ideal. These elements form the basis of copies of $F[G]$ within $R/R'$, completing the isomorphism.