Is there a way to see what the cup product of terms from $H^1$ is in $H^2$?
By the universal coefficient theorem there is an isomorphism
$$
H^2(G;k) \cong Hom_k(H_2(G;k),k)
$$
and an embedding
$$
H_2(G;\mathbb{Z}) \otimes k \hookrightarrow H_2(G;k).
$$
Identify $H_2(G;\mathbb{Z})=[F,F] \cap R / [F,R]$ and $H^2(G;k)=Hom_k(H_2(G;k),k)$.
Let $\alpha, \beta \in H^1(G;k)=Hom(G,k)$. Then the restriction of $\alpha \cup \beta$ to $H_2(G;\mathbb{Z}) \otimes k$ is given as follows:
For $x=\Pi_j [x_j,y_j] \in [F,F] \cap R$ and $\hat{x}=x$ mod $[F,R]$ we have
$$
(\alpha \cup \beta)(\hat{x} \otimes 1) = \sum_j [\alpha(\overline x_j)\beta(\overline y_j)-\alpha(\overline y_j)\beta(\overline x_j)],
$$
where $\overline x_j \in G$ denotes the image of $x_j \in F$ under the map $F \to G$.
This follows from the correspondence between elements of $[F,F] \cap R$ and 2-cycles of the bar resolution of $G$ that is described in exercise 4c) in II.5 of "Brown: Cohomology of Groups".
Can I get elements of $H^2,H^3,H^4$ this way?
Of course. But the description depends on the representation of the cohomology group. For example, if $H^n$ is represented by cocycles of the bar resolution, then the cup product $\alpha_1 \cup ... \cup \alpha_n $ of $\alpha_1, ..., \alpha_n \in Hom(G,k))$is represented by the cocyle $[g_1|...|g_n] \to \alpha_1(g_1)...\alpha_n(g_n).$