First of all, there's no need to use flat cohomology here. By Theorem III.3.9 in Milne's *Etale Cohomology*, the canonical map $H^i_{\mathrm{et}}(X, G) \rightarrow H^i_{\mathrm{fppf}}(X, G)$ is an isomorphism when $G$ is a sheaf represented by a smooth group scheme over $X$. In particular, this works for the constant group $\mathbf{Z}/p\mathbf{Z}$ and the vector group $\mathbf{G}_a$ (which is just the quasi-coherent sheaf $\mathscr{O}_X$ and thus its flat cohomology is isomorphic to its *Zariski* cohomology). For $H^1$ at least, this boils down to the statement that torsors for smooth group schemes always have sections over an étale cover.

Furthermore, the Artin-Schreier sequence is an exact sequence of étale sheaves, so we have, for any $R$:
$$
0 \rightarrow R/\mathscr{F}(R) \rightarrow H^1_{\mathrm{et}}(X_R, \mathbf{Z}/p\mathbf{Z}) \rightarrow (\ker \mathscr{F} \colon H^1_{\mathrm{Zar}}(X, \mathscr{O}_X) \rightarrow H^1_{\mathrm{Zar}}(X, \mathscr{O}_X))\otimes_k R \rightarrow 0
$$

On the other hand, $X \rightarrow \mathrm{Spec}(k)$ is proper, so we can use the proper base change theorem - note that unlike the smooth base change theorem, this works for *any* (locally constant) abelian torsion sheaf without a hypothesis that the torsion orders are units. Let $\pi \colon X \rightarrow \mathrm{Spec}(k)$ and $f \colon \mathrm{Spec}(R) \rightarrow \mathrm{Spec}(k)$ be the structure morphisms. Then the proper base change theorem tells us that the canonical map $$f^* R^1 \pi_* \mathbf{Z}/p\mathbf{Z} \rightarrow R^1 (\pi_R)_* \mathbf{Z}/p\mathbf{Z}$$ is an isomorphism. (Note that the pullback of the constant sheaf $\mathbf{Z}/p\mathbf{Z}$ from $X$ to $X_R$ is still this constant sheaf). Now the étale sheaf on the left is the constant sheaf on $\mathrm{Spec}(R)_{\mathrm{et}}$ associated to the cohomology group $$H^1_\mathrm{et}(X, \mathbf{Z}/p\mathbf{Z}) = \ker \mathscr{F} \colon H^1_{\mathrm{Zar}}(X, \mathscr{O}_X) \rightarrow H^1_{\mathrm{Zar}}(X, \mathscr{O}_X)$$

Finally, we have a Leray spectral sequence $$E_2^{p, q} = H^p_{\mathrm{et}}(\mathrm{Spec}(R), R^q (\pi_R)_* \mathbf{Z}/p\mathbf{Z}) \implies H^{p+q}_{\mathrm{et}}(X_R, \mathbf{Z}/p\mathbf{Z})$$

Now, we have an exact sequence of low-degree terms:
$$
0 \rightarrow H^1_{\mathrm{et}}(\mathrm{Spec}(R), \mathbf{Z}/p\mathbf{Z}) \rightarrow H^1_{\mathrm{et}}(X_R, \mathbf{Z}/p\mathbf{Z}) \rightarrow H^1_{\mathrm{et}}(X, \mathbf{Z}/p\mathbf{Z})^{\pi_0(\mathrm{Spec}(R))} \rightarrow H^2_{\mathrm{et}}(\mathrm{Spec}(R), \mathbf{Z}/p\mathbf{Z}) \rightarrow H^2_{\mathrm{et}}(X_R, \mathbf{Z}/p\mathbf{Z})
$$

This should let you compute $H^1(X_R, \mathbf{Z}/p\mathbf{Z})$ as long as you know a bit about $\mathrm{Spec}(R)$ and understand how $\mathscr{F}$ acts on the $k$-vector space $H^1_{\mathrm{Zar}}(X, \mathscr{O}_X)$.

(note that the above arguments do *not* work for $\mathbf{Z}/p\mathbf{Z}$ replaced by $\mu_p$, since the étale and flat cohomologies of this non-smooth group scheme differ. However, the Kummer sequence is exact in the flat topology, so you get some information. I wouldn't expect flat cohomology of something like this to play nearly as nicely with base change, and suspect it's very hard to say anything in a reasonable level of generality).