I think that the following might be a counterexample. Take $S = \mathrm{Spec}(\mathbb{R})$ and $X = \mathbb{A}^1_{\mathbb{R}}$. I will take $S' - \mathrm{Spec}(\mathbb{C})$, so $X' = \mathbb{A}^1_{\mathbb{C}}$. Consider the point $x: \mathrm{Spec}(\mathbb{R}[t]/(t^2+1)) \to \mathbb{A}^1_{\mathbb{R}}$. Let $\mathcal{F}$ be the sheaf of $\mathcal{O}_{\mathbb{A}^1_{\mathbb{R}}}$-modules defined as follows for opens $U \subset \mathbb{A}^1_{\mathbb{R}}$:
$$ \mathcal{F}(U) = \begin{cases} 0 \; \text{ if $x \notin U$} \\ \mathbb{R}[t]_{(t^2+1)} \; \text{ if $x \in U$} \end{cases} $$
(Note that this is actually a sheaf of $\mathcal{O}_{\mathbb{A}^1_{\mathbb{R}}}$-algebras in the evident way).
The pushforward $p_*(\mathcal{F})$ can be viewed as the $\mathbb{R}$-algebra $\mathbb{R}[t]_{(t^2+1)}$. The pullback $f^*p_*(\mathcal{F})$ is the base change $\mathbb{C}$-algebra $A:= \mathbb{C} \otimes_{\mathbb{R}} \mathbb{R}[t]_{(t^2+1)}$. Note that $A$ is a localization of $\mathbb{C}[t]$, and so it is a domain.
Let $x_1, x_2 \in \mathbb{A}^1_{\mathbb{C}}$ be the two preimages of $x$ (corresponding to $\pm \sqrt{-1}$). I think that the pullback $(f')^*(\mathcal{F})$ is the sheafification of the following presheaf $\mathcal{G}$ of $\mathcal{O}_{\mathbb{A}_{\mathbb{C}}^1}$-algebras:
$$ \mathcal{G}(V) = \begin{cases} A \; \text{ if $x_1 \in V$ and $x_2 \in V$} \\ A_{(t-t(x_1))} \; \text{ if $x_1 \in V$ and $x_2 \notin V$}\\ A_{(t-t(x_2))} \; \text{ if $x_2 \in V$ and $x_1 \notin V$} \\ 0 \; \text{ otherwise} \end{cases} $$
I believe that the sheafification is the following:
$$ (f')^*(\mathcal{F})(V) = \begin{cases} A_{(t-t(x_1))} \times A_{(t-t(x_2))} \; \text{ if $x_1 \in V$ and $x_2 \in V$} \\ A_{(t-t(x_1))} \; \text{ if $x_1 \in V$ and $x_2 \notin V$}\\ A_{(t-t(x_2))} \; \text{ if $x_2 \in V$ and $x_1 \notin V$}\\ 0 \; \text{ otherwise} \end{cases} $$
Therefore, the pushforward $(p')_*((f')^*(\mathcal{F}))$ is the product $A_{(t-t(x_1))} \times A_{(t-t(x_2))}$, which is not a domain. I believe that the base change map is given by the diagonal $A \to A_{(t-t(x_1))} \times A_{(t-t(x_2))}$, so it is not an isomorphism.
(Edited: corrected the computation of the pullback).