Here's an interesting example very similar to one I described in my dissertation (in a discussion of Stein factorisation and base change). In particular, it shows that assuming $f$ is flat and projective with geometrically connected fibres is not sufficient, both in the case where $S$ is integral and the case $S$ Artinian. (You can even show that the source and target are regular, cf. §2.3 of my dissertation.)
(I suspect there might be more elementary examples, but at least this one is somewhat conceptual.)
Example. Let $k = \bar{\mathbf F}_p(x)$ and $S = \mathbf A^1_k = \operatorname{Spec} k[t]$. Let $E$ be a supersingular elliptic curve over $\bar {\mathbf F}_p$, and let $$\mathcal E = E \underset{\operatorname{Spec}\bar{\mathbf F}_p}\times S = E_ k \underset{\operatorname{Spec} k}\times S.$$ Construct an $\pmb\alpha_p$-torsor $X \to \mathcal E$ that is geometrically nontrivial in all fibres over $S \setminus 0$, and in the special fibre is the purely inseparable map $E_{\bar{\mathbf F}_p(x^{1/p})} \to E_k$: the short exact sequence $$0 \to \pmb\alpha_p \to \mathcal O_{\mathcal E} \stackrel{F}\to \mathcal O_{\mathcal E} \to 0$$ on the flat site of $\mathcal E$ gives a long exact sequence $$\ldots \to H^0(E,\mathcal O_E) \underset{\bar{\mathbf F}_p}\otimes k[t] \stackrel\delta\to H^1(\mathcal E,\pmb\alpha_p) \to H^1(E,\mathcal O_E) \underset{\bar{\mathbf F}_p}\otimes k[t] \stackrel{F}\to H^1(E,\mathcal O_E) \underset{\bar{\mathbf F}_p}\otimes k[t] \to \ldots.$$ Since $E$ is supersingular, the Frobenius action on $H^1(E,\mathcal O_E)$ is $0$, so a nonzero class $\eta \in H^1(E,\mathcal O_E)$ gives $\eta t \in H^1(E,\mathcal O_E) \otimes_{\bar{\mathbf F}_p} k[t]$ mapping to $0$ under $F$. If $\beta \in H^1(\mathcal E,\pmb\alpha_p)$ is an element mapping to $\eta t$, then $\beta|_{\mathcal E_0}$ maps to $0$ in $H^1(\mathcal E_0,\mathcal O_{\mathcal E_0}) = H^1(E,\mathcal O_E) \otimes_{\bar{\mathbf F}_p} k$, hence comes from an element $f \in H^0(\mathcal E_0,\mathcal O_{\mathcal E_0}) = H^0(E,\mathcal O_E) \otimes_{\bar{\mathbf F}_p} k$. Replacing $\beta$ by $\beta + \delta(x-f)$ we may assume that $\beta|_{\mathcal E_0} = \delta(x)$.
Letting $X \to \mathcal E$ be the $\pmb\alpha_p$-torsor given by the class $\beta$, we see that $X_0 = E_{\bar{\mathbf F}_p(x^{1/p})}$ (corresponding to the class $\delta(x) \in H^1(\mathcal E_0,\pmb\alpha_p)$). It is clear that $X \to \mathcal E$ and $\mathcal E \to S$ are flat and proper, so the same goes for $f \colon X \to S$.
Claim. With $X$ as above, we have $H^0(X,\mathcal O_X) = k[t]$, i.e. $f_*\mathcal O_X = \mathcal O_S$.
Indeed, note that the fibres at $s \neq 0$ are smooth since the $\pmb\alpha_p$-torsors $X_s \to \mathcal E_s$ are geometrically nontrivial, hence $X_{\bar s} \to \mathcal E_{\bar s}$ is a degree $p$ inseparable cover of elliptic curves. Thus the geometric fibres of $f|_{S \setminus 0}$ are reduced and connected, so $f_* \mathcal O_{X|_{S\setminus 0}} = \mathcal O_{S \setminus 0}$. Since $S$ is normal, $f_* \mathcal O_X$ is the normalisation of $\mathcal O_S$ in $f_* \mathcal O_{X|_{S \setminus 0}}$, hence equals $\mathcal O_S$. $\square$
Since $f_* \mathcal O_X = \mathcal O_S$ but $H^0(X_0,\mathcal O_{X_0}) = \bar{\mathbf F}_p(x^{1/p}) \supsetneq k$, we already have a counterexample over an integral base (which can be made local by localising at $0$).
To get an Artinian counterexample, restrict the above to $\operatorname{Spec} k[t]/t^n \subseteq S$ for $n \gg 0$. By the theorem of formal functions, if the maps $H^0(X|_{\operatorname{Spec} k[t]/t^n},\mathcal O) \to H^0(X_0,\mathcal O_{X_0})$ are surjective for all $n$, then so is $H^0(X,\mathcal O_X) \to H^0(X_0,\mathcal O_{X_0})$, which we saw is not the case.
Remark. In fact, I suspect that already for $n = 2$ we have $$H^0\left(X\big |_{\operatorname{Spec} k[t]/t^2},\mathcal O\right) = k[t]/t^2,$$ somehow because the $\pmb\alpha_p$-torsor is not induced by one coming from the base $k[t]/t^2$ (but really involves the elliptic curve).