Here's an interesting example very similar to one I described in my dissertation (in a discussion of Stein factorisation and base change). In particular, it shows that assuming $f$ is flat and projective with geometrically connected fibres is *not* sufficient, both in the case where $S$ is integral and the case $S$ Artinian.

(I suspect there might be more elementary examples, but at least this one is somewhat conceptual.)

**Example.** Let $k = \bar{\mathbf F}_p(x)$ and $S = \mathbf A^1_k = \operatorname{Spec} k[t]$. Let $E$ be a supersingular elliptic curve over $\bar {\mathbf F}_p$, and let
$$\mathcal E = E \underset{\operatorname{Spec}\bar{\mathbf F}_p}\times S = E_ k \underset{\operatorname{Spec} k}\times S.$$
Construct an $\pmb\alpha_p$-torsor $X \to \mathcal E$ that is geometrically nontrivial in all fibres over $S \setminus 0$, and in the special fibre is the purely inseparable map $E_{\bar{\mathbf F}_p(x^{1/p})} \to E_k$: the short exact sequence
$$0 \to \pmb\alpha_p \to \mathcal O_{\mathcal E} \stackrel{F}\to \mathcal O_{\mathcal E} \to 0$$
on the flat site of $\mathcal E$ gives a long exact sequence
$$\ldots \to H^0(E,\mathcal O_E) \underset{\bar{\mathbf F}_p}\otimes k[t] \stackrel\delta\to H^1(\mathcal E,\pmb\alpha_p) \to H^1(E,\mathcal O_E) \underset{\bar{\mathbf F}_p}\otimes k[t] \stackrel{F}\to H^1(E,\mathcal O_E) \underset{\bar{\mathbf F}_p}\otimes k[t] \to \ldots.$$
Since $E$ is supersingular, the Frobenius action on $H^1(E,\mathcal O_E)$ is $0$, so a nonzero class $\eta \in H^1(E,\mathcal O_E)$ gives an element $\eta t \in H^1(E,\mathcal O_E) \otimes_{\bar{\mathbf F}_p} k[t]$ mapping to $0$ under $F$. If $\beta \in H^1(\mathcal E,\pmb\alpha_p)$ if an element mapping to $\eta t$, then $\beta|_{\mathcal E_0}$ maps to $0$ in $H^1(\mathcal E_0,\mathcal O_{\mathcal E_0}) = H^1(E,\mathcal O_E) \otimes_{\bar{\mathbf F}_p} k$, hence comes from an element $f \in H^0(\mathcal E_0,\mathcal O_{\mathcal E_0}) = H^0(E,\mathcal O_E) \otimes_{\bar{\mathbf F}_p} k$. Replacing $\beta$ by $\beta + \delta(x-f)$ we may assume that $\beta|_{\mathcal E_0} = \delta(x)$.

Letting $X \to \mathcal E$ be the $\pmb\alpha_p$-torsor given by the class $\beta$, we see that $X_0 = E_{\bar{\mathbf F}_p(x^{1/p})}$ (corresponding to the class $\delta(x) \in H^1(\mathcal E_0,\pmb\alpha_p)$). It is clear that $X \to \mathcal E$ and $\mathcal E \to S$ are flat and proper, so the same goes for $f \colon X \to S$.

> **Claim.** With $X$ as above, we have $H^0(X,\mathcal O_X) = k[t]$, i.e. $f_*\mathcal O_X = \mathcal O_S$.

Indeed, note that the fibres at $s \neq 0$ are smooth since the $\pmb\alpha_p$-torsors $X_s \to \mathcal E_s$ are geometrically nontrivial, hence $X_{\bar s} \to \mathcal E_{\bar s}$ is a degree $p$ inseparable cover of elliptic curves. Thus the geometric fibres of $f|_{S \setminus 0}$ are reduced and connected, so $f_* \mathcal O_{X|_{S\setminus 0}} = \mathcal O_{S \setminus 0}$. Since $S$ is normal, $f_* \mathcal O_X$ is the normalisation of $\mathcal O_S$ in $f_* \mathcal O_{X|_{S \setminus 0}}$, hence equals $\mathcal O_S$. $\square$

Since $f_* \mathcal O_X = \mathcal O_S$ but $H^0(X_0,\mathcal O_{X_0}) = \bar{\mathbf F}_p(x^{1/p}) \supsetneq k$, we already have a counterexample over an integral base (which can be made local by localising at $0$).

To get an Artinian counterexample, restrict the above to $\operatorname{Spec} k[t]/t^n \subseteq S$ for $n \gg 0$. By the theorem of formal functions, if the maps $H^0(X|_{\operatorname{Spec} k[t]/t^n},\mathcal O) \to H^0(X_0,\mathcal O_{X_0})$ are surjective for all $n$, then so is $H^0(X,\mathcal O_X) \to H^0(X_0,\mathcal O_{X_0})$, which we saw is not the case.

**Remark.** In fact, I suspect that already for $n = 2$ we have
$$H^0\left(X\big |_{\operatorname{Spec} k[t]/t^2},\mathcal O\right) = k[t]/t^2,$$
somehow because the $\pmb\alpha_p$-torsor is not induced by one coming from the base $k[t]/t^2$ (but really involves the elliptic curve).