Here is a example with proper morphism over a noetherian base. We consider a DVR $R$, and we note $S=\mathrm{Spec}(R)$. Let $X/S$ be a proper flat curve $X/R$ which is not cohomologically flat, that is $\mathrm{R}^{1}f_{\ast}\mathcal{O}_X$ has actually torsion, and that $\mathrm{H}^{0}(X,\mathcal{O} _{X})=R$ (such a curve exists, for example, one can consider the proper minimal regular model of some projective smooth curve on $\mathrm{Frac}(R)$ without rational point, see for example Raynaud's paper on picard functor where you can find example for genus one curve). Then I claim that for some integer $n\gg 1$, the reduction $X_n:=X\otimes_R R/\pi^n$ provides an example that we need. Indeed, since $X/S$ is flat, we have the following short exact sequence
$$
0\rightarrow \mathcal{O}_{X}\rightarrow \mathcal{O}_X\rightarrow \mathcal{O} _{X_n}\rightarrow 0
$$
here the first map is multiplication by $\pi^n$. Now the long exact sequence tells us
$$
0\rightarrow \mathrm{H}^{0}(X,\mathcal{O}_{X})\rightarrow \mathrm{H}^0(X,\mathcal{O}_X)\rightarrow \mathrm{H}^{0}(X,\mathcal{O} _{X_n})\rightarrow \mathrm{H}^{1}(X,\mathcal{O}_{X})[\pi^n]\rightarrow 0
$$
Here the first map is the multiplication by $\pi^n$, while the last member is the $\pi^n$-torsion of $\mathrm{H}^{1}(X,\mathcal{O} _{X})$. Now when $n$ becomes sufficiently large, the last member becomes stable. On the other hand, $\mathrm{H}^{0}(X,\mathcal{O} _{X})=R$, hence we get the following exact sequence of $R$-modules
$$
0\rightarrow R/\pi^n R \rightarrow \mathrm{H}^{0}(X,\mathcal{O} _{X_n})\rightarrow \mathrm{H}^{1}(X,\mathcal{O}_{X})[\pi^n]\rightarrow 0.
$$
So in this way, we see that $\mathrm{H}^{0}(X,\mathcal{O} _{X_n})$ cannot be flat (hence free) over $R/\pi^n$ as such a free module must be of length a multiple of n, which is impossible since the last member of the previous short exact sequence is stable and non zero for $n\gg 0$.