No. For instance there is a flat, projective morphism $f:X\rightarrow \Bbb{P}^1$ such that $X_t:=f^{-1}(t)$ is a smooth rational curve for $t\neq 0$, but $X_0$ is a rationalnodal plane cubic curve with an embedded point (see Hartshorne, III.9.8.4). Then $H^1(X_t,\mathcal{O}_{X_t})$ is zero for $t\neq 0$, but $\ \dim H^1(X_0,\mathcal{O}_{X_0})=1$ . By base change it follows that $R^1f_*\mathcal{O}_X$ is the skyscraper sheaf with 1-dimensional fiber at 0.