Suppose that $Y=C$ is a cuspidal curve, and let $\tilde C\to C$ be the normalization. Put $X = \mathbb{A}^1\times \tilde C$, and let $X\to Y$ be the obvious map. The fibers are all $\mathbb{A}^1$'s, but over the singularity in $Y$ the map cannot locally be the projection.
If the map $f$ is a submersion and $Y$ is smooth then I think things should work out, at least in characteristic zero.