I apologize for answering such an old question, but it seems fundamental.  A classical counterexample occurs for the abel map of a Prym variety with exceptional singularities on the theta divisor.  The point is that the fibers of the abel prym map  X-->Y of the double cover C'-->C are included among those for the abel map of C', hence are all smooth.  (A map obtained by restricting another map over a subvariety of the target has the same fibers.)  

Nonetheless X is singular at any exceptional divisor. (see lemma 2.13 of [A Riemann singularities theorem for Prym theta divisors, with applications](https://www.math.uga.edu/sites/default/files/inline-files/sv2rst.pdf)).  

The point of the previous paper was that generalizing the Riemann - Kempf singularity theorem to prym varieties is easy when X is smooth.  But when X is singular it is considerably harder:

[A necessary and sufficient condition for Riemann's singularity theorem to hold on a Prym theta divisor](https://www.math.uga.edu/sites/default/files/inline-files/sv5rst2.pdf)

[Singularities of the Prym theta divisor](https://annals.math.princeton.edu/2009/170-1/p05)


For a detailed discussion of the case of the abel prym map for a prym variety isomorphic to the intermediate jacobian of the cubic threefold, see:

[On parametrizing exceptional tangent cones to Prym theta divisors](https://www.math.uga.edu/sites/default/files/inline-files/onparam.pdf)

 The answer is yes however if the target Y is a smooth curve, since X is smooth at any point lying on a smooth cartier divisor,  (compare Mumford, chap.7, Prop. 2, redbook.)