Suppose that $X$ be an oriented, closed four-manifold. Suppose that $Y$ is an oriented, closed three-manifold smoothly embedded in $X$. Suppose also that $f:X \to X$ is a diffeomorphism that fixes $Y$ setwise: that is, $f(Y)=Y$. Finally, we suppose that that $f$ is topologically isotopic to the identity map.

Does this imply, perhaps after some further smooth isotopy, that $f$ is the identity on $Y$: that is, fixes $Y$ pointwise?

If it helps, we can also assume that $X$ is simply connected.

One thing in dimension four that requires care is that the existence of antopological isotopy may not imply that there is a smooth isotopy. Danny Ruberman found the existence of such examples.

Another thing: if $\mathrm{Diff}^+(Y)$ is connected, then the above question has a positive answer. (For example $S^3$.)