Let $W$ be $B^4$ or $S^3 \times I$. Let $Y$ be a properly embedded surface in $W$. Let $f : W \to W$ be a diffeomorphism which is the identity near $\partial W$. Very little is known about $\pi_0(\mbox{Diff($W$) rel $\partial W$})$ (see page 4 of this survey by Hatcher), so $f$ might not be isotopic to the identity (or, in the $S^3\times I$ case, isotopic to a standard twist diffeomorphism). Nevertheless, I believe it is true that $f(Y)$ is isotopic rel boundary to $Y$. In other words, exotic diffeomorphisms cannot do anything exotic to embedded surfaces.

The proof I have in mind (based on conversations with an expert) uses a "push-pull" type argument (and, in the $S^3\times I$ case, straightening along an arc) to concentrate any potential exoticness of $f$ in the neighborhood of one or two points. We can take these points to be far from $Y$, and it follows that $f(Y)$ is isotopic to $Y$.

Rather than write down the details of this proof, I would prefer to just cite a published result. So my question is:

Is there a citable reference for the above claim that $f(Y)$ is isotopic rel boundary to $Y$? Is there a citable reference for the claim that diffeomorphisms of simple 4-manifolds (like $B^4$ or $S^3 \times I$) can be "tamed" in the complement of a finite number of points?