Let $V$ be a smooth manifold obtained by attaching the ``open collar'' $[0,1)\times \partial N$ to a compact smooth manifold $N$ along the boundary. Let $\mathrm{Emb}(N, V)$ be the space of embeddings of $N$ into $V$ isotopic to the inclusion. The identity component $\mathrm{Diff\,} N$ of the diffeomorphism group of $N$ embeds into $\mathrm{Emb}(N, V)$ by precomposing a diffeomorphism with the inclusion.
Question: Under what conditions the embedding $\mathrm{Diff\,} N\to\mathrm{Emb}(N, V)$ is not a homotopy equivalence?
It is relevant to consider the $\mathrm{Diff\,} V$-action on $\mathrm{Emb}(N, V)$ by postcomposing. The orbit map $\mathrm{Diff\,} V\to\mathrm{Emb}(N, V)$ is a homotopy equivalence: its homotopy fiber consists of diffeomorphisms that are identity on $N$, and the group of such diffeomorphisms is contractible by the Alexander trick towards infinity.
The case when $V=\mathbb R^n$ and $N=D^n$ is fairly well understood: both $\mathrm{Diff\,} \mathbb R^n$ and $\mathrm{Diff\,}\mathbb D^n$ deformation retract onto $O(n)$ [Edit: as Allen Hatcher explains in comments $\mathrm{Diff\,}\mathbb D^n$ does not deformation retract onto $O(n)$]. On the other hand, $\mathrm{Diff\,}(\mathbb D^n, rel\,\partial D^n)$ is homotopically more complicated when $n$ large (computing it involves pseudoisotopy and Waldhausen K-theory).
Naively I hope the case of a disk be used to produce elements of $\pi_i(\mathrm{Diff\,} N)$ that die in $\pi_i(\mathrm{Emb}(N,V))\cong\pi_i(\mathrm{Diff\,} V)$, e.g. by considering boundary connected sum of $N$ with $D^n$, and extending diffeomorphisms of $D^n$ that are identity on $\partial D^n$ to $N$ by the identity. Has this ever been done?
I am even more interested in ways to produce elements of $\pi_i(\mathrm{Emb}(N,V))$ that do not come from $\pi_i(\mathrm{Diff\,} N)$.