Let us take a look at the case where $Y\equiv x_n=0$ in $\mathbb R^n$.
Let $a(\underbrace{\overbrace{x_1,\dots,x_{n-1}}^{x'},x_n}_x; \xi_n)$ be a symbol in $S^m(\mathbb R^n\times \mathbb R)$ and let us consider the following "oscillatory integral":
$$
u(x)=\int_{\mathbb R} e^{ix_n\cdot \xi_n} a(x,\xi_n) d\xi_n.
$$
This integral is not absolutely converging and we could define it as a distribution. Note that we may assume that $a$ is supported where $\vert \xi_n\vert\ge 1$ since the contribution of a compactly supported $a$ is a smooth function. Then we have for $\phi$ in the Schwartz space, $l\in 2\mathbb N$,
formally integrating by parts,
\begin{align*}
\langle u,\phi\rangle_{\mathscr S',\mathscr S}&=\int_{\mathbb R^n\times\mathbb R}
\vert \xi_n\vert^{-l} \vert D_{x_n}\vert^l (e^{ix_n\cdot \xi_n}) a(x,\xi_n)\phi(x) dx d\xi_n\\
&=\sum_{\alpha+\beta=l}c_{\alpha\beta}\int_{\mathbb R^n\times\mathbb R}
e^{ix_n\cdot \xi_n} \vert \xi_n\vert^{-l} (D_{x_n}^\alpha a)(x,\xi_n)(D_{x_n}^\beta\phi)(x) dx d\xi_n,
\end{align*}
which makes sense for $l>1+m$. We can take the last integral as the definition of the duality product.
We note also that $u$
can easily be differentiated with respect to $x'$ (tangential derivatives) since $\partial_{x'}a\in S^m$. However the distribution $u$ is more difficult to differentiate wrt $x_n$ since $\xi_na\in S^{1+m}$; however calculating
$$
x_n\partial_{x_n} u=\int_{\mathbb R} x_n\partial_{x_n}(e^{ix_n\cdot \xi_n}) a(x,\xi_n) d\xi_n
+\int_{\mathbb R} e^{ix_n\cdot \xi_n} x_n\partial_{x_n}(a)(x,\xi_n) d\xi_n.
$$
$$
=\int_{\mathbb R} x_n i\xi_n e^{ix_n\cdot \xi_n} a(x,\xi_n) d\xi_n
+\int_{\mathbb R} e^{ix_n\cdot \xi_n} x_n\partial_{x_n}(a)(x,\xi_n) d\xi_n.
$$
$$
=\int_{\mathbb R} \partial_{\xi_n}(e^{ix_n\cdot \xi_n} )\xi_n a(x,\xi_n) d\xi_n
+\int_{\mathbb R} e^{ix_n\cdot \xi_n} x_n\partial_{x_n}(a)(x,\xi_n) d\xi_n.
$$
$$
=-\int_{\mathbb R} e^{ix_n\cdot \xi_n} \underbrace{\partial_{\xi_n}(\xi_n a(x,\xi_n))}_{\in S^m} d\xi_n
+\int_{\mathbb R} e^{ix_n\cdot \xi_n} \underbrace{x_n\partial_{x_n}(a)(x,\xi_n)}_{\in S^m} d\xi_n.
$$
We can prove in particular that the wave-front-set of $u$ is included in the conormal bundle of $Y$,
i.e.
$$
WF u\subset\{(x',0;0,\xi_n)\}_{x'\in \mathbb R^{n-1}, \xi_n
\in \mathbb R^*}$$
Your simple layer $u(x)= \omega(x')\otimes \delta(x_n)$ with a smooth $\omega$ is indeed the prototypical conormal distribution since
$$
u(x)= \int_\mathbb R\omega(x') e^{2iπ x_n\xi_n}d\xi_n.
$$
It is not difficult to generalize the previous discussion to the case involving $k$ codimensions.