Let $K$ be a smoothing operator on $\mathbb{R}^n$, i.e., it defines a map on all Sobolev spaces $K\colon H^r(\mathbb{R}^n) \to H^s(\mathbb{R}^n)$ for all $r, s \in \mathbb{R}$. Now (a variation of) the Schwartz kernel theorem states that it is given by some smooth kernel $k \in C^\infty(\mathbb{R}^n \times \mathbb{R}^n)$, i.e., $(Kf)(x) = \int_{\mathbb{R}^n} k(x, y)f(y) dy$.
Now suppose we have kernels $k_n, k \in C^\infty(\mathbb{R}^n \times \mathbb{R}^n)$ that do define smoothing operator $K_n$ and $K$ (since not every element of $C^\infty(\mathbb{R}^n \times \mathbb{R}^n)$ defines a smoothing operator we have to assume this).
What type of convergence $k_n \to k$ is needed, so that $K_n$ does converge to $K$?
Suppose I want $K_n \to K$ only as maps $L^2(\mathbb{R}^n) \to L^2(\mathbb{R}^n)$. Does it suffice for this that $k_n$ converges uniformly to $k$?
Suppose I want $K_n \to K$ as maps $H^r(\mathbb{R}^n) \to H^s(\mathbb{R}^n)$ for all $r, s \in \mathbb{R}$. Is it sufficient for this that $k_n \to k$ uniformly and also all their derivatives?
I would prefer convergence in the norm topology, but any information about convergence in the strong / weak operator topology would also be nice.