Iterated integrations by parts using the fractional Laplacian

Question

Let $u \in C^\infty_c(\mathbb{\Omega})$ and $\varphi$ be an eigenfunction of the fractional Laplacian $(-\Delta)^s$ in $\Omega$ with eigenvalue $\lambda$. In what sense, if any, is it true that $$\langle u, \varphi \rangle = \frac{1}{\lambda} \langle u, (-\Delta)^s\varphi \rangle = \frac{1}{\lambda} \langle(-\Delta)^s u, \varphi \rangle = \frac{1}{\lambda^2} \langle (-\Delta)^{2s} u, \varphi \rangle = ........?$$ Here $ \langle \cdot, \cdot \rangle$ denotes the scalar product in $L^2(\Omega)$. In other words, is it true that we can integrate by parts using $(-\Delta)^su$, $(-\Delta)^{2s}u$ and so on even though their support is not compact anymore?

Answer 1

No, we cannot.

Formally, $\varphi$ is the eigenfunction of the unbounded operator $L_s$ on $L^2(\Omega)$, defined initially by $$ L_s u(x) = (-\Delta)^s u(x) \qquad \text{for } x \in \Omega , $$ where $u \in C_c^\infty(\Omega)$ (and it is understood that $u(x) = 0$ for $x \notin \Omega$), and then extended to an appropriate domain (e.g. by means of Friedrichs extension).

Now the key observation is that $L_s L_s$ is not equal to $L_{2s}$, unless $\Omega = \mathbb R^d$. Indeed, in $\Omega$ we have $$ L_s L_s u = (-\Delta)^s (\mathbb 1_{\Omega} \times (-\Delta)^s u) , $$ while $$ L_{2s} u = (-\Delta)^s (-\Delta)^s u , $$ and due to non-locality of $(-\Delta)^s$, the two are not equal.

For the above reasons, we have $$ \langle u, \varphi \rangle = \frac{1}{\lambda^2} \langle u, L_s L_s \varphi \rangle = \frac{1}{\lambda^2} \langle L_s L_s u, \varphi \rangle $$ (provided that $L_s u$ belongs to the domain of $L_s$, which is rarely the case!), and in general the right-hand side is not equal to $\lambda^{-2} \langle (-\Delta)^{2s} u, \varphi \rangle$.