Let $\Omega\subset\mathbb{R}^n$ open, bounded with smooth boundary, let $s\in(0,1)$. I know that the fractional Laplacian has a sequence of eigenfunctions $\{e_k\}_{k\in\mathbb{N}}\subset H^s(\mathbb{R}^N)$, $e_k=0$ a.e. on $\mathbb{R}^n\setminus\Omega$, $\forall k\in\mathbb{N}$. Moreover I know that these eigenfunctions are continuous on the whole of $\mathbb{R}^n$. My question is: are these eigenfunctions smooth in $\Omega$? Can you give me some reference on this result (if they exist)?
This is consequence of the theory of Pseudodifferential operators. Your fractional Laplacian $P_s$ is a PDO of order $2s$. Above all, it is elliptic. This is a classical result (certainly in Hörmander) that if $P_su\in H^r_{\rm loc}$, then $u\in H^{r+2s}_{\rm loc}$, where $H^{\cdots}$ is the class of Sobolev spaces.
Apply this to an eigenfunction $e_k$, which you know is continuous, hence is $H^0_{\rm loc}=L^2_{\rm loc}$. Bootstrapping, you find that $e_k$belongs to $$\bigcap_{\ell\in\mathbb N}H^{2\ell s}_{\rm loc}\subset C^\infty.$$


1$\begingroup$ @inoc. I apologize, but being confined at home, I don't have access to a library or my bookshelf. But every monography about PDOs contains this result. $\endgroup$ Nov 3 '20 at 10:16