Let $\Omega\subset\mathbb{R}^n$, let $s\in [1/2,1)$, let $u\in C^{1,2s1+\epsilon}(\Omega)$ such that: $u=0$ on $\mathbb{R}^n\setminus\Omega$, and: $u\in C^{0,s}(\mathbb{R}^n)$, is true that: $$\int_{\mathbb{R}^n}\phi(\Delta)^su\,dx=\int_{\mathbb{R}^n}u(\Delta)^s\phi,\quad\forall\phi\in C^\infty_c(\mathbb{R}^n)?$$ I kwon only that: $$ \int_{\mathbb{R}^n}\phi(\Delta)^sf\,dx=\int_{\mathbb{R}^n}f(\Delta)^s\phi\,dx,\quad\forall f,\phi\in \mathcal{S}(\mathbb{R}^n).$$ I have no idea on how to go on, any help is appreciated.

1$\begingroup$ The proof follows by approximation, but the correct setting is $u \in H^{2s}_0(\Omega)$. Whether your condition on $u$ implies that $u$ is in the Sobolev space $H^{2s}_0(\Omega)$ may depend on the exact definition of $C^{k,\alpha}(\Omega)$, as pointed out in many of your previous questions. $\endgroup$– Mateusz KwaśnickiCommented Nov 17, 2020 at 9:13

$\begingroup$ Can you give the details of this approximation in $H^{2s}(\Omega)$, please? I do not know any approximation results in $H^{2s}(\Omega)$. $\endgroup$– inocCommented Nov 17, 2020 at 9:20

$\begingroup$ @MateuszKwaśnicki: Moreover, for $u\in C^{1,2s1+\epsilon}(\mathbb{R}^n)$ i mean that: $u$ is $(2s1+\epsilon)$Hölder continuous and $\nabla u$ is $(2s1+\epsilon)$Hölder continuous. This hypothesis implies that: $u\in H^{2s}(\Omega)$? $\endgroup$– inocCommented Nov 17, 2020 at 9:35

1$\begingroup$ Any $H^{2s}(\mathbb R^n)$ function $u$ can be approximated by a sequence $u_n$ of Schwartz functions in the norm of $H^{2s}(\mathbb R^n)$ (so that $u_n$ converges to $u$ and $(\Delta)^s u_n$ converges to $(\Delta)^s u$, both limits in $L^2(\mathbb R^n)$). This can be found in most textbooks on Sobolev spaces, I believe. I am sure it is proved in Samko's Hypersingular integrals and Their Applications. $\endgroup$– Mateusz KwaśnickiCommented Nov 17, 2020 at 10:07

1$\begingroup$ Regarding your other comment: I already explained that some authors use $C^{k,\alpha}(\Omega)$ for the class of functions with $k$th partial derivatives Hölder continuous in any compact subset of $\Omega$, while others require them to by Hölder continuous in all of $\Omega$. In the latter case, $u$ extends to a function in $C^{k,\alpha}(\overline \Omega)$, so with this definition the assumptions in your questions would be odd (if $u \in C^{1,\alpha}(\Omega)$, then automatically $y \in C^{1,\alpha}(\mathbb R^n)$). $\endgroup$– Mateusz KwaśnickiCommented Nov 17, 2020 at 10:12
1 Answer
If $x, y \in \Omega$, then $$ u(y)  u(x)  \nabla u(x) \cdot (y  x) \leqslant C y  x^{2 s + \epsilon} ,$$ and so the integral $$ \iint_{\Omega \times \Omega} \frac{u(y)  u(x)  \nabla u(x) \cdot (y  x)}{y  x^{n + 2 s}} \, \phi(x) dx dy $$ converges absolutely.
Denote $d(x) = \operatorname{dist}(x, \partial \Omega)$. If $x \in \Omega$, $y \in \Omega^c$, then $u(x) \leqslant C d(x)$ (because $\nabla f$ is bounded) and $u(y) = 0$. Thus, $$ u(y)  u(x)  \nabla u(x) \cdot (y  x) \leqslant u(x) + \nabla u(x) \, y  x \leqslant C d(x) + C y  x .$$ Furthermore, $$ \int_{\Omega^c} \frac{1}{y  x^{n + 2 s}} \, dy \leqslant \frac{1}{(d(x))^{2s}} $$ and $$ \int_{\Omega^c} \frac{y  x}{y  x^{n + 2 s}} \, dy \leqslant \frac{1}{(d(x))^{2s  1}} \, . $$ Finally, $1 / (d(x))^{2s  1}$ is integrable. It follows that the integral $$ \iint_{\Omega \times \Omega^c} \frac{u(y)  u(x)  \nabla u(x) \cdot (y  x)}{y  x^{n + 2 s}} \, \phi(x) dx dy $$ converges absolutely, too.
Similarly, if $x \in \Omega^c$ and $y \in \Omega$, we find that $$ u(y)  u(x)  \nabla u(x) \cdot (y  x) \leqslant u(y) \leqslant C d(y) ,$$ and since $$ \int_\Omega \frac{1}{y  x^{n + 2 s}} \, dy \leqslant \min \biggl\{ \frac{1}{(d(x))^{2s}} , \frac{C \Omega}{x^{n + 2 s}} \biggr\} , $$ we have absolute convergence of $$ \iint_{\Omega^c \times \Omega} \frac{u(y)  u(x)  \nabla u(x) \cdot (y  x)}{y  x^{n + 2 s}} \, \phi(x) dx dy . $$
Finally, the integral over $\Omega^c \times \Omega^c$ is identically zero.
We conclude that the integral $$ \iint_{\mathbb R^n \times \mathbb R^n} \frac{u(y)  u(x)  \nabla u(x) \cdot (y  x)}{y  x^{n + 2 s}} \, \phi(x) dx dy $$ converges absolutely. Now the usual argument applies: $$\begin{aligned} \int_\Omega (\Delta)^s u(x) \phi(x) dx & = \iint_{\mathbb R^n \times \mathbb R^n} \frac{u(y)  u(x)  \nabla u(x) \cdot (y  x)}{y  x^{n + 2 s}} \, \phi(x) dx dy \\ & = \lim_{\delta \to 0^+} \iint_{x  y > \delta} \frac{u(y)  u(x)  \nabla u(x) \cdot (y  x)}{y  x^{n + 2 s}} \, \phi(x) dx dy \\ & = \lim_{\delta \to 0^+} \iint_{x  y > \delta} \frac{u(y)  u(x)}{y  x^{n + 2 s}} \, \phi(x) dx dy \\ & = \lim_{\delta \to 0^+} \iint_{x  y > \delta} \frac{\phi(y)  \phi(x)}{y  x^{n + 2 s}} \, u(x) dx dy \\ & = \lim_{\delta \to 0^+} \iint_{x  y > \delta} \frac{\phi(y)  \phi(x)  \nabla \phi(x) (y  x)}{y  x^{n + 2 s}} \, u(x) dx dy \\ & = \int_\Omega (\Delta)^s \phi(x) u(x) dx . \end{aligned}$$ (Here the second equality follows by dominated convergence, the fourth one by Fubini, and the sixth one again by dominated convergence.)

$\begingroup$ Regarding the integral expression for $(\Delta)^s u(x)$: isn't this the very definition of $(\Delta)^s u(x)$? The term $y  x^{n  2s} \nabla u(x) \cdot (y  x)$ disappears because it has zero integral over $\mathbb R^n \setminus B(x, r)$ for every $r > 0$. $\endgroup$ Commented Nov 24, 2020 at 11:58

$\begingroup$ (The above comment answers questions raised in other comments that now seem deleted.) $\endgroup$ Commented Nov 24, 2020 at 11:59