Equip $\mathbb{R}^{d+1}$ with the Lorentzian form $\langle x, y\rangle=-x^0y^0+{\bf x}\cdot{\bf y}$ where $x=(x^0,{\bf x})$ and $\cdot$ is the usual Euclidean dot product. We define the hyperboloid $\mathbb{H} = \{p\in\mathbb{R}^{d+1}\,\vert\, p^2 = -m^2,\, p^0 > 0\}$, $m>0$; it possesses a Euclidean metric induced by restriction from $\mathbb{R}^{d+1}$ and hence a measure $\mathrm{d}V_{\mathbb{H}}$, as well as an $L^2$ inner product $\langle \cdot ,\cdot\rangle_{\mathbb{H}}$. If we parametrize the points on $\mathbb{H}$ as $p=(\sqrt{{\bf p}^2 + m^2}, {\bf p})$, ${\bf p}\in\mathbb{R}^d$, the measure is expressed as $$ \mathrm{d}V_{\mathbb{H}} = \frac{m}{\sqrt{{\bf p}^2 + m^2}}\,\mathrm{d}{\bf p} $$ Now suppose $f,g\in\mathcal{S}(\mathbb{R}^d)$ are Schwartz functions. Denote the Fourier transform as $$ (\mathcal{F}f)(p) = \int_{\mathbb{R}^{d+1}}f(x)e^{-2\pi i\langle p, x\rangle}\mathrm{d}x $$ I would like to compute the integral $\langle \mathcal{F}f,\mathcal{F}\overline{g}\rangle_\mathbb{H}$, i.e. $$ \int_{\mathbb{H}}(\mathcal{F}f)(p)\,\overline{(\mathcal{F}\overline{g})(p)}\,\mathrm{d}V_{\mathbb{H}}(p)=\int_{\mathbb{R}^d}\frac{m\,\mathrm{d}{\bf p}}{\sqrt{{\bf p}^2+m^2}}\int_{\mathbb{R}^{d+1}}\int_{\mathbb{R}^{d+1}}\mathrm{d}x\,\mathrm{d}y\, f(x)g(y) e^{2\pi i\langle p, y-x\rangle} $$ I mean "compute" here in the sense that, if I am not mistaken, the assignment $$ g\longmapsto(f\longmapsto \langle \mathcal{F}f,\mathcal{F}\overline{g}\rangle_\mathbb{H}) $$ is a continuous linear map $\mathcal{S}(\mathbb{R}^{d+1})\longrightarrow\mathcal{S}'(\mathbb{R}^{d+1})$ and hence defines a distributional kernel by the Schwartz kernel theorem, which is what I'm after. Naively, one might change the order of integration above to get$$ \langle \mathcal{F}f,\mathcal{F}\overline{g}\rangle_\mathbb{H}=\int_{\mathbb{R}^{d+1}}\int_{\mathbb{R}^{d+1}}\mathrm{d}x\,\mathrm{d}y\, f(x)g(y) \int_{\mathbb{R}^d}\frac{m\,\mathrm{d}{\bf p}}{\sqrt{{\bf p}^2+m^2}}e^{2\pi i\langle p, y-x\rangle}, $$ which suggests that the kernel is the Fourier transform of the distribution $f\longmapsto \int_{\mathbb{H}}f\,\mathrm{d}V_{\mathbb{H}}$, but I cannot rigorously justify this nor can I compute this Fourier transform. (I tried first regularizing the ${\bf p}$-integral by a Gaussian factor, but I don't really know how to proceed from there.) I would appreciate it if someone clarified the situation, even just computing the Fourier transform of the hyperboloid would be a nice first step.
I am interested in this because I believe that in QFT, $\langle \mathcal{F}f,\mathcal{F}\overline{g}\rangle_\mathbb{H}$ equals $\langle \phi(g)\phi(f)\Omega,\Omega\rangle$ for a free real scalar boson field $\phi$. (Migrated from Math StackExchange for lack of activity.)
