Rankin-Selberg integral for GL(3) form with Odd Maass form on GL(2)

Question

Let $F$ be a Hecke-Maass cusp form for $SL_3(\mathbb Z)$. Let $u$ be a Hecke-Maass cusp form for $SL_2(\mathbb Z)$.

The following integral $$\mathcal L(s,F\times u)=\int_{{SL}(2,\mathbb{Z})\backslash GL_+(2,\mathbb R)/SO(2)}F\left(\begin{pmatrix}z&\\&1\end{pmatrix}\right)u(z)\det(z)^{s- \frac 1 2}{d^*z}$$

produces $$\text{gamma factors }\times\; L(s, F\times u), $$ where $L(s,F\times u)$ is the Rankin-Selberg $L$-functions and $u$ is an even Maass form.

However, when $u$ is an odd Maass form, $\mathcal L(s,F\times u)$ is identically zero because of trivial reason.

But still $L(s,F\times u)$ is well defined.

Question: What integral (maybe similar to $\mathcal L$) will give $L(s,F\times u)$ when $u$ is an odd Maass form?

Reference: Proposition 4.4 of Li, Xiaoqing, and Matthew P. Young. "The $L^2$ restriction norm of a $GL_3$ Maass form." Compositio Mathematica 148, no. 3 (2012): 675-717.

Answer 1

Let me translate this question into the adèlic language, because this question is more naturally posed there. Let $\pi$ be a cuspidal automorphic representation of $\mathrm{GL}_3(\mathbb{A_Q})$ and let $\pi'$ be a cuspidal automorphic representation of $\mathrm{GL}_2(\mathbb{A_Q})$. For vectors $\varphi \in \pi$ and $\varphi' \in \pi'$, we consider the global Eulerian integral \[\int\limits_{\mathrm{GL}_2(\mathbb{Q}) \backslash \mathrm{GL}_2(\mathbb{A_Q})} \varphi\begin{pmatrix} h & \\ & 1 \end{pmatrix} \varphi'(h) |\det h|^{s - \frac{1}{2}} \, dh.\] This is precisely the integral $\mathcal{L}(s,F \times u)$ if $\varphi$ is the adèlic lift of $F$ and $\varphi'$ is the adèlic lift of $u$.

The vector $\varphi$ has the Fourier expansion \[\varphi(g) = \sum_{\gamma \in N_2(\mathbb{Q}) \backslash \mathrm{GL}_2(\mathbb{Q})} W_{\varphi} \left(\begin{pmatrix} \gamma & \\ & 1 \end{pmatrix} g\right),\] where

\[W_{\varphi}(g) = \int\limits_{N_3(\mathbb{Q}) \backslash N_3(\mathbb{A_Q})} \varphi(ng) \psi^{-1}(n) \, dn.\] By unfolding, we find that the global Eulerian integral is equal to \[\int\limits_{N_2(\mathbb{Q}) \backslash \mathrm{GL}_2(\mathbb{A_Q})} W_{\varphi}\begin{pmatrix} h & \\ & 1 \end{pmatrix} \varphi'(h) |\det h|^{s - \frac{1}{2}} \, dh,\] which in turn is equal to \[\int\limits_{N_2(\mathbb{A_Q}) \backslash \mathrm{GL}_2(\mathbb{A_Q})} W_{\varphi}\begin{pmatrix} h & \\ & 1 \end{pmatrix} W'_{\varphi'}(h) |\det h|^{s - \frac{1}{2}} \, dh,\] where

\[W'_{\varphi'}(h) = \int\limits_{N_2(\mathbb{Q}) \backslash N_2(\mathbb{A_Q})} \varphi(ng) \psi(n) \, dn.\]

The point of this is that if we choose $\varphi$ and $\varphi'$ to be pure tensors $\bigotimes_{p \leq \infty} \varphi_p$ and $\bigotimes_{p \leq \infty} \varphi_p'$, then this global Eulerian integral factorises into the product of local Eulerian integrals of the form

\[\int\limits_{N_2(\mathbb{Q}_p) \backslash \mathrm{GL}_2(\mathbb{Q}_p)} W_{\varphi_p}\begin{pmatrix} h_p & \\ & 1 \end{pmatrix} W'_{\varphi_p'}(h_p) |\det h_p|^{s - \frac{1}{2}} \, dh_p.\]

Now if $\pi_p$ and $\pi_p'$ are both unramified and $\varphi_p$ and $\varphi_p'$ are both chosen to be spherical vectors (and normalised appropriately), then this local Eulerian integral is equal to $L(s, \pi_p \otimes \pi_p')$, the local component of the Rankin-Selberg $L$-function $L(s,\pi \otimes \pi')$ associated to $\pi$ and $\pi'$. So if both $\pi$ and $\pi'$ are unramified everywhere, then the global Eulerian integral is equal to \[\prod_{p \leq \infty}L(s, \pi_p \otimes \pi_p') = \Lambda(s, \pi \otimes \pi'),\] the completed Rankin-Selberg $L$-function (i.e. the finite part $L(s,\pi \otimes \pi) = \prod_p L(s, \pi_p \otimes \pi_p')$ multiplied by the infinite part, which is a product of gamma factors). In particular, this is the case if $\varphi$ is the adèlic lift of an even Hecke-Maaß cusp form $F$ of level $1$ on $\mathrm{GL}_3$ and $\varphi'$ is the adèlic lift of an even Hecke-Maaß cusp form $u$ of level $1$ on $\mathrm{GL}_2$.

What if either $\pi_p$ or $\pi_p'$ is ramified? Well, if $p$ is a prime (i.e. $p \neq \infty$), $\pi_p$ is ramified, but $\pi_p'$ is unramified, then Jacquet, Piatetski-Shapiro, and Shalika prove the existence of a vector $\varphi_p$ for which the local Eulerian integral is still equal to $L(s, \pi_p \otimes \pi_p')$ whenever $\varphi_p'$ is equal to the (appropriately normalised) spherical vector. They call $\varphi_p$ the essential vector, though a better name would be local newform, since this is the generalisation to $\mathrm{GL}_n$ of the $\mathrm{GL}_2$-notion of a newform. In particular, it is invariant under a congruence subgroup.

What about if $\pi_p'$ is ramified? In this case, if we take $\varphi_p$ and $\varphi_p'$ both to be local newforms, then the local Eulerian integral vanishes! So these are not the right choice of vectors. In fact, very little is known about "natural" choices of vectors for which the local Eulerian integral is equal to the local $L$-function.

In particular, let's take $p = \infty$ and $\pi_{\infty}'$ to be the ramified principal series representation $\mathrm{sgn} |\cdot|^{it} \boxplus \mathrm{sgn} |\cdot|^{-it}$ of $\mathrm{GL}_2(\mathbb{R})$; this is the component at infinity of an odd Hecke-Maaß cusp form on $\mathrm{GL}_2$ with Laplacian eigenvalue $1/4 + t^2$, while $\pi_{\infty}$ will be taken to be a spherical representation of $\mathrm{GL}_3(\mathbb{R})$, so that it is the local component at infinity of an even Hecke-Maaß cusp form on $\mathrm{GL}_3$. Then your question boils down to the following:

Do there exist vectors $\varphi_{\infty} \in \pi_{\infty}$ and $\varphi_{\infty}' \in \pi_{\infty}'$ such that the local Eulerian integral

\[\int\limits_{N_2(\mathbb{R}) \backslash \mathrm{GL}_2(\mathbb{R})} W_{\varphi_{\infty}}\begin{pmatrix} h_{\infty} & \\ & 1 \end{pmatrix} W'_{\varphi_{\infty}'}(h_{\infty}) |\det h_{\infty}|^{s - \frac{1}{2}} \, dh_{\infty}\] is equal to $L(s, \pi_{\infty} \otimes \pi_{\infty}')$?

Jacquet has shown that such vectors do exist (and that the vectors are $K$-finite, where $K$ is the maximal compact subgroup of the relevant reductive group), but the proof is completely nonconstructive. Moreover, there is no reason to expect uniqueness.

Very recently, Hirano, Ishii, and Miyazaki have announced that they have discovered an explicit description of a pair of vectors satisfying this. The description is unfortunately rather complicated (it involves the decomposition of $\pi_{\infty}$ and $\pi_{\infty'}$ into $K$-types, and choosing explicit vectors in particular $K$-types). Moreover, they don't explicitly state the relation between their vectors and the "natural" choice of vectors; here the "natural" vector is the spherical vector for $\pi_{\infty}$ and the unique vector (up to scalar) lying in the minimal $K$-type of $\pi_{\infty'} = \mathrm{sgn} |\cdot|^{it} \boxplus \mathrm{sgn} |\cdot|^{-it}$, which is the determinant representation of $K = \mathrm{O}(2)$ (which is one-dimensional). Finally, their result must be taken with a pinch of salt, since the proofs are yet to appear.

Answer 2

Like Peter Humphries suggests, you need to replace u with $\Lambda_2 u$ and replace $F$ with something that transforms appropriately on the upper-left copy of SO(2).

Thinking of the weight 2 raise $\tilde{u}=\Lambda_2 u$ as a function on $\Gamma\backslash G$, and provided I have my signs correct, this function transforms as $$ \tilde{u}\left(\begin{pmatrix}y&x\\&1\end{pmatrix}\begin{pmatrix}\cos\theta&-\sin\theta\\ \sin\theta&\cos\theta\end{pmatrix}\right) = \tilde{u}\left(\begin{pmatrix}y&x\\&1\end{pmatrix}\right) e^{-2i\theta}. $$

So you need some raise of $F$ that transforms by $e^{2i\theta}$.

A little background: The higher-weight structure of $GL(3)$ is attached to the representations of $SO(3,\Bbb{R})$, and there is exactly one of these (up to isomorphism) for every odd dimension (taking the trivial representation at dimension 1), these are Wigner-D matrices. The spherical forms have images in every dimension except 3. We like to collect the $K$-finite scalar-valued forms into (row) vector-valued forms which transform by the Wigner D-matrix. For $d \ge 2$, if $\tilde{F}=(\tilde{F}_{-d}, \ldots, \tilde{F}_d)$ is a vector-valued form that transforms by the $(2d+1)$-dimensional Wigner D-matrix (that is, $\tilde{F}(gk)=\tilde{F}(g)\mathcal{D}^d(k)$), then the entry at -2 has what you need, i.e. $$ \tilde{F}_{-2}\left(g\begin{pmatrix}\cos\theta&-\sin\theta\\ \sin\theta&\cos\theta\\&&1\end{pmatrix}\right) = \tilde{F}_{-2}(g) e^{2i\theta}. $$

In the notation of this paper, there are two vector-valued forms at $d=2$ which might work: $Y^2 F$ and $Y^0 Y^2 F$. I would guess that you want $$ (Y^2 F)_{-2} = \tilde{Y}^{0,2}_{-2,-2} F = Z_{-2} F=(2y_2 \partial_{y_2}-y_1\partial_{y_1}-2iy_2\partial_{x_2})F, $$ and this guess matches up nicely with Peter Humphries', as well.