# An interesting double coset in the theory of automorphic forms

Does anyone have some idea to describe the double coset $P(F)\backslash G(F)/H(F)$ , say using Weyl group elements ? Here $G=GL_n\times GL_{n-1}$ is defined over a number field $F$ , $H=GL_{n-1}$ diagnoal embedded into $G$ as a subgroup and $P$ is some standard parabolic of $G$ .

The interesting point is that $H$ is not the fixed point set of some involution on $G$ so the quotient is not a symmetric space. Such example appers e.g. in the theory of Rankin-Selberg convolutions.

Let's start from a special case: say P is maximal parabolic.

Any comments and references will be welcome. Thank you !

• unknown, this seems somewhat unpleasant for general $P$, but the $(n-1,1)\times (n-2,1)$-parabolic case should be pretty straightforward, since then $P\backslash G$ is "projective $n$-space times projective $(n-1)$-space".
– B R
Feb 3, 2012 at 4:19

First of all some remarks:

1. The pair that you discussed is spherical, so it is known that there is a finite number of such orbits (formally speaking it is implied only in char $0$ case, but it does not matter here)
2. A convenient way to think of the spherical space $G/H$ is as $GL_n$ where the action of $G$ is given by the left action of $GL_n$ and the right action of $GL_{n-1}$.

Now to your question: I'll try to give a set of representatives for the case when $P$ is the Borel. For arbitrary parabolic, the set of representatives should be a subset of the set I'll describe. It should be not so hard to find it, although it is not completely trivial even in the classical case of Bruhat cells.

After a suitable chose of the Borel the problem becomes equivalent to classifying the orbits in $GL_n$ under the left action of lower triangular matrices in $GL_n$ and right action of upper triangular matrices in $GL_{n-1}$. Here I consider $GL_{n-1}$ to be embedded into $GL_{n}$ as the upper left corner.

I think that the following set will do: the set of matrices of the type $w+b$ where $w$ is a permutation matrix and $b$ is a matrix with first $n-1$ columns equal to $0$, in the last column all the entries below (and including) the $j$-th entry also $0$, and the others allowed to be either $0$ or $1$. Here $j$ is the index of the non zero entry in the last column of $w$.

I'm ~95% sure that this set covers all the orbits and ~75% sure that it covers each orbit once. Basically it is an easy exercise, but one has to be careful when doing it. I was not, so please double check me.

2. A similar question is the description of $K \backslash G(F)/H(F)$ where $K$ is a maximal compact subgroup of $G(F)$. We discussed this question in your case in http://arxiv.org/abs/0910.3199.