Question

Dear all,

Does anyone have some idea to describe the double coset $P(F)\backslash G(F)/H(F)$ , say using Weyl group elements ? Here $G=GL_n\times GL_{n-1}$ is defined over a number field $F$ , $H=GL_{n-1}$ diagnoal embedded into $G$ as a subgroup and $P$ is some standard parabolic of $G$ .

The interesting point is that $H$ is not the fixed point set of some involution on $G$ so the quotient is not a symmetric space. Such example appers e.g. in the theory of Rankin-Selberg convolutions.

Let's start from a special case: say P is maximal parabolic.

Any comments and references will be welcome. Thank you !

Answer 1

First of all some remarks:

1. The pair that you discussed is spherical, so it is known that there is a finite number of such orbits (formally speaking it is implied only in char $0$ case, but it does not matter here)
2. A convenient way to think of the spherical space $G/H$ is as $GL_n$ where the action of $G$ is given by the left action of $GL_n$ and the right action of $GL_{n-1}$.

Now to your question: I'll try to give a set of representatives for the case when $P$ is the Borel. For arbitrary parabolic, the set of representatives should be a subset of the set I'll describe. It should be not so hard to find it, although it is not completely trivial even in the classical case of Bruhat cells.

After a suitable chose of the Borel the problem becomes equivalent to classifying the orbits in $GL_n$ under the left action of lower triangular matrices in $GL_n$ and right action of upper triangular matrices in $GL_{n-1}$. Here I consider $GL_{n-1}$ to be embedded into $GL_{n}$ as the upper left corner.

I think that the following set will do: the set of matrices of the type $w+b$ where $w$ is a permutation matrix and $b$ is a matrix with first $n-1$ columns equal to $0$, in the last column all the entries below (and including) the $j$-th entry also $0$, and the others allowed to be either $0$ or $1$. Here $j$ is the index of the non zero entry in the last column of $w$.

I'm ~95% sure that this set covers all the orbits and ~75% sure that it covers each orbit once. Basically it is an easy exercise, but one has to be careful when doing it. I was not, so please double check me.

Additional remarks:

1. This is only one possible choice, there are many others. I do not claim that this one is a good choice from some high level point of view, this is just the first one that came into my mind.
2. A similar question is the description of $K \backslash G(F)/H(F)$ where $K$ is a maximal compact subgroup of $G(F)$. We discussed this question in your case in http://arxiv.org/abs/0910.3199.
3. For general spherical case your question might be discussed in:

P. Delorme, Constant term of smooth H -spherical functions on a reductive p-adic group. Trans. Amer. Math. Soc. 362 (2010), 933-955. - http://iml.univ-mrs.fr/editions/publi2009/files/delorme_fTAMS.pdf.

or

Y. Sakellaridis and A. Venkatesh, Periods and harmonic analysis on spherical varieties. In preparation.

Good luck.