Orthonormal basis for $L^2(G/H)$.

Let $G$ be a locally compact group and $H$ be a closed subgroup of $G$. Is there any way to define a reasonable orthonormal basis for $L^2(G/H)$? By "reasonable" I mean elements of the orthonormal basis should be continuous and whose supports are compact and of bounded measure (i.e. there is a large number $M$ such that $\mu(supp f)\leq M$ for all $f$ in the basis).

At least, do you know any reference to learn about concrete orthonormal bases on $L^2(G)$, where $G$ is a locally compact group?

-
I'm afraid that, except for compact $G$'s, there is no general answer to your question. Already for $G$ non-compact abelian and $L^2(G)$, the question is very serious: after all, one of the goals of wavelet theory is to construct orthonormal bases of $L^2(\mathbb{R})$! –  Alain Valette Jan 21 '13 at 11:43
Thanks Alain for sharing your insight. –  Vahid Shirbisheh Jan 21 '13 at 11:49

I do not see what the group has to do with the question--maybe you should redefine 'reasonable' to include the group structure. I know nothing about the groups so here is a general argument which I hope can be made to work. Fix a sequence of disjoint, open sets $A_n\subset G$ such that $\mu(A_n)\leq 1$ for all $n$, $\mu(clA_n\setminus A_n)=0$ for all $n$ and $\bigcup_n cl A_n=G$. Those sets are like cubes in $R^n$. Let $C_0(A_n)$ be the space of all continuous functions on $cl A_n$ which are zero on $cl A_n\setminus A_n$. (If $G$ is metrisable) $C_0(A_n)$ is separable and dense in $L_2(A_n;\mu)$ so do a Gram-Schmidt orthogonalisation on any sequence spaning $C_0(A_n)$ to get the orthonormal basis in $L_2(A_n)$. It consists of continuous functions that are sero on the boundary so extend to continuous on $G$ and measure of the support is $\leq 1$. Take all those bases together and it fits your requirements.

-
I am interested in groups or homogeneous spaces, because I am looking for a suitable way to define a convolution like product and define a regular representation for certain algebras on $L^2(H\backslash G)$. I have been thinking about a similar construction as you described in your answer. But it does seem have several problems: 1. There is no general recipe to partition a general group as the union of cubes in $\mathbb{R}^n$. The Gram-Schmidt orthogonalisation process does not give us an explicit orthonormal basis to work with, but it proves the existence of an orthonormal basis. –  Vahid Shirbisheh Mar 14 '13 at 6:20

This was merely too long for a comment.

There is a certain preferable choice in a special situation, where one can choose an orthornormal basis of a rather specific nature.

Let $H$ be cocompact in $G$, then $L^2(H \backslash G)$ decomposes into a direct sum of unitary, irreducible subspaces $V_\pi$. Now assume furthermore that $G$ admits a large compact subgroup $K$ in the sense that $Res_K V_\pi$ decomposes with finite multiplicity, then you can construct an orthonormal basis of $V_\pi$ in terms of irreducible representations of $K$. This is often done in the theory of automorphic representations, where these orthonormal vectors are then called automorphic forms (or rather finite linear combinations of them).

For example, consider $H$ to be a uniform lattice in $G=SL_2(\mathbb{R})$ and $K=SO(2)$. The vectors introduced above will then be real analytic, but have no finite support.

In general, there is no hope for a canonical basis in terms of representations. Already for $H =SL_2(\mathbb{Z}) \subset G = SL_2(\mathbb{R})$, the decomposition will involve a direct integral of irreducible representations.

-
Your suggestion looks promising (in special cases), but my original problem has an easier solution when $H\backslash G$ is compact. Anyway, I would like to learn more about the technique you suggested. Could you give some references? –  Vahid Shirbisheh Mar 14 '13 at 6:21
I am curious now. What is the easier solution? For the discrete decomposition, there is Deitmar Echterhoff "Principles of Harmonic Analysis" Chapter 9. The decomposition with respect to K is standard for semisimple groups over local field.Harish Chandra "Discrete series 2" first few pages will do for general Lie groups. Writing general G locally compact as projective limit of Lie groups helps to extend HC results. I also discuss related stuff in my Phd Thesis Chapter 5 ff. available on my homepage. –  plusepsilon.de Mar 14 '13 at 7:14
Thanks for references. The original problem is to show the Hecke algebra $\mathcal{H}(G,H)$ has a left regular representation on $L^2(H\backslash G)$. When $H\backslash G$ is discrete, there is a proof based on an orthonormal basis which I wanted to generalize to locally compact case. However, when $H$ is cocompact, one can use standard techniques like the Fubini theorem to show that the convolution product defines a representation. –  Vahid Shirbisheh Mar 14 '13 at 7:33