When I first starting studying modular forms, I was told that we can consider any congruence subgroup $\Gamma\subset\operatorname{SL}_2(\mathbb{Z})$ as a level, but very soon the book/lecturer begins to focus on the three types which are usually denoted by $\Gamma(N),$ $ \Gamma_0(N)$ and $\Gamma_1(N)$, where $N\in \mathbb{Z}_{\geq 1}$ and pretty much forgets about the rest. Back then I thought it was because these were the only ones that had a nice moduli space description. But, now that I know that $(\operatorname{GL}_2,\mathcal{H}^{\pm})$ is a PEL-type Shimura datum, I am not entirely convinced by my old reasoning either.

I have two possible guesses as to why this might be happening: 

 1. Since the Shimura variety attached to $(\operatorname{GL}_2,\mathcal{H}^{\pm})$ is the inverse system $\{\operatorname{Sh}_K(\operatorname{GL}_2,\mathcal{H}^{\pm})\}_K$ for all small enough $K$, it is enough that we care about the 'really small $K$s', because we can study the cohomologies, Galois action, Hecke operators just using these ones.

2. Another theory I have is relate to the models over the reflex field. The complex points of the varieties of different levels sure have a nice moduli description using elliptic curves, but I believe there does not exist a moduli description over the reflex field (which I am guessing is $\mathbb{Q}$ in this case). What lead me to believe this is the issues caused by Weil pairing at level $\Gamma(N)$.

I am not entirely sure about either of these arguments though. Thanks in advance for your comments and answers.