This is a very soft question.

Let $n\geq 1$ and let $Y(n)$ be the (open) modular curve associated to $\Gamma(n)\subset SL_2(\mathbb{Z})$. Interpreted correctly, $Y(n)\to Y(d)$ is finite etale, whenever $d$ is a positive integer dividing $n$.

Now, it is a priori possible that $Y(n)\to Y(d)$ is a "trivial" finite etale cover, i.e., it has a section.

But, of course, it isn't. With a view towards higher-dimensional analogues of the modular curves I would like to know:

Why is $Y(n)\to Y(d)$ a non-trivial finite etale cover?

I'm looking for an argument which is as "soft" as possible, and could be applied to higher-dimensional situations (or even situations different from the moduli of abelian varieties).

For instance, the genus of $Y(n)$ grows with $n$, so these covers can't be trivial. But is there a "softer", slightly more direct, argument?