First, one should be a little careful about saying that $L^2(G_k\backslash G_\mathbb A)$ has $L^2(Z_\mathbb A G_k\backslash G_\mathbb A,\omega)$ inside it... since appearing as direct integral "integrands" is not a very strong commitment. If $G$ has non-compact center, $L^2(G_k\backslash G_\mathbb A)$ will have no discrete spectrum at all... which gives the wrong impression (by Selberg et al's proof of various forms of "Weyl's Law" for arithmetic subgroups, namely, that the bulk of the spectrum is cuspidal, hence, discrete).
That Gelfand condition about integrals over all unipotent radicals being $0$ is far from obviously "the right thing". It is less a leap to understand that instead of "cusps" we should think of "parabolics" (etc).
The constant terms along various parabolics correspond to "going to infinity" in the variety of fashions possible in general, in higher-rank groups.
Still, yes, it is mildly amazing that vanishing of all constant terms guarantees discreteness. In general, it is easy to fail to prove this... :)
I think that Y. Colin de Verdiere's argument, cast into general form by Jacquet, that appears in Moeglin-Waldspurger's book, is potentially the clearest in terms of describing the causality, as it shows a somewhat more general thing, that square-integrable automorphic forms ($K$-finite, $\mathfrak z$-finite) all of whose various constant terms vanish above some fixed height(s), is already discrete. This follows by proving that the resolvent for Casimir on such a space is compact, which follows by proving a sort of Rellich compactness lemma for an inclusion of a Sobolev $H^1$ into $H^o=L^2$, as appears in Lax-Phillips' book on Automorphic Scattering, about page 204 and following. (The earlier parts are not essential to understanding what's happening just there.)
The rough explanation I've heard, and sometimes repeat, although it doesn't truly explain so much, is that (eventual) vanishing of all constant terms says that the given afm has $0$ "average mass" "at infinity", so that it behaves as though it lived on a compact manifold, where a simpler Rellich lemma would apply (by a smooth partition of unity, and reducing to the essentially elementary case of a product of circles, and Fourier series).
The historical version of "holomorphic cuspform" (also in the Siegel and Hilbert modular cases) played on some good fortune, in some regards. If it seems lucky, you're probably right.
EDIT: in response to comment/query... No, there is no general rubric that says that parabolic subgroups determine spectral features of automorphic forms. Plausibly, in a different universe, the stratification of automorphic $L^2$ could be different. Thus, although the rational-rank-one case was relatively easy to (optimistically) extrapolate from the $SL_2(\mathbb R)$ case, where the cross-sections going out to the point-cusps were elementary objects, all the pseudo-down-to-earth ideas about "going to infinity" and "cusp" that seemed to be decisive for elliptic modular forms, and for Maass' waveforms, rather abruptly not only "fail", but fail qualitatively.
Thus (to my mind) Langlands' earning his spot at IAS in the 1960s, for, among other things, carrying out Selberg's highly-optimistic sketch of automorphic spectral decompositions. A number of important, critical surprises: there're not just two sorts of things, cuspforms and "continuous spectrum", and then a little leftover, constants, but a whole range of things. Yes, as has only been proven in recent years, the discrete spectrum dominates, and the discrete spectrum is dominated by cuspforms. But, first, there are cuspidal-data Eisenstein series, apparently not anticipated by Selberg. But, as is the subtlest part of Langlands' SLN 544, and addressed completely only for $GL_n$, in Moeglin-Waldspurger's 1989 paper, there are many non-constant $L^2$ residues of Eisenstein series, for $GL_n$ at least called Speh forms, because Birgit Speh discovered the corresponding repns of real Lie groups $GL_n(\mathbb R)$.
The "constant term" along a parabolic $P$ is the trivial-character Fourier component along the unipotent radical of $P$. It is not obvious that this shadow of the thing should be important, but, yes, one proves... "the theory of the constant term"... that the aggregate of the constant terms of a $K$-finite, $\mathfrak z$-finite automorphic form determines its asymptotic behavior "at infinity".
Indeed, decompositions along other subgroups are very interesting, especially for number theoretic applications.
However, as it happens, it seems that no other decompositions-along-subgroups adequately distinguish non-compact quotients from compact... and compact quotients have discrete spectrum with respect to their invariant Laplacian, or Laplacian on suitable vector bundles.
Perhaps in a different universe the non-compactness of interesting arithmetic quotients would have been mediated by different sorts of subgroups, but in this universe the collection of all parabolics seems to do the job. Yes, this was not obvious, and Gelfand deserves substantial credit for formulating things this way...