Cusp forms and L^2 I am confused about the "bigger picture" when one goes from classical modular forms on $SL_2(\mathbb{Z})$ and its subgroups to automorphic forms (possibly non-holomorphic). 
For classical modular forms we have cusp forms and Eisenstein series. There is the Peterson product defined when one of the forms is a cusp form (since cusp forms vanish at the cusps) and Eisenstein series are the orthogonal complement of cusp forms. For cusp forms there is Atkin-Lehner theory but not (as far as I can understand) for Eisenstein series.
Now from reading various sources, in order to generalise this to nonholomorphic forms, the correct thing is to look at $L^2(\Gamma\backslash SL_2(\mathbb{R}))$.  I have figured out how to make a modular form into a function on $SL_2(\mathbb{R})$, but unless it is a cusp form it doesn't look like it is in $L^2$ (the inner product is just the generalization of Peterson, isn't it?)
On the other hand there are series which look like Eisenstein series like 
$$
E_\phi(z) = \sum_{\gamma\in\Gamma_\infty\backslash\Gamma} \phi(\mathrm{Im}(\gamma(z)))
$$
where $\phi$ is a compactly supported $C^\infty$-function on the positive reals. These obviously vanish at the cusps (and are in $L^2$) but they are (apparently) not cusp forms. 
So my specific questions are: what is the right way to generalize modular forms so that holomorphic Eisenstein series still stay in the picture? Where do these $E_\phi$ series fit in? And what happens to Hecke theory in this setting? 
I would like to also understand the role of weight 2 "almost-holomorphic" Eisenstein series in a more conceptual way.
(I don't know much functional analysis, so Gelbart's book is proving rather difficult for me. Do I have to understand all the functional analysis to get a feeling for what is going on?)
 A: This is a long answer because the question asks quite a lot of things. I agree that Gelbart's book, although inspirational, is hard for someone without a strong analytic background. The Boulder and Corvallis proceedings are full of articles which are worth studying if you want to get an understanding of automorphic theory.  It is hard to summarise but here's a (necessarily oversimplified) sketch of the "big picture" you may be looking for.
First, two important things to get straight:
1) "Cuspidal" is not the same as "vanishing at the cusps". Sticking to the upper half-place for the moment, a function is cuspidal if at each cusp the 0-th Fourier coefficient vanishes. For a holomorphic function the Fourier coefficients are constant, so cuspidal and vanishing at cusps are the same thing. But for those functions $E_\phi$ the 0-th Fourier coefficient at $\infty$ is some function $c_0(y)$ vanishing in a neighbourhood of infinity, but certainly non-zero in general. In fact, $f\in L^2(\Gamma\backslash\mathfrak{H})$ is cuspidal iff it is orthogonal to all the $E_\phi$ (compute the Petersson product to see this), so $\{E_\phi\}$ generates a complement to the cusp forms in $L^2$. Arithmetically, though, they aren't interesting.
2) Automorphic forms and $L^2(\Gamma\backslash G)$ are different beasts entirely. An automorphic form is a smooth function on $\Gamma\backslash G$  satisfying various properties (moderate growth, K-finite, and killed by an ideal of finite codimension in the centre of the universal enveloping algebra).  It does not have to be in $L^2$. For example, holomorphic Eisenstein series are automorphic forms which are not in $L^2$.
Hecke theory: for $SL_2(\mathbb{Z})$ you can just mimic the classical definitions in this more general setting, but for groups other than $GL_2/\mathbb{Q}$ you really need to look at the adelic setting. Here an automorphic form (for a reductive group $G/\mathbb{Q}$) is a smooth function on $G(\mathbb{Q})\backslash G(\mathbb{A})$, satisfying a bunch of properties (see Borel and Jacquet's article in Corvallis for a precise definition, indeed for just about everything here). The finite adelic group $G(\mathbb{A}_f)$ acts by right translation on the space of automorphic forms, and this is the right generalisation of Hecke operators. The Lie group $G(\mathbb{R})$ does not act on the space of automorphic forms (K-finiteness is not preserved) but there is a suitable algebra of invariant differential operators (the Hecke algebra of $G(\mathbb{R})$) which does. 
Cuspidal automorphic forms - in the adelic setting - are those $F$ for which the function
$$
F_N(g) = \int_{N(\mathbb{Q})\backslash N(\mathbb{A})} F(hg)\ dh
$$
vanishes for suitable unipotent subgroups $N$. For $GL_2$ these are just the unipotent radicals of Borel subgroups (defined over $\mathbb{Q}$), and since these are all conjugate it's enough to verify the vanishing for the upper triangular unipotent subgroup. 
A classical cusp newform $f$ of weight $k$ then gives rise to a cuspidal automorphic form $F$ on $GL_2/\mathbb{Q}$. There are two different things one can now do with $F$:
1) The translates of $F$ under $GL_2(\mathbf{A}_f)$ generate an irreducible representation $V_f$, which encodes the action of the Hecke operators (and more). Elements of this representation are none other than oldforms associated to $f$. More precisely, $V_f$ is an infinite tensor product of representations $V_p$ of $GL_2(\mathbb{Q}_p)$. If $p$ doesn't divide the level of $f$, then $V_p$ tells you the Hecke $T_p$ and $R_p$ eigenvalues. At bad primes, it contains much more delicate information than classical Atkin-Lehner theory does (one reason to use the adelic approach even for $GL_2$).  
2) The translates of $F$ under the Hecke algebra at infinity, on the other hand, generate an irreducible representation $V_\infty$ of a particular type (discrete series with parameter given by $k$), inside which $F$ is characterised as the lowest weight vector (this is the group-theoretic interpretation of the holomorphy of $f$).
The space of all translates of $F$ (i.e. by the finite adelic group and the Hecke algebra at infinity) is just $V_\infty\otimes V_f$. It is an example of an automorphic representation. (In general, an automorphic representation is any irreducible subquotient of the spaces of automorphic forms under these actions.)
The map $F\mapsto F_N$ allows one to describe the quotient (automorphic forms)/(cusp forms) by induced representations from parabolic subgroups. Although explicit, this quotient is quite a complicated representation - in particular, it is very far from being semisimple, even for $GL_2$.
So, if you are looking at this from a number-theorist's perspective, why care about $L^2$ ? The reason is the trace formula, which needs to be formulated in the setting of Hilbert spaces. There is essentially no difference between an $L^2$-form which is cuspidal and an automorphic cusp form, so the trace formula, appropriately wielded, can tell you a lot about cusp forms. It is a powerful and indispensable tool. But to apply it you need also to look at the rest of the $L^2$, in which lives the continuous spectrum, accounted for by real-analytic Eisenstein series at $Re(s)=1/2$.   Here there is a lot of analysis but you can usually find a friendly expert to help you out.
PS: Some people like to define a cuspidal automorphic representation as an irreducible subspace of $L^2_{\mathrm{cusp}}$. This has some advantages: (a) it's concise, and (b) at infinity one is working with genuine (unitary) representations of the real Lie group, rather than $(\mathfrak{g},K)$-modules (equivalently, representations of the real Hecke algebra). But it only gives the correct answer for cuspidal representations.
