I sort of understand the definition of a spectral sequence and am aware that it is an indispensable tool in modern algebraic geometry and topology. But why is this the case, and what can one do with it? In other words, if one were to try to do everything without spectral sequences and only using more elementary arguments, why would it make things more difficult?

9$\begingroup$ Have you seen wwwmath.mit.edu/~tchow/spectral.pdf ? $\endgroup$ – Qiaochu Yuan Dec 7 '09 at 0:45

1$\begingroup$ Only vaguelyI was going to read that whenever I decided to make my next attempt to really understand the subject (i.e., probably soon). But that answers the question of "what is a spectral sequence." $\endgroup$ – Akhil Mathew Dec 7 '09 at 1:17

4$\begingroup$ I would also recommend the book by Bott and Tu and perhaps compare the discussion there with the one in Frank Warner's book on the equivalence between Cech, deRham, and singular cohomology. Spectral sequences, like other modern mathematical abstractions, are not absolutely necessary but they provide a convenient way to do bookkeeping and also "reuse" the same proof in otherwise apparently different settings. $\endgroup$ – Deane Yang Dec 7 '09 at 1:42

2$\begingroup$ I have one question (not worth its own question) that perhaps someone here can answer. When I learnt spectral sequences as a grad student (from a number of sources: Bott and Tu, Lang, HiltonStammbach if memory serves me) I used to refer, say, to the $E_2$ term of the spectral sequence. In recent times, and in some of the answers below, people refer to the $E_2$ page. Not having used them for a while, I first heard "page" from Mike Hopkins during his talk on the Kervaire problem at the Atiyah 80 earlier this year. I like the imagery, but wonder at the origin. Does anyone know? Thanks! $\endgroup$ – José FigueroaO'Farrill Dec 7 '09 at 20:42

1$\begingroup$ This is a comment from a paper of Mac Lane regarding the work of Charles Ehresmann (archive.numdam.org/article/CTGDC_1980__21_4_353_0.pdf), " At this point one should also note that the concept of a Serre fibration involves much more than the covering homotopy theorem, since the added structures given by the various associated spectral sequences also enter. Probably a spectral sequence should be viewed not as an independent structure, but as an (important) algebraic constituent of the structure <<fibre space>>. [...] $\endgroup$ – Samantha Y Dec 8 '17 at 20:22
Let's say you have a resolution $0\to A\to J^0\to J^1\to\dots$ (of a module, a sheaf, etc.) If $J^n$ are acyclic (meaning, have trivial higher cohomology, resp. derived functors $R^nF$), you can use this resolution to compute the cohomologies of $A$ (resp. $R^nF(A)$). If $J^n$ are not acyclic, you get a spectral sequence instead, and that's the best you can do.
Let us say you have two functors $F:\mathcal A\to\mathcal B$ and $G:\mathcal B\to \mathcal C$. Let us say you know the derived functors for $F$ and $G$ and would like to compute them for the composition $GF$. Answer: Grothendieck's spectral sequence.
1 and 2 account for the vast majority of applications of spectral sequences, and provide plenty of motivation  I am sure you will agree.
The reason for the spectral sequences in both cases is the same. Intuitively, $A$ in case 1 (resp. $F(A)$ in case 2) is made of parts which are not themselves elementary. Instead, they are made (via an appropriate filtration) from some other elementary, "acyclic" objects.
So there is a 2step process here. You can do the first step and the second step separately but they are not exactly independent of each other. Instead, they are entangled somehow. The spectral sequence gives you a way to deal with this situation.

$\begingroup$ Thanks! I think this (especially 1) is the motivation I was looking for. I was vaguely aware that there was a spectral sequence for composing functors but it seemed somewhat nonintuitive at first. $\endgroup$ – Akhil Mathew Dec 7 '09 at 2:30
Qiaochu links to a really nice article by Timothy Chow that says a lot about the mechanics of how to go from a filtered complex to its spectral sequence. Two questions that remain are, (1) why do filtered complexes show up so much, and (2) is there anything that you could do with a filtered complex other than compute its spectral sequence?
VA gives a very general motivation for why filtered complexes show up so much. Probably it is just more general than what I was going to say. To discuss things more concretely, (I am told that VA's interesting answer is not a generalization of this paragraph.) For question (1), here are two general ways that filtered complexes show up: (a) you might get a filtered complex if you are interested in a chain complex that comes from a filtered object. For instance it could be a stratified topological space. A CW complex is stratified, but it has a special structure so that its chain complex doesn't need a spectral sequence. (It is an acyclic resolution as VA says.) If you have a stratification that you wish was CW but isn't, then you just have a general filtered complex. (b) There are some cases where just a single map or similar gives you a filtered complex. These may all be special cases of what Grothendieck described. For example, suppose that you are interested in the de Rham cohomology of a manifold. (This is then a certain resolution of the sheaf of smooth constant functions on the manifold.) Suppose further that the manifold is fibered over another manifold, or maybe is just foliated. Then differential forms are filtered by how parallel they are to the foliation.
Concerning question (2), there is an interesting result for filtered complexes over a field coming from representation theory. (I learned this some years ago in a discussion with Michael Khovanov.) The theorem in this case is that there is no more information in a filtered complex than in its spectral sequence. For simplicity, let's look at filtrations of length $n$. Then a filtered vector space is a representation of the $A_n$ quiver with all arrows in the same direction. (Each term of the filtration is assigned to a vertex of the quiver, and the arrows correspond to the inclusions.) The quiver also has other representations, but filtered vector spaces are the projective modules. A filtered complex is thus a projective complex over the $A_n$ quiver algebra. It is just like homological algebra over any other ring; you usually look at projective complexes. Since the $A_n$ quiver has projective dimension 1, it is easy to identify the indecomposable chain complexes of filtered vector spaces. There are two types, with two terms and with one terms: $$0 \to k^{(i)} \to k^{(j)} \to 0 \qquad\qquad 0 \to k^{(i)} \to 0.$$ In this notation, $k$ is the ground field and $k^{(i)}$ is $k$ in degree $i$. The first type of complex is valid if $i \ge j$. If you compute the spectral sequence of an indecomposable complex, you will see that detects the first type of term at the $(ij)$th page, and kills it on the next page. The second type of complex is of course the surviving homology. You can also go backwards and reconstruct the filtered complex from its spectral sequence.
Of course it is simplistic to only discuss filtered complexes and spectral sequences over a field. Nonetheless, roughly speaking spectral sequences are no more than a framework for analyzing filtered complexes.
VA asked for more details about the indecomposable modules, which is a fair request because I was quite cryptic about the relation. In particular, I use nonstandard indexing in my own thinking about this. Unfortunately, I'm not sure that I can convert to standard indexing without making a mistake, so I won't convert. But I did change one thing above: I fixed the filtration degrees so that they are correct.
Suppose that $C = (C_n)$ is a complex of filtered vector spaces. Say that the filtration is increasing and indexed by $\mathbb{Z}_\ge 0$. The complex has two degrees: The chain degree $n$ and the filtration degree $k$. Suppose that $\partial$ is a differential with chain degree $1$ and filtration degree $0$. (This is where the numbering begins to be nonstandard, although it makes sense from the point of view of quiver representations.) Then the theorem is that over a field, the two kinds of indecomposable complexes are those listed above, where the term $k^{(i)}$ has chain degree $n$, and the term $k^{(j)}$ (if present) has chain degree $n1$. To be precise about what $k^{(i)}$ means, it is a filtration of the field $k$ in which the degree $j$ subspace is $0$ when $j < i$ and $k$ when $j \ge i$.
The page $E^0$ is the associated graded complex. The page $E^r$ has a differential of degree $(1,r)$. When $r = ij$, the differential $\partial^r$ of the first type of indecomposable complex connects the "tip" of $k^{(i)}$ to the "tip" of $k^{(j)}$ and kills them both on the next page. The other kind of indecomposable complex has a vanishing differential, so the induced differential on every page also vanishes.

$\begingroup$ Thanks! Do you have a reference for the de Rham spectral sequence for a manifold fibered/foliated over another? $\endgroup$ – Akhil Mathew Dec 7 '09 at 2:33

1$\begingroup$ The book "Singularities" by Peter Orlik has a discussion, and refers to what could be the original paper, "Characteristic invariants of foliated bundles", by Kamber and Tondeur. springerlink.com/content/pk53617932261r7m $\endgroup$ – Greg Kuperberg Dec 7 '09 at 2:48

$\begingroup$ A very nice example with a quiver! But what exactly is the statement? Knowing ALL terms of a spectral sequence (including $E_0^{pq}$) one can reconstruct the filtered complex $(A^n,d)$? Or only knowing $E_1^{pq}$? (The term $E_0^{p,q}$ already gives the graded pieces $Gr^pA^n$, and so each $A^n$ if it's a vector space.) Could you clarify? P.S. For the de Rham cohomology, one needs a resolution of the constant sheaf R; the sheaf of smooth functions is fine, and so acyclic. $\endgroup$ – VA. Dec 7 '09 at 4:41

$\begingroup$ Since the first type of indecomposable survives until the $(ji)$th page, you have to know all of the pages. Let me clarify it this way: Every filtered complex is a direct sum of indecomposable ones, and the spectral sequence is a corresponding direct sum. So just compute the spectral sequence of the indecomposables listed above, and you will be enlightened. $\endgroup$ – Greg Kuperberg Dec 7 '09 at 5:09

1$\begingroup$ Greg, you shouldn't be so down on your answer; yours is much more general, and gives a much completer view of where spectral sequences come from than VA's (though of course, that is a very important special case). $\endgroup$ – Ben Webster♦ Dec 7 '09 at 15:36
Let me first try to answer a simpler question:
Why are long exact sequences so ubiquitous?
Almost anything that is written as a capital letter, followed by a subscript _{i} or superscript ^{i}, i an integer, and finally some stuff in parentheses, can be interpreted as π_{i} of some spectrum (or sometimes space, as in nonabelian group cohomology, or maybe a sheaf of spectra or spaces...). And almost any long exact sequence which involves three similar terms in a cycle, with _{i} decreasing by 1 every three terms, comes from the long exact sequence of homotopy groups of a fiber sequence of spectra or spaces. For instance, the long exact sequence for the cohomology of a group G with coefficients in a short exact sequence of Gmodules A → B → C corresponds to (HA)^{hG} → (HB)^{hG} → (HC)^{hG}, since H^{i}(G, M) = π_{i}((HG)^{hM}) (which is nonzero only for nonpositive i), which is a fiber sequence because HA → HB → HC is one (since A → B → C is a SES) and (–)^{hG} is a homotopy limit, so it preserves fiber sequences. Or, I could draw a square with these three terms in it and 0 in the lower left, which is both a pullback and pushout square (since spectra form a stable (∞,1)category).
Long exact sequences are actually a special case of spectral sequences—those whose E^{1} page has only two columns. If you have never done this before, you should check for yourself that the d_{1} and the extension problems exactly tell you that there is a long exact sequence formed by the two columns and whatever the spectral sequence converges to. This suggests that we could try to generalize the picture with a pushout/pullback square of spectra to find something to which a more general spectral sequence corresponds. Two possibilites are: we could extend the top row of the square to a directed sequence of spectra, and take the homotopy cofiber of each map; or we could extend the right column of the square to an inverse sequence of spectra and take the fiber of each map. These are the homotopy version of filtered and cofiltered objects, respectively; however, there is no condition on the maps (the notion of "inclusion" does not make much sense in the homotopytheoretic world). Associated to each is a spectral sequence, though there are convergence issues when the sequence of spectra is infinite. It could be that most spectral sequences encountered in practice can be viewed as arising from an underlying sequence of spectra, though I have not attempted to convince myself of this fact.
Edit: Clark Barwick suggests that one may indeed view all "natural" spectral sequences as arising from filtered spectra. He and I would like to know whether there are any convincing counterexamples, so please let me know if you have any! Note however that 1 and 2 from VA's answer are not counterexamples.

$\begingroup$ Just a typo correction: the long exact sequence is formed by the entries in the $E_1$ page and the limit of the spectral sequence, not $E_2$ (or $E^1$ and not $E^2$ if you prefer homological spectral sequences to the cohomological ones). $\endgroup$ – VA. Dec 8 '09 at 2:28

$\begingroup$ Oops. There are various ways a spectral sequence can yield a long exact sequence, but you're definitely right about the one I care about. $\endgroup$ – Reid Barton Dec 8 '09 at 3:12

4$\begingroup$ With respect to your edit, there are several natural spectral sequences abutting to homotopy groups of certain spaces, which are less likely to arise from filtered spectra. For example, the unstable Adams spectral sequence for computing homotopy from cohomology rings. $\endgroup$ – Tyler Lawson Dec 16 '09 at 13:09

4$\begingroup$ Tyler is of course right. Perhaps one should say instead that all "natural" spectral sequences come from filtered objects in an ∞category $C$ with a conservative functor $C\to D$ to a stable ∞category with a tstructure. The terms of the spectral sequence live in the heart of the tstructure, and if $C$ isn't already stable, then you get "fringe effects," which in general can be a lot more disastrous than the gentle term suggests. This perspective doesn't explain the presence of nonabelian groups and pointed sets in, e.g., the spectral sequence of a tower of fibrations, though. $\endgroup$ – Clark Barwick Jan 7 '10 at 17:08
I still don't feel as though I understand at an intuitive level what a spectral sequence "is" or what it "means." But here is one nice explanation I heard about how one particular spectral sequence (the Serre spectral sequence) arises in topology.
 Homotopy groups behave nicely with respect to products: $\pi_n(X\times Y) \cong \pi_n(X) \times \pi_n(Y)$.
 Homotopy groups behave not quite so nicely with respect to fibrations (= "twisted products"): instead of an isomorphism you get a long exact sequence.
Homotopy groups are great, but they are often hard to compute. (Co)homology is easier to compute, but it also doesn't behave as nicely on products and fibrations.
 Homology doesn't quite preserve products (i.e. take them to tensor products)it almost does, but in general there's the Künneth theorem that relates them by an SES.
 Thus, when you apply homology to a fibration, you should expect to get something that has both the problems of a long exact sequence and and the problems of the Künneth theorem. This "something" is a spectral sequence.
Whenever you have a sequence of maps of (perhaps graded) abelian groups
$$ \cdots \to A^0 \to A^1 \to A^2 \to \cdots $$
and each pair $A^{p1} \to A^p$ is involved in a long exact sequence with third term $C^p$, there is a spectral sequence whose terms are the groups $C^p$. If the sequence eventually stabilizes on both ends (or many variants of weaker hypotheses), the spectral sequence detects the difference between the limit and the colimit. For instance, if you have a space filtered by subspaces, or a chain complex filtered by subcomplexes, or some arbitrary sequence of composable maps in a triangulated category, this kind of structure arises.
The core tool introduced by homological algebra is the long exact sequence, and the spectral sequence has proved to be an extremely useful organizational tool whenever you have two or more long exact sequences that interlock; the alternative is often to simply work with the long exact sequences one by one. It's led to a lot of the methodology in algebraic topology where you can take some difficult computation that looks approximately like a structure you can handle, and handle the "easy" portion first with the difficult issues encoded by the differentials and extensions in the spectral sequence.