# Grothendieck spectral sequence when one of the functors is contravariant

Let $f \colon X \rightarrow S$ be a morphism of schemes. I am interested in computing the cohomology groups of $$\mathbf{R}\mathscr{H}om(\mathbf{R}f_* \mathcal{O}_X, \mathcal{O}_S)$$ in terms of $\mathrm{Ext}^p(R^qf_* \mathcal{O}_X, \mathcal{O}_S)$. Is there a spectral sequence that achieves this? If so, what are the entries of its pages, how do the differentials go, which way does the eventual filtration on the infinity page go, etc.? Any references to the literature where this is explained would also be highly appreciated.

I am familiar with the Grothendieck spectral sequence for a composition of functors, but I cannot see how that applies because $\mathbf{R}f_*$ is covariant and $\mathbf{R}\mathscr{H}om$ is contravariant (in the first variable).

As alluded to in the title, the question may be posed in much greater generality, but let me stick with the above for the sake of concreteness.

EDIT. I realized that I forgot that the usual Grothendieck spectral sequence has a condition that the first functor maps injectives to acyclics for the second functor. A condition of this sort does not seem to hold in my setting, so perhaps there is no spectral sequence of the type I want. Further comments are nevertheless appreciated!

• Can you provide some motivation for the question (as it may turn out not to be necessary to answer the question posed for the purpose you have in mind)? – nfdc23 Dec 27 '15 at 2:50
• Just replace one of the categories involved with its opposite category. – Qiaochu Yuan Dec 27 '15 at 3:47
• @QiaochuYuan: I've tried that but I don't think it works (please, prove me wrong!). I don't see how to replace something by the opposite category and end up in the setting where the usual version of the spectral sequence applies (in particular, where the image category of the first functor is the source of the second). – Lisa S. Dec 27 '15 at 4:19
• @nfdc23: I don't have any more concrete question that I could pose. I've run into this type of computation in several settings in the past (manipulations with coherent sheaves), and sometimes I can end up bypassing it, but I feel I would benefit from knowing how to get the spectral sequence in such a situation. – Lisa S. Dec 27 '15 at 4:24
• @Lisa: a contravariant functor can be thought of as either a functor $C \to D^{op}$ or as a functor $C^{op} \to D$. So if you interpret Hom as a functor into $\text{QCoh}(S)^{op}$ then I think everything becomes covariant. I guess this doesn't answer your question but at least it shows that contravariance as such is not the issue. – Qiaochu Yuan Dec 27 '15 at 21:41

I think this case is actually not so obvious. The issue is that to derive $R\mathscr Hom$ in the first variable you would need to use a locally free resolution while $Rf_*$ being a covariant right derived functor would need an injective one. If you look at the usual proofs of the various forms of GSS, you'll see that the idea is to take a resolution of the first object in the first category, apply the first (derived) functor and then see that what you get in the second category computes the second derived functor. With this, as it is, you would need to switch resolutions and I am not sure I would want to get into that computation.

On the other hand, if your situation is such that $R^if_*\mathscr O_X=0$ for $i>d$ for some $d\in \mathbb Z$, for instance, $f$ is proper, $Y$ is (locally) Noetherian and $f$ has fibers of dimension at most $d$, then you can take your starting functor $R^df_*$ and consider $Rf_*\mathscr O_X$ as the left derived functor of that. (See Prop 7.4 on page 74 in [Hartshorne, R&D].) I would expect that if you know that

1. $R^if_*\mathscr F=0$ for $i>d$ for every coherent $\mathscr F$, and
2. $R^df_*\neq 0$,

then this should work. For simplicity let me assume that $f$ is a projective morphism of schemes of finite type over a field with $d$ equal the maximal fiber dimension of $f$. If you need it in more generality you can probably adjust the argument. (If it works!!).

Namely, fix a coherent sheaf $\mathscr F$ on $X$ and take a locally free resolution $\mathscr L^\bullet$ of $\mathscr F$. (Addendum: this should probably consist of locally free sheaves of the form $\oplus\mathscr O(-n)$ as we discussed in the comments, so the $R^df_*$ of this sequence would indeed be a locally free resolution of $R^df_*\mathscr F$ as claimed in the next sentence.) Then by cohomology and base change $R^df_*\mathscr L^i$ is locally free for all $i$ and hence $R^df_*\mathscr L^\bullet$ is a complex of locally free sheaves which is quasi-isomorphic to $\left(L(R^df_*\mathscr)\right)(F)$ (=the (total) left derived functor of $R^df_*$ applied to $\mathscr F$). Now if you just follow the usual proof, you should (check this!!!) get a spectral sequence something like $$E_2^{p,q}= \mathscr Ext^p(\underbrace{L^q(R^df_*)}_{R^{d-q}f_*}\mathscr F, \mathscr G)\Rightarrow E_\infty^{p,q}= h^{p+q}\left( R\mathscr Hom(Rf_*\mathscr F, \mathscr G)\right)$$

Actually I think that to do this right, here probably both $p$ and $q$ should be negative, since the locally free resolution goes that way. A simple first check is to see if you can write down the edge maps from obvious short exact sequences. For $E_2$ spectral sequences, the maps are usually compositions of two edge maps from two different short exact sequences.

So, in terms of the original functors this is not a first quadrant spectral sequence, but one should expect that anyway.

Anyway, as it is probably clear from what I have written, I did not work out the details, so it is quite possible that there is some issue I didn't consider and also, clearly I didn't even give you an explicit answer, but perhaps you can work it out. And, please do work out the details carefully (and perhaps post it here if you have).

• Thank you, this seems very interesting and I will try to work it out. I am having difficulty in applying Proposition 7.4 from Hartshorne that you mentioned: what should I take for $A$ and $P$ there? If $A$ is the category of coherent sheaves on $X$, then there may not be enough injectives, and if $A$ is the category of quasi-coherent sheaves on $X$, then I don't see any candidate $P$? As for $P$, supposing that $f$ is projective (and not merely proper), should $P$ contain all suitable $\bigoplus O(n_j)$? But then how to ensure that $R^if_* = 0$ for all $i \neq d$ and all objects in $P$? – Lisa S. Dec 27 '15 at 18:39
• Right, this is a difficulty. However, it is enough to take $\mathscr O(-n)$ for $n\gg 0$ and if $f$ is nice enough (maybe CM?) then you get the vanishing you need. What do you think? – Sándor Kovács Dec 27 '15 at 19:04
• OK, that looks promising. I guess we can get rid of the nuisance of "not enough injectives" by using quasi-coherent sheaves and by employing a limit argument as in the proof of Lemma 4.1 on p. 154. – Lisa S. Dec 27 '15 at 19:40
• (Convergence has to be dealt with, at least in the general case) – Mariano Suárez-Álvarez Dec 30 '15 at 3:12
• @MarianoSuárez-Alvarez: I'm not sure what you mean. If this idea works, then there are only finitely many $q$s for which $E_2^{p,q}\neq 0$, so there are only finitely many pages on which there are any non-zero differentials. How could this not converge? (And if it doesn't work, then convergence is not the issue). – Sándor Kovács Dec 30 '15 at 5:55

If $I^*$ and $J^*$ are global sections of appropriate injective resolutions of $f_*{\cal O}_X$ and ${\cal O}_S$, then you have a double complex that looks in part like this:

$$\matrix{ Hom(I^q,J^p)&\rightarrow&Hom(I^{q},J^{p+1})\cr &&\uparrow\cr &&Hom(I^{q+1},J^{p+1})&\rightarrow&Hom(I^{q+1},J^{p+2})}$$

Your $E_2$ terms come from taking cohomology vertically and then horizontally, so your $E_2$ differentials should go from $E_2^{p,q}$ to $E_2^{p+2,q+1}$.

• Thank you. I am having trouble coming up with a formula for $E_2^{p, q}$ that would not involve $I^q$ or $J^p$ (which depend on choices). Could you explain what the $E_2$ terms of the spectral sequence are? – Lisa S. Dec 27 '15 at 4:39
• The $E_2$ terms are exactly what you said they are in the second sentence of your post. – Steven Landsburg Dec 27 '15 at 5:19
• Thank you, I should have seen that through initially. I will think some more about convergence of the resulting spectral sequence and about what it computes. – Lisa S. Dec 27 '15 at 18:44
• @LisaS. Regarding convergence, etc, Sander Kovacs's answer has convinced me that this is a subtler business than I'd first thought. I see from the comments on that answer that you're hard at work digesting it. – Steven Landsburg Dec 27 '15 at 22:46