Consider a cartesian diagram

$$\require{AMScd} \begin{CD} X' @>{f'}>> X\\ @V{p'} VV @VV{p} V\\ S' @>{f}>> S \end{CD}$$

of schemes (or even locally ringed spaces). If $\mathcal{F}$ is any $\mathcal{O}_X$-module, then there is a base change morphism $$ f^* R^i p_* \mathcal F \to R^i(p')_* (f')^* \mathcal F. $$ The flat base change theorem says that if all objects are schemes, $f$ is flat, and $\mathcal F$ is quasicoherent, then base change is an isomorphism. Is there a nice counterexample if $\mathcal F$ is not quasicoherent?