In his paper "Sur la mauvaise reduction des courbes de Shimura" from '86 H. Carayol shows the following congruence relation:

Let $M$ be the tower of Shimura curves over a totally real $F$, associated to a quaternion algebra over $F$ that is split at one infinite place.

Fix a prime $\mathfrak{p}$ and consider the base change of $M$ to $F_\mathfrak{p}$. It carries a $p$-divisible group $E_\infty$ and using this, he constructs integral models $\mathbb{M}$ and $\mathbb{E}_\infty$ over $\mathcal{O}_\mathfrak{p}$.

The congruence relation (§ 10.3) is an assertion on the geometric special fiber of this model: It carries actions by $\rm GL_2(F_\mathfrak{p})$ and the unramified quotient of the Weil group $W(F_\mathfrak{p}^{\rm ab}/F_\mathfrak{p})$. I think these actions commute, since all geometric objects occuring, including all morphisms are actually defined over $\mathcal{O}_\mathfrak{p}$.

Each point $A$ in $\mathbb{P}^1(F_\mathfrak{p})$ determines a closed subscheme and the matrices $X:=(\begin{smallmatrix} u & * \\ 0 & 1\end{smallmatrix})$ stabilize the subscheme corresponding to $[1,0]$.

The **congruence relation** is: Each point in this subscheme is stabilized by $(X,\rm{Frob}^{val(u)})$ and this induces on $\mathbb{E}_\infty|_x$ the $val(u)$-power of absolute Frobenius.

Now to the application and my question: In § 5.2 of his paper "Sur les representations $\ell$-adic associées aux formes modulaires de Hilbert" (also from '86) Carayol applies the congruence relation to show that local-global compatibility holds in the Langlands correspondence for Shimura curves (and hence Hilbert modular forms of appropriate type).

He constructs $\lambda$-adic sheafs on $M_{et}$ and $\mathbb{M}_{et}$ much like $E_\infty$ and $\mathbb{E}_\infty$ (but instead of the standard representation, some weight enters).

The etale cohomology of the geometric special fiber at some prime $\mathfrak{p}$ then again carries (by functoriality of cohomology) an $\rm GL_2(F_\mathfrak{p}) \times W(F_\mathfrak{p}^{\rm ab}/F_\mathfrak{p})$-action.

Now to my **question**:
Carayol mentions that by the previous result the diagonal action (as above and suitably restricted to the $[1,0]$-part of cohomology)
on cohomology **must be trivial**. Why is this?

You might say this is easy: that absolute Frobenius acts trivially on cohomology. Yes, I understand the result for the etale cohomology of $\mathbb{E}_\infty$, or even nontrivial-weight-twists of it.

But the *crucial point* in the first paper is that the coefficients are a $\mathfrak{p}$-adic sheaf, while in the second paper $\lambda$ and $\mathfrak{p}$ very much are of different residue characteristic (T. Saito did the p=l case 15 years later).

I hope I could make the point of my confusion clear and apologize if this is a notational triviality.