[[ I have added a discussion of when $p$ is smooth or has quotient singularities. ]]
[[ I added a discussion on the cohomology of $[X/G]$. ]]

The étale case follows in a way that is altogether analogous to the topological
case. Let me give a proof that gives a teeny bit of extra information. I assume
that $\alpha_i$ is homogenous (with respect to cohomological degree) of degree
$d_i$. Then $\alpha_i$ gives a map in the derived category $A[-d_i]\to
Rp_\ast A$ and combining them a map $\bigoplus_iA[-d_i]\to Rp_\ast A$. If we can
show that this map is an isomorphism then we get an isomorphism
$\bigoplus_iR\Gamma(B,A)[-d_i]\to R\Gamma(E,A)$ which on taking
cohomology gives the L-H theorem. That this is an isomorphism can be checked
fibrewise and if the natural map $(R^ip_\ast A)_x\to H^i(F_x,A)$ is an isomorphism for
all geometric points $x$ we are through.

This condition is true under one of the following conditions:

- $p$ is proper (by the proper base change theorem).
- $p$ is locally trivial by the Künneth formula.
- The second case covers the case of a $G$-torsor. If $G$ acts on $X$ with
finite stabiliser scheme (a condition slightly stronger than having finite
stabilisers but which guarantees that $X/G$ exists) and the orders of the
stabilisers are invertible in the ring of coefficients it is still true. This
can be seen by looking at the stack quotient $X\to[X/G]$ which is a
$G$-torsor (though with base a stack) and at $[X/G]\to X/G$ which induces an
isomorphism in cohomology fulfilling the condition. It should also be possible
to do directly imitating Deligne's proof (in SGA 4 1/2 I think) that the
cohomology of $G$ is indendent of the characteristic (it use the sequence of
fibrations $G\to G/U\to G/B$ where $B$ is a Borel subgroup and $U$ its unipotent radical).

**Addendum**: Here are, as requested below by algori, some details on the fact that $\pi\colon[X/G]\to X/G$ induces an isomorphism for coefficients $A$ for which the order of the (group of connected components of the) stabilisers are invertible. (This of course is well-known, so well-known in fact that I don't know if there is a proper reference for it.) I will not use that the stack is a global quotient so we may as well consider $\pi\colon\mathcal X\to X$ where $\mathcal X$ is a stack with finite stabiliser scheme and $X$ is its spatial quotient. For simplicity I will assume that the automorphism groups are reduced (i.e., that $\mathcal X$ is a Deligne-Mumford stack). The general case can be proved along the same lines but would be longer and more technical. What we are going to show is that $R\pi_*A=A$. As the construction of the spatial quotient commutes with étale localisation on $X$ we may assume that $X$ is local strictly Henselian and then by the local structure theory of DM-stacks (to be found for instance in Laumon-Moret-Bailly) $\mathcal X$ has the form $[Y/G]$, where $G$ is a finite group which can be assumed to be the stabiliser of a point of $Y$ and hence has order invertible in $A$ and $Y$ is also local strictly Henselian. Now using the usual simplicial resolution $T_n=G^n\times Y$ of $[Y/G]$ we get that $H^*([Y/G],A)=H^*(G,A)=A$ as $H^*(T_n,A)=A^G$.
- $p$ is smooth. This is proved as follows: For any $A$-complex $K$ on $B$ we
get a map $\bigoplus_iK[-d_i]\to Rp_\ast p^\ast K$ which we want to show is an
isomorphism. This map is functorial so in particular if is an isomorphism for
two complexes in a distinguished triangle it is so for the third. By Noetherian
induction (assuming for simplicity $B$ is Noetherian) we may assume that $B$ is
local Henselian and that the statement is true for $B$ replaced by the
complement $U$ of the closed point. We start by showing that that implies that
if $K$ is an $A$-complex on $U$ and if $j\colon U\to B$ is the inclusion, then the
result is true for $Rj_\ast K$. Indeed, this follows directly from smooth base
change which implies that $p^\ast Rj_\ast K=Rj'_\ast p^\ast K$, where $j'\colon p^{-1}U\to E$ is the
inclusion, and then the result follows from the induction hypothesis. On the
other hand, if $K$ is supported on the closed point $x$ of $B$, then the map is
$\bigoplus_iK_x[-d_i]\to R\Gamma(E_x,K_x)$ which is an isomorphism as it is for
$K=A$. Now, the mapping cone of $K\to Rj_\ast j^\ast K$ has support at $x$ so the statement
follows.
- The only thing that is used about a smooth map is that $p$ is universally
locally acyclic for the torsion primes of $A$ (SGA IV: Exp. XVI,
Thm. 1.1). Unless I am mistaken this is true for $p$ that locally are of the
form $E\times_GU\to B\times_GU$, where $E\to B$ is a smooth $G$-map, $U$ a $G$-scheme and $|G|$
is invertible in $A$.

Another way of dealing with the $(G,X)$ case which I think should be more
efficient and general is, following Deligne, to split $X \to X/G$ up into $X\to
X\times_GG/U\to X\times_GG/B\to X/G$ (where $U$ is the unipotent radical of a Borel subgroup
$B$). Then $X\times_GG/B\to X/G$ is proper and in fact the Leray-Hirsch argument
applies provided a large enough integer is invertible in the coefficients (it is
in general not enough to invert the $|H|$ but one also needs to invert some
primes intrinsincally defined by $G$) and $X\to X\times_GG/U$ has more or less affine
spaces as fibres and induces an isomorphism if the $|H|$ are invertible. Finally
$X\times_GG/U\to X\times_GG/B$ is essentially a torus bundle and the cohomology of $X\times_GG/U$
can be analysed in terms of the cohomology of $X\times_GG/B$ and the characteristic
classes of the bundle.