homomorphism into reductive groups

Question

Let $k$ be an algebraically closed field with char($k$)$= p > 0$. Let $P$ be a finite $p$-group. For any homomorphism $\rho : P \rightarrow GL(n,k)$ we know that the image $im(\rho)$ can be put inside the group of upper triangular matrices(standard borel) after conjugation. The question I have is what can we say about homomorphisms of $P$ into a general reductive or even semi-simple algebraic groups. (The usual argument for $GL(n,k)$ case can be adapted to $SL(n,k)$ as well. )

Answer 1

This is a (rather) expanded version of Yves' answer, because for the specific focus of the question one can understand the proof without getting into many other technicalities of the Borel--Tits paper (which has bigger fish to fry, related to fields of definition and much more). We will explain the proof of the following result (which is given over perfect fields rather than just algebraically closed fields so that we include finite fields):

${\bf Theorem}$ (Borel--Tits): In a connected linear algebraic group $G$ over a perfect field $k$, every subgroup $H \subset G(k)$ consisting of unipotent elements is contained in a "canonical" connected unipotent smooth closed $k$-subgroup (i.e., one whose formation is compatible with respect to $k$-isomorphisms in the pair $(G,H)$ and perfect extension on $k$).

This will use a beautiful construction in section 2 of the Borel--Tits paper. The following is an exposition of some arguments in that paper, with an eye towards the proof of the preceding Theorem (which thereby answers the question posed); by all means one should look at the original paper too (all I have done is provide some more explanation about various aspects of the argument).

Proof: Due to the canonicity claim applied in the setting of Galois descent relative to $\overline{k}$ over $k$, we may and do assume $k$ is algebraically closed (and the compatibility with respect to any further perfect extension of the ground field will be clear from the construction). Since the Zariski closure of $H$ is (smooth and) unipotent (but perhaps disconnected), we can replace $H$ with its Zariski closure and instead work with unipotent smooth closed subgroups $U$ in $G$ (e.g., finite constant ones when in positive characteristic). We induct on the dimension of $G$ (dimension 0 being trivial). We can also replace $U$ with $U \cdot R_u(G)$ so that $U$ contains $R_u(G)$, and if $R_u(G) \ne 1$ then we can pass to $G/R_u(G)$ to drop the dimension and conclude, so we can assume that $G$ is reductive. We can assume $U \ne 1$, so by nilpotence of unipotent algebraic groups there is a nontrivial central element $u$ in $U(k)$.

We will show that $u$ lies in the unipotent radical of a parabolic subgroup $P$ that is "canonical" in terms of $u$ (i.e., one stable under all automorphisms of $G$ that preserve $u$, or even just preserve the cyclic subgroup generated by $u$). Once this is proved, such a $P$ is stable under $U$-conjugation on $G$ (as that leaves the central $u$ in $U(k)$ invariant), so by the self-normalizing property of parabolic subgroups it would follow that $U \subset P$. But $P \ne G$ (since $R_u(P)$ contains $u \ne 1$ whereas we arranged that $G$ is reductive), so we can use dimension induction to conclude that $U$ lies in a connected unipotent subgroup of $P$, but without canonicity in $(G,U)$. However, we gain the property that $U$ is "embeddable" (in the terminology of Borel--Tits), meaning that it does occur inside some connected unipotent subgroup (equivalently, in the unipotent radical of a Borel subgroup).

Observe that $u$ lies in the unipotent radical of some parabolic subgroup, or equivalently in the unipotent radical of some Borel subgroup: this is immediate from Grothendieck's important theorem that $G(k)$ is covered by the subgroups $B(k)$ for Borel subgroups $B$ of $G$ (a result that is proved near the end of section 14 of Borel's textbook but possibly not stated as an official result there; it can be extracted nonetheless). Thus, the Zariski closure of the cyclic subgroup generated by $u$ is "embeddable". Moreover, we saw above via the dimension induction that once this cyclic subgroup is realized inside the unipotent radical of a "canonically associated" parabolic subgroup of $G$, then $U$ itself is "embeddable". Hence, by two applications of the following result (which is to be regarded as a finer version of Grothendieck's theorem, but whose applicability in our inductive strategy ultimately rests on Grothendieck's theorem) we would be done:

${\bf Refined}$ ${\bf Theorem}$ (Borel--Tits). Any embeddable unipotent smooth closed subgroup $U$ of a connected linear algebraic group $G$ over a perfect field $k$ is contained in the unipotent radical of a "canonically associated" parabolic $k$-subgroup $P$. (As before, "canonical" means that the formation of $P$ is compatible with respect to $k$-isomorphisms in the pair $(G,U)$ and perfect extension on $k$.)

[Note that this result is interesting even when $U$ is connected, and it is interesting in characteristic 0. Also, the embeddable hypothesis is automatic once this is all over, so we see that the unipotent radicals of the minimal parabolic $k$-subgroups are precisely the maximal unipotent smooth closed $k$-subgroups; in particular, if $G$ has no proper parabolic $k$-subgroups then $G(k)$ contains no nontrivial unipotent smooth closed $k$-subgroups. The proof gives a "canonical" $P$, but it is not uniquely determined by the stated conditions in general, as we see by considering $G = \prod G_i$ for several pairwise non-isomorphic $k$-simple semisimple $G_i$ with $U \subset G_1$. I don't know offhand whether one can do better when $G$ is $k$-simple or absolutely simple.]

Proof: By Galois descent from $\overline{k}$ we may assume $k$ is algebraically closed (and it will be clear from the construction that we have compatibility with any further perfect extension of the ground field). To construct $P$, we will work with the normalizer $N(U)$, which might be non-unipotent and even more disconnected. Consider the unipotent radical of $N(U)$ (i.e., maximal unipotent connected normal subgroup) anyway. Note that $R_u(N(U))$ trivially contains $U^0$, but it isn't clear if it contains $U$ or not. So following Borel--Tits (see 2.4 in their paper), let's put $U$ back in: define the closed subgroup $$L(U) = U \cdot R_u(N(U))$$ that is unipotent and does contain $U$. By the "canonicity" of $L(U)$ in terms of $U$, any automorphism of $G$ that carries $U$ back to itself must do the same for $L(U)$. Also, just as we assumed that $U$ is embeddable (i.e., lies in the unipotent radical of a Borel), we claim the same holds for $L(U)$. This amounts to checking that $L(U)$ has a fixed point on the projective variety of Borel subgroups of $G$. By hypothesis $U$ has a fixed point, so the closed (!) locus of Borels containing $U$ is a non-empty projective variety (perhaps disconnected). Certainly $N(U)$ acts on that locus, so the connected solvable $R_u(N(U))$ does too. By the Borel fixed point theorem, $R_u(N(U))$ has a fixed point in that locus, so we have found a Borel subgroup of $G$ that contains $L(U)$. Hence, we lose nothing by replacing $U$ with $L(U)$.

We iterate a few times, and eventually reach the situation that the dimension stops growing. That is, we may assume $L(U)$ has the same dimension as $U$. Thus, the connected normal subgroup $R_u(N(U))$ in $L(U)$ must be contained in $U^0$, so $L(U) = U$. In other words, we $R_u(N(U)) \subset U$, so the containment $U^0 \subset R_u(N(U)) = R_u(N(U)^0)$ is an equality. Thus, the normal unipotent subgroup $U$ in $N(U)$ has $R_u(N(U))$ as its identity component. At this stage, we shall prove that $U$ is already connected (but we really need to go further and get the canonical parabolic, or else the plan of the argument would collapse; in fact, we will show that $N(U)$ is parabolic).

By our arranged "embeddable" hypothesis on $U$, we can pick a Borel subgroup $B$ of $G$ that contains $U$, so $L := B \cap N(U)$ is a solvable (possibly disconnected) closed subgroup of $N(U)$ that contains $U$. The connected solvable $L^0$ must lie in a Borel subgroup $B'$ of $N(U)^0$, and if $B''$ denotes the opposite Borel subgroup then $R_u(B' \cap B'') = R_u(N(U)^0) = U^0$. We have $B'' = gB'g^{-1}$ for some $g \in N(U)^0$, so $R_u(L^0 \cap gL^0g^{-1}) \subset U^0$. Equivalently, $R_u(L \cap gLg^{-1}) \subset U^0$. That is, $$R_u(N(U) \cap B \cap gBg^{-1}) \subset U^0$$ for some $g \in N(U)^0$ and some Borel $B$ of $G$ containing $U$. Clearly $U$ is contained in $gBg^{-1}$ too, so $$U \subset B \cap gBg^{-1}.$$ But any intersection of two Borel subgroups is always connected (even contains a maximal torus), and unipotent radicals are functorial for connected solvable groups, so $$U \subset R := R_u(B \cap gBg^{-1}).$$ We now show that this inclusion is an equality (so $U$ is connected).

Suppose to the contrary that it is a strict inclusion. Consideration of the descending central series of the connected unipotent $R$ (whose steps must eventually emerge from $U$, maybe at the first step or maybe later) yields a closed connected subgroup $U'$ in $R$ that normalizes $U$ but is not contained in $U$. This connected unipotent $U'$ then lies inside the identity component of the solvable $N(U) \cap B \cap gBg^{-1}$, so it lies in its unipotent radical, which we have seen is contained in $U^0$. The resulting inclusion $U' \subset U^0$ contradicts that $U'$ is not contained in $U$, so we have shown that $U = R$. Thus, now $U$ is connected, so we conclude that $U = U^0 = R_u(N(U))$.

Now it suffices to prove that $N(U)$ is a parabolic in $G$; i.e., we just have to produce a Borel of $G$ inside $N(U)$, where $U = R_u(B \cap B')$ for Borel subgroups $B$ and $B'$ of $G$. We'll show that $B$ is contained in $N(U)$ (by exploiting that also $U = R_u(N(U))$). Since $U$ contains $R_u(G)$, it is harmless to pass to $G/R_u(G)$ so that $G$ is reductive (if it wasn't already). We will soon make serious use of the links between reductive groups and root systems (and the uniqueness characterizations of root groups).

There is a maximal torus $T$ of $G$ contained in the smooth connected $B \cap B'$, so $T$ normalizes $R_u(B \cap B') = U$. Thus, $N(U)$ contains $T$. Since $R_u(B)$ is generated by the simple positive root groups for $\Phi(B,T)$ (this is a delicate fact in small positive characteristics), it suffices to show that each of those normalizes $U$. The $T$-stable connected subgroup $U \subset R_u(B)$ must be generated by the root groups for the nontrivial $T$-weights that occur on its Lie algebra, and we just have to consider simple positive $a \in \Phi(B,T)$ for which the root group $U_a$ is not contained in $U = R_u(B \cap B')$, so $U_a$ is not contained in $B'$. Hence, $-a \in \Phi(B',T)$ (so $U_{-a} \subset B'$).

Applying to $B$ the "reflection" associated to the simple $a \in \Phi(B,T)$ gives a Borel subgroup $B_a$ containing $T$ and having the same $T$-weights as $B$ except that $a$ is replaced with $-a$. Thus, $R_u(B \cap B_a)$ is directly spanned by the root groups associated to $\Phi(B,T) - (a) = \Phi(B_a,T) - (-a)$, so it is normalized by $U_{-a}$ (as $-a$ is simple for $\Phi(B_a,T)$). We conclude that $U_{-a}$ normalizes $R_u(B \cap B_a) \cap B'$, so it normalizes its unipotent radical, which is directly spanned by the roots in $$(\Phi(B,T) - (a)) \cap \Phi(B',T) = \Phi(B,T) \cap \Phi(B',T) = \Phi(R_u(B \cap B'),T) = \Phi(U,T).$$ In other words, $R_u(B \cap B_a) \cap B'$ has unipotent radical $U$, so $U_{-a}$ normalizes $U$ yet $U_{-a}$ is not contained in $U^0$ (since $-a$ is not even a $T$-weight for $B$). Now recall that $U^0 = R_u(N(U)) = R_u(N(U)^0)$, so the $T$-weight $-a$ occurring on $N(U)^0$ does not arise on the unipotent radical, and hence (by reductivity of the quotient $N(U)^0/R_u(N(U)^0)$ that contains $T$) the $T$-weight $-(-a) = a$ occurs on $N(U)^0$. This forces $U_a \subset N(U)^0$, so $U_a$ normalizes $U$. To summarize, for each simple $a \in \Phi(B,T)$, either $U_a \subset U$ or else $U_a$ normalizes $U$, so either way always $U_a \subset N(U)$. This forces $B \subset N(U)$, so we are done. QED

Answer 2

Corollary 3.7 of the Borel-Tits 1971 Inventiones paper includes: if $K$ is a perfect field, $H$ any connected linear algebraic $K$-group, the maximal unipotent $K$-subgroups of $H$ are the unipotent radicals of minimal parabolic $K$-subgroup of $H$, and those are pairwise conjugate (by elements in $H(K)$). Here "unipotent" does not assume connected (Borel-Tits are very careful with this). In particular, in characteristic $p$, every finite $p$-subgroup of $H(K)$ is contained in the $K$-points of the unipotent radical of a minimal parabolic $K$-subgroup.

Answer 3

Using a faithful reprentation of the reductive group, we can see from the argument for $GL_n$ that the image of $P$ lies inside a unipotent subgroup. Thus it lies inside a maximal unipotent subgroup. So the correct analogue of the group of upper-triangular unipotent matrices is a maximal unipotent subgroup of the reductive group. (Since conversely, every unipotent element has order a power of $p$ and so generates a $p$-group, since this is true in $GL_n$.)

These are all conjguate since they are all the unique maximal unipotent subgroup of a Borel subgroup, all of which are conjugate. So the image of a $P$-group is contained, up to conjugation, in a specific maximal unipotent subgroup.