Return to Answer

For reasons that will become clearer below, my expectation is that the answer is negative in positive characteristic, due to the failure of nontrivial connected semisimple groups (unlike tori) to have completely reducible representation theory in such characteristics. (EDIT: McNinch has now given such examples with $H = {\rm{SL}}_m$ over any algebraically closed field of characteristic $p > 0$, for suitable $m$ depending on $p$.) No doubt in char. 0 the argument via Cartan involutions by Aakumadula is simpler than what is below in char. 0 (though the proof of Mostow's theorem upon which Aakumadula's argument rests is not easy).

Observe that finding a counterexample to the original problem in positive characteristic amounts to showing that the above Theorem has optimal hypotheses; e.g., it fails for some connected semisimple $H$ (and a suitable $G$). This is a very natural question in positive characteristic, and so (as I hope Jim Humphreys will agree) adequately motivates the original problem.

I expect that counterexamples should exist in every positive characteristic (so the Theorem above is essentially optimal). (EDIT: McNinch's answer now confirms this expectation.)

For reasons that will become clearer below, my expectation is that the answer is negative in positive characteristic, due to the failure of nontrivial connected semisimple groups (unlike tori) to have completely reducible representation theory in such characteristics. (EDIT: McNinch has now given such examples over any algebraically closed field of characteristic $p > 0$: the adjoint semisimple subgroup $H = {\rm{PGL}}_n$ inside ${\rm{SL}}({\mathfrak{gl}}_n)$ embedded via "conjugation" for any $n$ divisible by $p$.) No doubt in char. 0 the argument via Cartan involutions by Aakumadula is simpler than what is below in char. 0 (though the proof of Mostow's theorem upon which Aakumadula's argument rests is not easy).

Observe that finding a counterexample to the original problem in positive characteristic amounts to showing that the above Theorem has optimal hypotheses; e.g., it fails for some connected semisimple $H$ (and a suitable $G$). This is a very natural question in positive characteristic, and so (as I hope Jim Humphreys will agree) adequately motivates the original problem.

For reasons that will become clearer below, my expectation is that the answer is negative in positive characteristic, due to the failure of nontrivial connected semisimple groups (unlike tori) to have completely reducible representation theory in such characteristics. No doubt in char. 0 the argument via Cartan involutions by Aakumadula is simpler than what is below in char. 0 (though the proof of Mostow's theorem upon which Aakumadula's argument rests is not easy).

I expect that counterexamples should exist in every positive characteristic (so the Theorem above is essentially optimal). For example, suppose some connected semisimple $H$ has an irreducible (finite-dimensional) linear representation $V$ such that there is a non-split $H$-equivariant linear extension $W$ of $V$ by $V$, and let $G = {\rm{GL}}(W)$ equipped with its natural $H$-action. (EDIT: see the bottom; no such examples exist.) For the group $G^H$ of $H$-equivariant linear automorphisms of $W$, every $g \in G^H(k)$ must preserve the extension structure and induce the same nonzero scaling effect on the steps of the filtration. Thus, the resulting nontrivial character $(G^H)_{\rm{red}} \rightarrow {\rm{GL}}_1$ has kernel identified with ${\mathbf{G}}_{\rm{a}}$, so $(G^H)^0_{\rm{red}}$ is not reductive. (Hence, for $G' = G \rtimes H$ the normalizer $N_{G'}(H)^0_{\rm{red}}$ is not reductive.)

So to give a counterexample in positive characteristic it is enough to find such a pair $(H, V)$, preferably in every positive characteristic. Maybe George McNinch has some ideas (perhaps using results in Jantzen's 1991 article "First cohomology groups for classical Lie algebras", since we want ${\rm{H}}^1(H,V \otimes V^{\ast}) \ne 0$)?

[EDIT: The above idea to make counterexamples doesn't work. It came to my attention from an expert that no such non-split extensions exist, due to results in positive characteristic in the book of Jantzen (see II, 2.14). Nonetheless, my guess is that there are counterexamples in positive characteristic.]

For reasons that will become clearer below, my expectation is that the answer is negative in positive characteristic, due to the failure of nontrivial connected semisimple groups (unlike tori) to have completely reducible representation theory in such characteristics. (EDIT: McNinch has now given such examples with $H = {\rm{SL}}_m$ over any algebraically closed field of characteristic $p > 0$, for suitable $m$ depending on $p$.) No doubt in char. 0 the argument via Cartan involutions by Aakumadula is simpler than what is below in char. 0 (though the proof of Mostow's theorem upon which Aakumadula's argument rests is not easy).

I expect that counterexamples should exist in every positive characteristic (so the Theorem above is essentially optimal). (EDIT: McNinch's answer now confirms this expectation.)

I expect that counterexamples should exist in every positive characteristic (so the Theorem above is essentially optimal). For example, suppose some connected semisimple $H$ has an irreducible (finite-dimensional) linear representation $V$ such that there is a non-split $H$-equivariant linear extension $W$ of $V$ by $V$, and let $G = {\rm{GL}}(W)$ equipped with its natural $H$-action. For the group $G^H$ of $H$-equivariant linear automorphisms of $W$, every $g \in G^H(k)$ must preserve the extension structure and induce the same nonzero scaling effect on the steps of the filtration. Thus, the resulting nontrivial character $(G^H)_{\rm{red}} \rightarrow {\rm{GL}}_1$ has kernel identified with ${\mathbf{G}}_{\rm{a}}$, so $(G^H)^0_{\rm{red}}$ is not reductive. (Hence, for $G' = G \rtimes H$ the normalizer $N_{G'}(H)^0_{\rm{red}}$ is not reductive.)

So to give a counterexample in positive characteristic it is enough to find such a pair $(H, V)$, preferably in every positive characteristic. Maybe George McNinch has some ideas (perhaps using results in Jantzen's 1991 article "First cohomology groups for classical Lie algebras", since we want ${\rm{H}}^1(H,V \otimes V^{\ast}) \ne 0$)?

I expect that counterexamples should exist in every positive characteristic (so the Theorem above is essentially optimal). For example, suppose some connected semisimple $H$ has an irreducible (finite-dimensional) linear representation $V$ such that there is a non-split $H$-equivariant linear extension $W$ of $V$ by $V$, and let $G = {\rm{GL}}(W)$ equipped with its natural $H$-action. (EDIT: see the bottom; no such examples exist.) For the group $G^H$ of $H$-equivariant linear automorphisms of $W$, every $g \in G^H(k)$ must preserve the extension structure and induce the same nonzero scaling effect on the steps of the filtration. Thus, the resulting nontrivial character $(G^H)_{\rm{red}} \rightarrow {\rm{GL}}_1$ has kernel identified with ${\mathbf{G}}_{\rm{a}}$, so $(G^H)^0_{\rm{red}}$ is not reductive. (Hence, for $G' = G \rtimes H$ the normalizer $N_{G'}(H)^0_{\rm{red}}$ is not reductive.)

So to give a counterexample in positive characteristic it is enough to find such a pair $(H, V)$, preferably in every positive characteristic. Maybe George McNinch has some ideas (perhaps using results in Jantzen's 1991 article "First cohomology groups for classical Lie algebras", since we want ${\rm{H}}^1(H,V \otimes V^{\ast}) \ne 0$)?

[EDIT: The above idea to make counterexamples doesn't work. It came to my attention from an expert that no such non-split extensions exist, due to results in positive characteristic in the book of Jantzen (see II, 2.14). Nonetheless, my guess is that there are counterexamples in positive characteristic.]