Let $k$ be an algebraically closed field and let $G$ be a reductive linear algebraic group over $k$ with Lie algebra $\mathfrak g$. If the characteristic of $k$ is $0$ then, by a classical result of Kostant, $k[\mathfrak g]$ is free over the subalgebra $k[\mathfrak g]^G$. Is there an analog of this result when the characteristic of $k$ is positive?

Roger Richardson amplified Kostant's result in characteristic 0, which in turn led Steve Donkin to work out a closely parallel version in prime characteristic: *On conjugating representations and adjoint representations of semisimple groups*, Invent. Math. 91 (1988), no. 1, 137–145. (This is available online at the GDZ archive here. See in particular Donkin's Theorem 2.2, which ends with the freeness result you want (under his conditions). All serious results of this sort unfortunately require some mild restrictions on the prime involved relative to the root system. Note too that both the algebraic group and its Lie algebra lead to precise statements.

Kostant's theorem itself goes back to his paper in *Amer. J. Math.* 85 (1963), available at JSTOR. Note that the first volume of his collected papers includes an extended up-to-date commentary on that paper in notes at the end. There is also an interesting account in lectures by Tony Joseph at the 1997 U. Montreal conference (published proceedings).

In a complementary direction there is a long tradition of studying the structure of the algebra of invariants in the polynomial algebra (and its relation to the center of the universal enveloping algebra), first in characteristic 0 (Chevalley, Bourbaki) and later in prime characteristic (Veldkamp, Kac-Weisfeiler, Mirkovic-Rumynin, etc.). As in Donkin's work, there are always complications for some primes and some Lie types. A lot has been written down, though perhaps not the absolutely last word.

ADDED: As Sasha Premet points out (I think correctly), there are real problems when Donkin's hypotheses (which I didn't quote in full) are not satisfied. I'm not sure how close Sasha's example gets to providing both necessary and sufficient conditions for freeness, but simple algebraic groups of Lie type $A$ which fail to be simply connected over a field of characteristic which is not very good definitely cause the most trouble. Donkin is getting freeness in prime characteristic as part of a more general argument about module filtrations which substitute for complete reducibility in characteristic 0 as used by Richardson. But it would be useful to state separately a best possible analogue of Kostant's original theorem.

Here is an example of $G$ for which $k[\mathfrak{g}]$ is not a free $A$-module, where $A=k[\mathfrak{g}]^G$ (I hope there is no mistake in what follows).

Let $G=\mathrm{PGL}_p$ where $p\ge 5$, a simple algebraic group of adjoint type with $\mathfrak{g}=\mathfrak{pgl}_p$. The Chevalley Restriction Theorem holds for $G$ as follows, e.g., from the Springer-Steinberg paper (see Ch. II, 3.17). Then we have that $A\cong k[\mathfrak{t}]^W$ where $\mathfrak{t}$ is the image of the diagonal matrices in $\mathfrak g$. Of course, $W\cong\mathfrak{S}_p$ and $\mathfrak{t}$ is the quotient of the natural $\mathfrak{S}_p$-module $V$ with basis $\{e_1,\ldots, e_p\}$, permuted by $\mathfrak{S}_p$, by its trivial submodule $k(e_1+\cdots+e_p)$. By a result of Gregor Kemper, for $p\ge 5$ the ring $S(V^*)^{\mathfrak{S}_p}\cong k[\mathfrak{t}]^W$ is not Cohen-Macaulay (see Corollary 2.8 and Example 2.9 in Kemper's paper published in J. Algebra, Vol. 215 (1999), 33--351). In particular, $k[\mathfrak{g}]^W$ is not a polynomial algebra, which resolves in the negative Problem 3.18 in the Springer-Steinberg paper.

Since $\mathfrak g$ is a restricted Lie algebra and $\mathfrak t$ is its toral Cartan subalgebra of dimension $p-1$, it follows from a result I proved that there exist homogeneous $f_0,\ldots, f_{p-2}\in A$ with ${\rm deg}\ f_i=p^{p-1}-p^i$ such that $x^{[p]^{p-1}}=\sum_{i=0}^{p-2}f_i(x)x^{[p]^i}$ for all $x\in\mathfrak{g}$. Furthermore, the zero locus of the $f_i$'s equals the nullcone $\mathcal N$ of $\mathfrak g$. Since $\mathcal N$ is irreducible of codimension $p-1$ it follows that {$f_0,\ldots, f_{p-2}$} a regular sequence in $k[\mathfrak{g}]$ and $k[\mathfrak{g}]$ is a free module over $A_0:= k[f_0,\ldots, f_{p-2}]$. Since $A\cong k[\mathfrak{t}]^W$ has Krull dimension $p-1$, the $f_i$'s form a homogeneous sytem of parameters for $A$. If $k[\mathfrak{g}]$ is free over $A$ then $A$ is finitely generated and projective over $A_0$. But then $A$ is free over $A_0$ implying that $A$ is Cohen-Macaulay. This contradiction shows that Kostant's freeness theorem fails in our case.

Math. Z.paper on centers of reduced enveloping algebras. $\endgroup$ – Jim Humphreys Aug 17 '12 at 22:21