Unipotent algebraic group action on quasi-affine (vs. affine) variety?

Question

This question arises from a comment by user nfdc23 on an unrelated recent MO question here. It concerns textbook treatments of what has been called the "Theorem of Kostant-Rosenlicht", stated as Theorem 2 in this 1961 paper by Rosenlicht. (Apparently Kostant had addressed a similar question earlier involving Lie algebras in characteristic 0, though this may be unpublished.) Rosenlicht was mainly concerned with quotient varieties related to the action of a unipotent algebraic group $G$ on a quasi-affine variety $V$ (an open set in an affine variety). By a somewhat complicated induction on the dimension of $G$ (which can just as well be assumed connected), he proved in Theorem 2 that the $G$-orbits in $V$ are all closed. As an immediate corollary, the conjugacy classes in $G$ itself are closed.

Each of the three textbooks called Linear Algebraic Groups includes some version of this theorem: Borel's 1969 Benjamin lecture notes (expanded to a second edition in the Springer GTM series in 1991) stated Rosenlicht's theorem as Proposition 4.10 but with a proof which works only when $V$ is affine. My own GTM book in 1975 included just an exercise (17.8) on the affine case, with an outline of the proof. Then Springer in 1981/1998 treated the affine case briefly in his Proposition 2.4.14.

Is there a natural example of a quasi-affine but not affine variety $V$ to which Rosenlicht's theorem applies in an essential way?

I've found it difficult to place the theorem in the study of algebraic groups and their actions. For example, it doesn't seem to play any role in the basic structure theory of reductive groups.

ADDED: The answers and comments by nsfdc23, YCor, and Friedrich Knop have been very helpful in clarifying the issues here. The implicit answer to my question seems to be that there are no examples of the sort I asked about. Indeed, there seems to be little distance between the affine and quasi-affine formulations. This is less obvious in Rosenlicht's paper, especially because the language he used isn't as flexible as the language of schemes.

Borel apparently saw how to simplify Rosenlicht's approach by invoking the action of $G$ on regular functions, where any finite dimensional $G$-invariant subspace has a fixed point. But he skipped over the shift from an affine to a quasi-affine variety. Perhaps this was due to impatience and also having too many other projects in mind. (I recall the kind of time pressure he faced in 1968 in splicing together two sets of lecture notes on arithmetic groups which differed a lot in notation. I found many minor corrections to be made but didn't share all of them in a timely way; so the published version still has uncorrected pages.)

I don't remember now why I stated my exercise just for affine varieties, since I was usually following Borel's lecture notes. Unfortunately it's too late to ask Borel or Springer how they arrived at their choices. Springer might for example have understood the extra argument needed to deal with the quasi-affine case and decided to avoid it.

Answer 1

Since the quasi-affine case is so easily reduced to the affine case, one doesn't really get much extra mileage out of it.

After checking my papers I am pretty sure that I never used the quasi-affine case seriously. To the contrary, in some cases I had to reduce to the affine case anyway because one is actually losing information by passing to the quasi-affine case.

For example, for affine varieties it is clear that any two closed orbits can be separated by a global function $f$ (i.e., $f$ is constant $0$ on one orbit and constant $1$ on the other). So, this also holds for unipotent groups acting on quasi-affine varieties. But this cannot be deduced just from all orbits being closed. Consider, e.g., $\mathbf G_m$ acting on $X=\mathbf A^2\setminus\{0\}$ by scalar multiplication. Then all orbits are closed but no two orbits can be separated. The reason is of course that in $\mathbf A^2$ none of the orbits stay closed.

So why bother stating theorems in the quasi-affine case, at all? I think, this is mostly to alert that one is dealing with a property which is passed on to open subsets. So one is allowed to remove all kinds of undesired points from your variety like singularities, fixed points etc.

Answer 2

Let $X$ be quasi-affine. Then $X$ canonically embeds in the affine variety $\hat{X}=\mathrm{Spec}(k[X])$. ($*$)

In particular, any algebraic group action on $X$ canonically extends to $\hat{X}$.

If the group is unipotent, then orbits are closed in $\hat{X}$ (by the affine version of Rosenlicht's theorem), and hence orbits inside $X$ are closed in $X$ (I'm just using here injectivity of $X\to\hat{X}$ at the level of closed points, which holds because regular functions separate closed points of $X$).

So there's no serious difference between the two versions of the theorem, and hence it's hard to say anywhere that we "seriously" use the quasi-affine version. Still it's natural and convenient to state it in this generality, since there are many natural instances of non-affine quasi-affine varieties occurring in the context of algebraic group actions, such as many homogeneous spaces (the simplest one being $\mathrm{SL}_2$ modulo the upper unipotent subgroup).

---

As mentioned by nfdc23, there is an issue, namely that $k[X]$ can be infinitely generated. I think this can be fixed as follows: let $G$ be an algebraic group acting on $X$. Unless I miss something, $k[X]$ is an increasing union of finite-dimensional sub-$G$-modules. Passing to the $k$-algebras they generate, we see that $k[X]$ is increasing union of finitely generated $k$-subalgebras $A_n$. So we have genuine affine varieties $Y_n=\mathrm{Spec}(A_n)$ with canonical $G$-equivariant morphisms $X\to Y_n$.

Since $X$ is quasi-affine, there exist a finite number of regular functions separating the points, so $X\to Y_n$ is injective for some $n$ and the argument goes through.