In several places (for example, Chriss & Ginzburg’s book “Representation Theory and Complex Geometry”), the author says that the set $X$ of Borel subalgebras of a semisimple Lie algebra $\mathfrak g$ forms a Zariskiclosed subset of a Grassmannian on this semisimple Lie algebra. A Borel subalgebra is a maximal solvable subalgebra, by general theorems, all of them have the same dimension, say $d$. Then the set $X$ can be identified with a subset of the Grassmannian $\mathrm{Gr}(d,\mathfrak g)$. My question is: why is $X$ a Zariskiclosed subset? How to translate the condition “solvable” to an algebraic condition? I thought of Lie’s criterion on solvability, but it is not an equivalence condition, so it did not work.

1$\begingroup$ I think this is addressed in Fulton and Harris. $\endgroup$– Ben McKaySep 4, 2021 at 17:02

1$\begingroup$ @BenMcKay Since you seem to know the answer, perhaps you can spell it out as an answer? $\endgroup$– Theo JohnsonFreydSep 4, 2021 at 21:52

3$\begingroup$ This is completely standard. It's definitely dealt with in Humphreys and Borel, and probably in Springer's book too. (All these books are called Linear Algebraic Groups.) $\endgroup$– Paul LevySep 5, 2021 at 10:10

1$\begingroup$ The question specifically asks how to show that the set of Borel subalgebras of a Lie algebra is a closed subvariety of the appropriate Grassmannian (of all subspaces of a given dimension). All of the references mentioned in the comments show that G/B is a projective variety, and then show that all Borel subgroups/algebras are conjugate, along with a normalizer theorem to identify the set of Borels with G/B. To me, this route definitively fails to answer the question (which is not directly about showing G/B is projective, or even about G/B at all without some nontrivial theorems). $\endgroup$– Grant B.Sep 9, 2021 at 22:39

2$\begingroup$ @Grant B.  Projective varieties are complete; the image of the map $G/B\rightarrow {\rm Gr}(d, {\mathfrak g})$ is therefore closed. Doesn't that answer the question? $\endgroup$– Paul LevySep 10, 2021 at 15:22
2 Answers
A nice source on this is Procesi's "Lie Groups: an approach through invariants and representations". There he defines parabolic subgroups as precisely those for which $G/P$ is a projective variety and shows that this is equivalent to $P$ containing a Borel subgroup.
In a later section (called quadratic relations, I think) he shows that you can cut them out with only quadratic polynomials in an appropriate projectivised representation of $G$ (a result originally due to Kostant I believe). This is, in effect, a generalisation of the Plucker embedding for the Grassmannian.
Note, you say generalised flag variety in the title which refers to any quotient $G/P$ of semisimple $G$ by a parabolic subgroup but in the body of the question only refer to the quotient by a Borel $G/B$ which I would call a "full" flag variety.
Edit: said Borel when I meant Kostant
I think, the statement "the set of Borel subalgebras of a semisimple Lie algebra $\mathfrak g$ forms a Zariskiclosed subset of a Grassmannian" can be understood in more than one way and then the question becomes less trivial than one thinks. Everything is over $\mathbb C$, by the way, otherwise things become even more complicated.
First interpretation: This is the one presumably intentend by ChrissGinzburg and others. It consists of two statements:
 The set of Borel subgroups is in bijection with $G/B$.
 The natural morphism $G/B\to Gr_d(\mathfrak g)$ is a closed embedding.
The first assertion uses the conjugacy of Borel subgroups and that Borel subgroups are selfnormalizing. The second uses that $G/B$ is projective and that $B$ is the normalizer of $\mathfrak b$ in $G$.
Second interpretation: The statement could also mean that the moduli space of Borel subalgebras is precisely the schematic image of $G/B$. This is more subtle and is not addressed in any of the standard text, as far as I know. One can see this as follows:
Step: A subalgebra $\mathfrak b\subseteq\mathfrak g$ is the Lie algebra of a Borel subgroup if and only if $\mathfrak b$ is solvable of maximal possible dimension $d=\dim B$. Here one has to prove mainly that a maximal dimensional solvable subalgebra is algebraic in the sense that it is the Lie algebra of some algebraic subgroup.
Step: There is a closed subscheme $X\subseteq Gr_d(\mathfrak g)$ which is a moduli scheme of $d$dimensional solvable subalgebras. For this one has to translate "solvable" into an algebraic condition. This condition comes from that fact that the degree of solvability of any $\mathfrak b$ is bounded by say $\ell$. Then $X$ is defined by the condition that all "multicommutators" $[\ldots[[\xi_1,\xi_2],[\xi_3,\xi_4]],\ldots]$ vanish for all $\xi_1,\ldots\xi_{2^\ell}\in\mathfrak b$. This is easily seen to be a closed condition.
Step: Step 1 implies that $X$ is set theoretically the image of $G/B$ in $Gr_d(\mathfrak g)$.
Step: It remains to prove that $X$ is smooth, hence reduced and normal. For this we compute the tangent space $Z$ of $X$ in $\mathfrak b$. First the tangent space of $Gr_d(\mathfrak g)$ in $\mathfrak b$ is $H:=\mathrm{Hom}(\mathfrak b,\mathfrak g/\mathfrak b)$. It turns out that $Z$ is the space of $1$cocycles (=derivations) in $H$. Moreover, the tangent space of $G/B$ in $\mathfrak b$ is the space of all $1$coboundaries (=inner derivations) $C\cong\mathfrak g/\mathfrak b$. So $Z/C=H^1(\mathfrak b,\mathfrak g/\mathfrak b)$. The latter is easily seen to be $0$ (take a filtration with $1$dimensional quotients and use that $\mathfrak b$ and $\mathfrak g/\mathfrak b$ have no weights in common). Thus $Z=C$ and therefore $\dim Z=\dim X$ and we are done.

$\begingroup$ Thank you a lot for your answer! The second interpretation is exactly what I expected. $\endgroup$– yzchenJan 15, 2022 at 23:35