Let $k$ be an algebraic closed charateristic zero field. Let $G$ be a reductive group over $k$ and let $X$ be a scheme (not necessarily separated) of finite type over $k$. Let $L$ be a line bundle over $X$. Let $G$ act on $X$ and linearise $L$, denoted by $G\curvearrowright(X,L)$.
In Mumford's Geometric invariant theory and Dogachev's Lectures on invariant theory (chapter 8), semistability is defined for geometric points, and they form an open subset $X^{\mathrm{ss}}$. The same is true for stable points $X^{\mathrm{s}}$. Unstable points are defined as the complement of semistable points, $X^{\mathrm{us}}:=X\setminus X^{\mathrm{ss}}$. I wonder why these notions are defined pointwise in stead of define “(semi)stable open subscheme” and “unstable subscheme”.
One reason for $X^{\mathrm{(s)s}}$ I guess is because open subschemes behave nicely. If we write $X_0\subset X$ for the subspace of closed points of $X$, then since $X$ is Jacobson, the set of open subsets of $X$ and the set of open subsets of $X_0$ are in bijection \begin{align*} \{\textrm{open subsets of } X\}&{}\overset{\textrm{1-1}}{\longleftrightarrow}\{\textrm{open subsets of }X_0\}\\O&{}\mapsto O\cap X_0. \end{align*} Also, open subsets of $X$ have natural open subscheme structures. So defining the “semistable open subscheme” of $X$ is equivalent to defining semistable closed points of $X$.
My question arises when I want to consider the unstable locus $X^{\mathrm{us}}$, and the strictly semistable locus $X^{\mathrm{sss}}:=X^{\mathrm{ss}}\setminus X^{\mathrm{s}}$. They are now defined as topological subspaces, $X^{\mathrm{us}}\subset X$ is closed and $X^{\mathrm{sss}}\subset X$ is locally closed. I think it is natural to consider subscheme structures on $X^{\mathrm{us}}$ and $X^{\mathrm{sss}}$.
There is research about $X^{\mathrm{us}}$ and $X^{\mathrm{sss}}$ when $X$ is a smooth variety. For example the instability decomposition (Kempf), characterizing how unstable a point is, and the partial desingulariastion (Kirwan), a $G$-equivariant blowing up $\tilde X\to X$ with a linearisation $\tilde L$ on $\tilde X$ such that $\tilde X^{\mathrm{ss}}=\tilde X^{\mathrm{s}}$. To generalise their results to appropriate schemes (e.g. non-reduced varieties), I think it is necessarily to consider subschemes on $X^{\mathrm{us}}$ and $X^{\mathrm{sss}}$.
Kempf, George R., Instability in invariant theory, Ann. Math. (2) 108, 299-316 (1978). ZBL0406.14031.
Kirwan, Frances Clare, Partial desingularisations of quotients of nonsingular varieties and their Betti numbers, Ann. Math. (2) 122, 41-85 (1985). ZBL0592.14011.
Mumford, D.; Fogarty, J.; Kirwan, F., Geometric invariant theory., Ergebnisse der Mathematik und ihrer Grenzgebiete. 2. Folge. 34. Berlin: Springer-Verlag. 320 p. (1994). ZBL0797.14004.
Dolgachev, Igor, Lectures on invariant theory, London Mathematical Society Lecture Note Series. 296. Cambridge: Cambridge University Press. xvi, 220 p. (2003). ZBL1023.13006.