4 added 1173 characters in body

Moreover, the hypothesis that $G$ is smooth may be removed as follows. First, we deal with the case where $X$ is separated and reduced. In this case, the arguments given by Anton in the question still work : the graphs of the action and of the projection are closed subschemes of $G\times X\times X$ isomorphic to $G\times X$ that coincide on an open dense subset $U$. Since $G\times X$ has no embedded point ($X$ is reduced and $G$ homogeneous), these two graphs need to coincide with the schematic closure of $U$ in $G\times X\times X$ and are thus equal.

Then we deal with the general case ($G$ connected, $X$ reduced) in the following way. Applying the result when the group is smooth connected and the space is reduced to $G^{red}$ acting on $X$ shows that $G^{red}$ acts trivially on $X$. In particular, $G^{red}$ stabilizes every affine open subset of $X$ ; this implies that $G$ also stabilizes every affine open subset of $X$. Applying the result when the group is connected and the space is separated and reduced to $G$ acting on any affine open subset of $X$ shows that $G$ acts trivially on any open subset of $X$, hence on $X$.

3 deleted 474 characters in body

Let me work over an algebraically closed field $k$, let $X$ be a reduced $k$-scheme of finite type and let $G$ be a smooth connected $k$-group scheme acting on $X$, and acting trivially on an open dense subset $U$ of $X$. I will show that $G$ acts trivially on $X$.

Let me first suppose $G$ is smooth.

I will only manipulate closed points. Since both $G$ and $X$ are reduced, it suffices to prove that for any $g\in G$ and $x\in X$, $gx=x$.

Let us fix $x\in X$. We may of course suppose that $x\notin U$. Let $V$ be an affine neighbourhood of $x$. By quasicompactness of $X$, the open subset $GV$ is covered by finitely many of the $gV$, say $GV=\cup_i g_iV$. Since $U$ is dense in $X$, it is possible to find a smooth curve $C$ and a morphism $C\to V$ whose image contains $x$ and intersects $U$. Up to removing points of $C$, we may suppose that $x$ is the only point of the image of $C$ not belonging to $U$.

Now, if $g\in G$, $gx\in g_i V$ for some $i$. Consider $g(C)$ (this is an abuse of notation for the composition of $C\to V$ and of the action of $g$) : its image contained in $g_iV$. It coincides generically with $g_i(C)$, hence, the morphisms $g(C)$ and $g_i(C)$ are equal by separation of $g_iV$ ($g_iV$ is even affine). This shows $gx=g_ix$. We have proved $Gx\subset${$g_1x,\dots, g_rx$}. Since $G$ is connected, $Gx$ is also connected. This shows $Gx=${$x$} and finishes the proofwhen $G$ is smooth.

When $G$ is not necessarily smooth, we may apply the result to $G^{red}$ : the group $G^{red}$ acts trivially on $X$. In particular, $G^{red}$ stabilizes every affine open subset of $X$ ; this implies that $G$ stabilizes every affine open subset of $X$. By the result mentionned by Anton in the question, since any open affine subset of $X$ is separated and reduced, $G$ acts trivially on it.Thus $G$ acts trivially on $X$.

2 added 451 characters in body

Let me work over an algebraically closed field $k$, let $X$ be a reduced $k$-scheme of finite type and let $G$ be a smooth connected $k$-group scheme acting on $X$, and acting trivially on an open dense subset $U$ of $X$. I will show that $G$ acts trivially on $X$.

In what follows,

Let me first suppose $G$ is smooth. I will only manipulate closed points. Since both $G$ and $X$ are reduced, it suffices to prove that for any $g\in G$ and $x\in X$, $gx=x$.

Let me us fix $x\in X$. We may of course suppose that $x\notin U$. Let $V$ be an affine neighbourhood of $x$. By quasicompactness of $X$, the open subset $GV$ is covered by finitely many of the $gV$, say $GV=\cup_i g_iV$. Since $U$ is dense in $X$, it is possible to find a smooth curve $C$ and a morphism $C\to V$ whose image contains $x$ and intersects $U$. Up to removing points of $C$, we may suppose that $x$ is the only point of the image of $C$ not belonging to $U$.

Now, if $g\in G$, $gx\in g_i V$ for some $i$. Consider $g(C)$ (this is an abuse of notation for the composition of $C\to V$ and of the action of $g$) : its image contained in $g_iV$. It coincides generically with $g_i(C)$, hence, the morphisms $g(C)$ and $g_i(C)$ are equal by separation of $g_iV$ ($g_iV$ is even affine). This shows $gx=g_ix$. We have proved $Gx\subset${$g_1x,\dots, g_rx$}. Since $G$ is connected, $Gx$ is also connected. This shows $Gx=${$x$} and finishes the proof when $G$ is smooth.

When $G$ is not necessarily smooth, we may apply the result to $G^{red}$ : the group $G^{red}$ acts trivially on $X$. In particular, $G^{red}$ stabilizes every affine open subset of $X$ ; this implies that $G$ stabilizes every affine open subset of $X$. By the result mentionned by Anton in the question, since any open affine subset of $X$ is separated and reduced, $G$ acts trivially on it. Thus $G$ acts trivially on $X$.

1