You have not specified anything about the linearizing invertible sheaf $\mathcal{L}$ that you are using to define the GIT quotient. If for every $g\in \text{Aut}(X)$, $g^*\mathcal{L}$ is isomorphic to $\mathcal{L}$, then there is an induced action of $H$ on $Y$. This follows from Section 5, Chapter 1 of "Geometric Invariant Theory" (particularly the last paragraph of the section). It is also fairly easy to see directly from the construction of $Y$ as $\text{Proj}\oplus_n H^0(X,\mathcal{L}^{\otimes n})^G$.
Jason Starr
- 4.1k
- 1
- 93
- 111