Let your base S be Spec k[x] (say k is an algebraically closed field), let X be Spec k[x,y], and let G be G_a,S, with action over the point s given by g_s(x_s) = (sg_s) + x_s. This action is transitive away from zero, so the orbit of the zero section is a plane with a slit. This is not an open subscheme of a closed subscheme of X, because it is not a scheme.

Question number 1: Let your base S be Spec k[x] (say k is an algebraically closed field), let X be Spec k[x,y], and let G be G_a,S, with action over the point s given by g_s(x_s) = (sg_s) + x_s. This action is transitive away from zero, so the orbit of the zero section is a plane with a slit. This is not an open subscheme of a closed subscheme of X, because it is not a scheme.